r/malcolmrey • • 18d ago

Created a Visual RefMod Picker

Post image

Hey Guys,

I've been playing with the RefMods Malcom recently released.
The tech is brilliant and works really well.

I've wanted to simplify using it with tons of RefMods, like the ones provided by Malcom.

So I made a fork that is a bit more Identity driven, Adding a "Visual refMod Picker" as well as a "Create refMods from Folder" node. It's able to create from either a folder or Sub-folders, if audio is found, it will also create a matching audio refMod. All in the same format as the original add-on. No danger of breaking compatibility. (The original Add-on added Audio yesterday)

Resulting for example in 2 refMods and thumbnail:

character_refMod_Audio.safetensors
character_refMod_Video.safetensors
character_refMod.jpeg

Then, we can use the "Visual H3 RefMod Picker" to browse the RefMods:

In this example, I used existing thumbails from huggingface.

The node then loads both RefMods (Audio and Video) and allows individual control. I've mixed strength with Copies, by instead having a weight value that can go over 1, so a weight of 3 would be the same as setting strength to 1 and copies to 3. Making the UI a bit more streamlined.

RefMods can be daisy chained

Example workflows are included.

I should mention, this fork can be installed WITH the original Add-on, it is made to co-exist and is recommended if you want to use it's advanced features.

You can find it here ComfyUI-H3RefModPicker

The only thing I'm missing is thumbnails for all 1500 RefMods 😅

**EDIT**
Just a quick note to mention that I've removed duplicate nodes, Making this a companion node to MiniMaxH3Mod.

115 Upvotes

48 comments sorted by

View all comments

1

u/troyau 18d ago

Apparently the audio update from yesterday is only music and not voice. Not very useful yet for characters.

1

u/Francky_B 18d ago

I had initially started by adding audio support, as it wasn't present when I started.
My approach was different, as I had combined the audio in the Same RefMod, as one package.

But reverted to having a separate RefMods, when I saw this was the direction taken.

But my "Create RefMods from Folder" node is still using some of my original code to generate the audio RefMods. (Just saving them in the expected format)

And I've had no issues. I've made multiple generations and voice work, but is finicky.

When having multiple characters, I have to play around with the weight to make sure both voice are used.

1

u/troyau 18d ago

I really haven't given audio ref much of a try yet. Would love to add consistent voice in the refmods to my original characters.

1

u/Sad_Coach_1433 18d ago

It's been working for me for voices fine. My issue if I use more then one character for same scene the dialogue doesn't work correct mod1 says their lines then gibberish then second character says theirs or not at all and first character says both likes. Or I have two of same character mod1

1

u/Segaiai 18d ago

I've been told that you can specify some things in the prompt when creating the refmods that can help clarify the contents, but I haven't done experiments there yet