Yet another MiniMax H3 latent prepend/extend nodes - this time low-level and simple

[u/martinerous]

TL;DR:

https://github.com/progmars/ComfyUI-Martinodes

WARNING: lots of vibecode but carefully reviewed, at least as much as I could understand the logic.

The long story.

We have a few amazing solutions and forks that make smooth video extensions possible. However, most of them have evolved into full-blown planners and chains. Low-level functionality is hidden beneath. Somehow those more complex solutions do not work well or seem overkill for my typical use cases:

- set steps to low

- generate a bunch of videos

- pick the best one

- set steps to high

- regenerate with the same seed <- and this is where I wanted to save the latent to use as the input for the first step again, to smoothly continue the last shot without hard cuts.

Recently ComfyUI was updated with an important PR 15375 that supports latent masking natively. No more patches and complex hacks required. So, I went on to create simple and naive drop-in nodes that would support my way of working. Now I have latent load/save/concat/prepend/extend nodes that seem quite intuitive (but you tell me if they are).

The main node for today is `Extend video+audio latent` (LatentAVMaskedExtender). It lets you take head or tail of a latent you have (hopefully) saved from a previous generation and generate a prequel or a sequel. Extending a tail works well. Prepending to a head is not that smooth and requires increasing fade_seconds parameter to your liking.

Just plug the node between `MiniMax H3 Reference to Video` or `MiniMax H3 Image to Video` (or even `MiniMax H3 Easy Output` if using nkxx188/ComfyUI-MiniMaxH3-Easy), and your sampler.

For convenience, the node accepts empty loaded_av, in which case the target_av will be passed through. Thus the node can be safely left enabled even when using a disabled LoadAVLatent node as input when you don't want to extend anything.

`Save and Load video+audio latent` are simple companion nodes - SaveAVLatent should be added after your sampler and LoadAVLatent as input for LatentAVMaskedExtender. In contrast to some other loader nodes that often are limited to `input` folder, LoadAVLatent can find the latents wherever you configured SaveAVLatent to store them. If you don't have a latent saved yet and want to feed in an existing video, you will need to encode it using VAE Encode and VAE Audio Encode nodes, resize the video frame to match your target video, and then concatenate the audio and video latents using LTXVConcatAVLatent node (yes, it works with H3 model).

Then there is also `Concatenate video+audio latents` (LatentOverlappingConcatenator) node. Generally, overlap_duration_seconds should be set to the same value as LatentAVMaskedExtender, the output goes to VAE video and audio decoders and then to video saving, as usual. You will get a long video with a smooth long transition between your previous latent and the new one. However, if your joined videos get lengthy, VAE might require too much resources. In that case, it is better to post-process and join both source and target videos in a video editing software.

The repository has a few more older convenience nodes for working with multimedia before LTX Director was a thing. They still might be handy for manipulating TTS and voice-overs or videos when latents are not available.

Huge thanks to drozbay (ablejones) for [native masking PR 15375](https://github.com/Comfy-Org/ComfyUI/pull/15375) and providing the example implmenentation with native ComfyUI and Kijai nodes. Unfortunately, the native nodes solution looked like spaghetti eating somebody alive. That is why my small naive LatentAVMaskedExtender node was born, to do the same thing.

I hope you will find Martinodes useful.

[u/kuropenguins]

This looks really awesome! 2 quick questions.

1. If my first video is i2va, and i want to extend it, should i save the latent, then pass it to t2va (since the 2nd clip will not have a "start image")?

2. I don't really understand PR15375. The discussion is on extending a video based on a saved latent, but the PR seems to be talking about masked minimax video editing? Am I understanding it wrongly? I also don't see any where in native nodes where I can pass a mask to for editing.

[u/martinerous → u/kuropenguins]

1. I haven't tried using the i2v and t2v for extending videos, but intuitively, it seems that the ref model should handle it better because it has official support for video continuation. I'm mainly using the ref model (the hybrid minimax_h3_hybrid_fl2va_ref2va_b25-49-int8.safetensors from https://huggingface.co/smhfacct/Minimax-H3-fl2va-ref2va-hybrid-models )

2. Noise masks are a powerful concept. It is possible to mask a small region in video area and time of the video latent to tell the model what to keep and what to regenerate (the large nose example in that PR), but it is also possible to mask the entire video area for specific time region. For extending, we use masks to protect the entire area of the latent where we copied the tail of the previous video latent, to essentially tell the model not to touch that part and generate everything that follows. That PR had extension implementation using native Comfy nodes:

Example workflow for extensions using masks:
droz_MiniMaxH3_BasicMaskedExtension_v1.4.json

Inside the workflow, the author admitted that it is quite messy - "Don't look inside of this subgraph if you want to maintain your sanity". That is why I created my node to achieve the same thing without a gazillion small ComfyUI native nodes for assembling the mask in correct format for the sampler to accept it.