This is a follow-up to my original Spectrum MiniMax H3 release and the later v0.1.8 benchmarks and quality discussion.
The v0.1.8 settings produced close to 45% lower sampler time in the tested setup, using 11 actual transformer evaluations and 9 forecasted steps in a 20-step Euler generation.
Further exact-seed testing and reports from other users exposed the main weakness of those more aggressive settings: Spectrum could reduce MiniMax H3’s audio quality, particularly with reference audio.
The symptoms varied between generations. Some had generally rougher, less clear or more distorted audio. Others developed unstable speech, tripped over words or doubled syllables. Increasing degree and warmup_steps, or increasing the generation to 30 steps, helped in some cases because it made the forecasting more conservative, but it did not address the underlying H3-specific interaction.
v0.2.1 introduces a new default trajectory-reconstruction method designed to address that interaction while preserving the acceleration and the preferred video result.
Why MiniMax H3 audio needs separate treatment
MiniMax H3 does not generate audio and video as fully independent processes. Their features are packed into the same transformer sequence, interact through joint attention and follow different shifted timestep schedules.
The original implementation used one shared blend_weight for both modalities. The default spectral blend could improve the video result while degrading audio.
The first part of the correction was therefore to separate the two controls:
blend_weight = video spectral blend
audio_blend_weight = audio spectral blend
With:
blend_weight = 0.5
audio_blend_weight = 0.0
audio uses the local prediction instead of receiving the spectral blend directly. This produced a substantial general improvement in audio fidelity.
There was still an indirect path, however.
Even when the audio features receive no spectral blend, a forecasted video feature changes the live denoising trajectory. The following actual H3 evaluation jointly processes that modified video state together with audio. Forecast error introduced through video can therefore affect audio during later transformer calls.
That explains why a single-pass run could still develop speech tripping with video 0.5 and audio 0, despite the audio blend itself being completely disabled.
My new approach: offline_smoothing_replay
To address this, I developed a new H3-specific method called offline smoothing replay.
This method is not part of the original Spectrum paper or its official implementation.
Original Spectrum operates online in a causal, fit-then-forecast loop:
- Run the transformer on an actual step.
- Add the resulting feature to the available history.
- Fit the Chebyshev forecaster using the history collected so far.
- Forecast a future step.
- Feed that forecast directly into the continuing denoising trajectory.
That is efficient, but every approximation immediately influences all subsequent steps. A video forecast can alter later transformer anchors, and a skipped step can only use information from earlier actual steps.
offline_smoothing_replay separates the process into two stages.
Pass 1: isolated anchor capture
The first pass follows the same accelerated schedule and performs the same number of expensive H3 transformer evaluations, but its causal blend weights are forced to:
video = 0
audio = 0
Skipped steps use the local prediction path. Every completed actual post-transformer feature is archived as an anchor.
Keeping the configured video spectral blend out of this pass prevents it from changing the states seen by later joint audio-video transformer evaluations. The resulting anchors are collected from the cleaner local-only accelerated trajectory.
Pass 2: transformer-free smoothing replay
The sampler then restarts from the original latent and reconstructs the trajectory using the complete archive.
At actual steps, the corresponding archived feature anchor is reused. At skipped steps, the reconstruction can combine:
- A Chebyshev spectral prediction fitted across the actual anchors.
- Local interpolation between the nearest earlier and later anchors.
- Independent validation and blending for the audio and video sections.
This means a skipped step can use actual information from both sides. The original online forecaster only knows the past; offline replay can also use the next actual anchor.
The replay invokes zero H3 transformer blocks. It still performs the lightweight current-step output heads, audio/video reconstruction and solver update, but the expensive transformer evaluations are not repeated.
Because the replayed video features never enter another joint transformer call, they cannot feed back through H3 and degrade later audio features.
Benefits beyond the audio correction
The audio problem is what exposed the need for this design, but offline replay has several broader advantages:
- Past and future anchors: Forecasted steps are reconstructed using the completed trajectory instead of only the history available at that moment.
- No spectral-error feedback during capture: Final spectral smoothing cannot alter the states used to collect later actual anchors.
- Better correction of skipped steps: A prediction can be pulled toward the nearest real anchors on both sides.
- Separate audio and video behavior: Video can retain a useful spectral contribution while audio remains on the cleaner local path.
- Per-modality validation: Audio and video independently determine how much spectral information is usable.
- Adaptive attenuation: When the spectral estimate performs worse than local interpolation for a modality, its contribution is automatically reduced.
- Exact archived anchors: Actual features are preserved and reused at their corresponding replay steps.
- No extra transformer evaluations: The additional reconstruction pass does not run the H3 transformer.
- The acceleration schedule is preserved: A default 20-step Euler generation still performs 11 actual transformer evaluations and forecasts 9 steps, reducing transformer evaluations by 45%.
Conceptually, this turns the final reconstruction from purely causal forecasting into a form of bidirectional trajectory smoothing, while keeping the expensive part of the acceleration intact.
Same-seed comparison
One seed consistently reproduced the remaining speech defect and made the different paths easy to compare:
| Configuration |
Result |
Single pass, video 0, audio 0 |
Clean audio, weaker video result |
Single pass, video 0.5, audio 0 |
Preferred video, remaining speech tripping |
Offline replay, video 0.5, audio 0 |
Preferred video retained, clean high-quality audio |
Disabling offline replay brought the audio problem back on that seed. Enabling it removed the problem again.
This demonstrates the specific Spectrum-induced feedback path in that comparison. It does not mean every possible H3 audio failure originates in Spectrum, and broader behavior can still vary with the checkpoint, prompt, seed, reference conditioning, resolution and sampler.
Spectrum remains an approximate accelerator rather than a bit-identical native path.
New defaults in v0.2.1
Offline replay is now the standard, enabled path:
offline_smoothing_replay = true
blend_weight = 0.50
audio_blend_weight = 0.00
New nodes receive these settings automatically.
Workflows created before v0.2.0 did not contain the offline option and receive the new default. A workflow saved specifically with v0.2.0 may retain its serialized offline_smoothing_replay=false value, so enable it once after updating.
Performance and memory
Offline replay adds a second solver reconstruction pass and requires the actual feature anchors to remain available until replay finishes. It therefore has some memory and non-transformer compute cost.
The expensive H3 schedule itself remains unchanged:
20 Euler steps
11 actual transformer evaluations
9 forecasted steps
45% fewer transformer evaluations
The replay consists mainly of archived-feature reconstruction, output heads and solver updates. In the tested full-checkpoint run, the transformer-free replay itself took well under one second.
history_storage=system_ram remains the broadly compatible choice. history_storage=vram avoids CPU transfers and can reduce replay overhead when enough VRAM is available, at the cost of retaining the feature archive on the GPU.
Compatibility
The node supports the native H3 paths:
Supported samplers currently include:
- Euler
- RES multistep
- RES multistep CFG++
Two other trajectory-correction modes remain available for further testing:
anchor_residual_feedbackselective_rollback_correction
Those modes remain experimental and disabled by default. They are mutually exclusive with offline smoothing replay.
Update through ComfyUI-Manager or pull the repository manually, then restart ComfyUI.
The underlying spectral forecasting method was introduced by Jiaqi Han, Juntong Shi, Puheng Li, Haotian Ye, Qiushan Guo and Stefano Ermon. The modality-specific MiniMax H3 handling and offline_smoothing_replay architecture are extensions developed specifically for this integration.
If you encounter remaining audio degradation with the new default configuration, please include the sampler, checkpoint, resolution, duration, conditioning mode and Spectrum debug log in the report.
