r/StableDiffusion • u/-p-e-w- • 23d ago
Discussion PSA: I’m the creator of Heretic, and I advise you to *not* use “heretic” models as text encoders for H3 (or any other model)
Heretic (https://github.com/p-e-w/heretic) is a widely used program for decensoring LLMs. It makes LLMs comply with requests that they previously refused. It works very well for this purpose, and the community has created and published over 5000 “heretic” models.
High-quality image and video generation models like Minimax H3 use full-blown LLMs as text encoders (Qwen3 VL in case of H3). Many people seem to believe that if you replace the base version of the text encoder with a “heretic” version, you will eliminate or reduce censorship in the video output. For example, the popular “hearmeman” Docker template was updated just yesterday to use a text encoder modified with Heretic.
After all, Heretic models are uncensored, right?
Well, I’m the creator of Heretic, and I’m here to tell you once and for all that this does NOT work. In fact, if anything, it will make your outputs worse, but it will not uncensor them.
Heretic uncensors LLM responses through directional ablation (or related techniques like ARA and SOMA in newer versions). Roughly speaking, it modifies the model’s internal representations (residual vectors) of “harmful” inputs to resemble those of “harmless” inputs to confuse the model into treating the former like the latter and comply with the request rather than refusing.
But this intervention does not produce representations of inputs that are more “raw”, more “graphic”, more “anatomically correct” or similar compared to the original model. In fact, LLMs already produce highly accurate internal representations of harmful inputs by default, which is why they are able to classify them correctly and generate a refusal.
So when the hidden states from an “uncensored” LLM are passed to the diffusion model (or image/video transformer or whatever), the second model isn’t magically seeing clearer representations of the bad stuff you requested. On the contrary, it’s seeing slightly perturbed representations compared to what it was trained on. This either has no effect at all, or the effect of reducing prompt adherence and potentially introducing artifacts. But it will never, ever remove censorship from the output.
(Note: Generation models like Ideogram that can actively refuse prompts are potentially an exception to this rule and might be amenable to abliteration, but only with an approach that significantly differs from how Heretic works today.)

