{
"$type": "site.standard.document",
"bskyPostRef": {
"cid": "bafyreigsgkb37ziiq6osfsn5zml6pzjpqk65zk5ki7wnwpr65cbog5dage",
"uri": "at://did:plc:pgryn3ephfd2xgft23qokfzt/app.bsky.feed.post/3mjqwajpwyjb2"
},
"path": "/t/1st-movie-clip/175306#post_6",
"publishedAt": "2026-04-18T06:15:19.000Z",
"site": "https://discuss.huggingface.co",
"tags": [
"ComfyUI",
"GitHub",
"Hugging Face",
"docs.comfy.org",
"github.com",
"huggingface.co"
],
"textContent": "Fine-tuning LoRA often requires a slightly more powerful GPU than generating content.\n\nIf you just want to maintain consistency in your generated videos, it seems you might be able to manage with improvements to the ComfyUI workflow. (I’m not sure exactly which video model you’re using, but I’m assuming it’s probably from the Wan family…)\n\n* * *\n\nWhat you are seeing is normal for current open video workflows: the model is good at maintaining the **scene idea** across clips, but much worse at maintaining **identity** unless you explicitly carry identity forward from shot to shot. Current Wan/ComfyUI docs and recent Wan tooling all point in the same direction: when you want continuity, rely on **I2V** , **first/last-frame control** , and other forms of temporal guidance, not just repeated prompting. The official ComfyUI Wan 2.2 workflow is designed to fit the **5B** model on **8 GB VRAM** with native offloading, and the official Wan ecosystem now includes dedicated tools for temporal control like **Prompt Relay** rather than assuming text prompts alone will keep long clips coherent. (ComfyUI)\n\n## What is probably happening in your movie\n\nYour workflow is likely generating each clip as a fresh event:\n\n * Clip 1: “invent two people walking”\n * Clip 2: “invent two people walking”\n * Clip 3: “invent two people sitting”\n\n\n\nEven if the prompt is similar, the model is not treating those as “the same actors from the previous shot.” So the broad structure survives, but the faces drift. That is why the action can remain simple and believable while the final people no longer look like the original pair. This is exactly the kind of problem newer temporal-control methods are trying to address: the model struggles to attach the right semantic content to the right time segment unless you constrain it. (GitHub)\n\n## My direct answer to your main question\n\nYou do **not** need to jump straight to training your own Wan character LoRA.\n\nFor your exact case, the better next move is to improve the workflow first:\n\n * lock your characters with **anchor frames**\n * use **I2V** to carry them into the next shot\n * use **first/last-frame** for the standing-to-sitting transition\n * repair only the key frames that drift\n\n\n\nThat is the highest-value path on 8 GB VRAM. Full Wan LoRA training is still materially heavier: Musubi-tuner currently recommends about **12 GB+ for image training** and **24 GB+ for video training** , with lower resolutions and memory-saving options only partially softening that requirement. (GitHub)\n\n## Are there any known LoRAs worth trying?\n\nYes, but with a caveat.\n\nThe most visible public Wan 2.2 LoRAs right now are mostly **utility LoRAs** , not “keep my two exact characters stable forever” LoRAs.\n\nTwo useful families are:\n\n * **Wan2.2-Lightning** , which reduces generation to about **4 steps** and is aimed at much faster iteration. That is valuable because faster iteration lets you test more anchors, more handoffs, and more seed choices in the same time. (Hugging Face)\n * **Wan2.2-Distill-Loras** , which expose a **high-noise** and **low-noise** behavior split. Their model card describes **high noise** as more creative/diverse and **low noise** as more faithful/stable. For continuity work, the **low-noise** direction is usually the one to test first. (Hugging Face)\n\n\n\nSo yes, there are public LoRAs worth trying, but I would frame them as:\n\n * **speed / iteration LoRAs**\n * **stability / controllability LoRAs**\n\n\n\nnot as “drop-in identity LoRAs” for your specific two people. That distinction matters. The well-known public Wan LoRAs are mostly aimed at inference behavior, not at preserving your private pair of characters. That is an inference from what is publicly available now. (Hugging Face)\n\n## Is inpainting the last frame of each clip the right approach?\n\nYes, but as a **bridge-and-repair technique** , not as the whole pipeline.\n\nFor your case, I think this is one of the best ideas.\n\nThe ideal use is:\n\n 1. Generate a clip.\n 2. Pick the **last good frame**.\n 3. If the face drifted, repair that frame.\n 4. Feed that repaired frame into the next clip as the starting image.\n\n\n\nThat is much more effective than generating each clip from text alone and hoping the same people come back. If you need something more than single-frame repair, Wan 2.2 video inpainting tools now exist. **LanPaint** supports Wan 2.2 video inpainting with temporal consistency, but its own docs warn that longer sequences become slower and less stable, and it recommends keeping video inpainting to around **40 frames or fewer** for better stability. (GitHub)\n\nSo my view is:\n\n * **single-frame repair** : excellent idea\n * **short-span video inpainting** : useful for stubborn shots\n * **full-movie inpainting workflow** : too heavy and unnecessary for your case\n\n\n\n## Why I think your scene is actually solvable\n\nYour scene is simple in a good way:\n\n * two characters\n * side-by-side walking\n * then sitting on a bench\n\n\n\nThat means you do not need a model to remember a hundred changing details. You mainly need it to preserve:\n\n * the same two faces\n * the same clothes\n * the same relative body shapes\n * the same scale relationship\n * the same pair relationship in space\n\n\n\nThat is exactly the kind of problem that anchor-frame workflows can handle well. If you were trying to do a crowded party scene, a fight, or a lot of close-up dialogue cutting, I would be much less optimistic. For two people walking and sitting, continuity-by-handoff is realistic. The official Wan FLF workflow is built for transitions where you know the start and end states and want the model to create the motion between them. (ComfyUI)\n\n## My strongest suggestion for your case\n\nBuild the film around a **character pair bible** first.\n\nBefore rendering more video, create a tiny library of stills:\n\n * both characters walking side by side\n * both characters standing near the bench\n * both characters seated on the bench\n * optionally one tighter face reference for each person\n\n\n\nDo not think of them as “nice images.” Think of them as **production anchors**.\n\nThen use those anchors like this:\n\n### Walking shots\n\nGenerate from a strong walking still with **I2V**. Keep motion modest.\n\n### Transition shot\n\nUse **first/last-frame** :\n\n * first frame = trusted standing / arriving pose\n * last frame = trusted seated pose\n\n\n\n### Bench shots\n\nStart from the seated anchor and allow only small pose or camera changes.\n\nThat is the workflow I would choose before touching training. (ComfyUI)\n\n## My advice on multiple LoRAs\n\nYou can load multiple LoRAs, but I would be cautious.\n\nThe practical rule is: every loaded LoRA is generally influencing the model. In some wrappers there are explicit limits and caveats around merging or scheduling multiple LoRAs; for example, the WanVideoWrapper code warns that some LoRA setups cannot be merged under certain low-memory modes and that scheduling can be disabled. (GitHub)\n\nFor your project, I would keep it simple:\n\n * base model only, or\n * base model + one utility LoRA, or\n * base model + one carefully tested identity/style LoRA\n\n\n\nI would **not** stack several creative LoRAs and hope they settle into a stable cast. In practice, that often makes debugging much harder and can worsen identity drift rather than improve it. That last point is partly experience-based reasoning, but it follows from the fact that multi-LoRA behavior is not a magical “smart mix”; it is just multiple influences being applied at once. (GitHub)\n\n## Why do some LoRAs have trigger words and many do not?\n\nBecause “trigger word” is a training convention, not a law of nature.\n\nA LoRA may have a trigger word when it was trained with a specific token deliberately inserted into captions. But many LoRAs are trained with more natural captions, or their metadata is incomplete, or the UI simply does not surface the tag information well. Recent AI Toolkit issue reports explicitly show cases where a LoRA was trained with a `trigger_word` in config, yet the resulting `.safetensors` did not expose the expected metadata cleanly in downstream tools. (GitHub)\n\nThere is another important wrinkle: a LoRA can still visibly affect output **even without the trigger word in the prompt**. There is a long-standing sd-scripts issue about exactly this behavior. So “trigger word” does not mean “the LoRA is off until this word appears.” Loading the LoRA already changes the model; the trigger word is more like a steering handle for a specific learned concept. (GitHub)\n\nA useful way to think about it is:\n\n * **loading the LoRA** = changes the model weights used for generation\n * **trigger word** = tells the text encoder which learned concept to emphasize\n\n\n\nThat is why some LoRAs feel active even with no visible trigger, while others feel weak unless you use the right token. The training style and saved metadata both matter. (GitHub)\n\n## What I would do on your 8 GB machine\n\nI would use the official **Wan 2.2 5B** path and keep the process very disciplined:\n\n### 1. Build anchors before rendering motion\n\nMake the two people look right in stills first. Do not move on until they do.\n\n### 2. Use I2V as your continuity backbone\n\nEvery new shot should start from a frame you already trust.\n\n### 3. Use FLF for the sit-down moment\n\nThis is exactly the kind of boundary where first/last-frame control is stronger than free prompting.\n\n### 4. Use standard VAE decode for finals\n\nThere is a documented ComfyUI issue where **VAE Decode (Tiled)** causes **blinking and discoloration** with Wan output, while standard **VAE Decode** is fine. If you can fit the final decode, avoid tiled decode for final renders. (GitHub)\n\n### 5. Use Lightning only if you want faster previews\n\nLightning is useful because it makes iteration faster, not because it magically solves character identity. (Hugging Face)\n\nThat is the cheapest serious route.\n\n## What I would avoid right now\n\nI would avoid these three things for now:\n\n### 1. Full Wan video LoRA training\n\nThe hardware guidance is still above your current VRAM comfort zone. (GitHub)\n\n### 2. Heavy multi-LoRA stacks\n\nToo many moving parts, too hard to debug, too easy to destabilize the pair.\n\n### 3. Long uncontrolled clips\n\nShort connected shots are easier to keep coherent than one long free-running shot.\n\n## If you later decide to train something\n\nIf you eventually decide that you really do need a custom identity asset, then the better stepping stone is usually:\n\n * **image-side LoRA first**\n * **video-side LoRA later**\n\n\n\nCurrent Wan training options do exist. **Musubi-tuner** supports Wan 2.1/2.2 LoRA training, **DiffSynth-Studio** advertises Wan support with memory-management and LoRA training, and AI Toolkit-based UIs have added Wan video training options. But the hardware and workflow complexity still make these “later” moves rather than “first fix” moves for an 8 GB setup. (GitHub)\n\nOne interesting detail from Musubi-tuner discussions is that Wan 2.2 is so much larger than SDXL that a given LoRA rank is correspondingly more capable, and experienced users report often training Wan around **rank 16 / alpha 16** for simple character LoRAs. That is useful background for later, because it suggests “bigger rank” is not automatically better on Wan. (GitHub)\n\n## My concrete recommendation for your exact project\n\nIf this were my project, I would do this:\n\n 1. Make one excellent still of the pair walking.\n 2. Make one excellent still of the pair seated on the bench.\n 3. Use the walking still to generate a short walking shot with I2V.\n 4. Take the best end frame.\n 5. Repair that end frame if the faces drifted.\n 6. Use the repaired frame as the start of the next shot.\n 7. Use FLF from “standing near bench” to “seated on bench.”\n 8. Render final clips with standard VAE decode if possible.\n 9. Only then test Lightning for faster preview cycles. (ComfyUI)\n\n\n\nThat is the path I think gives you the best odds of finishing the film with the same two recognizable people **without** needing immediate custom training.\n\n## Bottom line\n\nFor your case:\n\n * public Wan LoRAs are worth trying mainly for **speed** and **stability** , not as a complete identity fix. (Hugging Face)\n * inpainting the last frame is a **good idea** , especially as a bridge into the next clip. (GitHub)\n * multiple LoRAs can be loaded, but I would keep that to a minimum. (GitHub)\n * trigger words exist because of how a LoRA was trained and how metadata was saved; they are not always present, and LoRAs can still affect output without them. (GitHub)\n * the best fix for you is not “more prompting.” It is **anchor frames + I2V + FLF + selective repair**. (ComfyUI)\n\n\n\n* * *\n\nUse **Wan 2.2 5B** as the base, because the official ComfyUI guide is specifically aimed at getting that workflow onto **8 GB VRAM** with native offloading. For this project, keep the toolset narrow: **I2V for continuity** , **FLF for the sit-down transition** , **standard VAE Decode for finals if you can fit it** , and only optional short-span repair if a key frame drifts. That is the cleanest 8 GB path. (docs.comfy.org)\n\n# The overall production idea\n\nTreat the whole piece as **four linked shots** , not one long generation:\n\n 1. **Walk**\n 2. **Approach bench**\n 3. **Sit**\n 4. **Seated**\n\n\n\nEach shot hands one trusted frame to the next shot.\n\nThat is the backbone.\n\n* * *\n\n# Before generating any video\n\n## 0. Build the three anchors first\n\nMake these stills before you render motion:\n\n * **Anchor A — walking pair**\n * **Anchor B — standing / arriving near bench**\n * **Anchor C — seated on bench**\n\n\n\nDo not move on until all three are good.\n\nThese anchors should lock:\n\n * the same two faces\n * the same outfits\n * the same hair\n * the same relative height and body shape\n * the same bench / environment style\n\n\n\nFor your film, these anchors matter more than another LoRA.\n\n* * *\n\n# Shot 1 — Walk\n\n## Goal\n\nGet a short, believable walking shot where the pair still looks correct.\n\n## Workflow\n\nUse **I2V** from **Anchor A**.\n\n## Why I2V here\n\nYou already know what the pair should look like. The job is not “invent two people.” The job is “animate these two people a little.” The Wan-native workflow already supports this logic directly. (docs.comfy.org)\n\n## What to ask for\n\nKeep it modest:\n\n * side-by-side walking\n * small body motion\n * small camera movement\n * no dramatic pose change\n * no major turn away from camera\n\n\n\nThis shot should be **short**. Do not try to make the entire walking section in one go. On 8 GB, short controlled clips are much easier to keep coherent than long uncontrolled ones.\n\n## What to save\n\nAt the end of Shot 1, save:\n\n * the **best final frame**\n * optionally one **mid-shot frame** if the end frame is weak\n\n\n\nThat final frame becomes the handoff into Shot 2.\n\n* * *\n\n# Shot 2 — Approach bench\n\n## Goal\n\nMove from walking to “near the bench” without recasting the pair.\n\n## Workflow\n\nUse **I2V** again, but now the input image is **the repaired last frame from Shot 1**.\n\n## What to do first\n\nInspect the last frame from Shot 1.\n\nIf the faces are slightly off:\n\n * repair that frame first\n * then use the repaired version as input\n\n\n\nThis is where your “inpaint the last frame” idea is genuinely useful.\n\n## Why this works\n\nYou are turning the clip boundary into a controlled handoff. Instead of asking the model to recreate the pair from text, you are telling it, in effect:\n\n> Start here. These are the people.\n\nThat is exactly the kind of explicit continuity help current Wan workflows benefit from. (docs.comfy.org)\n\n## What to ask for\n\nAgain, keep it conservative:\n\n * slowing down\n * moving toward bench\n * slight posture shift\n * maybe slight turn of heads\n * no full sit yet\n\n\n\nDo **not** combine “approach bench” and “sit down” in this shot. Split them.\n\n## What to save\n\nSave the best frame where the pair is clearly **standing or just about to sit**.\n\nThat frame becomes the first endpoint for Shot 3.\n\n* * *\n\n# Shot 3 — Sit\n\n## Goal\n\nCreate the actual standing-to-seated transition.\n\n## Workflow\n\nThis is the one shot where I would switch away from pure I2V and use **FLF**.\n\nUse:\n\n * **first frame** = best standing-near-bench frame from Shot 2\n * **last frame** = **Anchor C** or a repaired seated still\n\n\n\n## Why FLF here\n\nThis is the one place in your film where you know both ends very clearly:\n\n * start: standing near bench\n * end: seated on bench\n\n\n\nThat is exactly what the official Wan FLF workflow is designed for. (docs.comfy.org)\n\n## What to ask for\n\nKeep the motion plain:\n\n * both lower into the bench\n * small coordinated posture change\n * no complex hand gestures\n * no big camera move\n\n\n\nThis shot should also be short.\n\n## If it fails\n\nIf the FLF output reaches the seated pose but the faces slip:\n\n * repair the seated endpoint still\n * rerun the transition using the corrected endpoint\n\n\n\nThat is much cheaper than trying to fix a whole long clip afterward.\n\n* * *\n\n# Shot 4 — Seated\n\n## Goal\n\nHold the pair on the bench with minimal drift.\n\n## Workflow\n\nGo back to **I2V** , using either:\n\n * the best final frame from Shot 3, or\n * **Anchor C** if Shot 3 ends weakly\n\n\n\n## What to ask for\n\nOnly subtle movement:\n\n * breathing\n * head turn\n * slight glance\n * small hand movement\n * slow camera drift\n\n\n\nBench shots are where you can make the film feel longer **without** asking the model to do too much. This is the safest place to get duration.\n\n## Editorial trick\n\nIf you have a really good seated clip, use editing to stretch its screen time:\n\n * hold a good frame briefly\n * crossfade into the next seated clip\n * use a gentle camera push in post if needed\n\n\n\nThat is often smarter than generating more motion.\n\n* * *\n\n# Repair policy\n\n## When to repair\n\nRepair only these:\n\n * the **last frame** of Shot 1\n * the **last frame** of Shot 2\n * the **seated endpoint** for Shot 3\n * any hero close-up or especially visible face\n\n\n\n## When not to repair\n\nDo not start frame-by-frame fixing everything unless the clip is genuinely important.\n\nFor your project, selective repair is the efficient path.\n\n## If more than one or two frames are broken\n\nThen a short-span video inpainting pass can make sense. **LanPaint** now supports **Wan 2.2 video inpainting** , but it is explicitly positioned for shorter spans, not big full-scene rewrites. (github.com)\n\n* * *\n\n# LoRA plan for this workflow\n\n## What I would do\n\nKeep LoRA usage minimal:\n\n * **Option A:** no LoRA\n * **Option B:** one utility LoRA for faster previews\n * **Option C:** one carefully tested style/identity LoRA\n\n\n\n## What I would not do\n\nDo not stack several stylistic LoRAs while you are still solving continuity.\n\nThat makes it much harder to tell whether the model is drifting because of:\n\n * the shot design,\n * the clip handoff,\n * or the combined LoRA influences.\n\n\n\n## If you want faster previews\n\nTry **Wan2.2-Lightning** for preview passes only. It is a 4-step LoRA family designed to speed Wan 2.2 generation, which is useful for look-dev and shot testing. Once you like the handoff strategy, do cleaner final passes with your preferred final setup. (huggingface.co)\n\n* * *\n\n# Decode plan on 8 GB\n\n## Previews\n\nFor rough previews, do whatever keeps the graph alive.\n\n## Finals\n\nFor final output, try to avoid **VAE Decode (Tiled)** if possible.\n\nThere is a documented ComfyUI issue where Wan output shows **blinking/discoloration** with tiled decode, while standard **VAE Decode** does not show the same problem. (github.com)\n\nSo the practical rule is:\n\n * preview flexibly\n * final decode carefully\n\n\n\nIf standard decode is too heavy at full size, render smaller for continuity approval, then do the final-quality pass only on the shots you are actually keeping.\n\n* * *\n\n# The exact order I would follow\n\n## Pass 1 — still approval\n\nMake and approve:\n\n * Anchor A\n * Anchor B\n * Anchor C\n\n\n\n## Pass 2 — motion approval\n\nGenerate:\n\n * Shot 1 walk\n * Shot 2 approach\n * Shot 3 sit\n * Shot 4 seated\n\n\n\nKeep them short.\n\n## Pass 3 — continuity approval\n\nAt every shot boundary:\n\n * inspect last frame\n * repair if needed\n * feed forward\n\n\n\n## Pass 4 — final render approval\n\nRender only the approved shots with your better decode path.\n\n## Pass 5 — edit\n\nAssemble in your video editor.\nDo not ask ComfyUI to do every bit of pacing that a normal edit can do more safely.\n\n* * *\n\n# Suggested shot-by-shot checklist\n\n## Shot 1 — Walk\n\n * Input = Anchor A\n * Mode = I2V\n * Duration = short\n * Motion = gentle walk\n * Output = save best final frame\n\n\n\n## Shot 2 — Approach bench\n\n * Input = repaired last frame from Shot 1\n * Mode = I2V\n * Duration = short\n * Motion = approach / slow / prepare to sit\n * Output = save best standing-near-bench frame\n\n\n\n## Shot 3 — Sit\n\n * First frame = best standing-near-bench frame\n * Last frame = Anchor C or repaired seated still\n * Mode = FLF\n * Duration = short\n * Motion = simple sit-down transition\n * Output = save best seated final frame\n\n\n\n## Shot 4 — Seated\n\n * Input = best seated frame\n * Mode = I2V\n * Duration = short to medium\n * Motion = very subtle\n * Output = use this shot to create calm screen time\n\n\n\n* * *\n\n# What not to do for this film\n\nDo not:\n\n * generate each shot from text only\n * ask one shot to do too many state changes\n * stack multiple unknown LoRAs\n * rely on trigger words as your main continuity tool\n * start Wan video LoRA training on 8 GB as your first fix\n\n\n\nThe official toolchain direction is already telling you the same thing indirectly: **5B on 8 GB for inference** , **I2V/FLF for structure** , and much higher memory expectations for training. (docs.comfy.org)\n\n* * *\n\n# My simplest version of the plan\n\nIf you want the cleanest possible summary:\n\n * **Walk** = I2V from walking anchor\n * **Approach bench** = I2V from repaired last frame of walk\n * **Sit** = FLF from standing-near-bench to seated anchor\n * **Seated** = I2V from seated anchor\n * **Repair only the bridge frames**\n * **Use standard VAE Decode for finals if possible**\n * **Use Lightning only for fast previews**\n\n\n\nThat is the shot-by-shot workflow I would use on an 8 GB machine for your exact scene.",
"title": "1st movie clip!"
}