{
"$type": "site.standard.document",
"bskyPostRef": {
"cid": "bafyreibpuotti47g6fjxtgml454p36x7msxdjy2oxx2eo2kxhnizg4ujvi",
"uri": "at://did:plc:pgryn3ephfd2xgft23qokfzt/app.bsky.feed.post/3mh6dp5apgmv2"
},
"path": "/t/how-to-convert-a-single-safetensors-file-to-peft-format/173103#post_10",
"publishedAt": "2026-03-16T11:15:19.000Z",
"site": "https://discuss.huggingface.co",
"tags": [
"vLLM",
"GitHub"
],
"textContent": "Oh! I’ve rewritten it for Qwen-Image. From what I’ve tested so far, it seems that tensors with keys other than `mlp*` can be converted. However, it’s unclear whether LoRA will actually work with the converter below…\n\n* * *\n\nQwen-Image support is mostly a **naming/prefix** problem.\n\nvLLM-Omni diffusion LoRAs must be a **PEFT adapter directory** (`adapter_config.json` + `adapter_model.safetensors`). (vLLM)\nvLLM is strict about module-name suffixes and PEFT key naming, and it breaks on `*.to_out.0.*` unless you normalize it to `*.to_out.*`. (GitHub)\nFor Qwen-Image specifically, the pipeline loads transformer weights under a `transformer.` prefix, and the pipeline has a `self.transformer = QwenImageTransformer2DModel(...)`. (GitHub)\nThe Qwen-Image transformer also exposes packed projection shard mappings and normalizes `.to_out.0.` → `.to_out.` when loading weights. (GitHub)\n\nBelow is a rewritten version of the gist that adds a **Qwen-Image converter** for ComfyUI-style keys like:\n\n`transformer_blocks.N.attn.to_q.lora_down.weight`\n\nIt converts them into PEFT keys like:\n\n`base_model.model.transformer.transformer_blocks.N.attn.to_q.lora_A.weight`\n\n### Rewritten script (drop-in, supports Qwen-Image)\n\n\n #!/usr/bin/env python3\n \"\"\"\n comfyui-to-vllm-omni-qwenimage.py\n\n Convert ComfyUI-style Qwen-Image LoRA safetensors (lora_down/lora_up) into a PEFT\n adapter folder accepted by vLLM-Omni diffusion LoRA loader.\n\n Why this works:\n - vLLM-Omni requires PEFT adapter directory format. (adapter_config.json + adapter_model.safetensors)\n https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/lora/\n - vLLM expects lora_A/lora_B naming; ComfyUI uses lora_down/lora_up.\n - vLLM has a known failure for ModuleList/Sequential numeric indices like \"to_out.0\".\n Fix by rewriting to \"to_out\". https://github.com/vllm-project/vllm/issues/35734\n - Qwen-Image pipeline loads transformer weights with prefix \"transformer.\" and defines self.transformer.\n https://raw.githubusercontent.com/vllm-project/vllm-omni/main/vllm_omni/diffusion/models/qwen_image/pipeline_qwen_image.py\n - Qwen-Image transformer exposes packed shard mapping and normalizes \".to_out.0.\" -> \".to_out.\" in load_weights.\n https://raw.githubusercontent.com/vllm-project/vllm-omni/main/vllm_omni/diffusion/models/qwen_image/qwen_image_transformer.py\n \"\"\"\n\n import argparse\n import json\n import re\n import sys\n from pathlib import Path\n\n import torch\n from safetensors.torch import load_file, save_file\n\n\n # -------------------------\n # Qwen-Image settings\n # -------------------------\n\n # vLLM strips \"base_model.model.\" internally, and Qwen-Image modules live under \"transformer.*\"\n # (pipeline uses prefix=\"transformer.\" and assigns self.transformer=QwenImageTransformer2DModel)\n PREFIX_QWEN = \"base_model.model.transformer.\"\n\n # Attention-only by default (recommended). You can optionally include MLP keys with --include-mlp.\n ALLOWED_QWEN_PREFIXES_ATTN = (\n \"attn.to_q\",\n \"attn.to_k\",\n \"attn.to_v\",\n \"attn.to_out\",\n \"attn.add_q_proj\",\n \"attn.add_k_proj\",\n \"attn.add_v_proj\",\n \"attn.to_add_out\", # present in Qwen-Image-Lightning\n )\n\n # Optional MLP keys observed in Qwen-Image-Lightning (ComfyUI-style)\n ALLOWED_QWEN_PREFIXES_MLP = (\n \"img_mlp.net.0.proj\",\n \"img_mlp.net.2\",\n \"txt_mlp.net.0.proj\",\n \"txt_mlp.net.2\",\n )\n\n # PEFT config fields vLLM-Omni documents as important: r, lora_alpha, target_modules, base_model_name_or_path\n # https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/lora/\n QWEN_TARGET_MODULES_ATTN = [\n \"to_q\", \"to_k\", \"to_v\", \"to_out\",\n \"add_q_proj\", \"add_k_proj\", \"add_v_proj\",\n \"to_add_out\",\n # packed names are fine to include even if unused:\n \"to_qkv\", \"add_kv_proj\",\n ]\n\n # If you include MLP keys, vLLM will validate suffixes against expected modules.\n # net.2 can be tricky; keep it optional.\n QWEN_TARGET_MODULES_MLP = [\n \"proj\",\n # caution: module suffix may be \"2\" for net.2; only enable if your vLLM-Omni build expects it\n \"2\",\n ]\n\n ADAPTER_CONFIG_TEMPLATE = {\n \"peft_type\": \"LORA\",\n \"bias\": \"none\",\n \"inference_mode\": True,\n \"lora_dropout\": 0.0,\n \"r\": None,\n \"lora_alpha\": None,\n \"target_modules\": None,\n \"base_model_name_or_path\": None,\n }\n\n\n # -------------------------\n # Helpers\n # -------------------------\n\n def _remap_direction(direction: str) -> str:\n \"\"\"lora_down -> lora_A, lora_up -> lora_B\"\"\"\n if direction == \"lora_down\":\n return \"lora_A\"\n if direction == \"lora_up\":\n return \"lora_B\"\n return direction\n\n\n def _normalize_modulelist_indices(frag: str) -> str:\n \"\"\"\n Fix vLLM numeric-index issue:\n attn.to_out.0 -> attn.to_out\n Similar normalization exists in Qwen-Image transformer's load_weights. (see qwen_image_transformer.py)\n \"\"\"\n frag = frag.replace(\"attn.to_out.0\", \"attn.to_out\")\n frag = frag.replace(\"attn.to_add_out.0\", \"attn.to_add_out\")\n return frag\n\n\n def detect_format(keys: list[str]) -> str:\n sample = [k for k in keys if not k.endswith(\".alpha\")][:50]\n # Qwen-Image-Lightning (ComfyUI style) looks like:\n # transformer_blocks.N.attn.to_q.lora_down.weight\n if any(re.match(r\"^transformer_blocks\\.\\d+\\..+\\.(lora_down|lora_up)\\.weight$\", k) for k in sample):\n return \"qwen_transformer_blocks_comfyui\"\n return \"unknown\"\n\n\n def extract_rank_and_alpha(tensors: dict[str, torch.Tensor]) -> tuple[int, float]:\n alpha = None\n for k, v in tensors.items():\n if k.endswith(\".alpha\"):\n try:\n alpha = float(v.item())\n break\n except Exception:\n pass\n\n r = None\n for k, v in tensors.items():\n if k.endswith(\".lora_down.weight\") and hasattr(v, \"shape\"):\n r = int(v.shape[0])\n break\n\n if r is None:\n raise ValueError(\"Could not infer LoRA rank r. Provide --rank.\")\n if alpha is None:\n alpha = float(r)\n return r, alpha\n\n\n # -------------------------\n # Converter: Qwen-Image transformer_blocks.* (ComfyUI lora_down/lora_up)\n # -------------------------\n\n def convert_qwen_transformer_blocks_comfyui(\n tensors: dict[str, torch.Tensor],\n include_mlp: bool,\n dtype: torch.dtype,\n ) -> tuple[dict[str, torch.Tensor], list[str]]:\n out: dict[str, torch.Tensor] = {}\n unmapped: list[str] = []\n\n allowed_prefixes = ALLOWED_QWEN_PREFIXES_ATTN + (ALLOWED_QWEN_PREFIXES_MLP if include_mlp else ())\n\n pat = re.compile(r\"^transformer_blocks\\.(\\d+)\\.(.+?)\\.(lora_down|lora_up)\\.weight$\")\n\n for k, v in tensors.items():\n if k.endswith(\".alpha\"):\n continue\n\n m = pat.match(k)\n if not m:\n unmapped.append(k)\n continue\n\n block_idx = int(m.group(1))\n frag = _normalize_modulelist_indices(m.group(2))\n direction = m.group(3)\n\n if not frag.startswith(allowed_prefixes):\n unmapped.append(k)\n continue\n\n ab = _remap_direction(direction)\n new_key = f\"{PREFIX_QWEN}transformer_blocks.{block_idx}.{frag}.{ab}.weight\"\n\n if v.dtype != dtype:\n v = v.to(dtype)\n out[new_key] = v\n\n # Final safety: remove any leftover \".to_out.0.\" in full key\n fixed: dict[str, torch.Tensor] = {}\n for k, v in out.items():\n nk = k.replace(\".to_out.0.\", \".to_out.\").replace(\".to_add_out.0.\", \".to_add_out.\")\n fixed[nk] = v\n\n return fixed, unmapped\n\n\n # -------------------------\n # Main\n # -------------------------\n\n def main():\n ap = argparse.ArgumentParser(\"Convert ComfyUI Qwen-Image LoRA -> vLLM-Omni PEFT adapter dir\")\n ap.add_argument(\"--input\", required=True, help=\"Input LoRA .safetensors\")\n ap.add_argument(\"--output\", required=True, help=\"Output adapter directory\")\n ap.add_argument(\"--base-model\", default=\"Qwen/Qwen-Image\", help=\"base_model_name_or_path in adapter_config.json\")\n ap.add_argument(\"--dtype\", choices=[\"bf16\", \"fp16\", \"fp32\"], default=\"bf16\")\n ap.add_argument(\"--include-mlp\", action=\"store_true\", help=\"Also convert img_mlp/txt_mlp LoRA keys (may fail if vLLM expects different suffixes)\")\n args = ap.parse_args()\n\n dtype_map = {\"bf16\": torch.bfloat16, \"fp16\": torch.float16, \"fp32\": torch.float32}\n out_dtype = dtype_map[args.dtype]\n\n in_path = Path(args.input)\n if not in_path.exists():\n sys.exit(f\"[ERROR] Input not found: {in_path}\")\n\n print(f\"[INFO] Loading: {in_path}\")\n tensors = load_file(str(in_path))\n keys = list(tensors.keys())\n\n fmt = detect_format(keys)\n print(f\"[INFO] Detected format: {fmt}\")\n if fmt != \"qwen_transformer_blocks_comfyui\":\n sys.exit(\n \"[ERROR] This rewrite currently targets Qwen-Image ComfyUI keys like:\\n\"\n \" transformer_blocks.N.attn.to_q.lora_down.weight\\n\"\n \"If your keys differ, paste 30 keys and adjust detect_format/regex.\"\n )\n\n r, alpha = extract_rank_and_alpha(tensors)\n print(f\"[INFO] Inferred r={r}, lora_alpha={alpha}\")\n\n converted, unmapped = convert_qwen_transformer_blocks_comfyui(\n tensors=tensors,\n include_mlp=args.include_mlp,\n dtype=out_dtype,\n )\n\n print(f\"[INFO] Converted tensors: {len(converted)}\")\n if unmapped:\n print(f\"[WARN] Unmapped keys: {len(unmapped)} (showing first 20)\")\n for k in unmapped[:20]:\n print(\" \", k)\n\n out_dir = Path(args.output)\n out_dir.mkdir(parents=True, exist_ok=True)\n\n cfg = dict(ADAPTER_CONFIG_TEMPLATE)\n cfg[\"r\"] = int(r)\n cfg[\"lora_alpha\"] = float(alpha)\n cfg[\"base_model_name_or_path\"] = args.base_model\n cfg[\"target_modules\"] = (\n QWEN_TARGET_MODULES_ATTN + (QWEN_TARGET_MODULES_MLP if args.include_mlp else [])\n )\n\n (out_dir / \"adapter_config.json\").write_text(json.dumps(cfg, indent=2), encoding=\"utf-8\")\n save_file(converted, str(out_dir / \"adapter_model.safetensors\"))\n\n print(f\"[DONE] Wrote PEFT adapter dir: {out_dir}\")\n print(\" - adapter_config.json\")\n print(\" - adapter_model.safetensors\")\n\n\n if __name__ == \"__main__\":\n main()\n\n\n### Usage (for Qwen-Image-Lightning)\n\n\n python comfyui-to-vllm-omni-qwenimage.py \\\n --input Qwen-Image-Lightning-8steps-V2.0-bf16.safetensors \\\n --output ./out_adapter \\\n --dtype bf16 \\\n --base-model Qwen/Qwen-Image\n\n\n### Why this matches Qwen-Image in vLLM-Omni\n\n * It writes LoRA keys under `...transformer...` which aligns with Qwen-Image pipeline weight source prefix `prefix=\"transformer.\"` and `self.transformer = QwenImageTransformer2DModel(...)`. (GitHub)\n * It keeps `to_q/to_k/to_v` and `add_q_proj/add_k_proj/add_v_proj`, which align with Qwen-Image transformer packed shard mapping (`to_qkv` shards and `add_kv_proj` shards). (GitHub)\n * It normalizes `to_out.0` to `to_out` to avoid the known vLLM numeric-index LoRA failure. (GitHub)\n * It outputs the PEFT adapter folder vLLM-Omni requires. (vLLM)\n\n",
"title": "How to convert a single safetensors file to PEFT format"
}