基于 instantX-research/diffusers_sd3_control: 🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX. (github.com)
和
ZHO-ZHO-ZHO/ComfyUI-SD3-Medium-CN-Diffusers: ComfyUI SD3-Medium ControlNet(Diffusers) (github.com)
改的,这两个repo都有些问题,中间看提交记录,有好些人在参与改动,为了能跑的了,我分别fork了这两个分支, 在这里:
prodogape/ComfyUI-SD3-Medium-CN-Diffusers: ComfyUI SD3-Medium ControlNet(Diffusers) (github.com)
prodogape/diffusers_sd3_control: refactor name from "diffusers" to "diffusers_sd3_control" (github.com)
1. 先说怎么装
打开第一个橘色背景的repo:
prodogape/ComfyUI-SD3-Medium-CN-Diffusers: ComfyUI SD3-Medium ControlNet(Diffusers) (github.com)
按照Install这里的步骤安装
时间有限只写了手动安装的步骤,分4步:
(1)git 克隆 comfyui的节点
(2)下载hf的两个仓库的模型(一个controlnet一个sd3)
(3)修改sd3模型文件夹下的文件model_index.json,因为要安装的特殊版本的diffusers我改名为diffusers_sd3_control了
(4)安装diffusers_sd3_control
2. 工作流及效果
这个版本的实现应该还是有些问题的,可能跟control的百分比和control了不太对的部分有关
3. 上图的工作流(非API版本工作流)
{"last_node_id": 82,"last_link_id": 139,"nodes": [{"id": 77,"type": "CannyEdgePreprocessor","pos": [-1756,-33],"size": {"0": 315,"1": 106},"flags": {},"order": 2,"mode": 0,"inputs": [{"name": "image","type": "IMAGE","link": 133}],"outputs": [{"name": "IMAGE","type": "IMAGE","links": [134,139],"shape": 3,"slot_index": 0}],"properties": {"Node name for S&R": "CannyEdgePreprocessor"},"widgets_values": [100,200,512]},{"id": 82,"type": "PreviewImage","pos": [-1753,110],"size": {"0": 299.701171875,"1": 326.380859375},"flags": {},"order": 4,"mode": 0,"inputs": [{"name": "images","type": "IMAGE","link": 139}],"properties": {"Node name for S&R": "PreviewImage"}},{"id": 80,"type": "PreviewImage","pos": [-970,-40],"size": {"0": 438.1802978515625,"1": 471.4967041015625},"flags": {},"order": 5,"mode": 0,"inputs": [{"name": "images","type": "IMAGE","link": 137}],"properties": {"Node name for S&R": "PreviewImage"}},{"id": 74,"type": "LoadImage","pos": [-2113,-24],"size": {"0": 349.269775390625,"1": 459.3564147949219},"flags": {},"order": 0,"mode": 0,"outputs": [{"name": "IMAGE","type": "IMAGE","links": [133],"shape": 3,"slot_index": 0},{"name": "MASK","type": "MASK","links": null,"shape": 3}],"properties": {"Node name for S&R": "LoadImage"},"widgets_values": ["IMG_20240617_070212_131.jpeg","image"]},{"id": 76,"type": "SD3MCN_BaseModelLoader_Zho","pos": [-1424,-42],"size": {"0": 428.3705749511719,"1": 95.87933349609375},"flags": {},"order": 1,"mode": 0,"outputs": [{"name": "pipe","type": "SD3MMODEL","links": [132],"shape": 3,"slot_index": 0}],"properties": {"Node name for S&R": "SD3MCN_BaseModelLoader_Zho"},"widgets_values": ["stabilityai/stable-diffusion-3-medium-diffusers","InstantX/SD3-Controlnet-Canny"]},{"id": 78,"type": "SD3MCN_Generation_Zho","pos": [-1436,94],"size": {"0": 444.2547607421875,"1": 344.82293701171875},"flags": {},"order": 3,"mode": 0,"inputs": [{"name": "pipe","type": "SD3MMODEL","link": 132},{"name": "image","type": "IMAGE","link": 134}],"outputs": [{"name": "IMAGE","type": "IMAGE","links": [137],"shape": 3,"slot_index": 0}],"properties": {"Node name for S&R": "SD3MCN_Generation_Zho"},"widgets_values": ["close-up of a young girl, masterpiece, best quality, photo, realistic, very aesthetic","worst quality, low quality, toon",512,512,40,9,1,542410872807786,"randomize"]}],"links": [[132,76,0,78,0,"SD3MMODEL"],[133,74,0,77,0,"IMAGE"],[134,77,0,78,1,"IMAGE"],[137,78,0,80,0,"IMAGE"],[139,77,0,82,0,"IMAGE"]],"groups": [],"config": {},"extra": {"workspace_info": {"id": "-gBBQ2xIYibjLVAhj5iML"},"ds": {"scale": 1.3513057093104035,"offset": {"0": 2165.8774539604974,"1": 287.4375476041587}}},"version": 0.4
}