您的位置:首页 > 房产 > 家装 > 【ComfyUI】Stable Diffusion 3 加Controlnet

【ComfyUI】Stable Diffusion 3 加Controlnet

2024/12/23 16:09:38 来源:https://blog.csdn.net/weixin_36047799/article/details/139758186  浏览:    关键词:【ComfyUI】Stable Diffusion 3 加Controlnet

基于 instantX-research/diffusers_sd3_control: 🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch and FLAX. (github.com)

和 

ZHO-ZHO-ZHO/ComfyUI-SD3-Medium-CN-Diffusers: ComfyUI SD3-Medium ControlNet(Diffusers) (github.com)

改的,这两个repo都有些问题,中间看提交记录,有好些人在参与改动,为了能跑的了,我分别fork了这两个分支, 在这里:

prodogape/ComfyUI-SD3-Medium-CN-Diffusers: ComfyUI SD3-Medium ControlNet(Diffusers) (github.com)

prodogape/diffusers_sd3_control: refactor name from "diffusers" to "diffusers_sd3_control" (github.com)

1. 先说怎么装

打开第一个橘色背景的repo:

prodogape/ComfyUI-SD3-Medium-CN-Diffusers: ComfyUI SD3-Medium ControlNet(Diffusers) (github.com)

按照Install这里的步骤安装

时间有限只写了手动安装的步骤,分4步:

(1)git 克隆 comfyui的节点

(2)下载hf的两个仓库的模型(一个controlnet一个sd3)

(3)修改sd3模型文件夹下的文件model_index.json,因为要安装的特殊版本的diffusers我改名为diffusers_sd3_control了

(4)安装diffusers_sd3_control

2. 工作流及效果

这个版本的实现应该还是有些问题的,可能跟control的百分比和control了不太对的部分有关

3. 上图的工作流(非API版本工作流)

{"last_node_id": 82,"last_link_id": 139,"nodes": [{"id": 77,"type": "CannyEdgePreprocessor","pos": [-1756,-33],"size": {"0": 315,"1": 106},"flags": {},"order": 2,"mode": 0,"inputs": [{"name": "image","type": "IMAGE","link": 133}],"outputs": [{"name": "IMAGE","type": "IMAGE","links": [134,139],"shape": 3,"slot_index": 0}],"properties": {"Node name for S&R": "CannyEdgePreprocessor"},"widgets_values": [100,200,512]},{"id": 82,"type": "PreviewImage","pos": [-1753,110],"size": {"0": 299.701171875,"1": 326.380859375},"flags": {},"order": 4,"mode": 0,"inputs": [{"name": "images","type": "IMAGE","link": 139}],"properties": {"Node name for S&R": "PreviewImage"}},{"id": 80,"type": "PreviewImage","pos": [-970,-40],"size": {"0": 438.1802978515625,"1": 471.4967041015625},"flags": {},"order": 5,"mode": 0,"inputs": [{"name": "images","type": "IMAGE","link": 137}],"properties": {"Node name for S&R": "PreviewImage"}},{"id": 74,"type": "LoadImage","pos": [-2113,-24],"size": {"0": 349.269775390625,"1": 459.3564147949219},"flags": {},"order": 0,"mode": 0,"outputs": [{"name": "IMAGE","type": "IMAGE","links": [133],"shape": 3,"slot_index": 0},{"name": "MASK","type": "MASK","links": null,"shape": 3}],"properties": {"Node name for S&R": "LoadImage"},"widgets_values": ["IMG_20240617_070212_131.jpeg","image"]},{"id": 76,"type": "SD3MCN_BaseModelLoader_Zho","pos": [-1424,-42],"size": {"0": 428.3705749511719,"1": 95.87933349609375},"flags": {},"order": 1,"mode": 0,"outputs": [{"name": "pipe","type": "SD3MMODEL","links": [132],"shape": 3,"slot_index": 0}],"properties": {"Node name for S&R": "SD3MCN_BaseModelLoader_Zho"},"widgets_values": ["stabilityai/stable-diffusion-3-medium-diffusers","InstantX/SD3-Controlnet-Canny"]},{"id": 78,"type": "SD3MCN_Generation_Zho","pos": [-1436,94],"size": {"0": 444.2547607421875,"1": 344.82293701171875},"flags": {},"order": 3,"mode": 0,"inputs": [{"name": "pipe","type": "SD3MMODEL","link": 132},{"name": "image","type": "IMAGE","link": 134}],"outputs": [{"name": "IMAGE","type": "IMAGE","links": [137],"shape": 3,"slot_index": 0}],"properties": {"Node name for S&R": "SD3MCN_Generation_Zho"},"widgets_values": ["close-up of a young girl, masterpiece, best quality, photo, realistic, very aesthetic","worst quality, low quality, toon",512,512,40,9,1,542410872807786,"randomize"]}],"links": [[132,76,0,78,0,"SD3MMODEL"],[133,74,0,77,0,"IMAGE"],[134,77,0,78,1,"IMAGE"],[137,78,0,80,0,"IMAGE"],[139,77,0,82,0,"IMAGE"]],"groups": [],"config": {},"extra": {"workspace_info": {"id": "-gBBQ2xIYibjLVAhj5iML"},"ds": {"scale": 1.3513057093104035,"offset": {"0": 2165.8774539604974,"1": 287.4375476041587}}},"version": 0.4
}

版权声明:

本网仅为发布的内容提供存储空间,不对发表、转载的内容提供任何形式的保证。凡本网注明“来源:XXX网络”的作品,均转载自其它媒体,著作权归作者所有,商业转载请联系作者获得授权,非商业转载请注明出处。

我们尊重并感谢每一位作者,均已注明文章来源和作者。如因作品内容、版权或其它问题,请及时与我们联系,联系邮箱:809451989@qq.com,投稿邮箱:809451989@qq.com