Combining an Opencode coding model and a ComfyUI image generation model

 

I setup ComfyUI as I posted about previously.  I also have Opencode setup with some local models.  For this I used Qwen 3.6 coder.

I had the model help me write a python script that can call the ComfyUI image generation api endpoint.  After you have a workflow setup in ComfyUI you can download json that can be posted to ComfyUI to run that workflow.  The python script allows passing the image prompt and the dimensions as command line arguments.

Then I asked the model to use the python script and generate an html image gallery with six pictures of nature scenes.  The model was able to come up with 6 prompts, run them through the script and save the images.  It then built a simple html page that shows the images.  You can see this in gallery.html in the repo.

I then setup another workflow in ComfyUI to remove backgrounds from images so you can make pngs with transparent backgrounds.  I took that script and the original one and moved them into a skills folder with a SKILL.md explaining how to use them.

Then I asked the coding agent to make a simple solar system simulation with a star and 4 planets.  I told it to use the new skill to generate images with transparent backgrounds.  This took a few tries and some rearranging of the scripts and the skill before it worked, but the end result looks good.  You can see it in simulation.html in the repo.

Here is the repo with all the code I talked about in this article https://github.com/mattdyer/comfyuiapi