Draw-Things://api
﹖
Preamble
﹦{
🤖:
Itʼs Automation April!}
What better time to start a conversation on how Draw Things could celebrate?!
- Since launch, I've come across numerous examples of community members requesting that automation capabilities of one sort or another get added to Draw Things. Ideas that repeatedly pop up include adding built-in Shortcuts app support using rich native Draw Things actions, or users wishing for the ability to control and script DT from the Mac's command line.
- Either of those options would be amazing (particularly the Shortcuts integration, as it would be both user friendly and entirely cross-platform), and for DT to support either would be undeniably awesome. With that said, if automation workflows are still on the roadmap and if the community and Liu Liu agree with my reasoning,
I'd like to propose that any automation-based development efforts be (at least initially) invested in setting up a dedicated
draw‑things://
offline uri scheme, bestowing users with a simple interface to send detailed commands to the app via URL. - I feel that a local API approach such as this would likely be much faster and easier to integrate, and significantly more versatile than the other options mentioned at the start. Additionally, since it's both straightforward and commonplace to interact with JSON-based APIs from within Apple Shortcuts and from the command line, users would be able to automate Draw Things using just about any tool at our disposal.
Auxiliary benefits / "feature freebies" with this approach that spring to mind:
- As Draw Things already uses a JSON based format to save a gen's formula into every creation's EXIF metadata, there's already a foundation in place for the various keys and values that a user might compose an API call with.
- With the API implemented, Draw Things would inherently be capable of dynamically adjusting all of its settings based on essentially the exact same data as what's included in image metadata. As a result, it would likely be developmentally trivial for the app to also gain the ability to update its settings to match this metadata whenever a user (re-)imports images manually from Files or Finder, allowing users to easily transfer gens+settings from Project to Project by simply exporting and importing, or even let users share the complete "blueprint" / "recipe" for a particular gen just by sharing the image file itself.
- The Custom Presets feature could be completed, as Presets could also share the same JSON format as what the API uses.
- Support for the "X-Callback-URL" framework is a staple with these types of APIs, allowing apps to automatically bounce back and forth exchanging data. When implemented, it would allow Draw Things to automatically output results after being called from a Shortcut for example, so that even after a lengthy image generation task (which normally would cause a Shortcut that's been waiting for a response to time out), DT could auto-launch the original SC which triggered it (or even a different Shortcut or seperate app entirely if specified), feeding the resulting image(s) back to Shortcuts for further processing.
X-callback-url's website recommends that devs check out the Inter-App Communication framework on Github which was recently updated/ported to Swift, to easily drop callback support into their app's main API.
- Having this kind of system in place could also serve as a solid backbone for creating other ways to automate DT in the future, and since it's quite common for apps to interact with each other using these types of APIs, it could even open up the possibility for other devs to step in and potentially extend Draw Things' capabilities even further.
For example, another dev could release a helper app for Shortcuts that adds those 'native' custom UI actions, which could interact with DT through the official API rather than needing to be built into the main app's code base, or someone could make a Prompt Library for storing frequently reused prompt snippets that could be sent into DT on demand.
TL;DR 💁 :
... Automation's cool mmkay! Gimme?
TS;NMI! 🙏 (too short; need more info):
... When I started drafting this incredibly brief suggestion a few
daysweeks (🥴) ago, I began contemplating precisely what a fully-formed, powerful, and feature-rich Draw Things API might actually look like.
- I then thought Hey! I should really jot down all these ideas into a little mini mock-documentation (mockumentation?) to include alongside my suggestion!
- I'd never used Rentry before, but I decided to put the post on here after quickly becoming frustrated with Discord's limited formatting capabilities. In retrospect, this may have been a poor move, as I'm very easily sucked into tinkering with formatting and making things look good (to my eyes at least), and as a result of Rentry's expanded Markdown toolset, I've ended up working on this way more than I had anticipated, which in turn has cursed me with ample time to come up with an ever expanding list of featureset ideas to put into a fictional documentation that could easily end up being completely pointless. *sigh*
- My hope is obviously for something resembling the API below to arrive one day for real, and if it does, then perhaps the thoughts I've put into all of this might be end up being a useful reference while it's actually being designed. However, I'm not a designer nor a coder, only a technically-inclined enthusiast, and although I've tried my best to construct a cohesive set of functionality which I think would be perfectly feasible (after researching similar APIs used by other apps and looking at each feature from as many angles as possible to figure out how everything would interact), it's nevertheless completely conceivable that OS limitations or something else entirely could prevent any of it from actually being possible.
So without further ado, you'll find the "documentation" below, and thanks so much for reading!
Documentation
Disclaimer
In case you stumbled upon this without reading the preamble above and were under the impression that the capabilities outlined below are real, they're not. This is a proposal for how such an API might work if implemented, and as of 04/2023 is entirely theoretical.
It was written as if it were already implemented and supported all the features I could dream up at the time, but purely as a technical exercise.
If we do someday get these features, then perhaps everything below might be of use with documentation efforts for the real thing ;-)
If you'd be as excited for something like this to exist as I'd be, make sure to check out the Draw Things Discord and voice your support!
Table of Contents: API /endpoints
& parameters
- Introduction
- Examples: a quick primer on how API calls are formulated
- x-callback-url: generic parameters which can be used with all API endpoints
callback_parameter
: "name" of URL parameter where API results will be storedx-success
: customize the App/URL to launch if task succeedsx-cancel
: customize the App/URL to launch if task is aborted by userx-error
: customize the App/URL to launch if task fails- Requesting and Using an API Key: a Key is required for certain endpoints and sensitive requests
1. Generating Images ❯ /generate
preset
: specify a DT preset to set a configuration baseline automaticallyinput
: specify the source used to transmit image layer dataimage_data
: the actual image layer to be used, encoded with base64image_format
: file extension of the decoded imagecount
: number of images to generateoutput
: destination for generated imagesproject
: project file where image generation should take placesettings={
…}
: primary JSON dictionary used to configure Draw Things
"prompts": [
{ ≔ }]
: an array of one or more dictionaries with text used for prompting"config": [
{ ≔ }]
: an array of one or more dictionaries with individual settings to customize (JSON)"additives": [
{ ≔ }]
: an array of one or more dictionaries containing settings for additional models (LoRA, ControlNet)- Multi-Config Mode: how to apply different settings variations in sequence, per image-generation
- Handling Output: what to expect when you're expecting (your
/generate
results)
2. Manipulating Prompts ❯ /prompts
get
: ouputs a prompt box's current contents as a stringset
: selects the prompt box you'd like to modifyprompt_text
: the prompt string you'd like to injectoperation
: specifies the type of text-manipulation task to be performedtarget
: narrows an operation's scope to a specific string within the promptprompt_data={
…}
: an alternative (JSON) method to send advanced prompting commands- Handling Output: what to expect when you're expecting (your
/prompts
results)
3. Querying Projects ❯ /projects
task
: defines the function that will be performedtask_data={
…}
: specifies alltask
parameters- Handling Output: what to expect when you're expecting (your
/projects
results)
4. Organizing Models ❯ /models
⓿ Misc.
API introduction and parameters applicable to all API endpoints
If you already have a general understanding of making API requests and x-callback-urls, feel free to jump directly to the documentation for
/generate
, although the example calls below might still be worthwhile to review.
Intro:
I spent a good deal of time researching similar APIs used by various other iOS apps, borrowing concepts applicable to this API, and attempting to make sure that everything listed is realistically plausible to implement (tying into the existing Draw Things featureset as much as possible).
With that said, once I got on a roll, I couldn't help but take a few creative liberties by "supporting" the ocasional feature which DT has yet to actually roll-out (i.e using Custom Presets even though the presets in DT are not yet customizable).
I tried my best to think through and address potential conflicts and roadblocks that might prevent particular features from working as expected, but I'm sure there's plenty which never crossed my mind. If any devs are reading this, I'd be very curious to know how difficult something like this would be to actually code in Swift (of course I'm most curious to hear Liu Liu's thoughts on the matter 😉)
If anyone has any neat ideas for other possible endpoints and API requests, please share!
Examples:
Here's a couple of example API calls to get a feel for what would be possible or to understand how integrations like these tend to work for anyone unfamiliar.
A Draw Things Automation URL at its most basic might look like this:
draw-things:///generate
?preset
=Upscale with Remacri&input
=clipboard&output
=clipboard
Triggering this link would simply launch Draw Things, paste whichever image is currently in your clipboard to the canvas, configure all settings using the existing preset "Upscale with Remacri", hit Generate, and copy the result back to the clipboard.
A more complex example:
draw‑things://x‑callback‑url/generate
?count
=3&output
=url+export&settings
={JSON_Data}&callback_parameter
=input&x‑success
=shortcuts://run‑shortcut?name=MyShortcut
This URL would launch DT and do the following:
- Set the number of images to be generated to 3.
- Tell DT to save each image it generates to the current "Save Generated Images to:" folder, and additionally output the images as base64 encoded text to the URL it launches upon completion. It's also worth mentioning that based on these
output
choices, Draw Things would be asked to delete the images it just generated from the project once export is completed. More details on this can be found in the API documentation.- Extract and set the prompt boxes to what's in the
"prompts": [
…]
array (part of thesettings={
…}
JSON data shown below), using slightly different prompts for each sequential gen.- Configure all app and model settings based on the dictionaries within the
"config": [
…]
and"additives": [
…]
arrays (below).- When finished, thanks to the x-callback-url framework, DT would auto-launch a Shortcut named MyShortcut, and pass the 3 generated images to it since url was in the chosen
output
actions. As the Shortcut looks for data passed to it in a parameter named input, we've manually named the URL parameter where DT's output is stored to "input" so that it's properly recognized.
{JSON_Data} Contents:
三👆⃡
The
"layer_data": {
…}
object within the"control": [{
…}]
array contains Pose information in an OpenPose JSON format.Sequential Prompts:
You'll notice there's multiple dictionaries in the
"prompts": [
{…},
{…},
{…}]
array. Ascount
is set above 1, Draw Things can modify its parameters for each sequential generation.In the example:
1. The first image created will use the first dictionary in theprompts
array.2. The second image's positive prompt gets replaced by what's in the second dictionary while keeping the negative prompt from the first (since there's no negative prompt specified in the second dictionary)
3. For the third image, both positive as well as negative prompts are replaced.
Although this example doesn't demonstrate it, changing
settings={
…}
on a per-gen basis isn't limited only toprompts
.
See the Multi-Config Mode section for details.
URL Encoding
- For legibility purposes, the example API calls above are not properly URL/Percent encoded, although any actual DT URLs would have to be.
Specifically, the space characters in Upscale with Remacri from the first example would each become %20, and all of the symbols in the
x-success
Shortcuts URL from the second example would use percent encoding as well. - The expanded {JSON_Data} is shown with added whitespace to enhance readability, and apart from needing percent-encoding for all non-alphanumeric characters within, it would additionally need to be on a single line once ultimately added to the URL.
Here's what those transformations would look like
- First, it should be stripped of all superfluous whitespace (something that is typically already done with any machine-produced JSON). When "minified" in this way, the example JSON would look like this:
三👆⃡ - Next, it would need to be fully URL percent-encoded, resulting in the actual string that would replace the {JSON_Data} placeholder:
三👆⃡ %7B%22 prompts
%22%3A%5B%7B%22positive
%22%3A%22analog%20style%20RAW%20(color%3A1.25)%20photo%2C%20lumberjack%20chopping%20wood%20next%20to%20a%20cabin%2C%20soft%20lighting%2C%20high%20quality%2C%20film%20grain%22%2C%22negative
%22%3A%20worst%20quality%2C%20%3Cbad_prompt%3E%22%7D%2C%7B%22positive
%22%3A%20analog%20style%20RAW%20photo%2C%20clown%20lumberjack%20chopping%20wood%20next%20to%20a%20carnival%20ferris%20wheel%2C%20soft%20lighting%2C%20high%20quality%2C%20film%20grain%22%7D%2C%7B%22positive
%22%3A%20analog%20style%20wide%20angle%20photo%2C%20sentient%20anthropormorphic%20tree%20chopping%20up%20lumberjacks%20at%20a%20party%2C%20dramatic%20lighting%2C%20high%20quality%2C%20film%20grain%22%2C%22negative
%22%3A%22%3Ceasynegative%3E%22%7D%5D%2C%22config
%22%3A%5B%7B%22model
%22%3A%22liberty_f16.ckpt%22%2C%22sampler
%22%3A%22DDIM%22%2C%22scale
%22%3A7.5%2C%22initial_seed
%22%3A2673245%2C%22seed_mode
%22%3A%22Scale%20Alike%22%2C%22size
%22%3A%22768x768%22%2C%22steps
%22%3A30%2C%22strength
%22%3A1%2C%22upscaler
%22%3A%22realesrgan_x2plus_f16.ckpt%22%7D%5D%2C%22additives
%22%3A%5B%7B%22lora
%22%3A%5B%7B%22model
%22%3A%22test_v10_lora_f16.ckpt%22%2C%22weight
%22%3A0.35%7D%5D%2C%22control
%22%3A%5B%7B%22model
%22%3A%22controlnet_pose_1.x_f16.ckpt%22%2C%22guidance_start
%22%3A0%2C%22guidance_end
%22%3A0.6%2C%22weight
%22%3A1%2C%22layer_input
%22%3A%22json%22%2C%22layer_data
%22%3A%7B%22width
%22%3A768%2C%22height
%22%3A768%2C%22keypoints
%22%3A%5B%5B280%2C246%5D%2C%5B267%2C359%5D%2C%5B147%2C357%5D%2C%5B121%2C538%5D%2C%5B121%2C538%5D%2C%5B390%2C358%5D%2C%5B440%2C493%5D%2C%5B423%2C604%5D%2C%5B214%2C654%5D%2C%5B214%2C654%5D%2C%5B214%2C654%5D%2C%5B367%2C647%5D%2C%5B367%2C647%5D%2C%5B367%2C647%5D%2C%5B260%2C222%5D%2C%5B298%2C224%5D%2C%5B226%2C223%5D%2C%5B321%2C229%5D%5D%7D%7D%5D%7D%5D%7D
X-Callback-URL
♼
parameters
Finetunes the action that occurs following your API request
Most /endpoint
URLs are prefaced with draw-things://x-callback-url, which instructs DT to perform the requested action and then send the results of that request to another app for further processing (typically the original app where the request was formulated such as Shortcuts.app).
The x—
parameters below are normally populated automatically by the originating app, but can be overriden manually if you'd like results to be sent elsewhere. Values must be URL-encoded.
x-success
(optional)
Allows specifying a URL to launch upon task completion
By default, this URL is set to the callback url of whichever application made the initial request.For example, if Draw Things were invoked by an Apple Shortcut using the 'Open X-Callback URL' action, then as soon as DT finishes processing the request, output will be routed back to the Shortcut which invoked it. If you'd like to specify a different Shortcut to run, or a different x-callback compliant application entirely, you can use the
x-success
parameter to do so.
x-cancel
(optional)
Allows specifying a URL to launch when task is manually cancelled
By default, this URL is set to the "cancelled" callback url of whichever application made the initial request.
x-error
(optional)
Allows specifying a URL to launch if task fails to complete due to error
By default, this URL is set to the "error" callback url of whichever application made the initial request.
callback_parameter
(optional)
The name of the parameter within the callback url where the output is stored. Default is "result".
Useful if the app receiving DT's output expects its input to be named something specific.
Using
⚿⧆⧆⧆
API Keys
API requests that deal with potentially destructive tasks or which use /endpoints
handling private data require the inclusion of a special password/token, or "API Key"
To use this password in your requests or to create one, you'll include the following parameter in the API url:
api_key
(required for certain requests, otherwise optional)
- dpv8qRfQhvyCjTv24AC... (the key): a long random alphanumeric string proving your identity for enhanced API privileges.
- new: instructs DT to create (or re-generate) a new API key.
Creating your api_key
:
Submitting a request with
api_key
=new will only work with non-x-callback-url URLs but can be sent to any of the available/endpoints
, even those which otherwise only respond to x-callback-url requests.When the API detects this parameter value, the "New Key Generation" event kicks off in the DT UI and any other parameters you may have included in the request are ignored, so it's best to make a dedicated request for the sole purpose of generating your key, like so:
draw-things://
/generate
?api_key
=newYou can replace
/generate
with/project
or any other/endpoint
.
- Upon requesting your key, Draw Things will launch and present a dialog in the UI, prompting you to explicitly confirm your request.
- Once you've tapped the Create/Replace API Key button, your API key will be shown on screen alongside a button allowing you to copy it to the system Clipboard.
- Store it someplace safe as it won't be shown again, although at any time you can simply recreate a new key (invalidating the old one).
- Once you have your key, you can add the
api_key
=your key parameter/value to any request that benefits from enhanced privileges.Don't share your API Key with anybody else
If someone else were to obtain your current key and use it to create a Shortcuts.app shortcut (for example) with malicious intent, if you were to then run this shortcut without first verifying precisely what it does (something you should always be doing by the way!), then that shortcut could perform destructive or invasive actions such as overwriting your presets or retrieving private data within DT like the images you've generated.
Endpoint
❶
䷀
Generating
䷼
Images:
⛾
/generate
draw-things:///generate | draw-things://x-callback-url/generate
Launches Draw Things and begins image generation based on the parameters included in the URL.
If any (optional) parameters are omitted, DT will use whatever is currently configured in the UI.
The following options are supported:
preset
: specify a DT preset to set a configuration baseline automaticallyinput
: specify the source used to transmit image layer dataimage_data
: the actual image layer to be used, encoded with base64image_format
: file extension of the decoded imagecount
: number of images to generateoutput
: destination for generated imagesproject
: project file where image generation should take placesettings={
…}
: primary JSON dictionary used to configure Draw Things
"prompts": [
{ ≔ }]
: an array of one or more dictionaries with text used for prompting"config": [
{ ≔ }]
: an array of one or more dictionaries with individual settings to customize (JSON)"additives": [
{ ≔ }]
: an array of one or more dictionaries containing settings for additional models (LoRA, ControlNet)ꜛx-success
: customize the URL to open if task succeedsꜛx-cancel
: customize the URL to open if task is aborted by userꜛx-error
: customize the URL to open in the event task fails to completeꜛcallback_parameter
: name of URL parameter where results will be appendedꜛapi_key
: only used with this endpoint when when overriding apreset
Additional Info
- Multi-Config Mode: how to apply different settings variations in sequence, per image-generation
- Handling Output: what to expect when you're expecting (your data results)
preset
(optional)
Configures DT according to the specified preset.
- preset name (required) : The name of an existing preset, written exactly as it appears within the DT 𓉘⚙︎ 𝐏𝐫𝐞𝐬𝐞𝐭𝐬𓉝 menu. If not found, operation will cease with an error.
input
(optional)
Method DT should use to populate the image canvas prior to image generation.
- none (default) : primary image layer is empty.
- clipboard : will import the image that has been copied to the clipboard.
- data : will import the image provided as parameter (requires
image_data
).
image_data
(optional)
Pre-fills the canvas' image layer, for img2img generation.
If you'd like to populate other layers instead of (or in addition to) image, such as the masking, depth map, pose, color palette, scribble, or custom layers, see the"ꜜadditives"
: [
≔]
array below.
- iVBORw0KGgoAAAANSUhEUg... (required) : base64 encoded image data that is provided as part of the URL (base64 data stream must be URL encoded). Image format can be any format supported by iOS (see
image_format
).
image_format
(optional)
This is a hint to help DT disambiguate actual format in some situations.
- ext (required) : Format of
image_data
, expressed as a file extension (ex: 'png').
count
(optional)
Integer representing the number of images to generate.
See Multi-Config Mode to learn how to apply different settings variations based oncount
- 1 (default) : generate a single image.
- 2—100 : generates the specified number of images in sequence.
- 0 : image generation will continue indefinitely until manually cancelled by selecting the 𓉘■⃝ 𝐒𝐭𝐨𝐩𓉝 button in the app.
𓉘■⃝ 𝐒𝐭𝐨𝐩𓉝 button behavior
When using
ꜛx-callback-url
with a finite (1-100)count
, pushing 𓉘■⃝ 𝐒𝐭𝐨𝐩𓉝 mid-generation would typically trigger thex-cancel
event/url, because the task was aborted prematurely. Whencount
is 0 however, pressing 𓉘■⃝ 𝐒𝐭𝐨𝐩𓉝 is considered normal and expected, and will activate thex-success
url instead.
output
(optional)
External destination for generated images.
Multipleoutput
options can be chosen at once by chaining them together using a + character
(ex: draw-things:///generate?output
=canvas+clipboard+export...)
- canvas (default) : images remain in the currently active
ꜜproject
. If otheroutput
options are selected instead of canvas, generated images will be removed from theproject
file's image history. This can be prevented by including canvas alongside the other options (as shown above).- url : used alongside
ꜛx-callback-url
to include the output images as base64 encoded text within theꜛcallback_parameter
. See ꜜHandling Output below.- export : following each image generation, the result is saved to whichever folder is currently selected in the app's Save Generated Images to setting.
- clipboard : after image generation has ceased, results are written to the system pasteboard.
Cleanup behavior if canvas is omitted
▸ When export is the only
output
used, gens are deleted from the Canvas after each image is generated and successfully saved in the export folder.▸ If
output
includes either url or clipboard however, the images aren't deleted until the entirecount
of images have finished processing.▸ Additionally, when
output
is set to non-canvas options, upon completion of all tasks and removal of image generations from the history, ALL settings in the UI will revert to what they were prior to your API request.Folder availability for export
If there's no export folder currently being used (Save Generated Images to is Disabled in the app), then images will be exported to a new folder called /Output/ that will be created within Draw Things' internal Documents folder.
▸ On iOS, this folder is simply named /Draw Things/, and can be found within the Files app under On My iPad/ or On My iPhone/.
▸ On Mac, it's located at ~/Library/Containers/Draw Things/Data/Documents/ (although the path can look somewhat different depending on your version of macOS).
project
(optional)
The name or ID of the Project file where image generations take place
Ignored unlessoutput
contains the canvas value, as nothing is stored in the project file otherwiseWill first attempt to match based on ID, if that fails, project filenames will be compared, if no match found then a new project file will be created using the name specfied.
settings
(optional, JSON)
The main JSON dictionary that defines the majority of image-generation parameters
Contains any combination of three primary arrays, namedprompts
,config
, andadditives
. Each array can include one (or multiple in Multi-Config Mode).
prompts
(required*)
A (URL-encoded) JSON dictionary containing one or two
key
:value pairs corresponding to the prompt boxes in DT.
- Valid keys are "
positive
": and "negative
".- value for either would be a string of text describing your prompt
Requirement caveat
Although
prompts
is generally considered "required", it isn't a technical requirement (i.e if excluded, your request won't result in error/cancellation), it's simply rare that you wouldn't want to include at least some form of prompting.One exception where
prompts
is ignored entirely and could be excluded if desired, is after adding a ControlNet object to the "control
":[{…
}] array with its "no_prompt
": key set to true. (see the"ꜜadditives"
:{…
} parameter below for details)
config
(optional)
A (URL-encoded) JSON dictionary containing one or more DT settings overrides, as
key
:value pairs.
Any setting that isn't specified byconfig
directly will remain in its current configuration, as shown in the app.Using
ꜛpreset
+config
togetherIf both parameters are provided, any individual settings specified within
config
takes precedence over that same setting in the preset used whenever there's overlap/conflict. This allows you to usepreset
to easily establish your base/default configuration, and reduce the number of options needed withinconfig
to only the specific overrides you'd like to apply to the Preset.
"config": {
…
}
Key Example Value Type Description " model
":"liberty_f16.ckpt", string filename* of the primary model " sampler
":"DDIM", string name of the sampler " scale
":7.5, integer CFG 0.0—25.0 (text guidance) " initial_seed
":2673245, integer starting seed (first image generated)* " seed_lock
":false, boolean true or false (default). DT won't randomize initial_seed
for additional gens*" seed_mode
":"Scale Alike", string one of "Scale Alike", "Legacy", or "Torch CPU Compatible" " size
":"768x768", string one of available dimensions listed within DT " steps
":30, integer step count 5—150 " strength
":1, integer 0.1—1 " upscaler
":"remacri_4x_f16.ckpt", string filename of upscale model* " clip_skip
":2, integer 1—23 " hires_fix
":true, boolean true or false " first_stage_size
":"512x512", string one of available dimensions listed within DT* " second_stage_strength
":0.7, integer 0.1—1 " new_preset
":"My Neat Settings", string saves final config as new preset* " preset_type
":"override", string complete (default) or override. Defines the preset's scope* 三👆⃡
* Notes
model
,upscaler
: Filename as shown in the /Models/ folder
initial_seed
: Normally, if a particular key isn't included withconfig
, the current value in the UI (or a default value if unset in UI) is used.initial_seed
is an exception: if not included, the seed will always be randomized, unlessinitial_seed
+seed_lock
were set in a priorconfig
instance earlier in the API request (see Multi-Config Mode)
seed_lock
: While true, whichever seed is currently set byinitial_seed
is locked in place for current and subsequent image-generations.
first_stage_size
: Smaller thansize
, of the same ratio (1:1, 2:3, etc). Ignored ifhires_fix
is uneccessary or false. Defaults to one image-size step belowsize
.
new_preset
: All configured settings are saved into DT's⛭ Presets
for later re-use using the name specified. If a preset already exists with the same name as what was set withnew_preset
, the preset will be updated/overriden, only if the API request also includes a validapi_key
parameter. If not, for data safety the existing preset won't be replaced, and Draw Things will instead add an index to the end of the new preset's name so that it's unique (i.e My Neat Settings2 3...4...etc). You can then delete the original preset within the UI and rename the new one you've saved.
preset_type
(requiresnew_preset
): The process that is used to assemble a new preset is controlled by this setting.Rethink this - maybe choice between a complete preset and an override preset? figure out the differences.
The new preset will be built in the following sequence:
1. A baseline is determined by storing a snapshot of the current UI configuration (prior to any current adjustments).
2. Settings are then overriden using whichever preset you've specified in the
preset
parameter (if any).3. Finally, new adjustments configured using
config
oradditives
are applied, and the new preset is stored.settings values within your new preset will first take into account anything loaded using the
preset
parameter, followed by any overrides to that preset specified inconfig
, oradditives
. preset you've saved will absorb its various options in the following order:1. serve as an "override" preset, meaning that only the settings which you have specifically configured individually using
config
additives
(optional)
A (URL-encoded) JSON dictionary containing arrays defining secondary models or functions, and their settings.
Valid array names are "control
":[ {…
} ], "lora
":[ {…
} ], and "mask
":[ {…
} ].
Each array can include one or more dictionary objects, with each one specifying the settings for an individual additive function."
mask
":[ {…
} ]Mask layers can be used for img2img/inpainting. They're applied to an image layer that has been previously defined using the
ꜛinput
parameter (required)
Key Example Value Type Description " layer_input
":"data", string source of masking input. data, or clipboard " layer_data
":{ …
},object dictionary defining input. Same contents as what's described for ControlNets (using data) below. 三👆⃡
"
lora
":[ {…
} ]LoRAs are defined using a dictionary with the following two keys:
Key Example Value Type Description " model
":"analog_diffusion_v1_lora_f16.ckpt", string .ckpt filename of the LoRA model " weight
":0.85, integer LoRA strength, between -0.5—1, default 0.6 三👆⃡
"
control
":[ {…
} ]ControlNets are defined using a dictionary with the following:
Key Example Value Type Description " model
":"pose", string name of cnet model. pose, depth, canny, scribble, palette, normal, lineart, lineart_anime, or softedge " version
":"1.1", string cnet model version*. 1.0, 1.1, t2i_adapter, or latest (default) " weight
":1, integer cnet strength, between 0 and 1 (default) " guidance_start
":0, integer 0 (default) to 1, moment cnet influence begins " guidance_end
":0.5, integer 0 to 1 (default), moment cnet influence stops " layer_input
":"data", string source of guidance input. data, json, image, or clipboard " layer_data
":{ …
},object dictionary defining input. See below for content possibilities " no_prompt
":false, boolean if true, ignores ꜛprompts
and generates from cnet alone.三👆⃡
Notes
version
: If not specified, DT automatically selects the latest CNet version available that matches the base version of Stable Diffusion used by the primarymodel
(in"config": {
…
}
). If an incompatibleversion
is chosen (for example, by specifying t2i_adapter while using an SD v2.x primarymodel
), DT will attempt to auto-select the latest appropriateversion
that is compatible. If a comparable and compatible version doesn't exist (for example if you attempt to use a lineart_anime cnet to control a SD v2.x primarymodel
), the entirecontrol
array will be ignored.
layer_input
: with image, the CN influence layer is extracted from the image layer on the canvas.
Requires mainꜛinput
parameter to have set this already.Layers which can be extracted this way (when possible) are depth (from Portrait photos only), pose, canny, or palette.
clipboard will use the image currently on the system pasteboard.
data uses a base64 encoded image and, along with json (usable with pose cnet only), relies on information within the "
layer_data
": {…
} object, shown here:if "
layer_input
": "data",
"
layer_data
": {≔
}
Key Example Value Type Description " data
":"iVBORw0KGgoAAAAN…", string (url encoded) base64 image data (required) " format
":"jpeg", string file extension of the encoded image 三👆⃡
if "
layer_input
": "json",
"
layer_data
": {≔
}
Key Example Value Type Description " width
":512, integer dimensions of OpenPose pose file " height
":512, integer dimensions of OpenPose pose file " keypoints
":[
[…
],[…
],[…
],[…
]]
array list of arrays that define and draw a pose. Note: there are multiple OpenPose JSON formats that are recognized, so for example the keypoints
array might instead be apeople
array三👆⃡
Multi-Config Mode
Apply varried configurations to a sequence of image-generations
- Subsequent settings individually override previous sticky settings (setting pos and neg prompts in first gen are kept for second gen unless new/empty prompts are given, applying a controlnet to first gen is kept for subsequent ones, but if a new controlnet dictionary is included, it overrides previous even if it's a different type
- if
count
exceeds number of variation dictionaries, DT cyles back to the first variation. Best for first variation to include a settings baseline so that override mechanic doesn't cause unanticipated results
Handling /generate
Output
Endpoint
❷
䷀
Manipulating
䷼
Prompts:
⛾
/prompts
draw-things:///prompts | draw-things://x-callback-url/prompts
Launches Draw Things and begins image generation based on the parameters included in the URL.
If any (optional) parameters are omitted, DT will use whatever is currently configured in the UI.
get
(optional)
Retrieves the contents of one or both Prompt boxes as shown in the Draw Things UI.
Usable withꜛx-callback-url
only. Unnecessary if using any other parameters, as all other prompt editing-based functions will include the same output asget
=both in their x-callback results
- both : the results/
callback_parameter
will consist of a dictionary with prompt data. See the ꜜHandling Output section for details.- positive : returns only the text from the Positive Prompt box, as a string (rather than a dictionary)
- negative : returns only the text from the Negative Prompt box, as a string (rather than a dictionary)
set
(optional)
Selects the prompt to modify
Can be used with either callback or normal endpoint request.
- positive (requires
prompt_text
)- negative (requires
prompt_text
)
𝕘𝕠! prompt_text
(required for set
, otherwise ignored)
Specifies the string of text used when updating a single prompt-box
- Some prompting text : the string of text you'd like to input
operation
(optional)
Specifies the type of text-manipulation to perform when modifying a single prompt-box
Ignored unlessset
=positive or negative.If
target
is set and successfully matches, only the matching text is modified rather than the entire prompt.
- replace (default) : all content within the prompt-box being
set
will be replaced byprompt_text
.- prepend :
prompt_text
is added to the beginning of theset
prompt-box's current contents (separated with a space character).- append :
prompt_text
is added to the end of theset
prompt-box's current contents (separated with a space character unless string begins with a comma).
target
(optional)
Adjusts which text gets modified by
operation
, enabling you to inject a string at a precise point in the existing prompt, or to substitute one string for another
If Draw Things cannot find a precise match for thetarget
string, the normal behavior ofoperation
takes effect.
- Query String : the existing prompt text to search for. If multiple matches are found, only the first is used.
Tips
Prompt adjustments using
target
can be very useful. You could build an Apple Shortcut that stores frequently re-used prompt snippets, or that contains a library of prompt Wildcards, and then easily trigger the Shortcut to insert a snippet anywhere in your prompt, or you could write the prompt with special placeholder tokens, which your Shortcut could swap out for items in the Wildcard library, for example, if your Positive Prompt were built like this:A detailed portrait of __subject__ in the style of __art:style__, art by __art:artists__
Your Shortcut could then make sequential x-callback-url
/set
calls to quickly make targeted replacements of each special placeholder token with a random item from the appropriate list of wildcards, or it could even replace all tokens in a single request by submitting multipleoperation
tasks within aprompt_data
dictionaryIt's best to ensure that whenever you're making a
target
ed replaceoperation
, you should ensure that the string you want to find/replace exists in the prompt box, because iftarget
cannot find a match the non-target
behavior is used as a fallback, meaning yourprompt_text
would replace the entire prompt rather than a simple word or two. This is less of an issue with the other forms ofoperation
, since in those cases your additional text would simply be put at the start or end of the prompt-box.
prompt_data
(optional, JSON)
When used, all other parameters are ignored, and the request is built entirely from the instructions included in the
prompt_data
JSON dictionary
The parameters you can configure using JSON (as "key
": "value" pairs) are identical to what's shown above, except that the formatting is a little different.set
is the equivalent of the primaryprompt_data
dictionary, with the positive and negative choice expressed as individual arrays containing one or more dictionary objects that define the text-manipulation parameters (prompt_text
/operation
/target
).If your prompt modification needs are met using the simple parameter-based functionality detailed above, then formulating your request using
prompt_data
is likely unecessary, although there are a couple features only available using JSON:
- With
prompt_data
, you can modify both positive and negative prompt-box texts at once, rather than needing two sequential API requests.- You can perform multiple, distinct, back-to-back
operation
commands on the prompt-boxes, letting you inject or substitute several chunks of text in quick sequence.
- insert (requires
target
) :prompt_text
is injected immediately after the first successfully matchedtarget
text.- substitute (requires
target
) :prompt_text
replaces the first successfully matchedtarget
text.
Handling /prompts
Output
positive
and negative
keys, with the respective prompt-box contents as values. The dictionary will also contain a third key called tokens
, with an integer value representing the total amount of CLIP (change all this to instead output a positive and negative dictionary, each with keys for text
, token_count
, and overflow_text
Endpoint
❸
䷀
Querying
䷼
Projects:
⛾
/projects
draw-things:///projects | draw-things://x-callback-url/projects
Allows you to retrieve info on your project files, extract or transfer images, and more.
All tasks require sending your ꜛapi_key
.
The following options are supported:
task
: defines the function that will be performed
list
: retrieves an array listing all saved project files and their basic metadataget_project
: returns expanded metadata on the contents of a specific project file, by project_idget_image
: returns one or more images from a specific project_id, by history_idtransfer_image
: copies or moves images between projectstask_data={
…}
: specifies alltask
parameters
filter={
…}
: instructslist
to limit its results to projects which match certain criteria
query
scope
project_id
images:[
≔
]
destination_id
transfer_mode
: copy/move
query
(optional)
This is a hint to help DT disambiguate actual format in some situations.
- ext (required) : Format of
image_data
, expressed as a file extension (ex: 'png').
Models Endpoint
drawthings:///models | drawthings://x-callback-url/models