[FEATURE] Saving of Dynamic Prompts text inputs with variables

[masslevel]

Hi, all!

I wanted to ask if anyone has got a good workflow (beside having a general txt file) how to manage and save their raw prompt builds with all the Dynamic Prompt variables?

I apologize if something like this does already exist and I have missed it!

Current workflow

Maybe it's just my workflow, but I have been repeating the same steps over and over to reconstruct Dynamic Prompt builds with their variables since I work on many different projects at the same time or on and off when doing prompt engineering - sometimes creating hundreds or thousands of images per day.

I have a general or project based txt file to save the raw dynamic prompt builds, but sometimes you do small prompt changes on the fly like moving variables around, adding new ones, restructuring the prompt etc. and you run a small batch with that.

It's cumbersome and hard to keep track of all the different versions and variations.

Idea

In my mind it would make sense having the raw unprocessed Dynamic Prompt with all variables also saved somewhere - ideally as a companion file like the text information files from Automatic1111.

For example

A raw Dynamic Prompt companion file could use the [Images filename pattern] set in the Automatic1111 webui settings and it would create a file for every image with a suffix like "filename-pattern-dp.txt" that includes the full unprocessed dynamic prompt with variables.

I'm aware that this would mean that all companion files using the same dynamic prompt would contain the same information, but at least you would have access to the image's full unprocessed prompt build and it's also attached to the images that used it.

Since I don't know if it's possible to add this to the already existing a1111 information files, I thought another companion file could be a workaround.

This might be an absolute edge case to many or maybe it doesn't make sense, but I always try to optimize my workflows and reduce DRY (don't repeat yourself).

This may very well be out of scope of Dynamic Prompts, but I wanted to ask if anyone has a better workflow how to store and manage raw dynamic prompts variations with their images and projects.

Thank you!

[adieyal]

To be clear, do you want to store the template (containing DPs constructs like wildcards and combinations,) or the final resolved prompt? In the latter case, Auto1111 already stores it in the png metadata

[masslevel]

Sorry, for not being clear enough. The parsed prompt is there - of course. I mean the raw DP construct containing all the variables, wildcards and combinations - without it being parsed / populated with random tokens.

So instead of red tulips in the summer, the unprocessed DP __colors__ __flowers__ in the __seasons__ would be saved to a new companion file (or a different solution).

For example this is one of my Dynamic Prompts character builds - and I sometimes do small changes, add or reorder stuff during a prompt session.

__camera-perspectives__ of a {|__ml/characters-emotions__} __misc/ethnicities__ {{|__ml/adj-beauty__} __ml/characters-identity__|{|__ml/adj-beauty__} __ml/characters-identity-type__|{|__ml/adj-beauty__} __ml/characters-type__ __ml/characters-identity__} __ml/jobs__ __ml/action__ __ml/countries__, __ml/embedding-list__

So it would be really helpful when coming back to a project or experiment having the whole unprocessed DP available and connected to the image that used it. I don't know if the companion file suggestion is the best solution for it, but it would make sense to me in the current workflow.

Of course I don't know if that is even possible or feasible. I guess adding it to the existing infofile created by a1111 would open a can of compatibility issues (2 prompt builds in one file).

I hope this makes more sense. Let me know.

[adieyal]

It does make sense and it's a good idea - I plan to look into it once the other issues related to the recent release settle down.

[masslevel]

Wonderful! Let me know if I can do anything to help - beside writing the code for it, which I would do if I knew how ;-).

If a companion file solution would make sense, some of the important parts could be:

• saving the unprocessed positive and negative DP
• using a1111's directories_filename_pattern + samples_filename_pattern + maybe a configurable suffix for a DP companion file
  • this way it would be saved and named like the rest of the files (image outputs, infofile) and placed right beside them.
• Attention grabber and Magic prompt functionality would need to be considered - e.g. saving (__color__:1.2) so the random weight is saved with the construct. Not sure yet if it makes sense.

Thank you so much for your work on Dynamic Prompts!

[adieyal]

@masslevel I was hoping you could test a new version for me. It's another big backend change and I don't want to release it without a few users testing it. You shouldn't noticed any changes on the frontend but the update should make it easier to add and test features in future.

to test:

1. Navigate to extensions/sd-dynamic-prompts/
2. git pull -a
3. git checkout v2.0.0

To get back to the current version
git checkout main

Let me know if it breaks anything or if something looks off.

Much appreciated.

[masslevel]

I did do the checkout. it's said HEAD is now at be23398 bumped version
If I do a git pull in the extension directory it says You are not currently on a branch. So I guess I'm not on the new version.

If I start web-ui now, Dynamic Prompts is not shown in the interface.

[adieyal]

is there are error message in the terminal?

[masslevel]

sorry, for being a bit of a noob here.

• I removed the current extension directory and did a clean pull
• git clone https://github.com/adieyal/sd-dynamic-prompting/ extensions/dynamic-prompts
• git pull -a in the extensions/dynamic-prompts directory resulted in Already up to date.
• git checkout v2.0.0 resulted in

Note: switching to 'v2.0.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at be23398 bumped version

[adieyal]

You've done everything correctly. You say that the extension doesn't appear when you start the webui. That usually means that something broke when it was loading up. Hopefully there is an error in the terminal when it does. You should at least see some messages about it installing dependencies.

[masslevel]

ok - I fixed it. I think something blocked (my firewall) the dependencies update on startup. Sorry, for the trouble!

It's working and shown in the ui - testing now!
(deleted my previous "bug" report comments since they are no longer relevant)

[adieyal]

That's great news - let me know if you encounter any issues with v2, I would like to release it by the weekend if there aren't any serious bugs. Don't update using the extensions tab yet as you will probably be downgraded to v1.5.

[masslevel]

All good so far :). Except a whitespace being added sometimes after a wildcard. But I think that is already reported on issues.

[adieyal]

should be resolved in 2.1.0

[adieyal]

Regarding your initial request for saving the raw prompt template. I've just push a version (v2.2.0) which gives you the option to write the prompt directly into the image metadata, viewable using the PNG Info tab. It's disabled by default but you can find the checkbox under advanced settings.

Let me know if that works for you.

[masslevel]

Wow. That's really great! I will try it out shortly.

If that works, would it be possible in the future to also have a companion file option (or both at the same time)? Of course I say this naively without knowing how much work that would take and if it makes sense to you. Just asking.

I'm very happy with a PNG metadata option. The companion file option would only make it easier for my own workflow, since I work a lot by manipulating the file system. For example I regex convert all my a1111 infofiles to xmp so I can work with them in any image management software (lightroom, darktable or xnviewmp in my case) and search for prompts etc.

A companion file would make it easier to index, process and copy the information.

Thank you so much again!

[adieyal]

Have you had a look at the write prompts to file checkbox? I think it does what you're asking for.

[masslevel]

Sorry, @adieyal for getting back this late. I had to finish up some work.

I first tried to switch to 2.2.0 and 2.3.0 but noticed that you already updated main. Very cool!

It's behaving a bit strange. I first checked the advanced section in the dynamic prompt accordion but than noticed that the settings are on the a1111 Settings > Dynamic Prompts page.

I'm still trying to find the best way how to reproduce it. I'm also adding what I think would make sense from my perspective.

Enabling Write prompts to file

If I enable Write prompts to file it will create a csv file directly in the outdir_txt2img_samples instead where the image is created.

The location and filename of the DP should be constructed like this - or equal to:
outdir_txt2img_samples + directories_filename_pattern + samples_filename_pattern

a1111 also has defaults that are used if those fields are empty. I don't know how you can hook into this.

Somebody already opened an issue for this: #178

Enabling Save template to metadata

If I enable Save template to metadata the metadata is added to the .png but it's not the construct. It duplicates the metadata with your Template and Negative Template sections but without variables. When this is enabled it also duplicates the data inside the original a1111 infofile as well.

This leads to that the a1111 infofile and the metadata from the .png can no longer be used to retrieve the generation parameters in the webui because the syntax is no longer correct. An issue already exists: #177

Suggestions

existing a1111 infofile

I would suggest not writing into the a1111 infofile since the workflow of pulling in the metadata and syntax breaks because of it. Also if you want to add the metadata to the png (which is a great bonus beside having a txt file), you may need to create your own png chunk block (if that is possible), because the a1111 syntax also breaks here because of it.

filename

The location and filename of the DP infofile should be constructed like this:
outdir_txt2img_samples + directories_filename_pattern + samples_filename_pattern

Suggestion A

Duplicating a1111 infofile but replacing the prompts with Dynamic Prompts constructs

This would be my favored solution. Maybe it's too much work to integrate, but I think a very useable solution would be by duplicating the a1111 infofile that is created with every image including all settings and parameters. The Dynamic Prompts version would just replace the parsed positive and negative prompts with the Dynamic Prompts constructs.

This way you could decide later if you want to pull in the original a1111 infofile or the Dynamic Prompts construct version of the prompt build / image.

0000-my-image.png = image
0000-my-image.txt = a1111 infofile
0000-my-image-dp.txt = Dynamic Prompts infofile

I would think suggestion A would work quite well in the current workflow. But of course I don't know how easy it will be to rebuild, duplicate or alter the a1111 infofile yourself or integrate that process.

Suggestion B

CSV file as a companion file

An alternative and maybe a more easier to implement version could be having the Dynamic Prompts construct in a companion file (same naming scheme as the image in the same directory)
So for example:
0000-my-image.png = image
0000-my-image.txt = a1111 infofile
0000-my-image.csv = Dynamic Prompts file

Please let me know if it doesn't make sense and I'll try to give better examples.
Thank you!

[masslevel]

@adieyal I don't know if you have seen my message above. If you want let me know if I should add any additional information or if you think the approach doesn't make sense.

[adieyal]

I haven't had a chance yet. I'll give it a read this weekend.

…

On Thu, 19 Jan 2023, 06:38 masslevel, ***@***.***> wrote: @adieyal <https://github.com/adieyal> I don't know if you have seen my message above. If you want let me know if I should add any additional information or if you think the approach doesn't make sense. — Reply to this email directly, view it on GitHub <#149 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADGXMRR2677A5LMZRWHXY3WTDAM3ANCNFSM6AAAAAATPI5NXI> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[adieyal]

@masslevel

Have you tried the branch mentioned in #178?

git pull -a
git checkout chore/prompts-filename

what I do there is take the name and path of the first image generated, with a csv extension. Does that address the "Enabling Write prompts to file" part of your message?

[adieyal]

I pushed a fix for "Enabling Save template to metadata" - see #177

[adieyal]

With regards to the .txt file - how do you get auto1111 to read parameters from it? I see the checkbox in settings to save it but I'm not sure how it gets read later.

I've fixed the ability to copy and paste the contents of that file into the prompt text box in case that's how you use it

[masslevel]

Thank you so much, @adieyal ! Please let me know if I can "buy you a coffee" anywhere ;-).
I will test it out tonight and report back.

Regarding the a1111 metadata

• You can either drag'n'drop the .png image that includes the metadata or the .txt file right into the prompt field in the web-ui.
• You than press the blue arrow icon and it will parse the parameters into the corresponding fields.

It uses a very basic text parser. People have suggested JSON as a better alternative, but it was decided against it - probably to not break all the older stuff. JSON would still have been a better format for further processing etc. in my opinion.

[adieyal]

:-) I appreciate the offer - I enjoy maintaining this project and especially when people find it valuable. No need for coffee - testing the tool and giving feedback and suggestions is the best way for non-coders to contribute.

[adieyal]

Regarding the a1111 metadata

• You can either drag'n'drop the .png image that includes the metadata or the .txt file right into the prompt field in the web-ui.
• You than press the blue arrow icon and it will parse the parameters into the corresponding fields.

It uses a very basic text parser. People have suggested JSON as a better alternative, but it was decided against it - probably to not break all the older stuff. JSON would still have been a better format for further processing etc. in my opinion.

This is now working in the bug/copypaste-parameters branch. It previously wasn't working with negative prompts, but that's fixed.

Re the format, JSON would definitely be a better bet although prompts would need to be properly encoded since json doesn't accept newlines in strings. In any case, you might be interested to know DP handles this.

The current metadata format looks something like:

A test positive prompt
Negative prompt: A test negative prompt
Steps: 20, Sampler: Euler a, CFG scale: 9, Seed: 1418630673, Size: 512x704, Model hash: cc6cb27103
Template:A test positive prompt
Negative Template:A test negative prompt

The auto1111 parser is fiddly. The parser gobbles up everything from the start until it finds a Negative prompt: and assigns it to Prompt. The negative prompt is then everything except the last line, which is reserved for parameters. The DP Template and Negative Template breaks this assumption.

I have added a hook that gets called whenever the metadata is copypasted (either using the blue arrow, or Send to txt2img). When the hook is called, the template and negative template bits are removed. The remaining metadata is then passed back to auto1111 for parsing as if they were never there. This should somewhat future proof if auto1111 adds additional parameters.

[masslevel]

First of all I'm sorry that we got 2 different conversations going on this thread. If you want to consolidate those, let me know. Otherwise I just keep replying to the corresponding post.

The hook is a great idea and you would only have one infofile.

But the problem I see with that is that I share a lot of my prompts and settings with people I'm working with, with the community and on social media. Wouldn't that mean if someone doesn't have Dynamic Prompts installed (which of course everyone should ;-] ) that the infofile would not work because the hook would not be present?

So that would break compatibility and I would rather have the original infofiles untouched.

Please let me know if I misunderstood.

If I would share images and infofiles that has the DP part in it, I would have to edit them all manually and remove the DP part before doing that. Otherwise the a1111 parser without Dynamic Prompts as an extension present wouldn't correctly work, right?

That's why I thought it would make sense to have a separate infofile with the Dynamic Prompts construct so you can choose which file to import into the web-ui or share.

Also the workflow would be faster, since you could drag'n'drop the original a1111 infofile, click the blue arrow and have the specific image settings OR drag in the Dynamic Prompts version of the infofile, click the blue arrow and are back in your Dynamic Prompts project.

Right now I would have to manual edit the prompt to get the DP construct in place as well.

I can understand that having an additional file might not be preferable, but the workflow would be really optimized if it work like described above.

These are just my thoughts and I'm just looking for the most effective and fastest workflow that I think would make sense.

[masslevel]

@masslevel

Have you tried the branch mentioned in #178?

git pull -a
git checkout chore/prompts-filename

So this works already really great, @adieyal ! I haven't checked the png chunk metadata fix, but the csv file is now placed and named correctly right beside the other files using the prompts-filename branch.

Now you can easily copy'n'paste the Dynamic Prompts construct that was used for the image. That is already a huge workflow optimization for me and I can stop using secondary workflows.

Wonderful! (see update below: #149 (comment))

As an optional idea

The ultimate workflow optimizer for this, and you would be absolutely right that this could really well be out of the scope of Dynamic Prompts, would be an additional .txt file that is an exact copy of the a111 info file but replaces the prompt and Negative prompt: bits of the file.

Maybe you have already thought about it and concluded that it doesn't make sense or is way too convoluted to implement.

With this solution you could use the standard text parser (blue icon in the web-ui) but it would include the Dynamic Prompt construct instead of the populated prompt.

The advantage would be that all the other settings would also be ready to go when using this file. Also since new parameters are added to the infofile every so often by a1111, if you only replace the beginning of the file (the positive prompt) and the Negative Prompt: part, if a1111 doesn't totally refactor the file format, it should stay compatible for some time.

I would definitely keep the csv option as well, since it's much better to process this further than a non-standard data structured text file. And you have both prompts available in one file (unprocessed and processed).

Example

0000-my-image.txt (default a1111 info file)

a cat in the forest
Negative prompt: ugly cartoon drawing
Steps: 15, Sampler: DPM++ SDE Karras, CFG scale: 8.5, Seed: 12345678, Size: 768x768, Model hash: ad2a33c361, Model: sd2_sai_v2-1_768-ema-pruned

0000-my-image-dp.txt (Dynamic Prompts info file)

a __animal__ in the __location__
Negative prompt: ugly cartoon drawing
Steps: 15, Sampler: DPM++ SDE Karras, CFG scale: 8.5, Seed: 12345678, Size: 768x768, Model hash: ad2a33c361, Model: sd2_sai_v2-1_768-ema-pruned

I hope this makes sense.

[masslevel]

Ok, I did run into some strange behavior with the chore/prompts-filename branch. It stopped working and I'm not sure how. Also it did only generate a couple of csv. Not for all images.

I first thought maybe it does it only for one batch. Which could make sense, but I would prefer it to have with every image since they might get sorted and copied to different locations. So one csv per image would be ideal.

I'm trying to reproduce the issue now. It's not generating any .csv's currently and I have not updated or switched the branch.

[adieyal]

can you try chore/prompts-filename again? It was an older branch which was out of sync with main. I've now brought it up to date in case that was the issue. CSVs are being created on my setup.

So one csv per image would be ideal.
I don't understand - would the csv then only have a single line, the one relating to that image?

[masslevel]

To make sure that I have the correct version, I made a clean clone and switched to the chore/prompts-filename branch.

When I launch the web-ui and create an image, it creates one .csv file and than it stops creating them for every new image. I have to restart the web-ui, than it creates one .csv and than stops again creating them. I tried it with single images and batches.