Custom prompts input bug.

[RuiqingTang]

Hi,
I tried to use custom prompts to generate motion, but it did not work.
The command is as follows:
python -m sample.conditional_synthesis --model_path ./save/condmdi_random_frames/model000750000.pt --edit_mode benchmark_sparse --transition_length 5 --num_samples 10 --num_repetitions 3 --text_prompt "a person does a backflip"
And I got the results:

As you can see, the prompt seems to work (it works for the directory name). The video results prove that it did not work.

The motion is still "a person throws a ball" although it is labeled as “a person does a backflip”

[icedwater]

Same bug here. Did you manage to find a solution? I've changed --seed but it doesn't help, and I noticed that --num_samples will be forced to 1 if you supply a text prompt.

[RuiqingTang]

@icedwater
I haven’t found a solution yet, so I am going to try other text2motion algorithms now.

[icedwater]

Thanks! I found with some help that the test set prompt is somehow being used in conditional_synthesis.py but forcing it to take the input text doesn't change the motion being sampled.

Need to further examine model and model.kwargs...

[icedwater]

@RuiqingTang just to add on, I was able to generate a slightly more useful sequence via the unconditioned generation mode:

python -m sample.conditional_synthesis --model_path ./save/randomframes/model000750000.pt --edit_mode uncond --num_samples 10 --num_repetitions 3 --text_prompt "a person does a backflip"

fwd_roll.mp4

It's clearly not a backflip, but it's also not throwing a ball, I guess.

[RuiqingTang]

@icedwater Thank you! I will try it later.