The vicuna-13B-v1.3 model uses the batch_ouline scheduler to dump the data output garble problem

[qtyandhasee]

Hello, I recently used sot code to study the evaluation results of vicuna-13B-v1.3 model. I found that when scheduler was selected as naive, my output was normal, but if I selected batch_outline as scheduler, the output was always messy. This is not consistent with the vicuna-13B-v1.3 evaluation results reported in the paper. After debugging, I found that in the case of the 13b model, when using batch input, the output is normal when batchsize is 1, but if batchsize is greater than 1, the output is garbled. Interestingly, these issues were not encountered when I used the vicuna-7b model, and the vicuna-7b evaluation results were consistent with the paper results.
the command i use
CUDA_VISIBLE_DEVICES=1 python sot/main.py --model fastchat --scheduler batch_outline --data-path /root/nfs/sot_jupiter/sot/data/ourdata/vicuna_select2/vicuna_generic_counterfactual_coding_math.csv --output-folder /root/nfs/sot_jupiter/sot/results/ourdata/vicuna_select2/sot_vicuna_select2_vicuna13b_bit4_counterfactual_generic_coding_math_noquantize_new_sot_batch_outline --model-path /root/nfs/sot_jupiter/sot/sot/models/download_models/vicuna-13b-v1.5 --num-gpus 1 --prompt-file prompts/sot_opensource.json