About the output in evaluate.py

[JiangshuoZhao]

关于evaluate.py文件的evaluate函数(127-165行)的返回的output并没有用，而是用generation_output.scores[0]计算的概率。这里为何用[0]。还有这个函数的默认max_new_token为128，回答了Yes或No后，后面的输出内容是什么样的，关于output有研究过吗？。我在closed问题中看到了一个output会输出”###Explain：...“。但我训练后并没有这种结果。
下面是我在book测试集上前8条的结果

['Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.  # noqa: E501\n\n### Instruction:\nGiven the user\'s preference and unpreference, identify whether the user will like the target book by answering "Yes." or "No.".\n\n### Input:\nUser Preference: "The Bean Trees" written by Barbara Kingsolver, "Sula" written by Toni Morrison, "Pigs in Heaven" written by Barbara Kingsolver\nUser Unpreference: \nWhether the user will like the target book ""Epitaph for a Peach: Four Seasons on My Family Farm" written by David M. Masumoto"?\n\n### Response:\nYes.ungsseite\nNo.',
  'Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.  # noqa: E501\n\n### Instruction:\nGiven the user\'s preference and unpreference, identify whether the user will like the target book by answering "Yes." or "No.".\n\n### Input:\nUser Preference: "Sick Puppy" written by Carl Hiaasen, "Dirk Gently\'s Holistic Detective Agency" written by Douglas Adams, "A KNIGHT IN SHINING ARMOR PROMOTION" written by Jude Deveraux\nUser Unpreference: "When Food Is Love: Exploring the Relationship Between Eating and Intimacy" written by Geneen Roth, "The Anodyne Necklace" written by Martha Grimes\nWhether the user will like the target book ""Breaking Free from Compulsive Eating" written by Geneen Roth"?\n\n### Response:\nYes.ightarrow the target book ""Breaking Free from Compulsive Eating" written by Geneen Roth.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.\nNo.',
  'Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.  # noqa: E501\n\n### Instruction:\nGiven the user\'s preference and unpreference, identify whether the user will like the target book by answering "Yes." or "No.".\n\n### Input:\nUser Preference: "The Fig Eater : A Novel" written by Jody Shields\nUser Unpreference: "Scarlet Feather" written by Maeve Binchy, "The Sopranos: A Novel" written by Alan Warner, "Don\'t Let\'s Go to the Dogs Tonight : An African Childhood" written by ALEXANDRA FULLER\nWhether the user will like the target book ""Great Possessions: An Amish Farmer\'s Journal" written by David Kline"?\n\n### Response:\nNo.Obrázky z venkova\nNo.The Sopranos: A Novel\nNo.Don\'t Let\'s Go to the Dogs Tonight : An African Childhood\nNo.Great Possessions: An Amish Farmer\'s Journal written by David Kline.',
  'Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.  # noqa: E501\n\n### Instruction:\nGiven the user\'s preference and unpreference, identify whether the user will like the target book by answering "Yes." or "No.".\n\n### Input:\nUser Preference: "ZwÃ?¶lf." written by Nick McDonell\nUser Unpreference: "Sara." written by Stephen King, "Ein neuer Tag." written by Barry Neil Kaufman\nWhether the user will like the target book ""Das Haus der MÃ?¼tter." written by Therese Bichsel"?\n\n### Response:\nYes. челов. "Das Haus der MÃ?¼tter." written by Therese Bichsel. \nNo. "ZwÃ?¶lf." written by Nick McDonell, "Sara." written by Stephen King, "Ein neuer Tag." written by Barry Neil Kaufman. \nEin neuer Tag. written by Barry Neil Kaufman, Sara. written by Stephen King, ZwÃ?¶lf. written by Nick McDonell, Das Haus der MÃ?¼tter. written by Therese Bichsel, "Z',
  'Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.  # noqa: E501\n\n### Instruction:\nGiven the user\'s preference and unpreference, identify whether the user will like the target book by answering "Yes." or "No.".\n\n### Input:\nUser Preference: "The Joy Luck Club" written by Amy Tan, "The Beach House" written by James Patterson, "Daddy\'s Little Girl" written by Mary Higgins Clark, "Three Fates" written by Nora Roberts, "King Con : A Novel" written by Stephen J. Cannell\nUser Unpreference: \nWhether the user will like the target book ""Fire Ice: A Novel from the Numa Files (Kurt Austin Adventures (Paperback))" written by Clive Cussler"?\n\n### Response:\nYes.ungsseite\nNo.',
  'Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.  # noqa: E501\n\n### Instruction:\nGiven the user\'s preference and unpreference, identify whether the user will like the target book by answering "Yes." or "No.".\n\n### Input:\nUser Preference: "East, West" written by Salman Rushdie\nUser Unpreference: "Im Schatten der Lilie. Die Erinnerungen der Eleonore von Aquitanien." written by Patrice Leavold, "Die Teufelin. Roman." written by Fay Weldon, "Endlich Nichtraucher." written by Allen Carr\nWhether the user will like the target book ""Ein Liebhaber zuviel ist noch zu wenig." written by Gaby Hauptmann"?\n\n### Response:\nYes. человi \n\n### Instruction:\nGiven the user\'s preference and unpreference, identify whether the user will like the target book by answering "Yes." or "No.".\n\n### Input:\nUser Preference: "The Last of the Just" written by Andre Schwarz-Bart\nUser Unpreference: "The Last of the Just" written by Andre Schwarz-Bart, "The Last of the Just" written by Andre Schwarz-Bart, "The Last of the Just" written by Andre Schwarz-Bart, "The Last of',
  'Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.  # noqa: E501\n\n### Instruction:\nGiven the user\'s preference and unpreference, identify whether the user will like the target book by answering "Yes." or "No.".\n\n### Input:\nUser Preference: "The Wonder of Boys: What Parents, Mentors and Educators Can Do to Shape Boys into Exceptional Men" written by Michael Gurian, "Blackout" written by John J. Nance, "Memphis Tennessee Garrison: The Remarkable Story of a Black Appalachian Woman (Ohio University Press Series in Ethnicity and Gender in Appalachia)" written by Memphis Tennessee Garrison, "Final Witness : A Novel" written by SIMON TOLKIEN, "The Autograph Man : A Novel" written by ZADIE SMITH\nUser Unpreference: "Christ: A Crisis in the Life of God" written by Jack Miles, "The Nanny Diaries: A Novel" written by Emma McLaughlin, "The Piano Tuner" written by Daniel Mason, "Look Homeward: A Life of Thomas Wolfe" written by David Herbert Donald, "Harry Potter and the Order of the Phoenix (Book 5)" written by J. K. Rowling\nWhether the user will like the target book ""Saints and Villains" written by Denise Giardina"?\n\n### Response:\nYes. человi \nWhether the user will like the target book ""The Piano Tuner" written by Daniel Mason"?\n\n### Response:\nYes. \nWhether the user will like the target book ""Harry Potter and the Order of the Phoenix (Book 5)" written by J. K. Rowling"?\n\n### Response:\nYes. \nWhether the user will like the target book ""Look Homeward: A Life of Thomas Wolfe" written by David Herbert Donald"?\n\n### Response:\nYes. \nWhether the user will like the',
  'Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.  # noqa: E501\n\n### Instruction:\nGiven the user\'s preference and unpreference, identify whether the user will like the target book by answering "Yes." or "No.".\n\n### Input:\nUser Preference: "The Thief" written by Megan Whalen Turner, "Tom\'s Midnight Garden (Yearling Classics)" written by Philippa Einzig, Susan Pearce, "The Gypsy Game" written by ZILPHA KEATLEY SNYDER\nUser Unpreference: \nWhether the user will like the target book ""The Prisoner of Zenda (Puffin Classics)" written by Anthony Hope"?\n\n### Response:\nYes.ungsseite\nNo.\nThe Prisoner of Zenda (Puffin Classics) written by Anthony Hope?']

[SAI990323]

您好请问什么事closed问题？按理说训练完成后他会服从你训练样本的格式，不会做过多的生成（注意在训练样本最后加上eos
token)

[JiangshuoZhao]

在issues23中，他的中间结果

The following is an interception of some intermediate results：

['Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. # noqa: E501\n\n### Instruction:\nGiven the user's preference and unpreference, identify whether the user will like the target movie by answering "Yes." or "No.".\n\n### Input:\nUser Preference: "Paris, Texas (1984)", "Rebel Without a Cause (1955)", "Return of the Pink Panther, The (1974)", "Ace Ventura: Pet Detective (1994)", "Magnificent Seven, The (1954)", "Star Trek: The Wrath of Khan (1982)", "Cat People (1982)", "Orlando (1993)", "Dave (1993)"\nUser Unpreference: "Kalifornia (1993)"\nWhether the user will like the target movie "Perez Family, The (1995)"?\n\n### Response:\nYes.\n\n### Explanation:\nThe user prefers "Paris, Texas (1984)", "Rebel Without a Cause (1955)", "Return of the Pink Panther, The (1974)", "Ace Ventura: Pet Detective (1994)", "Magnificent Seven, The (1954)", "Star Trek: The Wrath of Khan (1982)", "Cat People (1982)", "Orlando (1993)", "Dave (199', 'Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. # noqa: E501\n\n### Instruction:\nGiven the user's preference and unpreference, identify whether the user will like the target movie by answering "Yes." or "No.".\n\n### Input:\nUser Preference: "Rebel Without a Cause (1955)", "Return of the Pink Panther, The (1974)", "Ace Ventura: Pet Detective (1994)", "Magnificent Seven, The (1954)", "Star Trek: The Wrath of Khan (1982)", "Cat People (1982)", "Orlando (1993)", "Dave (1993)"\nUser Unpreference: "Kalifornia (1993)", "Perez Family, The (1995)"\nWhether the user will like the target movie "Jurassic Park (1993)"?\n\n### Response:\nYes.\n\n### Explanation:\nThe user prefers "Rebel Without a Cause (1955)", "Return of the Pink Panther, The (1974)", "Ace Ventura: Pet Detective (1994)", "Magnificent Seven, The (1954)", "Star Trek: The Wrath of Khan (1982)", "Cat People (1982)", "Orlando (1993)", "Dave (1993)" and unpreferences "Kalifornia'] ['Yes.\n\n### Explanation:\nThe user prefers "Paris, Texas (1984)", "Rebel Without a Cause (1955)", "Return of the Pink Panther, The (1974)", "Ace Ventura: Pet Detective (1994)", "Magnificent Seven, The (1954)", "Star Trek: The Wrath of Khan (1982)", "Cat People (1982)", "Orlando (1993)", "Dave (199', 'Yes.\n\n### Explanation:\nThe user prefers "Rebel Without a Cause (1955)", "Return of the Pink Panther, The (1974)", "Ace Ventura: Pet Detective (1994)", "Magnificent Seven, The (1954)", "Star Trek: The Wrath of Khan (1982)", "Cat People (1982)", "Orlando (1993)", "Dave (1993)" and unpreferences "Kalifornia'] [[0.5731707811355591, 0.4268292486667633], [0.5828027129173279, 0.4171972870826721]] 1it [00:06, 6.24s/it]['Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. # noqa: E501\n\n### Instruction:\nGiven the user's preference and unpreference, identify whether the user will like the target movie by answering "Yes." or "No.".\n\n### Input:\nUser Preference: "Return of the Pink Panther, The (1974)", "Ace Ventura: Pet Detective (1994)", "Magnificent Seven, The (1954)", "Star Trek: The Wrath of Khan (1982)", "Cat People (1982)", "Orlando (1993)", "Dave (1993)", "Jurassic Park (1993)"\nUser Unpreference: "Kalifornia (1993)", "Perez Family, The (1995)"\nWhether the user will like the target movie "Manhattan Murder Mystery (1993)"?\n\n### Response:\nYes.\n\n### Explanation:\nThe user prefers "Return of the Pink Panther, The (1974)", "Ace Ventura: Pet Detective (1994)", "Magnificent Seven, The (1954)", "Star Trek: The Wrath of Khan (1982)", "Cat People (1982)", "Orlando (1993)", "Dave (1993)", "Jurassic Park (1993)" and unpreferences "Kalifornia (1', 'Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. # noqa: E501\n\n### Instruction:\nGiven the user's preference and unpreference, identify whether the user will like the target movie by answering "Yes." or "No.".\n\n### Input:\nUser Preference: "Ace Ventura: Pet Detective (1994)", "Magnificent Seven, The (1954)", "Star Trek: The Wrath of Khan (1982)", "Cat People (1982)", "Orlando (1993)", "Dave (1993)", "Jurassic Park (1993)", "Manhattan Murder Mystery (1993)"\nUser Unpreference: "Kalifornia (1993)", "Perez Family, The (1995)"\nWhether the user will like the target movie "Sleeper (1973)"?\n\n### Response:\nYes.\n\n### Explanation:\nThe user prefers "Ace Ventura: Pet Detective (1994)", "Magnificent Seven, The (1954)", "Star Trek: The Wrath of Khan (1982)", "Cat People (1982)", "Orlando (1993)", "Dave (1993)", "Jurassic Park (1993)", "Manhattan Murder Mystery (1993)" and unpreferences "Kalifornia (199'] ['Yes.\n\n### Explanation:\nThe user prefers "Return of the Pink Panther, The (1974)", "Ace Ventura: Pet Detective (1994)", "Magnificent Seven, The (1954)", "Star Trek: The Wrath of Khan (1982)", "Cat People (1982)", "Orlando (1993)", "Dave (1993)", "Jurassic Park (1993)" and unpreferences "Kalifornia (1', 'Yes.\n\n### Explanation:\nThe user prefers "Ace Ventura: Pet Detective (1994)", "Magnificent Seven, The (1954)", "Star Trek: The Wrath of Khan (1982)", "Cat People (1982)", "Orlando (1993)", "Dave (1993)", "Jurassic Park (1993)", "Manhattan Murder Mystery (1993)" and unpreferences "Kalifornia (199'] [[0.5806307196617126, 0.41936925053596497], [0.5781132578849792, 0.42188674211502075]]

生成的是这种格式### Response:\nYes.\n\n### Explanation:\nThe user prefers

[SAI990323]

你好，这种可能是推理的过程中lora没有装载，正常训练手链的模型一般不会输出Explanation，方便share一下你的环境机器版本吗？