Help me understand the HELM Classic Leaderboard's missing results

[PaulJoeMaliakel]

Why were many models not evaluated on tasks like HellaSwag, OpenBookQA, MS MARCO, and summarization tasks like XSUM and CNN/Daily Mail? Is it because they are not suitable for these tasks?

[yifanmai]

Yes - many scenarios on HELM Classic require logprobs from the model, because they use an adapter that requires logprobs. Many recent model APIs do not provide logprobs.