Figure - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Download
View publication
The table presents the model evaluation scores for different context lengths. Model Name shows the name of the model. The columns 4k, 8k, 16k, 32k, 64k, 128k present evaluation scores averaged over all tasks. The Overall score is obtained by averaging the results over all lengths. The best score is put in bold, the second best is underlined.

The table presents the model evaluation scores for different context lengths. Model Name shows the name of the model. The columns 4k, 8k, 16k, 32k, 64k, 128k present evaluation scores averaged over all tasks. The Overall score is obtained by averaging the results over all lengths. The best score is put in bold, the second best is underlined.

Source publication
Figure 1: The LIBRA benchmark is a set of 21 longcontext tasks grouped...
The table presents the average model's fertility.
The table presents the model evaluation scores for different context...
The table presents the evaluation results. Model Name shows the name of...
The table presents the evaluation results of LLaMA-2-32K. Dataset Name...
Long Input Benchmark for Russian Analysis
Preprint
Full-text available
  • Aug 2024
Recent advancements in Natural Language Processing (NLP) have fostered the development of Large Language Models (LLMs) that can solve an immense variety of tasks. One of the key aspects of their application is their ability to work with long text documents and to process long sequences of tokens. This has created a demand for proper evaluation of l...
Cite
Download full-text

Context in source publication

Context 1
... baseline results with respect to context length are shown in Table 4 and with respect to tasks are shown in Tables 5, 6, 7. Detailed results for each model are given in Appendix D. Based on the obtained results we can draw the following conclusions for each group of tasks. ...

Similar publications

Fig. 4. HuggingFace Open LLM Leaderboard -top models by search filter...
Overcoming linguistic barriers in code assistants: creating a QLoRA adapter to improve support for Russian-language code writing instructions
Preprint
Full-text available
  • Sep 2024
In this paper, an approach to training and evaluating an adapter model for the popular language model "zephyr-7b-beta" is described. The adapter was developed to improve the performance of the base model in tasks related to programming and understanding the Russian language. Considering the high quality of the original model in tasks in the English...
View