Introduction: The ‘journal quality’ has been assessed using both objective (based on citations) and subjective (based on expert opinion) approches over the years. In recent years, bibliometric scores, such as Impact Factor, have become increasingly popular as a method of assessing ‘journal quality’. However, they do not provide a reliable measurement for this concept. They are quantitative in nature, and are based on mere citation frequency which present part of the picture (i.e., research quality/scientific impact). ‘Journal quality’ is a multidimensional concept and cannot be accurately defined and assessed by a single metric and/or few factors. Thus, new criteria and methods must be developed to incorporate all dimensions of a journal. The purpose of this study was to investigate the concept of “journal quality” in a comprehensive way and introduce its various components within a health and biomedical contexts.
Methods: The research was conducted in three qualitative and quantitative stages. As a first step, scoping review methods (including, online database search, manual search, grey literature search) were used to identify, categorize, and develop the initial conceptual framework. To conduct the scoping review, Arksey and O'Malley’s framework was used, and Brown and Clerk’s thematic analysis method was implemented to analyze the data. A group of 16 experts from seven countries and continents evaluated and edited the extracted criteria and the initial framework in the second stage of the research. In the third step, we validated and prioritized the extracted criteria and compiled the final model based on both objective (experts’ opinions) and subjective (leading health and biomedical journals’ data) approaches. Each criterion was weighed and prioritized based on the total weight of the expert group and journals’ data. Components with less than 70% agreement were removed from the final model.
Results: A total of 116 documents met the eligibility criteria and 216 unique ‘quality criteria’ were extracted during the first stage of the research (scoping review). ‘Quality criteria’ were categorized into five themes, including ethical, content structure & technical, scientific rigor, editorial structure, and promoting & indexing. At the second stage of the research, the expert groups did not change the five main themes, and no components were removed. However, by rearranging and merging criteria, and adding new ones, 216 components changed to 203, including 16 sub-themes, 83 main-statements, and 120 sub-statements. In the final stage (validation and weighting), five criteria were excluded from the analysis because they could not be assessed and were specific to a certain group of journals (e.g., open access). The 198 criteria were then validated and weighted. Finally, forty-seven criteria were removed because they did not meet the required score (70% agreement), and the final model was developed based on 151 ‘quality criteria’.
Conclusion: This is the first study focusing on the concept of ‘journal quality’ in health and biomedical contexts. This study presents a conceptual model and quality criteria in five themes, including ethical, content structure & technical, scientific rigor, editorial structure, and promoting & indexing, to facilitate the perception of this concept. The results of this study, besides Iranian journals, can also be used as a comprehensive guide by all members of the publishing community worldwide, especially young editors and new publishers, to develop policies and improve the quality of their journals. The results of this study can also be used to develop policies for improving the quality of publications on a national and international level, as well as a first step towards developing a quality assessment and rating system for journals.