ArticlePDF Available

Assessment of ESP students’ learning outcomes in a digital learning environment

Authors:

Abstract and Figures

Problem statement. The description of a pedagogic research in the field of formative assessment theory and practice is presented. The goal of the study is to define pedagogic and methodological conditions for effective evaluation of ESP students’ learning outcomes in a digital learning environment. Methodology. On the basis of key theoretical provisions of formative assessment and interdisciplinary approaches LSPCLIL, a complete set of criteria-based ESP evaluation tasks and assessment schemes were developed for Business English course delivered in the groups of BMSTU masters students majoring in economics. The approach to grading that correlates with BMSTU point-rating system was substantiated. The comparative analysis of modern platforms allowed to select the domestic service pruff.me, which was used as a ground for a digital assessment space (DAS), a part of BMSTU digital learning environment needed for implementing formative assessment of ESP learning outcomes. Results. The developed evaluation materials integrated in the DAS were tested by ESP students in a distant format. Conclusion. The research showed that creating DAS as an essential component of a university digital learning environment can contribute to effective assessment of ESP students’ competence-based learning outcomes provided that underlying methodology is taken into account and formative assessment technology is implemented in full volume.
Content may be subject to copyright.
RUDN Journal of Informatization in Education
ISSN 23128631 (Print); ISSN 2312864X (Online)
2022 Vol. 19 No. 4 300–311
http://journals.rudn.ru/informatizationeducation
Вестник РУДН. Серия: Информатизация образования
300
DIGITAL EDUCATIONAL ENVIRONMENT
ЦИФРОВАЯ ОБРАЗОВАТЕЛЬНАЯ СРЕДА
DIGITAL EDUCATIONAL ENVIRONMENT
DOI 10.22363/2312-8631-2022-19-4-300-311
UDC 373
Research article / Научная статья
Assessment of ESP students’ learning outcomes
in a digital learning environment1
Kira M. Inozemtseva
, Elizaveta V. Morozova , Ilya M. Kolesnikov
Bauman Moscow State Technical University, Moscow, Russian Federation
ikm@bmstu.ru
Abstract. Problem statement. The description of a pedagogic research in the field of
formative assessment theory and practice is presented. The goal of the study is to define peda-
gogic and methodological conditions for effective evaluation of ESP students’ learning out-
comes in a digital learning environment. Methodology. On the basis of key theoretical provi-
sions of formative assessment and interdisciplinary approaches LSP&CLIL, a complete set of
criteria-based ESP evaluation tasks and assessment schemes were developed for Business
English course delivered in the groups of BMSTU masters students majoring in economics.
The approach to grading that correlates with BMSTU point-rating system was substantiated.
The comparative analysis of modern platforms allowed to select the domestic service pruff.me,
which was used as a ground for a digital assessment space (DAS), a part of BMSTU digital
learning environment needed for implementing formative assessment of ESP learning outcomes.
Results. The developed evaluation materials integrated in the DAS were tested by ESP students
in a distant format. Conclusion. The research showed that creating DAS as an essential compo-
nent of a university digital learning environment can contribute to effective assessment of ESP
students’ competence-based learning outcomes provided that underlying methodology is taken
into account and formative assessment technology is implemented in full volume.
Keywords: formative assessment, English for specific purposes, digital learning envi-
ronment, digital assessment space
Authors’ contribution: Kira M. Inozemtseva – conceptualization, methodology, text
writing. Elizaveta V. Morozovaevaluation materials design, text editing. Ilya M. Kolesnikov
comparative analysis of platforms, materials placement.
Conflicts of interest. The authors declare no conflicts of interests.
Article history: received 17 June 2022; accepted 20 July 2022.
For citation: Inozemtseva KM, Morozova EV, Kolesnikov IM. Assessment of ESP
students’ learning outcomes in a digital learning environment. RUDN Journal of Informatiza-
tion in Education. 2022;19(4):300–311. http://doi.org/10.22363/2312-8631-2022-19-4-300-311
©
Inozemtseva K.M., Morozova E.V., Kolesnikov I.M., 2022
This work is licensed under a Creative Commons Attribution 4.0 International License
https://creativecommons.org/licenses/by-nc/4.0/legalcode
Иноземцева К.М., Морозова Е.В., Колесников И.М. Вестник РУДН. Серия: Информатизация образования. 2022. Т. 19. № 4. С. 300–311
ЦИФРОВАЯ ОБРАЗОВАТЕЛЬНАЯ СРЕДА 301
Оценивание результатов обучения иностранному языку
для профессиональных целей в цифровой среде
К.М. Иноземцева
, Е.В. Морозова , И.М. Колесников
Московский государственный технический университет имени Н.Э. Баумана,
Москва, Российская Федерация
ikm@bmstu.ru
Аннотация. Постановка проблемы. Представлено педагогическое исследование в
области теории и практики формирующего оценивания, целью которого является определе-
ние методико-педагогических условий эффективного оценивания результатов обучения ино-
странному языку для профессиональных целей в цифровой среде. Методология. На основе
анализа теоретических положений формирующего оценивания и междисциплинарных
подходов LSP и CLIL разработан полный комплект критериальных контрольных зада-
ний и оценочных схем для оценивания результатов обучения дисциплине «Деловой
иностранный язык (английский)» магистрантов экономических специальностей в МГТУ
имени Н.Э. Баумана. Приведено обоснование подхода к выставлению оценок, коррели-
рующее с принятой в МГТУ имени Н.Э. Баумана балльно-рейтинговой системой.
Сравнительный анализ современных платформ, используемых в целях оценивания,
позволил сделать выбор в пользу отечественной платформы pruff.me и создать на ее
основе пространство цифрового оценивания (ПЦО) как компонент цифровой образова-
тельной среды МГТУ имени Н.Э. Баумана, предназначенный для реализации форми-
рующего оценивания результатов обучения иностранному языку для профессиональ-
ных целей. Результаты. Разработанные контрольные материалы, интегрированные в ПЦО,
апробированы студентами в дистанционном формате. Заключение. Показано, что со-
здание пространства цифрового оценивания как важного компонента цифровой образо-
вательной среды университета может способствовать эффективному оцениванию ком-
петентностных результатов обучения иностранному языку для профессиональных це-
лей при условии учета рассматриваемых в исследовании методологических концепций,
а также при включении всех обязательных компонентов формирующего оценивания.
Ключевые слова: формирующее оценивание, иностранный язык, профессио-
нальные цели, цифровая образовательная среда, цифровое пространство оценивания
Вклад авторов: К.М. Иноземцева концептуализация, методология, написание
текста. Е.В. Морозова разработка оценочных материалов, редактирование текста.
И.М. Колесников – сравнительный анализ платформ, размещение оценочных материалов.
Заявление о конфликте интересов. Авторы заявляют об отсутствии конфликта
интересов.
История статьи: поступила в редакцию 17 июня 2022 г.; принята к публикации
20 июля 2022 г.
Для цитирования: Inozemtseva K.M., Morozova E.V., Kolesnikov I.M. Assessment of
ESP students’ learning outcomes in a digital learning environment // Вестник Российского
университета дружбы народов. Серия: Информатизация образования. 2022. Т. 19. № 4.
С. 300–311. http://doi.org/10.22363/2312-8631-2022-19-4-300-311
Problem statement. The pandemic period associated with the transfer
of educational processes to distant format aroused great interest of educationalists
in the theory and practice of ESP teaching in a digital environment. However,
Inozemtseva K.M., Morozova E.V., Kolesnikov I.M. RUDN Journal of Informatization in Education. 2022;19(4):300–311
302 DIGITAL EDUCATIONAL ENVIRONMENT
the problem of ‘digital’ assessment of ESP competence-based learning outcomes
has not received sufficient scientific coverage. In the context of a new educational
paradigm, which implies ‘digital transformation’ of universities, the issue of ‘digi-
tal’ assessment needs to be studied in details.
Over the last decade there have appeared a wide number of scientific papers on
teaching methodology in a digital environment. M.E. Vaindorf-Sysoyeva et al. (2020)
highlighted the issue of creating a virtual educational environment.1 V.I. Blinov et
al. (2019) put forward the concept of digital didactics as a scientific discipline on
the organization of educational processes in a digital society [1]. In the context of
linguistic education, the term digital linguodidactics’ is being increasingly used.
This term means a scientifically based system for organizing learning in a digital
environment. Professor S.V. Titova (2017, 2022) has made a significant contribu-
tion to the development of this branch of didactics, with some aspects of digital
assessment being considered in her scientific works [2; 3].
At present, teaching ESP to non-linguistic students is regulated by leading
methodological approaches Language for Specific Purposes (LSP) [4] and Con-
tent Language Integrated Learning (CLIL) [5]. The choice in favor of CLIL,
which is often made by foreign language departments, is focused on the parallel
acquisition of a subject-specific foreign language and disciplines of the profes-
sional cycle [3]. Despite the difference in goal setting, LSP and CLIL are both
interdisciplinary, which implies indirect assessment of both components of sub-
ject-language integration. This, in turn, requires development of interdisciplinary
evaluation tasks.
LSP and CLIL methodology involves a departure from the traditional
‘norm-referencing approach’ to evaluation2, in which the objects of evaluation are
compared with a certain standard (the norm), to formative assessment. An appeal
to formative assessment based on the theory of constructivism (L. Vygostky,
1934, J. Piaget, 1969) and the concept of ‘Mastery learning’ (B. Bloom, 1956,
M. Scriven, 1967) is due to the desire to create conditions for students’ activity-
based learning and their conscious interpretation of cognitive experience [6].
The concepts that are mentioned above are in line with the provisions of compe-
tence-based approach, requiring adequate assessment of learners’ competences as
the expected learning outcomes. Assessment of competencies is a complex prob-
lem of educational methodology. With a huge number of scientific papers devoted
to this issue, there is still no consensus on how to evaluate it. Nevertheless,
it is obvious that with a competence-based approach, both knowledge and practi-
cal skills should be evaluated. Otherwise, it is impossible to talk about achieving
the goal – formation of competences.
In competence-based education the expected learning outcome of ESP train-
ing is formation of foreign language professional communicative competence,
which represents the ability to “use a foreign language for solving professional
tasks” [7]. Undoubtedly, ESP course evaluation should combine knowledge and
1 Vaindorf-Sysoeva ME, Gryaznova NS, Shitova VA. Distance learning methodology:
study guide for universities. Moscow: Yurait Publ.; 2020. (In Russ.)
2 This type of assessment is also called ‘summative.’ Testing is considered to be the main
method of summative assessment.
Иноземцева К.М., Морозова Е.В., Колесников И.М. Вестник РУДН. Серия: Информатизация образования. 2022. Т. 19. № 4. С. 300–311
ЦИФРОВАЯ ОБРАЗОВАТЕЛЬНАЯ СРЕДА 303
skill-based communicative tasks based on clearly defined criteria. However,
the analysis of ESP assessment practices shows that even in offline format teach-
ers tend to simplify assessment procedures, perceiving competence-based learning
outcomes as an abstract concept and therefore making a final judgement on stu-
dents’ achievements based on tests. This approach is not consistent with either
LSP and CLIL methodology, or the competence-based paradigm.
Being a fundamental indicator of the education quality, assessment should
be a constant process of monitoring the educational and cognitive activities of
students. Formative assessment meets this requirement, as it is based on the dy-
namic observation of each student’ academic growth, does not prioritize compari-
son with the standard/norm and does not use repressive function. According to
L.V. Vilkova (2017), formative assessment is a “step-by-step movement of each
student to the best learning outcomes through active inclusion in the analysis (re-
flection) designed to identify difficulties and gaps in a course acquisition and ef-
fectively fill them” [8]. In the context of LSP and CLIL, any practical communi-
cative tasks combining subject-oriented content and being performed through
a foreign language3 (debates, problem discussion, case-study, presentation, labora-
tory experiment report, graph interpretation, diagram analysis, technical instruc-
tion, patent application, minutes, elevator pitch, essay, business letter, scenario,
leaflet, blog, etc.) can serve as objects of formative assessment [5]. Criteria as-
sessment rubrics are required to evaluate the above tasks.
L.V. Vilkova (2017) identifies essential components of formative assess-
ment, which are accepted in this study as constitutive:
1) ‘objectives-results’ dyad, which implies transfer of learning objectives to
expected learning outcomes, which is traditional for competence-based approach;
2) collaborative work with students to define the evaluation criteria, which
promotes students’ better understanding of the requirements and creates sense of
belonging to the assessment process;
3) self-assessment promoting the development of critical thinking;
4) peer assessment aimed at developing objectivity and mutual support;
5) feedback, providing students with recommendations and tips that make
them think better. Following L.V. Vilkova we consider it appropriate to comple-
ment summative and formative assessment methods, as it allows to get a clearer
picture of the program acquisition [8].
In a digital learning environment, it is advisable to implement formative as-
sessment of ESP learning outcomes on the basis of ‘integrated evaluation princi-
ple’, highlighted by the authors of digital didactics concept (Blinov et al., 2019).
According to this principle, “the traditional assessment process is transformed
into a continuous personalized diagnosis-forming assessment of academic suc-
cess” [1, p. 52]. This requires a comprehensive analysis of new didactic digital
tools capable of providing instant feedback, informing a teacher and a student
about the course and results of task fulfillment, issuing personalized ‘trouble-
shooting’ recommendation and adjusting the immediate goals and scenarios for
further development [1, p. 52].
3 In CLIL methodology subject content is delivered not in, but through a foreign language,
which is called ‘vehicle language’ [4].
Inozemtseva K.M., Morozova E.V., Kolesnikov I.M. RUDN Journal of Informatization in Education. 2022;19(4):300–311
304 DIGITAL EDUCATIONAL ENVIRONMENT
The purpose of this study is to determine conditions for effective assessment
of ESP students’ leaning outcomes in a digital learning environment. The study
took place at Linguistics faculty of BMSTU with the participation of 58 master
students of Engineering Business and Management (EBM) faculty studying
‘Business English’. The research materials included 1) didactic materials of
Business English course delivered at BMSTU, including case-studies, supplemen-
tary audio and video materials4 and tests developed and approved by Linguistics
faculty; 2) 20 modern platforms used for evaluation purposes Wizer.me,
iSpring, Genially, Interacty, PruffMe, Exam, Яндекс.Формы, StudySmarter,
Symbaloo, Google Формы, Quizizz, Pear Deck, Kahoot, Typeform, Lumio, Unio,
Learning Apps, Online Test Pad, BrandQuiz, Buncee.
Methodology. The methodological basis of the research included key theo-
retical provisions of formative assessment [9–11], interdisciplinary approaches
LSP and CLIL [4; 12], digital didactics [1], digital linguodidactics [2; 3], distant
learning.5 Based on the analysis of scientific literature and Russian educational
standards (FGOS), the goals and values of formative assessment, as well as the
requirements for ESP learning outcomes of master students majoring in econom-
ics were identified. Then, a comparative analysis of 20 platforms used for assess-
ment purposes was conducted with a view to select the best one. The selection
criteria included simplicity and convenience of the interface, availability of vide-
oconference mode, chat and cloud storage, uploading audio and video files, creat-
ing tests, presentations, sharing files and monitoring results online, free access.
As a result, the domestic service pruff.me was selected for creating a digital as-
sessment space (DAS), a part of BMSTU digital learning environment needed for
implementing formative assessment of ESP learning outcomes.
In the course of study, the analysis of didactic and evaluation materials used
for teaching ESP master students majoring in economics at BMSTU was conduct-
ed. All the materials were adapted to the tasks of formative assessment. In com-
pliance with the formative assessment methodology, evaluation criteria for oral
communicative tasks (case-studies, problem discussions, presentations) were
identified in collaboration with the participating students. Based on the identified
criteria, assessment schemes for the above practical tasks, as well as all other
evaluation materials, were developed and integrated into pruff.me. All the materi-
als have been tested by EBM master students in the process of Business English
distant learning. Based on the results of evaluation materials testing, methodologi-
cal recommendations for ESP teachers have been compiled.
Results and discussion. Business English course for EBM master students
at BMSTU is designed for 1 year and includes 6 modules. During the training,
the students are engaged in in the development of ESP speaking, listening, reading
and writing skills in the field of business communication. Within the framework
of a course, the students extend their knowledge of international business practice,
business education, sales and marketing strategies, quality management, financial
4 Townend J, Allison J, Emmerson P. The business upper intermediate student's book.
Macmillan Education; 2013.
5 Vaindorf-Sysoeva ME, Gryaznova NS, Shitova VA. Distance learning methodology:
study guide for universities. Moscow: Yurait Publ.; 2020. (In Russ.)
Иноземцева К.М., Морозова Е.В., Колесников И.М. Вестник РУДН. Серия: Информатизация образования. 2022. Т. 19. № 4. С. 300–311
ЦИФРОВАЯ ОБРАЗОВАТЕЛЬНАЯ СРЕДА 305
control. The ESP training is conducted on the basis of communicative method,
with a significant time being devoted to case analysis, discussions, role plays and
profession-oriented written tasks. Monitoring and assessment of ESP learning
outcomes is carried out on the basis of oral and written interdisciplinary assign-
ments (case-studies, presentations, problem discussions, tests).
Critical analysis of ESP online assessment practice at BMSTU showed that
despite the absence of explicit criticism of ESP control materials used, the process
of evaluation seemed to be one-sided, with the main focus made on electronic
tests and oral communicative tasks evaluated intuitively by the teachers. This ap-
proach allows to record the average level of ESP course acquisition, but doesn’t
contribute to students’ conscious interpretation of their academic achievements [13].
The virtual classroom observation confirmed the importance of creating a digital
assessment space for ESP teachers and students, that allows to monitor students
individual progress and make an adequate assessment of learning outcomes in
a digital learning environment.
In the course of this research evaluation materials of Business English
course were brought into line with the requirements of formative assessment and
integrated into pruff.me. The description of these materials including criteria-
based assessment schemes is given below.
One of the most popular competence-based techniques used for educational
and evaluation purposes at universities is a case study, a description of a situation
containing a problem or a contradiction and based on real facts. Case study is
widely used in interdisciplinary foreign language teaching as it demonstrates stu-
dents’ ability to apply theory for solving practical tasks and ensures content acqui-
sition through emotional involvement. On the platform pruff.me ESP students can
collaboratively solve the case tasks, listen to the audio and discuss the results in
videoconference mode. The sample of case study placement on the platform is
shown in Figure 1.
In the process of collaborative development of case studies evaluation cri-
teria, it was decided to move away from the traditional format of evaluation scale
(rubric) in favor of a criteria-based assessment scheme. The scheme combines
tasks fulfilment criteria and indicators in a single description of four levels that
describes ESP students’ ability to solve a profession-oriented communicative task
and informs the participants about the assigned points. For the convenience, each
level is assigned its own range of points correlating to the point-rating system
adopted at Linguistics faculty and to the traditional five-point scale familiar to
students. For instance, level 1 corresponds to a score range of 0–4 or ‘bad’,
level 2 – 5–6 points (‘satisfactorily’), level 3 – 7–8 points (‘good’), level 4 – 9–10
points (‘excellent’). The sample of case assessment scheme is presented in Figure 2.
The final communicative task fulfilled by ESP students participating in this
research in the end of each term is individual multimedia presentation. The public
defence of a presentation allows the students to demonstrate so-called ‘soft skills’
including an ability to work with the information sources, insight into the prob-
lem, an ability to communicate information in an accessible form, establish con-
tact with the audience. On pruff.me, presentation topics are given in a list.
The logic of designing a multimedia presentation assessment scheme is similar to
Inozemtseva K.M., Morozova E.V., Kolesnikov I.M. RUDN Journal of Informatization in Education. 2022;19(4):300–311
306
DIGITAL EDUCATIONAL ENVIRONMENT
the same of the case study (Figure 2). The assessment criteria for presentation
are: 1) relevance of the topic; 2) language and communication; 3) logic and per-
suasiveness; 4) design; 5) contact with the audience.
Figure 1. Case study ‘IT Solutions in Business’
Available from: https://pruffme.com/landing/u2696543/tmp1654084672#/26c59f270a6b86468e4de69ae03015d0
(accessed: 15.06.2022)
Figure 2. Casestudy assessment scheme
Available from: https://upload.pruffme.com/download/?media=963b78b3c4c8dee5419fc7492d1fa532
(accessed: 15.06.2022)
Иноземцева К.М., Морозова Е.В., Колесников И.М. Вестник РУДН. Серия: Информатизация образования. 2022. Т. 19. № 4. С. 300–311
ЦИФРОВАЯ ОБРАЗОВАТЕЛЬНАЯ СРЕДА 307
One more oral communicative task used in this study for evaluation purposes
is a problem discussion on the video. Problem discussion is a kind of an argument
aimed at achieving the truth, with participants’ argumentation being the most im-
portant characteristic of the task execution. In the course of this assignment, a subject-
related video is offered for ESP students’ viewing on the platform. Then the teacher
asks questions on the content initiating a discussion. Problem discussion on the video
implies an accurate understanding and correct interpretation of what was seen and
heard. The assessment scheme of this task is shown in Figure 3.
Figure 3. Assessment scheme for the discussion on the video
Available from: https://upload.pruffme.com/download/?media=5d9c676f88f4d1f6743199e0716e0889
(accessed: 15.06.2022)
When fulfilling all oral communicative tasks (cases, presentations, discus-
sions), the students carry out peer assessment by rating their groupmates
achievements on the collaboratively developed criteria (see assessment schemes
in Figures 2, 3). From the grades received, the teacher deduces the average point-
rating value in accordance with the results of each student.
According to the idea of formative and summative assessment tools’ com-
plementarity proposed by L.V. Vilkova, tests were also included in the set of our
evaluation tools and placed on pruff.me. In case of misunderstanding or difficul-
Inozemtseva K.M., Morozova E.V., Kolesnikov I.M. RUDN Journal of Informatization in Education. 2022;19(4):300–311
308
DIGITAL EDUCATIONAL ENVIRONMENT
ties encountered when performing the test, students have an opportunity to com-
municate with the teacher in a chat, where every participant can attach a file, re-
cord/send an audio or a video reply. The teacher in turn can see an avatar and
a name of every student, which makes it much easier to navigate on the platform.
Points are awarded for the correct answers to the test questions and summed up
with the points received for oral communication tasks. The resulted value is taken
as a final student’s score.
An opportunity to chat with a teacher on the platform allows students to
provide feedback. When clicking on the ‘Message section’ icon, the teacher can
see a window with the list of students and their module assignments completed.
The students’ messages in the chat are tied to control activities, which greatly fa-
cilitates the teacher’s checking work and allows to monitor every student’s aca-
demic achievement in dynamics.
Figure 4. ESP student selfassessment checklist
Available from: https://upload.pruffme.com/download/?media=a6eb11397d60d244d2343ffbd97941c1
(accessed: 15.06.2022)
Another important component of formative assessment is self-assessment.
The sample of self-assessment checklist filled by each student in the end of the 1
st
and 2
nd
terms is presented in Figure 4. The proposed self-assessment check-list
placed on pruff.me is designed on the principle of Likert scale, which allows the
respondents to carry out a qualitative evaluation of the educational process, stu-
dents’ perception of the course and also, if necessary, transfer the obtained qualita-
tive data to numerical form. Qualitative characteristics are given in the form of
statements, evaluating participants’ achievements during the period under review on
Иноземцева К.М., Морозова Е.В., Колесников И.М. Вестник РУДН. Серия: Информатизация образования. 2022. Т. 19. № 4. С. 300–311
ЦИФРОВАЯ ОБРАЗОВАТЕЛЬНАЯ СРЕДА 309
the Likert scale from 1 to 5 (‘Over the past semester I’ve learned how to…’),
the quality of the course and the teacher’s work (see statements 1–3 in Figure 4).
These statements characterize the learning outcomes from the students’ perspective.
The developed formative assessment tools were offered to the students par-
ticipating in the research and caused a positive response. The reviews written by
the participants in a free form contained the following comments: ‘it was conven-
ient to chat with the teacher, get explanations’, ‘evaluating each other is a bit te-
dious, but useful, you pay attention to the details’, it was important to evaluate
yourself at the end of the term, it becomes clear what worked and what didn’t’,
it’s nice that the teacher is interested in my opinion about the course’.
The experience gained during the described research showed the need to
compile methodological recommendations for ESP teachers willing to implement
formative assessment technology in a digital learning environment. The recom-
mendations represent a step-by-step description of ESP teachers actions, taking
into account the formative assessment logic and values [14] and technical charac-
teristics of the platform.
Conclusion. Within the framework of the performed study, it was defined
that ‘digital’ assessment of ESP students’ leaning outcomes requires creation of
a digital assessment space, a part of a university digital learning environment,
based on a thoroughly selected platform that allows to place all evaluation materi-
als and implies various modes of teacher-students online interaction.
In order to implement ESP ‘digital’ formative assessment effectively,
it is necessary to fulfill the following conditions:
1) take into account theoretical provisions of ESP&CLIL underlying metho-
dology, when developing evaluation materials;
2) consider formative assessment as a dynamic observation of every stu-
dent’s academic growth aimed at identifying difficulties and filling the gaps, not
at simple measurement of average results;
3) perform the evaluation procedures in accordance with formative assess-
ment essential components, not excluding any of them.
In conclusion, implementing formative assessment technology has had a po-
sitive effect on ESP students’ motivation making them the center of evaluation
process. The ESP teacher, in turn, has been informed about the problems and got
an opportunity to correct the course and teaching methods. The prospects for the
conducted research lie in the field of designing digital assessment space as an es-
sential component of universities’ digital learning environment and ESP teacher
professional development.
References
[1] Blinov VI, Dulinov MV, Esenina EYu, Sergeev IS. Pedagogic concept of digital voca-
tional education and training. Moscow: Pero Publ.; 2019. (In Russ.)
[2] Titova SV. Digital technologies in a foreign language teaching: theory and practice.
Moscow: Editus Publ.; 2017. (In Russ.)
[3] Titova SV. Assessment of project tasks in subject-language integration course: stages,
forms, online instruments. Higher Education in Russia. 2022;(2):94–106. (In Russ.)
http://doi.org/10.31992/0869-3617-2022-31-2-94-106
Inozemtseva K.M., Morozova E.V., Kolesnikov I.M. RUDN Journal of Informatization in Education. 2022;19(4):300–311
310 DIGITAL EDUCATIONAL ENVIRONMENT
[4] Coyle D, Hood P, Marsh D. CLIL: Content and Language Integrated Learning. Cam-
bridge: Cambridge University Press; 2010.
[5] Bloom BS. Taxonomy of educational objectives. Boston: Allyn and Bacon; 1956.
[6] Scriwen M. The methodology of evaluation. Perspectives of curriculum evaluation.
Chicago: Rand McNally; 1967. p. 39–83.
[7] Inozemtseva KM. Profile of professional and methodical competence of ESP teachers
of technical universities. Sovremennyje Problemy Nauki i Obrazovaniya. 2017;(3):111.
(In Russ.)
[8] Vilkova LV. Formative assessment in implementation of a personally oriented ap-
proach (scientific school of Professor M.A. Vikulina). Vestnik Permskogo Gosudar-
stvennogo Gumanitarno-Pedagogicheskogo Universiteta. Seriya 1. Psykhologicheskiye
i Pedagogicheskiye Nauki. 2017;(2):15–23. (In Russ.)
[9] Clark I. Formative assessment: policy, perspectives and practice. Journal of Educa-
tional Administration and Policy. 2011;4(2):158–180.
[10] Briggs D, Ruiz-Primo M, Furtak E, Shepard L. Meta-analytic methodology and infer-
ences about the efficacy of formative assessment. Educational Measurement: Issues
and Practice. 2012;31(4):13–17.
[11] Prilipko EV. Assessment activity of a teacher in the competence-based paradigm: de-
velopment of conscious competence. Innovatsionnyje Project i Programmy Obra-
zovanija. 2017;(1):30–35. (In Russ.)
[12] Dudley-Evans T, St John M-J. Developments in English for specific purposes: a multi-
disciplinary approach. Cambridge: Cambridge University Press; 1998.
[13] Vikulina MA, Vilkova LV. The elements of the methodology regulating formative
assessment of university students’ academic achievements in foreign language
mastering. PNRPU Linguistics and Pedagogy Bulletin. 2019;(2):131–139. (In Russ.)
http://doi.org/10.15593/2224-9389/2019.2.12
[14] Prilipko EV, Inozemtseva KM. Educational values in assessment: a professional development
course for tertiary English language teachers in Russia. Proceedings of the 10th Inter-
national Conference on Education and New Learning Technologies (EDULEARN18).
Mallorca; 2018. p. 11163–11170.
Список литературы
[1] Блинов В.И., Дулинов М.В., Есенина Е.Ю., Сергеев И.С. Проект дидактической
концепции цифрового профессионального образования и обучения. М.: Перо,
2019. 112 с.
[2] Титова С.В. Цифровые технологии в языковом обучении: теория и практика. М.:
Эдитус, 2017. 248 с.
[3] Титова С.В. Оценивание проектных заданий в предметно-языковом интегриро-
ванном курсе: этапы, формы, онлайн-инструменты // Высшее образование в Рос-
сии. 2022. Т. 31. № 2. С. 94–106. http://doi.org/10.31992/0869-3617-2022-31-2-94-106
[4] Coyle D., Hood P., Marsh D. CLIL: Content and Language Integrated Learning. Cam-
bridge: Cambridge University Press, 2010. 184 p.
[5] Bloom B.S. Taxonomy of educational objectives. Boston: Allyn and Bacon, 1956.
[6] Scriwen M. The methodology of evaluation. Perspectives of curriculum evaluation.
Chicago: Rand McNally, 1967. Pp. 39–83.
[7] Иноземцева К.М. Профиль профессионально-методической компетентности пре-
подавателя иностранного языка технического вуза // Современные проблемы
науки и образования. 2017. № 3. С. 111.
[8] Вилкова Л.В. Формирующее оценивание в реализации личностно-ориентирован-
ного подхода (научная школа профессора М.А. Викулиной) // Вестник Пермского
государственного гуманитарно-педагогического университета. Серия 1. Психо-
логические и педагогические науки. 2017. № 2–1. С. 15–23.
Иноземцева К.М., Морозова Е.В., Колесников И.М. Вестник РУДН. Серия: Информатизация образования. 2022. Т. 19. № 4. С. 300–311
ЦИФРОВАЯ ОБРАЗОВАТЕЛЬНАЯ СРЕДА 311
[9] Clark I. Formative assessment: policy, perspectives and practice // Journal of Educa-
tional Administration and Policy. 2011. Vol. 4. No. 2. Pp. 158–180.
[10] Briggs D., Ruiz-Primo M., Furtak E., Shepard L. Meta-analytic methodology and in-
ferences about the efficacy of formative assessment // Educational Measurement: Is-
sues and Practice. 2012. Vol. 31. No. 4. Pp. 13–17.
[11] Прилипко Е.В. Оценочная деятельность преподавателя в компетентностной обра-
зовательной парадигме: формирование осознанной компетентности // Инноваци-
онные проекты и программы в образовании. 2017. № 1. С. 30–35.
[12] Dudley-Evans T., St John M.-J. Developments in English for specific purposes: a multi-
disciplinary approach. Cambridge: Cambridge University Press, 1998.
[13] Викулина М.А., Вилкова Л.В. Элементы методики формирующего оценивания
учебных достижений в овладении иностранным языком обучающимися вуза //
Вестник Пермского национального исследовательского политехнического
университета. Проблемы языкознания и педагогики. 2019. 2. С. 131–139.
http://doi.org/10.15593/2224-9389/2019.2.12
[14] Prilipko E.V., Inozemtseva K.M. Educational values in assessment: a professional
development course for tertiary English language teachers in Russia // Proceedings
of the 10th International Conference on Education and New Learning Technologies
(EDULEARN18). Mallorca, 2018. Pp. 11163–11170.
Bio notes:
Kira M. Inozemtseva, PhD in Education, Associate Professor, Head of Romano-Germanic
Languages Department, Faculty of Linguistics, Bauman Moscow State Technical University,
5 2-ya Baumanskaya St, Moscow, 105005, Russian Federation. ORCID: 0000-0003-3143-0671.
E-mail: ikm@bmstu.ru
Elizaveta V. Morozova, senior lecturer, postgraduate student, English for Industrial Engineering
Department, Faculty of Linguistics, Bauman Moscow State Technical University, 5 2-ya Bauman-
skaya St, Moscow, 105005, Russian Federation. ORCID: 0000-0001-7028-9311. E-mail: liza-
tih@mail.ru
Ilya M. Kolesnikov, master of Linguistics, Faculty of Linguistics, Bauman Moscow State
Technical University, 5 2-ya Baumanskaya St, Moscow, 105005, Russian Federation. ORCID:
0000-0002-2240-7990. E-mail: krik.44442@rambler.ru
Сведения об авторах:
Иноземцева Кира Михайловна, кандидат педагогических наук, доцент, заведующая ка-
федрой «Романо-германские языки», факультет «Лингвистика», Московский государ-
ственный технический университет имени Н.Э. Баумана, Российская Федерация, Москва,
105005, 2-я Бауманская улица, д. 5. ORCID: 0000-0003-3143-0671. E-mail: ikm@bmstu.ru
Морозова Елизавета Вадимовна, старший преподаватель, аспирант, кафедра «Англий-
ский язык для приборостроительных специальностей», факультет «Лингвистика», Мос-
ковский государственный технический университет имени Н.Э. Баумана, Российская
Федерация, Москва, 105005, 2-я Бауманская улица, д. 5. ORCID: 0000-0001-7028-9311.
E-mail: lizatih@mail.ru
Колесников Илья Михайлович, магистр лингвистики, факультет «Лингвистика», Мос-
ковский государственный технический университет имени Н.Э. Баумана, Российская
Федерация, Москва, 105005, 2-я Бауманская улица, д. 5. ORCID: 0000-0002-2240-7990.
E-mail: krik.44442@rambler.ru
... These studies examined various ways of incorporating technology into English language instruction. One study concentrated on formative assessment in a digital learning environment (Inozemtseva et al., 2022), while another proposed an artificial intelligence-based online writing evaluation system (Jin, 2022). A third study developed an English teaching system that combined AI and knowledge recommendation (Sun et al., 2021), and a fourth examined AI-based writing tools for EFL students. ...
... Several studies have been conducted to determine the efficacy of AI-based online assessment in English language instruction, particularly ESP classes. The first study on formative assessment in the field of English for Specific Purposes (ESP) in a digital learning environment was conducted by Inozemtseva et al. (2022). The researchers designed criterionbased evaluation assignments and evaluation schemes for a Business English course for BMSTU master's degree students majoring in economics. ...
... However, addressing technical issues, providing clear instructions and explanations, ensuring the system is user-friendly, and making the exam more engaging and exciting are crucial for the continued success and effectiveness of such platforms. As noted by Gayed et al. (2022); Inozemtseva et al. (2022); Sun et al. (2021) education has the potential to significantly improve the quality of learning, and AI-based online testing platforms can be an important part of this technological revolution. ...
Article
Full-text available
The use of Artificial Intelligence (AI) technology in education has grown in popularity, especially in online assessments. This research evaluates the effectiveness of an AI-based online testing platform, Classtime.com, in teaching tenses in English for Specific Purposes (ESP). The study uses a mixed-methods approach, collecting data from 246 Economic Sharia Law students who took the midterm exam during the 2022-2023 school year. The results show that the average score on the exam was 88.21, indicating a high overall performance. Females scored higher and more consistently than males. The study suggests that Classtime.com's AI-based testing is an effective tool for teaching tenses in ESP assessment, providing immediate feedback and personalized learning experiences. However, the study's focus on one language subject limits its generalizability to other subjects. Nevertheless, the use of AI-based online testing in ESP assessment can improve language learning outcomes and contribute to effective teaching practices. Future research should explore the use of AI-based online testing platforms in other language subjects, investigate factors influencing student performance, and examine long-term effects on language learning outcomes and attitudes toward language education and assessment.
... Такие web-приложения, как Wizer, Quizlet, Myquiz, H5P, дают возможность создавать тематические интерактивные квизы, рабочие листы, тесты, викторины, флеш-карты, игры, видео со встроенными заданиями для проверки усвоения языкового материала [14]. Наконец, ИИ-генераторы заданий Twee, Quizbot, Quillionz и ChatGPT обладают широкими возможностями создания неограниченного количества вопросов к текстам и видео, самых разнообразных заданий на проверку сформированности четырех видов речевой деятельности, сократив до минимума подготовку преподавателя ИЯ к контрольно-оценочным мероприятиям [15]. ...
Article
Full-text available
Introduction. In the context of digital transformation of universities, which prescribes the active use of digital educational environment in teaching, the assessment of educational outcomes cannot be carried out without the use of digital tools that can help teachers to shift the focus of assessment from simple recording of the level of mastery of the curriculum to continuous monitoring of each student's progress. The aim of this study is to determine the readiness of EFL teachers to use modern digital technologies to implement personalized dynamic assessment of students' learning success in accordance with the methodology of formative assessment. Methodology. The basic provisions of summative and formative approaches and digital didactics to the assessment of educational results are considered as the theoretical and methodological foundations of this study. Based on the analysis of Russian and foreign scientific and methodological literature, the necessity of transforming traditional assessment procedures in order to turn them from a repressive measure into an incentive for learning (assessment for learning) was substantiated. From the perspective of the methodology of formative assessment, the conditions for realizing the corrective, motivating and formative functions of assessing the results of language learning with the help of digital tools are considered. The didactic possibilities of web and mobile applications, as well as AI-tools in creating assessment tasks for the development of foreign language communicative skills are analyzed. Results. The results of a survey of current technical university FL teachers (N = 60) conducted to determine their attitudes towards digital assessment and practical readiness to use digital technologies are presented. Conclusion. The results obtained necessitate the development of a professional development program to overcome stereotypes of assessment and to develop the necessary digital skills in FL teachers.
Article
Full-text available
Today, according to various educational documents, a university teacher must be able to control the learning process not only in order to compare the level achieved by students with a certain minimum of requirements laid down in the curriculum on the basis of competent and transparent assessment criteria and to record changes in the general level of preparedness of each student and the dynamics of his successes, but for the diagnosis of problems that arise among students in the learning process (formative assessment); for the development of reflection skills; skills of self-assessment and peer-assessment by students of their achievements. Unfortunately, at the current stage, peer-assessment, reflection and self-assessment are a weak point in the educational process, since usually the student does not participate in control and assessment, he is not involved in the process of developing or choosing assessment criteria, in peer and self-assessment, in reflective activity after completing the project assignment or taking a course. The article examines control and feedback as important components of the methodological system: the stages of control, assessment methods, control and reflection tools. The purpose of this article is to develop the approaches for assessing professionally oriented projects within the framework of a content-integrated language course (CLIL). The article analyzes the didactic potential of feedback during project activities and the conditions for providing effective feedback. The stages of web projects’ assessment, the pre requisites for the successful implementation of the activity in blended learning are described as well as online tools and mobile applications that allow to ensure effective feedback, reflection and assessment in a content-integrated language course.
Article
Full-text available
Proponents of formative assessment (FA) assert that students develop a deeper understanding of their learning when the essential components of formative feedback and cultural responsiveness are effectively incorporated as central features of the formative assessment process. Even with growing international agreement among the research community about the benefits of FA in improving student learning, examination performance and promoting life-long learning, the standards-based reform agenda found in the US has created a politically inhospitable climate for the assessment-driven transformation of classroom practice. This article draws from some 75 sources on instruction and assessment in order to make sense of a) the debate surrounding FA in the US and b) the cultural and individual developmental issues encapsulated within the extensive conceptual territory that is FA. The article has five main sections: the emergence of the term 'formative assessment'; FA in the context of the US; the strategies and principles on which formative practices are founded; the debate in the US on formative assessment; formative assessment and culturally responsive pedagogy.
Article
In a recent article published in EM:IP, Kingston and Nash report on the results of a meta-analysis on the efficacy of formative assessment. They conclude that the average effect of formative assessment on student achievement is about .20 SD units. This would seem to dispel the myth that effects between .40 and .70 can be attributed to formative assessment. They also find that there is considerable variability in effect sizes across studies, and that only the content area in which the treatment is situated explains a significant proportion of study variability. However, there are issues in the meta-analytic methodology employed by the authors that make their findings somewhat equivocal. This commentary focuses on four methodological concerns about the Kingston and Nash meta-analysis: (1) the approach taken to select studies for inclusion, (2) the application of study inclusion criteria, (3) the extent to which the effect sizes being combined are biased, and (4) the relationship between effect size magnitude and characteristics of outcome measures. After examining these issues in the context of the Kingston and Nash review, it appears that considerable uncertainty remains about the effect that formative assessment practices have on student achievement.
Book
CLIL (Content and Language Integrated Learning) has emerged since the millennium as a major trend in education. Written by Do Coyle, Philip Hood and David Marsh and drawing on their experience of CLIL in secondary schools, primary schools and English language schools across Europe, this book gives a comprehensive overview of CLIL. It summarises the theory which underpins the teaching of a content subject through another language and discusses its practical application, outlining the key directions for the development of research and practice. This book acknowledges the uncertainty many teachers feel about CLIL, because of the requirement for both language and subject knowledge, while providing theoretical and practical routes towards successful practice for all.
Profile of professional and methodical competence of ESP teachers of technical universities
  • K M Inozemtseva
Inozemtseva KM. Profile of professional and methodical competence of ESP teachers of technical universities. Sovremennyje Problemy Nauki i Obrazovaniya. 2017;(3):111. (In Russ.)
Pedagogic concept of digital vocational education and training
  • V I Blinov
  • M V Dulinov
  • Esenina Eyu
  • I S Sergeev
Blinov VI, Dulinov MV, Esenina EYu, Sergeev IS. Pedagogic concept of digital vocational education and training. Moscow: Pero Publ.; 2019. (In Russ.)