ArticlePDF Available

Smart device-based testing for medical students in Korea: satisfaction, convenience, and advantages

April 2017
Journal of Educational Evaluation for Health Professions 14:7

DOI:10.3352/jeehp.2017.14.7

License
CC BY 4.0

Authors:

Hallym University

The aim of this study was to investigate respondents’ satisfaction with smart device-based testing (SBT), as well as its convenience and advantages, in order to improve its implementation. The survey was conducted among 108 junior medical students at Kyungpook National University School of Medicine, Korea, who took a practice licensing examination using SBT in September 2015. The survey contained 28 items scored using a 5-point Likert scale. The items were divided into the following three categories: satisfaction with SBT administration, convenience of SBT features, and advantages of SBT compared to paper-and-pencil testing or computer-based testing. The reliability of the survey was 0.95. Of the three categories, the convenience of the SBT features received the highest mean (M) score (M= 3.75, standard deviation [SD]= 0.69), while the category of satisfaction with SBT received the lowest (M= 3.13, SD= 1.07). No statistically significant differences across these categories with respect to sex, age, or experience were observed. These results indicate that SBT was practical and effective to take and to administer.

Average responses in each category on a 5-point Likert scale on items assessing satisfaction with smart device-based testing, as well as its convenience and advantages, among medical students in Korea according to gender. Category

…

Figures - available via license: Creative Commons Attribution 4.0 International

Content may be subject to copyright.

Available via license: CC BY 4.0

Content may be subject to copyright.

Page 1 of 4

(page number not for citation purposes)

2017, Korea Health Personnel Licensing Examination Institute

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits

unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Brief report

J Educ Eval Health Prof 2017; 14: 7 • https://doi.org/10.3352/jeehp.2017.14.7

In 2020, the Korean medical licensing examination will be ad-

ministered as a smart device-based test using a tablet PC as the user

interface [1]. Therefore, medical schools have attempted to imple-

ment smart device-based testing (SBT) in the administration of

practice licensing examinations and have investigated students’ satis-

faction with SBT, as well as its effectiveness and convenience, in or-

der to improve students’ adaptation to it. The purpose of this study

was to investigate satisfaction with SBT and its perceived conve-

nience among medical students at Kyungpook National University,

Korea.

Study design

This was a cross-sectional observational study.

Materials and subjects

The evaluation survey included all 108 junior medical students at

Kyungpook National University, Korea. They took a practice licens-

ing examination using SBT and filled out an evaluation survey in

September 2015. The evaluation survey for SBT consisted of three

categories: satisfaction with SBT (two items), convenience of SBT

features (13 items), and advantages of SBT (13 items). The category

assessing the advantages of SBT was composed of two parts: one com-

pared respondents’ satisfaction with SBT to their satisfaction with

paper-and-pencil testing, and the other part contained a similar com-

parison with computer-based testing (CBT). The following 5-point

Likert scale was used for the survey: 1, strongly disagree; 2, disagree;

3, neutral; 4, agree; and 5, strongly agree. Table 1 presents the con-

tent of the three categories, the number of items, and the reliability

of the survey. For determining the reliability of the survey, internal

consistency (Cronbach α) was used. The Cronbach α coefficient of

the survey was 0.905, and the range of this coefficient for each item

was 0.807 to 0.936; this indicated strong internal consistency and a

high level of reliability. Table 2 presents the respondents’ background

in terms of gender, age, and experience with CBT and SBT. The

percentage of respondents with CBT experience was 96.3%, and the

corresponding value for SBT was 17.6%. This indicates that most of

the students had some experience with CBT, but not many students

had experience with SBT.

eISSN: 1975-5937

Open Access

Smart device-based testing for medical students in Korea:

satisfaction, convenience, and advantages

Eun Young Lim1, Mi Kyoung Yim2, Sun Huh3*

1Division of Educational Evaluation, Korea Institute for Curriculum and Evaluation, Seoul, Korea

2Research and Development Division, Korea Health Personnel Licensing Examination Institute, Seoul, Korea

3Department of Parasitology and Institute of Medical Education, Hallym University College of Medicine, Chuncheon, Korea

The aim of this study was to investigate respondents’ satisfaction with smart device-based testing (SBT), as well as its convenience and

advantages, in order to improve its implementation. The survey was conducted among 108 junior medical students at Kyungpook

National University School of Medicine, Korea, who took a practice licensing examination using SBT in September 2015. The survey

contained 28 items scored using a 5-point Likert scale. The items were divided into the following three categories: satisfaction with

SBT administration, convenience of SBT features, and advantages of SBT compared to paper-and-pencil testing or computer-based

testing. The reliability of the survey was 0.95. Of the three categories, the convenience of the SBT features received the highest mean

(M) score (M= 3.75, standard deviation [SD] = 0.69), while the category of satisfaction with SBT received the lowest (M= 3.13,

SD = 1.07). No statistically significant differences across these categories with respect to sex, age, or experience were observed. These

results indicate that SBT was practical and effective to take and to administer.

Keywords: Computers; Personal satisfaction; Republic of Korea; Medical students; Tablets

*Corresponding email: shuh@hallym.ac.kr

Editor: A Ra Cho, e Catholic University of Korea, Korea

Received: April 13, 2017; Accepted: April 22, 2017;

Published online: April 24, 2017

is article is available from: http://jeehp.org

Journal of Educational Evaluation

for Health Professions

Page 2 of 4

(page number not for citation purposes)

http://jeehp.org

J Educ Eval Health Prof 2017; 14: 7 • https://doi.org/10.3352/jeehp.2017.14.7

Statistics

The results of the evaluation of SBT were summarized using de-

scriptive statistics in SPSS for Windows ver. 23.0 (IBM Corp., Ar-

monk, NY, USA). The responses regarding satisfaction, convenience,

and advantages were compared according to sex, age, experience with

CBT, and experience with SBT. Two-way repeated-measures analysis

of variance test was conducted for a comparative analysis using DB-

STAT ver. 5.0 (DBSTAT Co., Chuncheon, Korea).

Ethical approval

Informed consent was obtained from the subjects. The study de-

sign was approved by the institutional review board of Hallym Uni-

versity (HIRB-2015-092).

Table 3 shows the descriptive statistics of the three categories and

the two subcategories regarding the advantages of SBT. The category

of the convenience of SBT received the highest mean (M) score (M=

3.75, standard deviation [SD]= 0.69). The next highest mean was

that of the category assessing the advantages of SBT compared to pa-

per-and-pencil testing (3.43). The lowest mean was that of the cate-

gory evaluating satisfaction with SBT (M = 3.13, SD= 1.07). The

means among the three categories were significantly different (P<

0.0001). Supplementary files of the coding books and integrated data

for each category are presented in Supplement 1. Appendix 1 pres-

ents the results of all the items. The range of the 28 items was 2.83

to 4.23. Satisfaction with SBT was assessed using two items: satisfac-

tion with using a tablet PC as a testing tool (M = 2.92, SD= 1.24) and

lack of difficulty in carrying out the testing according to the advice

for test-takers presented on the tablet PC (M=3.13, SD = 1.26).

The percentage of negative responses (strongly disagree and disagree)

was higher than 30% for the former item; this implies that further

improvement in SBT administration is necessary.

The category evaluating the convenience of SBT features was com-

posed of 13 items. The range of the items in this category was 2.83

to 4.23. The highest mean score (M= 4.23, SD = 0.74) was found

for the item assessing whether the function of identifying items that

were not marked before submitting the answers was convenient, and

the next highest mean score was found for the following item: ‘it was

convenient to have a ‘check’ function for later review’ (M =4.18, SD =

0.77). This indicates that students recognized that the features that

enabled them to review and correct their answers were the most con-

venient. The lowest mean was for the item assessing whether it was

convenient to use the functions of zoom in and zoom out when view-

ing an image or replaying a video file (M= 2.83, SD = 1.29). The

next lowest mean was of the following item: ‘the loading time was

adequate before the start of video playback’ (M= 3.25, SD =1.29).

Therefore, our results indicate that the features related to video need

Table 2. The number of subjects based on their background

Background Frequency (%)

Gender

Male 70 (64.8)

Female 38 (35.2)

Total 108 (100.0)

Age (yr)

20–24 2 (1.9)

25–30 93 (86.1)

≥31 13 (12.0)

Total 108 (100.0)

Experience of taking computer-based testing

No 4 (3.7)

Yes 104 (96.3)

Total 108 (100.0)

Experience of taking smart device-based testing

No 89 (82.4)

Yes 19 (17.6)

Total 108 (100.0)

Table 1. The contents, number of items, and Cronbach α coecient of each category in the survey

Categories Content No. of items Cronbach α

Satisfaction of SBT Satisfaction of students taking SBT 2 0.807

Convenience of SBT features Degree of convenience of SBT features 13 0.916

Advantages of SBT Advantage of SBT compared to a paper testing and pencil testing and a computer based testing. 13 0.936

1) Compared to a paper and pencil testing 7 0.905

2) Compared to a computer based testing 6 0.892

Total 28 0.905

SBT, smart device-based testing.

Table 3. The descriptive statistics of 3 categories and 2 subcategories

(N = 108)

Mean ± standard deviation

Satisfaction of SBT 3.13 ± 1.07

Convenience of SBT 3.75 ± 0.69

Advantages of SBT 3.33 ± 0.83

1) Compared to a paper and pencil testing 3.43 ± 0.89

2) Compared to a computer based testing 3.20 ± 0.87

SBT, smart device-based testing.

Page 3 of 4

(page number not for citation purposes)

http://jeehp.org

J Educ Eval Health Prof 2017; 14: 7 • https://doi.org/10.3352/jeehp.2017.14.7

to be improved.

The results for the advantages of SBT compared to paper-and-pen-

cil testing were as follows: The range of mean scores was 2.91 to 3.89,

indicating that the advantages of SBT were positively evaluated by

the students. The highest mean was for the item assessing whether

the absence of separate answer-card marking was helpful in manag-

ing the testing time (M = 3.89, SD= 1.06). The next highest mean

was for the following item: ‘it was convenient to select the correct

answer on the tablet PC screen rather than marking an optical mark

reader (OMR) answer card’ (M= 3.85, SD= 1.06). The lowest mean

was for the following item: ‘tablet PC-based testing was more conve-

nient than paper-and-pencil testing’ (M= 2.91, SD= 1.09). The next

lowest mean was for the following item: ‘it was good to be able to

see materials such as photographs, sounds, and video clips’ (M= 3.07,

SD = 1.09). The range of the advantages of SBT compared to CBT

was 2.94 to 3.34. The highest mean was for the following item: ‘there

was no noise or heat from the desktop computer, so the examination

environment was comfortable’ (M= 3.34, SD =1.02). The lowest

mean was for the following item: ‘the testing instrument was smart,

and it helped me to focus more on the testing’ (M= 2.94, SD= 1.14).

No statistically significant differences were observed in the mean

scores for satisfaction with SBT, its convenience, and its advantages

according to sex (P = 0.2979), age (P= 0.7126), experience with CBT

(P= 0.5024), or experience with SBT (P= 0.2640) (Fig. 1).

SBT has also been administered in several other medical schools

in Korea, where it has been known as ubiquitous-based testing (UBT).

UBT was established to be a satisfactory and convenient testing tool

in one of these medical schools [2]. In another medical school, stu-

dents had a positive attitude towards the UBT interface. The factors

of having experience with CBT and being male were associated with

favorable responses towards UBT [3]. In this study, students thought

Fig. 1. Average responses in each category on a 5-point Likert scale on

items assessing satisfaction with smart device-based testing, as well as

its convenience and advantages, among medical students in Korea ac-

cording to gender.

Category

Average of response

Satisfaction Convenience Advantages

4.0

3.5

3.0

2.5

2.0

1.5

1.0

0.5

Female

Male

that SBT could improve the effectiveness of taking a test, reduce ex-

tra work such as marking answers on an OMR sheet, and make the

testing environment more pleasant than that in the computer lab. As

the means of the items showed positive patterns for the satisfaction

and convenience of SBT, most items had a high rate for ‘agree’ and

‘strongly agree.’ In conclusion, the results of the survey indicate a posi-

tive attitude toward SBT. However, the technical features of SBT,

such as the zoom in and zoom out functions when viewing an image

and playing video clips, could be improved further.

ORCID: Eun Young Lim: http://orcid.org/0000-0001-6838-

183X; Mi Kyoung Yim: http://orcid.org/0000-0001-5048-0477;

Sun Huh: http://orcid.org/0000-0002-8559-8640

Authors’ contribution

Conceptualization: SH. Data curation: MKY. Formal analysis:

EYL. Funding acquisition: SH. Methodology: EYL MKY SH. Proj-

ect administration: SH. Visualization: EYL MKY SH. Writing–orig-

inal draft: EYL. Writing–review & editing: EYL MKY SH.

Conict of interest

No potential conflict of interest relevant to this article was reported.

Funding

This work is supported by the National Health Personnel Licensing

Examination Board of Korea (Fundref ID: 10.13039/501100003647)

research fund (RE02-1604-00).

Supplementary materials

Supplement 1. Data file is available from https://dataverse.har-

vard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/GT2IZY

Supplement 2. Audio recording of the abstract.

References

1. Kim CH. Presidential address: introduction of smart device-based test-

ing and item exposure policy for Korea Health Personnel Licensing

Examination. J Educ Eval Health Prof 2017;14:5. https://doi.org/

10.3352/jeehp.2017.14.5

2. Kwon OY, Rhee SY, Choi JM, Kim YS. Usefulness of ubiquitous-based

testing for evaluations in me dical education. Korean J Med Educ 2015;

27:3-10. https://doi.org/10.3946/kjme.2015.27.1.3

3. Roh H, Lee JT, Rhee BD. Ubiquitous-based testing in medical educa-

tion. Med Teach 2015;37:302-303. https://doi.org/10.3109/014215

9X.2014.956070

Page 4 of 4

(page number not for citation purposes)

http://jeehp.org

J Educ Eval Health Prof 2017; 14: 7 • https://doi.org/10.3352/jeehp.2017.14.7

Appendix 1. Frequency and mean of each item in the survey given to medical students in Korea after smart device-based testing in 2015

Item Mean ± standard

deviation

Frequency

Strongly

disagree Disagree Neutral Agree Strongly

agree

Satisfaction with SBT

1. I was satised with using the tablet PC as the testing tool. 2.92 ± 1.24 19 20 30 29 10

2. There was no diculty in carrying out the test according to the advice for test takers presented on the tablet PC. 3.13 ± 1.26 15 18 20 30 16

Convenience of SBT

3. Function of the remaining testing time notication was more convenient than an announcement of the remaining testing

time.

3.35 ± 1.26 11 19 20 37 21

4. The selection and correction function of the correct answer was convenient. 3.80 ± 1.00 310 19 50 26

5. It was convenient to see one item at a time on the screen. 3.85 ± 1.01 211 19 45 31

6. The function of identifying items that were not marked before submitting the answers was convenient. 4.23 ± 0.74 0 3 11 52 42

7. It was convenient to be able to see the previous items, the next items, and all items. 3.97 ± 0.86 2 4 17 57 28

8. It was convenient to be able to see the solved items, unsolved items, checked items, and the items noted for later review

selectively.

3.91 ± 1.03 3 9 17 45 34

9. It was convenient to have a ‘check’ function for later review. 4.18 ± 0.77 0 3 15 50 40

10. The ‘check wrong answers’ function to check the wrong answer option was convenient. 3.95 ± 0.95 1 7 24 40 36

11. The font size was appropriate. 3.85 ± 0.96 4 6 16 58 24

12. The typeface was appropriate. 3.93 ± 0.85 3 2 19 60 24

13. The loading time was adequate before the star t of video playback . 3.25 ± 0.94 512 55 26 11

14. It was convenient to use then functions of zoom in and zoom out when viewing an image and replaying a video le. 2.83 ± 1.29 22 22 27 26 11

15. The overall screen conguration was appropriate. 3.58 ± 0.91 4 6 34 51 13

Advantages of SBT

Comparison with paper–and-pencil testing

16. It was convenient to select the correct answer on the tablet PC screen rather than marking the optical mark reader answer

card.

3.85 ± 1.03 5 6 18 50 29

17. The absence of separate answer-card marking was helpful in managing the testing time. 3.89 ± 1.06 5 6 19 44 34

18. The psychological burden (tension) decreased because there was no separate answer-card marking. 3.75 ± 1.16 710 17 43 31

19. The font size was large and clear, minimizing eye fatigue. 3.35 ± 1.08 910 36 40 13

20. It was good to be able to see materials such as photographs, sounds, and video clips. 3.07 ± 1.09 11 18 40 30 9

21. Tablet PC-based testing was more convenient than paper-and-pencil testing. 3.21 ± 1.25 14 18 21 41 14

22. The test taker’s ability is assessed more accurately by tablet PC-based testing than by paper-and-pencil testing. 2.91 ± 1.09 16 16 43 28 5

Comparison with desktop-based testing

23. The touch-screen method was more convenient than the mouse-click method for selecting the correct answer. 3.33 ± 1.12 10 12 31 42 13

24. There was less eye strain in the case of the tablet PC than in the case of the desktop PC monitor. 3.22 ± 1.09 914 42 30 13

25. There may be a decrease in cheating behavior during tablet PC-based testing as compared to desktop PC-based testing. 3.24± 1.03 910 45 34 10

26. The testing instrument was smart, and it helped me to focus more on the test. 2.94 ± 1.14 16 15 46 21 10

27. There was no noise or heat from the desktop computer, so the examination environment was comfortable. 3.34 ± 1.02 811 34 46 9

28. Tablet PC-based testing is an improved testing method compared to the existing desktop PC-based testing method. 3.12± 1.09 10 18 39 31 10

SBT, smart device-based testing.

Supplementary Material

Data

April 2017

Eun Young Lim · Mi Kyoung Yim · Sun Huh

Evaluation of Student Satisfaction with Ubiquitous-Based Tests in Women’s Health Nursing Course

Article

Full-text available

Nov 2021

Learning evaluation using ubiquitous-based tests may be essential during a public health crisis, such as the COVID-19 pandemic, during which theoretical classes and clinical practice are conducted online. However, students may not be as familiar with ubiquitous-based tests as they are with paper-based tests. This survey study aimed to evaluate students’ satisfaction with ubiquitous-based tests and compare the evaluation results of a paper-based test with that of a ubiquitous-based test in nursing education. For the midterm exam of the Women’s Health Nursing course, a paper-based test was conducted, while a ubiquitous-based test using a tablet computer was used for the final exam. The Ubiquitous-Based Test Usefulness and Satisfaction tool, which has a five-point Likert-type response scale, was employed to evaluate the post-test usefulness and satisfaction scores of the ubiquitous-based test. The mean score of the ubiquitous-based test usefulness was 4.01 ± 0.67. There was a significant difference in satisfaction levels between the ubiquitous-based and the paper-based test (t = −3.36, p = 0.001). Specifically, the evaluation scores were not affected by different evaluation methods. Study participants deemed the ubiquitous-based test highly useful and satisfactory, suggesting that such tests may be a future-oriented evaluation method, potentially replacing paper-based tests.

Evaluation of usefulness of smart device-based testing: a survey study of Korean medical students

Article

Full-text available

Sep 2020

Purpose: This study aims to understand the characteristics of smart device-based testing (SBT) by comparing the typical characteristics of students' satisfaction with SBT, its usefulness, advantages, and disadvantages when compared with existing testing methods. Methods: A total of 250 students from the first to third year were selected as the final targets of the study and the questionnaire was developed by faculty members who participated in the survey from the start of the SBT. The total number of questions is 12, and the questionnaire used a 4-point scale. The data obtained were analyzed using the IBM SPSS ver. 23.0 (IBM Corp., Armonk, USA). Results: Answers to the "satisfaction with SBT" were generally negative, while answers to the "usefulness of SBT" were generally positive. There was no difference in satisfaction across gender and smart device ownership, whereas there were significant differences across years. With reference to the usefulness of SBT, students responded positively, while about the overall configuration and completeness of SBT, students responded negatively. Students also seemed to show a greater preference toward the pencil-paper test. Conclusion: On the other hand, students generally thought that SBT helped to assess medical knowledge better and was a more objective method of knowledge assessment than a pencil-paper test. We believe that students preferred the traditional paper-pencil test due to their unfamiliarity with SBT. We believe that an appropriate and careful remedy for drawbacks of the SBT will have a significant impact in the accumulation of actual clinical knowledge and in the improvement of practical skills for medical students.

The relationship of individual examinee characteristics and acceptability of smart device-based testing to test score in the practice test of the Korea Emergency Medicine Technician Licensing Examination

Article

Full-text available

Dec 2018

Purpose Smart device-based testing (SBT) is being introduced into the Republic of Korea’s high-stakes examination system, starting with the Korean Emergency Medicine Technician Licensing Examination (KEMTLE) in December 2017. In order to minimize the effects of variation in examinees’ environment on test scores, this study aimed to identify any associations of variables related to examinees’ individual characteristics and their perceived acceptability of SBT with their SBT practice test scores. Methods Of the 569 candidate students who took the KEMTLE on September 12, 2015, 560 responded to a survey questionnaire on the acceptability of SBT after the examination. The questionnaire addressed 8 individual characteristics and contained 2 satisfaction, 9 convenience, and 9 preference items. A comparative analysis according to individual variables was performed. Furthermore, a generalized linear model (GLM) analysis was conducted to identify the effects of individual characteristics and perceived acceptability of SBT on test scores. Results Among those who preferred SBT over paper-and-pencil testing, test scores were higher for male participants (mean± standard deviation [SD], 4.36± 0.72) than for female participants (mean± SD, 4.21± 0.73). According to the GLM, no variables evaluated— including gender and experience with computer-based testing, SBT, or using a tablet PC—showed a statistically significant relationship with the total score, scores on multimedia items, or scores on text items. Conclusion Individual characteristics and perceived acceptability of SBT did not affect the SBT practice test scores of emergency medicine technician students in Korea. It should be possible to adopt SBT for the KEMTLE without interference from the variables examined in this study.

Item Equivalence Verification according to Test Information Media of the Optician National Licensing Examination: Focused on the Smart Device Based and Paper Based Tests Including Multimedia Items

Article

Full-text available

Aug 2022

This study conducts the validity of the pen-and-paper and smart-device-based tests on optician’s examination. The developed questions for each media were based on the national optician’s simulation test. The subjects of this study were 60 students enrolled in E University. The data analysis was performed to verify the equivalence of the two evaluation methods, specifically, through split-plot factorial design of the evaluation method as a partition variable. As a result of the statistical significance test for the difference in achievement for each type of test information medium, indicating that there was no difference in achievement according to the type of test information medium at the significance level of .05. Although the validity of the smart device-based test and the paper-and-pencil test was verified through this study. To develop and set multimedia items in the optician national licensing examination, it is necessary to establish guidelines for how to develop the items.

Effects of School-Based Exercise Program on Obesity and Physical Fitness of Urban Youth: A Quasi-Experiment

Article

Full-text available

Mar 2021

(1) Purpose: The purpose of this study was to evaluate if an after-school intervention program could prevent obesity and promote the physical fitness of urban sedentary school children. (2) Methods: A 16-week after-school physical fitness program was provided to 36 middle school students (7th, 8th, and 9th graders) recruited from a middle school for three days a week. They were high-risk youth showing poor health status (level 4 or 5) in the regular physical fitness evaluation conducted. Their body mass index (BMI), cardiovascular endurance, muscular strength and endurance, quickness, and flexibility were evaluated. (3) Results: A paired sample t-test was used (α = 0.05). There were statistical differences ((x ± s), p < 0.05) between the pre-Progressive Aerobic Cardiovascular Endurance Run (PACER) (13.36 ± 4.98 (# of laps)) and post-PACER (18.64 ± 6.31 (# of laps)) (p < 0.001), between the pre-sit-up (18.06 ± 7.22 (# of sit-ups)) and post-sit-up (24.89 ± 7.52 (# of sit-ups)) (p < 0.001), and between the pre-Trunk Flexion (2.64 ± 3.49 ((cm))) and post-Trunk Flexion (5.97 ± 2.78 ((cm)) (p < 0.001). There was no statistical difference between the pre-50m-Run (10.74 ± 1.30 ((sec))) and post-50m-Run results (10.69 ± 1.25 (sec)) (p = 0.063) or between the pre-BMI (24.84 ± 3.97 (kg/m2)) and post-BMI (24.76 ± 3.61 (kg/m2)) (p = 0.458). Overall, the physical fitness measures improved, whereas BMI did not change after 16 weeks. (4) Conclusion: Sixteen weeks of the school-based Health-related Physical Fitness (HrPF) program can be effective in improving overall physical fitness levels of adolescents, although additional treatments would be required to change BMI, which showed no improvement in the present study. It is concluded that in order for adolescents to maintain and promote physical fitness and health, participation in a school-based HrPF program is recommended for at least 30 min a day, three days a week. In addition, schools should provide high-risk youth with easy access to physical activities that are similar to the objectives of the physical education curriculum. In order to promote the health of school-age children, each school should establish and operate school-based systematic intervention programs.

Journal Metrics of Infection & Chemotherapy and Current Scholarly Journal Publication Issues

Article

Full-text available

Sep 2018

Sun Huh

Background: In 2013, Infection & Chemotherapy changed the main language of its articles to English so that they could be submitted to PubMed Central. This study presents the recent status of journal metrics for Infection & Chemotherapy and introduces scholarly journal publishing policies or guidelines that have recently appeared. Materials and methods: A variety of journal metrics were analyzed based on the Web of Science Core Collection, including the nationality of authors the proportion of funded articles to Journal Articles , manually calculated impact factor, the titles of journals in which articles were cited", and the Hirsch index. Results: Out of 181 articles of Infection & Chemotherapy published between 2015 and 2018, the highest number of authors were from Korea (165, 91.2%). There were articles from 11 other countries. The proportion of funded articles to Journal Articles has increased yearly and has reached 46.7% in 2018. The manually calculated impact factor of the year 2017 was 1.728, which corresponded to 21.5% of the 2017 Journal Citation Reports (JCR) category of "infectious diseases". There were 196 source journal titles that cited Infection & Chemotherapy in the 2015-2018 issues. The Hirsch index was 15. Conclusion: The metrics results above demonstrate that over the years, Infection & Chemotherapy was developed into a top-level international-level journal so that it could be utilized by researchers across the world. The adoption of new policies including author taxonomy, an open data policy, a clinical data sharing policy, the principles of transparency and best practice in scholarly publishing 3rd edition will help increase the transparency of the authorship and the scientific integrity of the articles.

(NON)COMPUTER-ORIENTED TESTING IN HIGHER EDUCATION: VIEWS OF THE PARTICIPANTS OF THE EDUCATIONAL PROCESS ON (IN)CONVENIENCE USING

Article

Full-text available

Apr 2024

Volodymyr Starosta

The article focuses the problems of using computer-based testing learning outcomes under modern conditions of accelerated development of distance and blended learning. The aim of the study is to identify and compare the views of the participants of the educational process in higher education (students, PhD students, university teachers) on convenience using during computer-based testing as compared to paper-based testing and oral-based testing. The author included 3206 respondents in the anonymous online survey in 2020-2023, including 2897 students (Bachelor and Master students), 140 PhD students, and 169 university teachers; used IBM SPSS Statistics 23 for statistical analyses. There are research results: the convenience of testing improves in the direction (the mean values of convenience and the corresponding percentages of respondents): computer-based testing – students (М=3,68; 58,3%) → university teachers (M=3,81; 65,7%) → PhD students (M=4,31; 84,3%); paper-based testing – students (M=3,21; 42,2%) → PhD students (M=3,28; 44,3%) → university teachers (M=3,46; 54,6%); oral-based testing – students (M=2,49; 19,9%) → PhD students (M=2,84; 31,4%) → university teachers (M=3,69; 60,8%). Oral-based testing is the least convenient for students and PhD students. Paper-based testing is the least convenient for university teachers. Respondents have maximum convenience during computer-based testing for all studied groups. The t-test shows a statistically significant difference (p≤0,05) of the convenience testing in the majority of compared independent groups (students, PhD students, university teachers). The author obtained the results in dynamics over four years: convenience increases in the direction oral-based testing → paper-based testing → computer-based testing for all groups according to various characteristics (educational level, gender, specialty, form of education, year of the survey). The study shows that in most cases there are no statistically significant differences (p≤0,05) of the convenience testing in the following cases: between all independent students groups for all testing forms according to the feature «form of education»; according to the feature «academic performance level» for computer-based testing. We believe that the obtained results regarding the (in)convenience testing of participants in the educational process should be taken into account in the practice of higher education institutions.

Development and application of a mobile-based multimedia nursing competency evaluation system for nursing students: A mixed-method randomized controlled study

Article

Sep 2022
Nurse Educ Pract

Aim To develop a mobile-based multimedia Nursing Competency Evaluation (NCE) system based on the Attention, Relevance, Confidence, Satisfaction model and verify its effectiveness. Background In education, mobile devices can enable the delivery of learning content without time and spatial constraints. Mobile-based test is emerging as a novel method using technologies to appraise students’ performance on practicum. This mobile-based test go beyond the simple evaluation of memorised knowledge, a limitation common to paper-based tests. They are useful because they can include multimedia items such as videos, animations and pictures to comprehensively evaluate students’ clinical competencies. Methods This study was conducted in a nursing university in Seoul, South Korea in September 2021. A mixed method randomised controlled design was employed to evaluate its usability. The participants in the experimental group used the Nursing Competency Evaluation system and joined in focus group interviews for verifying the effects of the Nursing Competency Evaluation system qualitatively. Those in the control group responded to the mobile-based test, but which has only text-based test items. The system usability, effectiveness and learning satisfaction in both groups were measured after the mobile-based test experience. Quantitative and qualitative data were analysed using t-tests and thematic analysis using focus group interviews, respectively. Results Sixty nursing students participated, with 30 each in the experimental and control groups. There were no significant differences in nursing competency scores between the two groups. However, average scores for effectiveness and learning satisfaction were significantly higher in the experimental than in the control group. Nineteen experimental group participants partook in the interviews, with many describing that the Nursing Competency Evaluation system allowed them to experience new learning contents and efficiently learn practical nursing skills that can be useful in clinical settings. Conclusions The Nursing Competency Evaluation system is a promising method because it used mobile technologies and multimedia to appraise students’ performance on nursing practicum. It was found that the Nursing Competency Evaluation system with multimedia items is more realistic, interactive and satisfactory compared with text-based mobile test. Thus, we expect it to be used in future nursing curriculums to improve students’ nursing competencies.

Development, Application, and Effectiveness of a Smart Device–based Nursing Competency Evaluation Test: A Mixed-Method Study

Article

Apr 2021
CIN-COMPUT INFORM NU

We aimed to develop and evaluate the effectiveness of a smart device-based test to assess Korean undergraduate students' clinical nursing competency, named SBT-NURS. The 65-item SBT-NURS comprises questions that simulate clinical situations, are problem solving-oriented, use multimedia (ie, videos/photos/animations), and involve the following topics: medical-surgical nursing, fundamentals of nursing, pediatrics, maternity, management, and psychiatric. We utilized a quantitative method to analyze the effects of the SBT-NURS (ie, via a single-group, post-experimental survey design) and a qualitative method to analyze students' experiences of using the SBT-NURS (ie, via seven focus group interviews [FGIs]). Students' overall adult health nursing paper-based test scores (ie, combining their scores in group activity, presentation, attendance, and attitude toward the midterm and final tests on adult health nursing) (r = 0.552, P < .001) and clinical practicum scores (r = 0.268, P = .040) in the last semester showed a statistically significant positive correlation with their SBT-NURS scores. Their paper-based testing practice average scores (ie, combination between paper-based tests and clinical practicum scores) showed a similar significant correlation (r = 0.506, P < .001). Students deemed the SBT-NURS advantageous, satisfactory, convenient, and useful. The SBT-NURS may be an effective learning and evaluation method for nursing education that help improve students' clinical competency and learning outcomes.

How to Deal with the Concept of Authorship and the Approval of an Institutional Review Board When Writing and Editing Journal Articles

Article

Full-text available

Jun 2020

Sun Huh

Presidential address: smart device-based test and item exposure policy introduced to Korea Health Personnel Licensing Examination

Article

Full-text available

Mar 2017

Chang Hwi Kim

Usefulness of ubiquitous-based testing for evaluations in medical education

Article

Full-text available

Mar 2015

The purpose of this study was to evaluate the suitability (convenience, objectiveness, and satisfaction) of ubiquitous-based testing (UBT) as a medical education evaluation tool. UBT was administered using a smart pad in our medical school in May 2012. A questionnaire was given twice. The pre-UBT questionnaire examined possession of a tablet computer, skillfulness of smart devices, the convenience of UBT, and the usefulness of a medical educational assessment tool. The post-UBT questionnaire evaluated the satisfaction, convenience, and preference of UBT and the usefulness of a medical educational assessment tool, as in the pre-UBT test. The survey was measured on a 4-point scale: 1 is "strongly disagree" and 4 is "strongly agree." One hundred three students (male, 55.3%) participated in the UBT. The mean age was 29.2±2.4 years. In the pre-UBT questionnaire analysis, students responded affirmatively to the items about the skillfulness of smart devices, clinical skill assessment, and achievement of educational objectives. The responses to the items on the convenience and satisfaction with the UBT were positive in the post-UBT. The factors that affected the post-UBT questionnaire were as follows: knowledge assessment (p=0.041) and achievement of educational objectives (p=0.015) were significant, based on gender, and satisfaction with the UBT (p=0.002) was significant, based on possession of a tablet computer. The relationship between the ranks of this UBT and the average ranks of the three previous semesters was statistically significant (p<0.001). Convenience, objectiveness, knowledge assessment, and composition and completion were useful items in the UBT.

Ubiquitous-based testing in medical education

Article

Full-text available