Available via license: CC BY 4.0
Content may be subject to copyright.
e-ISSN 1135-9250
EDUTEC. Revista Electrónica de Tecnología Educativa.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 157
This work is licensed under a Creative Commons Attribution 4.0 International License.
Received: 27-02-2024
Accepted: 28-03-2024
Co-design in electronic portfolio for learning:
pilot validation of the Co-PIRS Model
Codiseño en portafolios electrónicos para el aprendizaje:
validación piloto del Modelo Co-PIRS
Peng Zhang; p.zhang@uib.es
Laia Riera Negre; laia.riera@uib.es
Gemma Tur; gemma.tur@uib.es
University of the Balearic Islands (Spain)
Abstract
This study performs a pilot validation of the Co-PIRS
model., a co-design paradigm for integrating e-
portfolios into learning environments and addressing
students’ agency, organizing roles into four phases.
This paper examines the model's effectiveness by
exploring learners' satisfaction with the co-design
approach, their evaluation of each Co-PIRS phase, and
their suggestions for modifications, using an ad hoc
questionnaire adaptation. The research utilizes a
mixed-methods design, incorporating quantitative
statistical analysis to explore students' satisfaction
with multiple factors in the learning process,
descriptive quantitative data about students'
evaluation of each phase regarding coherence,
adequacy, and clarity, and participants' qualitative
feedback to refine the model. The results show that
there is a positive correlation between students’
satisfaction, motivation, dedication, and evaluation,
and there is a negative correlation between students’
satisfaction with teacher instruction clarity and the
perceived usefulness of materials. Suggestions by
students are mainly for eliminating tasks in each
phase. The study offers a pilot validation of the model
and reflects on the implications for educational
research, policy-making and practices, which may
further contribute to instructional design and
educational innovation.
Keywords: e-portfolio, electronic portfolio, co-design,
learning design, learner’s agency
Resumen
Este estudio realiza una validación piloto el modelo
Co-PIRS, un paradigma de codiseño para integrar e-
portafolios en entornos de aprendizaje, que organiza
los roles en cuatro fases. Se examina la efectividad del
modelo explorando la satisfacción del alumnado con
el enfoque de codiseño, su evaluación de cada fase del
Co-PIRS y sus sugerencias para modificaciones usando
un cuestionario adaptado ad-hoc. La investigación
utiliza un diseño de métodos mixtos, incorporando un
análisis estadístico cuantitativo para explorar la
satisfacción del estudiantado con múltiples factores
en el proceso de aprendizaje, datos cuantitativos
descriptivos sobre la evaluación de cada fase en
cuanto a coherencia, adecuación y claridad, y
feedback cualitativo para refinar el modelo. Los
resultados muestran que existe una correlación
positiva entre la satisfacción, la motivación, la
dedicación y la evaluación del alumnado, aunque
existe también una correlación negativa entre la
satisfacción del alumnado con la claridad de las
instrucciones del profesor y la utilidad percibida de los
materiales. Las sugerencias del alumnado son
principalmente para eliminar tareas. El estudio ofrece
una validación piloto del modelo y reflexiona sobre las
implicaciones para la investigación, políticas y
prácticas educativas, que pueden contribuir al diseño
y la innovación educativa.
Palabras clave: e-portafolio, portafolio electrónico,
codiseño, diseño de aprendizaje, agencia del
estudiante
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 158
1. INTRODUCTION
Student agency, collaborative learning, and reflective practice have been brought to the
forefront of education with the development of technological advances in education. One of
the developments that have significantly impacted learning environments is the advent of
electronic portfolios or e-portfolios as dynamic learning instruments. It has been extensively
documented that they have an effect on student engagement, self-directed learning, and
assessment, demonstrating their capacity to transform learning experiences and outcomes
(López-Crespo et al., 2021) and, more recently, students’ agency (Zhang & Tur, 2023a). Despite
the recognition of e-portfolios as a catalyst for educational advancement, the full extent of their
benefits, particularly when integrated within a co-designed learning framework in which close
collaboration between teachers and students is well defined, is yet to be fully explored.
1.1. E-portfolio implementation in education
E-portfolios are gaining recognition for their role in enhancing student engagement, self-
directed learning, and innovating assessment practices, as discussed by multiple scholars
(López-Crespo et al., 2022; Mudau, 2022; Muin et al., 2021; Wang & Wang, 2012). Their
importance in improving student self-efficacy and engagement, crucial for academic success,
was emphasized by López-Crespo et al. (2021). Furthermore, e-portfolios contribute to better
metacognitive skills and deeper concept understanding (Lukitasari et al., 2014), with Mudau
(2022) highlighting their utility in open-distance e-learning for fostering student-centered
learning via authentic assessments.
Ngui et al. (2020) highlighted the transformative role of e-portfolios in fostering interactive and
student-centered learning, a sentiment echoed by Wang & Wang (2010) who noted their
contribution to integrating technology and critical thinking in education. López-Crespo et al.
(2021) and Khalid et al. (2015) recognized e-portfolios for enhancing student self-efficacy,
engagement, and reflection. Recent discussions by Buchem et al. (2020) have expanded the
focus to include e-portfolios as tools for promoting student agency.
Research on e-portfolios has mainly targeted higher education, with their role in K-12 emerging
as a new frontier (Chang & Kabilan, 2022). Despite their established presence in higher
education, challenges such as educator reluctance, technical issues, privacy concerns (Cheng,
2022), and the need for continuous support hinder broader adoption (López-Crespo et al.,
2021; Vance et al., 2013). Collaboration is a core process that allows for addressing feedback
and supporting learning, as stated in early research by Zubizarreta (2009).
Co-design in education is a collaborative method where students, educators, and other
stakeholders jointly shape learning experiences and curricula, fostering equal partnerships and
dialogue (Aldridge & Bianchet, 2022). It involves diverse participants in creating educational
innovations, and enhancing personal connections among students and between students and
educators (Brown et al., 2021; Lam, 2020). This approach not only democratizes the design
process by incorporating various perspectives (Vezzoli et al., 2020) but also plays a crucial role
in the ongoing evaluation and refinement of educational practices, ensuring they resonate with
the actual experiences of learners and teachers (Michos et al., 2017).
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 159
The scope of co-design extends to various educational innovations, including digital learning
(Adachi et al., 2022), blended learning (Albó et al., 2021), narrative digital game-based learning
(Breien et al., 2022), and academic agency (Villatoro & de-Benito, 2022).
1.2. The Co-PIRS model: e-portfolio co-design enacting students’ agency
Zhang and Tur (2023 b,c) introduced the Co-PIRS model, a co-design framework for e-portfolio
learning to enhance student agency. Initially conceptualized as a collaborative learning
approach, it evolved into a detailed framework guiding e-portfolio implementation through
joint efforts of students and educators, incorporating elements of student agency as outlined
by Jääskelä et al. (2017; 2020). Drawing from Zubizarreta's (2009) collaborative and mentoring
strategies, the Co-PIRS model fosters a partnership between learners and educators in e-
portfolio development and use. It extends to include sociocultural aspects of student agency,
integrating peer and material support into the e-portfolio process. The model is structured
around four key phases: Planning, Implementation, Revision and Reflection, and Showcase,
embedding agency elements within a co-design collaboration between students and teachers.
Zhang and Tur argue that the Co-PIRS model promotes more engaging and effective e-portfolio
practices, emphasizing the importance of user validation for quality assurance (see Figure 1).
Figure 1
The Co-PIRS model (Zhang & Tur, 2023c, p. 88)
This paper aims to pilot validate and refine the Co-PIRS model to verify its effectiveness in
authentic learning environments by investigating learners’ satisfaction with the co-design
approach in e-portfolio learning, their evaluation of each phase, and modification and revision
suggestions. The three research questions were posed, serving as the focal points of our study:
● RQ1: To what extent does self-perceived satisfaction correlate with motivation and
dedication among students engaged in the e-portfolio co-designed learning journey?
● RQ2: What is the impact of students’ perceptions of teachers, learning design, and e-
portfolio tools on their satisfaction levels?
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 160
● RQ3: What are the students' evaluations of the coherence, adequacy, and clarity of each
phase of the Co-PIRS model, and what revisions or modifications do they suggest?
Through the provision of empirical evidence regarding the practicality and effectiveness of the
Co-PIRS model, this study contributes to the ongoing dialogue concerning e-portfolio
implementation and co-design learning. The goal of the study is to establish a connection
between pedagogical practice and theoretical frameworks by examining the experiences and
perspectives of students regarding the practical implications of co-design in e-portfolio
learning.
2. METHOD
The research utilizes a mixed-methods approach, primarily quantitative, featuring statistical
correlations and descriptive analyses, complemented by qualitative components to enhance
the pilot validation of the Co-PIRS model. Regarding the validation phase, this study employs
the Sarabia and Alconero (2019) model and initiates with a pilot test involving a preliminary
group of participants. This initial step is designed to pave the way for a subsequent, more
comprehensive validation phase, which will incorporate more sophisticated statistical
techniques, including Exploratory and Confirmatory Factor Analysis, on an expanded sample
size.
2.1. The Instrument
2.1.1. Instrument development
An ad hoc questionnaire, adapted from Negre-Bennasar et al. (2023), was used to evaluate the
Co-PIRS model, focusing on the coherence, adequacy, and clarity of its phases and associated
tasks. The questionnaire begins with a consent form, followed by sections that collect general
participant data and assess satisfaction with the e-portfolio experience, offering insights into
the model's impact from the learners' perspective. It then specifically examines the Co-PIRS
model, evaluating its coherence, adequacy, and clarity to understand its effectiveness and
alignment with educational objectives. An open-ended section allows for participant feedback
on potential model improvements, providing valuable insights for future enhancements.
2.1.2. Reliability
The satisfaction scale used in this study demonstrated high reliability, as evidenced by a
Cronbach's Alpha of .909. This indicates that the items on the scale consistently measure the
same underlying concept of learner satisfaction, implying that the questionnaire is a reliable
measure of learner satisfaction.
2.1.3. Validity
To ensure the content validity of our survey, we adopted a collaborative development process,
involving two researchers and subject matter experts who reviewed and refined the questions,
drawing on a validation study by Negre-Bennasar et al. (2023) to ground our instrument in
proven research while tailoring it to our specific needs. For face validity, we aimed to make the
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 161
survey visibly relevant to measuring learner satisfaction, piloting it with a small group of 5
students not involved in the study to identify and rectify any confusing elements. We prioritized
simplicity and clarity in question design and offered the survey in both English and Chinese to
accommodate all participants, enhancing accessibility and understanding.
2.2. Participants
The study involved thirty-nine 10th and 11th graders from a Hong Kong private school,
following an international curriculum and serving upper-middle socio-economic families. Due
to incomplete responses, the sample was narrowed to thirty-six students, aged 15-17,
comprising eleven females and twenty-five males. These participants, all experienced in e-
portfolio use, voluntarily engaged in a semester-long co-design e-portfolio learning journey in
their Chinese language class, offering critical insights into the Co-PIRS model's effectiveness.
2.3. Data Collection
Data collection began on-site at the school five months into the academic year, during the last
class before the midterm break. Students received a detailed briefing on the study's goals and
processes, with an emphasis on the voluntary nature of their participation and their right to
withdraw at any time. To ensure informed responses, a comprehensive review of the Co-PIRS
model was provided, aimed at reinforcing understanding and encouraging reflective
engagement with the co-designed e-portfolio learning experience. With consent obtained, the
survey was distributed electronically via Google Forms, with students using their school-issued
MacBooks for access. This method streamlined data collection, allowing for prompt, organized
responses in a controlled environment, and ensured respondent anonymity, facilitating the
effective gathering and analysis of data.
2.4. Data Analysis
The collected data was analyzed using Microsoft Excel and the Statistical Package for the Social
Sciences (SPSS) version 29. Excel served as the primary tool for organizing and visualizing
descriptive statistics, and SPSS was used for inferential statistical analysis.
Excel was implemented to look at the satisfaction-related data distribution through line graphs
and bar charts. SPSS 29 was used for correlation analysis. After confirming the non-normal
distribution of variables through preliminary tests (Sig. values < 0.001), Spearman's correlation
was chosen due to its suitability for non-parametric data. This method analyzed the
relationships between variables like student satisfaction, motivation, and dedication within the
e-portfolio learning context, including perceptions of teachers, learning design, and tools.
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 162
3. RESULTS
The findings exhibit the following sub-sections to address the three research questions.
3.1. RQ1: Students' satisfaction, motivation, and dedication
The study found a perfect correlation between motivation and dedication, suggesting students
view these aspects nearly identically. Additionally, a significant positive correlation between
satisfaction and both dedication and motivation indicates that higher satisfaction is linked to
increased motivation and dedication. According to Figure 2, all participants rated their
satisfaction, motivation, and dedication above 3 on a 5-point Likert scale, reflecting overall
positive perceptions of the e-portfolio learning experience.
Table 1
Correlations satisfaction, motivation, dedication
**Correlation is significant at the 0.01 level (2-tailed)
Figure 2
Data distribution satisfaction, motivation, dedication
Satisfaction
Motivation
Dedication
Spearman's
rho
Satisfaction
Correlation Coefficient
1.000
.648**
.648**
Sig. (2-tailed)
.
<.001
<.001
N
36
36
36
Motivation
Correlation Coefficient
.648**
1.000
1.000**
Sig. (2-tailed)
<.001
.
.
N
36
36
36
Dedication
Correlation Coefficient
.648**
1.000**
1.000
Sig. (2-tailed)
<.001
.
.
N
36
36
36
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 163
3.2. RQ 2: Students’ satisfaction and its components
3.2.1. Teacher
The study explored the impact of teacher-related factors on student satisfaction, focusing on
four key areas (Table 2):
● Teacher Attitude and Encouragement: A significant moderate positive correlation was
found, indicating that positive teacher attitudes and encouragement are linked to
higher student satisfaction. This suggests that a supportive teacher demeanor can
positively affect learner contentment.
● Teacher Instruction Clarity: A non-significant negative correlation with student
satisfaction was observed, suggesting that instruction clarity alone may not be a strong
determinant of student satisfaction. This indicates that other factors might play a more
crucial role in influencing satisfaction levels.
● Teachers Foster Participation: A strong positive correlation highlighted the significant
impact of teachers promoting participation on student satisfaction. This emphasizes the
value of interactive and engaging teaching methods in enhancing student contentment.
● Teacher Evaluation and Feedback: A significant moderate positive correlation was
found, indicating that constructive feedback and fair evaluation are important to
students and positively influence their satisfaction with the learning experience.
Table 2
Correlations_teacher related factors
Satisfaction
Attitude and
encouragement
Instruction
clarity
Foster
participation and
engagement
Evaluation
and
feedback
Spearman's
rho
Satisfaction
Correlation
Coefficient
1.000
.385*
-.166
.498**
.367*
Sig. (2-tailed)
.
.020
.333
.002
.028
N
36
36
36
36
36
*Correlation is significant at the 0.05 level (2-tailed)
**Correlation is significant at the 0.01 level (2-tailed).
Based on the descriptive data (see Figure 3), teachers' instruction clarity and encouragement
for engagement and participation are highly valued, with a rating of 4-5 out of the 5 Likert scale.
This emphasizes the importance of teachers' influence on students' sense of engagement. Also,
the participants valued the teacher's attitude and encouragement. Compared with other
factors, teacher's evaluation and feedback appeared to have a few 3, lower than other
components; this implies that teachers' ongoing assessment and feedback should be further
addressed.
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 164
Figure 3
Data distribution_teacher related factors
3.2.2. Learning design
The analysis of student satisfaction in relation to learning design revealed (see Table 3):
● Materials Usefulness: Exhibited a slight negative, but insignificant correlation,
suggesting materials' utility might not strongly affect satisfaction.
● After-classroom Activities: Showed a moderate positive correlation, indicating their
positive impact on satisfaction.
● Workload Adequacy: Had a positive but non-significant correlation, implying workload
perceptions might not greatly influence satisfaction.
● Learning Activities: Demonstrated a moderate positive and significant correlation,
highlighting their importance in enhancing satisfaction.
Table 3
Correlations_learning design-related factors
Satisfaction
Materials
usefulness
Usefulness of
after classroom
learning activities
Workload
adequacy
Learning
activities
Spearman's
rho
Satisfaction
Correlation
Coefficient
1.000
-.207
.385*
.190
.371*
Sig.
(2-tailed)
.
.225
.020
.266
.026
N
36
36
36
36
36
*. Correlation is significant at the 0.05 level (2-tailed).
**. Correlation is significant at the 0.01 level (2-tailed).
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 165
In Figure 4, the descriptive data entails that the students are satisfied with in-class activities
overall, with all participants rating over 3. Some students rated 2 for workload adequacy,
indicating they perceive the workload to be insufficient or too much. Several students also rate
the usefulness of materials and after-class activities as 2, but most rates are 4-5.
Figure 4
Image caption
3.2.3. Tool utilization: e-portfolio
The analysis of ePortfolio components indicated their significant impact on student satisfaction,
with each aspect interlinked to enhance the educational experience (Table 4). The ePortfolio's
utility as a learning aid showed a moderate positive correlation, suggesting that students find
it a valuable tool, thereby increasing their satisfaction. Furthermore, the assessment and
evaluation functions within the ePortfolio demonstrated a very strong positive relationship
with satisfaction, underscoring the importance of effective evaluation methods in the
educational journey. Most notably, the role of the ePortfolio in knowledge and skills
development was highlighted by an extremely strong positive correlation, pointing to its critical
influence in enriching students' learning outcomes and overall satisfaction with their
educational experience.
Table 4
Correlations_e-portfolio related factors
Satisfaction
Learning
aid
Assessment and
evaluation
Knowledge/skills
development
Spearman's
rho
Satisfaction
Correlation
Coefficient
1.000
.362*
.732**
.926**
Sig. (2-tailed)
.
.030
<.001
<.001
N
36
36
36
36
*. Correlation is significant at the 0.05 level (2-tailed).
**. Correlation is significant at the 0.01 level (2-tailed).
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 166
Figure 5 demonstrates that all participants scored 3 and above for their satisfaction with the e-
portfolio as a learning tool. Particularly on assessment, evaluation, and skills and knowledge
development, with all participants rating 4 or 5. The participants also approve E-portfolios used
for learning aids, but some rate it 3. This indicates that the e-portfolio’s function as a learning
aid can be further addressed.
Figure 5
Data distribution-portfolio related factors
3.3. RQ 3: Validation and suggestions
This section presents the findings from the evaluation of the Co-PIRS model's implementation,
focusing on the assessment of specific actions within each phase of the model. The evaluation
criteria were centered on three key aspects:
● Coherence: how well the actions within each phase were interconnected and aligned
with the objectives of that particular phase of the e-portfolio implementation.
● Adequacy: how well the actions were tailored to fit the context and conditions of the
process during the respective phase of e-portfolio utilization.
● Clarity: in relation to the ease of understanding the actions to be taken in this phase of
the implementation.
The suggestions for elimination, modification, and inclusion from the participants are also
included in the following sections.
3.3.1. Planning Phase
Figure 6 indicates participants’ rating on the coherence, adequacy, and clarity of the tasks on
the planning phase of the Co-PIRS model.
● Coherence: Majority of participants (24) rated Coherence at 4, indicating that the
actions were well-connected and aligned with the objectives of the Planning phase but
might benefit from minor improvements. A notable number of participants (12) felt the
Coherence deserved the highest rating (5), suggesting excellent interconnection of
actions within this phase.
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 167
● Adequacy: Ratings were evenly split between 4 and 5 (16 responses each), indicating
that most participants found the actions to be well-tailored to the project's conditions
during the Planning phase. A small number (4) rated Adequacy at 3, suggesting that
while generally adequate, there could be room for better customization.
● Clarity: Clarity received the highest number of top ratings (20 for 5), showing that most
participants found the actions to be clearly understandable. Fewer participants (12)
rated it at 4, with a very small group (4) giving it a 3, indicating a slight need for improved
clarity.
Figure 6
Coherence, adequacy, and clarity_Planning
According to Table 5, almost half of the participants (n=17, 47%) wanted to eliminate some
tasks in the Planning phase, including Discuss with teachers (Students’ actions) and Discuss with
peers (Students’ actions). They further stated that planning and objectives-setting should be
more independent. Some students intended to modify some actions (n=13, 36%), such as
Platform/tools selection (Teachers’ actions), which is highlighted by multiple students; one
students claimed “I am not satisfied with the platform that I am using at the moment”. Also,
Make detailed plans collaboratively (Students’ actions) were suggested to be modified, the
students stated “having some thoughts is fine, no need to have detailed plan at the very
beginning”.
Regarding the items to be included, two students (6%) recommended adding a modelling
component before some complicated tasks, in the teachers’ action.
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 168
Table 5
Participants’ suggestion_Planning
Actions
Frequency
%
Elimination
17
47%
Modification
13
36%
Inclusion
2
6%
3.3.2. Implementation phase
The evaluation of the implementation phase is shown in Figure 7.
● Coherence: Ratings are identical to the Planning phase for the top two tiers (12 for 5
and 24 for 4), indicating a consistent perception of logical consistency and alignment
during the Implementation phase.
● Adequacy: Adequacy received more top ratings (5) in this phase (20 responses),
suggesting that the actions were seen as more tailored to the process conditions
compared to the Planning phase. Like Planning, some room for improvement is
indicated by the 16 responses at a 4 and the 4 responses at a 3.
● Clarity: Clarity is rated highest among the three aspects in this phase (24 for 5), implying
that the actions to be taken were most understandable during the Implementation
phase. Consistent with Planning, a smaller number of participants rated Clarity at 4 (12
responses), and none rated it below 3.
Figure 7
Coherence, adequacy, and clarity_Implementation
It is observed that some students expressed their feedback on eliminating (n = 7, 19%) and
modifying (n = 10, 28%) some action in this phase. Several students wanted to remove the
action of Document and design e-portfolios collaboratively with peers (students’ action) since
they prefer to do it themselves. The action of encouraging students’ autonomy by guiding the
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 169
use of learning materials and resources (teachers’ action) was also emphasized in students’
comment since some of believe that “students’ autonomy is far from enough.”
For task inclusion suggestions, one student recommended adding the action of enabling
students’ to choose the tasks to record on e-portfolios, since some prefer electronic tasks,
some prefer paper-pen-based work.
Table 6
Participants’ suggestion_Implementation
Actions
Frequency
%
Elimination
7
19%
Modification
10
28%
Inclusion
3
8%
3.3.3. Reflection and Revision phase
Figure 8 demonstrates the assessment of the Reflection and Revision phase in terms of clarity,
adequacy, and clarity.
● Coherence: The highest ratings (5) increased slightly (17 responses) compared to the
previous phases, indicating better alignment of actions within this phase. There is a
slight increase in the number of 3 ratings (6 responses) suggesting a few participants
found some actions less coherent.
● Adequacy: Adequacy remained high with the majority rating it at 5 (19 responses) and
4 (13 responses), but there was one response at 2, indicating a single concern regarding
the fit of the actions.
● Clarity: Clarity ratings are similar to Adequacy with a high level of agreement (18 for 5
and 15 for 4), but again, there are a few responses at 3, signaling a need for minor
improvements in understanding.
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 170
Figure 8
Coherence, adequacy, and clarity_Reflection and Revision
Table 7 portrays the participants' suggestions on the Reflection and Revision phase. There is no
mention of including additional actions. Eleven students (31%) and eight (22%) would like to
modify and eliminate some actions, respectively. Peer evaluation (students' action) was
mentioned by multiple participants to be removed. They think peer feedback is good enough
and that peer evaluation is unnecessary since this may affect their mark. For modification,
Revising and optimizing (students' action) were most emphasized; some students stated, "I
don't think revising is necessary; it is too much work. Reflection is enough."
Table 7
Participants’ suggestion_Reflection and Revision
Actions
Frequency
%
Elimination
8
22%
Modification
11
31%
Inclusion
0
0
3.3.4. Showcase phase
The rating of the Showcase phase is depicted in Figure 9.
● Coherence: Coherence ratings remain high, with more responses at 4 (19) than 5 (15),
suggesting consistent but not perfect alignment. A small number of participants rated
it at 3 (2 responses), indicating minor inconsistencies.
● Adequacy: Adequacy ratings are quite balanced across the top two tiers (16 for 5 and
15 for 4), with a slight increase in the number of 3 ratings (5 responses), suggesting
some actions were less tailored than others.
● Clarity: Clarity is perceived slightly better than Adequacy, with a higher number of top
ratings (18 for 5) and an equal number of 4 ratings (16), indicating clear understanding
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 171
for most participants. Only a couple of participants rated Clarity at 3, hinting at the need
for marginal enhancements in articulation.
Figure 9
Coherence, adequacy, and clarity_Showcase
The students' feedback on the actions in the Showcase phase is shown in Table 8. There was
one comment on additional inclusion (3%); the student did not mention what to include. Seven
students (19%) would like to remove the Self-reflection (students' action) and Acknowledge
the usefulness of learning materials and resources for future learning (students' action) actions
since they believe it is repetitive with the previous phase. One claimed, "This can be part of the
reflection."
Nine participants suggested modifying some actions, including Giving summative feedback
(teacher's action) and Summative evaluation (teacher's action). Some said, "I am figuring out
how to work on this with e-portfolio," since they believe that e-portfolio learning is formative
learning with ongoing formative assessments.
Table 8
Participants’ suggestion_Showcase
Actions
Frequency
%
Elimination
7
19%
Modification
9
25%
Inclusion
1
3%
Overall, the data reveals that the Co-PIRS model was well received throughout all phases, with
the highest ratings consistently given to Clarity. This indicates that the participants considered
the actions easy to understand. Strong ratings were also given to Coherence and Adequacy.
However, there are indications that specific elements of the model could be enhanced to align
and customize the actions with the conditions and objectives of the model more precisely. The
model could be improved to maximize its efficacy, as indicated by the few 3 ratings spanning
various phases and aspects. Besides, the students suggested eliminating, modifying, or adding
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 172
some actions. Those suggestions guide the further update of the Co-PIRS model to make it
more concise and streamlined. The updated model is described in the Discussion section.
4. DISCUSSION AND CONCLUSIONS
The Co-PIRS model pilot validation study contributes to understanding e-portfolio learning,
particularly the interplay between co-design practices and learner experiences. This discussion
reflects on the study's findings, focusing on students' satisfaction, motivation, commitment,
and the impact of teacher-related factors, instructional design, and e-portfolio tool
implementation. Additionally, it addresses the Co-PIRS model refinement implications.
4.1. Interrelation of Satisfaction, Motivation, and Dedication
The study reveals a perfect correlation between motivation and dedication and their significant
positive correlation with satisfaction. It underlines a foundational principle in educational
psychology: the intrinsic link between learners' emotional states and their engagement and
perseverance in learning tasks (Ryan & Deci, 2000). The findings suggest that students are
simultaneously motivated when dedicated to their e-portfolio learning journey, which is
consistent with the literature linking motivation and dedication as parallel constructs in
educational experiences (Schunk et al., 2014). Furthermore, the significant positive correlation
between satisfaction and dedication, as well as satisfaction and motivation, highlights the
importance of satisfaction as a driver for student engagement, aligning with Ryan and Deci's
(2000) previous publication on self-determination theory, which states that satisfaction of basic
psychological needs enhances motivation and engagement. The high levels of satisfaction
reported by participants in co-designed e-portfolio learning environments suggest that such
settings may effectively meet these psychological needs, thereby enhancing motivation and
dedication.
4.2. Students' Satisfaction and Teacher-related Factors
The findings on teacher-related factors and student satisfaction shed light on the teacher's role
within the Co-PIRS framework, suggesting that teacher encouragement and positive attitudes
moderately enhance student satisfaction, aligning with Göktaş and Kaya's (2023) emphasis on
the importance of teacher-student relationships. Surprisingly, a negative correlation between
instruction clarity and satisfaction challenges conventional beliefs about the value of clear
instruction, posited by Fendler et al. (2016). This might indicate that in a co-design
environment, where student agency is prioritized, overly directive instruction could feel
restrictive, hinting at the need for a balance between clear guidance and learner autonomy.
The positive impact of teacher-facilitated participation on satisfaction further highlights the
importance of active learning, resonating with Campen et al. (2023), who underscore the value
of teacher feedback in enhancing the educational experience.
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 173
4.3. Students' Satisfaction and Learning Design
The negative correlation between material usefulness and student satisfaction contradicts
common beliefs about material quality impacting satisfaction (Pham et al., 2019), suggesting a
misalignment with students' goals or co-design needs, emphasizing the importance of material
relevance. The positive correlation between after-classroom activities and satisfaction supports
the idea that practical application enhances engagement (Doo, 2021), highlighting the value of
experiential learning outside the classroom. The lack of strong correlation between workload
adequacy and satisfaction implies students prioritize assignment quality over quantity
(Khonamri & Pavlíková, 2020), suggesting the significance of assignment relevance. The positive
relationship between engaging learning activities and satisfaction points to the necessity of
activities that promote deep understanding and reflection (Sølvik & Glenna, 2022), stressing
the importance of interactive and meaningful activities in the learning process.
4.4. Students' Satisfaction and e-Portfolio
The e-portfolio's positive impact on learning aid satisfaction underscores its value in enhancing
student engagement by facilitating reflection and progress tracking (Ismailov & Laurier, 2021).
Its strong correlation with assessment satisfaction highlights the importance of personalized
feedback through e-portfolios, promoting a more engaging learning experience (Muin et al.,
2021). Furthermore, the significant link between e-portfolios and knowledge and skills
development satisfaction emphasizes their role in visualizing learning progress, crucial for
student fulfillment (Jääskelä et al., 2017).
4.5. Co-PIRS Model Refinement
Feedback on the Co-PIRS model indicates high overall satisfaction but identifies areas for
improvement to enhance its flexibility and adaptability. Participants suggest streamlining
actions, simplifying language, and offering clear explanations to improve user-friendliness.
Specific recommendations include:
• Planning Phase: Lessen the focus on mandatory collaboration to allow for more
autonomy.
• Implementation Phase: Provide options for both collaborative and individual
documentation, with a preference for collaborative efforts.
• Reflection Phase: Eliminate revision components, deemed redundant within the
reflection process, to alleviate workload concerns; shift from peer evaluation to
feedback, and streamline the reflection for efficiency without sacrificing depth.
• Showcase Phase: Enhance distinction from earlier phases and clarify the balance
between formative and summative assessments.
Considering these elements, an updated Co-PIRS model has been proposed, integrating these
refinements (see Figure 10).
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 174
Figure 10
The updated Co-PIRS model
4.6. Implication
The pilot validation of the Co-PIRS model emphasizes its importance for educational research,
highlighting the impact of co-design on learner agency and the need for further study across
various educational levels. It points to the significance of teacher strategies, material utility
versus learner satisfaction, and suitable workload in co-designed e-portfolios. The validation
process in this work is in itself a contribution to research in educational innovation, and in
particular in the e-portfolio field, since the pilot stage allows implementing in small contexts,
informs the model and prepares both the practice and the research instrument for further
scalable contexts.
For educators, the findings stress the importance of co-design training, creating relevant
materials, and using e-portfolios for reflection and assessment to enhance learner autonomy
and feedback mechanisms.
Policymakers are advised to support e-portfolio co-design learning infrastructure and training,
ensuring the availability of adaptable e-portfolio systems that promote student agency and
satisfaction.
The model's analysis supports e-portfolios' role in boosting student satisfaction and skill
development, advocating for their adaptation in diverse educational settings. The study
champions a learning paradigm that values personalized and co-designed experiences,
emphasizing learner empowerment.
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 175
4.7. Conclusion
The Co-PIRS model's pilot validation highlights its effectiveness in enhancing learner agency,
satisfaction, and engagement through e-portfolio co-design, emphasizing the importance of
teacher involvement and tool functionality. These insights guide educators and policymakers
in fostering learner-centered approaches and inform ongoing refinements to the model and e-
portfolio learning practices.
The study's limitations include a small sample size of 36 and its setting in a Hong Kong private
school, potentially affecting the generalizability and cultural applicability of the findings. The
reliance on self-reported questionnaires and the short evaluation period may also limit the
study's scope, suggesting a need for more diverse and longitudinal research.
Future studies should aim for broader participant diversity and incorporate expert insights,
possibly through Delphi studies. Employing methodological triangulation and extending the
research timeframe could enhance validity and provide a comprehensive understanding of the
Co-PIRS model's long-term impact. These efforts will deepen the understanding of co-design e-
portfolio practices and their role in educational innovation.
5. REFERENCES
Adachi, C., O’Donnell, M., & Elliott, J. (2022). Co-creating a digital learning innovation
framework through design thinking approaches. ASCILITE Publications, e22140.
https://doi.org/10.14742/apubs.2022.140
Albó, L., Stylianidou, N., Chalatsis, X., Dieckmann, M., & Hernández-Leo, D. (2021). Including
Students’ Voices in the Design of Blended Learning Lesson Plans. In T. De Laet, R.
Klemke, C. Alario-Hoyos, I. Hilliger, & A. Ortega-Arranz (Eds.), Technology-Enhanced
Learning for a Free, Safe, and Sustainable World (Vol. 12884, pp. 419–423). Springer
International Publishing. https://doi.org/10.1007/978-3-030-86436-1_47
Aldridge, J. M., & Bianchet, S. (2022). Using student feedback about the learning environment
as a starting point for co-construction. Learning Environments Research, 25(3), 939–
955. https://doi.org/10.1007/s10984-021-09403-9
Breien, F., Wasson, B., Greiff, S., & Hauan, N. P. (2022). The eLuna mixed-reality visual language
for co-design of narrative game-based learning trails. Frontiers in Education, 7,
1061640. https://doi.org/10.3389/feduc.2022.1061640
Brown, B., Hurrell, C., Roberts, V., Jacobsen, M., Neutzling, N., & Travers-Hayward, M. (2021).
Open Education Co-Design as a Participatory Pedagogy in an Online Graduate Program.
The Open/Technology in Education, Society, and Scholarship Association Conference,
1(1), 1–8. https://doi.org/10.18357/otessac.2021.1.1.41
Buchem, I., Tur, G., & Salinas, J. (2020). Preface. Designing for ownership in technology-
enhanced learning (TEL): a core element for learners’ SRL and agency. Interaction
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 176
Design and Architecture(s) Journal - IxD&A, 45, pp. 5 - 14. https://doi.org/10.55612/s-
5002-045-001psi
Campen, C. A. N. K., Wise, A., & Molenaar, I. (2023). The equalizing effect of teacher dashboards
on feedback in K-12 classrooms. Interactive Learning Environments, 31(6), 3447–3463.
https://doi.org/10.1080/10494820.2021.1931346
Chang, S. L., & Kabilan, M. K. (2022). Using social media as e-Portfolios to support learning in
higher education: A literature analysis. Journal of Computing in Higher Education.
https://doi.org/10.1007/s12528-022-09344-z
Cheng, Y.-H. (2022). Benefits and Barriers of Utilizing E-portfolios in EFL Writing [Preprint]. In
Review. https://doi.org/10.21203/rs.3.rs-1829171/v1
Doo, M. Y. (2021). Understanding Flipped Learners’ Perceptions, Perceived Usefulness,
Registration Intention, and Learning Engagement. Contemporary Educational
Technology, 14(1), ep331. https://doi.org/10.30935/cedtech/11368
Fendler, R. J., Ruff, C., & Shrikhande, M. (2016). Evaluating Characteristics of Top and Bottom
Performance: Online Versus In-Class. American Journal of Distance Education, 30(2),
109–120. https://doi.org/10.1080/08923647.2016.1153350
Göktaş, E., & Kaya, M. (2023). The Effects of Teacher Relationships on Student Academic
Achievement: A Second Order Meta-Analysis. Participatory Educational Research,
10(1), 275–289. https://doi.org/10.17275/per.23.15.10.1
Ismailov, M., & Laurier, J. (2022). We are in the “breakout room.” Now what? An e-portfolio
study of virtual team processes involving undergraduate online learners. E-Learning and
Digital Media, 19(2), 120–143. https://doi.org/10.1177/20427530211039710
Jääskelä, P., Poikkeus, A.-M., Vasalampi, K., Valleala, U. M., & Rasku-Puttonen, H. (2017).
Assessing agency of university students: Validation of the AUS Scale. Studies in Higher
Education, 42(11), 2061–2079. https://doi.org/10.1080/03075079.2015.1130693
Khalid, F., Ahmad, M., Karim, A. A., Daud, Md. Y., & Din, R. (2015). Reflective Thinking: An
Analysis of Students’ Reflections in Their Learning about Computers in Education.
Creative Education, 06(20), 2160–2168. https://doi.org/10.4236/ce.2015.620220
Khonamri, F., Pavlikova, M., & Constantine the Philosopher University in Nitra, Nitra, Slovakia.
(2020). Exploring Teachers’ and Learners’ Attitude towards Homework: The case of
English versus Non-English-Major Teachers’ Homework Practices. Education & Self
Development, 15(4), 32–49. https://doi.org/10.26907/esd15.4.07
Lam, J. (2020). Co-designing expressive textile arts: Student advancements in service-learning.
Arts & Humanities Open Access Journal, 4(4), 119–128.
https://doi.org/10.15406/ahoaj.2020.04.00162
López-Crespo, G., Blanco-Gandía, M. C., Valdivia-Salas, S., Fidalgo, C., & Sánchez-Pérez, N.
(2022). The educational e-portfolio: Preliminary evidence of its relationship with
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 177
student’s self-efficacy and engagement. Education and Information Technologies, 27(4),
5233–5248. https://doi.org/10.1007/s10639-021-10827-2
Lukitasari, M., Susilo, H., Ibrohim, I., & Duran Corebima, A. (2014). Lesson Study in Improving
the Role of E-Portfolio on the Metacognitive Skill and Concept Comprehension: A Study
on Cell Biology Subject in IKIP PGRI Madiun, Indonesia. American Journal of Educational
Research, 2(10), 919–924. https://doi.org/10.12691/education-2-10-11
Michos, K., Fernández, A., Hernández-Leo, D., & Calvo, R. (2017). Ld-Feedback App: Connecting
Learning Designs with Students’ and Teachers’ Perceived Experiences. In É. Lavoué, H.
Drachsler, K. Verbert, J. Broisin, & M. Pérez-Sanagustín (Eds.), Data Driven Approaches
in Digital Education (Vol. 10474, pp. 509–512). Springer International Publishing.
https://doi.org/10.1007/978-3-319-66610-5_51
Mudau, P. K. (2022). Lecturers’ Views on the Functionality of e-Portfolio as Alternative
Assessment in an Open Distance e-Learning. International Journal of Educational
Methodology, 8(1), 81–90. https://doi.org/10.12973/ijem.8.1.81
Muin, C. F., Hafidah, H., & Daraini, A. M. (2021). Students’ Perceptions on the Use of E-Portfolio
for Learning Assessment. AL-ISHLAH: Jurnal Pendidikan, 13(1), 497–503.
https://doi.org/10.35445/alishlah.v13i1.485
Negre-Bennasar, F., Benito, B., Tur, G., & Villatoro-Moral, S. (2023). Diseño e implementación
de un modelo Aprendizaje-Servicio dirigido a los Objetivos de Desarrollo Sostenible
aplicando técnicas de codiseño. Revista Electrónica Interuniversitaria de Formación Del
Profesorado, 26(3), 177–193. https://doi.org/10.6018/reifop.558221
Ngui, W., Pang, V., & Hiew, W. (2020). Designing and Developing an e-Portfolio for Second
Language Learners in Higher Education. International Journal of Information and
Education Technology, 10(5), 362–366. https://doi.org/10.18178/ijiet.2020.10.5.1390
Pham, L., Limbu, Y. B., Bui, T. K., Nguyen, H. T., & Pham, H. T. (2019). Does e-learning service
quality influence e-learning student satisfaction and loyalty? Evidence from Vietnam.
International Journal of Educational Technology in Higher Education, 16(1), 7.
https://doi.org/10.1186/s41239-019-0136-3
Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic
motivation, social development, and well-being. American Psychologist, 55(1), 68–78.
https://doi.org/10.1037/0003-066X.55.1.68
Sarabia Cobo, C. M., & Alconero Camarero, A. R. (2019). Claves para el diseño y validación de
cuestionarios en Ciencias de la Salud. Enfermería en cardiología, 77(1), 69-73.
https://hdl.handle.net/10902/26747
Schunk, D. H., Meece, J. L., & Pintrich, P. R. (2014). Motivation in education: Theory, research,
and applications (4th ed). Pearson.
Sølvik, R. M., & Glenna, A. E. H. (2022). Teachers’ potential to promote students’ deeper
learning in whole-class teaching: An observation study in Norwegian classrooms.
EDUTEC. Revista Electrónica de Tecnología Educativa. e-ISSN 1135-9250
Zhang, P., Riera Negre, L., & Tur, G.
Issue 88 – June 2024
DOI: https://doi.org/10.21556/edutec.2024.88.3181
Page 178
Journal of Educational Change, 23(3), 343–369. https://doi.org/10.1007/s10833-021-
09420-8
Vance, G., Williamson, A., Frearson, R., O’Connor, N., Davison, J., Steele, C., & Burford, B.
(2013). Evaluation of an established learning portfolio. The Clinical Teacher, 10(1), 21–
26. https://doi.org/10.1111/j.1743-498X.2012.00599.x
Vezzoli, Y., Mavrikis, M., & Vasalou, A. (2020). Inspiration cards workshops with primary
teachers in the early co-design stages of learning analytics. Proceedings of the Tenth
International Conference on Learning Analytics & Knowledge, 73–82.
https://doi.org/10.1145/3375462.3375537
Villatoro, S. M., & de-Benito, B. C. (2022). Self-Regulation of Learning and the Co-Design of
Personalized Learning Pathways in Higher Education: A Theoretical Model Approach.
Journal of Interactive Media in Education, 2022(1), 6. https://doi.org/10.5334/jime.749
Wang, S., & Wang, H. (2012). Organizational schemata of e-portfolios for fostering higher-order
thinking. Information Systems Frontiers, 14(2), 395–407.
https://doi.org/10.1007/s10796-010-9262-0
Zhang, P., & Tur, G. (2023a). Exploring Students’Agency in ePortfolio Implementation: A Case
Study. Proceedings of the 2023 6th International Conference on Educational Technology
Management, 208–214. https://doi.org/10.1145/3637907.3637975
Zhang, P., & Tur, G. (2023b). Collaborative e-Portfolios Use in Higher Education During the
COVID-19 Pandemic: A Co-Design Strategy. International Journal of Educational
Methodology, 9(3), 585–601. https://doi.org/10.12973/ijem.9.3.585
Zhang, P. & Tur, G. (2023c). A Co-design Approach for Collaborative e-Portfolio Use in Higher
Education: Introducing the Co-PIRS Model. In T. Bastiaens (Ed.), Proceedings of EdMedia
+ Innovate Learning (pp. 87-92). Vienna, Austria: Association for the Advancement of
Computing in Education (AACE). https://www.learntechlib.org/primary/p/222486/
Cite this work:
Zhang, P., Riera Negre, L., & Tur, G. Co-design in electronic portfolio for learning: pilot
validation of the Co-PIRS Model. Edutec, Revista Electrónica de Tecnología Educativa, (88),
157-178. https://doi.org/10.21556/edutec.2024.88.3181