Self-Assessment - Science topic

Thank you so much.

"The increased focus of pedagogic discourse on ‘new paradigms’ of feedback practice is attributed to the theoretical move towards focusing on students’ actions and sense of agency in shared dialogue and socially constructed meaning.

Formative student peer reviews, often used in creative disciplines, can present a significant strategy for enhancing learning through discursive interaction, negotiation and collaboration as students become more actively engaged in feedback processes.

Peer review involves students reviewing each other’s work by giving and receiving peer feedback comments, while offering suggestions for development, and thereby enhancing their opportunities to make academic judgements...

There are five key considerations for educators to facilitate effective practice in peer reviewing activities with dialogic feedback. The following strategies can increase opportunities for the development of critical thinking skills, feedback literacy and active engagement in feedback processes, while enhancing the student experience...

As educators, the responsibility lies with us to positively influence power dynamics. We need to teach students to recognise power as a capability that can be attained by cooperating with others to help distribute power more equitably among all students, while supporting inclusive spaces for peer review practice. We must continue to endeavour to facilitate equal access to peer learning opportunities for students who might otherwise be excluded or marginalised within higher education settings.

Therefore, we need to get a deeper understanding of the power dynamics that students experience in our classrooms, how these imbalances may impact student learning from peer reviews and feedback and how to counteract these dynamics to create more inclusive spaces. By doing so, we will enhance opportunities for peer feedback and cooperative learning experiences centred on student peer review."

Self-assessment itself is not sufficient for students since they lack this skill. Yet students have ideas for learning a skill (e.g. language, math, history). Therefore, it is essential for students to first recognize the characteristics of an effective language learner. Once they have identified the requirements for successful learning, they can evaluate themselves based on their established criteria. This process will enhance their metacognitive awareness and promote a sense of learning responsibility and self-regulation.

There are several tools and assessments available for measuring information literacy skills in a practical and objective way. Here are a few examples:

Hence, before selecting a tool or assessment method, it's important to consider your specific assessment goals, target audience, and available resources. Additionally, you may want to explore whether any existing tools align closely with your needs or if customization is required.

Much like self criticism practiced by the Communist party. https://en.wikipedia.org/wiki/Criticism_and_self-criticism_(Marxism%E2%80%93Leninism)

This served to enforce the DEI ideology on the entire organization - esp. those non-acceptors unwilling to voice their positions..

Here are some discussions about ResearchGate in general:

The new age of education has brought about significant changes in the identity and role of teachers. Previously, the perception of teachers was often associated with devoted and monk-like personalities who followed established dogma without requiring advanced pedagogical knowledge or social charisma. However, the landscape of education has shifted, and these traditional notions of teacher identity have evolved.

Embracing new concepts and technologies:

Teachers in the modern era are expected to engage with concepts such as self-assessment, learning communities, digital technologies, hybrid learning and teaching, inclusion, self-management of the school unit, tele-education, and distance education. These terms not only enrich the educational vocabulary but also shape a new educational environment and redefine the identity of teachers.

Adopting a learner-centered approach:

With the emergence of self-assessment and learning communities, the focus has shifted towards learner-centered approaches. Teachers now play a role in facilitating student reflection and self-assessment, empowering students to take ownership of their learning process, and fostering collaborative learning environments.

Integrating digital technologies:

The prevalence of digital technologies has transformed the way teaching and learning take place. Teachers are now expected to incorporate digital tools and resources into their instruction, providing students with blended learning experiences that combine traditional classroom teaching with online learning tools . This integration of technology requires teachers to have a certain level of technological proficiency and adaptability.

Facilitating inclusive education:

Inclusion has become a key aspect of modern education, and teachers are tasked with creating inclusive learning environments where all students, regardless of their abilities or backgrounds, can thrive. This requires teachers to have a deep understanding of diverse learning needs, implement differentiated instruction, and provide appropriate support for every student.

Navigating new educational models:

The advent of hybrid learning and tele-education has introduced new models for teaching and learning. Teachers now have the opportunity to navigate these flexible approaches, conducting classes that incorporate both in-person and online components, or even delivering education remotely through tele-education. This necessitates a willingness to adapt and innovate within these evolving educational contexts.

The new age of education has redefined teacher identity. Teachers are expected to be knowledgeable, adaptable, technologically proficient, and skilled in fostering inclusive learning environments. The expanding educational vocabulary and changing educational models have transformed the way teachers approach their roles, embracing new opportunities for effective teaching and learning.

wow , let me take a look on that .

Sincere thanks Zhi Yang

Certainly the analysis of the correlation is fundamental, regardless of the type of indicators chosen.

Possible tests come later.

However, in the University of Tartu there will be someone who deals with the subject, I imagine.

Oskar Sierka If the differences between pairs of data are not normally distributed, the Wilcoxon signed-rank test should be employed. The Wilcoxon test compares two matched groups and has two variants: the rank-sum test and the signed-rank test. The test's purpose is to assess if two or more sets of pairings differ from one another in a statistically meaningful way.

Depends upon the structure and culture of the organisations

It is better if you directly request access from the original author. Full text of the manuscript you are seeking is available for download on researchgate ( ).

Here is the list of items you are looking for:

1. I often neglect my personal needs because of the demands of my work.

2. My personal life suffers because of my work.

3. I have to miss out on important personal activities because of my work.

4. I come home from work too tired to do things I would like to do.

5. My job makes it difficult to maintain the kind of personal life I would like.

6. My personal life drains me of the energy I need to do my job.

7. My work suffers because of everything going on in my personal life.

8. I am too tired to be effective at work because of things I have going on in my personal life.

9. When I am at work, I worry about things I need to do outside of work.

10. I have difficulty getting my work done because I am preoccupied with personal matters.

11. My job gives me energy to pursue activities outside of work that are important to me.

12. Because of my job, I am in a better mood at home.

13. I am in a better mood at work because of everything I have going for me in my personal life.

14. My personal life gives me the energy to do my job.

15. I struggle with trying to juggle both my work and nonwork responsibilities.

16. I feel overwhelmed when I try to balance my work and personal life.

17. I have difficulty scheduling vacation time because of my workload.

18. I am unable to relax at home because I am preoccupied with my work.

19. I am happy with the time I spend doing activities not related to work.

20. I often have to make difficult choices between my work and my personal life.

21. I am able to accomplish what I would like in both my personal and work lives.

22. I feel that I allocate appropriate amounts of time to both work and nonwork activities.

23. I make personal sacrifices to get work done.

24. I have to put aspects of my personal life “on hold” because of my work.

All the best.

Thank you, Stefano! I'm well, thank you. I hope you are doing fine as well.

Thanks for your suggestion, I will look into it, even though my issues was much smaller :)

Thank you though!

Nurses must be observed from time to time to see the extent of commitment to implementing the guidelines. Also, role modes of practice for identifying knowledge development.

You are welcome! Let me know if you need any help!

Regards,

Al

Items that are loaded with affect (emotions) such as satisfaction tends to be highly subjectivea s you might want to compare error rates and and power when analysing the Likert scale. As you are attempting to quantify perceived efficacy and the numbered measures are ordinal ,this might have some statistical violation issues.You may want to use visual analog scales (VAS) instead. VAS measures a characteristic/ attitude across a continuum of values which cannot easily be directly measured, which can give you better and accurate results.Caution should be exercised with regards to statistical power and type 1/2 errors.Good luck.

José-Ignacio Antón

Thank your for the article! I'm reading it right now, hopefully it can help with my question, and otherwise it made me wiser in general (:

Bonjour Julie,

j'ai utilisé deux échelles de mesure de la communication dans mes travaux sur la confiance. Une comme antécédent et l'autre (com confidentielle) comme conséquence.

Akrout, H., & Diallo, M. F. (2017). Fundamental transformations of trust and its drivers: A multi-stage approach of business-to-business relationships. Industrial Marketing Management, 66, 159-171.

Hop it helps !

Best wishes,

Houcine

https://thebestschools.org/magazine/eliminating-grading-system-college-pros-cons/

Please go to this useful link:

http://www.cnbc.pt/jpmatos/29.%20Bradley.pdf

Dear Arcady Putilov, Survey completed

Use Metacognitive Awareness Inventory(MAI).

PACT IPE Assessment Tools; https://nexusipe.org/advancing/assessment-evaluation/performance-assessment-communication-and-teamwork-tools-set-pact

Yes I am partially but not completely satisfied. Many clinical, academic, and administrative duties have limited the time I would like to spend doing research.

Thank you Maria. I will look into it during the weekend.

Anita Singh Thank you.. But the link direct me to this address http://www.iinet.com/index.php

Here is someone else using a self-assessment instrument that might be helpful to you - https://www.surveymonkey.com/r/servantslead

And here's a useful chapter on the topic - http://www.irma-international.org/viewtitle/20257/

Bill

Hi William,

I personally have not done a check on participant accuracy or reliability of walking speed or sit/stand tasks. However, some of my students and I encountered a lot of literature on accuracy of self-report of weight and height and noted that: (a) while self-reported weight is usually biased, it tends to correlate well (.90) with actual weight. By sex, women are more likely to bias weight reports downward, while men are more likely to bias weight reports upward.

Good luck with your work!

Thank you very much Prof. Azer for your kind concern to my query.

I will search more and more literatures

Hello Alexander,

Yes, it is preferable to present evidence for score reliability and validity (for the intended use of scores) for any new measure. Doing so helps justify the use of your measure(s) in your own study, as well as to guide other researchers who might wish to use or adapt your measure(s).

In this case of your study, who's to know whether the rational groupings you discuss at the end of your question will correspond to empirically derived groupings of the items?

Good luck with your work.

As others have already articulated, there are many methods already out there:

Afasahi asked this question: https://www.researchgate.net/post/How_do_you_evaluate_your_class_after_teaching

Which may also help.

I suppose it is also a question of what you are seeking to evaluate? (Engagement, Effectiveness, Pace etc...) as each are likely to be assessed and evaluated differently

Thanks Philip! This is so helpful. All the tools in the same spot!

Hello, most insights concerning the relationship between noncognitive qualities and educational or work-related outcomes stem from research conducted with questionnaires. Self-assessments usually ask individuals to describe themselves by answering a series of standardized questions. The answer format is usually a Likert-type rating scale (for example, in a range from Strongly Disagree to Strongly Agree) but other formats may also be used (such as Yes-No or open answer). Typically, questions assessing the same construct are aggregated and this aggregated score serves as an indicator of the relevant area of science.

My colleagues and I in Australia are developing a Death Literacy Index based on the public health palliative care approach.

Please see the following link

Best Regards Ali Mohammed Yaseen Al Hashimi

EU information

There is a shift towards technology-enhanced assessment that allows student self-assessment to catch errors quickly and correct these before hand to improve final performance. The following manuscript sheds more light on the latter:

Best regards,

Debra

I do believe that u are searching something like this:

- DOI: 10.1891/1061-3749.24.1.5

Hope to have helped

It depends on characteristics and objective of the research. Please review the synopsis with effective appropriate statics tool, so the result can explain the findings of objective.

The questionnaire must be ask in such a way, so participant can response easily. further must easy in crux the data from questionnaire. I hope that you have been read various book on this and my opinions are following:

1. Read the questionnaire frequently as much as you and colleagues can do and ask opinion too.

2. Ask and meet with expert and follow their suggestion.

3. before final touch of questionnaires, Do and conclude with pilot study.

Thanks for the recommendation. I will look into it

Dear Nate,

I think if your sample size is small and you have choose the paradigm of pre and post where the sample is same, then you have to go with paired t-test because this is more suitable and authentic test for analysis and apart from this you may be multivariate analysis.

Dear Michael:

The e-portfolio can be a challenge if there are no set clear guidelines and rubrics for students to follow. Here's a dissertation study that would give more in-depth insights into the answer to your question:

Hi Muhammad,

I've looked over what others have provided and hopefully add to what you now have. 

In the broadest sense you can approach self-report scale development and validation from two approaches; exploratory or confirmatory.

If the scale you are developing is based on well established/grounded theory then you're best to go with a confirmatory study using a Structural Equation Modelling approach such as Confirmatory Factor Analysis (CFA). Basically you create your scale items in reference to the theoretical construct you wish to measure. You would want to have your scale looked over by a panel of "experts" to check for face/content validity, maybe run a pilot study, and then deploy your scale. You can analyse the data using AMOS or SAS. AMOS has a nice visual GUI interface, but SAS handles coding better (that's my opinion). In CFA you set your model (theory) up in advanced ( a priori) and try and "fit" the data to your predefined model. Your data either fits or it doesn't, although there is some room for debate on that, but not much. So the stakes are high if you come up with nothing. If your data fits you have establish evidence that your theoretical construct has quantifiable psychometric properties. You are also providing evidence for the grounded theory. This is interesting work if you are passionate about scale design, and Structural Equation Modelling.

If you are not starting from grounded theory, you should approach your scale design using Exploratory Factor Analysis. Which is a lot easier than CFA, but you are not likely to establish any evidence for construct validity. If you find something worthwhile in your EFA study, then you would run multiple studies (at least one) to formulate your grounded theory, and the run a CFA. 

Just a word of caution on before you start planning, you CAN'T run both CFA and EFA in the same study. To do so is not rigorous scholarship and unethical in my opinion.

On my profile you can find the references to some of the scales I have worked on. Have a look at Sun's  Educational Stress Scale for Adolescents. There are several articles that describe the scale development from start to multinational/multilingual normative studies. It is a good example of EFA and CFA for scale development on the same scale in different studies, and starting from a less grounded theoretical, construct.

You can also look at my Attitudes Towards Acculturative Behaviour Scale for an example of scale development from grounded theory using CFA. I have provided a link to a dissertation with this scale that you could use a guide or example of what's involved and with plenty of detail. 

I hope this helps, and please read what my colleagues above have provided, as it will give you the best insight into what is involved and how committed you will need to be. My golden rule is that if there is already a scale to measure what you want to, use that first! There's no need to reinvent the wheel.

"The higher education literature testifies to an extensive interest in self and peer assessment. The interest in self and peer assessment is partly driven by changing conceptions of teaching and learning which emphasize the active engagement of students in their own learning, learner responsibility, metacognitive skills and a dialogical, collaborative model of teaching and learning. Assessment processes in which the teacher holds all the power and makes all the choices limit the potential for learner development in all of these aspects..."

The foundation of self assessment is inherent in eduction: eventually we want a learner to do without help, to be autonomous in a certain area. To have learned something consciously (i.e. not by covert indoctrination or sheer habit) is equal to having mastered and understood the required criteria. Learners needs self evaluation to be able to decide when they are ready. 

The foundation for peer assessment is inherent in learning to co-operate: a group needs mutual feed back to optimize working together. Peer assessment is a means for learning how to address feed back in a friendly and effective way. 

Dear Amna,

You obviously want to analyse the relationship between the scores on the two questionnaires. I would suggest adding 1 to your negative observed behaviour scores and subtracting 1 from your observed positive behaviour scores to create a single scale from -3 to +3 with 0 as neutral.

You could look at the relationship between the scores for the individual behaviours but there are a lot of them. This means you run the risk of making Type I errors and the data itself is rather coarse, so even if you treat the data as ordinal (which it is) a nonparametric correlation (e.g. Spearman) is still not advisable because there are too few values, leading you into the Chi-square test which would not explain the relationships adequately.

An alternative would be a dimension reduction approach such as exploratory factor analysis. This would reduce the number of variables and also allow you to treat the derived variables as scale rather than ordinal. You could then investigate the scatter plots and run an appropriate correlation with each pair of results. Has your questionnaire already been validated?

Sorry am unable to get it as well

Adam, my team's assessments are all Rasch-based, and Computer-Adaptive, so they will allow you to determine how short and precise you wish them to be.  For Collaborative leadership, we have a scale for Teamwork based on my own dissertation and the much more recent work from Eduardo Salas.

Our assessments are mobile-centric (Android, iOS) for self-assessments, and for multisource (360), use the web.  Feel free to reach out if you'd like to explain how you're conceptualizing collaborative leadership, antecedents, and the science that underlies our approach.

Juha Kettunan, thank you very much. I think this paper is really useful for my research.

Thank you Helena for sharing information about CAF model, the 2013 version does not contain self assessment questionnaire.

@ Ani : As an afterthought I have to say, that the 20 criteria/questions on the questionnaire are still very coarse: they only point to aspects or locations in a report which are relevant, important and frequently misunderstood. They don't tell the student yet, what precisely wen't wrong in his or her case (if anything went wrong indeed).

Thus I used the actual results of the first group of about 15 students to collect a much longer list of about 80 very specific violations of the technical writing guidelines and recommendations. That's indeed a long list!

I gave this list to the other two groups of the same size. And see: suddenly the results were much better, even after adjusting (i.e., lowering) my assessment tolerance, because these groups had a definite advantage over the first group, which acted as the source of  this longer list.

What does that tell us? It reconfirms a well-known truth about guidelines: they are less useful during application as many people think or hope, because (most) guidelines only tell the newcomer in a general way what to do; they don't usually list all the ways, in which application of a specific guideline can go wrong, for whatever reason.

Furthermore, in the case of rather simple guidelines (from my point of view), it tells me, that many students have difficulties to "decode" the meaning inherent in the heavily "encoded" guidelines: the way back from abstract to concrete is much more difficult, mostly, than the original forward way from concrete to abstract. In the latter case the expert leaves out certain "irrelevant" details, in the former case the novice has to intuite them again, which is exactly what we want him or her to learn in the first place.

Dear Peter

Holmes and Rahe stress scale is a simple measure that lists 43 stressful life events and can be completed on line.  I like this scale and find it useful. 

Not necessarily as these Agencies do not always take into consideration the culture of the immediate society/institution. I know from experience in the Caribbean, some internationally funded projects at some institutions failed to meet accreditation standards and were not accredited by the International Agencies. In one instance that involved setting up a garment construction training institution with a workforce component, whereby trainees can work and study, the international accreditation standards required that the trainees/employees be searched everyday on leaving/entering the compound. Well, students were not use to such measures and did not subject themselves to these searches. The latter affected the success of the project negatively.

Best regards,

Debra

That's great, thanks for the advice.

I will do my best to describe what I do! I issue my students with cards at the beginning of each class, they are instructed to keep all comments anonymous and I provide a box in class for them to post their cards in. At the end of each session I collect the cards, read them, and type up a summary. I then make some brief notes (typed) of what I think are the most poignant pieces to take forward. I issue the summaries and my response notes to my students prior to next class. Often their and my responses are classroom pedagogy related - sometimes with implications for me as a teacher, but more often than not they are relevant to a broader pedagogical issue (for example, equitable  assessment strategies). I then use the synthesized notes and my responses for two main purposes, (1) to get feedback upon, inform and develop my own practice, and (2) as ideas for further discussion of issues in class through class debate, or through a case study etc., so in this way they help to shape the content of future lessons. I always give students a prompt before they complete the exit cards so that we can "stay on topic" to some extent, as I am mindful of the need to complete what is laid down in curriculum. But in the few years I have been using them, I have found that by sharing my own reflections and ways I am adapting/changing/reinforcing my own practice really helps teacher candidates to see how this can be helpful to their own developing practice. It usually instigates a deeper level of thinking about what it means to teach, and a commitment to continuous development. I guess that in simple terms  I model that it is OK to keep on learning and developing!

Dear all, thank you for your replies to my question. In the meantime, I have searched in the literature for relevant evidence on the possibility of counselors/therapists to assess their own MI skills. I have found relevant information in an article of Miller et al (2001/2004). They suggest that counselors' self-report of their MI skills is inaccurate and unrelated to ratings by skilled coders. 

Thank you so much for your recommendations and for taking the time to respond! Have a great day! Jennifer :-)

In the research we did at Infosys, we addressed the issue of what construct to measure by whom, based on the literature and on which stakeholder (e.g. self, manager, peer, mentor, client) had the best opportunities to observe that facet of leadership.  As for literature, Hogan's Socioanalytic theory has been supported in three meta-analyses showing that asking for others' views of your personality is more predictive than self-report.  But for self-regulatory constructs like identity, the leader herself/himself is likely the best judge.

I'm fond of Linacre's Many Facet Rasch Model to adjust for biases with modern psychometrics.

An outcome is a behavior or measurement that can be objectively measured after an intervention including education.  I added a link that describes outcome research in health care.  Much of evaluation in education, especially continuing ed, has been about process.  Did the instructor follow the objectives?  Were there adequate handouts?  etc.  They were more happiness indexes. 

Outcomes research focuses on achievement rather than process.   If someone attends a workshop of any kind, can they now do something different or better? 

If you want to send me what content area you deal with I could give you specific examples. 

Hope that helps.

You might want to ask Laura King at the University of Missouri. Her and her colleagues are studying insight, and they might know the answer to this question, or have a similar measure that they can point you toward.

Dear colleaugues,

There are an enormous number of instruments and questionnaires available to assess all type of patient variables. The total number is now estimated to be between 500 and 600 in the field of musculoskeletal disorders alone. For pratical purposes, a set of 10 - 15 instruments / questionnaires are enough to describe about 85% of all patients with musculoskeletal disorders. The first issue raised is how to choose a clinimetric index. The second concerns the methodological or psychometric properties of the indexes.

The European Research Group on Health Outcome (ERGHO) has published a statement entitled 'Choosing a health outcome measurement outcome which is applicable to all clinimetrical indexes'. In chossing a measurement instrument (for example an evaluative instrument) to be asked is 'What is the aim (construct) of the index.

The International Classification of Functioning, Disability and Health (ICF) is a good starting point for the first choice of outcome measures and measurement instruments in clinical situatuions, particurly because diagnostic assessment by physical therapist is concerned mainly with functional disorders such as low back pain.

Based on the ICF and the clinimetric properties, we have developed a 5-step model in selecting the appropriate instruments and / or questionnaires. See reference 1.

Based on this model, I give you an overview of the evaluative instruments and questionnaires we have used in different type of studies (case study, observational studies and randomised controlled studies):

. pain: visual analogue scale (VAS), Numeric Pain Rating Scale (NPRS), McGill Pain Questionnaire (MPQ) and pain drawing;

. activities and partiipation: Roland-Morris disability questionnaire (RDQ) and Quebec Back Pain Disability Scale (QBPDS),;

. personal factors: acute low back pain screening questionnaire, chronic pain grade questionnaire, Tampa Scale Kinesiophobia (TSK), Pain Coping Inventory (PCI) and Short Form 36 (SF-36 [general health]).

My answer on your question 'How many self-reported questionnaires are available (currently being used by clinicians and researchers) to assess non-specific lower back pain?' is 11 instruments or questionnaires.

I wish you and qall the colleaugues succes and best greetings from The Netherlands,]

Rob Oostendorp, PhD, MScPT, MPT.

Reference

Swinkels RAHM, Oostendorp RAB. Outcomes assessment and measurement in spinal musculoskeletal disorders. In: Grieve's Modern Manual Therapy. The Vertebral Column (edited by: Boyling, JD, Jull GA). Edinburgh, Churchill Livingstone, 2004.