Article

The Convergent and Discriminant Validity of NSSE Scalelet Scores

Authors:
To read the full-text of this research, you can request a copy directly from the author.

Abstract

Faculty and administrators are more likely to take responsibility for student learning and development if they believe that assessment data represent their students and identify specific actions for improvement. An earlier study found that NSSE scalelets provide dependable metrics for assessing student engagement at the university, college, and department levels. Building on the earlier study, the findings of the current research indicate that the NSSE scalelets have greater explanatory power and provide richer detail than the NSSE benchmarks.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the author.

... were selected from institutions that were in both data sets and had more than 50 first year students responded. The model's scalelets developed by Pike (2006aPike ( , 2006b were used for the 2011 analysis and the EIs were used for 2013 analysis. According to Pike (2006aPike ( , 2006b, scalelets consist of a small number of survey items that are related and represent students' educational experiences. ...
... The model's scalelets developed by Pike (2006aPike ( , 2006b were used for the 2011 analysis and the EIs were used for 2013 analysis. According to Pike (2006aPike ( , 2006b, scalelets consist of a small number of survey items that are related and represent students' educational experiences. Scalelets can be used to focus on specific measures while reducing the amount of time it takes for students to respond to a survey. ...
... Pike developed 12 scalelets from items on the original NSSE. He found evidence that the scalelets had both convergent and discriminant validity (Pike, 2006a). This finding suggested that the scalelets were more nuanced than the benchmarks. ...
... According to Kuh (2001) college rankings became a component of pop culture in the last decade of the twentieth century. However, many administrators and faculty at these individual institutions found the rankings and specifically their methodologies to be lacking in rigor and robustness (Kuh, 2001;McClenney, 2007;Pike, 2006). This is mainly because they are considered to be, "too general…that is, the results do not suggest specific courses of action" (Pike, 2006, p. 550). ...
... A component of a successful assessment system is an active and engaged faculty that serves to also systematically gather data on student mastery of learning objectives (Kuh & Ikenberry, 2009;Laird, Niskode-Dossett, & Kuh, 2009;Pike, 2006;Wang & Hurley, 2012). Faculty members understanding of the importance and relevance of assessment is clearly fundamental to successful educational leadership and effective teaching. ...
... However, the study they conducted concluded that motivation was the pivotal factor in faculty willingness to incorporate assessment. As a result, many faculty fight the implementation of assessment strategies within the administration of their individual institutions (Astin, 1993;Bok, 2006;Laird, Niskode-Dossett, & Kuh, 2009;Pike, 2006). In fact, the lack of assessment is so widespread that many departments don't even require graduating students to have completed a substantial research paper prior to completing classwork (Bok, 2006;Laird, Niskode-Dossett, & Kuh, 2009). ...
Thesis
The State of Texas started an educational endeavour called the Texas Common Curriculum in 1987 which was structured to ascertain and require that all graduates from Texas institutions of higher learning, who received a baccalaureate degree from a public college or university, would be required to successfully finish the state’s general education Core curriculum within the components of their specific degree requirements. Over time changes were made to the curriculum initially on an institutional level and then on a state level. Designed, almost wholly, to allow for easy and free transferability between accredited institutions in Texas the Core curriculum has yet to actually satisfy this most basic of its foundational missions. The history of this process for educational reform mirrors changing times in the state, and nation. This is a descriptive study.
... Thus, looking at the findings of Q 1 of the questionnaire, all respondents across both samples are members of the Net Generation according to the definition in [5]. Furthermore, a list of characteristics of the Net Generation can be found in [23]. With regards to being digitally literate, connected and social, the findings confirmed that all respondents across both samples described themselves as being digitally literate [25]. ...
... With respect to teamwork most of the students across both samples answered that they did not prefer to learn and work in teams [25]. This is an interesting finding as it is contrary to the literature reviewed and is consistent amongst both samples of students surveyed [23]. A possible reason for this could be the student's field of study. ...
... Lastly, in terms of engagement and experience, visual and kinesthetic and things that matter, the findings indicated that across both samples, respondents adhered to the characteristics noted in [23]. ...
Conference Paper
Full-text available
This paper aims to determine the effect on the engagement of ‘Net Generation’ students from using ‘YouTube’ in the classroom. The education and engagement of the Net Generation students are a growing challenge among institutions of higher learning. Net Generation arrival overlaps with the advent of digital technology. Thus, this explains why the students have dissimilar styles of learning due to their comfort with and use of digital technology. Literature on educating and engaging the Net Generation asserts incorporating the Web 2.0 elements; YouTube inside and/or outside classroom. The target sample includes Net Generation students in their 2nd-3rd year at the University of Cape Town in the Commerce faculty enrolled in an Information Systems course. The research instrument was a questionnaire. Two samples were included, the first consisted of students currently enrolled in the course and the other, students previously enrolled in the course. The result shows that the use of YouTube had a positive effect on the engagement of Net Generation students.
... Pike's scalelets measured deep approaches to learning, satisfaction, gains in general learning, in-class activities, and aspects of the campus environment. Pike (2006aPike ( , 2006b) developed these scalelets to respond to the need to have survey data that were more specific at the department level rather than at the university level. On the basis of his findings, he argued that the NSSE scalelets produced dependable group means and provided richer detail than the NSSE benchmarks, which he believed would lead to information for improvement at the program level (Pike, 2006a). ...
... Pike (2006aPike ( , 2006b) developed these scalelets to respond to the need to have survey data that were more specific at the department level rather than at the university level. On the basis of his findings, he argued that the NSSE scalelets produced dependable group means and provided richer detail than the NSSE benchmarks, which he believed would lead to information for improvement at the program level (Pike, 2006a). The SPSS syntax for determining these scalelets can be found at http://nsse.indiana.edu/ ...
... Using the SPSS syntax developed by Pike (2006a), 15 questions from the 2011 NSSE were used to create five scalelets for the 1st-year sample and five scalelets for the 4th-year sample: general learning outcomes (four items), overall satisfaction (two items), emphasis on diversity (three items), support for student success (three items), and interpersonal environment (three items) (see Appendix A). Scalelets for each student respondent were created by first transforming all related items to a common scale ranging from 0 to 100 and then summing the transformed ratings across the selected items using an unweighted approach, which means that each survey item used contributed equally in creating the overall scalelet (Comrey & Lee, 1992). ...
Article
Full-text available
Aggregate survey responses collected from students are commonly used by universities to compare effective educational practices across program majors, and to make high-stakes decisions about the effectiveness of programs. Yet if there is too much heterogeneity among student responses within programs, the program-level averages may not appropriately represent student-level outcomes, and any decisions made based on these averages may be erroneous. Findings revealed that survey items regarding students’ perceived general learning outcomes could be appropriately aggregated to the program level for 4th-year students in the study but not for 1st-year students. Survey items concerning the learning environment were not valid for either group when aggregated to the program level. This study demonstrates the importance of considering the multilevel nature of survey results and determining the multilevel validity of program-level interpretations prior to making any conclusions based on aggregate student responses. Implications for institutional effectiveness research are discussed.
... Active engagement has been found to maximize learning and enhance personal development in the classroom (Ahlfeldt, Mehta and Sellnow 2005;Carini, Kuh and Klein 2006;Chung et al. 2009;LaNasa, Cabrera and Transgrud 2009;Pike 2006a;Preszler 2009;Elder et al. 2011). Students reported a better understanding of course concepts and displayed greater active participation, when actively engaged (Ahlfeldt, Mehta and Sellnow 2005;Chung et al. 2009;Preszler 2009). ...
... Although institutional surveys -such as the NSSE -provide a variety of data to participating institutions, barriers can arise when attempting to use the data to make changes, as the results are often too general (Pike 2006a). Pike proposed that departments and colleges use scalelets to select areas of measure that are pertinent to that department or college. ...
... Pike proposed that departments and colleges use scalelets to select areas of measure that are pertinent to that department or college. Specifically, a scalelet is a set of the NSSE survey questions that relate to a specific aspect of the educational experience (Pike 2006a). Pike created 12 scalelets based upon a selection of questions from the original NSSE survey, and reported that all 12 scalelets produced dependable group means (Pike 2006b). ...
Article
Full-text available
Active engagement has been found to maximize learning, enhance personal development, and enrich communication in the classroom. This study examined how participants' perceptions of engagement – measured by questions from the National Survey of Student Engagement (NSSE)-changed in response to three problem-based learning (PBL) activities; peer-led workshops, service learning, and journal clubs. Sixty-two undergraduate students enrolled in an introductory class to a career in physical therapy were recruited for three consecutive semesters. Participants were administered a shortened version of the NSSE at the beginning and end of their semester of class. Two NSSE benchmarks-active and collaborative learning and supportive campus environment – were perceived by participants as improved, partially supporting the hypothesis. Participants reported increased engagement in six scalelets (specific subsets of educational experiences)-course challenge, writing, active learning, collaborative learning, diversity, and interpersonal environment. Responses to several scalelets, however, showed a decline in engagement-higher order thinking and information technology. This is the first study to look at changes in NSSE scores across only one semester of class. The study supports PBL, however, recognizes that class design, composition, and instructor may impact student perceptions.
... In addition, a search of Google Scholar identified more than 7000 articles and conference presentations dealing with NSSE. Although the appropriateness of NSSE for student-level assessment is a topic of considerable debate (see McCormick and McClenney 2012;Porter 2011), several studies have documented the reliability and validity of using NSSE scores for institution-and group-level assessment and research (Kuh 2001;Kuh et al. 2001Kuh et al. , 2007Ouimet et al. 2004;Pascarella et al. 2009;Pike 2006aPike , b, 2013b. In addition, a variety of institutions have reported using NSSE data to make substantial improvements in student engagement and learning outcomes (Banta et al. 2009;Kuh 2005; National Survey of Student Engagement 2009). ...
... Pike (2006a, b) found that the NSSE scalelets produced very dependable measures of engagement and were differentially related to self-reports of student learning outcomes. Most important, the results reported by Pike (2006a) were consistent with student development theory. ...
... NSSE's conceptual framework is based on Pace's (1984) deceptively simple premise that students learn from what they do, coupled with Kuh's (2003) extension that colleges and universities influence what students do based on the activities and experiences they make available to students. As previously noted, there is ample evidence to demonstrate that the activities in which students are engaged during college are related to their learning outcomes, based on objective and subjective measures of learning (Anderson et al. 2015;Carini et al. 2006;Gellin 2003;Pascarella et al. 2009;Pike 2006a;Pike et al. 2012). ...
Article
Full-text available
The present study examined the relationships between student engagement, represented by two versions of the National Survey of Student Engagement (NSSE) and self-reported gains in learning. The study drew on institutional-level data from participating institutions in 2011 and 2013. The objective of the research was to compare evidence of convergence and discrimination for the two versions of NSSE using canonical correlation analysis. Results indicated that both versions of NSSE provided clear evidence of convergence in that student engagement measures were significantly and positively related to perceived gains in learning. However, only the most recent version of NSSE provided strong evidence of discrimination (i.e., differential relationships between engagement measures and self-reported learning outcomes). Thus, the revised NSSE appears to offer substantial advantages for institutions interested in more nuanced understandings of the relationships between student engagement and perceived learning outcomes. Implications for educators, with goals of enhancing student learning, and for researchers, who often compare complex sets of data, are included.
... Studies show that engagement is positively related to test scores and students' reports of learning (Gellin, 2003;Kuh, Hu, & Vesper, 2000;Pike, Kuh, & Gonyea, 2003) and that institutional actions influence levels of engagement and learning on campus (Kuh, Schuh, Whitt, & Associates, 1991). Moreover, different types of engagement have been found to be differentially related to learning outcomes (Pike, 2006). ...
... Items were selected from the COS that measured students' time allocations and engagement in various activities reflecting benchmarks of effective educational practices, in addition to valued outcomes like students' growth, satisfaction, and their recommendation of the University. Such an approach of developing institutional scalelets was recommended by Pike (2006) for institutions not participating in NSSE but is using other instruments and who wanted to document impact on students' development and learning. Appendix provides list of COS items used to answer research questions. ...
... Students' experiencing academic challenge (given opportunity to develop thinking, writing, application skills, etc) followed next as a contributor to students' growth and word of mouth recommendation. This confirms research findings of Pike (2006) that different types of student engagement have been found to be differently related to learning outcomes. For example, greater involvement with writing was positively related to gains in general education and in writing, and experience with information technology was positively related to gains in practical skills. ...
Article
Full-text available
It is of great significance to an institution of higher education to meet its goals and to establish its institutional effectiveness and that it has a framework for discussing its institutional performance results, accordingly this study aims to investigate a) the conditions for student success at the University with respect to the five benchmarks of effective educational practices (Kuh, 2009); b) significant differences in conditions for student success across important student populations (gender, GPA, number of credits completed, and academic year); and c) how do these conditions contribute to outcomes valued by the institution (students’ growth, satisfaction, and recommendation of the University). Responses of 1853 students’ on the College Outcomes Survey (COS) for years 2007-2010 were used to answer the various research questions of the study. COS Items were selected that measured students’ time allocations and engagement in various activities reflecting effective educational practices, in addition to valued outcomes. Data analysis involved first testing the measurement model and estimating overall fit of the data using confirmatory factor analysis (CFA). Descriptive statistics, and correlations were reported for the benchmarks of effective educational practices, and differences in benchmark experiences by subgroup were investigated. Finally, a structural model tested the influence of benchmarks of academic practice on valued outcomes and a regression was conducted to investigate relationship between student activities and the benchmarks. Results revealed good fit of the data for the model, identified University’s performance on benchmarks of effective educational practices and their relationship to outcomes valued by the University. Implications for practice were discussed.
... The psychometric self-report instrument of NSSE (2009) is an example of a single-dimensional measurement of student engagement, increasingly used by the universities worldwide over the last two decades. In brief, the NSSE measures the extent to which students are engaged in educational practices, and how this contributes to learning and personal development (Kuh, 2002;Pike, 2006). The NSSE (2014) comprises four themes/benchmarks of engagement (formerly five) representing ten engagement indicators that emerged from "a blend of theory and empirical analysis" [Kuh, (2003), p.30]. ...
... Moreover, the NSSE benchmarks do not include engagement items of abstract/latent measures such as students' individual thoughts, meta-cognitive understanding, perception of self-identity and autonomy, motivation or feelings towards education institution, peers, or learning environment. The lists of variables that constitute engagement concept theoretically expand beyond those that were used to create the NSSE (Pike, 2006). Burch et al. (2015) claim that the NSSE was developed to compare universities to one another, and therefore, aggregates student engagement to the college/university level, thereby making it impossible to investigate course/class level engagement. ...
Article
This study examines the relationship between student engagementand academic attainment. A random sample of 548 full time second-year andthird-year undergraduate students from six higher education institutions inAzerbaijan was obtained. The st udy methodologically compares andmerges two psychometric self-report measures, the National Survey of StudentEngagement (NSSE) and the Burch Engagement Survey for Students (BESS),which capture three fundamentally different dimensio ns of engagement,namely behavioural, cognitive and emot ional. The data we re analysed usingconfirmatory factor analysis (CFA) and multiple linear regression. The findingsof the study support the multidimensionality notion of engagement assuggested by the prominent student invo lvement theories and claim that allthree dimensions are to a different extent statistically significant determinantsof academic success. Regression results reveal significant relationshipsbetween students’ academic ach ievement and engagement both forone-dimensional and three-dimensional engagement models. Compared to Page 2 The impact of behavioural, cognitive and emotional dimensions 11traditionally used measurements, the three-dimensional CFA model providesbetter explanation of academic outcomes. The results of this study could haveimplications for university academic staff and policymakers in utilising thebenefits of each engagement factor to improve institutional quality. The impact of behavioural, cognitive and emotional dimensions of student engagement on student learning: the case of Azerbaijani higher education institutions.
... The psychometric self-report instrument of NSSE (2009) is an example of a single-dimensional measurement of student engagement, increasingly used by the universities worldwide over the last two decades. In brief, the NSSE measures the extent to which students are engaged in educational practices, and how this contributes to learning and personal development (Kuh, 2002;Pike, 2006). The NSSE (2014) comprises four themes/benchmarks of engagement (formerly five) representing ten engagement indicators that emerged from "a blend of theory and empirical analysis" [Kuh, (2003), p.30]. ...
... Moreover, the NSSE benchmarks do not include engagement items of abstract/latent measures such as students' individual thoughts, meta-cognitive understanding, perception of self-identity and autonomy, motivation or feelings towards education institution, peers, or learning environment. The lists of variables that constitute engagement concept theoretically expand beyond those that were used to create the NSSE (Pike, 2006). Burch et al. (2015) claim that the NSSE was developed to compare universities to one another, and therefore, aggregates student engagement to the college/university level, thereby making it impossible to investigate course/class level engagement. ...
Article
Full-text available
This study examines the relationship between student engagement and academic attainment. A random sample of 548 full time second-year and third-year undergraduate students from six higher education institutions inAzerbaijan was obtained. The st udy methodologically compares and merges two psychometric self-report measures, the National Survey of StudentEngagement (NSSE) and the Burch Engagement Survey for Students (BESS),which capture three fundamentally different dimensions of engagement, namely behavioural, cognitive and emotional. The data we re analysed using confirmatory factor analysis (CFA) and multiple linear regression. The findingsof the study support the multidimensionality notion of engagement assuggested by the prominent student involvement theories and claim that allthree dimensions are to a different extent statistically significant determinants of academic success. Regression results reveal significant relationships between students’ academic achievement and engagement both for one-dimensional and three-dimensional engagement models. Compared toThe impact of behavioural, cognitive and emotional dimensions 11traditionally used measurements, the three-dimensional CFA model provides better explanation of academic outcomes. The results of this study could have implications for university academic staff and policymakers in utilising the benefits of each engagement factor to improve institutional quality.
... Variables that measured dimensions of student involvement, institutional engagement, and sense of belonging were computed for our analysis. Survey items were initially selected that we hypothesized would capture the essence of our study constructs based on prior literature (Kuh, 2009;Pike, 2006;Strayhorn, 2012;Wolf-Wendel, Ward, & Kinzie, 2009). Principal component analysis with varimax rotation was performed to determine the structure of each variable and to reduce the number of items into a valid set of factors. ...
... The institutional engagement variable (α = 0.89) was operationalized using a composite scale of eight summated NSSE survey items that asked students to rate how much their institution emphasized engagement and involvement. Items were considered that measured student perceptions of institutional emphasis on student engagement and involvement opportunities (NSSE, 2007;Pike, 2006). For example, one item asked students to rate how much their institution emphasized contact among students from different backgrounds. ...
Article
Full-text available
Persistence rates among Black and Latinx students continue to fall behind those of their White peers. One way to address this issue is to promote a stronger college sense of belonging. While student involvement has been linked to a sense of belonging, postsecondary institutions need to seriously reflect on, and recommit to, their own role in engaging Black and Latinx students to promote their sense of belonging, a strong correlate of persistence and college completion.
... The exploratory component is that the analysis focused on the observed variables as a basis for the analysis (Kim, & Mueller, 1979), but at the same time, it has a confirming approach, as it looks at comparing earlier structure patterns as a basis for the proper identification of meaningful clusters of variables (Byrne, 1994). This approach is consistent with Pike's (2006) recommendation for a single institution study to identify a substantive scale that is meaningful for the institution. Although factor analysis is most directly relevant for evaluating a scale's internal structure, it also provides information about a scale's internal consistency, and it can be used to evaluate convergent and discriminant validity evidence. ...
... Likewise, a recent study reported a six-factor revised model fitting so well for a single institution data (Chosang, Steven, & Penny, 2013). Regardless of this, it has been consistently confirmed that the instructional and academic interaction sub-components had relatively higher influence in representing the dimensions of student engagement scale (Krause & Coates, 2008;Kuh, Kinzie, Buckley, & Hayek, 2006;Pike, 2006). This was similarly true for single-institution studies as well (LaNasa et al., 2009;Lutz & Culver, 2010). ...
Article
This paper is positioned to make a sound contribution to knowledge of higher education (HE) quality through its study of student engagement in a large university in Ethiopia. We establish that the assessment of student engagement is a valid facet in determining HE quality, contributing holistically alongside quality assurance and university rankings. The authors present an example of this through reporting on a statistical analysis of an adjustment of the Australasian Survey of Student Engagement survey and its administration in the Ethiopian university. The results of factor analysis illustrate empirical support for the nine-factor engagement scale and a number of associated factors related to student demographics and university experience. The results of the multi-validation approach provide specific guidelines to universities using this approach to evaluate the validity and reliability of this construct. The paper provides a supporting tool that can be used by other universities to assist in the evaluation of students’ engagement with their undergraduate programmes.
... Following the approach by Kuh, Cruce, Shoup, Kinzie and Gonyea (2008) Cronbach"s alpha as .83 in a sample of 11,420 students from 18 colleges. In an assessment of the NSSE scalelet scores, Pike (2006) found moderate to strong associations in the expected direction between scalelet scores and improvement in Daniel Bowen -11354484 academic achievement, indicating convergent validity (r ranged from -.44 to .80, p < .05). ...
... The NSSE assesses students" participation and level of engagement in effective educational practices that are related to positive learning outcomes(Kuh, 2001). The reliability and validity of the CSEQ has been well documented (seeKuh, 2001; Kuh, 2009; Kuh, Kinzie, Cruce, Shoup, & Gonyea, 2006;Pike, 2006).Two items asked respondents to indicate the number of hours spent in a typical 7-day week studying and participating in co-curricular activities(0, 1-5, 6-10, 11-15, 16- 20, 21-25, 26-30 and more than 30). The other 19 items (e.g., "discussed grades or assignments with an instructor") were rated on a 4-point Likert scale ("Very Often", "Often", "Sometimes" and "Never") and equally contributed toward a global measure of student engagement. ...
Thesis
Full-text available
Undergraduates face numerous challenges and pressures in pursuing a degree. This study explored variables and measures which can be used to assess and predict undergraduates’ everyday academic resilience (i.e., academic buoyancy) and confirmed the nature of some key relationships between variables. This provides important insights for researchers, higher education providers and psychological practice with specific directions for policy and praxis. This study primarily aimed to test a model which maps undergraduates’ academic buoyancy. The secondary objective was to assess the reliability and validity of the measures Martin, Marsh and colleagues (2007, 2008a) used to investigate high school students’ academic buoyancy. Undergraduates (n = 236) aged 18 to 55 were recruited via snowball sampling to complete an online questionnaire comprising a battery of measures assessing psychosocial and educational variables. Acceptable (> .70) to excellent (> .90) internal consistency values were found for each measure investigated, while four of the six measures used by Martin, Marsh and colleagues demonstrated strong to very strong convergent validity with other psychometrically validated measures of the same construct. Support for the theoretical model was found using structural equation modelling, highlighting specific variables (parental attachment, self-efficacy, anxiety, control and engagement) which could constitute the basis for effective intervention. Suggestions for psychological and educational policy and praxis are offered for universities and individual students. Directions for future research could include stratified-random sampling, multiple sources of data and a longitudinal design to qualify results and increase generalisability. Research aimed at developing and testing effective applications of these findings is recommended.
... Существует ряд исследований, в которых показано, что участие в научно-исследовательской деятельности положительно влияет на аналитическое и критическое мышление [18,27,28,31,34], повышает академическую успеваемость и снижает долю отчислившихся студентов [21,26,30], позволяет студентам уточнить выбор академической специальности [33,35], способствовать дальнейшему поступлению в аспирантуру [17,24,29,32,33]. ...
Chapter
Full-text available
В разделе рассматривается проблема включения студентов в научную деятельность университета. Представлены результаты анкетирования студентов Института психологии КГУ им. К.Э. Циолковского по вопросам их мотивации и факторов, влияющих на участие в научной работе. Выявлены ключевые мотивы, подчеркнута связь между вовлеченностью в научную деятельность и опытом неформальных исследований в школьные годы. Представлены результаты самооценки научных компетенций студентов выпускного курса. Выделены группы научно-исследовательских компетенций: содержательные и технико-оформительские, показана разница в самооценке технических и аналитических навыков. Установлена связь общей самооценки готовности к самостоятельному выполнения выпускной квалификационной работы с такими компетенциями как выполнение теоретического анализа и выбор адекватных методов исследования. Предложены рекомендации по активизации студенческой научной деятельности. (The section deals with the problem of students' inclusion in the scientific activities of the university. The results of a survey of students of the Institute of Psychology of K.E. Tsiolkovsky KSU on their motivation and factors influencing participation in scientific work are presented. The key motives are identified, the connection between involvement in scientific activities and the experience of informal research in school years is emphasized. The results of the self-assessment of the scientific competencies of graduate students are presented. The groups of research competencies are highlighted: substantive and technical design, the difference in self-assessment of technical and analytical skills is shown. The connection of the general self-assessment of readiness for independent completion of the final qualification work with such competencies as performing theoretical analysis and choosing adequate research methods has been established. Recommendations on the activation of student scientific activity are proposed).
... Segundo os responsáveis pelo desenvolvimento, os itens e escalas da NSSE são criados com base em fundamentos teóricos e empiricamente testados, apresentando propriedades psicométricas sólidas, como validade aparente e de construção robustas, além de confiabilidade consistente (Kuh, 2001). Entretanto, alguns outros autores evidenciam alguns problemas com a validade da NSSE (Pike, 2006;Porter, 2011). Porter (2011) destaca que até mesmo elementos simples, como "teve conversas sérias com alunos", podem ser interpretados de maneiras diferentes. ...
Preprint
Full-text available
O conceito de engajamento é polissêmico na literatura, muitas vezes empregado de maneira informal e sem definições precisas. Dada a sua ampla utilização no contexto educacional, é fundamental esclarecer os significados atribuídos a este termo no campo investigativo. Esta revisão sistemática da literatura visa elucidar essa polissemia, identificando as concepções, modelos, indicadores, facilitadores e métodos de mensuração do engajamento estudantil. Metodologicamente, realizamos uma análise de revisões de literatura já publicada sobre o tema. A busca foi conduzida nas bases de dados Scopus e Web of Science e, após processos de filtragem, selecionamos 52 artigos para compor o corpus. Como resultado, identificamos quatro perspectivas principais de engajamento estudantil na literatura: comportamental, psicológica, sociocultural e integrativa. Além disso, encontramos 18 facilitadores, que classificamos em quatro categorias: relações de apoio entre professor e aluno (o mais frequente), práticas docentes, elementos ambientais, e pessoais. Também identificamos seis métodos de mensuração, sendo a observação o mais recomendado e o questionário o mais utilizado nas pesquisas. Além de fornecer uma perspectiva abrangente sobre as formas como o conceito de engajamento estudantil é explorado em pesquisas educacionais, o que é de grande valor especialmente para pesquisadores recém-interessados na área, os resultados categorizados nesta revisão da literatura também oferecem orientações para a implementação de práticas pedagógicas que fomentam o engajamento estudantil, que são fundamentais tanto para docentes como para a formação de professores.
... Hinged on the Tyler and Pace's contribution, Astin (1984) stressed on the effect of participation on student accomplishment. Since then, student endeavor and time on duties to a wide range expected outcomes of education institution has drawn the attentions of the academician community (Pascarella & Patrick 2005;Pike 2006). Student engagement was identified as the time and exertion students place to the practices which were empirically related to expected results of education institutions and what institutions conducted to encourage the students to take part in these practices (Kuh, 2009) and their doggedness in and contentment with studying (Fredricks et al., 2004). ...
Article
Full-text available
An increasing focus has been placed on enhancing accounting student engagement (ASE) in blended learning courses (BLCs) because to its advantageous effects, particularly in the context of the pandemic. The primary objective of current study is to identify the distinct forms of intelligence possessed by lecturer that are relevant to teaching methods, and to investigate how these intelligences may affect student engagement in blended learning environments. The present study made use of a mixed-method technique. Consequently, the goal of analyzing the expert interviews’ qualitative perspectives was to determine how the constructs were put into practice and to have a better grasp of various issues related to the proposed model. The structural equation modelling was conducted using AMOS 28.0 software to analyze a theoretical model that explores the relationships between the mentioned components. Statistical response data was collected from a convenient and snowball sample of 323 informants from public higher education institutions. The conducted outcome analyses highlighted significant and positive relationships between the hypothesized constructs related to significance and effect size. Specifically, creativity intelligence had the highest path coefficient, followed by passion quotient, while adversity quotient had the lowest path coefficient among the drivers of ASE in BLCs. From a policymaking perspective, the current research recommended implementing necessary policy measures to make informed decisions and developing future action plans aimed at increasing and enhancing the intelligence of lecturers. Alternatively, the observations may provide practitioners and policymakers with fresh perspectives to develop specific tactics that can improve the implementation of BLCs.
... This stance confirms the importance, described in previous studies [22,24,26,30,33,37,50,74,75], of students' involvement in educational activities beyond the classroom boundaries. Corroborating literature, participants in this study posit that student life cannot be limited to class attendance and the preparation of papers and exams-this experience should be complemented with recreational or, at least, less formal activities. ...
Article
Full-text available
Research studies worldwide have focused on higher education dropout, persistence, and success. Given the profound changes in higher education that have taken place in recent decades, higher education institutions need to compete for students by attracting, retaining, and, ultimately, graduating them. Thus, higher education institutions increasingly offer actions that aim to foster student success. While a smooth and supported process of student transition from secondary to tertiary education is one of the key variables in higher education student retention and paramount for preventing student dropout, the student’s overall experience in higher education plays a pivotal role in their performance and success. This paper focuses specifically on higher education students’ academic and social involvement, notably through their engagement in extracurricular activities and decision-making processes, which are perceived as critical mechanisms in their persistence in higher education. The study used a qualitative approach with the analysis of four Portuguese higher education institutions. Data were collected through in-depth interviews with students and institutional leaders, complemented with document analysis, and explored through content analysis. The results reveal that, from the wide range of opportunities for involvement offered to students by the higher education institution, activities of an academic nature are the most sought after by students to complement their educational experience. However, students perceive involvement in extracurricular activities in general as critical, both to their overall education and preparation for the labor market and to an easier integration into the institutional environment. Hence, student involvement in cultural or recreational activities, alongside their involvement in institutional decision-making bodies and associative movements, is a privileged way of complementing students’ academic training and is perceived by them as important in their overall education, both as professionals and as individuals.
... Overall, the results of this research reveal that students acknowledge the relevance of extracurricular activities, as a complement to their scientific training, in a more comprehensive and global training logic. This positioning confirms the importance, described in previous studies [59,60], of the involvement of students in educational activities beyond the classroom. Students get involved and participate in numerous events, from cultural to playful and/or sporting and associative events. ...
Article
Full-text available
In the current higher education context, higher education institutions need, more than ever before, to compete for students, attracting, retaining and, ultimately, graduating them. To this end, actions are increasingly developed, and conditions are created to promote student success. The literature demonstrates that there is a strong link between the students’ experience and success. However, students’ experience cannot be controlled by the higher education institution, given the existence of previous subjective experiences that students bring when they enroll in higher education, which act as filters of their current experiences. The central goal of this study is to unveil the factors that students perceive as influencers on their global experience in higher education, which are reflected in their path, performance and success. The methodology used is qualitative, with in-depth interviews with students and institutional leaders from four Portuguese higher education institutions, complemented with documentary analysis. The results reveal that individual and organizational factors, alongside the students’ global experience, clearly influence their definition of a successful higher education student. Students build their representations of success based on the multiplicity and complexity of their experiences in higher education, affected by the features of the higher education institution and mediated by their personal history and life project.
... Education researchers employ multiple psychometric techniques to determine survey quality and validity. Statistics, such as factor analysis and inter-item reliability, are used to confirm the validity and reliability of survey items and the validity of constructs [12][13][14][15]. In addition, researchers incorporate qualitative assessments such as enlisting subject matter experts to support content validity about survey instruments and cognitive interviewing [11,14,16]. ...
Article
Full-text available
Addressing issues of inequity and exclusion on college campuses requires a comprehensive assessment strategy inclusive of diverse student populations’ cultures and communities, as it is integral to understanding students and their environment. Developing instruments that can accurately and robustly measure student culture is necessary for campus leaders to contextualize their data. The purpose of this paper is to validate and describe the use of the term cultural community in the Culturally Engaging Campus Environments survey. We used cognitive interviewing to illicit racially and ethnically diverse students’ interpretations of the term and its use in the survey. Findings suggest that students’ interpretations are mitigated by larger societal and institutional discourses, although race was a common mitigating factor.
... This notion is better explained by understanding the potentials and weaknesses of AL approaches or perspectives. A detailed account of strengths and weaknesses of Active Learning is described in Barling et al., 2008;Chi & Wylie, 2014;Coates, 2007;Kahu, 2013;Pike, 2006;Zhao & Kuh, 2004). It fails to distinguish between engagements and its antecedents. ...
Article
Full-text available
Remote teaching is the “new normal” especially in higher educational institutions as the world grapples with the raging COVID-19 pandemic and innumerable changes caused by disruptive technologies. Higher educational institutions have resorted to remote teaching as a means of ensuring student safety and an effective learning approach. For this approach to succeed, didactic methods must be carefully chosen to ensure student active engagement during learning processes. Specifically, cognitive engagement has been often cited as a critical component of students’ educational experience which includes active, constructive, interactive, and passive cognitive modes of engagement. These modes allow students to demonstrate knowledge as node-link structures through different overt behavioural knowledge-change processes that require individuals to store, activate, link, and infer varying stages of learning also known as Active Learning (AL). AL enables students’ engagement activities to construct knowledge, improve subject content retention and better their performance in achieving learning outcomes. AL can leverage both synchronous and asynchronous teaching pedagogical methods. The primary goal of this review was to examine AL literature during COVID-19 pandemic in order to propose solutions for improving student cognitive engagement in higher educational institutions.
... Consequently, the questionnaire was handed over to students of two consecutive semesters during the year of observation. According to Pike (2006), NSSE scale-let scores provide valid measures of students' educational experiences, and they can be used for teaching evaluation. The questionnaire was hand-delivered to each student and participation was entirely voluntary and anonymous. ...
Article
Validation of ideas are of paramount importance in STEM fields. Learning and converting ideas into practical application is the main purpose of technical education. Aviation is a highly safety sensitive field where confusion and mistakes are not acceptable. This brings serious challenges for academia that provides higher education in this field. A yearlong observation of the reflective practices was done at an Australian university while teaching aviation students to analyse outcomes of reflection on teaching and learning. Reflection provides a powerful opportunity to a teacher in improving teaching qualities and to identify training needs for enhancing teaching capabilities.
... Although these results suggest that the four factors suggested by NSSE are meaningful, a more comprehensive depiction might be required to include factors that might be important from the point of view of different institutions, different settings, and even different countries (Pike 2006). If the measurement instrument had included more items relevant to Indian Management Institutes and students, perhaps a few other components might have emerged from the study. ...
Article
Full-text available
The study seeks to develop a model to measure and evaluate Students’ Engagement in Indian Management Institutes for which the India Management Students’ Engagement Scale (IMSES) was evolved from the National Survey of Student Engagement (NSSE) of the Indiana State University, USA. The study was based on the premise that a measurement scale that had its origins in a standardized model like the NSSE could help predict important educational outcomes for Indian students. The construct validity approach was adopted for the study conducted in three management institutes in Surat, located in Gujarat in India. The responses from 156 MBA students were included in the study. The results suggest that the four broad themes namely Academic Challenge, Relationships with faculties, Peer Interactions and Campus Environment do converge to explain the concept of Student Engagement in the urban Indian setting. Originally, a questionnaire with 30 items was fielded to the students. Upon exploratory analysis, ten items with issues related to low factor loadings, content validity and error variances were removed. The final IMSES emerged as a lean 14 item scale after modifications which makes it is extremely easy to administer. The CFA supported a four-dimensional model based on the NSSE Version 2.0, with all the dimensions showing high internal consistency based on Chronbach’s alpha coefficients. The model fit was adequate and acceptable based on well-documented thresholds. Although the four factors suggested by NSSE are meaningful in urban Indian educational settings, the need for a more India-centric approach to designing the survey has been discussed.
... Collecting self-reported confidential and anonymous information is a standard method of enquiry into student engagement and has been used elsewhere in the AUSSE and other widely recognized applied surveys within the field of education research. For this evidence to be reliable and valid, certain prerequisites needed to be met (Pike, 2006;Pohlmann, 1976) that include ensuring respondents were aware of information being requested; there was no ambiguity or unclearly phrased questions; activities being investigated were recent; respondents believe there was merit to answering the question; there was a possibility to verify respondents' answers; and finally, the respondents did not feel a threat of embarrassment or violation of privacy by answering the questions. In weeks prior to the survey, students were provided with formal participant information indicating the background, purpose, and demands on respondents as well as the confidentiality provisions. ...
Article
Full-text available
The quantitative evaluation of student engagement has been difficult to achieve. This study uses Kahu’s (2013) conceptual framework to investigate the effectiveness of active teaching strategies and how they influence Business students’ engagement in a blended learning environment. First, we quantify the influence of various in-class active teaching activities and out-of-class support tools upon student engagement. The link between engagement and student outcomes in terms of academic results and personal and professional skills development is then captured in our empirical modelling. Results are compared between first year and senior students to understand significant differences in their engagement and experience. Our findings suggest that first year students display a higher propensity to utilize in-class learning activities and out-of-class support tools. This in turn, establishes a strong link with their engagement patterns. However, there is a weaker link between first year student engagement and outcomes compared to senior students. Overall, this study reinforces the usefulness of Kahu’s framework to guide curricula developments that cater for learners’ different needs.
... NSSE measures students' participation in educationally purposeful activities that prior research has found to be linked to desired outcomes of college (Chick ering & Gamson, 1987;Pascarella & Terenzini, 2005). The validity and reliabilit of the instrument have been extensively tested (Kuh, 2002;Kuh, Kinzie, Cruce, Shoup, & Gonyea, 2006;Pike, 2006). The NSSE questionnaire, in the field since 2000, collects data in five general areas. ...
... To explore this objective, a construct-validation approach was used to substantiate the presumed dimensionality of working alliance. As noted by Pike (2006), construct validity calls for a systematic examination of the extent to which an item relates to other observable variables. Simply put, items that measure the same factor should correlate strongly with one another (convergent validity) while displaying low correlations with those items indexing different factors (discriminant validity; e.g., Kline, 2005). ...
... Student engagement theory also has its roots in the research of C. Robert Pace conducted in the 1960s and 1980s, which focused on quality of effort. Additionally, student engagement theory has its roots in the research of Alexander Astin conducted in the 1980s, which focused on students' involvement (Axelson & Flick, 2011;Pike, 2006). ...
... Theoretically, convergent and discriminant validity should measure similarity and dissimilarity of the constructs respectively. For ensuring the construct validity of the scale, both validities should be assessed simultaneously ( Pike 2006, Watson et al 1995). We used rotated component matrix table of factor analysis to test convergent and discriminant validity of the scale. ...
Thesis
Full-text available
Despite various efforts maternal mortality is still high in most low resource countries of Sub-Saharan Africa and South East Asia. Nepal has made remarkable progress in maternal mortality reduction. However, most indicators of Millennium Development Goal Five i.e. completion of four antenatal care visits, skilled care at birth, institutional deliveries and postnatal care visits have not attained set targets. The high utilization of maternal health care services makes it possible to reduce maternal and neonatal morbidity and mortality. Demographic factors, socio-economic characteristics of women, geographical location, availability and accessibility of health care services, needs of health care are some factors which determine the utilization of maternal health care services. Women’s autonomy is one of the determinants of maternal health care services utilization in developing countries. Most studies have focused on assessing the association with the demographic and socioeconomic factors and the utilization of maternal health care services rather than exploring the role of women’s autonomy as a mediating factor in the utilization of maternal health care services. There are a few studies which looked at women’s autonomy and its influence on the utilization of maternal health care in Nepal; however, most women’s autonomy measurement tools included very limited components of women’s autonomy. Only in a few cases their psychometric characteristics were assessed. Autonomy is regarded as a multidimensional construct and difficult to quantify. However, it is explained as the capacity of a person to work independently in accordance with his/her aims and objectives. Women’s autonomy has contextual meaning and it depends upon the personal attributes of women as well as norms, values and practices of the society. There is no unanimous view on women’s autonomy. Most researchers prefer proxy indicators i.e. educational attainment, employment, income, spousal age difference, and type of family to measure women’s autonomy and utilization of maternal health care services. The plains area of western Nepal has access to road transportation and various health care facilities. However, there is the low utilization of maternal health care services for achieving the target of the Millennium Development Goal five in Kapilvastu district. It is a challenge for Nepal as one of the signatory members of the Millennium Declaration 2000. For assessing women’s autonomy and the utilization of maternal health care services, and for better understanding of its key predictors in Nepal, we chose this district as the study area and conducted this study. The overall aim of this study was to study women’s autonomy and its influence on the utilization of maternal health care services. Specific objectives were - 1) to construct and validate a women’s autonomy measurement scale based on the field evidence; 2) to examine the relationship between women’s demographic and socioeconomic characteristics, and their utilization of maternal health care services; 3) to correlate women’s autonomy with their demographic characteristics and socioeconomic factors; and, 4) to examine the role of women’s autonomy as a mediating factor in the utilization of maternal health care services. Our study hypothesis is- women who have high autonomy utilize more maternal health care services than women with restricted or low autonomy. This was a population based cross-sectional study, conducted in two phases. First, we constructed a women’s autonomy scale using a sample of 250 women from Rupandehi scale, and tested its psychometric characteristics as well as validated it. Second, we conducted a cross-sectional survey for assessing women’s autonomy and factors associated with the utilization of maternal health care services in Kapilvastu District, Nepal, using the scale which we developed. We used the survey data also for the validation of the scale using factor analysis for assuring the construct validity of the scale. For constructing a new scale, we defined women’s autonomy (construct) as a capacity of the women to control decision-making, financial and physical resources, and freedom of mobility. We generated an items pool reviewing published literature and prepared a preliminary draft of the scale. We conducted pretest, psychometric analysis and validation test for assuring measurement capacity of the scale. Development of the scale was done on a non-random sample of 250 women of child-bearing age in Rupandehi district, which is similar to the study area in its characteristics. Study population of the survey comprised all women of reproductive age who had full term delivery in the preceding year and completed their postnatal period preceding the survey. We selected ten village development committees (VDCs) out of 76 VDCs of the district using simple random sampling method. The final number of women at VDC level was fixed proportionately considering the total population of VDC. We interviewed 500 women from all five electoral constituencies (ECs) areas and 10 VDCs of the district. The sample size was computed based on the proportion of skilled care at birth (15.92%) of Kapilvastu district, with design effect = 2 and non-response rate = 20% using online OpenEpi statistics software. For identifying the respondents in each village, we identified the center of the village and chose a random direction for identifying households with subjects. We continued the household visit in the clock-wise direction until obtaining the required number of respondents. Considering Cronbach’s Alpha value (0.84), average content validity ratio/ index (0.8) and overall agreement- kappa value (0.83), we accepted all 24 items of the scale. The minimum and maximum score of the new women’s autonomy measurement scale was zero and forty eight respectively. We also did exploratory factor analysis with the survey data and finalized 23 items of the scale where the items had good convergent and discriminant validity. All 23 items were loaded in five factors. All five factors had single loading items by suppressing absolute coefficient value less than 0.45 and average coefficient was more than 0.60 for each factor. The new scale is a reliable tool for assessing women’s autonomy in developing countries which quantifies the possible score of women’s autonomy between zero and forty six. We suggest use and validate the scale for assuring the performance of the scale in large samples and different settings. The mean score of the autonomy measured by our scale was 23.34 8.06 out of the possible maximum scoring forty-eight. We found 83.6% pregnant women sought at least one antenatal care visit in one year preceding the study period. More than one-third (37.6%) women sought institutional delivery for their last childbirth. Antenatal care visit had strong positive association with the place of delivery (chi-square = 20.05, df = 1 and P <0.001) and postnatal care (chi-square = 16.77, df = 1 and P <0.001). Nearly six per cent deliveries were conducted by general health workers in health facilities. Out of the total institutional deliveries, nearly 58% women visited health facilities for the self-reported emergency obstetric care services. A few home deliveries (6.2%) were assisted by health workers, 14.7% households used safe delivery kit (SDK) for home delivery care and 22.0% women sought postnatal care in their last postnatal period. We examined the association with explanatory variables and outcome variables i.e. antenatal care, delivery care and postnatal care. We found that having at least one antenatal care visit and institutional delivery care had positive association with five years or greater difference of age with the spouse, some ethnic groups, better education of couples, occupation of husband, and economic status of the household. Having at least one postnatal care visit was found to be positively associated with women’s age more than 20 years at marriage, parity of two or below, some caste/ethnic groups, the better education of couple, occupation of the husband, and economic status of the household. Women’s autonomy was found to be positively associated with five years or more of age difference with the spouse at marriage, some caste/ethnic groups, better employment for the husband, women’s and husband’s education more than 10 years schooling, and high economic status. Out of several explanatory variables, women’s education (OR = 8.14, CI = 3.77–17.57), husband’s education (OR = 2.63, CI = 1.69–4.10) and socio-economic status of the household (OR = 1.42, CI = 1.01–2.03) were found as major predictors of women’s autonomy. Women’s education, husband’s education and women’s autonomy were found to be key predictors of the utilization of maternal health care services. Women’s education had strong positive association (OR = 24.11, CI = 9.43–61.64) with institutional delivery care. The stratified multivariate analysis further showed that when the husband is not educated, women’s education works independently of the effect of her autonomy, and is a dominant influence. On the other hand, when the husband is educated, women’s education seems to work partly through her autonomy where around 40% of the effect is explained by her autonomy. Low education and economic status of women are significantly associated with the low utilization of maternal health care services in Kapilvastu district of Nepal. Women’s autonomy seems as a mediating factor of the pathways in the utilization of maternal health care services. Women’s education, husband’s education and economic status of the household are key predictors of women’s autonomy and the utilization of maternal health care services. Improvement of women’s education, husband’s education and economic status of the household would be among the effective strategies for increasing their autonomy and the utilization of maternal health care services in Kapilvastu district of Nepal. In spite of intensive implementation of incentive programmes to increase the utilization of maternal health care services during pregnancy, childbirth and postnatal period, we do not see an encouraging response. It points to the very basic and strong relationship between women’s position in the household and the society, and their health status. There are limits to how far financial incentives can overcome these obstacles.
... The analysis of the interviewees' narratives allows concluding that students' academic experience cannot be disconnected from their social and functional experience in the HEI. This stance confirms the importance, described in previous studies (Astin 1985;Hassan 2014;Pike 2006), of students' involvement in educational activities beyond the classroom boundaries. Indeed, and corroborating literature, participants in this study posit that the student life cannot be limited to class attendance and the preparation of papers and exams -it is important that this experience is complemented with ludic or, at least, less formal activities. ...
... A large body of empirical research in higher education has documented myriad benefits of undergraduate student participation in faculty-mentored research. For student participants, these include improved analytic and critical thinking, increased academic achievement and retention, persistence to degree completion in their chosen field, improved ability to think and work like a scientist, clarification of career plans, and improved preparedness or desire for graduate study Bennett 2003, 2008;Cole and Espinoza 2008;Espinosa 2009;Hathaway et al. 2002;Hunter et al. 2007;Ishiyama 2002;Kardash 2000;Kuh et al. 2007;Laursen et al. 2010;Lopatto 2004;Nagda et al. 1998;Pike 2006;Russell 2008;Seymour et al. 2004;Tompkins 1998;Volkwein and Carbone 1994;Wasserman 2000). The retention and success of underrepresented minority students in particular has been shown to be strongly related to faculty-mentored URE (Brown, Davis, and McClendon 1999;Campbell and Campbell 1997;Cronan-Hillix et al. 1986;Eagan et al. 2013;Hurte 2002;Jones et al. 2010;Luna and Cullen 1995;Schultz et al. 2011;Stromei 2000;Villarejo et al. 2008). ...
Article
Full-text available
Undergraduate research experiences are a “high impact” educational practice that confer benefits to students. However, little attention has been paid to understanding faculty motivation to mentor undergraduate students through research training programs, even as the number of programs has grown, requiring increasing numbers of faculty mentors. To address this, we introduce a conceptual model for understanding faculty motivation to mentor and test it by using empirical data to identify factors that enable and constrain faculty engagement in an undergraduate research program. Using cross-sectional survey data collected in 2013, we employed generalized linear modeling to analyze data from 536 faculty across 13 research institutions to examine how expected costs/benefits, dispositional factors, situational factors, previous experience, and demographic factors predicted faculty motivation to mentor. Results show that faculty who placed greater value on the opportunity to increase diversity in the academy through mentorship of underrepresented minorities were more likely to be interested in serving as mentors. Faculty who agreed more strongly that mentoring undergraduate students was time consuming and their institution’s reward structures were at odds with mentoring, or who had more constrained access to undergraduate students were less likely to be interested in serving as mentors. Mid-career faculty were more likely than late-career faculty to be interested in serving as mentors. Findings have implications for improving undergraduate research experiences, since the success of training programs hinges on engaging highly motivated faculty members as mentors.
... A benchmarking approach assumes that the unit of analysis is the institution and that the group-level score is reliable. Generalizability studies have shown that NSSE's engagement measures are dependable measurements of group means (Fosnacht & Gonyea, 2012 ;Pike, 2006aPike, , 2006b . Of course, group scores need not be limited to entire institutions. ...
Chapter
This chapter traces the development of student engagement as a research-informed intervention to shift the discourse on quality in higher education to emphasize matters of teaching and learning while providing colleges and universities with diagnostic, actionable information that can inform improvement efforts. The conceptual lineage of student engagement blends a set of related theoretical propositions (quality of effort, involvement, and integration) with practice-focused prescriptions for good practice in undergraduate education. The development of survey-based approaches to measuring student engagement is reviewed, including a treatment of recent criticisms of these approaches. Next, we summarize important empirical findings, including validation research, typological research, and research on institutional improvement. Because student engagement emerged as an intervention to inform educational improvement, we also present examples of how engagement data are being used at colleges and universities. The chapter concludes with a discussion of challenges and opportunities going forward.
... For ensuring the construct validity of the scale, both validities should be assessed simultaneously. 19,20 We used rotated component matrix table of factor analysis to test convergent and discriminant validity of the scale. We assessed correlation coefficient of the items within a factor to ensure the convergent validity and loading of factors to ensure the discriminant validity of the scale. ...
... For ensuring the construct validity of the scale, both validities should be assessed simultaneously. 19,20 We used rotated component matrix table of factor analysis to test convergent and discriminant validity of the scale. We assessed correlation coefficient of the items within a factor to ensure the convergent validity and loading of factors to ensure the discriminant validity of the scale. ...
Article
Full-text available
Introduction: Women's autonomy is one of the predictors of maternal health care service utilization. This study aimed to construct and validate a scale for measuring women's autonomy with relevance to developing countries. Methods: We conducted a study for construction and validation of a scale in Rupandehi and further validated in Kapilvastu districts of Nepal. Initially, we administered a 24-item preliminary scale and finalized a 23-item scale using psychometric tests. After defining the construct of women's autonomy, we pooled 194 items and selected 24 items to develop a preliminary scale. The scale development process followed different steps i.e. definition of construct, generation of items pool, pretesting, analysis of psychometric test and further validation. Results: The new scale was strongly supported by Cronbach's Alpha value (0.84), test-retest Pearson correlation (0.87), average content validity ratio (0.8) and overall agreement- Kappa value of the items (0.83) whereas all values were found satisfactory. From factor analysis, we selected 23 items for the final scale which show good convergent and discriminant validity. From preliminary draft, we removed one item; the remaining 23 items were loaded in five factors. All five factors had single loading items by suppressing absolute coefficient value less than 0.45 and average coefficient was more than 0.60 of each factor. Similarly, the factors and loaded items had good convergent and discriminant validity which further showed strong measurement capacity of the scale. Conclusions: The new scale is a reliable tool for assessing women's autonomy in developing countries. We recommend for further use and validation of the scale for ensuring the measurement capacity.
... NSSE measures students' participation in educationally purposeful activities that prior research has found to be linked to desired outcomes of college (Chickering & Gamson, 1987;Pascarella & Terenzini, 2005). The validity and reliability of the instrument have been extensively tested (Kuh, 2002;Kuh, Kinzie, Cruce, Shoup, & Gonyea, 2006;Pike, 2006). The NSSE questionnaire, in the field since 2000, collects data in five general areas. ...
Article
Conducted through a collaboration between the Council of Writing Program Administrators (CWPA) and the National Survey of Student Engagement (NSSE), this study identified and tested new variables for examining writing's relationship to learning and development. Eighty CWPA members helped to establish a consensus model of 27 effective writing practices. Eighty US baccalaureate institutions appended questions to the NSSE instrument based on these 27 practices, yielding responses from 29, 634 first-year students and 41, 802 seniors. Confirmatory factor analysis identified three constructs: Interactive Writing Processes, Meaning-Making Writing Tasks, and Clear Writing Expectations. Regression analyses indicated that the constructs were positively associated with two sets of established constructs in the regular NSSE instrument-Deep Approaches to Learning (Higher-Order Learning, Integrative Learning, and Reflective Learning) and Perceived Gains in Learning and Development as defined by the institution's contributions to growth in Practical Competence, Personal and Social Development, and General Education Learning-with effect sizes that were consistently greater than those for the number of pages written. These were net results after controlling for institutional and student characteristics, as well as other factors that might contribute to enhanced learning. The study adds three empirically established constructs to research on writing and learning. It extends the positive impact of writing beyond learning course material to include Personal and Social Development. Although correlational, it can provide guidance to instructors, institutions, accreditors, and other stakeholders because of the nature of the questions associated with the effective writing constructs. © 2015 by the National Council of Teachers of English. All rights reserved.
... To explore this objective, a construct-validation approach was used to substantiate the presumed dimensionality of working alliance. As noted by Pike (2006), construct validity calls for a systematic examination of the extent to which an item relates to other observable variables. Simply put, items that measure the same factor should correlate strongly with one another (convergent validity) while displaying low correlations with those items indexing different factors (discriminant validity; e.g., Kline, 2005). ...
Article
Full-text available
Relationships with teachers have been found to be particularly salient for elementary-age students, as they relate to successful adjustment to school. The construct of working alliance reconceptualizes traditional definitions of relationship to consider elements of emotional connection, as well as the collaboration central to the working relationship between two individuals. The current study sought to examine the construct validity of the Classroom Working Alliance Inventory (CWAI). Multilevel confirmatory factor analyses supported a two-factor model, representing the emotional and collaborative elements of relationship. These findings provide evidence for the validity of the construct of classroom working alliance in capturing the working relationship between teacher and student.
... A multitude of studies call upon NSSE data, and several are of interest in the present study. At least three data reduction models are widely recognized as effective and reliable means of gaining a broad sense of NSSE data: (a) the Indiana University-Bloomington's NSSE Benchmark Scores (Kuh, 2001) Pike's (2006) Scalets; and (c) Kuh, Kinzie, Cruce et al.'s (2006) Educationally Purposeful Activities. The existence of strong positive correlations between NSSE Benchmark Scores and GPAs has been well documented (Carini, Kuh, & Klein, 2006;Gordon, Ludlum, & Hoey, 2008;Kuh, 2001). ...
Article
Full-text available
Researchers evaluated the effects of Educationally Purposeful Activities (EPAs) on transfer and nontransfer students’ cumulative GPAs. Hierarchical, linear, and multiple regression models yielded seven statistically significant educationally purposeful items that influenced undergraduate student GPAs. Statistically significant positive EPAs for transfer students were: (a) receiving prompt written or oral feedback from faculty on academic performance, (b) tutoring or teaching other students (paid or voluntary), (c) asking questions in class or contributing to class discussions, and (d) working harder than they thought they could to meet an instructor’s standards or expectations. A negative statistically significant activity of transfer students was having serious conversations with students of a race different from their own. Overall, the effects of EPAs on undergraduate GPAs are somewhat more robust for transfer students. A recommendation to incorporate specific EPAs in transfer students’ curriculum followed.
... For ensuring the construct validity of the scale, both validities should be assessed simultaneously. 19,20 We used rotated component matrix table of factor analysis to test convergent and discriminant validity of the scale. We assessed correlation coefficient of the items within a factor to ensure the convergent validity and loading of factors to ensure the discriminant validity of the scale. ...
... The NSSE (NSSE, 2014) is a validated (Pike, 2006) tool used to assess factors related to student engagement, including collaborative learning, reflective and integrative learning, student faculty Digital badging in university courses dataset 939 interaction, higher order learning, effective teaching practices, learning strategies and student satisfaction. Some questions were removed to better suit the structure of the course. ...
Article
Full-text available
This dataset contains participant data related to the use of badging (achievement) feedback in pedagogical design. Two sections each of web‐based graphic design and web design undergraduate courses were offered at the University of Central Florida. A badging system for achievements was included in one section of each. Performance, engagement and behavior‐related data were collected. The dataset comprises complete data from 44 undergraduate students. This paper includes a justification for data collection, methodology and preexisting usage of the data. It also discusses limitations of the dataset.
... El estudio del compromiso estudiantil se ha operacionalizado en un instrumento diseñado por la Universidad de Indiana denominado National Survey of Student Engagement aplicado de manera sistemática desde el año 2000 en aproximadamente 1500 instituciones de educación superior en Estados Unidos y Canadá y que ha proporcionado información clave para la cualificación de la experiencia educativa que se ofrece a los estudiantes en esos contextos. La validez y confiabilidad de la encuesta se ha demostrado en diversos estudios (National Survey of Student Engagement, 2012a; Kuh, Cruce, Shoup, Kinzie & Gonyea, 2008;Pike, 2006) e incluye una versión en español que ha sido empleada hasta el momento en México, Puerto Rico y España. ...
Article
Full-text available
Despite the increase in Latin America of Higher Education coverage, grave dropout problems persist that question the role of educational experiences to foster students’ academic engagement. This study was carried out in Colombia and sought to establish the relationship between the five benchmarks that compose academic engagement and the academic performance of a group of Colombian university students. The transversal and correlational study used the Spanish version of the National Survey of Student Engagement (NSSE) that measures students’ level of participation in five dimensions: Academic challenge, active and collaborative learning, student-faculty interaction, enriching educational experiences, and supportive campus environment and its relationship to academic performance. The findings of 1906 students from 7 universities indicate that there are statistically significant, but weak correlations between the items that compose the benchmarks and students’ academic performance, which lead to reflect upon key aspects to strengthen the education experiences offered to university students.
Article
The study aims to disclose the role of the universities and students’ backgrounds in predicting student engagement. The study uses Monitoring of Education Markets and Organizations (MEMO) of 2015 and 2017, which is hierarchical nationally representative data set of 5,251 undergraduate students nested into 135 universities in Russia. Four indices were developed to measure student engagement based on behavioural approach, namely, class engagement, learning engagement, research engagement, and extracurricular engagement. The mixed-effects multilevel modelling was used to test the hypotheses on university roles and students’ background characteristics. Students’ parental family characteristics did not appear to have a strong influence on student engagement except mothers’ higher education. High school achievements are very important for further student engagement. The findings highlight the importance of motivation and career ambitions of students. Plans to work within the field of study or to pursue further studies to obtain a MA or Ph.D. degree positively associated with student engagement. The results confirmed that the academic environment is the strongest predictor of student engagement. Universities need to develop active teaching practices to improve the academic environment and increase student engagement.
Article
The focus of the present study is whether student engagement as measured at the class level via the Classroom Level Survey of Student Engagement (CLASSE) is associated with higher levels of student satisfaction. To do so, we administered the CLASSE survey to 370 first-year, sophomore, junior, and senior students across nine classrooms. There were no statistically significant relationships between student engagement and overall satisfaction. When analyzed with student responses to questions about their satisfaction in the classroom, we found no variation in student satisfaction by race, gender, or class level. Implications for future research and practice are discussed.
Article
Chinese universities have opened their doors for foreign students in recent years. Finding internationally accepted academic degrees programs and relevantly qualified academics however remains a challenge for the Chinese universities. The universities too are facing difficulties in engaging international students in studies and university activities. Many students remain regularly absent from academic activities because they feel isolated due to one institution-two systems concept of education administration at the universities. International education management and effectiveness of teaching and learning practices of the universities under the system are examined in this study.
Article
This chapter reviews the history and various definitions of student engagement and proposes a multidimensional model from which one can develop a variety of engagement opportunities that lead to a rich and challenging higher education experience.
Article
The focus of the present study is whether student engagement as measured at the class level via the Classroom Level Survey of Student Engagement (CLASSE) is associated with higher levels of student satisfaction. To do so, we administered the CLASSE survey to 370 first-year, sophomore, junior, and senior students across nine classrooms. There were no statistically significant relationships between student engagement and overall satisfaction. When analyzed with student responses to questions about their satisfaction in the classroom, we found no variation in student satisfaction by race, gender, or class level. Implications for future research and practice are discussed.
Article
Despite the popularity of student engagement and, by association, student disengagement, the academic literature is unclear about the meaning of these terms. This review extends existing conceptual studies of student engagement by offering clear definitions and conceptualisations of both student engagement and disengagement in the classroom context. To develop these conceptualisations, the present review draws upon organisational behaviour theory on work engagement and disengagement because the literature in this discipline is notably more refined than in educational research. Using an organisational behaviour backdrop, student engagement and disengagement are defined by the degree of students’ activation and pleasure. In order to operationalise student engagement, measures that are aligned with the proposed conceptualisation are recommended. Recommendations are also suggested for the development of a measure of student disengagement. The proposed measurement of student engagement and disengagement should provide a unified direction for future empirical research on these topics.
Article
We examined persuasive and expository writing samples collected from more than 300 college students as part of a nine-year cross-sectional and longitudinal study of undergraduate writing performance, conducted between 2000 and 2008. Using newly developed scoring rubrics, longitudinal analyses of writing scores revealed statistically significant growth in writing performance over time. These findings held for both persuasive and expository writing. Although writing performance was better among women than men, and better among students majoring in the humanities and social sciences than in natural sciences and engineering, neither women nor humanities and social science majors showed differential improvement over time from freshman to senior year. Our findings showed reliable increases in writing performance during a student’s college years, and moreover demonstrated that such longitudinal changes can be effectively measured. We call for more such outcome assessment in higher education as an essential tool to enhance student learning.
Article
By superimposing digital learning content on top of the video stream of the mobile device's camera, Augmented Reality (AR) enables learners to interact with real-world context in mobile learning setting. Technically, the AR technology can be considered as consisting of two major aspects: identification of the real-world object and display of computergenerated digital contents related the identified real-world object. In this paper, we will present a location-based object identification algorithm that has been used to identify the location-based learning objects in the 5R adaptive mobile learning system. We will also discuss some issues in using the algorithm. As an example of using this algorithm, an AR mobile application on iPhone will be shown to demonstrate the effectiveness of the location-based object identification algorithm.
Chapter
Systems, institutions and researchers in Canada started using the National Survey of Student Engagement (NSSE) in 2004. This chapter begins with a brief review of the postsecondary education sector in Canada. It discusses the introduction of NSSE and how it came to be administered by the majority of university-level education providers. Harvesting insights from rich data, the chapter describes the main findings of the initial engagement surveys and samples, and reactions of the various stakeholders, including a suite of research projects, which flowed from these reactions. The chapter concludes by speculating on the future developments of NSSE in Canada.
Article
Michelle, a first-generation college student from a predominantly Black urban area, was a senior health and recreation major at Midwest University. Although successful in her health and recreation coursework and an engaged campus student leader, Michelle “often talked about her time on campus as ‘painful’” (Winkle-Wagner, 2009, p. 99):You might get the one person who's like, “Well I don't like Black people,” but, then you have a bunch of other people ganging up on him saying, “That is so old, nobody does that anymore.” And I feel like I am more accepted by White people than I am [by] the Black people. Because they're like, well she doesn't dress a certain way, or … “Why are you listening to that type of music?” (Winkle-Wagner, p. 99)
Article
South Africa is currently faced with the challenge of undesirably low throughput rates in higher education. The need to keep students interested and motivated to succeed are key objectives of many lecturers and institutions. Empirical studies have shown that one of the factors influencing student success at university is student engagement. This paper presents lecturers’ and students’ perspectives on levels of student engagement in four (Engineering, Business, Applied Sciences, and Informatics and Design) of the six faculties at CPUT as well as factors that contribute to and constrain these levels of engagement in learning. Finally, the paper offers some practical strategies for promoting student engagement and success in learning.
Book
Full-text available
Accessible to any professional or researcher who has a basic understanding of analysis of variance, "Generalizability Theory: A Primer" offers an intuitive development of generalizability theory, a technique for estimating the relative magnitudes of various components of error variation and for indicating the most efficient strategy for achieving desired measurement precision. Covering a variety of topics such as generalizability studies with nested facets and with fixed facets, measurement error and generalizability coefficients, and decision studies with same and with different designs, the text includes exercises so the reader may practice the application of each chapter's material. By using detailed illustrations and examples, Shavelson and Webb clearly describe the logic underlying major concepts in generalizability theory to enable readers to apply these methods when investigating the consistency of their own measurements. (PsycINFO Database Record (c) 2010 APA, all rights reserved)
Article
Full-text available
"This paper advocates a validational process utilizing a matrix of intercorrelations among tests representing at least two traits, each measured by at least two methods. Measures of the same trait should correlate higher with each other than they do with measures of different traits involving separate methods. Ideally, these validity values should also be higher than the correlations among different traits measure by the same method." Examples from the literature are described as well as problems in the application of the technique. 36 refs. (PsycINFO Database Record (c) 2010 APA, all rights reserved)
Article
Full-text available
It is argued that in the next edition of Technical Recommendations for Psychological Tests and Diagnostic Techniques "there should be a considerable strengthening of a set of precautionary requirements more easily classified under construct validity than under concurrent or predictive validity as presently described." (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Chapter
Article
A student development theory based on student involvement is presented and described, and the implications for practice and research are discussed.
Article
Although institutional characteristics are assumed to influence student learning and intellectual development, this link has not been confirmed empirically. This study examined whether institutional mission, as represented by Carnegie classification, is related to student learning and development. After controlling for student background characteristics, no meaningful differences were found in students' perceptions of the college environment, levels of academic and social involvement, integration of information, or educational outcomes by Carnegie classification.
Article
The Review of Higher Education 26.4 (2003) 541-542 Since Ernest Boyer introduced the idea of different types of scholarship in 1990, types have proliferated. Trudy Banta, this volume's editor, maintains that in 1999 "scholarship of assessment" spontaneously joined that list. This book, written by her and twenty associates from diverse positions in the U.S. higher education system, aims to give a state of the art report on this new branch of scholarship, underpin it by showing different theoretical contributions to it, and finally convince the skeptics of its worth. The book therefore has theoretical as well as practical aims and is targeted mainly to faculty and administrators in assessment (p. xi). What does "scholarship of assessment" mean? Banta defines it as systematic enquiry about assessment (p. x), in the words of Barbara Wright "sophisticated thinking about assessment" (p. 242; italics Wright's). Assessment has received much attention for at least a decade from higher education institutions, from state and federal policymakers, and from accreditation agencies. As a result, U.S. higher education has seen many developments in assessment, much data-collection, and therefore fertile ground for this developing area of scholarship. Victor Borden (Chapter 9) makes the point that we know more than we know: institutional research offices often collect assessment-related data that could be used in a sophisticated, scholarly way. Scholarship of assessment builds upon scholarly assessment, i.e., sophisticated "practice of assessment" (p. 242). Therefore, the first of this volume's five parts gives a history and an overview of the current status of assessment. Peter Ewell concludes that assessment has not (yet) shown the usual fate of "movements," which is, in Lewis Carroll's memorable phrase about the usual fate of snark hunters "to softly and suddenly vanish away" or to become institutionalized in mainstream practices (p. 21). Assessment practice is "broad," but not "deep." Part 2, "Theoretical Foundations of Assessment," consists of four chapters, each approaching the issue of theoretical deepening through different "lenses" (ranging from objectivist versus subjectivist epistemologies), a critique of indicator practices, and an overview of lessons learnt about assessment in Alverno College. At the end of this section and the next, readers find a plethora of methods of assessment, among others, Web-based techniques. This volume is not a handbook, and the descriptions stop short of that, yet they do provide readers with a feeling for their strengths. Because the writers are proponents, readers have to find the weaknesses themselves. Also, further readings and Web-sites are given for those looking for handbooks. Part 4 surveys actual experiences with scholarly assessment, beginning with classroom assessment, and continuing on through assessment capturing the essence of students' experiences (capstones, portfolios), to assessment of whole programs and institutions. External assessment, especially by professional and regional accreditation agencies, progressively enters the scene, a motif that makes me, a non-U.S. resident, ask again what is meant by assessment? The term covers a lot, and so does its scholarship. In Part 5, Banta summarizes this book's findings and arguments in two chapters. First, from a survey of higher education institutions across the United States, she identifies 17 characteristics of effective outcomes assessment from planning and implementation to "improving and sustaining" in the assessment process. Her final chapter is an eight-page agenda for strategies toward transformation in institutions, based on the previous chapters of the book. The book does, in fact, largely deliver what it promised. If I had to start a job involving student assessment, program review, or accreditation, I would keep this volume at hand to check on the field's status and underlying principles. Yet if I, as a "scholar of assessment," had the higher ambitions of assuring that my school practiced scholarly assessment, I would feel a bit disappointed by the final chapters, humming softly along with Peggy Lee, "Is that all there is?" In the end, it all seems to come down to commonsense precepts based on obtaining faculty engagement and—to be honest—three further...
Article
This monograph is part of a series on institutional support for student assessment. This report, which covers the second stage of the study, analyzes the results of a national survey of postsecondary education institutions that examined how institutions approach student assessment, their patterns of organizational and administrative support, and the uses and impacts of these efforts, focusing on undergraduate education. The survey instrument was based on a literature review and conceptual framework developed during the first stage. The survey population was all public and private postsecondary institutions that offer undergraduate programs at the associate or baccalaureate level. Analysis of the data suggest that student assessment is becoming a common practice in the academic management of U.S. postsecondary education, but it is not yet institutionalized or deeply embedded in institutional support patterns, policies, and practices. The researchers also conclude that the conceptual framework used appears useful, and that the survey instrument provides a checklist for institutions to examine their assessment approach. Appendixes include the survey instrument, the "Inventory of Institutional Support for Student Assessment"; correspondence related to the Inventory; institutional response rates by state; factor analysis results; and related reports, publications, and presentations. (Contains approximately 155 references.) (CH)
Article
Higher education's principal reason for being is seen as that of developing student talents. This premise suggests that a high-quality institution is one that maximizes the intellectual and personal development of its students. Student involvement is seen as the cornerstone of academic excellence. (MLW)
Article
A study investigated the appropriateness of the American College Testing Program's College Outcome Measures Program, conducted at the University of Tennessee, Knoxville, by applying the criterion of construct validity. Results indicated that while the test primarily measures individual differences, it is also sensitive to the effects of higher education. (Author/MSE)
Article
Responses to the College Student Experiences Questionnaire from 51,155 undergraduates at 128 institutions were analyzed to develop a student typology based upon their patterns of engagement in educationally purposeful activities and the progress they reported making toward important outcomes of college. The Intellectual, Individualist, Scientist, Conventional, and Collegiate groups were above average in terms of gains and the Disengaged, Grind, Recreator, Socializer and Artist were below average.
Article
In the 2002-2003 academic year, a series of roundtable discussions was conducted to explore uses of data from the National Survey of Student Engagement (NSSE) for the improvement of student learning. This paper reports on one roundtable, held in June 2003 in Seattle, Washington. Participants included campus officers responsible for student assessment and institutional research and others with experience using NSSE data. The paper summarizes the key points about uses of the data, effective communication strategies, and ideas for further uses of NSSE data. NSSE data are being used to assess the first year experience, assess graduating seniors, and assess student experiences by academic department and major. The data have been embedded in larger improvement initiatives that include accreditation reports and reviews, agendas for improvement, and faculty participation in improvement. Campus leaders are referring to NSSE data in their work, and survey participants have developed good communication strategies to emphasize the portions of NSSE data that are most useful for a designated purpose. Among the suggestions for future uses of NSSE data are: (1) develop a comparison group of institutions; (2) develop a strategy for oversampling; (3) pay close attention to response rates; and (4) enlist student interest in and support of the NSSE survey. (SLD)
Article
The original exposition of convergent-discriminant validation by D. T. Campbell and D. W. Fiske (see record 1960-00103-001) emphasized use of the multitrait-multimethod matrix as a procedure for examining the adequacy of tests and other methods for measuring traits. It paid less attention, however, to the set of concepts underlying the rationale for this procedure (e.g., content, concurrent, and predictive validity). The present author considers how the procedure has fared in the last 2 decades and examines the concepts' relationship to other validity constructs and role in the methodology of behavior research. (33 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Summarizes findings from the 1st year of the federally funded 3-yr longitudinal National Study of Student Learning begun in 1992. 18 4-yr and 5 2-yr postsecondary predominantly White or historically Black colleges participated. Focus was on the influence of academic and nonacademic experiences on student learning, attitudes about learning, cognitive development, and persistence. 3,840 students participated in the Fall 1992 data collection, and 2,685 of these Ss participated in the follow-up in Spring 1993. Findings highlight the interconnected influence of students' college experiences as they shape learning and point to a variety of curricular, instructional, out-of-class, and organizational climate variables that affect how students learn and develop. Implications for higher education policy and practice are discussed. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
construct validity (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
It is observed that many sorts of difficulties may preclude the uneventful construction of tests by a computerized algorithm, such as those currently in favor in Computerized Adaptive Testing (CAT). In this essay we discuss a number of these problems, as well as some possible avenues of solution. We conclude with the development of the “testlet,” a bundle of items that can be arranged either hierarchically or linearly, thus maintaining the efficiency of an adaptive test while keeping the quality control of test construction that is possible currently only with careful expert scrutiny. Performance on the separate testlets is aggregated to yield ability estimates.
Article
This article outlines a general process for faculty use in comparing the relative efficacy of college outcomes assessment instruments for gauging student progress toward goals considered important by the faculty. Analysis of two standardized general education exams—the ACT COMP and the ETS Academic Profile—illustrates the process.
Article
College and university administrators, as well as faculty members, are more likely to take responsibility for student learning if they believe that the assessment data represent their students and suggest specific actions for improvement. This study examined whether it is feasible to develop scalelets (i.e., focused measures, usually consisting of four or five items) that provide dependable metrics for assessing student learning at the college or department level. A generalizability analysis of 12 scalelets from the National Survey of Student Engagement (NSSE) indicated that the scalelets provided dependable measures of educational effectiveness with 25–50 respondents.
Article
This study evaluated the feasibility of using students' self reports of the academic development and college experiences as proxies and policy indicators for a national assessment of undergraduate education. Results indicated that self reports of academic development should be used with care as proxies for a national test. Particularly important are issues of content overlap and differences in the measurement methods. Results from the second part of this study supported using self reports of college experiences as policy indicators to improve undergraduate education.
Article
Thesis (Ph. D.)--State University of New York at Buffalo, 2003. Includes bibliographical references (leaves 185-202). Photocopy of typescript.
Involvement: The cornerstone of excellence A call for transformation Building a scholarship of assessment (pp. 284­292) Methods for evaluating assessment instruments
  • A W Astin
Astin, A. W. (1985). Involvement: The cornerstone of excellence. Change, 17(4), 35­39. Banta, T. W. (2002). A call for transformation. In T. Banta (Ed.), Building a scholarship of assessment (pp. 284­292). San Francisco: Jossey­Bass. Banta, T. W., & Pike, G. R. (1989). Methods for evaluating assessment instruments. Research in Higher Education, 30, 455­470.
Testing algorithms Computerized adaptive testing: A primer (pp. 103­136)
  • D Thissen
  • R L Mislevy
Thissen, D., & Mislevy, R. L. (1990). Testing algorithms. In J. Wainer (Ed.), Computerized adaptive testing: A primer (pp. 103­136). Hillsdale, NJ: Lawrence Erlbaum.
Student affairs administrators: Building collaborations with students and academic affairs for institutional improvement. Documenting effective educational practices: National roundtable series
  • A Kezar
Kezar, A. (2003, June). Student affairs administrators: Building collaborations with students and academic affairs for institutional improvement. Documenting effective educational practices: National roundtable series (Issue 3). Bloomington, IN: Indiana University Center for Postsecondary Research.
The components of construct validity: A comparison of two measures of general education outcomes
  • G R Pike
Pike, G. R. (1992). The components of construct validity: A comparison of two measures of general education outcomes. Journal of General Education, 41, 130150.
A call for transformation Building a scholarship of assessment (pp. 284­292)
  • T W Banta
Banta, T. W. (2002). A call for transformation. In T. Banta (Ed.), Building a scholarship of assessment (pp. 284­292). San Francisco: Jossey­Bass.
Faculty developers using the National Survey of Student Engagement (NSSE) to be change agents. Documenting effective educational practices: National roundtable series
  • A Kezar
Kezar, A. (2002, December). Faculty developers using the National Survey of Student Engagement (NSSE) to be change agents. Documenting effective educational practices: National roundtable series (Issue 1). Bloomington, IN: Indiana University Center for Postsecondary Research.
Student engagement: Pathways to collegiate success
  • Research
Pike Indiana University Center for Postsecondary Research. (2004). Student engagement: Pathways to collegiate success. Blooming­ ton, IN: Author.
Measurement issues in outcomes assessment
  • G R Pike
Pike, G. R. (2002). Measurement issues in outcomes assessment. In T. Banta (Ed.), Building a scholarship of assessment (pp. 131147). San Francisco: JosseyBass.
Involving colleges: Encouraging student learning and personal development through out-of-class experiences
  • G D Kuh
  • J H Schuh
  • E J Whitt
  • Associates
Kuh, G. D., Schuh, J. H., Whitt, E. J., & Associates. (1991). Involving colleges: Encouraging student learning and personal development through out-of-class experiences. San Francisco: JosseyBass.
Applications of generalizability theory in higher education assessment research
  • G R Pike
Pike, G. R. (1994). Applications of generalizability theory in higher education assessment research. In J. Smart (Ed.), Higher education: Handbook of theory and research (Vol. X, pp. 4587). New York: Agathon.
Building a scholarship of assessment
  • T W Banta
Banta, T. W. (2002). A call for transformation. In T. Banta (Ed.), Building a scholarship of assessment (pp. 284292). San Francisco: JosseyBass.
Methods for evaluating assessment instruments. Research in Higher Education
  • T W Banta
  • G R Pike
Banta, T. W., & Pike, G. R. (1989). Methods for evaluating assessment instruments. Research in Higher Education, 30, 455470.
Documenting effective educational practices: National roundtable series (Issue 5)
  • E Elkhawas
ElKhawas, E. (2003, August). Using NSSE data for assessment and institutional improvement. Documenting effective educational practices: National roundtable series (Issue 5). Bloomington, IN: Indiana University Center for Post secondary Research.