ArticlePDF Available

Abstract and Figures

Today, a major goal in higher education is the advancement of students’ vocational competences. To assess the extent to which this goal is met, both competences acquired during university studies and later vocational success need to be measured. In our study, we collected self‐ratings of competences (t1) and indicators of vocational success (t2) in 210 alumni of the Freie Universität Berlin. Using structural equation models along with this longitudinal data, we found that self‐ratings of competences accounted for a substantial proportion of the variance in different measures of vocational success five years later.
Content may be subject to copyright.
Assessment & Evaluation in Higher Education
Vol. 36, No. 4, July 2011, 417–427
ISSN 0260-2938 print/ISSN 1469-297X online
© 2011 Taylor & Francis
DOI: 10.1080/02602938.2010.534762
http://www.informaworld.com
Self-rated competences and future vocational success: a
longitudinal study
Edith M.P. Brauna*, Hammad Sheikhb and Bettina Hannovera
aSchul- und Unterrichtsforschung, Freie Universität Berlin, Habelschwerdter Allee 45, Berlin
14195, Germany; bNew School for Social Research, New York City, USA
Taylor and FrancisCAEH_A_534762.sgm10.1080/02602938.2010.534762Assessment & Evaluation in Higher Education0260-2938 (print)/1469-297X (online)Article2011Taylor & Francis3640000002011Dr. EdithBraunedith.braun@fu-berlin.de
Today, a major goal in higher education is the advancement of students’
vocational competences. To assess the extent to which this goal is met, both
competences acquired during university studies and later vocational success need
to be measured. In our study, we collected self-ratings of competences (t1) and
indicators of vocational success (t2) in 210 alumni of the Freie Universität Berlin.
Using structural equation models along with this longitudinal data, we found that
self-ratings of competences accounted for a substantial proportion of the variance
in different measures of vocational success five years later.
Keywords: self-rated competences; vocational success; learning outcome in
higher education; higher education reform; Bologna Process
Introduction
As a result of the Bologna Reform, today the advancement of vocational competences
is seen as the main goal of higher education (van der Wende 2000). Accordingly,
university teaching should move its focus from the transfer of knowledge to the
promotion of students’ vocational competences. In this context, the term ‘compe-
tences’ is used to denote skills related to solving authentic problems, thus encompass-
ing more than the acquisition of factual knowledge. Coatsworth and Masten defined
competence as ‘a pattern of effective adaptation in the environment, … broadly
defined in terms of reasonable success with major developmental tasks expected for a
person of a given age and gender in the context of his or her culture, society, and time
… It carries the dual meaning that there is a track record of such achievement (compe-
tent performance) and also that the individual has the capability to perform well in the
future. It refers to good adaptation and not necessarily to superb achievement’ (1998,
206). In a similar way, Weinert (2001) described competences as those cognitive abil-
ities, attitudes and skills which are at an individual’s command or can be learnt. This
definition includes motivational, volitional and social dispositions as well as the
specific skills necessary to solve problems in a responsible and successful way.
The acquisition of competences being a major goal of higher education is also
reflected in how educational quality is typically being measured today: students are
asked to describe the extent to which they have acquired competences in different
content domains during their university studies. In this study, we aim to show that
such ratings of gained competences are, in fact, predictive of students’ vocational
*Corresponding author now at HIS – Hochschul-Informations-System GmbH, Hannover,
Germany. Email: edith.braun@fu-berlin.de.
418 E.M.P. Braun et al.
success later on. Using self-ratings of competences of alumni of the Freie Universität
Berlin, we expect to show that a substantial part of the variance in alumni’s vocational
success can be explained by key competences that were acquired during the course of
higher education.
Domains of competences
In order to test whether competences acquired during university studies predicted
later vocational success, we needed to measure both competences and vocational
success.
According to the definitions given previously, the concept of competence encom-
passes expert knowledge in a given field in addition to specific skills of problem-
solving, as well as motivational and social aspects. Consistent with this view, Braun
and Leidner (2009) suggested four distinct domains of competences that higher
education is expected to foster: Knowledge Processing, Systematic Competence
(including problem-solving), Social Competence and Personal Competence (includ-
ing motivation):
(1) Knowledge Processing is the expansion of knowledge to applied and
performed skills (Dochy and Alexander 1995). Hence, it is more than factual
knowledge in that it emphasises the ability to put knowledge to use for solving
authentic and specific problems in a given field.
(2) Systematic Competence consists of skills related to planning and organising
one’s work effectively. As Rychen and Salganik put it, ‘using tools interac-
tively requires more than having access to the tool and the technical skills
required to handle it. Individuals also need to create and adapt knowledge and
skills’ (2000, 10).
(3) Social Competence is defined as the capability to achieve a balance between
positive outcomes for oneself and compliance to context-specific expectations
by others (see also Wentzel 1999). In other words, a socially competent person
is able to work effectively in teams and adhere to social norms.
(4) Personal Competence refers to a positive attitude towards learning and self-
development. As an outcome of higher education, students should develop a
reflexive and creative attitude towards learning (Braun, Soellner, and Hannover
2006; Hannover 1998).
The skills in these four domains are key competences that enable graduates to be
successful vocationally.
Vocational success
To test whether the competences acquired in these four key domains predicted success
in students’ vocational careers, we measured both objective and subjective aspects of
vocational success (Dette, Abele, and Renner 2004; Ng et al. 2005). Income and posi-
tion are regarded as objective criteria of vocational success (Judge et al. 1999). The
higher the income of people, the more successful they are regarded to be vocationally.
Similarly, the position people hold is often used as an indicator of their vocational
success. The measurement of objective criteria is relatively straight forward (e.g.
Assessment & Evaluation in Higher Education 419
assessing annual income or the number of people who are supervised by an individual).
In our study, we chose annual income as an indicator for objective vocational success.
In addition to this objective criterion, we operationalised subjective vocational
success as a match for the person’s individual preferences and the perceived charac-
teristics of his or her job (Gusy, Braun, and Scheffer 2005; Heslin 2005). In this line
of thinking, ‘being successful’ can be seen as achieving one’s goals. For instance,
people who attempted to achieve a balance of family and work and managed to realise
this goal are regarded as successful as people who intended and succeeded in finding
a particularly well-paid position (Gusy, Braun, and Scheffer 2005). Or people with a
low income may nevertheless be considered as successful if they have never aimed for
a well-paid position. Consistent with this point of view, Seibert and Kraimer
suggested that career success should be defined ‘as the accumulated positive work and
psychological outcomes resulting from one’s work experiences’ (2001, 2). In our
study, we defined ‘subjective vocational success’ as the match of an individual’s pref-
erences and the perceived characteristics of his or her job.
Individual job-related preferences can be assigned to broader categories of orien-
tation towards work. von Rosenstiel and Nerdinger (2000) suggested three types of
work orientation:
(1) Career Orientation: ‘Career-orientated’ individuals are interested in supervis-
ing others and being involved in making important decisions. They are likely
to work above the average working hours and usually attain a relatively higher
income.
(2) Leisure Orientation: Individuals in this category attached importance to a
good atmosphere at work and regulated working hours. Job security and the
possibility to achieve a balance between work and family are central charac-
teristics of their ideal job.
(3) Societal Orientation: For individuals having a societal orientation towards
work, the opportunity to exert political influence is substantial. The core ideas
of this orientation are creating an environment ‘worth living’ and contributing
to a collective good.
Based on this categorisation, we operationalised subjective vocational success as the
extent to which individuals’ preferences (e.g. societal orientation) match the subjec-
tively perceived characteristics of their job.
Method
Participants
All alumni who graduated in 1995 from the Freie Universität Berlin, Germany (N =
2970), were contacted five years after graduation. The questionnaire was delivered
and returned by letter post. In total, 621 envelopes were returned as undeliverable,
leaving 2349 valid addresses. Five hundred ninety-one alumni completed and returned
the questionnaires on competences that were acquired during their studies (return rate
of 25%). These 591 alumni were surveyed on their current vocational situation again
five years after initial contact. The sample of analysis consisted of the 210 individuals
who responded to this follow-up survey (return rate of 36%; 7% of the graduates
initially contacted).
420 E.M.P. Braun et al.
Measures
Competences
In the first survey, participants were given a list of competences and asked to indicate
to what extent they possessed each competence at the time of their graduation (Likert
scale ranging from 1= ‘not at all’ to 5 = ‘to a very great extent’). Combining the
competences of each of the four key domains suggested by Braun et al. (2008) (knowl-
edge processing, systematic competence, social competence and personal compe-
tence), four latent factors were estimated using confirmatory factor analysis. Each
factor was estimated using four to six items that described the respective competence.
For all four factors, reliabilities, as indicated by high internal consistencies, were
satisfactory. The competences and the internal consistency of the factors (ρ, as
suggested by Jöreskog et al. 1999) are as follows:
(1) Knowledge Processing (ρ = .73)
Knowledge of one’s own field
Knowledge of complex, social, organisational and technical systems
Analytical thinking
Critical thinking
(2) Systematic Competence (ρ = .76)
Using time efficiently
Performing well under pressure
Meeting a challenge
Asserting one’s authority
Coordinating activities
Working independently
(3) Social Competence (ρ = .75)
Negotiating with others
Working productively with others
Being loyal towards others
Managing co-workers
(4) Personal Competence (ρ = .79)
Willingness to question one’s own work
Creativity
Being proactive
Personal commitment
Ability to make decisions
Vocational success
Using the data of the follow-up survey, two measures of participants’ vocational
success were created: an ‘objective’ and a ‘subjective’ one. The objective measure
consisted of participants’ self-reported annual income. Annual income is typically
distributed unequally, i.e. a large majority of the population attains low earnings,
while a small minority attains high earnings. This leads to highly skewed data that
may produce overestimation or underestimation when calculating covariations with
Assessment & Evaluation in Higher Education 421
other variables. To address this issue, the annual income was mathematically trans-
formed using the log function, as suggested by Cohen et al. (2003). This transforma-
tion resulted in a more normally distributed variable, which allowed us to perform
analyses such as structural equation modelling.
The subjective measure was an indicator of how well participants’ current position
met the job characteristics that were valued by them. To obtain this measure, partici-
pants were provided with a list of job characteristics covering the three different types
of orientation towards work (career orientation, leisure orientation and societal orien-
tation), as suggested by von Rosenstiel and Nerdinger (2000). They were then asked
to indicate how much they valued each (from 1 = ‘not important at all’ to 5 = ‘very
important’) and how well their current position met it (from 1 = ‘not at all’ to 5 = ‘a
very large extent’) on five-point Likert scales. The job characteristics along with the
relevant vocational orientation read as follows:
(1) Career Orientation:
High earnings
Social status
Coordinating and supervising
(2) Leisure Orientation:
Concise and regulated working hours
Job security
Good chance to combine work with family tasks
(3) Societal Orientation:
Opportunity to exert political influence
Possibility to work with others
Opportunity to contribute to the collective good
If the current position offered a given job characteristic that was greater than or
equally valued by the participant, the respective characteristic was classified as an
indicator of ‘subjective vocational success’. This procedure resulted in nine dichoto-
mous indicators (e.g. desired level of job security achieved or not) that were used to
estimate a latent factor of ‘subjective vocational success’.
Analyses
To test our assumption that vocational success can be predicted from competence
ratings from five years earlier, we estimated regressions analyses using two structural
equation models (SEM): one for objective vocational success (Model A) and one for
subjective vocational success (Model B). A SEM consists of measurement models
and a structural model. Measurement models consider measurement errors and use
the actual responses to several items to estimate error-free latent constructs that they
are supposed to measure. For instance, the Likert scale items in our study (which
included measurement errors) measuring Career Orientation were used as indicators
for the true (i.e. error free) amount of the latent construct ‘subjective vocational
success’. The structural model revealed the relationships between error-free latent
constructs.
422 E.M.P. Braun et al.
Model A and its coefficients were calculated using a maximum likelihood estima-
tor; more precisely, MLR of software package MPlus 4.1 was used. Model B used an
approach that allowed categorical indicators. It was calculated via a weighted least
square estimator (in particular, the WLSMV of MPlus 4.1). Both estimators were
robust to violations of the normality assumption. Additionally, the missing data
handling techniques provided by MPlus were utilised (full-information maximum
likelihood for Model A; analysis with MAR covariates for Model B). Due to multicol-
linearity, values and algebraic signs of the beta weights could not be interpreted
(Cohen et al. 2003).
Results
Measurement models
Table 1 gives the measurement models of the latent competences. The factor load-
ings ranged from
λ
= .49 (‘knowledge of one’s own field’ on Knowledge Process-
ing) to
λ
= .76 (‘being proactive’ on Personal Competence). In total, the four areas
of competences can be regarded as sufficiently well measured by the given items.
Intercorrelations between latent competences
Consistent with theory (Braun et al. 2008), the four domains of competences were
interrelated. As seen in Table 2, the empirical correlations between self-rated
competence domains ranged from r = .560, p < .05 (for Knowledge Processing and
Social Competence) to r = .912, p < .05 (for Knowledge Processing and Personal
Competence). These intercorrelations lead to mutilcollinearity in our analysis, i.e.
the magnitude of the coefficients of single predictors and their standard errors can
be biased (Cohen et al. 2003). Therefore, parameters should be interpreted with
care. However, the focus of this paper was on the overall explanatory power of
Table 1. Measurement models of competences (confirmatory factor analysis with a maximum
likelihood estimator).
Knowledge
Processing
λ
Systematic
Competence
λ
Social
Competence
λ
Personal
Competence
λ
Knowledge of
one’s own field
.49 Ability to use
time efficiently
.60 Ability to
negotiate with
others
.71 Willingness to
question one’s
own work
.67
Knowledge of
complex, social,
organisational
and technical
systems
.59 Ability to
perform well
under pressure
.51 Ability to work
productively
with others
.67 Creativity .71
Analytical thinking .70 Ability to meet a
challenge
.56 Being loyal
towards others
.56 Being proactive .76
Critical thinking .74 Ability to assert
one’s authority
.66 Ability to manage
co-workers
.72 Personal
commitment
.57
Ability to
coordinate
activities
.65 Ability to make
decisions
.57
Ability to work
independently
.51
Assessment & Evaluation in Higher Education 423
competences for vocational success rather than on the power of specific predictors.
Fortunately, the estimation of the total explanatory power (R2) was not influenced
by multicollinearity and can therefore be trusted (Cohen et al. 2003).
Association between subjective and objective vocational success
The empirical correlation between income and attainment of job-related personal
goals was r = .29 (n.s.). Though not statistically significant, the coefficient indicated
that both measures of vocational success were associated but distinct.
Objective vocational success
In a longitudinal regression analysis (Model A), self-reported competences explained
9.7% of the variance in future income (R2 = .097). Given real data, the model was
(Marsh, Balla, and McDonald 1988) rejected by the
χ
2-test (
χ
2 = 398.753 and df =
161, p .0001). With a
χ
2/df ratio of 2.45, this model was acceptable according to
Backhaus et al. (2005). Also, SRMR (standardised root mean square residual) indi-
cated a good fit (with SRMR = .068) (Hu and Bentler 1999). In contrast, the close fit
indices, CFI (comparative fit index) = .834 and RMSEA (root mean square error of
approximation) = .083, suggested an inadequate fit. In summary, the fit indices were
unbalanced (see Table 3). Despite this fact, almost 10% of the variance in logged
income 10 years after graduation was explained by competences reported five years
after graduation. Considering that the annual income was not a latent construct (it was
not free from random error), the real amount of variance explained by competences
was likely to be greater.
Three of the four regression weights of competences on annual income were
negative. Knowledge Processing (
γ
= .24), Social Competence (
γ
= .21) and
Personal Competence (
γ
= .16) were related to income in an unexpected direction,
Table 2. Intercorrelations of competences.
Knowledge
Processing
Systematic
Competence
Social
Competence
Knowledge Processing
Systematic Competence .799*
Social Competence .560* .652*
Personal Competence .912* .835* .701*
*p < .05.
Table 3. Model A: structural equation model predicting income from self-rated competences.
Beta
Knowledge Processing .24
Systematic Competence .35
Social Competence .21
Personal Competence .16
Explained variance .097
424 E.M.P. Braun et al.
i.e. participants who indicated high competences in these domains seemed to be less
successful in their job. Only Systematic Competence was positively related with
future salary (
γ
= .35). As we have pointed out, the competence scales were intercor-
related, which lead to multicollinearity. Thus, the negative signs and values of param-
eters could be biased, but the total explanatory power was unbiased (Cohen et al.
2003).
Subjective vocational success
Compared with Model A (predicting objective vocational success), in Model B (see
Table 4) (predicting subjective vocational success), self-rated competences explained
almost twice of the variance in vocational success (R2 = 18.8%). As in Model A, indi-
ces showed inconsistent results considering the fit of the model. While a
χ
2/df ratio of
1.69 indicated a very good fit to the data (Backhaus et al. 2005; Tabachnik and Fidell
2004), CFI = .821, RMSEA = .058 and WRMR (weighted root mean square residual)
= .931 (weighted root mean square was an additional index for models with categori-
cal indicators which was expected to be smaller than .9 for fitting models) indicated
an unsatisfactory fit. Despite the inconsistency in the fit indices, the explained vari-
ance of almost 19% implies that a substantial part of the variance in vocational success
could be explained by self-rated competences assessed five years before.
Mirroring the results of Model A, Knowledge Processing (
γ
= .32) and Personal
Competence (
γ
= .32) were negatively related to subjective vocational success.
Again, Systematic Competence seemed to be of high importance for future subjective
success (
γ
= .81), while Social Competence was not related to subjective success
(
γ
= .05) at all.
Discussion
In this study, we asked alumni who had graduated from the Freie Universität Berlin
five years before to describe key competences they had acquired at the end of their
university studies and related their self-ratings to indicators of vocational success
measured five years after the first survey. Results showed that self-rating measures of
competences gained in the course of higher education predicted future vocational
success. Considering the five-year time lag between the measurement of the predictors
and the outcome measures, the model explains a considerable amount of the variance.
This study examined two different kinds of vocational success: objective success
operationalised by annual income, and subjective success conceptualised by the
achievement of individual goals in the current occupation. The latter was especially
Table 4. Model B: structural equation model predicting subjective vocational success from
self-rated competences.
Beta
Knowledge Processing .32
Systematic Competence .81
Social Competence .05
Personal Competence .32
Explained variance .188
Assessment & Evaluation in Higher Education 425
related to alumni’s self-ratings of the competences they had obtained at the time of
their university graduation which dated back 10 years.
While most empirical research on the success of higher education have used self-
ratings as measures of educational outcome (Allen and van der Velden 2005; Schaeper
2009; Schomburg and Teichler 2006), self-ratings have been criticised by some schol-
ars for not being objective (e.g. Ward, Gruppen, and Regehr 2002). Our findings
suggest that self-ratings of competences are, in fact, valid measures since they
predicted relevant outcomes. Leaving all other well-known predictors, such as familial
and educational context, age or sex, aside, in SEMs, self-ratings of competences could
account for 9.7% of the variance in income and 18.8% of the variance in the degree
of overlap between personal values and their attainment in the current occupational
position five years later.
Although other biographic variables such as age and sex are well known for being
important in vocational success, we have left such information aside to be able to get
an impression of the power of self-rating competences.
Consistent with theory, the four domains of competence measured in our sample
were highly interrelated (Braun et al. 2008; Weinert 2001). Unfortunately, these inter-
relations made it difficult to disentangle the effects of the predictors mathematically,
possibly resulting in biased estimates. Therefore, the negative parameters that our
analysis yielded should be interpreted with care. Future research should investigate
whether the specific effects can be replicated in other samples.
A possible interpretation of the negative effects is the sequence in which the data
had been collected. The first survey was conducted five years after the participants’
graduation. Thus, it is somewhat unclear whether the reported competences reflect the
actual level at graduation or whether they were also influenced by the occupational
status and situation at the time of the first survey. The negative weights of self-ratings
of competences on later vocational success might reflect underlying cognitive
processes that might determine how people judge their competences retrospectively.
For example, individuals who are more successful in the present might have a more
negative perception of the competences they possessed in the past. Unfortunately, we
were not able to investigate this matter with our data.
Future research should obtain additional measures in order to gain insight into how
cognitive processes relate to the perception of competences and vocational success.
The overall predictive power of self-rated competences for future vocational success
can be interpreted, however, independent of an understanding of these underlying
processes. Thus, our data unequivocally indicated that self-rated competences are
correlated with indicators of later vocational success. Future research should look into
the relationships between other established predictors of vocational success and
specific domains of competences as well as into the cognitive processes underlying
the perception and evaluation of one’s competences and vocational success.
Notes on contributors
Edith M.P. Braun is the scientific director of the Hochschul-Informations-System GmbH in
Hannover. Her research interests relate to quality assurance in higher education and to teach-
ing evaluation. In 2008, she was awarded the ‘Ulrich Teichler Preis der Gesellschaft für
Hochschulforschung’.
Hammad Sheikh is a PhD student of social psychology at the New School for Social Research
(NYC), and is involved in research on the role of values for educational success and intergroup
426 E.M.P. Braun et al.
conflicts. Further, he is interested in the resilience of children in the face of adversity, such as
intergroup conflicts and racial discrimination.
Bettina Hannover is a professor for school and teaching research at the Freie Universität Berlin.
In her research, she focuses on the cognitive mechanisms underlying the processing of self-
related information, on social and cultural influences on the self, and the self’s impact on the
development of competencies, scholastic interests and occupational career motivation.
References
Allen, J., and R. van der Velden. 2005. The role of self-assessment in measuring skills. REFLEX
working paper. Maastricht: Research Centre for Education and the Labour Market.
Backhaus, K., B. Erichson, W. Plinke, and R. Weiber. 2005. Multivariate Analysemethoden.
11th ed. Berlin: Springer.
Braun, E., B. Gusy, B. Leidner, and B. Hannover. 2008. Kompetenzorientierte Lehrevalua-
tion: Das Berliner Evaluationsinstrument fuer studentische Kompetenzen. Diagnostica 54:
30–42.
Braun, E., and B. Leidner. 2009. Academic course evaluation: Theoretical and empirical
distinctions between self-rated gain in competences and satisfaction with teaching behav-
ior. European Psychologist 14: 297–306.
Braun, E., R. Soellner, and B. Hannover. 2006. Ergebnisorientierte Lehrveranstaltungs evalu-
ation. In Qualitaetsentwicklung an Hochschulen – Erfahrungen und Lehren aus 10 Jahren
Evaluation, ed. Hochschulrektorenkonferenz (HRK), 60–7. Bonn: HRK.
Coatsworth, J.D., and A. Masten. 1998. The development of competence in favorable and
unfavorable environments: Lessons from research on successful children. American
Psychologist 53: 205–20.
Cohen, J., P. Cohen, S.G. West, and L.S. Aiken. 2003. Applied multiple regression/correla-
tion: Analysis for the behavioural sciences. Hillsdale, NJ: Lawrence Erlbaum.
Dette, D.E., A.E. Abele, and O. Renner. 2004. Zur Definition und Messung von Berufser-
folg: Theoretische Überlegungen und metaanalytische Befunde zum Zusammenhang von
externen und internen Laufbahnerfolgsmaßen. Zeitschrift für Personalpsychologie 3:
170–83.
Dochy, F.J.R.C., and P.A. Alexander. 1995. Mapping prior knowledge: A framework for
discussion among researcher. European Journal of Psychology of Education 10: 225–42.
Gusy, B., E. Braun, and S. Scheffer. 2005. Berufseinstieg und Berufserfolg. Ergebnisse einer
Absolventenbefragung der Jahrgänge 1996 bis 2000 an der Freien Universität Berlin.
Empirische Paedagogik 19: 265–80.
Hannover, B. 1998. The development of self-concept and interests. In Interest and learning,
ed. L. Hoffmann, A. Krapp, K.A. Renninger, and J. Baumert, 105–25. Kiel: IPN.
Heslin, P. 2005. Conceptualizing and evaluating career success. Journal of Organizational
Behavior 26: 113–36.
Hu, L.T., and P.M. Bentler. 1999. Cut off criteria for fit indexes in covariance structure analy-
sis: Conventional criteria versus new alternatives. Structural Equation Modeling 6: 1–55.
Jöreskog, K.G., D. Sörbom, S. du Toit, and M. du Toit. 1999. LISREL 8: New statistical
features. Chicago, IL: Scientific Software International.
Judge, T.A., C.A. Higgins, C.J. Thoreson, and M.R. Barrick. 1999. The Big Five personality
traits, general mental ability, and career success across the life span. Personnel Psychology
52: 621–52.
Marsh, H.W., J.R. Balla, and R.P. McDonald. 1988. Goodness of fit in confirmatory factor
analysis: The effect of sample size. Psychological Bulletin 103: 391–410.
Ng, T.W.H., L.T. Eby, K.L. Sorensen, and D.C. Feldman. 2005. Predictors of objective and
subjective career success: A meta-analysis. Personnel Psychology 58: 367–408.
Rychen, D.S., and L.H. Salganik. 2000. Definition and selection of key competences: A report
to the INES General Assembly. Neuchatel: OECD.
Schaeper, H. 2009. Development of competencies and teaching–learning arrangements in
higher education: Findings from Germany. Studies in Higher Education 34: 677–97.
Schomburg, H., and U. Teichler. 2006. Higher education and graduate employment in
Europe: Results of graduates surveys from 12 countries. Dordrecht: Springer.
Assessment & Evaluation in Higher Education 427
Seibert, S.E., and K.L. Kraimer. 2001. The five-factor model of personality and career
success. Journal of Vocational Behavior 58: 1–21.
Tabachnik, B.G., and L.S. Fidell. 2004. Using multivariate statistics. Boston, MA: Allyn &
Bacon.
van der Wende, M.C. 2000. The Bologna Declaration: Enhancing the transparency and
competitiveness of European higher education. Higher Education in Europe 25: 205–310.
von Rosenstiel, L., and F.W. Nerdinger. 2000. Die Münchner Wertestudien – Bestand-
saufnahme und (vorläufiges) Resümee (The Munich Value Study – preliminary resumé).
Psychologische Rundschau 51: 146–57.
Ward, M., L. Gruppen, and G. Regehr. 2002. Measuring self-assessment: Current state of the
art. Advances in Health Sciences Education 7: 63–80.
Weinert, F.E. 2001. Concept of competence: A conceptual clarification. In Defining and
selecting key competences, ed. D. Rychen and L. Salganik, 17–31. Goettingen: Hogrefe &
Huber.
Wentzel, K.R. 1999. Social-motivational processes and interpersonal relationships: Implica-
tions for understanding students’ academic success. Journal of Educational Psychology
91: 76–97.
Copyright of Assessment & Evaluation in Higher Education is the property of Routledge and its content may
not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written
permission. However, users may print, download, or email articles for individual use.
... Graduates' career success has been explored in many studies (Braun, Sheikh, & Hannover, 2011;Vermeulen & Schmidt, 2008). Career success can be divided into objective and subjective career success (e.g. ...
... There is evidence that self-assessments of the development of competences during university studies are related to graduates' career success or job satisfaction (Baruch & Peiperl, 2000;Braun, Sheikh, & Hannover, 2011;Mora, Garcia-Aracil, & Vila, 2007;Semeijn et al., 2006;Vermeulen & Schmidt, 2008). More precisely, systematic competences such as time management skills and the ability to organise work effectively are related to career success (Braun, Sheikh, & Hannover, 2011). ...
... There is evidence that self-assessments of the development of competences during university studies are related to graduates' career success or job satisfaction (Baruch & Peiperl, 2000;Braun, Sheikh, & Hannover, 2011;Mora, Garcia-Aracil, & Vila, 2007;Semeijn et al., 2006;Vermeulen & Schmidt, 2008). More precisely, systematic competences such as time management skills and the ability to organise work effectively are related to career success (Braun, Sheikh, & Hannover, 2011). In addition, problem-solving skills and strategic thinking are related to career satisfaction (Van Dierendonck & Van der Gaast, 2013). ...
... Liu and Wei (2022) mentioned the third-level indicator of occupational ethics in their evaluation system for the professional competence of masters in library and information science. Braun et al. (2011) referred to first-level indicators such as personal abilities when assessing whether higher education enhances students' vocational capabilities, measuring the skills acquired during university studies and subsequent career success. In our research, we consider all these as second-level indicators. ...
... They should consciously and purposefully concentrate on cultivating essential qualities for students to adapt to the workplace. The weights of personal skills/ abilities (B2) (27.61%) and character qualities (B4) (25.28%) are not significantly different, indicating that for master's students majoring in educational technology, as evaluated by Braun et al. (2011), personal skills/abilities are generally important in the future employment process. Universities should provide more practical opportunities to enhance their practical skills during education. ...
Article
Full-text available
Purpose The purpose of this study is to determine the competency characteristics required for the employment of master’s degree students in educational technology. Design/methodology/approach A combined qualitative and quantitative method was used to consult multiple experts through a modified Delphi method. Competency characteristics were extracted from Chinese recruitment apps, national recruitment websites and university training programs. Ten senior teacher experts who teach educational technology master’s students were consulted through a questionnaire consultation to validate the proposed competency model. The weights of competency characteristics were determined through a combination of the analytic hierarchy process and entropy method. Findings The results show that when recruiting educational technology master’s students, more emphasis is placed on operational skills. The majority of companies tend to assess practical abilities rather than theoretical knowledge. Relevant knowledge of educational technology, psychology, computer science and education is considered to be the basic knowledge components of educational technology master’s students, while professional skills are the core skills required for their positions. Therefore, universities need to focus on training, educational technology graduate students in these areas of competence. The study also found that professional qualities (such as physical and mental fitness) and personality traits (interpersonal communication and interaction) receive more attention from companies and are essential competencies for educational technology master’s students. Originality/value A competence model for educational technology master’s students is proposed, which includes aspects such as knowledge, personal skills/abilities, professional qualities and personality traits. The competence elements included in this model can serve as reference indicators for universities to cultivate the competence of educational technology master’s students, as well as reference points for recruiting units to help them select talents. This represents a new dimension in research related to the employment of educational technology master’s students. The study enriches the research objects and competence dictionary in the field of competence research.
... Survey participants often overestimate or report the "right" answer as opposed to the facts. Respondents often do not take the time to read the questions fully and may rush through Likert scales, ticking the boxes to finish quickly [35]. ...
... Self-reporting can also introduce social desirability bias where answers reflect an anticipated social norm, as the participants presume about their knowledge which, they do not possess [36]. Authors have recognized and demonstrated the difference between students' perceptions or feelings about their own knowledge and skills and their actual knowledge and skills as measured, for example, by tests or assessments administered by the institution [35] [37] [38]. ...
... There have been many studies on career success among graduates (Braun, Sheikh, and Hannover 2011;Vermeulen and Schmidt 2008;Tuononen, Parpala, and Lindblom-Ylänne 2019), which could be considered an indicator of successful transition (Tuononen 2019). Two types of career success have been described, objective and subjective (e.g., Van Dierendonck and Van der Gaast 2013). ...
... There is a lot of evidence that graduates' skills and competences are related to career success or job satisfaction (Braun, Sheikh, and Hannover 2011;Tuononen, Parpala, and Lindblom-Ylänne 2019;Vermeulen and Schmidt 2008). Tuononen, Parpala, and Lindblom-Ylänne (2019) identified several reasons why graduates had difficulties finding employment after graduation: poor employment opportunities in the field, inadequate networks, a lack of work experience, the subjects included in their degrees, uncertainty about their own competences and a lack of clarity about their own goals. ...
Article
The transition from university to working life is a challenging phase for graduates. The focus in the present longitudinal study is on employability factors and their association with this transition and with early career success. The participants were 43 graduates who were interviewed at the time of their graduation and filled in a follow-up questionnaire three years later. The data were analysed using qualitative content analysis. The results revealed five employability factors relating early career success: (1) career plans and goals, (2) perceived competences related to the degree, (3) self-efficacy beliefs, (4) activity and (5) work experience and networks. Three transition groups emerged based on the differences in employability factors and career success, which we labelled smooth transition, progressive transition and a rocky road. The results revealed individual variation in employability factors and in the kind of challenges these graduates encountered in the transition phase and in their early career. An awareness of the ways in which graduates differ could help educators to develop practises that better support students and graduates in the transition to working life. These findings highlight the importance of active career planning during one’s studies.
... The term research competence is commonly used to address research skills and the ability to exercise these skills in carrying out research activities. It can be conceptualized into a three-dimensional model, comprising knowledge, skill, and attitude, although the distinction among these dimensions might be elusive [25][26][27]. ...
Article
Full-text available
Action research (AR), which involve rigorous cycles of implementing and reflecting on educational changes, brings genuine values to language teaching. However, due to several factors related to school cultures and the teacher-researchers, AR does not always live up to its expectations. Generally, previous studies have steered towards instigating an awareness of AR at the expense of obtaining empirical evidence on competencies and conditions for AR. This study examines teachers’ action research competencies and challenges in doing AR. A questionnaire (70 items, α = 0.809) was administered to 105 EFL teachers in various high schools in Vietnam. Among these participants, eight partook in follow-up semi-structured interviews. Chi-square analysis confirmed that top-rated competencies aligned with the traditional roles of high school English teachers, including giving feedback, using visual aids for presentations, and self-assessment. In contrast, teachers were least sure of their research skills and techniques. Thematic analysis complemented the quantitative data, ascribing the underdeveloped research capacity to a lack of guidelines, motivation, and resources. Consequently, the practice of AR might be improved with the cultivation of a teacher research culture. Other measures to incentivize high school EFL teachers in research engagement are also discussed in this paper.
... Although selfratings of knowledge are widely used in the literature, research is inconclusive on the matter of how valid and reliable such measures are. Some authors [65,66] advocate that individuals generally have accurate perceptions of their knowledge, while others disagree [67]. Still, reporting bias will be equally problematic (or non-problematic) for both types of knowledge. ...
Article
Full-text available
Background: Alcohol consumption is a major public health challenge; the majority of employees consume alcohol regularly and a considerable proportion of employees can be characterized as risky drinkers in need of interventions. Occupational health services (OHS) are uniquely positioned for implementing alcohol prevention interventions targeting employees, but rarely do so. Studies have shown that lack of knowledge among OHS personnel is a barrier to alcohol prevention activity. This study aimed to explore OHS personnels' levels of theoretical and practical alcohol knowledge, and whether these two ways of knowing were differentially associated with alcohol prevention activity. Methods: In this cross-sectional study, survey data were collected from 322 OHS personnel in Norway in 2018 (response rate = 53.6%). The survey included variables of two ways of knowing (theoretical and practical) and three types of doing (intervention frequency, conducting individual interventions, and conducting group interventions). Data were analyzed with descriptive statistics, paired sample t-tests, bivariate correlations, and adjusted linear and logistic regression analyses. Results: OHS personnel rated their theoretical alcohol knowledge higher than their practical knowledge (η2 = 0.33, p < 0.001). Higher reported levels of practical knowledge were associated with higher intervention frequency (b = 0.39, β = 0.60, p < 0.001) and greater likelihood of conducting individual interventions (OR = 1.60, p < .001) as well as group interventions (OR = 1.84, p < 0.001). Theoretical knowledge was not associated with conducting interventions, and there was no evidence of an interaction between the two ways of knowing in their association with doing. Sensitivity analyses did not indicate clustering effects of OHS personnel being employed within different units. Conclusions: Different ways of knowing about alcohol among OHS personnel were dissimilarly associated with conducting alcohol prevention interventions in occupational health settings. For doing, knowing how seems to be more important than knowing that. Training programs for OHS personnel should emphasize knowledge about how to deal with alcohol-related issues and how to conduct prevention interventions, rather than focus on detrimental effects of alcohol.
... According to Tomlinson (2017), human capital refers to the knowledge and skills which graduates acquire and which are a foundation of their labour market outcomes. The role of generic competencies or skills, such as critical thinking, problem solving, communication and communication skills is emphasised in successful transition and career success (Braun, Sheikh, and Hannover 2011;Tuononen, Parpala, and Lindblom-Ylänne 2019). On the other hand, there is evidence that there is a gap between the skills needed in the workforce and developed during studies (Andrews and Higson 2008;García-Aracil and Van der Velden 2008;Tuononen, Parpala, and Lindblom-Ylänne 2019). ...
Article
Full-text available
An unprecedented number of autistic people are completing university and they frequently face unemployment after graduation. However, research focusing on the forms of graduate capital and their employability is scarce. The focus of existing research has been on non-autistic, or neurotypical, graduates. The human, social, cultural, identity and psychological capital might be different for autistic graduates due to the characteristics of autism. Using a participatory approach, our aim was to examine the five areas of graduate capital in the context of autistic graduates. The study involved semi-structured interviews with 15 autistic university graduates from England, Finland, France and the Netherlands. Data were analysed using theory guided content analysis and ‘data-driven’ approaches. Findings indicate that the five areas of graduate capital are particularly relevant to autistic graduates, who typically expose gaps in several capital, jeopardising their employability. © 2022 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.
... Aunque seamos conscientes de las limitaciones que ello conlleva (Braun, Woodley, Richardson y Leidner, 2012), sabemos que trabajar con los estudiantes sobre sus percepciones acerca del nivel de logro de sus competencias es un modo de involucrarle directamente en su propia evaluación y de potenciar la reflexión sobre la misma. Esta opinión, junto con la del profesorado, permite evidenciar si las calificaciones del alumnado son coherentes con el grado de desarrollo de las competencias (Goodman, Arbona y Domínguez, 2008), ya que, si bien esta relación debería darse en todas las ocasiones (Braun, Sheikh y Hannover, 2011), no siempre ocurre así (Lopes, Branco y Jimenez-Aleixandre, 2011), puesto que prevalece una sobreestimación de los conocimientos adquiridos (Baartman y Ruijs, 2011). ...
Article
Full-text available
Resumen Tras las profundas modificaciones experimentadas por las instituciones educativas universitarias en los últimos años, se declara al estudiante como el mayor protagonista de su propio proceso formativo, lo que le promulga como agente activo de su aprendizaje. Dada la importancia que tiene la idoneidad de las competencias que los alumnos han de adquirir durante su paso por la universidad, así como el nivel de logro alcanzado en ellas, con el presente estudio se pretende evaluar y contrastar la percepción del profeso-rado y del alumnado en relación con la pertinencia de los conocimientos que conforman el grado en Pedagogía de la Universidad de Murcia, así como el nivel en que se han logrado, tanto de forma global como en función del curso en que se encuentran matri-culados los estudiantes. A la luz de los resultados analizados se concluye que, en algunos casos, las opiniones de alumnos y docentes coinciden en cuanto a la pertinencia de las competencias, pero no en relación con el nivel de logro de las mismas. Únicamente los estudiantes de último curso, a nivel global, demuestran ideas concordantes con el pro-fesorado en cuanto al grado competencial adquirido. Palabras clave: educación; educación superior; alumnado; profesorado; modelo de for-mación competencial
... At present, the competence model of accounting vocational education is constructed in the methods such as derivation, citation revision, and induction, etc., but the general competency model has too many factors, not always in line with the actual accounting industry [8]. Some research scholars constructed the competency model system of accounting vocational education from the four aspects: accounting tasks, task environment, specific performance standards, and ability to accomplish performance standards [9]. In light of the construction and application of competence model, this paper explores the competency of accounting practitioners under the accounting vocational education. ...
Article
Full-text available
Accounting vocational education directly bears on the competence of accountants, and affects the development of the accounting industry. In light of the back-ground of vocational education, this paper develops a competence model of ac-counting vocational education in China, applies the established model in an actual case, and puts forward measures to improve the competence of accounting voca-tional education. The research results show that: the evaluation index system of accounting vocational education should highlight importance, incentives and ef-fectiveness; the management and leadership ability is the primary factor of the competence of accounting vocational education; an excellent accounting vocation-al educator must enjoy outstanding management and leadership ability, account-ing ability and accounting skills; the competence of accounting vocational educa-tion should be improved mainly through incentive measures, from the perspective of adjusting the education purpose. The research findings lay the theoretical basis for self-evaluation, self-management and career planning of accounting vocational educators and accountants.
... In this context, data is a measured value concerning an objective measured phenomenon where human individuals will not be used as a measuring instrument. Therefore (self)-evaluations can not be seen as real data -rather a sort of "semi-or pseudo-data" (Braun et al. 2011; Roszyk-Kowalska, 2016) ...
Article
Full-text available
Competency-based education, as a representative concept addressing the educational and employee development challenges, encompasses multiple theoretical and pragmatic approaches with various scopes of analyses. In this research, the reader is tried to introduce to a data-driven[1]or big-data-based method. The goals of this research are to investigate how behaviour patterns related competencies can be identified and measured using the strength of log file and records, and to develop a professional system that is able to foresee training needs based on similarity analyses enabling artificial intelligence-based term-creation processes.
Article
Full-text available
The conceptualisation and definition of vocational success is equivocal. The present paper attempts to clarify the construct. We suggest a model that distinguishes vocational success with respect to three parameters: 1) the criterion to be assessed with the extremes of specific work and task vs. global career, 2) the data content with the distinction of external records, comparative judgements and satisfaction ratings, and 3) the data source, i.e., documents and ratings by others (both external measures) and self-report data (internal measures). We then take a closer look at career success and give an overview of current operationalizations with respect to data source and data content. Subsequently, we report findings of a metaanalysis on the relationship between external measures and internal measures of career success. The analysis yields an estimated population correlation of ρest = .28, which is homogeneous for the correlation of external measures and career satisfaction (ρest = .30), but not for external measures and self-rated career success. Suggestions for the measurement of career success are given. Key words: vocational success, career, work, satisfaction, external measures, internal measures, self report, meta-analysis
Article
Full-text available
This article contributes to the conceptual and empirical distinction between (the assessment of) appraisals of teaching behavior and (the assessment of) self-reported competence acquirement within academic course evaluation. The Bologna Process, the current higher-education reform in Europe, emphasizes education aimed toward vocationally oriented competences and demands the certification of acquired competences. Currently available evaluation questionnaires measure the students’ satisfaction with a lecturer’s behavior, whereas the “Evaluation in Higher Education: Self-Assessed Competences” (HEsaCom) measures the students’ personal benefit in terms of competences. In a sample of 1403 German students, we administered a scale of satisfaction with teaching behavior and the German version of the HEsaCom at the same time. Using confirmatory factor analysis, the estimated correlations between the various scales of self-rated competences and teaching behavior appraisals were moderate to strong, yet the constructs were shown to be empirically distinct. We conclude that the self-rated gains in competences are distinct from satisfaction with course and instructor. In line with the higher education reform, self-reported gains in competences are an important aspect of academic course evaluation, which should be taken into account in the future and might be able to restructure the view of “quality of higher education.” The English version of the HEsaCom is presented in the Appendix.
Article
Full-text available
Zusammenfassung. Im Rahmen des Bologna-Prozesses werden europaweit Bachelor-Studiengange eingefuhrt, deren Abschlusse berufsqualifizierend sind. Die beteiligten Lander haben sich auf einen Qualifikationsrahmen, d.h. auf eine Liste von studienfachunspezifisch formulierten Kompetenzen verstandigt, die in Lehrveranstaltungen vermittelt werden sollen. Inwieweit diese Kompetenzen tatsachlich von den Studierenden erworben werden, ist empirisch nachzuweisen. Bisherige Lehrveranstaltungsevaluationsinstrumente konnen zu diesem Zweck nur bedingt genutzt werden, denn sie messen eher den Prozess als das Ergebnis einer Veranstaltung. Deshalb haben wir ein Instrument entwickelt, das in sechs Subskalen mit insgesamt 29 Items den selbsteingeschatzten Zuwachs an Fach-, Methoden-, Prasentations-, Kommunikations-, Kooperations- sowie Personalkompetenz erhebt. In einer Stichprobe mit insgesamt 2507 Fragebogen wurde das Instrument entlang der klassischen Testtheorie uberpruft. Die Ergebnisse zeigen gute Reliabilitaten und die...
Article
Social-motivational processes and socialization experiences can play a critical role in students' academic success. However, the search for specific mechanisms and processes that explain these social influences on motivation is still in its inception. The purpose of this article was to begin to articulate some of these processes in the hope that more precise explanations of influence will emerge. The Ist section of the article focuses on ways in which social-motivational processes are relevant for understanding motivation to achieve academically, using goal pursuit as a case in point. Models describing complementary, developmental, and hierarchical relations among social and task-related goals and their implications for understanding student achievement are presented. Then, ways in which students' social encounters and experiences with parents, teachers, and peers might influence their adoption and internalization of socially valued goals are examined. New directions for theoretical and empirical inquiry are presented.
Article
The Bologna Process places special emphasis on the outcomes of higher education in terms of employability and key competencies. Taking Germany as an example, this article examines whether the introduction of a two‐tier degree structure actually has led to an enhanced acquisition of key competencies. Based on constructivist learning theories, in addition, the article tests the hypothesis that an activating learning environment enhances the acquisition of both disciplinary and key competencies. Data are used from a survey among higher education graduates from 2005. The results of linear regression analyses suggest that the new bachelor programmes provide better conditions for developing key competencies, and also indicate what the ingredients of a competence‐oriented teaching approach might be.
Article
Zusammenfassung. Ausgangspunkt der Munchner Wertestudien war das Phanomen eines gesellschaftlichen Wandels von materialistischen zu postmaterialistischen Werten. Demnach war zu erwarten, das sich Studenten und Studentinnen aus Studienzweigen, die fur Fuhrungspositionen qualifizieren, weniger mit den Zielen von Unternehmen identifizieren als die Unternehmen von ihnen erwarten und ihre berufliche Zukunft weniger in den herkommlichen Karrierebahnen planen. Diese Annahmen konnten in einer Reihe empirischer Studien - durchgefuhrt auf der Basis qualitativer und quantitativer Methoden im Quer- und Langsschnittdesign - teilweise bestatigt werden. Daruber hinaus zeigte sich, das die Integration in die Unternehmen auch von der Moglichkeit, personliche Ziele zu realisieren, abhangt.