Conference PaperPDF Available

Competency Evaluation Applied to Human Resource Development Program Designed to Solve Social Issues using Space Applications

Authors:

Abstract and Figures

Who are you? This is a crucial question when one wants to assert himself/herself in a global competitive environment. Characteristics of individuals differentiated from others are affected by 4 key components; (1) Values: foundation of how one finally decide, (2) Skills: technical expertise and language proficiency, (3) Knowledge: information and experience accumulated and (4) Competencies: soft skills succeeding as an influential person. In this paper, among the 4 elements, we emphasize the importance of "competency" and its concept adapted to the G-SPASE (Geo-spatial and SPAce technology for Society and Earth) program which focuses on the social service reconstruction for more efficient use of global scale information obtained from space and geospatial information infrastructures. Especially, in space exploitation where competencies beyond skills and knowledge are imperative to solve social issues using space applications, it is vital to measure competencies and design an education program that fosters the development. It is critical to investigate the balance of competencies required, which may vary by the situation such as the target issues, solutions and social roles the person is dealing with. We designed and verified the method of competency evaluation and had program participants use the GROW online system 1) to measure their competencies. The approach was based on self-evaluation as well as peer-evaluation. The 1st competency evaluation was conducted between June to July and the 2nd survey was carried out between December and January, where the results were distributed in August (2016) and February (2017), respectively. Here, we compare and evaluate how the 25 competencies changed during the time being and describe how competency-based education assists individuals to reflect on their personal traits and discover their blind self to pave their journey.
Content may be subject to copyright.
1
Competency Evaluation Applied to Human Resource Development Program
Designed to Solve Social Issues using Space Applications
By Aria Iwasawa 1*), Masahiro Fukuhara 2), Fabien Roudier 2) and Naohiko Kohtake 1)
1) Graduate School of System Design and Management, Keio University, Japan
2) Institution for a Global Society
(Received June 21st, 2017)
Who are you? This is a crucial question when one wants to assert himself/herself in a global competitive environment.
Characteristics of individuals differentiated from others are affected by 4 key components; (1) Values: foundation of how
one finally decide, (2) Skills: technical expertise and language proficiency, (3) Knowledge: information and experience
accumulated and (4) Competencies: soft skills succeeding as an influential person. In this paper, among the 4 elements, we
emphasize the importance of “competency” and its concept adapted to the G-SPASE (Geo-spatial and SPAce technology
for Society and Earth) program which focuses on the social service reconstruction for more efficient use of global scale
information obtained from space and geospatial information infrastructures. Especially, in space exploitation where
competencies beyond skills and knowledge are imperative to solve social issues using space applications, it is vital to
measure competencies and design an education program that fosters the development. It is critical to investigate the balance
of competencies required, which may vary by the situation such as the target issues, solutions and social roles the person is
dealing with.
We designed and verified the method of competency evaluation and had program participants use the GROW online
system 1) to measure their competencies. The approach was based on self-evaluation as well as peer-evaluation. The 1st
competency evaluation was conducted between June to July and the 2nd survey was carried out between December and
January, where the results were distributed in August (2016) and February (2017), respectively. Here, we compare and
evaluate how the 25 competencies changed during the time being and describe how competency-based education assists
individuals to reflect on their personal traits and discover their blind self to pave their journey.
Key Words: Competency, Peer-evaluation, Self-evaluation, Social Issues, Space Applications
1. Introduction
The concept of “competency” was acknowledged in the
beginning of 1970s as the core value of the recruitment
method for the U.S. Government. It was to respond to such
question “Why are there personnel who can succeed and
others that cannot even though they have the same level of
knowledge and skills?” 2) In May 2016, G7 Kurashiki
Education Ministers’ Meeting was held in Okayama, Japan.
The Kurashiki Declaration mentions competencies required in
the new era. “In a rapidly-changing world, we recognize the
need to build up competencies to solve both given problems
and newly arising challenges.” Moreover, Minerva School at
KGI, adopts educational philosophies to nurture practical
competencies of “personal skills” such as critical thinking
(e.g., evaluating claims, weighing decisions) & creative
thinking (e.g., solving problems, developing new ideas) and
“interpersonal skills” like effective communication (e.g.,
writing, speaking, presenting) & effective interaction (e.g.,
negotiating, working on teams). 3) Although the definitions
differ at times and critical views exist, the concept of
“competencies” are evolving and still gain wide attention. The
study of systems engineering competencies are discussed 4)
and various requirement models are invented 5).
Here, we define “competency” as the behavioral ability to
do something that contribute to enhance personal performance
which ultimately result in organizational success.
Competencies can be improved through experience, challenge,
reflection and peer-feedback. Based on careful research and
analysis, Institution for a Global Society (IGS) has selected 25
competencies (Table 1) required to execute leadership and
implement ideas in the global environment. IGS has
developed the GROW application which supports the
surveying of the competency and the hiring of global talent
for companies using Artificial Intelligence.
This paper explains how the concept of “competency” was
applied to the G-SPASE program. G-SPASE is an
international education program lead by GESTISS (Geospatial
and Space Technology Consortium for Innovative Social
Services) which is consisted of 5 universities in Japan: (1) The
University of Tokyo, Center for Spatial Information Service,
(2) Keio University, Graduate School of System Design and
Management, (3) Tokyo University of Marine Science and
Technology, Department of Marine Systems Engineering, (4)
Aoyama Gakuin University and (5) Graduate School of
Project Design. The G-SPASE program was established in
purpose of tackling global common issues. In particular, it
focuses on the social service reconstruction for more efficient
use of global scale information obtained from space and
ground infrastructures. Space infrastructures such as earth
monitoring, navigation and information transmission using
satellite have been providing rich information for human
society. At the same time, internet and cell phone services are
expanding at an unprecedented rate. The program educates
2
personnel who design and manage social benefits using space
technology and geospatial information. In collaboration with
domestic and international universities, G-SPASE enables to
offer wide range of experience to students. Expertise required
in the space/geo-spatial industry, perhaps common among all
fields, are (1) Leadership; to plan and implement new
programs and involve others to commit to the goal, (2)
International cooperation, skill and network and (3)
Cross-sectoral cooperation, skill and network. 6)
There are 2 reasons why “competency” was adopted in
the G-SPASE program. First, to challenge in designing an
education program that fosters personnel who can act
competitively in the real-world. Second, to quantitatively
assess the educational effectiveness of the program. In
introducing “competency” to the syllabus, we made sure of 3
things. One, giving out lectures on competency to have the
participants understand the notion. Two, grouping of the
peer-review members according to their research project team
so to gain accurate results. Three, preparing the GROW
application in both English and Japanese in order to provide
equal opportunity to students. The following section describes
the actual implementation of the idea.
Table 1. 25 Competencies
2. Competency Evaluation Design in G-SPASE Program
The G-SPASE program attracts students from diverse
background and from varying countries such as China,
Indonesia, Myanmar, Mozambique, Taiwan, Thailand and
Zambia. Consequently, the career one wishes to pursue after
graduation diverges. Some may join in domestic private
companies, seek a job in international organizations, continue
to work in the academia or become entrepreneurs. Therefore,
G-SPASE aims to display competencies required for each
career path. Fundamentally, G-SPASE’s approach to
competency is analyzing the current competency scores of the
participants, finding star players among the pool, displaying
of role model competencies according to the career path and
the planning of education program to foster growth. In order
to evaluate the competency, participants used the GROW
application developed by IGS. The characteristic of the
GROW system is obtaining data through not only
self-evaluation but also peer-evaluation which produce more
precise results. This makes it possible to comprehend whether
the nature one recognizes is consistent with the personality
others perceive. By observing the difference, one can expect
to understand oneself deeply, improve communication and
find direction of development. To analyze the current status
and growth of competencies in the annual program, the survey
was conducted 2 times in the following timeframe.
1st Evaluation:
From June 12th, 2016 - July 21st, 2016
Deadline for evaluation was extended to August 11th
2nd Evaluation:
From December 11th, 2016 - January 15th, 2017
Deadline for evaluation was extended to February 3rd
The members participating in the survey were divided into
groups consisted of 5 to 7 people categorized by their research
themes. In total, there were 11 groups in the G-SPASE
program for the fiscal year 2016. In addition to self-evaluation,
peer-evaluation was conducted among the group members.
For personality assessment, “Big Five Factors”, an analytical
method based on social psychology, was adopted. The Big
Five Factors of personality are five broad domains which
define human personality and account for individual
differences (Table 2).
Table 2. Big Five Factors (1)
E
Extroversion: (outgoing/energetic vs. solitary/reserved).
Energy, positive emotions, urgency, assertiveness,
sociability and the tendency to seek stimulation in the
company of others, and talkativeness. High extroversion is
often perceived as attention-seeking, and domineering. Low
extroversion causes a reserved, reflective personality, which
can be perceived as aloof or self-absorbed.
Introvert
O
Openness: (inventive/curious vs. consistent/cautious).
Appreciation for art, emotion, adventure, unusual ideas,
curiosity, and variety of experience. Openness reflects the
degree of intellectual curiosity, creativity and a preference
for novelty and variety a person has. It is also described as
the extent to which a person is imaginative or independent,
and depicts a personal preference for a variety of activities
over a strict routine. High openness can be perceived as
unpredictability or lack of focus. Moreover, individuals with
high openness are said to pursue self-actualization
specifically by seeking out intense, euphoric experiences,
such as skydiving, living abroad, gambling, etc. Conversely,
those with low openness seek to gain fulfillment through
perseverance, and are characterized as pragmatic and
data-drivensometimes even perceived to be dogmatic and
closed-minded. Some disagreement remains about how to
interpret and contextualize the openness factor.
Conservative
1
Inner Values
16
Self-efficacy
2
Decisiveness
17
Growth
3
Ability to Seek
Cooperation
18
Interests
4
Emotional Control
19
Empathy and
Listening Skills
5
Resilience
20
Ability to Lead a Team
6
Ability to Express
Yourself
21
Extroversion
7
Problem Setting
22
Flexibility
8
Ability to Get Things
Done
23
Sense of Ethics
9
Commitment to a
Team
24
Open-minded
10
Logical Thinking
25
Ability to Wield
Influence
11
Doubt What is Said to
be True
12
Solution-oriented
13
Creativity
14
Global Mindset
15
Vision
3
Table 2. Big Five Factors (2)
N
Neuroticism: (sensitive/nervous vs. secure/confident).
The tendency to experience unpleasant emotions easily,
such as anger, anxiety, depression, and vulnerability.
Neuroticism also refers to the degree of emotional stability
and impulse control and is sometimes referred to by its low
pole, "emotional stability". A high need for stability
manifests as a stable and calm personality, but can be seen
as uninspiring and unconcerned. A low need for stability
causes a reactive and excitable personality, often very
dynamic individuals, but they can be perceived as unstable
or insecure.
Sensitive
Calm
A
Agreeableness:
(friendly/compassionate vs. analytical/ detached)
A tendency to be compassionate and cooperative rather
than suspicious and antagonistic towards others. It is also a
measure of one's trusting and helpful nature, and whether a
person is generally well-tempered or not. High
agreeableness is often seen as naive or submissive. Low
agreeableness personalities are often competitive or
challenging people, which can be seen as argumentative or
untrustworthy.
Cooperative
Independent
C
Conscientiousness:
(efficient/organized vs. easy-going/careless).
A tendency to be organized and dependable, show
self-discipline, act dutifully, aim for achievement, and
prefer planned rather than spontaneous behavior. High
conscientiousness is often perceived as stubborn and
obsessive. Low conscientiousness are flexible and
spontaneous, but can be perceived as sloppy and unreliable.
Hard working
Laid back
3. Competency Evaluation Result
In the 1st evaluation, a “Full Report” was distributed to the
contributors who completed both self and peer-evaluation. On
the other hand, participants only received a “Partial Report” if
they only finished self-evaluation. The result of the group
profile is shown in Figure 1 and 2. The profiles of the 4
groups that completed the survey represented that team
members “can act” but there are less preparation and thoughts
before action and they are “good to others” but not
functioning well as a group. It was advised that more
preparation with structural way (critical thinking), leadership
and project management are required.
In the 2nd evaluation, a “Personal Report” with explanation
of competency scores and personal traits were handed out. In
addition, we were able to extract the competency of the star
players who showed significant growth gaining high score in
both 1st and 2nd peer-evaluation, improved competency in the
overall G-SPASE program, transition of group competencies
and comparison of high performer competencies among
G-SPASE program and companies.
3.1 Results of the 1st Evaluation
- 54 registrations
- 4 groups completed the survey (24 persons)
- 5 groups were unable to complete the survey (30 persons)
3.2 Products of the 1st Evaluation
1. Full Report consisted of 5 parts: (1) Description, (2)
Introduction of competency stage, (3) Competency profile, (4)
Advice to the future and (5) Personality analysis profile.
2. Partial Report consisted of 2 parts: (1) Description of
character analysis and (2) Personality analysis profile.
3. Group profile of the Big Five Factors (Figure 1 and 2)
Fig. 1. The Big Five Personality Report Group Profile.
Fig. 2. The Big Five Personality Report Group Profile Personal Type.
3.3 Results of the 2nd Evaluation
- 43 registrations
- 4 groups completed the survey (17 persons)
- 5 groups were unable to complete the survey (9 persons)
3.4 Products of the 2nd Evaluation
1. Personal Report which shows competency scores and
personal traits (Figure 3)
2. Improved Competency in the G-SPASE Program (Figure 4)
3. High Performance Player Competency (Figure 5)
4. Group Competency Comparison (Figure 6, 7, 8 and 9)
5. Comparison of High Performer Competency among
G-SPASE Program and Companies (Table 3)
4
Fig. 3. Image of Personal Report - Competency Score and Personal
Traits, Example.
Fig. 4. Improved Competency in the G-SPASE Program.
Fig. 5. Competency Growth of Top Performers.
Fig. 6. 1st and 2nd Evaluation Comparison (Group: ASIAN BASE
STATION).
Fig. 7. 1st and 2nd Evaluation Comparison (Group: LOG ANALYSIS).
Fig. 8. 1st and 2nd Evaluation Comparison (Group: UAV).
Fig. 9. 1st and 2nd Evaluation Comparison (Group: URBAN
MAPPING).
5
Table 3. G-SPASE and Company Competency Comparison
G-SPASE
Company A
Company B
1
Logical
Thinking
1
Inner Values
1
Interests
2
Solution-orien
ted
2
Interests
2
Growth
3
Resilience
3
Control of
one’s emotion
3
Open-minde
d
4
Teamwork
4
Extroversion
4
Sense of
Ethics
5
Doubt what is
said to be true
5
Growth
5
Inner Values
6
Decisiveness
6
Passion,
Evangelize
6
Empathy
and listening
skills
7
Control of
one’s emotion
7
Sense of
Ethics
7
Commitment
to a team
8
Extroversion
8
Empathy and
listening
skills
8
Global
Mindset
9
Global
Mindset
9
Global
Mindset
9
Resilience
10
Creativity
10
Ability to get
things done
10
Doubt what
is said to be
true
11
Inner Values
11
Creativity
11
Control of
one’s
emotion
12
Vision
12
Doubt what
is said to be
true
12
Creativity
4. Comparison and Evaluation
First, we successfully evaluated the growth of the
program’s and participants’ competency quantitatively.
Examining the overall result and data comparison with that of
companies, IGS analyzes that the 1st evaluation data collected
in the university are rather unreliable and the 2nd evaluation
result seems more stable. Perhaps, this fluctuation was
prominent because peer-evaluation for the 1st evaluation was
conducted when group members were not familiar with each
other in the beginning of the program.
In the G-SPASE program, progression in 12 competencies
were recorded in the order of “Doubt what is said to be true”,
“Creativity”, Teamwork”, “Inner Values”, “Logical
Thinking”, “Extroversion”, “Control of one’s emotion”,
“Resilience”, “Global Mindset”, “Decisiveness”, “Vision” and
“Solution-oriented.” Figure 4 only shows the improved
competencies but low score for the “Ability to get things done”
means that individuals were experiencing difficulty in
executing one’s strength in a team working environment. As
the student research project is carried out in joint efforts, we
considered the outcome as reasonable.
Table 3 shows the comparison of high performer
competency ranking of the G-SPASE program, company A
and company B. The colored cells are the competencies that
marked growth in the G-SPASE program.
Second, we were able to observe how well the teamwork
were exercised among groups. Among the 11 groups, there
were only 4 groups that all members completed the 1st and
2nd evaluation. This means that groups that couldn’t complete
peer-evaluation lacked effective communication and
coordination to get things done among their members. The
groups that completed the survey were ASIAN BASE
STATION (working on “High Precision Satellite Positioning
and its Application in Asian Countries”), LOG ANALYSIS
(“Taxi Probe Data Analysis to Estimate Customer Trend with
Addressing Urban Traffic Issues”), UAV (“Monitoring of
Coconut Farm using UAV”), and URBAN MAPPING
(“Developing Urban Mapping & Analysis Infrastructure for
Developing Countries”). Among the 4 groups, except for
LOG ANALYSIS, the competency for the 2nd evaluation was
higher than the 1st. The LOG ANALYSIS team showed a
peculiar behavior. For some sort of reason, this maybe the
result of soft peer-evaluation in the 1st evaluation or a harsh
assessment by the team members in the 2nd one. The
competency score for one of the members became
significantly low for the 2nd peer-evaluation so that may have
affected the end result.
Third, we distinguished the differing level of motivation
towards competency evaluation. We estimate 3 reasons why
the participants who completed the survey decreased in the
2nd evaluation. First, there was a time lag between the 1st and
2nd evaluation so the participants may have forgotten its
worth. Second, advantage of measuring competency may have
not been conveyed to the participants in the first place. Third,
the schedule was in conflict with submitting the master’s
research in February, lowering the priority of completing the
1-hour duration survey. It was intriguing to identify even
though the GROW application was provided in Japanese with
an English translation sheet for assistance, studying abroad
students were more willing to participate in the survey than
Japanese students. G-SPASE program does not provide credit.
In such circumstance where joining the survey is not
mandatory, it is relevant to provide incentive and enforcement
to collect more data.
5. Conclusion & Future Work
The competency evaluation was conducted in the G-SPASE
program. In the 1st and 2nd evaluation, 54 and 26 participants
took the survey, respectively. As the figure shows, the number
of members that made contribution for the 2nd evaluation was
lower than first expected. To compile more data, incentive and
enforcement is the key. As a measure, in the 2017 fiscal year
G-SPASE program, we plan to adjust the schedule and change
strategy. First, we will arrange the timetable so that the 2nd
evaluation is finished in autumn. Second, to lower the entry
barrier, we have selected 14 crucial competencies targeted for
the G-SPASE program to shorten the survey time.
Additionally, for the 1st peer-evaluation, participants will
choose 3 personal acquaintances to evaluate himself/herself
rather than having their research group members grade them.
In the future, we wish to display competencies dependent
on organizations so that participants may discover model
competencies of their career. In order to do so, during April -
May 2017, the competencies of approximately 15 employees
working at Japan Aerospace Exploration Agency (JAXA) is
6
being collected. Other than government agencies, data from
private companies, international organizations and
entrepreneurial entities are desired.
Competency evaluation results may be used not only by
third parties, for example in the hiring and promotion process
in companies, but also for personal reference to recognize
one’s own character. We need to be aware of our “intrinsic
tendency-personality traits”, which is difficult to change but
we can behave differently by “knowing who we are.”
Acknowledgments
The research funding from Ministry of Education, Culture,
Sports, Science and Technology - Japan for the “Human
Resource Development Program to Solve Social Issues using
Space Applications” is greatly appreciated.
References
1) GROW Online System URL:
https://survey.grow-to-global.com/users/sign_in
2) Competency-based Interview Manual, Shinji Kawakami, Ryozo Saito,
Koubundou, 2005 (in Japanese)
3) Minerva Schools, The Future for Higher Education, G-SPASE
Conference, Feb 23 2017, Hideki Yamamoto
4) Mr. Charles S. Wasson, Wasson Strategics, LLC, Systems Engineering
Competency: The Missing Course in Engineering Education, ASEE
Annual Conference & Exposition, 2012
5) Frank, M. (2014), 7.2.1 Towards a 4-D Systems Engineering Cognitive
Competency Model. INCOSE International Symposium, 24: 617634
6) Yusuke Muraki, The 3rd Asia-Pacific Space Generation Workshop
(AP-SGW 2016), Los Baños, Laguna, Philippines, 12-13 November,
2016
ResearchGate has not been able to resolve any citations for this publication.
Conference Paper
Full-text available
This paper addresses the need for and proposes solutions to bolster the competency of the engineering professionals at two levels: 1) strengthen undergraduate and graduate level engineering education to include a robust Systems Engineering(SE) problem solving / solution development course and 2) shift the Systems Engineering paradigm found in many organizations through education and training to employ scalable SE methodologies for projects ranging in size from small to large complex systems. The objective is to educate and train engineers to efficiently and effectively develop systems, products, and services that can be verified and validated as meeting user operational needs and expectations with acceptable risk and life cycle costs within project triple constraints – i.e., technical, cost, and schedule performance. To fill the void, the paper proposes a minimum set of topics required for a Fundamentals of Systems Engineering course and instructor / instructional team qualifications. Universities, colleges, government, and industry often create or procure Systems Engineering courses that may provide necessary but insufficient content for educating and training engineers in concepts, principles, and practices required to achieve system, product, or service development success. The problem is exacerbated by the need for Systems Engineering instructors with industry experience, multi-disciplined systems thinking, and excellent instructional communications skills qualifications.
  • Competency-Based Interview
  • Shinji Manual
  • Ryozo Kawakami
  • Saito
Competency-based Interview Manual, Shinji Kawakami, Ryozo Saito, Koubundou, 2005 (in Japanese)