ArticlePDF Available

Abstract and Figures

This paper presents the process used to devise the Teamwork Failure Prevention Questionnaire (TFP Questionnaire), a tool that allows teams with problems in functioning to be detected early. The TFP Questionnaire was formulated in a project management course at the University of Zaragoza (Spain). In this course, teams of five or six students have to manage a project for a real client. The questionnaire was then tested on students on this course and on a similar one at Aalborg University (Denmark). This article analyses the psychometric characteristics of the TFP Questionnaire and then presents and discusses its results, before moving onto examine the implications of this research for engineering education research and engineering education in general.
Content may be subject to copyright.
Journal of Engineering Education
Volume 36, Issue 4, 2010, Pages 784-794
A Tool for Preventing Teamwork Failure: the TFP
Questionnaire
Rubén Rebollar1, Iván Lidón1, Juan L. Cano1, Fernando Gimeno2 and Palle Qvist3
1) Department of Design and Manufacturing Engineering. Project Engineering Group. Universidad de Zaragoza.
2) Department of Psychology and Sociology. Universidad de Zaragoza.
3) Department of Planning and Development. Aalborg University.
This paper presents the process used to devise the Teamwork Failure
Prevention Questionnaire (TFP Questionnaire), a tool that allows teams with
problems in functioning to be detected early. The TFP Questionnaire was
formulated in a project management course at the University of Zaragoza
(Spain). In this course, teams of five or six students have to manage a project
for a real client. The questionnaire was then tested on students on this course
and on a similar one at Aalborg University (Denmark). This article analyses
the psychometric characteristics of the TFP Questionnaire and then presents
and discusses its results, before moving onto examine the implications of
this research for engineering education research and engineering education
in general.
1. Introduction.
In recent years, active learning methods, such as project-based learning [1] and cooperative
learning [2], have been increasingly adopted in engineering schools. This trend is evident in the
many references in journals such as the International Journal of Engineering Education and
the Journal of Engineering Education which cite the introduction of classroom activities that
employ active learning methods. Active learning in the engineering classroom helps students to
develop specific skills and competencies that will prepare them for the profession of engineer
[3–5]. In this learning process, students assume a more active role than in traditional processes,
where they are typically more passive agents. This more active role is usually focused on
working on a team project.
Today, one of the skills that employers most frequently require from candidates for
engineering positions is the ability to work as part of a team [6, 7]. In order to equip students
with the skills they need to work successfully in team situations [8], increasing efforts are being
made in the engineering classroom to teach them this ability [9–11].
_____________________________
*Corresponding author. Email: rrevollar@unizar.es
ISSN 2168-9830 © 2010 SEFI
Downloaded By: [Lidon, Ivan] At: 12:18 29 December 2010
R. Rebollar, et al 785
The research presented in this paper was conducted in a project management course run by
the School of Engineering of the University of Zaragoza. During this course, groups of students,
supervised by instructors that adopt the role of coach, have to manage a project for a real client.
This learning method was first used on the course in 2003. Since that year, the course organisers
have analysed the projects that have obtained poor results. Their analyses revealed that one of
the most frequent causes of project failure was related to functioning problems in teams. This
discovery led to the conception of this research project, which went onto devise a tool that
allows conflicts or problems within a group to be detected early.
This tool has also been tested in another learning context: the Cooperation, Learning and
Project Management course at Aalborg University. In this second context, students work in
teams on a project that is based on a real-life situation as opposed to a real-life project.
According to Prince, Felder and Brent [12, p.291] “doing research on teaching and
integrating successful innovations into classroom practice clearly has the potential to improve
teaching and learning”. In the same spirit, the results obtained in this research have been used to
make improvements in these project management courses therefore improving student team
functioning and learning processes.
2. Context and objective of the research.
The research was conducted in the context of a project management course that is run in the
final year of the Industrial Engineering Degree at the University of Zaragoza. Each year 100
students and five instructors participate in this course. From a learning perspective, the course is
designed to give students first-hand experience of professional project management in order to
prepare them for the profession of engineer. The course’s learning approach simulates the
operations of a consultancy company: teams of five or six students manage a project for a client,
which they must find for themselves. Clients typically consist of small and medium-sized
companies, non-governmental organisations (NGOs), cultural associations, sports clubs or town
councils from small municipalities. Student projects vary greatly, ranging from company
relocations, to event organisation, to preparing proposals for financing NGO activities.
Students are free to form their own teams, although the instructors recommend that group
members’ timetables be compatible. Compatibility is important for students to be able to meet
easily to work as a team on their project and to attend meetings with their coach and client. Each
team has two compulsory roles, those of coordinator and secretary. Elected by group consensus,
the former is in charge of coordinating the group, and the latter takes control of administrative
matters; both combine their role-specific responsibilities with their other project tasks.
During the client search process, the course instructors hold two seminars with all the groups
to advise students on the type of projects that best suit the course characteristics and objectives.
When deciding whether or not to approve students’ project proposals, the instructors assess
whether the scope of the project is compatible with the planned course workload and whether
the project’s emphasis would be on management skills rather than technical skills. The
instructors use their accumulated experience to assist them in this approval process.
The students on this course have no prior experience in team working or project
management. In order to address with these shortcomings, in the first part of the course, the
instructors deliver a series of ten theory lectures explaining the concepts and tools of
cooperation and project management that the students will have to use during their projects.
These lectures are reinforced with two seminars or workshops on specific teamwork topics and
the causes of project failure.
Downloaded By: [Lidon, Ivan] At: 12:18 29 December 2010
Journal of Engineering Education 786
The project deliverable is a written report the Project Management Plan. This includes all
the information needed for the client to implement the project. In addition to the report, groups
must also deliver an oral presentation in which they have to defend their project before a panel
of three instructors and their client. Both the report and presentation count in the final
assessment of each project. The assessment criteria are based upon the scope of the project, its
difficulty, the quality of the report and the presentation, and the technical and financial viability
of the project. Projects are marked on the following scale of 0 to 10:
0-4.9 unsatisfactory (fail): the project is unsuitable. Groups with this mark cannot pass
the course
5.0-7.9 improvable: the project meets the minimum criteria required, although there are
still many areas with room for improvement.
8.0-10.0 very good: the project fulfils all the criteria required.
All the members of a project team receive the same mark. This project mark is then added to
their mark from an individual multiple-choice exam on basic project management concepts.
With the aim of improving course functioning and student learning, the course organisers
analysed the projects marked as “unsatisfactory” in order to try to understand the causes of
failure [13, 14]. It was through this analysis that the instructors detected that group functioning
problems related to coordination were the main cause of project failure. This situation does not
differ greatly from the situation that occurs in professional project management. In that case, it
has been determined that some of the most common causes of failure are related to the
individuals that work in project teams [15] and with team coordination [16, 17].
In light of these findings, the course instructors included two seminars in the learning
programme entitled “Causes of Project Failure” and “Conflict Management”. In the first
seminar, instructors and former alumni present their past experiences on the biggest problems
identified by earlier teams, and strategies to overcome them. The second seminar is delivered by
a psychologist and is based on a series of role-playing exercises. The students are split into
groups and given a fictitious project. They are then given a series of group dynamics in order to
experience situations that frequently occur when working in a team. For example, they are asked
to elect a project coordinator and leader, and to define the different roles needed in the project
and the functions and responsibilities of each one. In another task, students are asked to think
about the kinds of attitudes and behaviour that may improve or harm a group’s functioning. The
groups also simulate different types of group conflicts and discussions (for example, when a
team member fails to submit work correctly or promptly) and then propose ways of addressing
these situations, which will not jeopardise the group’s functioning.
Based on the fact that conflict is an inherent part of team working and is practically
unavoidable, the objective of this research project was to:
Create a teaching tool that allows instructors to detect groups with functioning problems
early on in the course, enabling action to be taken before these problems affect the end
quality of students’ projects.
The next section presents the process followed when designing the TFP Questionnaire and
compares this questionnaire with other such tools devised in the context of engineering
education.
3. The design of the TFP Questionnaire.
3.1. Group functioning assessment tools. Literature review.
Downloaded By: [Lidon, Ivan] At: 12:18 29 December 2010
R. Rebollar, et al 787
As mentioned earlier, most engineering schools set team-working skills as a learning
objective for their students. However, simply placing students in a group work situation does
not suffice to meet this objective [18]; tools are also needed that measure how these groups are
functioning and which help them to work as effectively and efficiently as possible.
On that score, Brewer and Mendelson [19], propose a methodology based on a series of
applied psychology metrics in order to evaluate the effectiveness of a group. They chose
physical, emotional, mental and energy levels as performance criteria and used 29 metrics
grouped together in these levels in order to measure the effectiveness of groups in terms of
creativity, cooperation and productivity. Their groups were assessed 14 times per term using
these metrics, which allowed a group’s effectiveness to be predicted from the characteristics of
its members. This technique proved that fully integrated, multidisciplinary, diverse teams are at
least twice as effective as non-integrated, single-disciplinary, student-selected teams.
Yang and Yin [20] propose a methodology to measure effectiveness in distributed and co-
located engineering teams. They devised a questionnaire consisting of ten team effectiveness
characteristics that allow students to self-assess their groups. These characteristics consist of
goals and objectives, utilisation of resources, trust and conflict, leadership, interpersonal
communication, problem-solving/decision-making, control and procedures,
experimentation/creativity, and evaluation and cohesion. Their questionnaire was adapted from
one developed by Alexander [21] for teams in the workplace. The results of their study suggest
that co-located teams are more socially orientated while distributed teams are more task-
focused.
Along the same line, Adams et. al [22] propose a conceptual model to foster and assess team
effectiveness in order to improve the quality of team results. Their model is based on the most
important characteristics that a group needs to be effective (common purpose, clearly defined
goals, psychological safety, role clarity, mature communication, productive conflict resolution
and accountable interdependence). These characteristics were compiled from previous studies in
the field. In the first stage of this model, group members complete a self-assessment in order to
determine their starting point with respect to these characteristics and to detect potential
deficiencies, which they then receive training on. In the next step, the team carries out a series
of set tasks before once more assessing its effectiveness in terms of the same characteristics.
Likewise, Imbrie, Maller and Immekus [23] propose a scale to measure team effectiveness
based on considerations proposed by other authors. Their model contemplates four factors
(interdepency, learning, potency and goal-setting) with 24 items.
Loo [24] conducted another student team assessment project using a tool called the Team
Climate Inventory® (TCI). This tool was developed by Anderson and West [25] and is now on
the market. The TCI consists of a questionnaire comprising 44 items and has been used in
educational, professional and cultural situations. Loo focused on applying this tool to assess
team climate in a project management classroom context where students worked on research
projects. Loo also suggests that there are no previous studies on using TCI for detecting groups
with problems in functioning.
Although these tools assess different aspects of teamwork they are not designed to detect
team functioning problems in on-going projects. It would therefore be very useful to have a
specific tool for this purpose with the following characteristics:
Useful for instructors. It should facilitate the monitoring of student group functioning
and the provision of guidance in order to improve group results.
Agile and quick. It should be compatible with instructor and student workloads during
the course and contain a limited number of items in order to be implemented quickly.
Downloaded By: [Lidon, Ivan] At: 12:18 29 December 2010
Journal of Engineering Education 788
Predictive character. It should be able to identify student groups whose internal
functioning problems will lead to project failure and consequently a negative result in
their final project assessment.
Reliable and valid. It should have psychometric characteristics that ensure is it a good
measuring tool.
A tool with these characteristics may be of interest to any instructor who works with teams
of students. In the learning context where the TFP Questionnaire was formulated, such a tool is
essential given that students carry out projects for real clients who must receive a professional
service. In this situation, poor teamwork may be detrimental to client satisfaction and therefore
lead to project failure.
The next section details the background to the development of the TFP Questionnaire.
3.2. Background. The Teamwork Questionnaire.
After determining that poor internal group functioning [13] was one of the most common
causes of failure in the groups on the project management course at the University of Zaragoza,
the course organisers set up a research project to study the relationship between the quality of
the work carried out and the quality of group functioning, as well as underlying variables in
both constructs.
The course instructors started by examining the course objectives and the bodies of
knowledge of the leading project management associations (IPMA and PMI). From this
analysis, they defined ten key variables for teaching project management and assessing group
projects. These ten variables, listed below, were used to formulate the first version of a tool
called the Teamwork Questionnaire (TQ).
TQ1. Perceived quality of the project group’s work.
TQ2. Perceived quality of the project group’s functioning.
TQ3. Quality of technical/individual contributions to the project group
TQ4. Quality of individual contributions to the project group’s functioning
TQ5. Teamwork competence in the project group
TQ6. Work motivation in the project group.
TQ7. Satisfaction with work in the project group.
TQ8. Advice for improving the quality of the project work.
TQ9. Advice for improving teamwork in project groups.
TQ10. Proposals for improving individual work in project groups.
The content, development and results of the TQ will be briefly described below,
concentrating on the issues that justified the formulation of the superseding tool, the TFP
Questionnaire. Further information on the TQ and its results can be found in the article An
assessment of behavioural variables implied in teamwork: an experience with engineering
students of the University of Zaragoza [26].
The TQ was drawn up in two stages. In the first stage, the 05/06 course, 21 students were
randomly chosen from a total of 149 to fill in the questionnaire once they had finished their
projects. The students were asked to score each variable using a subjective scale of 0 (very
negative, highly deficient) to 10 (very positive, excellent) and to give the reasons for each score
in their own words.
This article analyses the first two variables of the TQ (TQ1 and TQ2) since these are the only
ones with implications for the TFP Questionnaire. Table 1 shows the mean and standard
deviation of these variables obtained in this first stage.
Downloaded By: [Lidon, Ivan] At: 12:18 29 December 2010
R. Rebollar, et al 789
Mean Standard deviation
TQ1. Perceived quality of the project group’s work. 8.175 1.413
TQ2. Perceived quality of the project group’s functioning. 7.621 1.918
Table 1. Mean and average deviation of the TQ1 and TQ2 variables
The reasons the 21 students gave to explain their scores were analysed qualitatively by three
instructors using the “inter-judge consensus” method [27]. The instructors had received prior
training for this analysis process. The instructors created, by consensus, a series of categories
that grouped together reasons with the same meaning. The instructors did not judge the validity
of students’ contributions. Instead, they restricted their work to compiling all the reasons.
Moreover, the instructors tried to use the same vocabulary and language as the students in order
to minimise interference.
This analysis allowed content categories (items) to be identified for the different variables.
These enabled a new, final version of the TQ to be created comprising ten variables and their
respective items. In the second stage (06/07 course), 92 students filled in this second
questionnaire – which can be found at (www.didyf.unizar.es/pi/teamworkquestionnaire.pdf).
This time the students had to assess the variables and their items using an ordinal scoring
system of 0 (very negative, completely disagree) to 10 (very positive, completely agree). Table
2 shows the overall assessment of the TQ1 and TQ2 variables together with the TQ2 items,
which were the only items used as a basis for formulating the TFP Questionnaire.
Before the results of the TQ questionnaire were analysed, its psychometric properties were
studied, in particular its reliability and validity. Reliability, or the internal consistency of results,
can be estimated using Cronbach’s alpha coefficient. Variables TQ1 and TQ2 were deemed to
have good internal consistency since the coefficients were greater than 0.7 [28]: 0.8761 and
0.8619 respectively. In turn, validity can be defined as “the level with which a questionnaire
measures what it aims to measure” [29, pp.113–139]. The model that is most commonly used
today to determine validity is the model that allows a combined evaluation of content validity,
construct validity and criteria validity [30].
Content validity aims to determine whether the sample of questions in a questionnaire
represents the relevant content in the domain to be measured. In this questionnaire, the content
validity was validated by the process of identifying and analysing the content of the variables
through the opinions of a representative group of “experts” (the students).
Construct validity refers to the idea that a questionnaire should measure the construct, or
thing, for which it was constructed. In this case, the construct validity was determined through
analysing the “item-total” correlation in order to evaluate the integration of each item in the
total of the variable. In accordance with Nunnally and Bernstein [31], the criteria chosen to
include an item in a scale was a correlation index with a value of 0.25 or above. As a result of
this analysis, items 5, 18, 19 and 25 were eliminated from the TQ2 variable. A low correlation
may be caused by a range of factors, such as an item being poorly expressed or not measuring
what is was designed to measure.
Finally, criteria validity shows if a test is useful for predicting a certain response in a specific
situation. The summary of the findings of the TQ results below shows that the criteria variable
mark obtained in the projectwhich measures the success of each project group is an
indicator of the predictive validity of the TQ. This variable is the mark given by the panel to the
group for its report and the defence of its project.
In addition to these tests, exploratory factor analyses were carried out to check the factor
structure of each of the variables and to try to reduce the number of items in each one. However,
Downloaded By: [Lidon, Ivan] At: 12:18 29 December 2010
Journal of Engineering Education 790
the factor groups constructed through this analysis were rejected because they did not group
together items with the same conceptual meaning.
The most important findings from the analysis of the TQ results were:
There is a statistically significant positive correlation between the “mark obtained in the
project” and the “perceived quality of the project group’s work” (TQ1) (φ= 0.397 and p
< 0.001). This result suggests that the criteria used by the instructors to assess the
projects matched students’ individual perceptions of the quality of their projects and
also allows the TQ1 variable to be used as an indicator of project quality.
The groups of students that received a mark of very good (vg, between 8.0 and 10.0)
had a significantly higher score in the TQ2 variable “perceived quality of the project
group’s functioning” in comparison to the group that received a mark of improvable
(im, between 5.0 and 7.9) according to Mann-Whitney’s nonparametric test (U = 551, p
< .05; Mdvg = 8, RIvg = 2; Mim = 7, RIim = 2). This analysis showed that students who
received a better mark perceived a better quality in their group functioning than those
who received lower marks.
When studying the relationship between the 21 items in the TQ2 variable and its overall
value, statistically significant positive correlations were found in 18 items (see Table 2).
The results of the TQ gave the course instructors the idea of going one step further with this
research in order to design a tool that would assess how student groups were functioning in
order to prevent functioning problems leading to poor quality projects. This next step, as
described below, led to the creation of the TFP Questionnaire.
Downloaded By: [Lidon, Ivan] At: 12:18 29 December 2010
R. Rebollar, et al 791
Variable /item Mean Standard
Deviation “Item-total”
correlation index Spearman Correlation between TQ2 and its items.
TQ1 Perceived quality of the project group’s work 8.051 1.004
TQ2 Perceived quality of the project group’s functioning 7.419 1.455
Item 1 All group members had a clear idea of what was to be done. 7.086 1.592 0.456 rs=0.480; p=0.000
Item 2
The group was united by a common goal.
7.624
1.882
0.541
rs
=0.624;
p
=0.000
Item 3 There was a pleasant working environment. 7.978 1.877 0.613 rs=0.550; p=0.000
Item 4 Time allocation for the work was uneven throughout the project. 6.913 2.696 -0.264 rs=-0.376; p=0.000**
Item 5 Work was allocated on a weekly basis to each group member. 6.984 2.244 0.246*
Item 6 There were differences among group members. 3.591 3.080 0.273 rs=0.438; p=0.135**
Item 7 Communication within the group was good. 7.685 1.735 0.624 rs=0.598; p=0.000
Item 8 Work was performed as a team. 7.935 1.680 0.683 rs=0.634; p=0.000
Item 9
Timetable incompatibilities were solved.
8.183
1.894
0.544
rs
=0.399;
p
=0.000
Item 10 All group members worked hard. 7.086 2.595 0.645 rs=0.647; p=0.000
Item 11 All group members worked well. 7.323 2.091 0.682 rs=0.664; p=0.000
Item 12
Each group member had a clearly defined role.
7.505
1.827
0.407
rs
=0.408;
p
=0.000
Item 13 The work brought the group together on a personal level. 7.022 2.436 0.430 rs=0.450; p=0.000
Item 14 A positive attitude prevailed in the group. 7.538 1.704 0.738 rs=0.741; p=0.000
Item 15
The group members had worked together before.
5.731
3.392
0.364
rs
=0.127;
p
=0.226**
Item 16
The meetings led to steady work.
6.774
1.906
0.610
r
s
=0.502; p=0.000
Item 17 Although the beginning was difficult. day-by-day things improved. 7.826 1.745 0.498 rs=0.424; p=0.000
Item 18 The group members had incompatible timetables. 6.806 3.015 -0.067*
Item 19
The group members had different interests.
3.901
3.044
-0.159*
Item 20 The group held together even in moments of difficulty. 7.828 1.851 0.642 rs=0.490; p=0.000
Item 21 Tasks were distributed evenly. 6.145 2.781 0.635 rs=0.458; p=0.000
Item 22
All group members fulfilled their part.
7.344
2.361
0.640
rs
=0.597;
p
=0.000
Item 23 Group coordination was good. 7.527 1.926 0.555 rs=0.672; p=0.000
Item 24 Before the group was formed, its members were friends. 6.860 3.091 0.448 rs=0.220; p=0.000
Item 25 There was not enough time 4.909 3.316 -0.053*
* Items removed after the itemtotal correlation analysis.
** Items without a statistically significant positive correlation with the TQ2 variable
Table 2. Statistical data of the TQ1 variable and the TQ2 variable and its items,
Downloaded By: [Lidon, Ivan] At: 12:18 29 December 2010
Journal of Engineering Education 792
3.3. The Teamwork Failure Prevention Questionnaire.
At that point, it would have been possible to create a tool to assess group functioning using
the 18 TQ2 items with a statistically significant positive correlation with the overall TQ2 value,
since this variable represents students’ perceptions of the quality of their group functioning.
However, this high number of items did not match some of the characteristics planned for this
tool – such as agility and speed. The instructors therefore decided to reduce this number through
selecting the items that they believed to show the different aspects of group internal
coordination, since this had historically been the main cause of failure detected in the groups.
They short-listed six items:
All group members had a clear idea of what was to be done.
Work was performed as a team.
Timetable incompatibilities were solved.
Each group member had a clearly defined role.
Tasks were distributed evenly.
Group coordination was good.
The instructors then carried out a confirmatory factor analysis to check whether these items
formed a single or multiple-dimension variable or construct. The results showed these items
were coherently grouped in a single-dimension construct with high internal consistency
(Cronbach's alpha coefficient = 0.804).
Once these items were defined as a single-dimension variable or construct called
"perceived quality of the project group’s internal coordination" the correlation was studied
between this variable and the TQ1 "perceived quality of the project group’s work" variable. The
results of the overall correlation between these two variables were not statistically significant (rs
= 0.14; p > 0.05). However, the individual analyses of each of the six items with the “perceived
quality of the project’s group work variable produced statistically significant positive
correlations in five of the six items, as shown in Table 3.
Items in the perceived quality of the project group’s internal coordination”
variable Spearman
Correlation
All group members had a clear idea of what was to be done. rS=0,276;p=0,006
Work was performed as a team. rS=0,373;p=0,000
Timetable incompatibilities were solved. rS=0,319;p=0,001
Each group member had a clearly defined role. rS=0,222;p=0,028
Tasks were distributed evenly. rS=0,314;p=0,002
Group coordination was good. rS=0,120;p=0,240
Table 3. Spearman correlation between the items in the “perceived quality of the project group’s internal
coordination” variable with the TQ1 “perceived quality of the project group’s work” variable
This correlation led to the idea of creating a tool that would be able to check these internal
coordination values in order to predict which groups would have a poor quality final project,
thus enabling preventative actions to be put into place.
The TFP Questionnaire was therefore constructed using the first five variables listed in Table
3 together with the assessment of the quality of group functioning (TQ2). The item “group
coordination was good” was excluded from the questionnaire, since this aspect of internal group
coordination had no correlation with the TQ1 variable. The students participating in this study
had to respond using an ordinal scale with a continuous classification of 10 intervals between 0
(very negative, strongly disagree) and 10 (very positive, completely agree). Figure 1 shows the
Downloaded By: [Lidon, Ivan] At: 12:18 29 December 2010
R. Rebollar, et al 793
TFP Questionnaire that was distributed to students. This template can be downloaded at
www.didyf.unizar.es/pi/ptfquestionnaire.pdf.
Figure 1. Teamwork Failure Prevention Questionnaire.
In order to identify if a group perceived it had internal functioning problems, three variables
were created:
Pmember”: sum of the values of an individual’s questions (TFP Questions 2 to 6).
Pgroup”: average “Pmember” value of the members of a team.
Pcourse”:25th percentile of the “Pmember” values of all students on the course.
A group was identified as potentially having problems if its Pgroupvalue was below the
Pcourse value. A quasi-experimental regression-discontinuity design was therefore used [32].
More specifically, groups were either allocated to a “treatment group” or “non-treatment group”
according to a cut-off point in the 25th percentile of the assessments of all students. Groups that
scored below the cut-off point were assigned to the “treatment group” and those with scores
above this level were allocated to the “non-treatment group”.
The next section presents the results obtained with the TFP Questionnaire.
4. TFP Questionnaire Results.
4.1 Results in the project management course at the University of Zaragoza.
The TFP Questionnaire was tested at the University of Zaragoza during the 07/08 and 08/09
courses. In the 07/08 course, 84 students filled it in (97.7% of the total), representing all the 18
groups on the course. In the following year, 75 questionnaires were filled in (93.7% of the
total), representing 17 groups. In both years, the questionnaire was distributed mid-course
Downloaded By: [Lidon, Ivan] At: 12:18 29 December 2010
Journal of Engineering Education 794
because at this point the workload increases considerably and this is when the first conflicts and
problems in functioning tend to appear.
Students completed the questionnaires individually and away from other members of their
groups. The questionnaires were anonymous; students only had to indicate their group reference
number. In order to guarantee this anonymity, the completed questionnaires were placed in a
sealed box. Students were also informed that the questionnaires would not be marked.
As with the TQ, the psychometric characteristics of the TFP Questionnaire were analysed:
The reliability was gauged using Cronbach’s alpha coefficient, reaching a value of
0.852.
The content validity was justified because the items were taken from the TQ and had
been validated earlier.
The construct validity was confirmed by the high internal consistency of the TQ, as
validated earlier, and by the item discrimination analysis in which all items obtained an
item-total correlation index above 0.25 using the criteria of Nunnally and Bernstein
[31].
The validity criteria results in both years, as shown below, confirm the predictive
capability of this tool. In this respect, the results show a statistically significant
association between the variables measured with the TFP Questionnaire and the marks
obtained by students in their projects.
The TFP questionnaire detected internal functioning problems in five out of 18 groups on the
07/08 course and three out of 17 groups on the 08/09 course. Table 3, below, shows the values
obtained in each of these groups (Pgroup variable) together with the cut-off values that were
used to determine whether a group had problems in functioning (“Pcourse variable).
The system used to reference the groups in Tables 3 and 4 is GA–B–C, where:
A is the group reference number.
B is the year of the questionnaire.
C is the location of the questionnaire (Z=Zaragoza and A=Aalborg).
07/08 COURSE 08/09 COURSE
Cut-off value “Pcourse = 33 Cut-off value “Pcourse = 33
Groups “Pgroup Groups “Pgroup
G2-08-Z 24.0 G6-09-Z 31.5
G5-08-Z 32.0 G10-09-Z 31.0
G9-08-Z 28.0 G13-09-Z 25.0
G12-08-Z 32.0
G17-08-Z 28.5
Table 3. TFP Questionnaire results at the University of Zaragoza.
Downloaded By: [Lidon, Ivan] At: 12:18 29 December 2010
R. Rebollar, et al 795
Each group and its coach were informed of the questionnaire results as soon as these were
calculated so that the results could be discussed at their next meeting. This meeting was
particularly important when the tool detected a group with problems. Each coach was given a
series of recommendations to help them draw up a tailored action plan for each group affected.
Action plans were established because both the students and coaches in the groups affected
verified the results obtained with this tool, confirming that the groups had problems in
functioning.
The actions recommended by coaches to groups with problems functioning are very diverse
and depend on the type of problems identified. One of the most common types of problems in
the groups was that students complained that the distribution of the group workload was uneven
(TFP Question 6). In these cases, students are recommended to make optimum use of the course
timesheet, a tool on which group members record the time they spend on the project each week.
Although students’ names on the timesheet are hidden from the coach, students can see the
names therefore allowing them to identify the team members that are working fewer hours.
Another common problem was that students did not find enough time to meet up (TFP Question
4). In these cases, the coach encourages the groups to establish compulsory weekly meetings,
which cannot be postponed. In order to get the most out of their meetings, students are also
advised to prepare an agenda for each meeting to focus their attention on the most important
matters.
After the action plan has been defined, the coaches then monitor the functioning of the
groups affected and the effectiveness of any corrective measures. The coaches also compile a
report containing all the information gathered in these meetings as well as any corrective
measures adopted. This report is then shared and discussed with the other course coaches.
In all groups in which the TFP Questionnaire has detected problems in functioning, both the
group members and coaches have verified this diagnosis. After adopting corrective measures,
the five groups with problems on the 07/08 course passed the course and their projects were
assessed positively. On the 08/09 course, two of the groups overcame their problems, however
the third could not sit the exam because their problems prevented them from completing their
project on time. Nevertheless, the group passed the course in the second round of exams. In all
cases, students informed their coaches that after applying the corrective measures, their group’s
functioning improved and they were able to solve their problems.
4.2 Results in the Cooperation, Learning and Project Management course at Aalborg
University.
In order to test the TFP Questionnaire in a different context, it was distributed to students on
the 08/09 Cooperation, Learning and Project Management course at Aalborg University. The
students at the two universities completed the questionnaire at the same time.
This context was chosen firstly because both courses are based on project management and
secondly because they use a similar classroom methodology based on group projects. A third
factor influencing this choice was the fact that students at Aalborg possess much more honed
teamwork skills than those at Zaragoza. This difference is because Aalborg University focuses
more on developing these skills. It was therefore interesting to use the TFP Questionnaire on
students with more teamwork experience.
The questionnaire was written in Spanish and translated into English using a back translation
quality assurance procedure involving two different translators. In this process, one translator
translated the questionnaire from Spanish to English and a second translator then translated the
English version back to Spanish. The differences between the original and back-translated
Downloaded By: [Lidon, Ivan] At: 12:18 29 December 2010
Journal of Engineering Education 796
documents were then analysed in order to verify that the content and wording of the translated
questionnaire were true to the original.
A total of 144 anonymous responses were collected from the 252 students on this course
(57.1%), representing 36 groups. The reliability of the questionnaire in this context was gauged
using Cronbach's alpha coefficient, obtaining a value of 0.892. The questionnaire’s validity was
justified by using the criteria mentioned above in the case of the University of Zaragoza.
Problems were detected in five of the 36 groups analysed at Aalborg. Table 4, below, shows
the scores obtained in each of these groups (“Pgroupvariable) together with the cut-off value
used to determine groups with problems in functioning (“Pcourse variable).
Cut-off value Pcourse” = 26
Groups “Pgroup
G6-09-A 19.0
G16-09-A 25.0
G18-09-A 25,5
G24-09-A 25.0
G33-09-A 21.0
Table 4. TFP Questionnaire results at Aalborg University.
The groups had to submit a self-assessment report on the functioning of their group. In order
to validate the results of the TFP Questionnaire at Aalborg, the self-assessment reports of the
groups detected to have problems were consulted. All these groups reported that they were
experiencing problems, although none indicated a crisis that they felt they could not overcome
themselves. One of the most frequent problems identified in the Danish groups was that students
did not share the same vision of the project and therefore did not have a clear idea of the project
tasks (TFP Question 2). Another common problem, as in Zaragoza, was that project workloads
were distributed unfairly (TFP Question 6).
5. Discussion.
Firstly, with respect to the methodology, since the research involved project management
students at Zaragoza from different academic years, it was impossible to establish a control
group against which the results could be assessed. A total of 272 students participated in the
Zaragoza programme in the 05/06 to the 08/09 courses. The subjects were similar although
not identical in each year: they had similar weaknesses and strengths and no prior knowledge
on project management or teamwork. Nevertheless, the TFP Questionnaire’s reliability and
validity results in the groups at both Zaragoza and Aalborg suggest that it is a robust and
reliable measurement tool in both cultural contexts.
Another important aspect of the methodology is the cut-off value used for determining the
existence of functioning problems in a group. This research project set this level as the 25th
percentile of the scores of all those who completed the questionnaire (“Pcourse). This criteria is
commonly used to identify the critical region or risk groups in preventive and diagnostic
contexts. However, the 25th percentile may be rather conservative and therefore exclude some
groups that are experiencing problems in their functioning. If it were necessary, this cut-off
value could be raised to a higher percentile. Nevertheless, it was proven that all the groups
detected with this cut-off value confirmed the diagnosis made by this tool.
Downloaded By: [Lidon, Ivan] At: 12:18 29 December 2010
R. Rebollar, et al 797
Continuing with the theme of methodology, another key aspect is the procedure used by
students when completing the questionnaires. This procedure always ensured the questionnaires
were anonymous because group members may be hesitant to complain about group functioning
problems or conflicts in front of their peers or instructors. This situation may be exacerbated if
the students are also friends, as often occurs in the University of Zaragoza where students are
free to form their own groups. The procedure used helps to bring these issues to light and gives
instructors an opportunity to intervene to resolve them.
Secondly, it is important to discuss the different cut-off values defined for identifying groups
with problems at each university. These differences can be explained by the fact that the Danish
students had more experience in team working than the Spanish students, and that the two
cohorts encountered different types of problems. The questionnaire scores of the Aalborg
students were generally much lower than their counterparts in Zaragoza. The cut-off values
were therefore stricter in Zaragoza than in Aalborg, set at 33 and 26 respectively. This
difference may be due to the fact that the Danish students, owing to their experience of working
in groups, judged their team functioning more critically, whereas the students at Zaragoza
seemed to be more optimistic. The different types of problems affecting the two cohorts may
also have come into play here. For example, the Danish students did not have problems meeting
up, whereas the Spanish students did. Nevertheless, these scores cannot be compared because
these students come from different universities with different characteristics, problems and
learning experiences. What is important is that, owing to the idiosyncrasies of each context, the
TFP Questionnaire has a uniform way of calculating the cut-off point through the Pcourse
variable.
Finally, this research has contributed to both engineering education research and engineering
education in general.
The main contribution to engineering education research has been a tool, the TFP
Questionnaire, which can detect groups with problems in functioning during an on-going project
in order to prevent these problems from jeopardising the project’s quality. This tool was
designed to have good psychometric characteristics, and results show that it does. Moreover, it
has been applied successfully in the project management course at the University of Zaragoza,
as well as in another context – leading to the ability to improve group functioning in both cases.
The TFP Questionnaire has contributed to engineering education through helping students to
reflect on the most important aspects of team working. The questionnaires and subsequent
corrective measures, where applicable, have helped students to become aware of behaviours and
attitudes they can adopt to improve team working or group functioning. In turn, this awareness
has led to better quality work and has ultimately aided their learning. This learning experience
also benefits instructors who, through the reports they prepare after interventions in groups with
problems in functioning, learn to support their students in resolving group conflicts and
problems. Moreover, this learning experience aids instructors in future academic years, which,
in turn, benefits future students.
From a project management course design perspective, the TFP Questionnaire has become a
regular teaching tool that coaches use to monitor their groups in the course at the University of
Zaragoza. However, instructors on this course do not neglect the education of their students in
the key areas of cooperation and project management just because they have a tool that can
identify groups with problems in functioning. In fact, having realised the importance of these
areas, the instructors will place greater emphasis on them in the future.
6. Conclusions.
The aim of this research was to devise a tool that allows the early detection of project groups
with problems in functioning.
Downloaded By: [Lidon, Ivan] At: 12:18 29 December 2010
Journal of Engineering Education 798
The result has been the creation of the TFP Questionnaire. This questionnaire has proven to
be an agile and flexible tool that instructors can employ to improve their monitoring and
supervision of student groups during a course.
In order to test its validity and reliability, the TFP Questionnaire has been tested in project
management courses at the universities of Zaragoza and Aalborg. These tests have been a prime
example of transferring educational research results to the engineering classroom. In both cases,
the reliability and validity values of this questionnaire show it is a suitable measuring tool,
which, in practice, has detected eight groups with problems in functioning out of the 35 groups
analysed at Zaragoza and five groups out of the 36 studied at Aalborg.
These psychometric characteristics and the TFP Questionnaire’s simple and rapid
completion and analyses processes make it a good tool to be applied in educational contexts
where students work in groups – an increasingly popular learning approach – because the results
show that the tool is both effective and efficient.
Future work on this research would consist in testing this tool in different educational
contexts, perhaps in other engineering contexts outside of project management.
References.
1. E. De Graaff, and A. Kolmos, Characteristics of problem-based learning, International Journal of
Engineering Education, 19 (5), 2003, pp. 657662.
2. R. Felder, D. Woods, J. Stice and A. Rugarcía, The future of engineering education II. Teaching
methods that works, Chemical Engineering Education, 34(1), 2000, pp. 26–39.
3. X.Y. Du, and A. Kolmos, Process competencies in a problem and project based learning environment,
35th SEFI annual conference: engineering education and active students, Uppsala, Sweden (2006).
4. T.J. Brumm, L.F. Hanneman, and K. Mickelson, Assessing and Developing Program Outcomes
through Workplace Competencies, International Journal of Engineering Education, 23(1), 2006, pp.
123129.
5. R.M. Felder, and R. Brent, Designing and teaching courses to satisfy the ABET engineering criteria,
Journal of Engineering Education, 92(1), 2003, pp. 7–25.
6. I. Markes, A review of literature on employability skill needs in engineering, European Journal of
Engineering Education, 31 (6), 2006, pp. 637650.
7. L. Pascail, The emergence of the skills approach in industry and its consequences for the training of
engineers, European Journal of Engineering Education, 31 (1), 2006, pp. 5561.
8. B.A. Oakley, R.M. Felder, R. Brent, and I. Elhajj, Turning students groups into effective teams,
Journal of Student Centered Learning, 2 (1), 2004, pp. 934.
9.K. Tonso, Teams that Work: Campus Culture, Engineer Identity, and Social Interactions, Journal of
Engineering Education, 95 (1), 2006, pp. 2537.
10.B.A. Oakley, D. Hanna, Z. Kuzmyn, and R.M. Felder, Best practices involving teamwork in the
classroom: results from a survey of 6435 engineering students respondents, IEEE Transactions on
Education, 50 (3), 2007, pp. 266272.
11. P.L. Hirsch, and A.F. McKenna, Using Reflection to Promote Teamwork Understanding in
Engineering Design Education, International Journal of Engineering Education, 24(2), 2008, pp. 377
385.
12.M.J. Prince, R.M. Felder, and R. Brent, Does faculty research improve undergraduate teaching? An
analysis of existing and potential synergies, Journal of Engineering Education, 96 (4), 2007, pp. 283
294.
13. J.L. Cano, I. Lidón, R. Rebollar, P. Román, and M.J. Sáenz, Students groups solving real-life projects.
A case study of experiential learning, International Journal of Engineering Education, 22 (6), 2006, pp.
1252–1260.
14. J.L. Cano, I. Lidón, and R. Rebollar, Learning Project Management through working for real clients,
International Journal of Engineering Education, 24 (6), 2008, pp. 1199–1209.
15. J. Pinto, and S. Mantel, The causes of project failure, IEEE Transactions on Engineering
Management, 37(4), 1990, pp. 269276.
16. K.N. Jha, and K.C. Iyer, Critical determinants of project coordination, International Journal of
Project Management, 24(4), 2006, pp. 314–322.
Downloaded By: [Lidon, Ivan] At: 12:18 29 December 2010
R. Rebollar, et al 799
17. K.N. Jha, and K.C. Iyer, Commitment, coordination, competence and the iron triangle, International
Journal of Project Management, 25 (5), 2007, pp. 527–540.
18. S.K. Williams, and J.S Anderson, Teams Lab: promoting effective teamwork in Operations
Management classes. Decisions Sciences Journal of Innovative Education, 6(1), 2008, pp. 159–166.
19. W. Brewer, and M.I. Mendelson, Methodology and metrics for assessing team effectiveness,
International Journal of Engineering Education, 19 (6), 2003, pp. 777787.
20. M.C. Yang, and Y. Jin, An examination of team effectiveness in distributed and co-located
engineering teams, International Journal of Engineering Education, 24(2), 2008, pp. 400408.
21. M. Alexander, The team effectiveness critique, in The 1985 Annual: Developing Human Resources,
University Associates, San Diego, pp. 101106 (1985).
22. S.G. Adams, L.C. Simon, B.C. RuizUlloa, and F. Pereira, A conceptual model for the development
and assessment of teamwork, Proceedings of the 2002 American Society for Engineering Education
Annual Conference & Exposition, Montreal, Canada (2002).
23.P.K. Imbrie, S.J. Maller, and J.C. Immekus, Assessing Team Effectiveness, Proceedings of the 2005
American Society for Engineering Education Annual Conference & Exposition, Portland (USA).
24. R. Loo, Assessing “team climate” in project teams, International Journal of Project Management,
21(7), 2003, pp. 511517.
25. N.R. Anderson, M.A. West, The team climate inventory manual and user’s guide. Nelson, Windsor
(1994).
26. J.L. Cano, I. Lidón, R. Rebollar, and F. Gimeno, An assessment of behavioral variables implied in
teamwork: an experience with engineering students of Zaragoza University, European Journal of
Engineering Education, 34(2), 2009, pp. 113122.
27. B.G. Glaser, and A.L. Strauss, The discovery of grounded theory: strategies for qualitative research.
Aldine, Chicago (1967).
28. J.C. Nunnally, Psychometric theory 2nd ed, McGraw-Hill, New York (1978).
29. A. Anastasi, and S. Urbina, Psychological testing, 7th ed. Upper Saddle River, Prentice Hall, New
Jersey (1997).
30.R. Sartori, and M. Pasini, Quality and quantity in test validity: how can we be sure that psychological
tests measure what they have to? Quality and Quantity, 41, 2006, pp. 359374.
31. J.C. Nunnally, and I.J. Bernstein, Teoría psicométrica. McGraw-Hill, Madrid (1995).
32. D. Thistlethwaite, and D. Campbell, Regression-Discontinuity Analysis: An alternative to the ex post
facto experiment, Journal of Educational Psychology 51, 1960, pp. 309317.
About the authors
Ruben Rebollar. MSc and Doctor from the Engineering Faculty of University of Zaragoza. He is
Associate Professor in the Design and Manufacturing Department at this university.
Ivan Lidón. MSc and Doctor from the Engineering Faculty of Zaragoza University. He has worked as
assistant at the Design and Manufacturing Engineering Department of the University of Zaragoza since
2004.
Juan Luis Cano. MSc and Doctor from the Engineering Faculty of Madrid Polytechnic University.
Actually he is Professor in the Design and Manufacturing Department at Universidad de Zaragoza.
Fernando Gimeno. Ph.D. has held different positions at public and private organizations in the sport and
health scopes. Since 2001, he is Associate Professor in the Psychology and Sociology Department at the
University of Zaragoza.
Palle Qvist. He is associate professor and lecturer in Technology, Humans and Society and in
Cooperation, Learning and Project Planning. He is staff member at the UNESCO Chair in Problem Based
Learning in Engineering Education.
Downloaded By: [Lidon, Ivan] At: 12:18 29 December 2010
... For example, if the teachers' learning objective is to develop students' skills (i.e. communication and collaboration skills), there may be better methods than putting students together in a group and assuming that skills will develop without any guidance and/or formative and summative assessment of it (Felder and Brent 2001;Williams and Anderson 2008;Rebollar et al. 2010). As was pointed out by Johnson and Johnson (1999), a prerequisite for students learning together is that they are proficient in collaboration and communication; and if not, that they should be taught. ...
... For example, Rebollar et al. (2010) aimed to address students' shortcomings concerning collaboration in preparation for a collaborative learning assignment. To this end, students received ten theory lectures about collaboration and project management. ...
... This can be beneficial for the construct validity, while still adhering to the objective of developing collaboration skills. In addition, a training program as described by Rebollar et al. (2010) might better prepare students for collaborative learning outside of class (e.g. preparation for the future workplace). ...
Article
Full-text available
Over the past two decades, curricula in higher education have increasingly incorporated collaborative learning. However, due to (a) large variations in students’ domain-specific abilities (e.g. knowledge and/or skills) and the effort they invest into the collaboration and (b) teachers’ limited knowledge about how to assess collaborative learning, two main challenges arise. The first challenge concerns ensuring construct validity of the assessment methods, that is, whether an assessment accurately measures students’ domain-specific abilities. The second challenge originates from the potential of assessment methods to elicit student behaviour that is misaligned with the objectives of collaborative learning (e.g. free-riding behaviour). This paper aims to enhance teachers’, researchers’, and students’ awareness for and need to develop what we refer to as ‘collaborative learning assessment literacy’. In particular, we will discuss the two challenges in relation to three frequently used and discussed methods for assessing collaborative learning - group assessment, individual assessment, and group assessment combined with intra-group peer assessment - with specific attention to the purpose of assessment (i.e. formative and summative). Implications of the two challenges as well as their relation to other core components in the design of any collaborative learning setting (e.g. group constellation) will be discussed.
... For example, male engineering students might prefer male-typical speech over femaletypical speech and evaluate them as the weak team members [38], or peer assessment might be biased base on students' race [39]. Anyway, any dysfunctionalities can be diagnosed by tools and innovative heuristics [40]- [42]. For finding the problems in teams, we need team member assessment and team performance evaluation. ...
... She found that no direct correlation between support-oriented discourse and achievement and only a moderate positive correlation between post self-efficacy and the extent to which a student engaged in support-oriented discourse. Furthermore, Rebollar et al [27] developed the Teamwork Failure Prevention questionnaire to help instructors and teams to better manage their conflicts and prevent team project failure. ...
Article
Our study investigates the role of team conflict in the context of student design project work. Using data collected on 55 teams enrolled in a team-based engineering design program over three time periods, our results showed that the proportion of women and the existence of multiplex ties among team members have a positive influence on team performance, while the number of subgroups and team conflict were negatively related to team performance. Implications for team-based engineering programs are drawn.
... There are scarce evidences of application of principles of project management in the context of project based learning. Most of the published works are related to teamwork (some examples: [25][26][27][28][29][30] ), which is a fundamental knowledge area of projects in learning environments. However, not only the dimensions related with teamwork are important in coordination of student teams and other knowledge areas of Project Management should be considered. ...
Article
Engineering learning processes are expected to develop technical and transversal competencies on students that are demanded by the engineering professional bodies. The need for the development of competencies raised an incremental interest in applying innovative approaches in Engineering education. One of the methodologies used in this context is Project-Based Learning (PBL). At the University of Brasilia, a course degree in Production Engineering was created having as a main reference the Project-Based Learning (PBL) approach. The use of PBL implies a change of behavior of teachers who play a main role of facilitators of competencies development, and of the students, who learn in a collaborative way, working with others in teams. Students’ working in team and developing a project during a semester will require effective coordination models. The objective of this paper is to propose a model of coordination among students’ teams based on project management knowledge. Furthermore, a qualitative approach is applied to evaluate the application of this model during one semester. The results show that a coordination model for student teams developing projects is important to support their learning process, which is not solely dependent on students, as teachers/tutors have an important role before, during and at the end of the project. The tutor is especially important, as he/she is responsible for supporting the teams in several project management dimensions. This is an essential support for students to know how to manage the team, communicate, define goals, carry out the activities on time, plan the milestones and understand the impact of their decisions. These competencies are also part of student learning and are an important part of engineering education.
Article
Students routinely take part in group assignments, but many faculty fail to provide a supportive work environment. The author argues the value of team building and introduces a conceptual framework for developing multipart group assignments. The author’s purpose is to provide management faculty a framework for an interactive group assignment in an experiential management course. The framework uses competencies from the adaptive leadership model.
Article
Full-text available
Interprofessional team-based care is increasingly regarded as an important feature of delivery systems redesigned to provide more efficient and higher quality care, including primary care. Measurement of the functioning of such teams might enable improvement of team effectiveness and could facilitate research on team-based primary care. Our aims were to develop a conceptual framework of high-functioning primary care teams to identify and review instruments that measure the constructs identified in the framework, and to create a searchable, web-based atlas of such instruments (available at: http://primarycaremeasures.ahrq.gov/team-based-care/). Our conceptual framework was developed from existing frameworks, the teamwork literature, and expert input. The framework is based on an Input-Mediator-Output model and includes 12 constructs to which we mapped both instruments as a whole, and individual instrument items. Instruments were also reviewed for relevance to measuring team-based care, and characterized. Instruments were identified from peer-reviewed and grey literature, measure databases, and expert input. From nearly 200 instruments initially identified, we found 48 to be relevant to measuring team-based primary care. The majority of instruments were surveys (n = 44), and the remainder (n = 4) were observational checklists. Most instruments had been developed/tested in healthcare settings (n = 30) and addressed multiple constructs, most commonly communication (n = 42), heedful interrelating (n = 42), respectful interactions (n = 40), and shared explicit goals (n = 37). The majority of instruments had some reliability testing (n = 39) and over half included validity testing (n = 29). Currently available instruments offer promise to researchers and practitioners to assess teams’ performance, but additional work is needed to adapt these instruments for primary care settings.
Article
Background Engineering student team projects are frequently used to meet professional learning outcomes. Industrial and organizational psychologists study teams in the industry settings for which we prepare students, yet this research does not effectively inform engineering education. PurposeThis research review sought to demonstrate the relevance of literature on teams literature from industrial and organizational psychology to engineering education and to identify implications for practice and future directions for research. Scope/Method Phase 1 systematically reviewed 104 articles published from 2007 to 2012 describing engineering and computer science student team projects and sought to answer the following questions: What professional learning outcomes have been met by team projects? What negative student team behaviors have faculty sought to minimize? What literature has been used to inform development of teamwork outcomes? Phase 2 reviewed five team effectiveness constructs selected according to the results of Phase 1: social loafing, interdependence, conflict, trust, and shared mental models. Examples from Phase 1 articles and our own work explain how this research informs facilitation and assessment of engineering student teams. Conclusions Engineering faculty sought to achieve a variety of outcomes through team projects, including teamwork, communication, sustainability, and consideration of global/societal design context. They sought to avoid social loafing and conflict while building trust to ensure equal team effort. That few Phase 1 articles engaged the literature about team effectiveness indicates there is great opportunity to apply industrial and organizational psychology research to engineering education.
Article
Full-text available
A Project Management (PM) training approach is presented based on the solution of real-life projects by groups of students in their last year at the School of Engineering of the University of Zaragoza. The training dynamics simulates the functioning of a consulting firm, where students are the consultants and the teachers work as mentors, sharing a common methodology to help the student groups to successfully bring their projects to completion. The `problems' to be solved are proposed by real customers. During the academic years 2003/04 and 2004/05, 41 customers and 240 students have taken part in the course, with very satisfactory results. With the ultimate goal of increasing the satisfaction of all participants in this course, our priority has evolved in this period and we currently seek to ensure that no project group ends in failure. The results of this experience are based mainly on the analysis of `failed' groups (5 out of 41 cases), the periodic group selfassessment sessions and the feedback received from students. According to this research, coordination within a project group appears to be a key aspect in influencing the results obtained. Those groups with an assigned `coordinator' worked better. It is also worth pointing out that, as the project advanced, it was noted that all of the groups felt more confident and more optimistic about the quality of their work and the level of satisfaction expected from the customer, regardless of the actual results. Also, the vast majority of the groups grossly underestimated the time needed to complete the work. The paper concludes by describing future lines of work and collaboration in the model framework presented.
Article
Full-text available
and be involved in the accreditation process on a continuing basis, not just in the months preceding each visit. Understanding the engineering criteria is no trivial goal, however; the jargon they contain (objectives, outcomes, outcome indicators, performance targets, etc. ) is dense and confusing, and universally agreed-upon operational definitions of the terms do not yet exist. Moreover, while much has been written in the past few years about the assessment of program outcomes (more specifically, of Outcomes 3a-3k), relatively little attention has been paid so far to the central role of the individual faculty member in attaining those outcomes. The primary purpose of this paper is to examine that role. Since the new ABET accreditation system was first introduced to American engineering education in the middle 1990s as Engineering Criteria 2000, most discussion in the literature has focused on how to assess Outcomes 3a-3k and relatively little has concerned how to equip students with the skills and attitudes specified in those outcomes. This paper seeks to fill this gap. Its goals are to (1) overview the accreditation process and clarify the confusing array of terms associated with it (objectives, outcomes, outcome indicators, etc. ); (2) provide guidance on the formulation of course learning objectives and assessment methods that address Outcomes 3a-3k; (3) identify and describe instructional techniques that should effectively prepare students to achieve those outcomes by the time they graduate; and (4) propose a strategy for integrating program-level and course-level activities when designing an instructional program to meet the requirements of the ABET engineering criteria.
Article
This paper describes a new systematic methodology and measurements for assessing the effectiveness of engineering/business student teams. This new methodology is unique because it predicts team effectiveness. Our approach replaces the traditional methods of indirect surveying and interviewing with direct surveys and systematic exercises designed to measure team effectiveness. Effectiveness was defined and evaluated using three outcomes: creativity, collaboration and productivity. These three outcomes were measured using applied psychology metrics. By quantifying creativity, collaboration and productivity, this new assessment methodology provided a more objective way of measuring effectiveness. The outcomes were equally weighted to calculate an effectiveness rating. The teams that were faculty-selected and properly coached were confirmed to be at least twice as effective as those that were student-selected and received little coaching.
Article
Coordination among project participants is an important function having considerable effect on the outcome of a construction project. Literature review and interviews of experts led to the identification of 59 construction coordination activities. Results of a questionnaire survey conducted among Indian construction professionals on these coordination activities recognized 20 important coordination activities essential for achieving day-to-day project coordination. A second level of questionnaire survey was then conducted using the 20 coordination activities. Analyses of responses on the 20 coordination activities found that only six activities are significant in enhancing coordination rating of the project. The analyses indicated that the extent of contribution of different coordination activities varies with the present coordination ratings of a given project. While estimation of the optimum resource requirements has the highest positive effect on achieving coordination at low coordination rating levels, the activity preparation of a project quality plan in line with contract specification is observed to contribute most when the coordination rating is already at high level. A model is also suggested that can evaluate the impact of these six coordination activities in achieving the coordination rating of a project.
Article
One of the functions of construction project management is to ensure success of the construction project. However achieving success in a construction project is not a small task. Moreover, measurement of performance of a construction project itself is considered to be a debatable issue as there are no universally accepted criteria for it. Traditionally schedule, cost and quality compliances commonly referred to as ‘iron triangle’ have been accepted as the most widely used criteria to measure performance. Further assuming that the project success is repeatable, researches have indicated certain attributes/factors which when present or absent in a project are likely to make the project successful. In the present study, 55 project performance attributes are identified and a two-stage questionnaire survey is conducted. While 11 success and nine failure factors are identified from the analysis of the first stage questionnaire responses, the second stage questionnaire survey has helped in evaluation of the extent of criticality of these factors with respect to a given performance rating of the project. It is found that extent of contribution of various success or failure factors varies with current level performance ratings of the project. The crux of the findings of this study has been the emergence of commitment, coordination, and competence as the key factors for achievement of schedule, cost, and quality objectives respectively. The results may be of a great help to construction professionals and researchers since it helps to identify value adding factors, fewer in number and allow them to focus on these select factors rather than attending to numerous factors that do not yield any commensurate returns.
Article
Problem-based learning (PBL) is widely regarded as a successful and innovative method for engineering education. Since the development of the PBL model at McMaster University in Canada in the late 1960s, many different varieties have emerged. This paper highlights the Dutch approach of directing the learning process through problem analysis and the Danish model of project-organised learning. Various definitions of the concept PBL identify characteristics at the levels of theoretical learning principles, educational models and educational practices. The McMaster-Maastricht PBL model and the Aalborg model of project work share characteristic features such as the theoretical principle of the problem analysis at the basis of the learning process, integration of knowledge and practice, collaboration and group work. Notable differences were found with respect to the type of assignments, assessment methods and organisation of the group work. In comparison to traditional engineering curricula, the PBL models appear to inspire a higher degree of involvement in study activities and, consequently, a higher level of complex comprehension. A possible drawback is the risk of gaps in specific knowledge areas. Therefore, it is crucial that the students in a PBL curriculum become lifelong learners who have learned to take responsibility for their own learning process.
Article
Not all student teams are created equal. Some manage to produce excellent engineering results, others fabricate it. Social interactions in some teams are respectful, while on other teams some members expect others to carry the load, but take credit for it later. With engineering teamwork becoming more prevalent on engineering campuses, knowing more about student design teams that work is especially important. This article uses two teamwork cases from a large-scale ethnographic study of an engineering design program to describe not only the ways that student engineers practiced design teamwork, but also how campus culture reached into social interactions between teammates via engineering identities produced on campus. A model for effective teamwork emerged that implies producing high quality engineering products, and doing so through respectful social interactions. Implications for teaching about teamwork, teaching with teams, and thinking about ways to change campus cultures to better promote design engineering are developed.
Article
Encouraging skills development is a key part of the UK Government’s strategy. This emphasis on skills has led to changes in Higher Education, where there is an increasing recognition of the need to enhance students’ employability. This paper reviews the data on the type of employer skill needs available in the existing literature with a particular emphasis on those skills required by engineering and manufacturing employers. Statistical data related to skill gaps in engineering and manufacturing in the UK and London/Thames Gateway are presented with the intention to determine whether and how these relate to the employability skills and competences listed by various organisations. The paper arrives at a number of conclusions. In summary, for the ‘engineering and manufacturing workforce skills development’ ethos to become widespread in London (and the UK in general), a lot more needs to be done to understand the needs of industry to be able to develop provision that supports Londoners (and the UK citizens in general) in gaining appropriate skills in engineering. This can only be achieved through cooperative, inclusive, transparent and centrally coordinated approaches to skills assessment, monitoring and development.
Article
It is striking to notice that the notion of skill is not only used in industry but also in higher education in most major industrialized countries. If a significant number of companies have taken a skills approach, one notices that this term is also used in higher education to determine teaching programmes according to a simple idea: higher education no longer simply seeks to transmit knowledge but to train students in skills that they will be able to use themselves and in the development of the economy, as well as work situations that they will encounter once they leave school. One cannot help but notice the emergence of this notion of skills both in industry and in higher education. As researchers, we are in a position to observe, follow and evaluate the establishment of the skills approach in industry; as teachers we must take part in the skills approach initiated by our engineering school, the Ecole des Mines de Nantes. The use of the term ‘skills approach’ in both cases leads one to believe that there is some form of coherence between the two projects and even that the skills approach in higher education is a direct response and an adaptation to the skills approach initiated in industry. This is why it seems worthwhile to elucidate what we mean by the skills approach in industry, to clear up any ambiguities and try to measure what could be, from our point of view, the conclusions that an engineering school can draw for itself given the emergence of the notion of skills in industry.