NSF-Style Peer Review for Teaching Undergraduate Grant-Writing
- SourceAvailable from: Nancy M Trautmann[Show abstract] [Hide abstract]
ABSTRACT: Two studies analyzed impacts of writing and receiving web-mediated peer reviews on revision of research reports by undergraduate science students. After conducting toxicology experiments, 77 students posted draft reports and exchanged double-blind reviews. The first study randomly assigned students to four groups representing full, partial, or no peer review. Students engaging in any aspect of peer review made more revisions than students confined to reviewing their own reports. In the second study, all students engaged in peer review, and the influence of writing versus receiving critiques was analyzed using linear regression. Both studies showed receiving reviews to be more significant than writing them in terms of triggering report revisions. Students valued the peer review experience and credited it with giving them insights about their work. Conclusions address implications for optimal design of online peer review systems and for further research into student learning gains.Educational Technology Research and Development 01/2009; 57(5):685-704. · 1.09 Impact Factor
- [Show abstract] [Hide abstract]
ABSTRACT: The journal IMPULSE offers undergraduates worldwide the opportunity to publish research and serve as peer reviewers for the submissions of others. Undergraduate faculty have recognized the journal's value in engaging students working in their labs in the publication process. However, integration of scientific publication into an undergraduate laboratory classroom setting has been lacking. We report here on a course at Ursinus College where 20 students taking Molecular Neurobiology were required to submit manuscripts to IMPULSE. The syllabus allowed for the laboratory research to coincide with the background research and writing of the manuscript. Students completed their projects on the impact of drugs on the Daphnia magna nervous system while producing manuscripts ready for submission by week 7 of the course. Findings from a survey completed by the students and perceptions of the faculty member teaching the course indicated that students spent much more time writing, were more focused on completing the assays, completed the assays with larger data sets, were more engaged in learning the scientific concepts and were more thorough with their revisions of the paper knowing that it might be published. Further, the professor found she was more thorough in critiquing students' papers knowing they would be externally reviewed. Incorporating journal submission into the course stimulated an in depth writing experience and allowed for a deeper exploration of the topic than students would have experienced otherwise. This case study provides evidence that IMPULSE can be successfully used as a means of incorporating scientific publication into an undergraduate laboratory science course. This approach to teaching undergraduate neuroscience allows for a larger number of students to have hands-on research and scientific publishing experience than would be possible with the current model of a few students in a faculty member's laboratory. This report illustrates that IMPULSE can be incorporated as an integral part of an academic curriculum with positive outcomes on student engagement and performance.Journal of Undergraduate Neuroscience Education 01/2011; 9(2):A84-A91.
inquiry & investigation
benJie g. blAir george r. Cline WilliAM r. boWen
graduate research experience (URE) into its curriculum
for both BA and BS degree programs . This experience
involves the following semester courses:
• BY 370 (Introduction to Research in Biology): two
semester credits, sophomore level
• BY 327 (Directed Studies in Biology) or BY 427
(Independent Study in Biology): variable credits,
• BY 496 (Senior Seminar) two credits, senior level
The Department has recognized the need for both
substantive coursework and an undergraduate research
experience in its biology curriculum . The JSU URE is the
outcome of curricular revision over the past 14 years . BY
370 and BY 496 are required courses, whereas indepen-
dent research involves optional courses intended to inter-
est students in research . BY 327 and BY 427 had been
in the curriculum for some time, available for student
research projects under the tutelage of a faculty mentor .
BY 370 was initiated in 1991 to introduce majors to the
science and art of biological research and grant-writing
in an effort to strengthen the outcome of BY 327 and BY
427 . This course is now offered with multiple sections
and instructors both semesters every year . Today, BY 327
and BY 427 enable students to conduct original field
and/or laboratory research, educational research in biol-
ogy, or substantive library-based research of an approved
biological topic . BY 496, a required course since 2002,
involves both a senior thesis and an oral presentation, the
latter given in a department-wide symposium format . BY
496 is the Biology Department’s capstone course for all
matriculating biology majors . The overall URE Experience
has enabled the Department to integrate “communication
across its curriculum” in a way that was never possible
This paper concerns a modification to BY 370 . In this
course, students now actively participate in an NSF-style
peer review of their own undergraduate research propos-
als in biology .
he Biology Department at Jacksonville State
University (JSU) in Alabama has incorporated an under-
Teaching Grant-Writing to
Undergraduate Biology Majors
Why? Incoming JSU students generally lacked a coher-
ent understanding and appreciation of what was required
to actively engage in scientific inquiry . Hence, they were
unable to construct meaningful hypotheses or to design
appropriate experimentation . Many lacked the verbal and
oral skills to effectively communicate scientific activities to
their peers and the rest of the world . None had written a
grant proposal and few, if any, had engaged in substantive
biological research . Most were not aware that the JSU fac-
ulty who teach biology also engage in research .
The majority of incoming students, as well as those
matriculating in the past, rarely perceived the postgradu-
ate benefits that accrued from doing undergraduate
research . Undergraduates frequently received little or
no formal training in grant-writing and/or research until
they were doing postgraduate research . While some
undergraduates did not always aspire to continue their
education in biology or seek professional education, such
activities did benefit those seeking teaching, technical, or
other positions simply because their resumes now stood
out from other applicants!
In the early years of BY 370, biology faculty and their
research interests were introduced early in the course,
thereby enabling students to identify a potential research
project mentor . An exposure to biological literature fol-
lowed, from searching and accessing literature through
library journals and databases, including the use of appli-
cable and appropriate Internet search engines . Students
then made a critical analysis of selected research articles,
including the good and the bad (for examples of both,
contact one of the authors) . The concept of scientific
process and inquiry (both experimental and descriptive
approaches) were developed with an emphasis on: the role
of observation, asking questions, developing an appropri-
ate hypothesis, and materials and methods . Experimental
design included a basic introduction to statistical analysis,
construction of appropriate tables and graphs, and the use
of other tools (Ambrose & Ambrose, 1995) . The continu-
ing importance of professionalism and ethics in scientific
research and the need to avoid plagiarism were noted . The
value of a meaningful and competitive portfolio at matricu-
lation was emphasized . The course culminated with a
student-generated limited research proposal with budget
(in consultation with a faculty mentor) whose format was
patterned after Sigma Xi’s application form for Grant-in-
Aid of Research (Sigma Xi, 2004) .
benJie g. blAir (e-mail: email@example.com) is Associate
Professor of Biology; george r. Cline (e-mail: gcline@jsu.
edu) is Professor of Biology; and WilliAM r. boWen (e-mail:
firstname.lastname@example.org) is Professor and Department Head
Emeritus of Biology, all at Jacksonville State University,
Jacksonville, AL 36265.
34 The AmericAn Biology TeAcher, Volume 69, no. 1, JAnuAry 2007
NSF-Style Peer Review for Teaching UNdeRgRadUaTe gRaNT-WRiTiNg
undergrAduATe grAnT-WriTing 35
The Initial BY 370 Format: A
The number and quality of student research projects in
BY 327 and BY 437 did increase . A few students gave oral
presentations at an on-campus undergraduate symposium
and later at state and regional meetings . Several students
submitted their research proposals to the Sigma Xi Grants-in-
Aid of Research program and received funding . An occasional
research article was published and more majors were inclined
to continue postgraduate study .
But the faculty soon realized that the project description
in the Sigma Xi application format was limited and did little
more than develop an abstract . It was flawed in that it failed to
expose a student’s potential weakness in effective communica-
tion skills, i .e ., grammar, spelling, technical writing, etc . More
importantly, interactive student participation was limited to
individual contacts with a faculty mentor .
As a result, the student-generated research proposal
became more in-depth with a more extensive analysis of per-
tinent literature leading to an appropriate hypothesis . The
Materials and Methods section, especially the experimental
design, was now expected to be well-developed . However,
it was not until 2002 that the participation issue resolved
itself when the lead author was invited to an NSF CCLI
panel review . The idea emerged that an NSF-style student
peer review of student-generated research proposals could
involve student interaction to a degree never before possible .
This concept was immediately incorporated into the format
of BY 370 .
The Student Research Proposal
Format and style of student-generated research proposals
for BY 370 must adhere to departmental guidelines for format
(font size, etc), literature citation, overall page limits, etc . Each
research proposal now has seven or eight components:
1 . A Title Page with author (proposed student investi-
gator), a brief abstract or summary of the proposed
research project (not more than 12 lines), course and
section, and a line for approval by the faculty mentor .
2 . A second and incomplete Title Page, with title of pro-
posal, abstract, and an identification number provided
by the instructor (for use by the student peer review
panel (student author is not identified) .
3 . An Introduction section with literature review and
4 . A Materials and Methods section with sufficient detail
to support the proposed research . Instrumentation
not currently available in the Department must have
approval for use .
5 . A Timetable for completion of proposed research .
6 . A Literature Cited section that includes those journal
articles, popular articles, books, and Web sites that
were cited in the Introduction and Materials and
Methods sections .
7 . A Bibliography section that includes those publica-
tions read but not cited .
8 . A Budget page whose total estimated costs should not
exceed the maximum funding level set by all instruc-
tors participating in BY370 for that term . Estimated
costs for chemicals and other supplies are based on
prices available in current biological and chemical
supply catalogs maintained in the departmental office .
Estimated travel costs for field work are based on
current university mileage allowances . Expenditures
exceeding $100 and for travel must be justified .
Budget pages are printed on colored paper .
Why budget pages for a student-generated research pro-
posal? Students during routine laboratory exercises in courses
were never confronted with the costs involved . But students
gain an appreciation and respect for the “cost” of doing
research when confronted with assessing equipment needs
and operating costs, the costs associated with routine chemi-
cal and other supplies, and the costs and logistics associated
with travel .
Evaluation of Student-Generated
Each student-generated research proposal submitted
for BY 370 is subjected to two concurrent and independent
evaluations by the student peer review panelist and the course
instructor . The overall criteria for both student and instructor
evaluations are the same, emphasizing the quality and science
of the proposal and its budget, the overall presentation (such
as the review and critical analysis of applicable literature), the
hypothesis, and justification of the funding request . Attention
to spelling and grammar is also expected .
Student Peer Review Evaluation
Student evaluation involves an NSF-style peer review
panel whose purpose is to constructively critique each pro-
posal . A student is arbitrarily assigned as a panelist on one or
more peer review panels, each with four to six members . The
instructor convenes each panel, distributes evaluation forms,
and discusses the criteria for evaluation . Anonymous copies of
the proposals (either electronic or paper versions) submitted
by another section of BY 370 are distributed to each panelist .
Directions and advice (similar to that given to NSF review
panels) are given to each panel for conducting its evaluations .
The amount of imaginary funding allotted to each panel is
announced . The funding level is determined by the instructor,
usually about 25% of the total funding request for all propos-
als assigned to that panel . When possible, a second student
peer review panel is convened to review proposals, thereby
generating a second opinion .
The student panel determines the order in which propos-
als are to be reviewed . Each panelist independently conducts
an evaluation of all proposals assigned to that panel, using the
evaluation forms shown in Figures 1-2 . Panelists are required
to make written comments for each type of evaluation . They
are encouraged to be constructive in their commentary,
providing specific examples of areas within each proposal
that could be improved . The student panelist then rates each
1 = Excellent • 2 = Very good • 3 = Good • 4 = Below
The panel as a whole
reconvenes to critically dis-
cuss each proposal and to
establish an overall panel
rating and ranking . During
this process, a panelist may
abstain from the review
process if either a panelist
or the panel as a whole
has determined a conflict
of interest . Proposals are
recommended for fund-
ing (at or below the level
requested) or rejection . A
ist, serving as an NSF-style
scribe, writes a compos-
ite summary justifying
the panel’s overall rank-
ing and recommendation .
The scribe’s report must
be approved by all panel-
ists . Where there is a dif-
fering opinion, a signed
minority report also may
be submitted . The evalua-
tions by each panelist and
the scribe’s summary, and any minority reports,
are submitted to the instructor . Funding recom-
mendations along with anonymous copies of the
evaluations by each panelist and the scribe (and
minority reports) are distributed to the student
grant applicants .
The grade for this part of BY 370 (the research
proposal and peer review) is determined solely by
the instructor to avoid student bias . In so doing,
the instructor not only independently evaluates
each proposal, but also considers the written
comments by panelists and the composite panel
(i .e ., the scribe) . The instructor looks for the thor-
oughness of an evaluation, grammatical and spell-
ing assessment, constructive critiques, innovative
ideas, and overall participation in the evaluation
process by each panelist . The number of such
evaluations can vary depending on the number
of proposals submitted from the other section .
Therefore, the total number of proposals to be
evaluated is assigned a 100% and evaluation of all
the documents could earn a student full marks . It
is essential that a student peer panel’s rating of a
proposal never be a factor in the final outcome, i .e .,
the course grade .
Undergraduate Student Peer
Review: A Success
Since student peer evaluations do not impact
directly on a student’s grade in BY 370, we believe
the outcome of student peer review has proven to
figure 1. peer panel Composite evaluation form.
Peer Panel Composite evaluation
By 370 section ________________ date ______________ Proposal number ___________________
Proposal Title _________________________________________________________________
Proposal rating ____________
not recommended? ______
______ Amount Funded? ____________
other Panelists __________________________
figure 2. peer panelists evaluation.
Peer Panelist evaluation
Proposal number ___________________________________________
Proposal title _____________________________________________
rATing scAle: 1- excellent; 2 - Very good; 3 - good; 4 - Fair; 5 – Poor
PArT i - QuAliTy oF science
Part i rating
review/hypothesis adequate? _____
literature search adequate? _____
estimated costs appropriate?
PArT ii – QuAliTy oF ProPosAl
Part ii rating
__________ literature review adequate?
PArT iii – guidelines
cover page format met? __________
Page limits met?
Budget format met?
Part iii rating
36 The AmericAn Biology TeAcher, Volume 69, no. 1, JAnuAry 2007
be an invaluable learning experience . This was evident from
some of the written comments by panelists and scribes:
Quality of proposal
This proposal is great. • Excellent, a great idea. • Even
I would fund it . • There is no justification for funding
This proposal is easy to follow. • Proposal confusing. •
Does not flow. Does not appear to have been read before
submission. • Needs to be reworked. • Abstract does not
Technical skills lacking. • Sentences repeated or contra-
dictory statements. Too many mistakes. • Needs rework-
ing. • Mentor’s name misspelled or not identified. •
There is “Spell Check.”
Quality of the science
Proposal not well-developed. • No hypothesis. • Literature
review insufficient and/or lacking. • It is not clear how
research can be accomplished with existing facilities.
No timeline was given. • No double spacing. • Etc .
These comments reflect the serious and critical nature
of student peer review . Their expectations were high; in fact,
they were far more demanding of themselves than were the
faculty . In general, they thoroughly scrutinized proposals .
Their reviews, especially those of the panels as a whole, often
provided insight that otherwise might have been overlooked .
Student conflict of interest surprisingly was infrequent .
Students were assessed as to their perception of scientific
inquiry before and after BY 370 . Only those results pertinent
to grant-writing and peer review were considered in this paper
and these were not statistically significant due to small sample
size . The survey did suggest that student perception of grant-
writing (i .e ., development of a research proposal and budget)
improved as students became aware of the competitive edge
that such activities provide for postgraduate employment and
education as well as professional advancement . Awareness
and understanding of the scientific process in acquiring
scientific knowledge as well as appreciation of the faculty’s
research activities has certainly increased . This course has
made students more cognizant of the need for basic courses
in genetics, cell biology, and even ecology . Finally, it has pro-
moted the need for biology majors to take computer science
and scientific writing .
Student peer review not only provides valuable input
into a project’s experimental design but also ensures meritori-
ous dispersal of the available research funds (Gift & Krasny,
2003) . Carlsen, Cunningham, and Trautmann (2001) con-
cluded that peer review separates science from non-science
and therefore is an essential component of scientific inquiry .
There is little doubt that NSF-style peer review in com-
bination with grant-writing as an integral component of
a required undergraduate research experience has had a
positive impact on the JSU biology major . The latter certainly
provides BY 370 students with a unique learning experience
through the peer review panel process . Now, the JSU major
is better prepared to experience scientific inquiry through
BY 327 and BY 427, culminating in the capstone BY 496, the
Senior Seminar . Verbal and oral communicative skills also
have improved considerably, a fact that has been noted by
other JSU departments . It also means that our matriculat-
ing majors now have a substantive and meaningful resume;
hence, they have established a competitive edge over students
from other institutions when seeking either postgraduate edu-
cation (including professional schools such as medical and
pharmacy schools), or employment in teaching or technical
positions . Even those majors entering the K-12 teaching field
no longer fear the daunting task of writing grant proposals in
order to acquire external funding .
This approach to undergraduate scientific inquiry has
been a highly successful endeavor at JSU, a regional public
institution of higher education, and is very applicable to
any science program . It also has greatly enhanced the aca-
demic reputation of the Biology Department throughout the
Southeast region .
The authors wish to thank other members of the JSU
Biology Faculty for their input into the successful implemen-
tation of BY 370, especially LaJoyce Debro, Mark Meade and
Frank Romano .
Ambrose III, H .W . & Ambrose, K .P . (1995) . A Handbook of
Biological Investigation, Fifth Edition. Winston-Salem, NC:
Hunter Textbooks, Inc .
Carlsen, W .S ., Cunningham, C .M . & Trautmann, N .M . (2001) .
Peer review by school science students: Its role in scientific
inquiry . Paper presented at the Annual Meeting of the National
Association for Research in Science Teaching . St . Louis, Missouri .
Available online at: http://www .ei .cornell .edu/pubs/ .
Gift, N . & Krasny, M . (2003) . The great fossil fiasco: Teaching about
peer review . The American Biology Teacher, 65(4), 270-273 .
Sigma Xi . (2004) . Grants-in-Aid of Research program . Available
online at: http://www .giar .sigmaxi .org/Giar .asp/ .
undergrAduATe grAnT-WriTing 37
mAke plAns for
see page 60.