1887

Assessment of Student Skills for Critiquing Published Primary Scientific Literature Using a Primary Trait Analysis Scale

    Authors: MANUEL F. VARELA1,*, MARVIN M. F. LUTNESKY1, MARCY P. OSGOOD2
    VIEW AFFILIATIONS HIDE AFFILIATIONS
    Affiliations: 1: Biology Department, Eastern New Mexico University, Portales, New Mexico 88130 and; 2: Department of Biochemistry and Molecular Biology, University of New Mexico School of Medicine, Albuquerque, New Mexico 87131
    AUTHOR AND ARTICLE INFORMATION AUTHOR AND ARTICLE INFORMATION
    • *Corresponding author. Mailing address: Department of Biology, Station 33, Eastern New Mexico University, Portales, NM 88130. Phone: (505) 562-2464. Fax: (505) 562-2192. E-mail: Manuel.Varela@enmu.edu.
    • Copyright © 2005, American Society for Microbiology. All Rights Reserved.
    Source: J. Microbiol. Biol. Educ. May 2005 vol. 6 no. 1 20-27. doi:10.1128/154288105X14285806518972
MyBook is a cheap paperback edition of the original book and will be sold at uniform, low price.
  • XML
  • PDF
    147.36 Kb
  • HTML
    48.88 Kb

    Abstract:

    Instructor evaluation of progressive student skills in the analysis of primary literature is critical for the development of these skills in young scientists. Students in a senior or graduate-level one-semester course in Immunology at a Masters-level comprehensive university were assessed for abilities (primary traits) to recognize and evaluate the following elements of a scientific paper: Hypothesis and Rationale, Significance, Methods, Results, Critical Thinking and Analysis, and Conclusions. We tested the hypotheses that average recognition scores vary among elements and that scores change with time differently by trait. Recognition scores (scaled 1 to 5), and differences in scores were analyzed using analysis of variance (ANOVA), regression, and analysis of covariance (ANCOVA) ( = 10 papers over 103 days). By multiple comparisons testing, we found that recognition scores statistically fell into two groups: high scores (for Hypothesis and Rationale, Significance, Methods, and Conclusions) and low scores (for Results and Critical Thinking and Analysis). Recognition scores only significantly changed with time (increased) for Hypothesis and Rationale and Results. ANCOVA showed that changes in recognition scores for these elements were not significantly different in slope (F = 0.254, = 0.621) but the Results trait was significantly lower in elevation (F = 12.456, = 0.003). Thus, students improved with similar trajectories, but starting and ending with lower Results scores. We conclude that students have greatest difficulty evaluating Results and critically evaluating scientific validity. Our findings show extant student skills, and the significant increase in some traits shows learning. This study demonstrates that students start with variable recognition skills and that student skills may be learned at differential rates. Faculty can use these findings or the primary trait analysis scoring scale to focus on specific paper elements for which they desire to improve recognition.

References & Citations

1. Anderson LW, Krathwohl DR2001A taxonomy for learning, teaching, and assessing: a revision of Bloom’s educational objectivesAllyn & BaconBoston
2. Antony-Cahill S2001Using the protein folding literature to teach biophysical chemistry to undergraduatesBiochem Mol Biol Educ294549
3. Bloom B1956Taxonomy of educational objectives: the classification of educational goals Handbook I Cognitive domainLongmans, Green and CoLondon
4. Borthick AF, Dangel H, Springer C2003Pedagogy and assessment that support critical thinkingAAHE learning to change conference 2003, communities of practice, role, and identityAmerican Association for Higher EducationWashington D.C.
5. Fiancarlo CA, Facion PA1997A look across four years at the disposition toward critical thinking among undergraduate studentsJ Gen Educ50295510.1353/jge.2001.0004 http://dx.doi.org/10.1353/jge.2001.0004
6. Fortner RW1999Using cooperative learning to introduce undergraduates to professional literatureJ Coll Sci Teaching28261265
7. Herman C1999Reading the primary literature in the jargon-intensive field of molecular geneticsJ Coll Sci Teaching28252253
8. Houde A2000Student symposia on primary research articlesJ Coll Sci Teaching30184187
9. Janick-Buckner D1997Getting undergraduates to critically read and discuss the primary literatureJ Coll Sci Teaching272932
10. Kitchen E, Bell JD, Reeve S, Sudweeks RR, Bradshaw WS2003Teaching cell biology in the large-enrollment classroom: methods to promote analytical thinking and assessment of their effectivenessCell Biol Educ218019410.1187/cbe.02-11-005514506506192442 http://dx.doi.org/10.1187/cbe.02-11-0055
11. Levine E2001Reading your way to scientific literacyJ Coll Sci Teaching31122125
12. Mangurian L, Feldman S, Clements J, Boucher L2001Analyzing and communicating scientific informationJ Coll Sci Teaching30440445
13. Martin P, Bateson P1986Measuring behaviour: an introductory guideCambridge University PressCambridge, England
14. Muench SB2000Choosing primary literature in biology to achieve specific educational goalsJ Coll Sci Teaching29255260
15. National Research Council2003Bio2010: transforming undergraduate education for future research biologistsThe National Academies PressWashington, D.C
16. Nelson CE1999On the persistence of unicorns: the tradeoff between content and critical thinking revisited Pescosolido BA, Aminzade RThe social worlds of higher education: handbook for teaching in a new centuryPine Forge PressThousand Oaks, Calif
17. Parslow GR2002Commentary: critical thinking: can we teach it? Should we teach it?Biochem Mol Biol Educ306510.1002/bmb.2002.494030010013 http://dx.doi.org/10.1002/bmb.2002.494030010013
18. Perry WG1970Forms of intellectual and ethical development in the college years: a schemeHolt, Rinehart & WinstonNew York
19. Smith CN2002Using the cell signaling literature to teach molecular biology to undergraduatesBiochem Mol Biol Educ3038038310.1002/bmb.2002.494030060154 http://dx.doi.org/10.1002/bmb.2002.494030060154
20. Smith G2001Guided literature explorationsJ Coll Sci Teaching30465469
21. Stokstad E2001Trends in undergraduate educationScience2931608161010.1126/science.293.5535.160811533469 http://dx.doi.org/10.1126/science.293.5535.1608
22. Systat1996SYSTAT for windows, statistics, version 6 edSystatEvanston, Ill
23. Tabor D, Jakobsson E2004The Bio2010 revolution: what it is, why NIGMS is helping to lead it, and why you should join itNIGMS Minority Programs UpdateSpring20043
24. Walvood BE, Johnson-Anderson V1998Effective grading: a tool for learning and assessmentJossey-Bass PublishersSan Francisco
25. Zar JH1974Biostatistical analysisPrentice HallEnglewood Cliffs, N.J
26. Zar JH1999Biostatistical analysis4th edPrentice HallUpper Saddle River, N.J.
154288105X14285806518972.citations
jmbe/6/1
content/journal/jmbe/10.1128/154288105X14285806518972
Loading

Citations loading...

Supplemental Material

No supplementary material available for this content.

Loading

Article metrics loading...

/content/journal/jmbe/10.1128/154288105X14285806518972
2005-05-01
2017-09-20

Abstract:

Instructor evaluation of progressive student skills in the analysis of primary literature is critical for the development of these skills in young scientists. Students in a senior or graduate-level one-semester course in Immunology at a Masters-level comprehensive university were assessed for abilities (primary traits) to recognize and evaluate the following elements of a scientific paper: Hypothesis and Rationale, Significance, Methods, Results, Critical Thinking and Analysis, and Conclusions. We tested the hypotheses that average recognition scores vary among elements and that scores change with time differently by trait. Recognition scores (scaled 1 to 5), and differences in scores were analyzed using analysis of variance (ANOVA), regression, and analysis of covariance (ANCOVA) ( = 10 papers over 103 days). By multiple comparisons testing, we found that recognition scores statistically fell into two groups: high scores (for Hypothesis and Rationale, Significance, Methods, and Conclusions) and low scores (for Results and Critical Thinking and Analysis). Recognition scores only significantly changed with time (increased) for Hypothesis and Rationale and Results. ANCOVA showed that changes in recognition scores for these elements were not significantly different in slope (F = 0.254, = 0.621) but the Results trait was significantly lower in elevation (F = 12.456, = 0.003). Thus, students improved with similar trajectories, but starting and ending with lower Results scores. We conclude that students have greatest difficulty evaluating Results and critically evaluating scientific validity. Our findings show extant student skills, and the significant increase in some traits shows learning. This study demonstrates that students start with variable recognition skills and that student skills may be learned at differential rates. Faculty can use these findings or the primary trait analysis scoring scale to focus on specific paper elements for which they desire to improve recognition.

Highlighted Text: Show | Hide
Loading full text...

Full text loading...

/deliver/fulltext/jmbe/6/1/jmbe-6-1-20.xml.a.html?itemId=/content/journal/jmbe/10.1128/154288105X14285806518972&mimeType=html&fmt=ahah

Figures

Image of FIG. 1

Click to view

FIG. 1

Average score for the six elements evaluated in published papers. Grand average scores shown for all students for all papers during the whole of the semester. There was a significant difference among traits (single-factor ANOVA, F = 8.623, < 0.001). Different letters (a versus b) indicate significant differences among means using Tukey multiple comparison tests (least significant, < 0.04). Sample size () was equal to 10 papers for each bar; error bars equal standard error of the mean.

Source: J. Microbiol. Biol. Educ. May 2005 vol. 6 no. 1 20-27. doi:10.1128/154288105X14285806518972
Download as Powerpoint
Image of FIG. 2

Click to view

FIG. 2

Scores for student recognition and evaluation of components in published literature. The students’ scores are indicated for recognition and critical evaluation of Hypothesis and Rationale (HR), Significance of the Biological Sciences (SBS), Methods, Results, Conclusions, and Critical Thinking Skills and Analysis of validity of the published primary literature in the course as a function of time. Regression lines are shown only for significant relationships ( < 0.05). Sample size was = 10.

Source: J. Microbiol. Biol. Educ. May 2005 vol. 6 no. 1 20-27. doi:10.1128/154288105X14285806518972
Download as Powerpoint
Image of FIG. 3

Click to view

FIG. 3

Scores for student recognition and evaluation of hypothesis and rationale versus results. Student scores are indicated for ability to find and critically evaluate HR (•) and R (○) as a function of time. The regression lines are not significantly different in slope, but they are in elevation (see text), thus students learned with the same trajectory, but had different initial abilities.

Source: J. Microbiol. Biol. Educ. May 2005 vol. 6 no. 1 20-27. doi:10.1128/154288105X14285806518972
Download as Powerpoint

This is a required field
Please enter a valid email address
Please check the format of the address you have entered.
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error