1887

Assessment of Student Skills for Critiquing Published Primary Scientific Literature Using a Primary Trait Analysis Scale

    Authors: MANUEL F. VARELA1,*, MARVIN M. F. LUTNESKY1, MARCY P. OSGOOD2
    VIEW AFFILIATIONS HIDE AFFILIATIONS
    Affiliations: 1: Biology Department, Eastern New Mexico University, Portales, New Mexico 88130 and; 2: Department of Biochemistry and Molecular Biology, University of New Mexico School of Medicine, Albuquerque, New Mexico 87131
    AUTHOR AND ARTICLE INFORMATION AUTHOR AND ARTICLE INFORMATION
    • *Corresponding author. Mailing address: Department of Biology, Station 33, Eastern New Mexico University, Portales, NM 88130. Phone: (505) 562-2464. Fax: (505) 562-2192. E-mail: [email protected].
    • Copyright © 2005, American Society for Microbiology. All Rights Reserved.
    Source: J. Microbiol. Biol. Educ. May 2005 vol. 6 no. 1 20-27. doi:10.1128/154288105X14285806518972
MyBook is a cheap paperback edition of the original book and will be sold at uniform, low price.
  • XML
  • PDF
    147.36 Kb
  • HTML
    48.88 Kb

    Abstract:

    Instructor evaluation of progressive student skills in the analysis of primary literature is critical for the development of these skills in young scientists. Students in a senior or graduate-level one-semester course in Immunology at a Masters-level comprehensive university were assessed for abilities (primary traits) to recognize and evaluate the following elements of a scientific paper: Hypothesis and Rationale, Significance, Methods, Results, Critical Thinking and Analysis, and Conclusions. We tested the hypotheses that average recognition scores vary among elements and that scores change with time differently by trait. Recognition scores (scaled 1 to 5), and differences in scores were analyzed using analysis of variance (ANOVA), regression, and analysis of covariance (ANCOVA) ( = 10 papers over 103 days). By multiple comparisons testing, we found that recognition scores statistically fell into two groups: high scores (for Hypothesis and Rationale, Significance, Methods, and Conclusions) and low scores (for Results and Critical Thinking and Analysis). Recognition scores only significantly changed with time (increased) for Hypothesis and Rationale and Results. ANCOVA showed that changes in recognition scores for these elements were not significantly different in slope (F = 0.254, = 0.621) but the Results trait was significantly lower in elevation (F = 12.456, = 0.003). Thus, students improved with similar trajectories, but starting and ending with lower Results scores. We conclude that students have greatest difficulty evaluating Results and critically evaluating scientific validity. Our findings show extant student skills, and the significant increase in some traits shows learning. This study demonstrates that students start with variable recognition skills and that student skills may be learned at differential rates. Faculty can use these findings or the primary trait analysis scoring scale to focus on specific paper elements for which they desire to improve recognition.

References & Citations

1. Anderson LW, Krathwohl DR 2001 A taxonomy for learning, teaching, and assessing: a revision of Bloom’s educational objectives Allyn & Bacon Boston
2. Antony-Cahill S 2001 Using the protein folding literature to teach biophysical chemistry to undergraduates Biochem Mol Biol Educ 29 45 49
3. Bloom B 1956 Taxonomy of educational objectives: the classification of educational goals Handbook I Cognitive domain Longmans, Green and Co London
4. Borthick AF, Dangel H, Springer C 2003 Pedagogy and assessment that support critical thinking AAHE learning to change conference 2003, communities of practice, role, and identity American Association for Higher Education Washington D.C.
5. Fiancarlo CA, Facion PA 1997 A look across four years at the disposition toward critical thinking among undergraduate students J Gen Educ 50 29 55 10.1353/jge.2001.0004 http://dx.doi.org/10.1353/jge.2001.0004
6. Fortner RW 1999 Using cooperative learning to introduce undergraduates to professional literature J Coll Sci Teaching 28 261 265
7. Herman C 1999 Reading the primary literature in the jargon-intensive field of molecular genetics J Coll Sci Teaching 28 252 253
8. Houde A 2000 Student symposia on primary research articles J Coll Sci Teaching 30 184 187
9. Janick-Buckner D 1997 Getting undergraduates to critically read and discuss the primary literature J Coll Sci Teaching 27 29 32
10. Kitchen E, Bell JD, Reeve S, Sudweeks RR, Bradshaw WS 2003 Teaching cell biology in the large-enrollment classroom: methods to promote analytical thinking and assessment of their effectiveness Cell Biol Educ 2 180 194 10.1187/cbe.02-11-0055 14506506 192442 http://dx.doi.org/10.1187/cbe.02-11-0055
11. Levine E 2001 Reading your way to scientific literacy J Coll Sci Teaching 31 122 125
12. Mangurian L, Feldman S, Clements J, Boucher L 2001 Analyzing and communicating scientific information J Coll Sci Teaching 30 440 445
13. Martin P, Bateson P 1986 Measuring behaviour: an introductory guide Cambridge University Press Cambridge, England
14. Muench SB 2000 Choosing primary literature in biology to achieve specific educational goals J Coll Sci Teaching 29 255 260
15. National Research Council 2003 Bio2010: transforming undergraduate education for future research biologists The National Academies Press Washington, D.C
16. Nelson CE 1999 On the persistence of unicorns: the tradeoff between content and critical thinking revisited Pescosolido BA, Aminzade R The social worlds of higher education: handbook for teaching in a new century Pine Forge Press Thousand Oaks, Calif
17. Parslow GR 2002 Commentary: critical thinking: can we teach it? Should we teach it? Biochem Mol Biol Educ 30 65 10.1002/bmb.2002.494030010013 http://dx.doi.org/10.1002/bmb.2002.494030010013
18. Perry WG 1970 Forms of intellectual and ethical development in the college years: a scheme Holt, Rinehart & Winston New York
19. Smith CN 2002 Using the cell signaling literature to teach molecular biology to undergraduates Biochem Mol Biol Educ 30 380 383 10.1002/bmb.2002.494030060154 http://dx.doi.org/10.1002/bmb.2002.494030060154
20. Smith G 2001 Guided literature explorations J Coll Sci Teaching 30 465 469
21. Stokstad E 2001 Trends in undergraduate education Science 293 1608 1610 10.1126/science.293.5535.1608 11533469 http://dx.doi.org/10.1126/science.293.5535.1608
22. Systat 1996 SYSTAT for windows, statistics, version 6 ed Systat Evanston, Ill
23. Tabor D, Jakobsson E 2004 The Bio2010 revolution: what it is, why NIGMS is helping to lead it, and why you should join it NIGMS Minority Programs Update Spring 2004 3
24. Walvood BE, Johnson-Anderson V 1998 Effective grading: a tool for learning and assessment Jossey-Bass Publishers San Francisco
25. Zar JH 1974 Biostatistical analysis Prentice Hall Englewood Cliffs, N.J
26. Zar JH 1999 Biostatistical analysis 4th ed Prentice Hall Upper Saddle River, N.J.

Supplemental Material

No supplementary material available for this content.

Loading

Article metrics loading...

/content/journal/jmbe/10.1128/154288105X14285806518972
2005-05-01
2019-10-14

Abstract:

Instructor evaluation of progressive student skills in the analysis of primary literature is critical for the development of these skills in young scientists. Students in a senior or graduate-level one-semester course in Immunology at a Masters-level comprehensive university were assessed for abilities (primary traits) to recognize and evaluate the following elements of a scientific paper: Hypothesis and Rationale, Significance, Methods, Results, Critical Thinking and Analysis, and Conclusions. We tested the hypotheses that average recognition scores vary among elements and that scores change with time differently by trait. Recognition scores (scaled 1 to 5), and differences in scores were analyzed using analysis of variance (ANOVA), regression, and analysis of covariance (ANCOVA) ( = 10 papers over 103 days). By multiple comparisons testing, we found that recognition scores statistically fell into two groups: high scores (for Hypothesis and Rationale, Significance, Methods, and Conclusions) and low scores (for Results and Critical Thinking and Analysis). Recognition scores only significantly changed with time (increased) for Hypothesis and Rationale and Results. ANCOVA showed that changes in recognition scores for these elements were not significantly different in slope (F = 0.254, = 0.621) but the Results trait was significantly lower in elevation (F = 12.456, = 0.003). Thus, students improved with similar trajectories, but starting and ending with lower Results scores. We conclude that students have greatest difficulty evaluating Results and critically evaluating scientific validity. Our findings show extant student skills, and the significant increase in some traits shows learning. This study demonstrates that students start with variable recognition skills and that student skills may be learned at differential rates. Faculty can use these findings or the primary trait analysis scoring scale to focus on specific paper elements for which they desire to improve recognition.

Highlighted Text: Show | Hide
Loading full text...

Full text loading...

/deliver/fulltext/jmbe/6/1/jmbe-6-1-20.xml.a.html?itemId=/content/journal/jmbe/10.1128/154288105X14285806518972&mimeType=html&fmt=ahah

Figures

Image of FIG. 1

Click to view

FIG. 1

Average score for the six elements evaluated in published papers. Grand average scores shown for all students for all papers during the whole of the semester. There was a significant difference among traits (single-factor ANOVA, F = 8.623, < 0.001). Different letters (a versus b) indicate significant differences among means using Tukey multiple comparison tests (least significant, < 0.04). Sample size () was equal to 10 papers for each bar; error bars equal standard error of the mean.

Source: J. Microbiol. Biol. Educ. May 2005 vol. 6 no. 1 20-27. doi:10.1128/154288105X14285806518972
Download as Powerpoint
Image of FIG. 2

Click to view

FIG. 2

Scores for student recognition and evaluation of components in published literature. The students’ scores are indicated for recognition and critical evaluation of Hypothesis and Rationale (HR), Significance of the Biological Sciences (SBS), Methods, Results, Conclusions, and Critical Thinking Skills and Analysis of validity of the published primary literature in the course as a function of time. Regression lines are shown only for significant relationships ( < 0.05). Sample size was = 10.

Source: J. Microbiol. Biol. Educ. May 2005 vol. 6 no. 1 20-27. doi:10.1128/154288105X14285806518972
Download as Powerpoint
Image of FIG. 3

Click to view

FIG. 3

Scores for student recognition and evaluation of hypothesis and rationale versus results. Student scores are indicated for ability to find and critically evaluate HR (•) and R (○) as a function of time. The regression lines are not significantly different in slope, but they are in elevation (see text), thus students learned with the same trajectory, but had different initial abilities.

Source: J. Microbiol. Biol. Educ. May 2005 vol. 6 no. 1 20-27. doi:10.1128/154288105X14285806518972
Download as Powerpoint

This is a required field
Please enter a valid email address
Please check the format of the address you have entered.
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error