1887

Context Matters: Volunteer Bias, Small Sample Size, and the Value of Comparison Groups in the Assessment of Research-Based Undergraduate Introductory Biology Lab Courses

    Authors: Sara E. Brownell1,*, Matthew J. Kloser2, Tadashi Fukami3, Richard J. Shavelson4
    VIEW AFFILIATIONS HIDE AFFILIATIONS
    Affiliations: 1: School of Life Sciences, Arizona State University, Tempe, AZ 85287; 2: Institute for Educational Initiatives, University of Notre Dame, Notre Dame, IN 46556; 3: Department of Biology, Stanford University, Stanford, CA 94305; 4: Graduate School of Education, Stanford University, Stanford, CA 94305
    AUTHOR AND ARTICLE INFORMATION AUTHOR AND ARTICLE INFORMATION
    • Published 02 December 2013
    • *Corresponding author. Mailing address: School of Life Sciences, PO Box 874501, Arizona State University, Tempe, AZ 85287-4501. Phone: 480-965-0803. Fax: 480-965-6899. E-mail: [email protected].
    • ©2013 Author(s). Published by the American Society for Microbiology.
    Source: J. Microbiol. Biol. Educ. December 2013 vol. 14 no. 2 176-182. doi:10.1128/jmbe.v14i2.609
MyBook is a cheap paperback edition of the original book and will be sold at uniform, low price.
  • HTML
    56.10 Kb
  • PDF
    531.02 Kb
  • XML

    Abstract:

    The shift from cookbook to authentic research-based lab courses in undergraduate biology necessitates the need for evaluation and assessment of these novel courses. Although the biology education community has made progress in this area, it is important that we interpret the effectiveness of these courses with caution and remain mindful of inherent limitations to our study designs that may impact internal and external validity. The specific context of a research study can have a dramatic impact on the conclusions. We present a case study of our own three-year investigation of the impact of a research-based introductory lab course, highlighting how volunteer students, a lack of a comparison group, and small sample sizes can be limitations of a study design that can affect the interpretation of the effectiveness of a course.

Key Concept Ranking

Flowering Plants
0.40466627
Clinical Trials
0.39240363
Fig
0.33799812
0.40466627

References & Citations

1. American Educational Research Association 2006 Standards for reporting on empirical social science research in AERA publications Educ Researcher 35 33 40
2. American Psychological Association 2001 Publication manual of the American Psychological Association 5th ed Washington, DC
3. Aronson BD, Silveira LA 2009 From genes to proteins to behavior: a laboratory project that enhances student understanding in cell and molecular biology CBE Life Sci Educ 8 291 308 10.1187/cbe.09-07-0048 19952098 2786280 http://dx.doi.org/10.1187/cbe.09-07-0048
4. Baumler DJ, et al 2012 Using comparative genomics for inquiry-based learning to dissect virulence of Escherichia coli O157: H7 and Yersinia pestis CBE Life Sci Educ 11 81 93 10.1187/cbe.10-04-0057 22383620 3292067 http://dx.doi.org/10.1187/cbe.10-04-0057
5. Brame CJ, Pruitt WM, Robinson LC 2008 A molecular genetics laboratory course applying bioinformatics and cell biology in the context of original research CBE Life Sci Educ 7 410 421 10.1187/cbe.08-07-0036 19047427 2592051 http://dx.doi.org/10.1187/cbe.08-07-0036
6. Brewer CA, Smith D 2011 Vision and change in undergraduate biology education: a call to action American Association for the Advancement of Science Washington, DC
7. Brownell SE, Kloser MJ, Fukami T, Shavelson RJ 2012 Undergraduate biology lab courses: comparing the impact of traditionally based “cookbook” and authentic research-based courses on student lab experiences J Coll Sci Teach 41 18 27
8. Campbell DT, Stanley JC 1963 Experimental and quasi-experimental designs for research 1 37 Houghton-Mifflin Boston, MA
9. Casem ML 2006 Student perspectives on curricular change: lessons from an undergraduate lower-division biology core CBE Life Sci Educ 5 65 75 10.1187/cbe.05-06-0084 17012192 1635130 http://dx.doi.org/10.1187/cbe.05-06-0084
10. Cox-Paulson EA, Grana TM, Harris MA, Batzli JM 2012 Studying human disease genes in Caenorhabditis elegans: a molecular genetics laboratory project CBE Life Sci Educ 11 165 179 10.1187/cbe-11-06-0045 22665589 3366902 http://dx.doi.org/10.1187/cbe-11-06-0045
11. Cunningham SC, McNear B, Pearlman RS, Kern SE 2006 Beverage-agarose gel electrophoresis: an inquiry-based laboratory exercise with virtual adaptation CBE Life Sci Educ 5 281 286 10.1187/cbe.06-01-0139 17012220 1618695 http://dx.doi.org/10.1187/cbe.06-01-0139
12. Derting TL, Williams KS, Momsen JL, Henkel TP 2011 Education research: set a high bar Science 2 1220 1221 10.1126/science.333.6047.1220-c http://dx.doi.org/10.1126/science.333.6047.1220-c
13. Derting TL, Ebert-May D 2010 Learner-centered inquiry in undergraduate biology: positive relationships with long-term student achievement CBE Life Sci Educ 9 462 472 10.1187/cbe.10-02-0011 21123693 2995764 http://dx.doi.org/10.1187/cbe.10-02-0011
14. DiBartolomeis SM 2011 A semester-long project for teaching basic techniques in molecular biology such as restriction fragment length polymorphism analysis to undergraduate and graduate students CBE Life Sci Educ 10 95 110 10.1187/cbe.10-07-0098 21364104 3046893 http://dx.doi.org/10.1187/cbe.10-07-0098
15. Feldman KA 1986 The perceived instructional effectiveness of college teachers as related to their personality and attitudinal characteristics: a review and synthesis Res Higher Educ 24 139 213 10.1007/BF00991885 http://dx.doi.org/10.1007/BF00991885
16. Fukami T 2013 Integrating inquiry-based teaching with faculty research Science 339 1536 1537 10.1126/science.1229850 23539590 http://dx.doi.org/10.1126/science.1229850
17. Gess-Newsome J 2002 Pedagogical content knowledge: an introduction and orientation 3 17 Examining pedagogical content knowledge Springer Netherlands
18. Gormally C, Brickman P, Lutz M 2012 Developing a test of scientific literacy skills (TOSLS): measuring undergraduates’ evaluation of scientific information and arguments CBE Life Sci Educ 11 364 377 10.1187/cbe.12-03-0026 23222832 3516792 http://dx.doi.org/10.1187/cbe.12-03-0026
19. Grossman PL, Wilson SM, Shulman LS 1989 Teachers of substance: Subject matter knowledge for teaching 23 36 Reynolds M The knowledge base for beginning teachers Pergamon New York
20. Kloser MJ, Brownell SE, Shavelson RJ, Fukami T 2013 Effects of a research-based ecology lab course: a study of nonvolunteer achievement, self-confidence, and perception of lab course purpose J Coll Sci Teach 42 72 81
21. Kloser MJ, Brownell SE, Chiariello NR, Fukami T 2011 Integrating teaching and research in undergraduate biology laboratory education PLoS biology 9 11 e1001174 10.1371/journal.pbio.1001174 22110400 3216991 http://dx.doi.org/10.1371/journal.pbio.1001174
22. Magnusson S, Krajcik J, Borko H 1999 Nature, sources, and development of pedagogical content knowledge for science teaching 95 132 Gess-Newsome J, Lederman N Examining pedagogical content knowledge: The construct and its implications for science education Kluwer Academic Publishers Dordrecht, The Netherlands
23. Marczyk G, DeMatteo D, Festinger D 2005 Essentials of research design and methodology 65 94 John Wiley & Sons Hoboken, NJ
24. McComas W 2005 Laboratory instruction in the service of science teaching and learning Science Teacher 27 24 29
25. Morine-Dershimer G, Kent T 2002 The complex nature and sources of teachers’ pedagogical knowledge 21 50 Examining pedagogical content knowledge Springer Netherlands
26. National Research Council (US) Committee on Undergraduate Biology Education to Prepare Research Scientists for the 21st Century, & NetLibrary, Inc 2003 BIO 2010: transforming undergraduate education for future research biologists
27. Rosenthal R, Rosnow RL 1975 The volunteer subject 121 133 John Wiley & Sons New York
28. Ruiz-Primo MA, Briggs D, Iverson H, Talbot R, Shepard LA 2011 Impact of undergraduate science course innovations on learning Science 331 1269 1270 10.1126/science.1198976 21393529 http://dx.doi.org/10.1126/science.1198976
29. Rushton S, Morgan J, Richard M 2007 Teacher’s Myers-Briggs personality profiles: identifying effective teacher personality traits Teaching and Teacher Education 23 432 441 10.1016/j.tate.2006.12.011 http://dx.doi.org/10.1016/j.tate.2006.12.011
30. Shadish WR, Cook TD, Campbell DT 2002 Experimental and quasi-experimental designs for generalized causal inference 1 81 Houghton Mifflin Company New York
31. Shavelson RJ, Towne L 2002 Scientific research in education 80 126 The National Academies Press Washington, DC
32. Shulman LS 1986 Those who understand: knowledge growth in teaching Educ Res 15 4 14
33. Sirum K, Humburg J 2011 The Experimental Design Ability Test (EDAT) Bioscene: J Coll Biol Teach 37 8 16
34. Sundberg MD, Armstrong JE, Wischusen EW 2005 A reappraisal of the status of introductory biology laboratory education in US colleges & universities Am Biol Teach 67 525 529 10.1662/0002-7685(2005)067[0525:AROTSO]2.0.CO;2 http://dx.doi.org/10.1662/0002-7685(2005)067[0525:AROTSO]2.0.CO;2
35. Tanner K, Allen D 2004 Approaches to biology teaching and learning: learning styles and the problem of instructional selection—engaging all students in science courses Cell Biol Educ 3 197 201 10.1187/cbe.04-07-0050 15592590 533116 http://dx.doi.org/10.1187/cbe.04-07-0050
36. Wilson S, Shulman L, Richert A 1987 “150 different ways of knowing”: representations of knowledge in teaching 104 123 Calderhead J Exploring teachers’ thinking Cassell Eastbourne, UK

Supplemental Material

No supplementary material available for this content.

Loading

Article metrics loading...

/content/journal/jmbe/10.1128/jmbe.v14i2.609
2013-12-02
2019-03-20

Abstract:

The shift from cookbook to authentic research-based lab courses in undergraduate biology necessitates the need for evaluation and assessment of these novel courses. Although the biology education community has made progress in this area, it is important that we interpret the effectiveness of these courses with caution and remain mindful of inherent limitations to our study designs that may impact internal and external validity. The specific context of a research study can have a dramatic impact on the conclusions. We present a case study of our own three-year investigation of the impact of a research-based introductory lab course, highlighting how volunteer students, a lack of a comparison group, and small sample sizes can be limitations of a study design that can affect the interpretation of the effectiveness of a course.

Highlighted Text: Show | Hide
Loading full text...

Full text loading...

/deliver/fulltext/jmbe/14/2/jmbe-14-176.xml.a.html?itemId=/content/journal/jmbe/10.1128/jmbe.v14i2.609&mimeType=html&fmt=ahah

Figures

Image of FIGURE 1.

Click to view

FIGURE 1.

Likert-scale survey data from a three-year study of a research-based biology lab course. Students were asked a series of questions about (A) their interest in future research (2 questions), (B) their confidence in their ability to do lab-based tasks (6 questions), and (C) their attitudes towards authentic research (4 questions). Student scores on each question on the pre-course survey were subtracted from their scores on the post-course survey and averaged for that block of questions to get the main gain per question. Data shown are from three years that the course was offered to: volunteer students (n = 20 for the cookbook, n = 20 for research-based course), randomized students (n = 33 for cookbook, n = 33 for research-based course), and scaled-up research-based course students (n = 128). * < 0.05 (Note: Data from Cookbook and Research-based Volunteers ( 7 ) and Research-based Randomized students ( 20 ) have previously been published.)

Source: J. Microbiol. Biol. Educ. December 2013 vol. 14 no. 2 176-182. doi:10.1128/jmbe.v14i2.609
Download as Powerpoint
Image of FIGURE 2.

Click to view

FIGURE 2.

Conclusions about the effectiveness of the course differ based on which data are used. (A) The conclusion from only examining the data from the volunteers in the research-based course is that students show gains in both confidence in lab-based tasks and interest in pursuing future research. (B) The conclusion from comparing the volunteers in the research-based course with non-volunteers in the cookbook course is that students in the research-based course show higher gains than those in the cookbook course. (C) The conclusion from assessing non-volunteer students in the research-based course is that students show gains in confidence but no gains in interest. (D) The conclusion from comparing non-volunteers in the research-based course to non-volunteers in the cookbook course is that there are no differences between the students in interest or confidence in their ability to do lab-based tasks.

Source: J. Microbiol. Biol. Educ. December 2013 vol. 14 no. 2 176-182. doi:10.1128/jmbe.v14i2.609
Download as Powerpoint

This is a required field
Please enter a valid email address
Please check the format of the address you have entered.
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error