1887

Context Matters: Volunteer Bias, Small Sample Size, and the Value of Comparison Groups in the Assessment of Research-Based Undergraduate Introductory Biology Lab Courses

    Authors: Sara E. Brownell1,*, Matthew J. Kloser2, Tadashi Fukami3, Richard J. Shavelson4
    VIEW AFFILIATIONS HIDE AFFILIATIONS
    Affiliations: 1: School of Life Sciences, Arizona State University, Tempe, AZ 85287; 2: Institute for Educational Initiatives, University of Notre Dame, Notre Dame, IN 46556; 3: Department of Biology, Stanford University, Stanford, CA 94305; 4: Graduate School of Education, Stanford University, Stanford, CA 94305
    AUTHOR AND ARTICLE INFORMATION AUTHOR AND ARTICLE INFORMATION
    • Published 02 December 2013
    • *Corresponding author. Mailing address: School of Life Sciences, PO Box 874501, Arizona State University, Tempe, AZ 85287-4501. Phone: 480-965-0803. Fax: 480-965-6899. E-mail: sebbers@gmail.com.
    • ©2013 Author(s). Published by the American Society for Microbiology.
    Source: J. Microbiol. Biol. Educ. December 2013 vol. 14 no. 2 176-182. doi:10.1128/jmbe.v14i2.609
MyBook is a cheap paperback edition of the original book and will be sold at uniform, low price.
  • HTML
    56.10 Kb
  • PDF
    531.02 Kb
  • XML

    Abstract:

    The shift from cookbook to authentic research-based lab courses in undergraduate biology necessitates the need for evaluation and assessment of these novel courses. Although the biology education community has made progress in this area, it is important that we interpret the effectiveness of these courses with caution and remain mindful of inherent limitations to our study designs that may impact internal and external validity. The specific context of a research study can have a dramatic impact on the conclusions. We present a case study of our own three-year investigation of the impact of a research-based introductory lab course, highlighting how volunteer students, a lack of a comparison group, and small sample sizes can be limitations of a study design that can affect the interpretation of the effectiveness of a course.

Key Concept Ranking

Flowering Plants
0.40466627
Clinical Trials
0.39240363
Fig
0.33799812
0.40466627

References & Citations

1. American Educational Research Association2006Standards for reporting on empirical social science research in AERA publicationsEduc Researcher353340
2. American Psychological Association2001Publication manual of the American Psychological Association5th edWashington, DC
3. Aronson BD, Silveira LA2009From genes to proteins to behavior: a laboratory project that enhances student understanding in cell and molecular biologyCBE Life Sci Educ829130810.1187/cbe.09-07-0048199520982786280 http://dx.doi.org/10.1187/cbe.09-07-0048
4. Baumler DJ, et al2012Using comparative genomics for inquiry-based learning to dissect virulence of Escherichia coli O157: H7 and Yersinia pestisCBE Life Sci Educ11819310.1187/cbe.10-04-0057223836203292067 http://dx.doi.org/10.1187/cbe.10-04-0057
5. Brame CJ, Pruitt WM, Robinson LC2008A molecular genetics laboratory course applying bioinformatics and cell biology in the context of original researchCBE Life Sci Educ741042110.1187/cbe.08-07-0036190474272592051 http://dx.doi.org/10.1187/cbe.08-07-0036
6. Brewer CA, Smith D2011Vision and change in undergraduate biology education: a call to action AmericanAssociation for the Advancement of ScienceWashington, DC
7. Brownell SE, Kloser MJ, Fukami T, Shavelson RJ2012Undergraduate biology lab courses: comparing the impact of traditionally based “cookbook” and authentic research-based courses on student lab experiencesJ Coll Sci Teach411827
8. Campbell DT, Stanley JC1963Experimental and quasi-experimental designs for research137Houghton-MifflinBoston, MA
9. Casem ML2006Student perspectives on curricular change: lessons from an undergraduate lower-division biology coreCBE Life Sci Educ5657510.1187/cbe.05-06-0084170121921635130 http://dx.doi.org/10.1187/cbe.05-06-0084
10. Cox-Paulson EA, Grana TM, Harris MA, Batzli JM2012Studying human disease genes in Caenorhabditis elegans: a molecular genetics laboratory projectCBE Life Sci Educ1116517910.1187/cbe-11-06-0045226655893366902 http://dx.doi.org/10.1187/cbe-11-06-0045
11. Cunningham SC, McNear B, Pearlman RS, Kern SE2006Beverage-agarose gel electrophoresis: an inquiry-based laboratory exercise with virtual adaptationCBE Life Sci Educ528128610.1187/cbe.06-01-0139170122201618695 http://dx.doi.org/10.1187/cbe.06-01-0139
12. Derting TL, Williams KS, Momsen JL, Henkel TP2011Education research: set a high barScience21220122110.1126/science.333.6047.1220-c http://dx.doi.org/10.1126/science.333.6047.1220-c
13. Derting TL, Ebert-May D2010Learner-centered inquiry in undergraduate biology: positive relationships with long-term student achievementCBE Life Sci Educ946247210.1187/cbe.10-02-0011211236932995764 http://dx.doi.org/10.1187/cbe.10-02-0011
14. DiBartolomeis SM2011A semester-long project for teaching basic techniques in molecular biology such as restriction fragment length polymorphism analysis to undergraduate and graduate studentsCBE Life Sci Educ109511010.1187/cbe.10-07-0098213641043046893 http://dx.doi.org/10.1187/cbe.10-07-0098
15. Feldman KA1986The perceived instructional effectiveness of college teachers as related to their personality and attitudinal characteristics: a review and synthesisRes Higher Educ2413921310.1007/BF00991885 http://dx.doi.org/10.1007/BF00991885
16. Fukami T2013Integrating inquiry-based teaching with faculty researchScience3391536153710.1126/science.122985023539590 http://dx.doi.org/10.1126/science.1229850
17. Gess-Newsome J2002Pedagogical content knowledge: an introduction and orientation317Examining pedagogical content knowledgeSpringerNetherlands
18. Gormally C, Brickman P, Lutz M2012Developing a test of scientific literacy skills (TOSLS): measuring undergraduates’ evaluation of scientific information and argumentsCBE Life Sci Educ1136437710.1187/cbe.12-03-0026232228323516792 http://dx.doi.org/10.1187/cbe.12-03-0026
19. Grossman PL, Wilson SM, Shulman LS1989Teachers of substance: Subject matter knowledge for teaching2336 Reynolds MThe knowledge base for beginning teachersPergamonNew York
20. Kloser MJ, Brownell SE, Shavelson RJ, Fukami T2013Effects of a research-based ecology lab course: a study of nonvolunteer achievement, self-confidence, and perception of lab course purposeJ Coll Sci Teach427281
21. Kloser MJ, Brownell SE, Chiariello NR, Fukami T2011Integrating teaching and research in undergraduate biology laboratory educationPLoS biology911e100117410.1371/journal.pbio.1001174221104003216991 http://dx.doi.org/10.1371/journal.pbio.1001174
22. Magnusson S, Krajcik J, Borko H1999Nature, sources, and development of pedagogical content knowledge for science teaching95132 Gess-Newsome J, Lederman NExamining pedagogical content knowledge: The construct and its implications for science educationKluwer Academic PublishersDordrecht, The Netherlands
23. Marczyk G, DeMatteo D, Festinger D2005Essentials of research design and methodology6594John Wiley & SonsHoboken, NJ
24. McComas W2005Laboratory instruction in the service of science teaching and learningScience Teacher272429
25. Morine-Dershimer G, Kent T2002The complex nature and sources of teachers’ pedagogical knowledge2150Examining pedagogical content knowledgeSpringerNetherlands
26. National Research Council (US)Committee on Undergraduate Biology Education to Prepare Research Scientists for the 21st Century, & NetLibrary, Inc2003BIO 2010: transforming undergraduate education for future research biologists
27. Rosenthal R, Rosnow RL1975The volunteer subject121133John Wiley & SonsNew York
28. Ruiz-Primo MA, Briggs D, Iverson H, Talbot R, Shepard LA2011Impact of undergraduate science course innovations on learningScience3311269127010.1126/science.119897621393529 http://dx.doi.org/10.1126/science.1198976
29. Rushton S, Morgan J, Richard M2007Teacher’s Myers-Briggs personality profiles: identifying effective teacher personality traitsTeaching and Teacher Education2343244110.1016/j.tate.2006.12.011 http://dx.doi.org/10.1016/j.tate.2006.12.011
30. Shadish WR, Cook TD, Campbell DT2002Experimental and quasi-experimental designs for generalized causal inference181Houghton Mifflin CompanyNew York
31. Shavelson RJ, Towne L2002Scientific research in education80126The National Academies PressWashington, DC
32. Shulman LS1986Those who understand: knowledge growth in teachingEduc Res15414
33. Sirum K, Humburg J2011The Experimental Design Ability Test (EDAT)Bioscene: J Coll Biol Teach37816
34. Sundberg MD, Armstrong JE, Wischusen EW2005A reappraisal of the status of introductory biology laboratory education in US colleges & universitiesAm Biol Teach6752552910.1662/0002-7685(2005)067[0525:AROTSO]2.0.CO;2 http://dx.doi.org/10.1662/0002-7685(2005)067[0525:AROTSO]2.0.CO;2
35. Tanner K, Allen D2004Approaches to biology teaching and learning: learning styles and the problem of instructional selection—engaging all students in science coursesCell Biol Educ319720110.1187/cbe.04-07-005015592590533116 http://dx.doi.org/10.1187/cbe.04-07-0050
36. Wilson S, Shulman L, Richert A1987“150 different ways of knowing”: representations of knowledge in teaching104123 Calderhead JExploring teachers’ thinkingCassellEastbourne, UK
jmbe.v14i2.609.citations
jmbe/14/2
content/journal/jmbe/10.1128/jmbe.v14i2.609
Loading

Citations loading...

Supplemental Material

No supplementary material available for this content.

Loading

Article metrics loading...

/content/journal/jmbe/10.1128/jmbe.v14i2.609
2013-12-02
2017-07-26

Abstract:

The shift from cookbook to authentic research-based lab courses in undergraduate biology necessitates the need for evaluation and assessment of these novel courses. Although the biology education community has made progress in this area, it is important that we interpret the effectiveness of these courses with caution and remain mindful of inherent limitations to our study designs that may impact internal and external validity. The specific context of a research study can have a dramatic impact on the conclusions. We present a case study of our own three-year investigation of the impact of a research-based introductory lab course, highlighting how volunteer students, a lack of a comparison group, and small sample sizes can be limitations of a study design that can affect the interpretation of the effectiveness of a course.

Highlighted Text: Show | Hide
Loading full text...

Full text loading...

/deliver/fulltext/jmbe/14/2/jmbe-14-176.xml.a.html?itemId=/content/journal/jmbe/10.1128/jmbe.v14i2.609&mimeType=html&fmt=ahah

Figures

Image of FIGURE 1.

Click to view

FIGURE 1.

Likert-scale survey data from a three-year study of a research-based biology lab course. Students were asked a series of questions about (A) their interest in future research (2 questions), (B) their confidence in their ability to do lab-based tasks (6 questions), and (C) their attitudes towards authentic research (4 questions). Student scores on each question on the pre-course survey were subtracted from their scores on the post-course survey and averaged for that block of questions to get the main gain per question. Data shown are from three years that the course was offered to: volunteer students (n = 20 for the cookbook, n = 20 for research-based course), randomized students (n = 33 for cookbook, n = 33 for research-based course), and scaled-up research-based course students (n = 128). * < 0.05 (Note: Data from Cookbook and Research-based Volunteers ( 7 ) and Research-based Randomized students ( 20 ) have previously been published.)

Source: J. Microbiol. Biol. Educ. December 2013 vol. 14 no. 2 176-182. doi:10.1128/jmbe.v14i2.609
Download as Powerpoint
Image of FIGURE 2.

Click to view

FIGURE 2.

Conclusions about the effectiveness of the course differ based on which data are used. (A) The conclusion from only examining the data from the volunteers in the research-based course is that students show gains in both confidence in lab-based tasks and interest in pursuing future research. (B) The conclusion from comparing the volunteers in the research-based course with non-volunteers in the cookbook course is that students in the research-based course show higher gains than those in the cookbook course. (C) The conclusion from assessing non-volunteer students in the research-based course is that students show gains in confidence but no gains in interest. (D) The conclusion from comparing non-volunteers in the research-based course to non-volunteers in the cookbook course is that there are no differences between the students in interest or confidence in their ability to do lab-based tasks.

Source: J. Microbiol. Biol. Educ. December 2013 vol. 14 no. 2 176-182. doi:10.1128/jmbe.v14i2.609
Download as Powerpoint

This is a required field
Please enter a valid email address
Please check the format of the address you have entered.
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error