1887

Development of a Tool to Assess Interrelated Experimental Design in Introductory Biology

    Authors: Tess L. Killpack1,‡,*, Sara M. Fulmer2,‡
    VIEW AFFILIATIONS HIDE AFFILIATIONS
    Affiliations: 1: Biology Department, Salem State University, Salem, MA 01970; 2: Open Learning and Educational Support, University of Guelph, Guelph, ON N1G 2W1
    AUTHOR AND ARTICLE INFORMATION AUTHOR AND ARTICLE INFORMATION
    • Received 14 May 2018 Accepted 14 September 2018 Published 31 October 2018
    • ©2018 Author(s). Published by the American Society for Microbiology.
    • [open-access] This is an Open Access article distributed under the terms of the Creative Commons Attribution-Noncommercial-NoDerivatives 4.0 International license (https://creativecommons.org/licenses/by-nc-nd/4.0/ and https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode), which grants the public the nonexclusive right to copy, distribute, or display the published work.

    • Supplemental materials available at http://asmscience.org/jmbe
    • *Corresponding author. Mailing address: Biology Department, Salem State University, 352 Lafayette Street, Salem, MA 01970. Phone: 978-542-3085. E-mail: [email protected].
    • These authors contributed equally to this work.
    Source: J. Microbiol. Biol. Educ. October 2018 vol. 19 no. 3 doi:10.1128/jmbe.v19i3.1627
MyBook is a cheap paperback edition of the original book and will be sold at uniform, low price.
  • PDF
    566.15 Kb
  • HTML
    70.55 Kb
  • XML
    80.12 Kb

    Abstract:

    Designing experiments and applying the process of science are core competencies for many introductory courses and course-based undergraduate research experiences (CUREs). However, experimental design is a complex process that challenges many introductory students. We describe the development of a tool to assess interrelated experimental design (TIED) in an introductory biology lab course. We describe the interrater reliability of the tool, its effectiveness in detecting variability and growth in experimental-design skills, and its adaptability for use in various contexts. The final tool contained five components, each with multiple criteria in the form of a checklist such that a high-quality response—in which students align the different components of their experimental design—satisfies all criteria. The tool showed excellent interrater reliability and captured the full range of introductory-student skill levels, with few students hitting the assessment ceiling or floor. The scoring tool detected growth in student skills from the beginning to the end of the semester, with significant differences between pre- and post-assessment scores for the Total Score and for the Data Collection and Observations component scores. This authentic assessment task and scoring tool provide meaningful feedback to instructors about the strengths, gaps, and growth in introductory students’ experimental-design skills and can be scored reliably by multiple instructors. The TIED can also be adapted to a number of experimental-design prompts and learning objectives, and therefore can be useful for a variety of introductory courses and CUREs.

References & Citations

1. American Association for the Advancement of Science2011Vision and change in undergraduate biology education: a call to action: a summary of recommendations made at a national conference organized by the American Association for the Advancement of ScienceJuly 15–17, 2009Washington DChttp://visionandchange.org/files/2011/03/Revised-Vision-and-Change-Final-Report.pdf
2. Coil D, Wenderoth MP, Cunnigham M, Dirks C2010Teaching the process of science: faculty perceptions and an effective methodologyCBE Life Sci Educ10.1187/cbe.10-01-0005211236992995770 http://dx.doi.org/10.1187/cbe.10-01-0005
3. Jeffery E, Nomme K, Deane T, Pollock C, Birol G2016Investigating the role of an inquiry-based biology lab course on student attitudes and views toward scienceCBE Life Sci Educ15ar6110.1187/cbe.14-11-0203278565495132358 http://dx.doi.org/10.1187/cbe.14-11-0203
4. Shortlidge EE, Bangera G, Brownell SE2016Faculty perspectives on developing and teaching course-based undergraduate research experiencesBioScience66546210.1093/biosci/biv167 http://dx.doi.org/10.1093/biosci/biv167
5. Shortlidge EE, Brownell SE2016How to assess your CURE: a practical guide for instructors of course-based undergraduate research experiencesJ Microbiol Biol Educ17339940810.1128/jmbe.v17i3.1103 http://dx.doi.org/10.1128/jmbe.v17i3.1103
6. Dasgupta AP, Anderson TR, Pelaez N2014Development and validation of a rubric for diagnosing students’ experimental design knowledge and difficultiesCBE Life Sci Educ1326528410.1187/cbe.13-09-01924041504 http://dx.doi.org/10.1187/cbe.13-09-0192
7. Brownell SE, Wenderoth MP, Theobald R, Okoroafor N, Koval M, Freeman S, Walcher-Chevillet CL, Crowe AJ2014How students think about experimental design: novel conceptions revealed by in-class activitiesBioScience64212513710.1093/biosci/bit016 http://dx.doi.org/10.1093/biosci/bit016
8. Deane T, Nomme K, Jeffery E, Pollock C, Birol G2014Development of the biological experimental design concept inventory (BEDCI)CBE Life Sci Educ13354055110.1187/cbe.13-11-0218251852364152214 http://dx.doi.org/10.1187/cbe.13-11-0218
9. Shi J, Power JM, Klymkowsky MW2011Revealing student thinking about experimental design and the roles of control experimentsInt J Scholarsh Teach Learn5116
10. Cooper KM, Soneral PAG, Brownell SE2017Define your goals before you design a CURE: a call to use backward design in planning course-based undergraduate research experiencesJ Microbiol Biol Educ1821710.1128/jmbe.v18i2.1287 http://dx.doi.org/10.1128/jmbe.v18i2.1287
11. National Research Council2003BIO2010: Transforming undergraduate education for future research biologistsThe National Academies PressWashington, DC10.17226/10497 http://dx.doi.org/10.17226/10497
12. Wiggins G1998Ensuring authentic performance2142Educative assessment: designing assessments to inform and improve student performanceJossey-BassSan Francisco, CA
13. Oh DM, Kim JM, Garcia RE, Krilowicz BL2005Valid and reliable authentic assessment of culminating student performance in the biomedical sciencesAdv Physiol Ed292839310.1152/advan.00039.2004 http://dx.doi.org/10.1152/advan.00039.2004
14. Kloser MJ, Brownell SE, Chiariello NR, Fukami T2011Integrating teaching and research in undergraduate biology laboratory educationPLOS Biol911e100117410.1371/journal.pbio.1001174221104003216991 http://dx.doi.org/10.1371/journal.pbio.1001174
15. Dogan A, Kaya ON2009Poster sessions as an authentic assessment approach in an open-ended university general chemistry laboratoryProcedia Soc Behavior Sci1182983310.1016/j.sbspro.2009.01.148 http://dx.doi.org/10.1016/j.sbspro.2009.01.148
16. Laungani R, Tanner C, Brooks TD, Clement B, Clouse M, Doyle E, Dworak S, Elder B, Marley K, Schofield B2018Finding some good in an invasive species: introduction and assessment of a novel CURE to improve experimental design in undergraduate biology classroomsJ Microbiol Biol Educ19219.2.6810.1128/jmbe.v19i2.1517299838456022745 http://dx.doi.org/10.1128/jmbe.v19i2.1517
17. Makarevitch I, Frechette C, Wiatros N2015Authentic research experience and “big data” analysis in the classroom: maize response to abiotic stressCBE Life Sci Educ143ar2710.1187/cbe.15-04-0081261635614710385 http://dx.doi.org/10.1187/cbe.15-04-0081
18. Dolan EL, Collins JP2015We must teach more effectively: here are four ways to get startedMol Biol Cell26122151215510.1091/mbc.e13-11-0675260675664462934 http://dx.doi.org/10.1091/mbc.e13-11-0675
19. Jordan TC, Burnett SH, Carson S, Caruso SM, Clase K, DeJong RJ, Dennehy JJ, Denver DR, Dunbar D, Elgin SCR, Findley AM, Gissendanner CR, Golebiewska UP, Guild N, Hartzog GA, Grillo WH, Hollowell GP, Hughes LE, Johnson A, King RA, Lewis LO, Li W, Rosenzweig F, Rubin MR, Saha MS, Sandoz J, Shaffer CD, Taylor B, Temple L, Vazquez E, Ware VC, Barker LP, Bradley KW, Jacobs-Sera D, Pope WH, Russell DA, Cresawn SG, Lopatto D, Bailey CP, Hatfull GF2014A broadly implementable research course in phage discovery and genomics for first-year undergraduate studentsmBio51e01051-1310.1128/mBio.01051-13244967953950523 http://dx.doi.org/10.1128/mBio.01051-13
20. Dirks C, Cunningham M2006Enhancing diversity in science: is teaching science process skills the answer?CBE Life Sci Educ521822610.1187/cbe.05-10-0121170122131618688 http://dx.doi.org/10.1187/cbe.05-10-0121
21. Sirum K, Humburg J2011The experimental design ability test (EDAT)BioScience371816
22. Brookhart SM2013How to create and use rubrics for formative assessment and gradingASCDAlexandria, VA
23. Peat B2006Integrating writing and research skills: development and testing of a rubric to measure student outcomesJ Public Aff Educ12329531110.1080/15236803.2006.12001437 http://dx.doi.org/10.1080/15236803.2006.12001437
24. Andrade HG2000Using rubrics to promote thinking and learningEduc Leadersh571318
25. Kishbaugh TLS, Cessna S, Horst SJ, Leaman L, Flanagan F, Neufeld DG, Siderhurst M2012Measuring beyond content: a rubric bank for assessing skills in authentic research assignments in the sciencesChem Educ Res Pract1326827610.1039/C2RP00023G http://dx.doi.org/10.1039/C2RP00023G
26. Science Olympiad2017Experimental design checklist for B/CRetrieved from https://www.soinc.org/sites/default/files/uploaded_files/ExDesChecklist17Final.pdf
27. Hallgren KA2012Computing interrater reliability for observational data: an overview and tutorialTutor Quantitat Meth Psychol8233410.20982/tqmp.08.1.p023 http://dx.doi.org/10.20982/tqmp.08.1.p023
28. Mandrekar J2011Measures of interrater agreementBiostat Clinic6167
29. Cicchetti DV1994Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychologyPsychol Assess628429010.1037/1040-3590.6.4.284 http://dx.doi.org/10.1037/1040-3590.6.4.284

Supplemental Material

Loading

Article metrics loading...

/content/journal/jmbe/10.1128/jmbe.v19i3.1627
2018-10-31
2018-12-18

Abstract:

Designing experiments and applying the process of science are core competencies for many introductory courses and course-based undergraduate research experiences (CUREs). However, experimental design is a complex process that challenges many introductory students. We describe the development of a tool to assess interrelated experimental design (TIED) in an introductory biology lab course. We describe the interrater reliability of the tool, its effectiveness in detecting variability and growth in experimental-design skills, and its adaptability for use in various contexts. The final tool contained five components, each with multiple criteria in the form of a checklist such that a high-quality response—in which students align the different components of their experimental design—satisfies all criteria. The tool showed excellent interrater reliability and captured the full range of introductory-student skill levels, with few students hitting the assessment ceiling or floor. The scoring tool detected growth in student skills from the beginning to the end of the semester, with significant differences between pre- and post-assessment scores for the Total Score and for the Data Collection and Observations component scores. This authentic assessment task and scoring tool provide meaningful feedback to instructors about the strengths, gaps, and growth in introductory students’ experimental-design skills and can be scored reliably by multiple instructors. The TIED can also be adapted to a number of experimental-design prompts and learning objectives, and therefore can be useful for a variety of introductory courses and CUREs.

Highlighted Text: Show | Hide
Loading full text...

Full text loading...

/deliver/fulltext/jmbe/19/3/jmbe-19-98.html?itemId=/content/journal/jmbe/10.1128/jmbe.v19i3.1627&mimeType=html&fmt=ahah

Figures

Image of FIGURE 1

Click to view

FIGURE 1

TIED: A scoring tool for interrelated experimental design.

Source: J. Microbiol. Biol. Educ. October 2018 vol. 19 no. 3 doi:10.1128/jmbe.v19i3.1627
Download as Powerpoint
Image of FIGURE 2

Click to view

FIGURE 2

Histograms representing the number of students achieving each total TIED score on the pre- and post-assessment.

Source: J. Microbiol. Biol. Educ. October 2018 vol. 19 no. 3 doi:10.1128/jmbe.v19i3.1627
Download as Powerpoint

This is a required field
Please enter a valid email address
Please check the format of the address you have entered.
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error