1887

Development, Validation, and Application of the Microbiology Concept Inventory

    Authors: Timothy D. Paustian1,*, Amy G. Briggs2, Robert E. Brennan3, Nancy Boury4, John Buchner5, Shannon Harris1, Rachel E. A. Horak6, Lee E. Hughes7, D. Sue Katz-Amburn8, Maria J. Massimelli9, Ann H. McDonald10, Todd P. Primm11, Ann C. Smith5, Ann M. Stevens12, Sunny B. Yung11
    VIEW AFFILIATIONS HIDE AFFILIATIONS
    Affiliations: 1: University of Wisconsin-Madison, Madison, WI 53706; 2: Beloit College, Beloit, WI 53511; 3: University of Central Oklahoma, Edmond, OK 73034; 4: Iowa State University, Ames, IA 50011; 5: University of Maryland, College Park, MD 20742; 6: American Society of Microbiology, Washington, DC 20036; 7: University of North Texas, Denton, TX 76203; 8: Rogers State University, Claremore, OK 74017; 9: University of California – Irvine, Irvine, CA 92697; 10: Concordia University Wisconsin, Mequon, WI 53097; 11: Sam Houston State University, Huntsville, TX 77340; 12: Virginia Tech, Blacksburg, VA, 24061
    AUTHOR AND ARTICLE INFORMATION AUTHOR AND ARTICLE INFORMATION
    • Received 13 March 2017 Accepted 28 July 2017 Published 05 October 2017
    • ©2017 Author(s). Published by the American Society for Microbiology
    • [open-access] This is an Open Access article distributed under the terms of the Creative Commons Attribution-Noncommercial-NoDerivatives 4.0 International license (https://creativecommons.org/licenses/by-nc-nd/4.0/ and https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode), which grants the public the nonexclusive right to copy, distribute, or display the published work.

    • Supplemental materials available at http://asmscience.org/jmbe
    • *Corresponding author: Mailing address: Microbial Sciences Building, Room 2517, University of Wisconsin-Madison, 1550 Linden Drive, Madison, WI 53706. Phone: 608-263-4921. E-mail: [email protected].
    Source: J. Microbiol. Biol. Educ. October 2017 vol. 18 no. 3 doi:10.1128/jmbe.v18i3.1320
MyBook is a cheap paperback edition of the original book and will be sold at uniform, low price.
  • PDF
    577.79 Kb
  • XML
    84.60 Kb
  • HTML
    67.70 Kb

    Abstract:

    If we are to teach effectively, tools are needed to measure student learning. A widely used method for quickly measuring student understanding of core concepts in a discipline is the concept inventory (CI). Using the American Society for Microbiology Curriculum Guidelines (ASMCG) for microbiology, faculty from 11 academic institutions created and validated a new microbiology concept inventory (MCI). The MCI was developed in three phases. In phase one, learning outcomes and fundamental statements from the ASMCG were used to create T/F questions coupled with open responses. In phase two, the 743 responses to MCI 1.0 were examined to find the most common misconceptions, which were used to create distractors for multiple-choice questions. MCI 2.0 was then administered to 1,043 students. The responses of these students were used to create MCI 3.0, a 23-question CI that measures students’ understanding of all 27 fundamental statements. MCI 3.0 was found to be reliable, with a Cronbach’s alpha score of 0.705 and Ferguson’s delta of 0.97. Test item analysis demonstrated good validity and discriminatory power as judged by item difficulty, item discrimination, and point-biserial correlation coefficient. Comparison of pre- and posttest scores showed that microbiology students at 10 institutions showed an increase in understanding of concepts after instruction, except for questions probing metabolism (average normalized learning gain was 0.15). The MCI will enable quantitative analysis of student learning gains in understanding microbiology, help to identify misconceptions, and point toward areas where efforts should be made to develop teaching approaches to overcome them.

Key Concept Ranking

Horizontal Gene Transfer
0.44602397
0.44602397

References & Citations

1. President’s Council of Advisors on Science and Technology (PCAST) 2012 Engage to excel: producing one million additional college graduates with degrees in science, technology, engineering, and mathematics Executive Office of the President Washington, DC
2. Handelsman J, Ebert-May D, Beichner R, Bruns P, Chang A, Dehaan R, Gentile J, Lauffer S, Stewart J, Tilghman SM, Wood WB 2004 Scientific teaching Source Sci New Ser 304 521 522
3. Dirks C, Wenderoth MP, Withers M 2014 Assessment in the college science classroom W. H. Freeman United States
4. Brown PC, Roediger HL III, McDaniel MA 2014 Make it stick: the science of successful learning Belknap Press United States 10.4159/9780674419377 http://dx.doi.org/10.4159/9780674419377
5. Freeman S, Eddy SL, McDonough M, Smith MK, Okoroafor N, Jordt H, Wenderoth MP 2014 Active learning increases student performance in science, engineering, and mathematics Proc Natl Acad Sci USA 111 8410 8415 10.1073/pnas.1319030111 24821756 4060654 http://dx.doi.org/10.1073/pnas.1319030111
6. Hestenes D, Wells M, Swackhamer G 1992 Force concept inventory Phys Teach 30 141 10.1119/1.2343497 http://dx.doi.org/10.1119/1.2343497
7. Savinainen A, Scott P 2002 The force concept inventory: a tool for monitoring student learning Phys Educ 37 45 52 10.1088/0031-9120/37/1/306 http://dx.doi.org/10.1088/0031-9120/37/1/306
8. Sadler PM, Coyle H, Miller JL, Cook-Smith N, Dussault M, Gould RR 2009 The astronomy and space science concept inventory: development and validation of assessment instruments aligned with the K–12 national science standards Astron Educ Rev 8 10111 10.3847/AER2009024 http://dx.doi.org/10.3847/AER2009024
9. Aslanides JS, Savage CM 2013 Relativity concept inventory: development, analysis, and results Phys Rev Spec Top Phys Educ Res 9
10. Herman GL, Loui M, Zilles C 2010 Creating the digital logic concept inventory 102 106 Proceedings of the 41st ACM Technical Symposium on Computer Science Education
11. Stone A, Allen K, Rhoads TR, Murphy TJ, Shehab RL, Saha C 2003 The statistics concept inventory: a pilot study T3D1 T3D6 Proceedings – Frontiers in Education Conference, FIE
12. Epstein J 2007 Development and validation of the calculus concept inventory 165 170 Proceedings of the Ninth International Conference on Mathematics Education in a Global Community
13. Almstrum VL, Henderson PB, Harvey V, Heeren C, Marion W, Riedesel C, Soh L-K, Tew AE 2006 Concept inventories in computer science for the topic discrete mathematics ACM SIGCSE Bull 38 132 10.1145/1189136.1189182 http://dx.doi.org/10.1145/1189136.1189182
14. Krause S, Birk J, Bauer R, Jenkins B, Pavelich MJ 2004 Development, testing, and application of a chemistry concept inventory 103 107 34th Annual Frontiers in Education, 2004. FIE 2004
15. Knudson D 2006 Biomechanics concept inventory Percept Mot Skills 103 81 82 10.2466/pms.103.1.81-82 17037646 http://dx.doi.org/10.2466/pms.103.1.81-82
16. D’Avanzo C 2008 Biology concept inventories: overview, status, and next steps Bioscience 58 1079 10.1641/B581111 http://dx.doi.org/10.1641/B581111
17. Newman DL, Snyder CW, Fisk JN, Wright LK 2016 Development of the central dogma concept inventory (CDCI) assessment tool CBE Life Sci Educ 15 2 ar9 10.1187/cbe.15-06-0124 27055775 4909347 http://dx.doi.org/10.1187/cbe.15-06-0124
18. Wang JR 2004 Development and validation of a two-tier instrument to examine understanding of internal transport in plants and the human circulatory system Int J Sci Math Educ 2 131 157 10.1007/s10763-004-9323-2 http://dx.doi.org/10.1007/s10763-004-9323-2
19. Deane T, Nomme K, Jeffery E, Pollock C, Birol G 2014 Development of the biological experimental design concept inventory (BEDCI) CBE Life Sci Educ 13 540 551 10.1187/cbe.13-11-0218 25185236 4152214 http://dx.doi.org/10.1187/cbe.13-11-0218
20. Lin SW 2004 Development and application of a two-tier diagnostic test for high school students’ understanding of flowering plant growth and development Int J Sci Math Educ 2 175 199 10.1007/s10763-004-6484-y http://dx.doi.org/10.1007/s10763-004-6484-y
21. Odom AL, Barrow LH 1995 Development and application of a two-tier diagnostic test measuring college biology students’ understanding of diffusion and osmosis after a course of instruction J Res Sci Teach 32 45 61 10.1002/tea.3660320106 http://dx.doi.org/10.1002/tea.3660320106
22. Smith MK, Wood WB, Knight JK 2008 The genetics concept assessment: a new concept inventory for gauging student understanding of genetics CBE Life Sci Educ 7 422 430 10.1187/cbe.08-08-0045 19047428 2592048 http://dx.doi.org/10.1187/cbe.08-08-0045
23. Kalas P, O’Neill A, Pollock C, Birol G 2013 Development of a meiosis concept inventory CBE Life Sci Educ 12 655 664 10.1187/cbe.12-10-0174 24297292 3846516 http://dx.doi.org/10.1187/cbe.12-10-0174
24. Couch BA, Wood WB, Knight JK 2015 The molecular biology capstone assessment: a concept assessment for upper-division molecular biology students CBE Life Sci Educ 14 ar10 10.1187/cbe.14-04-0071 25713098 4353076 http://dx.doi.org/10.1187/cbe.14-04-0071
25. Stefanski KM, Gardner GE, Seipelt-Thiemann RL 2016 Development of a lac operon concept inventory (LOCI) CBE Life Sci Educ 15 ar24 10.1187/cbe.15-07-0162 27252300 4909346 http://dx.doi.org/10.1187/cbe.15-07-0162
26. Marbach-Ad G, Briken V, El-Sayed NM, Frauwirth K, Fredericksen B, Hutcheson S, Gao LY, Joseph SW, Lee V, McIver KS, Mosser D, Booth Quimby B, Shields P, Song W, Stein DC, Yuan RT, Smith AC 2009 Assessing student understanding of host-pathogen interactions using a concept inventory J Microbiol Biol Educ 10 43 50 10.1128/jmbe.v10.98 http://dx.doi.org/10.1128/jmbe.v10.98
27. Horak REA, Merkel S, Chang A 2015 The ASM curriculum guidelines for undergraduate microbiology: a case study of the advocacy role of societies in reform efforts J Microbiol Biol Educ 16 100 104 10.1128/jmbe.v16i1.915 25949769 4416496 http://dx.doi.org/10.1128/jmbe.v16i1.915
28. Stevens A, Liao M-K, Merkel S, Chang A 2014 ASM task force for learning outcomes – ASM general microbiology learning outcome examples https://www.asm.org/images/Education/FINAL_Learning_Outcomes_w_title_page.pdf
29. Seitz HM, Horak REA, Howard MW, Kluckhohn Jones LW, Muth T, Parker C, Rediske AP, Whitehurst MM 2017 Development and validation of the microbiology for health sciences concept inventory J Microbiol Biol Educ 18 3 1 10
30. Briggs AG, Hughes LE, Brennan RE, Buchner J, Horak REA, Katz-Amburn DS, McDonald AH, Primm TP, Smith AC, Stevens AM, Yung SB, Paustian TD 2017 Concept inventory development reveals common student misconceptions about microbiology J Microbiol Biol Educ 18 3 1 9
31. Adams WK, Wieman CE 2011 Development and validation of instruments to measure learning of expert-like thinking Int J Sci Educ 33 1289 1312 10.1080/09500693.2010.512369 http://dx.doi.org/10.1080/09500693.2010.512369
32. Merkel S ASM Task Force on Curriculum Guidelines for Undergraduate Biology 2012 The development of curricular guidelines for introductory microbiology that focus on understanding J Microbiol Biol Educ 13 32 38 10.1128/jmbe.v13i1.363 23653779 3577306 http://dx.doi.org/10.1128/jmbe.v13i1.363
33. Meltzer DE 2002 The relationship between mathematics preparation and conceptual learning gains in physics: a possible “hidden variable” in diagnostic pretest scores Am J Phys 70 1259 10.1119/1.1514215 http://dx.doi.org/10.1119/1.1514215
34. Crocker LM, Algina J 1986 Introduction to classical and modern test theory Holt, Rinehart, and Winston New York
35. Ebel RL 1954 Procedures for the analysis of classroom tests Educ Psychol Meas 14 352 364 10.1177/001316445401400215 http://dx.doi.org/10.1177/001316445401400215
36. Bardar EM, Prather EE, Brecher K, Slater TF 2006 Development and validation of the light and spectroscopy concept inventory Astron Educ Rev 5 103 113 10.3847/AER2006020 http://dx.doi.org/10.3847/AER2006020
37. Ding L, Chabay R, Sherwood B, Beichner R 2006 Evaluating an electricity and magnetism assessment tool: brief electricity and magnetism assessment Phys Rev Spec Top Phys Educ Res 2
38. Ferguson GA 1949 On the theory of test discrimination Psychometrika 14 61 68 10.1007/BF02290141 18113205 http://dx.doi.org/10.1007/BF02290141
39. Anderson DL, Fisher KM, Norman GJ 2002 Development and evaluation of the conceptual inventory of natural selection J Res Sci Teach 39 952 978 10.1002/tea.10053 http://dx.doi.org/10.1002/tea.10053

Supplemental Material

Loading

Article metrics loading...

/content/journal/jmbe/10.1128/jmbe.v18i3.1320
2017-10-05
2019-10-22

Abstract:

If we are to teach effectively, tools are needed to measure student learning. A widely used method for quickly measuring student understanding of core concepts in a discipline is the concept inventory (CI). Using the American Society for Microbiology Curriculum Guidelines (ASMCG) for microbiology, faculty from 11 academic institutions created and validated a new microbiology concept inventory (MCI). The MCI was developed in three phases. In phase one, learning outcomes and fundamental statements from the ASMCG were used to create T/F questions coupled with open responses. In phase two, the 743 responses to MCI 1.0 were examined to find the most common misconceptions, which were used to create distractors for multiple-choice questions. MCI 2.0 was then administered to 1,043 students. The responses of these students were used to create MCI 3.0, a 23-question CI that measures students’ understanding of all 27 fundamental statements. MCI 3.0 was found to be reliable, with a Cronbach’s alpha score of 0.705 and Ferguson’s delta of 0.97. Test item analysis demonstrated good validity and discriminatory power as judged by item difficulty, item discrimination, and point-biserial correlation coefficient. Comparison of pre- and posttest scores showed that microbiology students at 10 institutions showed an increase in understanding of concepts after instruction, except for questions probing metabolism (average normalized learning gain was 0.15). The MCI will enable quantitative analysis of student learning gains in understanding microbiology, help to identify misconceptions, and point toward areas where efforts should be made to develop teaching approaches to overcome them.

Highlighted Text: Show | Hide
Loading full text...

Full text loading...

/deliver/fulltext/jmbe/18/3/jmbe-18-49.html?itemId=/content/journal/jmbe/10.1128/jmbe.v18i3.1320&mimeType=html&fmt=ahah

Figures

Image of FIGURE 1

Click to view

FIGURE 1

Item difficulty and Item discrimination pre- vs. posttest. A total of 1,161 student surveys were used to determine question difficulty and discrimination. The dashed line indicates where each question would land if there was no change in difficulty or discrimination. Measured difficulty of each question decreased after instruction (the difficulty score increased). The discriminatory power of each question increased in the posttest.

Source: J. Microbiol. Biol. Educ. October 2017 vol. 18 no. 3 doi:10.1128/jmbe.v18i3.1320
Download as Powerpoint
Image of FIGURE 2

Click to view

FIGURE 2

Performance by question, pre- vs. posttest. Comparison of the average number of students answering correctly in the Pre-Test ( ) vs. the Post-Test ( ). Students showed improvement in all but questions 11, 12, and 13.

Source: J. Microbiol. Biol. Educ. October 2017 vol. 18 no. 3 doi:10.1128/jmbe.v18i3.1320
Download as Powerpoint
Image of FIGURE 3

Click to view

FIGURE 3

Normalized learning gains pre- vs. posttest. The normalized learning gains for each student by question. A total of 1,161 pre- and post- surveys from 10 colleges were analyzed per question. Positive learning gains were found for all but questions 11, 12, and 13.

Source: J. Microbiol. Biol. Educ. October 2017 vol. 18 no. 3 doi:10.1128/jmbe.v18i3.1320
Download as Powerpoint

This is a required field
Please enter a valid email address
Please check the format of the address you have entered.
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error