1887

Promoting the Multidimensional Character of Scientific Reasoning

    Authors: William S. Bradshaw1,*, Jennifer Nelson2, Byron J. Adams3, John D. Bell2
    VIEW AFFILIATIONS HIDE AFFILIATIONS
    Affiliations: 1: Department of Microbiology and Molecular Biology, Brigham Young University, Provo, UT 84602; 2: Department of Physiology and Developmental Biology, Brigham Young University, Provo, UT 84602; 3: Department of Biology, Brigham Young University, Provo, UT 84602
    AUTHOR AND ARTICLE INFORMATION AUTHOR AND ARTICLE INFORMATION
    • Received 14 November 2016 Accepted 20 December 2016 Published 21 April 2017
    • ©2017 Author(s). Published by the American Society for Microbiology.
    • [open-access] This is an Open Access article distributed under the terms of the Creative Commons Attribution-Noncommercial-NoDerivatives 4.0 International license (https://creativecommons.org/licenses/by-nc-nd/4.0/ and https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode), which grants the public the nonexclusive right to copy, distribute, or display the published work.

    • Supplemental materials available at http://asmscience.org/jmbe
    • *Corresponding author. Mailing address: Department of Microbiology and Molecular Biology, Brigham Young University, Provo, UT 84602. Tel: 80l-225-8437. E-mail: [email protected].
    Source: J. Microbiol. Biol. Educ. April 2017 vol. 18 no. 1 doi:10.1128/jmbe.v18i1.1272
MyBook is a cheap paperback edition of the original book and will be sold at uniform, low price.
  • XML
    101.09 Kb
  • HTML
    86.41 Kb
  • PDF
    926.37 Kb

    Abstract:

    This study reports part of a long-term program to help students improve scientific reasoning using higher-order cognitive tasks set in the discipline of cell biology. This skill was assessed using problems requiring the construction of valid conclusions drawn from authentic research data. We report here efforts to confirm the hypothesis that data interpretation is a complex, multifaceted exercise. Confirmation was obtained using a statistical treatment showing that various such problems rank students differently—each contains a unique set of cognitive challenges. Additional analyses of performance results have allowed us to demonstrate that individuals differ in their capacity to navigate five independent generic elements that constitute successful data interpretation: biological context, connection to course concepts, experimental protocols, data inference, and integration of isolated experimental observations into a coherent model. We offer these aspects of scientific thinking as a “data analysis skills inventory,” along with usable sample problems that illustrate each element. Additionally, we show that this kind of reasoning is rigorous in that it is difficult for most novice students, who are unable to intuitively implement strategies for improving these skills. Instructors armed with knowledge of the specific challenges presented by different types of problems can provide specific helpful feedback during formative practice. The use of this instructional model is most likely to require changes in traditional classroom instruction.

Key Concept Ranking

Endoplasmic Reticulum
0.47655177
Signal Transduction
0.45427856
Natural Selection
0.40273297
0.47655177

References & Citations

1. American Association for the Advancement of Science 2011 Vision and Change in Undergraduate Biology Education: A Call to Action: a summary of recommendations made at a national conference organized by the American Association for the Advancement of Science, July 15–17, 2009 Washington, DC
2. Seymour E, Hewitt NM 1997 Talking about leaving: why undergraduates leave the sciences Westview Boulder, CO
3. Chen XL, Soldner M 2013 STEM attrition: college students’ paths into and out of STEM fields Statistical analysis report US Department of Education IES. National Center for Education Statistics Nces.ed.gov/pubs2014/2014001rev.pdf
4. Bao L, Cai T, Koening K, Fang K, Han J, Wang J, Lu Q, Ding L, Cui L, Lluo Y, Wang Y, Li L, Wu N 2009 Learning and scientific reasoning Science 323 586 587 10.1126/science.1167740 19179514 http://dx.doi.org/10.1126/science.1167740
5. Mervis J 2010 Shanghai students lead global results on PISA Science 330 146 10.1126/science.330.6010.1461 http://dx.doi.org/10.1126/science.330.6010.1461
6. Wood WB 2009 Innovations in teaching undergraduate biology and why we need them Annu Rev Cell Dev Biol 25 93 112 10.1146/annurev.cellbio.24.110707.175306 http://dx.doi.org/10.1146/annurev.cellbio.24.110707.175306
7. Stanger-Hall KF 2012 Multiple-choice exams: an obstacle for higher-level thinking in introductory science classes CBE Life Sci Educ 11 294 306 10.1187/cbe.11-11-0100 22949426 3433302 http://dx.doi.org/10.1187/cbe.11-11-0100
8. Pellegrino JW 2013 Proficiency in science: assessment challenges and opportunities Science 340 320 323 10.1126/science.1232065 23599485 http://dx.doi.org/10.1126/science.1232065
9. Handelsman J, Miller S, Pfund C 2007 Scientific teaching WH Freeman and Co New York, NY
10. Tanner KD 2010 Order matters: using the 5E model to align teaching with how people learn CBE Life Sci Educ 9 159 164 10.1187/cbe.10-06-0082 20810945 2931660 http://dx.doi.org/10.1187/cbe.10-06-0082
11. Corwin LA, Graham JJ, Dolan EL 2015 Modeling course-based undergraduate research experiences: an agenda for further research and evaluation CBE Life Sci Educ 14 1 13 10.1187/cbe.14-10-0167 http://dx.doi.org/10.1187/cbe.14-10-0167
12. Brownell SE, Hekmat-Scafe DS, Singla V, Seawell PC, Iman JFC, Eddy SL, Steams T, Cyert MS 2015 A high-enrollment course-based undergraduate research experience improved student conceptions of scientific thinking and ability to interpret data CBE Life Sci Educ 14 1 13 10.1187/cbe.14-05-0092 http://dx.doi.org/10.1187/cbe.14-05-0092
13. Russell JE, D’costa AR, Runck C, Barnes PW, Barrera AL, Hurst-Kennedy J, Sudduth EE, Quinlan EL, Schlueter M 2014 Bridging the undergraduate curriculum using an integrated course-embedded undergraduate research CBE Life Sci Educ 14 1 10
14. Bangera G, Brownell SE 2014 Course-based undergraduate research experiences can make scientific research more inclusive CBE Life Sci Educ 13 602 606 10.1187/cbe.14-06-0099 25452483 4255347 http://dx.doi.org/10.1187/cbe.14-06-0099
15. Couch BA, Wood WB, Knight JK 2015 The Biology Capstone Assessment: a concept assessment for upper-division molecular biology students CBE Life Sci Educ 14 1 11 10.1187/cbe.14-04-0071 http://dx.doi.org/10.1187/cbe.14-04-0071
16. Deane T, Nomme K, Jeffery E, Pollock C, Birol G 2014 Development of the biological experimental design concept inventory (BEDCI) CBE Life Sci Educ 13 540 551 10.1187/cbe.13-11-0218 25185236 4152214 http://dx.doi.org/10.1187/cbe.13-11-0218
17. Brownell SE, Freeman S, Wenderoth MP, Crowe AJ 2014 BioCore Guide: a tool for interpreting the core concepts of Vision and Change for biology majors CBE Life Sci Educ 13 200 211 10.1187/cbe.13-12-0233 4041499 http://dx.doi.org/10.1187/cbe.13-12-0233
18. Allen D, Tanner K 2007 Putting the horse back in front of the cart: using visions and decisions about high-quality learning experiences to drive course design CBE Life Sci Educ 6 85 89 10.1187/cbe.07-03-0017 17548870 1885907 http://dx.doi.org/10.1187/cbe.07-03-0017
19. Crowe A, Dirks C, Wenderoth MP 2008 Biology in bloom: implementing Bloom’s taxonomy to enhance student learning in biology CBE Life Sci Educ 7 368 380 10.1187/cbe.08-05-0024 19047424 2592046 http://dx.doi.org/10.1187/cbe.08-05-0024
20. Linton DL, Pangle WM, Wyatt KH, Powell KN, Sherwood RE Identifying key features of effective active learning: the effects of writing and peer discussion CBE Life Sci Educ 2014 13 469 477 10.1187/cbe.13-12-0242 25185230 4152208 http://dx.doi.org/10.1187/cbe.13-12-0242
21. Freeman S, Eddy SL, McDonough M, Smith MK, Okoroator N, Jordt H, Wenderoth MP 2014 Active learning increases student performance in science, engineering, and mathematics Proc Natl Acad Sci USA 111 8410 8415 10.1073/pnas.1319030111 24821756 4060654 http://dx.doi.org/10.1073/pnas.1319030111
22. Light RJ 2001 Making the most of college: students speak their minds Harvard University press Cambridge, MA
23. Wiggins G, McTighe J 1998 Understanding by design Association for Supervision and Curriculum Development Alexandria, VA
24. Momsen JL, Long TM, Wyse SA, Ebert-May D 2010 Just the facts? Introductory undergraduate biology courses focus on low-level cognitive skills [Reports-Evaluative] CBE Life Sci Educ 9 435 440 10.1187/cbe.10-01-0001 21123690 2995761 http://dx.doi.org/10.1187/cbe.10-01-0001
25. Alberts B 2013 Failure of skin-deep learning Science 338 1263 10.1126/science.1233422 http://dx.doi.org/10.1126/science.1233422
26. Alberts B 2009 Restoring science to science education Iss Sci Technol 25 77 80
27. Jensen J, McDaniel M, Woodard S, Kummer T 2014 Teaching to the test… or testing to teach: exams requiring higher-order thinking skills encourage greater conceptual understanding Educ Psychol Rev 26 307 329 10.1007/s10648-013-9248-9 http://dx.doi.org/10.1007/s10648-013-9248-9
28. Anderson GL, Krathwohl DR 2001 A taxonomy for learning, teaching, and assessing: a revision of Bloom’s taxonomy of educational objectives Allyn & Bacon Boston, MA
29. Reeve S, Hammond JW, Bradshaw WS 2004 Inquiry in the large-enrollment science classroom. Simulating a research investigation J Coll Sci Teach 34 44 48
30. Bravo A, Porzecanski A, Sterling E, Bynum N, Cawthorn M, Fernandez DS, Freeman L, Ketcham S, Leslie T, Mull J, Vogler D 2016 Teaching for higher levels of thinking: developing quantitative and analytical skills in environmental courses Ecosphere 7 4 e01290 10.1002/ecs2.1290 http://dx.doi.org/10.1002/ecs2.1290
31. Hogan TP, Murphy G 2007 Recommendations for preparing and scoring constructed-response items: What the experts say Appl Meas Educ 20 427 441 10.1080/08957340701580736 http://dx.doi.org/10.1080/08957340701580736
32. Knight JK, Wood WB 2005 Teaching more by lecturing less Cell Biol Educ 4 298 310 10.1187/05-06-0082 16341257 1305892 http://dx.doi.org/10.1187/05-06-0082
33. Coil D, Wenderoth MP, Dirks C 2010 Teaching the process of science: faculty perceptions and effective methodology CBE Life Sci Educ 9 524 535 10.1187/cbe.10-01-0005 21123699 2995770 http://dx.doi.org/10.1187/cbe.10-01-0005
34. Kitchen E, Bell JD, Reeve S, Sudweeks RR, Bradshaw WS 2003 Teaching cell biology in the large-enrollment classroom: methods to promote analytical thinking and assessment of their effectiveness Cell Biol Educ 2 180 194 10.1187/cbe.02-11-0055 14506506 192442 http://dx.doi.org/10.1187/cbe.02-11-0055
35. Nelson J, Robison DF, Bell JD, Bradshaw WS 2009 Cloning the professor, an alternative to ineffective teaching in a large course CBE Life Sci Educ 8 252 263 10.1187/cbe.09-01-0006 19723819 2736028 http://dx.doi.org/10.1187/cbe.09-01-0006
36. Kitchen E, King SH, Robison DF, Sudweeks RR, Bradshaw WS, Bell JD 2006 Rethinking exams and letter grades: how much can teachers delegate to students? Cell Biol Educ 6 270 280 10.1187/cbe.05-11-0123 http://dx.doi.org/10.1187/cbe.05-11-0123
37. Kitchen E, Reeve S, Bell JD, Sudweeks RR, Bradshaw WS 2007 The development and application of affective assessment in an upper-level bell biology course J Res Sci Teach 44 1057 1087 10.1002/tea.20188 http://dx.doi.org/10.1002/tea.20188
38. Reeve S, Kitchen E, Sudweeks RR, Bell JD, Bradshaw WS 2011 Development of an instrument for measuring self-efficacy in cell biology J Appl Meas 12 242 260
39. Alberts B 1994 Molecular biology of the cell 3rd ed Garland Science New York, NY
40. Alberts B 2002 Molecular biology of the cell 4th ed Garland Science New York, NY
41. Bartsch D, Casadio A, Karl KA, Serodio P, Kandel ER 1998 Creb1 encodes a nuclear activator, a repressor, and a cytoplasmic modulator that form a regulatory unit critical for long-term facilitation Cell 95 211 223 10.1016/S0092-8674(00)81752-3 9790528 http://dx.doi.org/10.1016/S0092-8674(00)81752-3
42. Crick JE, Brennan RL 1982 GENOVA: a generalized analysis of variance system (computer program and manual) American College Testing Program Iowa City, IA
43. Cronbach LJ, Gleser GC, Nanda H, Rajaratnam N 1972 The dependability of behavioral measurement: theory of generalizability for scores and profiles John Wiley and Sons, Inc New York, NY
44. Roeser RW, Shavelson RJ, Kupermintz H 2002 The concept of aptitude and multidimensional validity revisited Educ Assess 8 191 205 10.1207/S15326977EA0802_06 http://dx.doi.org/10.1207/S15326977EA0802_06
45. Lemons PP, Lemons JD 2013 Questions for assessing higher-order cognitive skills: it’s not just Bloom’s CBE Life Sci Educ 12 47 58 10.1187/cbe.12-03-0024 23463228 3587855 http://dx.doi.org/10.1187/cbe.12-03-0024
46. Dunbar SB, Koretz DM, Hoover HD 1991 Quality control in the development and use of performance assessment Appl Meas Educ 4 289 303 10.1207/s15324818ame0404_3 http://dx.doi.org/10.1207/s15324818ame0404_3
47. Shavelson RJ, Baxter GP, Gao X 1993 Sampling variability of performance assessment J Educ Measure 30 215 232 10.1111/j.1745-3984.1993.tb00424.x http://dx.doi.org/10.1111/j.1745-3984.1993.tb00424.x
48. Smith MK, Wood WB, Adams WK, Wieman C, Knight JK, Guild N, Su TT 2009 Why peer discussion improves student performance on in-class concept questions Science 323 122 124 10.1126/science.1165919 19119232 http://dx.doi.org/10.1126/science.1165919
49. Goldstein GS 2007 Using classroom assessment techniques in an introductory statistics class Coll Teach 55 77 82 10.3200/CTCH.55.2.77-82 http://dx.doi.org/10.3200/CTCH.55.2.77-82
50. Smith G 2007 How does student performance on formative assessments relate to learning assessed by exams? J Coll Sci Teach 36 28 34
51. Tanner KD 2012 Promoting student metacognition CBE Life Sci Educ 11 113 120 10.1187/cbe.12-03-0033 22665584 3366894 http://dx.doi.org/10.1187/cbe.12-03-0033

Supplemental Material

Loading

Article metrics loading...

/content/journal/jmbe/10.1128/jmbe.v18i1.1272
2017-04-21
2019-10-23

Abstract:

This study reports part of a long-term program to help students improve scientific reasoning using higher-order cognitive tasks set in the discipline of cell biology. This skill was assessed using problems requiring the construction of valid conclusions drawn from authentic research data. We report here efforts to confirm the hypothesis that data interpretation is a complex, multifaceted exercise. Confirmation was obtained using a statistical treatment showing that various such problems rank students differently—each contains a unique set of cognitive challenges. Additional analyses of performance results have allowed us to demonstrate that individuals differ in their capacity to navigate five independent generic elements that constitute successful data interpretation: biological context, connection to course concepts, experimental protocols, data inference, and integration of isolated experimental observations into a coherent model. We offer these aspects of scientific thinking as a “data analysis skills inventory,” along with usable sample problems that illustrate each element. Additionally, we show that this kind of reasoning is rigorous in that it is difficult for most novice students, who are unable to intuitively implement strategies for improving these skills. Instructors armed with knowledge of the specific challenges presented by different types of problems can provide specific helpful feedback during formative practice. The use of this instructional model is most likely to require changes in traditional classroom instruction.

Highlighted Text: Show | Hide
Loading full text...

Full text loading...

/deliver/fulltext/jmbe/18/1/jmbe-18-4.html?itemId=/content/journal/jmbe/10.1128/jmbe.v18i1.1272&mimeType=html&fmt=ahah

Figures

Image of FIGURE 1

Click to view

FIGURE 1

Sample data-interpretation problem: Secretion. Valid conclusion statements are: 1) BiP is an ER-resident protein. 2) K and D near the C-terminus are required for return of BiP to the ER from the Golgi. 3) BiP is secreted from the cell if not returned to the ER. 4) Coatomer-coated vesicles are required for the movement of BiP to the Golgi. 5) The default secretory pathway uses coatomer-coated vesicles. 6) BiP binds to a protein in the ER membrane via the K and D sequence.

Source: J. Microbiol. Biol. Educ. April 2017 vol. 18 no. 1 doi:10.1128/jmbe.v18i1.1272
Download as Powerpoint
Image of FIGURE 3

Click to view

FIGURE 3

(A) Student performance on data interpretation items that vary in complexity (see text for details on the items). Problems were administered at the onset (“Pre”) and again near the end of the semester (“Post”), selectively from 2002 to 2006. The main effects (overall pre and post gains, and differences among problems) and the interaction between the two were all significant ( < 0.0001 by two-way ANOVA, = 111–353). (B) Gains realized by students for items shown in (A). See text for explanation of labels. (C) Correlation of student performance on two sets of three data interpretation items that contain all four elements from (B). One set focused on hormone signal transduction from a fourth midterm exam, 2006 (signal), and the context for the other was regulation of the globin gene from the third midterm exam (Hb) ( < 0.0001 by linear regression, r = 0.87). (D–F) Histograms of student performance on an item that displays a normal distribution (D), bimodal distribution (E), and skewed distribution (F). See text for details on these three items.

Source: J. Microbiol. Biol. Educ. April 2017 vol. 18 no. 1 doi:10.1128/jmbe.v18i1.1272
Download as Powerpoint
Image of FIGURE 2

Click to view

FIGURE 2

Comparison of student performance on two data-interpretation problems. See text for details on the items. (A) Scores on EX-2 and MMP2 were compared over five years (2002–2006) and nine semesters for 1,192 students ( < 0.0001 by linear regression, r = 0.066). (B) Analysis over consecutive winter and fall semesters (2005) of scores on student responses to EX-2 divided between understanding of data presentation and experimental protocol from those that relied on biological concepts from the course (-test of protocol vs. concepts, < 0.0001, = 295). (C) Responses over the same time period as in (B) to MMP2 were rescored using a rubric that distinguished student understanding of the data presentation from understanding of the experimental protocol (-test of data vs. protocol, < 0.01, = 295). (D) Responses to MMP2 on a test taken at the beginning of a summer term (2006) (“pre”) and again during a midterm exam (“post”) were scored using the rubric of (C). Two-way analysis of variance: pre vs. post, < 0.0001; data vs. protocol, = 0.06; interaction, = 0.01; = 35.

Source: J. Microbiol. Biol. Educ. April 2017 vol. 18 no. 1 doi:10.1128/jmbe.v18i1.1272
Download as Powerpoint

This is a required field
Please enter a valid email address
Please check the format of the address you have entered.
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error