1887

Promoting the Multidimensional Character of Scientific Reasoning

    Authors: William S. Bradshaw1,*, Jennifer Nelson2, Byron J. Adams3, John D. Bell2
    VIEW AFFILIATIONS HIDE AFFILIATIONS
    Affiliations: 1: Department of Microbiology and Molecular Biology, Brigham Young University, Provo, UT 84602; 2: Department of Physiology and Developmental Biology, Brigham Young University, Provo, UT 84602; 3: Department of Biology, Brigham Young University, Provo, UT 84602
    AUTHOR AND ARTICLE INFORMATION AUTHOR AND ARTICLE INFORMATION
    • Received 14 November 2016 Accepted 20 December 2016 Published 21 April 2017
    • ©2017 Author(s). Published by the American Society for Microbiology.
    • [open-access] This is an Open Access article distributed under the terms of the Creative Commons Attribution-Noncommercial-NoDerivatives 4.0 International license (https://creativecommons.org/licenses/by-nc-nd/4.0/ and https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode), which grants the public the nonexclusive right to copy, distribute, or display the published work.

    • Supplemental materials available at http://asmscience.org/jmbe
    • *Corresponding author. Mailing address: Department of Microbiology and Molecular Biology, Brigham Young University, Provo, UT 84602. Tel: 80l-225-8437. E-mail: groverclan@q.com.
    Source: J. Microbiol. Biol. Educ. April 2017 vol. 18 no. 1 doi:10.1128/jmbe.v18i1.1272
MyBook is a cheap paperback edition of the original book and will be sold at uniform, low price.
  • XML
    101.09 Kb
  • HTML
    86.41 Kb
  • PDF
    926.37 Kb

    Abstract:

    This study reports part of a long-term program to help students improve scientific reasoning using higher-order cognitive tasks set in the discipline of cell biology. This skill was assessed using problems requiring the construction of valid conclusions drawn from authentic research data. We report here efforts to confirm the hypothesis that data interpretation is a complex, multifaceted exercise. Confirmation was obtained using a statistical treatment showing that various such problems rank students differently—each contains a unique set of cognitive challenges. Additional analyses of performance results have allowed us to demonstrate that individuals differ in their capacity to navigate five independent generic elements that constitute successful data interpretation: biological context, connection to course concepts, experimental protocols, data inference, and integration of isolated experimental observations into a coherent model. We offer these aspects of scientific thinking as a “data analysis skills inventory,” along with usable sample problems that illustrate each element. Additionally, we show that this kind of reasoning is rigorous in that it is difficult for most novice students, who are unable to intuitively implement strategies for improving these skills. Instructors armed with knowledge of the specific challenges presented by different types of problems can provide specific helpful feedback during formative practice. The use of this instructional model is most likely to require changes in traditional classroom instruction.

Key Concept Ranking

Endoplasmic Reticulum
0.47655177
Signal Transduction
0.45427856
Natural Selection
0.40273297
0.47655177

References & Citations

1. American Association for the Advancement of Science2011Vision and Change in Undergraduate Biology Education: A Call to Action: a summary of recommendations made at a national conference organized by the American Association for the Advancement of Science, July 15–17, 2009Washington, DC
2. Seymour E, Hewitt NM1997Talking about leaving: why undergraduates leave the sciencesWestviewBoulder, CO
3. Chen XL, Soldner M2013STEM attrition: college students’ paths into and out of STEM fields Statistical analysis reportUS Department of Education IES. National Center for Education StatisticsNces.ed.gov/pubs2014/2014001rev.pdf
4. Bao L, Cai T, Koening K, Fang K, Han J, Wang J, Lu Q, Ding L, Cui L, Lluo Y, Wang Y, Li L, Wu N2009Learning and scientific reasoningScience32358658710.1126/science.116774019179514 http://dx.doi.org/10.1126/science.1167740
5. Mervis J2010Shanghai students lead global results on PISAScience33014610.1126/science.330.6010.1461 http://dx.doi.org/10.1126/science.330.6010.1461
6. Wood WB2009Innovations in teaching undergraduate biology and why we need themAnnu Rev Cell Dev Biol259311210.1146/annurev.cellbio.24.110707.175306 http://dx.doi.org/10.1146/annurev.cellbio.24.110707.175306
7. Stanger-Hall KF2012Multiple-choice exams: an obstacle for higher-level thinking in introductory science classesCBE Life Sci Educ1129430610.1187/cbe.11-11-0100229494263433302 http://dx.doi.org/10.1187/cbe.11-11-0100
8. Pellegrino JW2013Proficiency in science: assessment challenges and opportunitiesScience34032032310.1126/science.123206523599485 http://dx.doi.org/10.1126/science.1232065
9. Handelsman J, Miller S, Pfund C2007Scientific teachingWH Freeman and CoNew York, NY
10. Tanner KD2010Order matters: using the 5E model to align teaching with how people learnCBE Life Sci Educ915916410.1187/cbe.10-06-0082208109452931660 http://dx.doi.org/10.1187/cbe.10-06-0082
11. Corwin LA, Graham JJ, Dolan EL2015Modeling course-based undergraduate research experiences: an agenda for further research and evaluationCBE Life Sci Educ1411310.1187/cbe.14-10-0167 http://dx.doi.org/10.1187/cbe.14-10-0167
12. Brownell SE, Hekmat-Scafe DS, Singla V, Seawell PC, Iman JFC, Eddy SL, Steams T, Cyert MS2015A high-enrollment course-based undergraduate research experience improved student conceptions of scientific thinking and ability to interpret dataCBE Life Sci Educ1411310.1187/cbe.14-05-0092 http://dx.doi.org/10.1187/cbe.14-05-0092
13. Russell JE, D’costa AR, Runck C, Barnes PW, Barrera AL, Hurst-Kennedy J, Sudduth EE, Quinlan EL, Schlueter M2014Bridging the undergraduate curriculum using an integrated course-embedded undergraduate researchCBE Life Sci Educ14110
14. Bangera G, Brownell SE2014Course-based undergraduate research experiences can make scientific research more inclusiveCBE Life Sci Educ1360260610.1187/cbe.14-06-0099254524834255347 http://dx.doi.org/10.1187/cbe.14-06-0099
15. Couch BA, Wood WB, Knight JK2015The Biology Capstone Assessment: a concept assessment for upper-division molecular biology studentsCBE Life Sci Educ1411110.1187/cbe.14-04-0071 http://dx.doi.org/10.1187/cbe.14-04-0071
16. Deane T, Nomme K, Jeffery E, Pollock C, Birol G2014Development of the biological experimental design concept inventory (BEDCI)CBE Life Sci Educ1354055110.1187/cbe.13-11-0218251852364152214 http://dx.doi.org/10.1187/cbe.13-11-0218
17. Brownell SE, Freeman S, Wenderoth MP, Crowe AJ2014BioCore Guide: a tool for interpreting the core concepts of Vision and Change for biology majorsCBE Life Sci Educ1320021110.1187/cbe.13-12-02334041499 http://dx.doi.org/10.1187/cbe.13-12-0233
18. Allen D, Tanner K2007Putting the horse back in front of the cart: using visions and decisions about high-quality learning experiences to drive course designCBE Life Sci Educ6858910.1187/cbe.07-03-0017175488701885907 http://dx.doi.org/10.1187/cbe.07-03-0017
19. Crowe A, Dirks C, Wenderoth MP2008Biology in bloom: implementing Bloom’s taxonomy to enhance student learning in biologyCBE Life Sci Educ736838010.1187/cbe.08-05-0024190474242592046 http://dx.doi.org/10.1187/cbe.08-05-0024
20. Linton DL, Pangle WM, Wyatt KH, Powell KN, Sherwood REIdentifying key features of effective active learning: the effects of writing and peer discussionCBE Life Sci Educ20141346947710.1187/cbe.13-12-0242251852304152208 http://dx.doi.org/10.1187/cbe.13-12-0242
21. Freeman S, Eddy SL, McDonough M, Smith MK, Okoroator N, Jordt H, Wenderoth MP2014Active learning increases student performance in science, engineering, and mathematicsProc Natl Acad Sci USA1118410841510.1073/pnas.1319030111248217564060654 http://dx.doi.org/10.1073/pnas.1319030111
22. Light RJ2001Making the most of college: students speak their mindsHarvard University pressCambridge, MA
23. Wiggins G, McTighe J1998Understanding by designAssociation for Supervision and Curriculum DevelopmentAlexandria, VA
24. Momsen JL, Long TM, Wyse SA, Ebert-May D2010Just the facts? Introductory undergraduate biology courses focus on low-level cognitive skills [Reports-Evaluative]CBE Life Sci Educ943544010.1187/cbe.10-01-0001211236902995761 http://dx.doi.org/10.1187/cbe.10-01-0001
25. Alberts B2013Failure of skin-deep learningScience338126310.1126/science.1233422 http://dx.doi.org/10.1126/science.1233422
26. Alberts B2009Restoring science to science educationIss Sci Technol257780
27. Jensen J, McDaniel M, Woodard S, Kummer T2014Teaching to the test… or testing to teach: exams requiring higher-order thinking skills encourage greater conceptual understandingEduc Psychol Rev2630732910.1007/s10648-013-9248-9 http://dx.doi.org/10.1007/s10648-013-9248-9
28. Anderson GL, Krathwohl DR2001A taxonomy for learning, teaching, and assessing: a revision of Bloom’s taxonomy of educational objectivesAllyn & BaconBoston, MA
29. Reeve S, Hammond JW, Bradshaw WS2004Inquiry in the large-enrollment science classroom. Simulating a research investigationJ Coll Sci Teach344448
30. Bravo A, Porzecanski A, Sterling E, Bynum N, Cawthorn M, Fernandez DS, Freeman L, Ketcham S, Leslie T, Mull J, Vogler D2016Teaching for higher levels of thinking: developing quantitative and analytical skills in environmental coursesEcosphere74e0129010.1002/ecs2.1290 http://dx.doi.org/10.1002/ecs2.1290
31. Hogan TP, Murphy G2007Recommendations for preparing and scoring constructed-response items: What the experts sayAppl Meas Educ2042744110.1080/08957340701580736 http://dx.doi.org/10.1080/08957340701580736
32. Knight JK, Wood WB2005Teaching more by lecturing lessCell Biol Educ429831010.1187/05-06-0082163412571305892 http://dx.doi.org/10.1187/05-06-0082
33. Coil D, Wenderoth MP, Dirks C2010Teaching the process of science: faculty perceptions and effective methodologyCBE Life Sci Educ952453510.1187/cbe.10-01-0005211236992995770 http://dx.doi.org/10.1187/cbe.10-01-0005
34. Kitchen E, Bell JD, Reeve S, Sudweeks RR, Bradshaw WS2003Teaching cell biology in the large-enrollment classroom: methods to promote analytical thinking and assessment of their effectivenessCell Biol Educ218019410.1187/cbe.02-11-005514506506192442 http://dx.doi.org/10.1187/cbe.02-11-0055
35. Nelson J, Robison DF, Bell JD, Bradshaw WS2009Cloning the professor, an alternative to ineffective teaching in a large courseCBE Life Sci Educ825226310.1187/cbe.09-01-0006197238192736028 http://dx.doi.org/10.1187/cbe.09-01-0006
36. Kitchen E, King SH, Robison DF, Sudweeks RR, Bradshaw WS, Bell JD2006Rethinking exams and letter grades: how much can teachers delegate to students?Cell Biol Educ627028010.1187/cbe.05-11-0123 http://dx.doi.org/10.1187/cbe.05-11-0123
37. Kitchen E, Reeve S, Bell JD, Sudweeks RR, Bradshaw WS2007The development and application of affective assessment in an upper-level bell biology courseJ Res Sci Teach441057108710.1002/tea.20188 http://dx.doi.org/10.1002/tea.20188
38. Reeve S, Kitchen E, Sudweeks RR, Bell JD, Bradshaw WS2011Development of an instrument for measuring self-efficacy in cell biologyJ Appl Meas12242260
39. Alberts B1994Molecular biology of the cell3rd edGarland ScienceNew York, NY
40. Alberts B2002Molecular biology of the cell4th edGarland ScienceNew York, NY
41. Bartsch D, Casadio A, Karl KA, Serodio P, Kandel ER1998Creb1 encodes a nuclear activator, a repressor, and a cytoplasmic modulator that form a regulatory unit critical for long-term facilitationCell9521122310.1016/S0092-8674(00)81752-39790528 http://dx.doi.org/10.1016/S0092-8674(00)81752-3
42. Crick JE, Brennan RL1982GENOVA: a generalized analysis of variance system (computer program and manual)American College Testing ProgramIowa City, IA
43. Cronbach LJ, Gleser GC, Nanda H, Rajaratnam N1972The dependability of behavioral measurement: theory of generalizability for scores and profilesJohn Wiley and Sons, IncNew York, NY
44. Roeser RW, Shavelson RJ, Kupermintz H2002The concept of aptitude and multidimensional validity revisitedEduc Assess819120510.1207/S15326977EA0802_06 http://dx.doi.org/10.1207/S15326977EA0802_06
45. Lemons PP, Lemons JD2013Questions for assessing higher-order cognitive skills: it’s not just Bloom’sCBE Life Sci Educ12475810.1187/cbe.12-03-0024234632283587855 http://dx.doi.org/10.1187/cbe.12-03-0024
46. Dunbar SB, Koretz DM, Hoover HD1991Quality control in the development and use of performance assessmentAppl Meas Educ428930310.1207/s15324818ame0404_3 http://dx.doi.org/10.1207/s15324818ame0404_3
47. Shavelson RJ, Baxter GP, Gao X1993Sampling variability of performance assessmentJ Educ Measure3021523210.1111/j.1745-3984.1993.tb00424.x http://dx.doi.org/10.1111/j.1745-3984.1993.tb00424.x
48. Smith MK, Wood WB, Adams WK, Wieman C, Knight JK, Guild N, Su TT2009Why peer discussion improves student performance on in-class concept questionsScience32312212410.1126/science.116591919119232 http://dx.doi.org/10.1126/science.1165919
49. Goldstein GS2007Using classroom assessment techniques in an introductory statistics classColl Teach55778210.3200/CTCH.55.2.77-82 http://dx.doi.org/10.3200/CTCH.55.2.77-82
50. Smith G2007How does student performance on formative assessments relate to learning assessed by exams?J Coll Sci Teach362834
51. Tanner KD2012Promoting student metacognitionCBE Life Sci Educ1111312010.1187/cbe.12-03-0033226655843366894 http://dx.doi.org/10.1187/cbe.12-03-0033
jmbe.v18i1.1272.citations
jmbe/18/1
content/journal/jmbe/10.1128/jmbe.v18i1.1272
Loading

Citations loading...

Supplemental Material

Loading

Article metrics loading...

/content/journal/jmbe/10.1128/jmbe.v18i1.1272
2017-04-21
2017-11-24

Abstract:

This study reports part of a long-term program to help students improve scientific reasoning using higher-order cognitive tasks set in the discipline of cell biology. This skill was assessed using problems requiring the construction of valid conclusions drawn from authentic research data. We report here efforts to confirm the hypothesis that data interpretation is a complex, multifaceted exercise. Confirmation was obtained using a statistical treatment showing that various such problems rank students differently—each contains a unique set of cognitive challenges. Additional analyses of performance results have allowed us to demonstrate that individuals differ in their capacity to navigate five independent generic elements that constitute successful data interpretation: biological context, connection to course concepts, experimental protocols, data inference, and integration of isolated experimental observations into a coherent model. We offer these aspects of scientific thinking as a “data analysis skills inventory,” along with usable sample problems that illustrate each element. Additionally, we show that this kind of reasoning is rigorous in that it is difficult for most novice students, who are unable to intuitively implement strategies for improving these skills. Instructors armed with knowledge of the specific challenges presented by different types of problems can provide specific helpful feedback during formative practice. The use of this instructional model is most likely to require changes in traditional classroom instruction.

Highlighted Text: Show | Hide
Loading full text...

Full text loading...

/deliver/fulltext/jmbe/18/1/jmbe-18-4.html?itemId=/content/journal/jmbe/10.1128/jmbe.v18i1.1272&mimeType=html&fmt=ahah

Figures

Image of FIGURE 1

Click to view

FIGURE 1

Sample data-interpretation problem: Secretion. Valid conclusion statements are: 1) BiP is an ER-resident protein. 2) K and D near the C-terminus are required for return of BiP to the ER from the Golgi. 3) BiP is secreted from the cell if not returned to the ER. 4) Coatomer-coated vesicles are required for the movement of BiP to the Golgi. 5) The default secretory pathway uses coatomer-coated vesicles. 6) BiP binds to a protein in the ER membrane via the K and D sequence.

Source: J. Microbiol. Biol. Educ. April 2017 vol. 18 no. 1 doi:10.1128/jmbe.v18i1.1272
Download as Powerpoint
Image of FIGURE 3

Click to view

FIGURE 3

(A) Student performance on data interpretation items that vary in complexity (see text for details on the items). Problems were administered at the onset (“Pre”) and again near the end of the semester (“Post”), selectively from 2002 to 2006. The main effects (overall pre and post gains, and differences among problems) and the interaction between the two were all significant ( < 0.0001 by two-way ANOVA, = 111–353). (B) Gains realized by students for items shown in (A). See text for explanation of labels. (C) Correlation of student performance on two sets of three data interpretation items that contain all four elements from (B). One set focused on hormone signal transduction from a fourth midterm exam, 2006 (signal), and the context for the other was regulation of the globin gene from the third midterm exam (Hb) ( < 0.0001 by linear regression, r = 0.87). (D–F) Histograms of student performance on an item that displays a normal distribution (D), bimodal distribution (E), and skewed distribution (F). See text for details on these three items.

Source: J. Microbiol. Biol. Educ. April 2017 vol. 18 no. 1 doi:10.1128/jmbe.v18i1.1272
Download as Powerpoint
Image of FIGURE 2

Click to view

FIGURE 2

Comparison of student performance on two data-interpretation problems. See text for details on the items. (A) Scores on EX-2 and MMP2 were compared over five years (2002–2006) and nine semesters for 1,192 students ( < 0.0001 by linear regression, r = 0.066). (B) Analysis over consecutive winter and fall semesters (2005) of scores on student responses to EX-2 divided between understanding of data presentation and experimental protocol from those that relied on biological concepts from the course (-test of protocol vs. concepts, < 0.0001, = 295). (C) Responses over the same time period as in (B) to MMP2 were rescored using a rubric that distinguished student understanding of the data presentation from understanding of the experimental protocol (-test of data vs. protocol, < 0.01, = 295). (D) Responses to MMP2 on a test taken at the beginning of a summer term (2006) (“pre”) and again during a midterm exam (“post”) were scored using the rubric of (C). Two-way analysis of variance: pre vs. post, < 0.0001; data vs. protocol, = 0.06; interaction, = 0.01; = 35.

Source: J. Microbiol. Biol. Educ. April 2017 vol. 18 no. 1 doi:10.1128/jmbe.v18i1.1272
Download as Powerpoint

This is a required field
Please enter a valid email address
Please check the format of the address you have entered.
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error