Designing a Curriculum-Aligned Assessment of Cumulative Learning about Marine Primary Production to Improve an Undergraduate Marine Sciences Program †
-
Authors:
Ryan A. Weatherbee1,*,
Sara M. Lindsay2
-
Received 31 July 2017 Accepted 22 August 2018 Published 14 December 2018
- ©2018 Author(s). Published by the American Society for Microbiology
-
[open-access] This is an Open Access article distributed under the terms of the Creative Commons Attribution-Noncommercial-NoDerivatives 4.0 International license (https://creativecommons.org/licenses/by-nc-nd/4.0/ and https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode), which grants the public the nonexclusive right to copy, distribute, or display the published work.
- *Corresponding author. Mailing address: Husson University, 1 College Circle, Bangor, ME 04401. Phone: 207-941-7031. E-mail: [email protected].
Abstract:
We developed an assessment to track changes in understanding about marine primary production, a key concept taught across our undergraduate curriculum. Question content was informed by investigating student misunderstandings, conducting faculty interviews, and mapping primary production concepts to the curriculum. Content questions were paired with questions asking students how confident they were in their answers. Although students gained knowledge of marine primary production across educational levels, confidence data and item analysis indicated student misunderstandings on several concepts. Many students had difficulty on questions that required interpreting graphs or other higher-order thinking skills. The results set the stage for additional focused assessment and curriculum revision, and the questions may be useful in developing a large-scale, interdisciplinary marine sciences concept inventory.
References & Citations
Supplemental Material
-
Appendix 1: Methods of survey development, Appendix 2: Implemented curriculum concept map, Appendix 3: Marine primary production assessment (MPPA) reported on in this paper and graphs of student self-reported confidence by question item, Appendix 4: Recommendations for future deployments of the MPPA
-
MyBook is a cheap paperback edition of the original book and will be sold at uniform, low price.
-
PDF
4.35 MB
-
PDF
-

Article metrics loading...
Abstract:
We developed an assessment to track changes in understanding about marine primary production, a key concept taught across our undergraduate curriculum. Question content was informed by investigating student misunderstandings, conducting faculty interviews, and mapping primary production concepts to the curriculum. Content questions were paired with questions asking students how confident they were in their answers. Although students gained knowledge of marine primary production across educational levels, confidence data and item analysis indicated student misunderstandings on several concepts. Many students had difficulty on questions that required interpreting graphs or other higher-order thinking skills. The results set the stage for additional focused assessment and curriculum revision, and the questions may be useful in developing a large-scale, interdisciplinary marine sciences concept inventory.

Full text loading...
Author and Article Information
-
Received 31 July 2017 Accepted 22 August 2018 Published 14 December 2018
- ©2018 Author(s). Published by the American Society for Microbiology
-
[open-access] This is an Open Access article distributed under the terms of the Creative Commons Attribution-Noncommercial-NoDerivatives 4.0 International license (https://creativecommons.org/licenses/by-nc-nd/4.0/ and https://creativecommons.org/licenses/by-nc-nd/4.0/legalcode), which grants the public the nonexclusive right to copy, distribute, or display the published work.
- *Corresponding author. Mailing address: Husson University, 1 College Circle, Bangor, ME 04401. Phone: 207-941-7031. E-mail: [email protected].
Figures

Click to view
FIGURE 1
Flowchart describing general assessment development process. Potential misunderstandings about marine primary production were identified from existing first-year student homework and then mapped onto a concept map describing the implemented curriculum regarding marine primary production; assessment questions were drafted to target key topics for which we had evidence of some misunderstandings. Faculty experts guided question development, and some questions were revised based on student performance and comments following a pilot deployment. See Appendix 1 for details. MPP = marine primary production.

Click to view
FIGURE 2
Rasch analysis results. (a) Item map showing how each survey question varied in difficulty (y-axis) and how well it fit the Rasch model (x-axis). Question difficulty increases with vertical logit value. The vertical size of each symbol shows the standard error of the estimated difficulty. (b) Person map showing each student’s ability on the assessment (y-axis) and how well each student’s responses fit the Rasch model (x-axis). Students with more ability had larger vertical ability values. To aid readability, error estimates are not shown by symbol size. The average standard error of estimated ability was 0.65 (±0.11 SD). For both maps, the fit of the questions/students to the model is indicated by their position on the x-axis. Items/students further from 0 fit the model less well. The vertical lines at −2 and 2 infit t values indicate the acceptable range for items fitting the Rasch model.

Click to view
FIGURE 3
Mean assessment scores. Mean person parameter logit values in each SMS educational level. Larger values (less negative) indicate higher performance on the assessment. Error bars show standard error. Number of students in each educational level was level 1 = 20, level 2 = 17, level 3 = 34 and level 4 = 15. The mean score at level 1 was significantly different than all other levels (Tukey’s LSD, α = 0.1). LSD = least significant difference; SMS = School of Marine Sciences.

Click to view
FIGURE 4
Relationship between assessment difficulty and cognitive demand of assessment questions. The linear correlation of mean logit assessment difficulty for each question is pooled over all SMS educational levels and the cognitive demands of each question as categorized by Bloom’s Taxonomy levels (R2 = 0.37, p = 0.028). Note that the signs of item difficulty values have been inverted to emphasize increased difficulty of questions with higher Bloom’s levels. SMS = School of Marine Sciences.

Click to view
FIGURE 5
Relationship between self-reported response confidence and assessment score. The graph indicates a linear relationship between categorical mean self-reported question response confidence and overall assessment score over all questions and SMS educational levels. Each symbol represents a unique student (n = 86). The least-squares regression line is plotted (Pearson correlation, R2 = 0.00, α = 0.1, p = 0.956). SMS = School of Marine Sciences.
- Molecules containing carbon. (correct answer)
- Particulate matter.
- Dissolved forms of nitrogen.
- Energy from the sun.

Click to view
FIGURE 6
Self-reported confidence for survey Question 1. Responses are pooled over all educational levels, with confidence in each item indicated by color. The question was: The majority of actual weight (dry biomass) gained by open-ocean photosynthetic marine algae as they grow comes from which of the following substances? The possible responses were:
- Molecules containing carbon. (correct answer)
- Particulate matter.
- Dissolved forms of nitrogen.
- Energy from the sun.

Click to view
FIGURE 7
Relationship between question cognitive demand and mean student confidence. The graph shows a linear relationship between Bloom’s Taxonomy level for each question and categorical mean self-reported question response confidence pooled over all educational levels. The least-squares regression line is plotted (Pearson correlation, R2 = 0.24, α = 0.1, p = 0.089).