Scientific Reasoning Report
Fall 2024 and Spring 2025
Overview
Virginia Tech has chosen to measure two outcomes within the Reasoning in the Natural Sciences area of the Pathways general education program for the assessment of scientific reasoning. These two outcomes were chosen based on their alignment with the concept of Reasoning in the Natural Sciences.
Reasoning in the Natural Sciences courses involve the acquisition of the detailed knowledge of one or more of the natural sciences, hands-on experience with how science is conducted, what science can and cannot tell us about the universe, and the relationship between sciences and society. (Students take two courses in this area of the curriculum.)
During the Fall 2024 and Spring 2025 semesters, Virginia Tech sampled 45 course sections that addressed the Core Concept of Reasoning in the Natural Sciences (RNS). In addition, some sections sampled for other Pathways Core Concepts also covered RNS. Instructors of sampled sections were asked to submit data for all Core Concepts. The final group of sections included in the analyses below were sections that were not required to submit data but contributed data voluntarily. As a result, 139 sections reported data for RNS.
Table 1 summarizes the number of Pathways sections sampled, the number of sections reporting data, and enrollment totals included in the analyses for RNS. A total of 45 Pathways sections were sampled, and 139 sections reported data. Of these reporting sections, 45 were sampled sections and 94 were “bonus” sections that were not part of the original sample but still submitted data. Enrollment in reporting sections totaled 7,815 students. Footnotes clarify that some sections were sampled for other Core Concepts that also include RNS and that enrollment totals may include students enrolled in or assessed across multiple Pathways courses.
Outcomes Virginia Tech Measured this Reporting Cycle
- Apply principles and techniques of scientific inquiry.
- Evaluate the credibility and the use/misuse of scientific information.
Virginia Tech's Target for this Reporting Cycle
At least 70% of students will be rated as meeting or exceeding competency in this area.
Methods
All undergraduate students are required to participate in the Pathways general education program. As of Fall 2024, Virginia Tech utilizes a sampling methodology for general education assessment that aims to collect data from a subset of course sections that is representative of the general education curriculum. All courses approved for Pathways are part of the sampling pool. Based on an estimated enrollment population for general education course sections of 80,000 each fall and spring semester, along with a confidence level of 99% and a 2% margin of error, approximately 4,000 students should be assessed each semester. Based on the minimum number of students that can be assessed in any one section (i.e., 20), this equates to approximately 200 sections per semester. The number of course sections sampled for each Core Concept is then based on the required number of credit hours that students must complete in each area (i.e., Core Concepts that require students to complete nine credit hours have a greater number of sampled course sections than areas in which six credit hours are required).
When developing a final sample each semester, if a course section was already selected to participate in another Core Concept area, this course section was excluded from selection in other areas. Except for extremely rare cases, if an instructor was already selected during the current semester or a previous semester within the four-semester data collection period, this instructor was not required to participate again. Additionally, large-enrollment course sections from each Core Concept were sampled. Large-enrollment course sections are defined as those having more than 200 students enrolled on average.
Instructors develop their own assessment methodologies based on the content and structure of the course. Students complete the required assessment(s) developed by the instructor teaching the sampled course section. All assessments must be direct measures of student learning. Some possibilities include: written assignments, multiple-choice items, projects, case studies, and artistic creations. Faculty determine what is a developing competence, competent, and advanced competence level of performance on each learning outcome. Rubrics have been developed to guide faculty in this process.
Data and Analysis
Table 2 summarizes competency level results for two Student Learning Outcomes (SLOs) in the RNS area. For SLO 1, which focuses on applying principles and techniques of scientific inquiry, 134 sections and 6,149 students were included in the analyses. Of these students, 7% (444) demonstrated Developing Competence, 21% (1,317) were rated Competent, and 71% (4,388) demonstrated Advanced Competence. For SLO 2, which focuses on evaluating the credibility and use or misuse of scientific information, 9 sections and 375 students were included in the analyses. Of these students, 5% (17) demonstrated Developing Competence, 18% (66) were rated Competent, and 78% (292) demonstrated Advanced Competence. A footnote explains that instructors must teach and assess a majority of the four RNS SLOs, so section counts vary between SLO 1 and SLO 2.
SLO 1: Apply principles and techniques of scientific inquiry.
Table 3 summarizes the Pathways sections and student enrollments that reported data and were included in the analyses for SLO 1. A total of 134 Pathways sections submitted data. Enrollment across these sections totaled 7,327 students, and 6,149 students were included in the analyses. A footnote notes that enrollment counts may include students who were enrolled in or assessed across multiple Pathways courses.
Figure 1 shows the competency level distribution for SLO 1, which focuses on applying principles and techniques of scientific inquiry. Based on 6,149 students, 7% (444 students) demonstrated Developing Competence, 21% (1,317 students) were rated Competent, and 71% (4,388 students) demonstrated Advanced Competence.
SLO 2: Evaluate the credibility and the use/misuse of scientific information.
Table 4 summarizes the Pathways sections and student enrollments that reported data and were included in the analyses for SLO 2. A total of 9 Pathways sections reported data. Enrollment across these sections totaled 727 students, and 375 students were included in the analyses. A footnote notes that enrollment values may include students who were enrolled in or assessed across multiple Pathways courses.
Figure 2 shows the competency level distribution for SLO 2, which focuses on evaluating the credibility and the use or misuse of scientific information. Based on 375 assessed students, 5% (17 students) demonstrated Developing Competence, 18% (66 students) were rated Competent, and 78% (292 students) demonstrated Advanced Competence.
