Category: STEM Education

We would like to congratulate some newly graduated PhD and MS students from the ACSHF program as of this summer:

Isaac Flint, PhD
Nishat Alam, MS
Anne Linja, PhD

Well done all!

Titles and abstracts for each can be found below:

Isaac Flint
Title:The Impact of Cognitive Ability and Age on Movement Corrections: An Exploration of the Neurocognitive and Physiological Contributors to Optimal Feedback Control

Abstract: Making successful movement corrections is an important part of navigating dynamic environments where unexpected obstructions can occur. Failure to engage in successful movement corrections can result in injury and, in some cases, death. One theory used to explain people’s ability to make movement corrections is the optimal feedback control theory, which follows the minimal intervention principle. Experiment 1 shows older adults are as likely as young adults to choose hand paths that require the least effort following a visual perturbation; however, they also commit more collisions and have slower movement speeds. Regression analyses show that success rates and movement times on the obstacle avoidance task are related to individuals’ measures of executive control and processing speed. P3b components, often associated with executive control, were elicited following medium and large cursor jumps. These ERP responses were different between the two conditions for young adults; however, they were not different for older adults. Experiment 2 shows young adults’ performance on obstacle avoidance tasks aligned with what would be predicted according to the minimal intervention principle, regardless of if responding to haptic/proprioceptive or visual feedback. The modality of the perturbation did have an impact on performance when the optimal path was ambiguous. The P3bs observed in Experiment 2 were also affected by the difference in the modality of feedback. When these findings are evaluated with the experiment’s other findings showing 1) greater P3b related activity for collision trials than non-collision trials, 2) very little differences between P3bs from trials with inward and outward movement corrections, and 3) EMG indicators of movement correction onset occur before the P3b peak, it suggests that the neural activity observed in this study has more to do with monitoring the movement corrections than path selection. The regression models from Experiment 2 also show the top-down processing of visual feedback is associated with a greater number of cognitive variables; yet some level of executive control is still associated with participants; tendency to make optimal reaching movements following physical perturbations.

Nishat Alam
Title: Types of Questions Teachers Ask to Engage Students in Making Sense of a Student Contribution

Abstract: In the student-centered classroom, a teacher’s interpretation and response to student mathematical contributions plays an important role to shape and direct students’ opportunities for sense-making. This research used a scenario-based survey questionnaire to examine what types
of questions middle and high school mathematics teachers indicate they would ask to engage
students in making sense of a high-leverage student mathematical contribution and their
reasoning about why particular questions are or are not productive. From the results, it could be
concluded that teachers asked more productive questions after seeing a set of possible questions.
Their beliefs about the productivity of the questions related to a variety of factors, including the
specificity of the question, student participation, student ability and whether incorrect solutions
should be discussed. The results could inform future work with teachers to productively use
student thinking in their teaching.

Anne Linja
Title:
EXPLICIT RULE LEARNING : A COGNITIVE TUTORIAL METHOD TO TRAIN
USERS OF ARTIFICIAL INTELLIGENCE/MACHINE LEARNING SYSTEMS

Abstract:
Today’s intelligent software systems, such as Artificial Intelligence/Machine Learning systems, are sophisticated, complicated, sometimes complex systems. In order to effectively interact with these systems, novice users need to have a certain level of understanding. An awareness of a system’s underlying principles, rationale, logic, and goals can enhance the synergistic human-machine interaction. It also benefits the user to know when they can trust the systems’ output, and to discern boundary conditions that might change the output. The purpose of this research is to empirically test the viability of a Cognitive Tutorial approach, called Explicit Rule Learning. Several approaches have been used to train humans in intelligent software systems; one of them is exemplar-based training. Although there has been some success, depending on the structure of the system, there are limitations to exemplars, which oftentimes are post hoc and case-based. Explicit Rule Learning is a global and rule-based training method that incorporates exemplars, but goes beyond specific cases. It provides learners with rich, robust mental models and the ability to transfer the learned skills to novel, previously unencountered situations. Learners are given verbalizable, probabilistic if…then statements, supplemented with exemplars. This is followed up with a series of practice problems, to which learners respond and receive immediate feedback on their correctness. The expectation is that this method will result in a refined representation of the system’s underlying principles, and a richer and more robust mental model that will enable the learner to simulate future states. Preliminary research helped to evaluate and refine Explicit Rule Learning. The final study in this research applied Explicit Rule Learning to a more real-world system, autonomous driving. The mixed-method within-subject study used a more naturalistic environment. Participants were given training material using the Explicit Rule Learning method and were subsequently tested on their ability to predict the autonomous vehicle’s actions. The results indicate that the participants trained with the Explicit Rule Learning method were more proficient at predicting the autonomous vehicle’s actions. These results, together with the results of preceding studies indicate that Explicit Rule Learning is an effective method to accelerate the proficiency of learners of intelligent software systems. Explicit Rule Learning is a low-cost training intervention that can be adapted to many intelligent software systems, including the many types of AI/ML systems in today’s world.

The Department of Cognitive and Learning Sciences will host two speakers at the next Applied Cognitive Science and Human Factors forum: Tauseef Ibne Mamun and Erin Matas, both ACSHF graduate students. Their presentations will be from 2:00 to 3:00 p.m. Monday (April 17) in Meese 109 and via Zoom.

Mamun will present  “The Use of Social Forums to Train Users about Shortcomings of Tesla Full Self-driving (FSD)”

Abstract: In the past decade, consumer adoption of commercial semi-autonomous vehicles has increased, and along with it user concerns about shortcomings of these systems, especially regarding safety. Users often turn to social media forums to discuss these shortcomings, find workarounds, and confirm their experience is common. We suggest that these forums may provide some of the best training for users to understand the limitations of AI, as they are not controlled by the vendor who has a vested interest in hiding the limitations of their systems. In two laboratory experiments, we examined how information from Tesla FSD forums impact participants’ ability to detect and predict hazardous driving situations in simulated scenarios. Drivers who received the training were better at anticipating and recognizing dangerous driving conditions, suggesting that exposure to user-generated explanations of the shortcomings of the system may in fact improve safety and acceptance of the systems.

Matas will present “Practicum Project: Leadership Program Evaluation Using Cognitive Task Analysis (CTA)”

Abstract: The Association of Research Libraries (ARL) Leadership Fellows Program is designed to prepare emerging library leaders for senior-level positions in research libraries and other types of organizations. Erin Matas is a 2021-2022 cohort Fellow. For her ACSHF practicum project, Erin used Cognitive Task Analysis (CTA) methods to identify areas where improvement is needed for the leadership program’s training and development. The Leadership Fellows program has a year-long curriculum that targets different learning topics each month. Using human factors methods, Erin analyzed challenges in the Fellows’ current jobs and compared them with the topics covered in the program to determine if there are any training gaps. Erin interviewed 7 Fellows using CTA to identify cognitively complex aspects of their work and systematically analyzed the data. She developed task diagrams from each interview, identified themes, and presents results in a concept map. To round out the project, Erin will deliver the concept map and an executive summary report of her findings and recommendations regarding training to the Director of the ARL Leadership Fellows Program. The Program Director will include the recommendations in the final assessment of the program for the year to the ARL executive leadership team. Did CTA techniques identify overlooked training topics, pinpoint where more support is needed, and/or reinforce the strength of the current curriculum? Find out on April 17th at the ACSHF Forum!

The Department of Cognitive and Learning Sciences will host two speakers at the next Applied Cognitive Science and Human Factors forum: Katrina Carlson and Brittany Nelson, both ACSHF graduate students. Their presentations will be from 2:00 to 3:00 p.m. Monday (April 3) in Meese 109 and via Zoom.

Carlson will present ” Engineering Self-efficacy and Spatial Skills: A two-part study”

Abstract:

The research team behind previous work on the increased academic and retention outcomes of students who have taken a Spatial Visualization Intervention course at MTU postulates that affective changes within the students as a result of the course may be responsible for downstream academic success. One possible explanation may be related to the students’ confidence in their ability (self-efficacy) to gain the skills needed to become an engineer.  Extensive research has been conducted on self-efficacy, and academic self-efficacy has been shown to be significantly correlated with academic performance.

The first part of this study examines the general and engineering self-efficacy of students at the beginning and end of the Spatial Skills Intervention course, Spring 2023 (N= 9), and compares these to students at the beginning and end of a first year engineering class. One hundred sixty-eight students completed general and engineering self-efficacy surveys. The General Self-Efficacy Scale was used; this tool was developed from longer scales and was found to be a reliable and valid measure of overall self-efficacy and not a specific skill area.  The Assessment of Engineering Self-Efficacy V3.0 for undergraduate engineering students  was also used. Items on this measure are related to predictions of future academic ability and their sense of belonging in engineering and STEM classes.

The second part of this study will examine students’ visual and spatial perception, memory, and skills through a battery of tasks using the PEBL Platform.  Previous research has examined the development of spatial skills and the resulting increase in problem-solving skills across domains that require spatial reasoning.  Research has also been conducted to examine whether spatial visualization, the ability to mentally maneuver 2D and 3D objects, is a single ability or is composed of more than one skill or ability. Participants (N=80) will include both Intro to Psychology students, who will also take the Purdue Spatial Visualization Test with Rotations (PSVT:R) as a part of this battery, and first year engineering students.This part of the study will examine the relationships between students’ visual and spatial skills, drawing skills (engineering students only), and their PSVT:R scores and seeks to examine a possible taxonomy of spatial skills.  This battery may serve as a reliable and valid assessment in the future of student skills at the high school and/or college level to indicate a need for additional instruction and practice of skills. There may be applications of these findings in other fields and for other purposes, such as geography, computing education, and military use.

Nelson will present “Title: Preliminary Evaluation for an Educational Intervention: Insights from a Usability Survey”

Abstract: 

Increasing whole grain intake can reduce the risk of chronic health conditions such as cancer and heart disease. However, people continue to make poor dietary health decisions, and the life expectancy for Americans is declining. Therefore, a novel intervention is needed to boost informed dietary decision-making. This study aimed to (1) provide preliminary evidence on the effectiveness, enjoyment, and efficiency of a novel intervention and (2) identify practices for making scientific information more usable. The study used a self-report online survey. Qualitative and quantitative data were collected to test the effectiveness, enjoyment, and efficiency of the educational intervention and how to improve it. Results suggest that the intervention is effective at increasing informed preventative decision-making. One hundred percent of participants showed adequate gist understanding across the four knowledge domains: habit gist understanding, whole-grain gist understanding, gist understanding of benefits, and gist understanding of susceptibility and severity. The results also revealed several strategies for increasing the usability of other educational interventions for a student sample demographic: increase/incorporate graphs, data, and references to increase the trustworthiness of an intervention. These results suggest that an educational video intervention effectively increases informed decision-making for preventative behaviors. These findings are also valuable for future intervention development and testing, making this proposal the next step for preventative care.

The Department of Cognitive and Learning Sciences will host MTU Humanities Assistant Professor Jason Archer at the next Applied Cognitive Science and Human Factors forum.

The presentation, “The Embrace of the Surgical Machine: Touch, Practice, and Power in the Operating Room”, will be from 2:00 to 3:00 p.m. Monday (February 6) in Meese 109 and via Zoom

In this presentation, Jason Archer will talk about his work in the area of Human Machine Communication, focusing on research related to the da Vinci Surgical System (dVSS), a system widely used in robotic-assisted surgery. Dr. Archer will discuss how concerns with touch-oriented media sparked his investigation of the dVSS, explain the challenges of doing research in a surgical setting, and share stories from interviews with robotic surgeons, and observations from the OR, that help highlight some of his findings.

The Department of Cognitive and Learning Sciences will host ACSHF PhD Students Lauren Sprague and Brandon Woolman at the next Applied Cognitive Science and Human Factors forum. Their presentations will be from 2:00 to 3:00 p.m. Monday (January 23) in Meese 109 and via Zoom.

Sprague will present “Pilot test of critical flicker fusion in combination with functional near-infrared spectroscopy (fNIRS) in order to accurately measure cognitive workload during a visuospatial vigilance task.”

Abstract:
Vigilance tasks are largely considered to be stressful to perform, difficult to stay on task, and cognitively draining due to the mental demands of sustaining attention. These tasks, which involve the monitoring of an environment for critical signals while avoiding more frequent neutral signals, induce what has been dubbed the vigilance decrement. The vigilance decrement typically involves a decline in performance as well as an increase in response time. During the investigation of this decrement, some tools need to be validated before they should be used to investigate it. Functional near-infrared spectroscopy is a non-invasive brain imaging technique that provides real-time data on changes in light absorption caused by the hemodynamic activity of the brain region in question. This hemodynamic activity provides information about cognitive effort. The flicker fusion threshold is the frequency at which an observer perceives a flickering light as static. This threshold can provide information about cortical arousal, alertness, fatigue, and cognitive workload. This study seeks to determine if these two methods, utilized alongside the NASA-TLX a measure of mental workload, can provide detailed information about the cognitive effort of a task as well as any decline in mental resources due to the mental effort of a visuospatial vigilance task.  

Woolman will present “Assessing Cognitive Impairment and Early Alzheimer’s Disease Using a Reverse Visually Guided Reaching Task.”

Abstract:
Alzheimer’s Disease (AD) is the most common form of dementia, which is known for its impacts on cognitive functions, especially memory. Early signs of AD can be difficult to diagnose (Porsteinsson et al., 2021), neuropsychological test batteries designed for dementia are only moderately reliable. Recent findings in the field of motor behavior have show novel motor tasks to be sensitive to cognitive differences between younger and older adults. Some motor tasks have shown to be more sensitive to cognitive deficits compared to neuropsychological test batteries (Watral & Trewartha, 2021). For example, tasks like the visuomotor rotation task, where participants adapt to a visuomotor perturbation, have been identified as a means for assessing cognition (Buch, Young & Contreras-Vidal, 2003). Recent work by Tippet and Sergio (2006) developed a reverse visually guided reaching task (rVGR) in which participants make a series of aimed movements toward a target. During the rVGR task, the visual cursor moves in the opposite direction of the physical reach, forcing the participant to correct their movements by reversing the reaching direction. Measures of performance in this task, such as movement speed and inconsistency of movements, have been shown to change in preclinical Alzheimer’s populations (Hawkins & Sergio, 2014). The current investigation seeks to further characterize rVGR performance differences between younger adults, older adults, and individuals with early AD (diagnosed with MCI or mild AD). For this purpose, we are recruiting 20 younger adults, 20 healthy older adults, and 20 early AD patients. We are testing the prediction that participants with AD should perform similarly to the controls on a VGR task but show significant deficits on the rVGR task. Additionally, correlations will be examined between performance on a neuropsychological battery and the rVGR task performance to test the prediction that performance on the motor task are related to changes in cognition in AD. This work may provide the foundation for using motor tasks as a diagnostic tool for cognitive impairments in preclinical stages of MCI and Alzheimer’s Disease. Early diagnosis of cognitive impairments due to MCI and AD could allow physicians to maximize the effectiveness of available treatment methods for slowing the progression of the disease.

The Department of Cognitive and Learning Sciences will host ACSHF PhD Students Anne Linja and Alex Watral at the next Applied Cognitive Science and Human Factors forum. Their presentations will be from 2:00 to 3:00 p.m. Monday (November 28) in Meese 109 and via Zoom.

Linja will present her research titled “Through the Eyes of Tesla FSD Drivers: Tesla Drivers’ Social Media Posts Never Run Out of Gas”

Abstract: With the recent deployment of the latest generation of Tesla’s Full Self-Driving (FSD) mode, consumers are using semi-autonomous vehicles in both highway and residential driving for the
first time. As a result, drivers are facing complex and unanticipated situations with an unproven technology, which is a central challenge for cooperative cognition. One way to support cooperative
cognition in such situations is to inform and educate the user about potential limitations. Because these limitations are not always easily discovered, users have turned to the internet and social media
to document their experiences, seek answers to questions they have, provide advice on features to others, and assist other drivers with less FSD experience. In this presentation, I will explore a novel approach to supporting cooperative cognition: Using social media posts can help characterize the limitations of the automation in order to get information about the limitations of the system and explanations and workarounds for how to deal with these limitations. Ultimately, our goal is to determine the kinds of problems being reported via social media that might be useful in helping users anticipate and develop a better mental model of an AI system that they rely on. To do so, we examine a corpus of social media posts about FSD problems to identify (1) the typical problems reported, (2) the kinds of explanations or answers provided by users, and (3) the feasibility of using such user-generated information to provide training and assistance for new drivers. The results reveal a number of limitations of the FSD system (e.g., lane-keeping and phantom braking) that may be anticipated by drivers, enabling them to predict and avoid the problems, thus allowing better mental models of the system and supporting cooperative cognition of the human-AI system in more situations.

Watral will present her research titled “Sensitivity of a Robotic Hit & Avoid Task to Executive Control and Global Cognitive Changes in Healthy Aging and Cognitive Impairment”

Abstract: We recently found that a rapid motor decision-making task is sensitive to age differences in executive control and can isolate the cognitive from the sensorimotor contributions to task performance (Watral & Trewartha, 2021). However, we are also interested in this task’s ability to distinguish between healthy aging and cognitive impairment as seen in the early stages of Alzheimer’s. In this presentation, I will revisit the results from Watral & Trewartha (2021) and show preliminary findings comparing task performance between healthy older adults and those who exhibit cognitive impairment. Additionally, task parameters thought to be associated with executive control will be compared to a traditional measure of executive functioning (Trail Making Test) and a global measure of overall cognitive functioning (Montreal Cognitive Assessment).

The Department of Cognitive and Learning Sciences will host CS and CLS Assistant Professor Dr. Briana Bettin at the next Applied Cognitive Science and Human Factors forum Monday (November 14) from 2:00pm to 3:00pm in Meese 109 and via Zoom.

Briana Bettin is an assistant professor of both computer science and cognitive and learning sciences. She received her master’s in human-computer interaction from Iowa State University and her bachelor’s and Ph.D. in computer science from Michigan Tech. Her research blends user experience methodologies with education research to better understand programming students and the impacts of the classroom environment.

Abstract: Our increasingly digital society requires citizens to effectively communicate about and with computing technologies to thrive. All too often, these technologies impacting our lives are suggested to be apolitical, while details of their design – including their limitations – are obfuscated, ignored, or considered inevitable. Navigating the world of computing and digital landscapes already poses a variety of challenges which can make new obstacles, “glitches”, and outcomes feel insurmountable. Coupled with stereotypical notions suggesting increased difficulty and limited societal impacts of computing and programming, learners of all ages and skills can easily become frustrated and discouraged from learning skills and topics necessary for today’s society. 

This talk explores the need for and explorations toward increasing awareness, understanding, and agency toward computing as a crucial component of modern society. From learning to code to understanding data collection, opportunities abound to transform learner relationships with computing material. Empowering learners to communicate confidently about computing gives them the power through language to begin critically analyzing and reimagining these technologies. With technology demystified and new pathways opened, learners may feel more capable to advocate for and/or create change. This approach to agency formation draws from theories of “punk DIY subculture”, positing that “punk programmers” might be defined as individuals who recognize faulty societal norms in technology design, and “DIY” approaches to subvert them. By toppling barriers to entry, giving learners a voice, and inspiring agency, more “punk programming pedagogy” may play a key part in reimagining the multifaceted possibilities of our sociotechnical futures.