Program Evaluation Methods

Evaluation of the IDEAS Modules and Center was planned to be conducted in two ways: Pre-Post assessment of global changes in CT and creativity, and post-only assessments of student products from classes/modules.  Three cohort groups were utilized: a calculus ready cohort, a pre-calculus cohort prior to implementation of the IDEAS Modules and Center, and a pre-calculus cohort following implementation of the IDEAS Modules and Center. A pre-post within subjects design was utilized as well.  Thus, the resulting mixed design examined the impact of the design modules on CT and creativity across time in the following manner:

  1. Pre-calculus cohort prior to implementation of the Ideas Center served as one control group
  2. Calculus-ready cohort after implementation of the Ideas Center served as a second control group
  3. The experimental group consisted of pre-calculus students who were taught using the new Ideas modules and center.

The hypotheses were centered on observed changes in CT or creativity across a 3 course sequence for the experimental group (pre-calc cohort who were taught utilizing the IDEAs modules).  We anticipated that the modules/center would enhance student’s CT, creativity, and communication skills.

Students from the first year engineering program were surveyed utilizing the following measures:

  • Critical Thinking – Critical Thinking Basic Concepts Online Test was used to assess global critical thinking .  Pre and post assessment of the scale’s subscores (basic critical thinking insight, understanding of components of critical thinking, intellectual standards, intellectual traits or virtues, and obstacles to critical thinking) were t conducted
  • Creativity –items assessing self-reported attitudes and behavioral intentions designed to measure creativity in engineering and creative self-efficacy in engineering were created for use in this project.  Some existing assessment tools, including Oreg’s (2003) Resistance to Change Scale (negatively correlated with creativity) and the Curiosity and Exploration Inventory (Kashdan, Rose & Fincham, 2004; predictive of creativity) were also measured.  Finally, a scale created by Zampetakis and Moustakis (2006) designed to measure one’s own creativity, the university environment that supports creativity, the importance placed on creativity by family and entrepreneurial intentions was used.