Category: Uncategorized

Novel In-vehicle Interaction Design and Evaluation

Researchers: Philart Jeon, PI

Sponsor: Hyundai Motor Company

Amount of Support: $130, 236

Duration of Support: 1 year

Purpose and Target: To investigate effectiveness of an in-vehicle gesture control system and culture-specific sound preference, Michigan Tech will design a prototype of in-air gesture system and auditory displays and conduct successive experiments using a medium fidelity driving simulator.

Technical Background: Touchscreens in vehicles have increased in popularity in recent years. Touchscreens provide many benefits over traditional analog controls like buttons and knobs. They also introduce new problems. Touchscreen use requires relatively high amounts of visual-attentional resources because they are visual displays. Driving is also a visually demanding task. Competition between driving and touchscreen use for visual-attentional resources has been shown to increase unsafe driving behaviors and crash risk

[1]. Driving researchers have been calling for new infotainment system designs which reduce visual demands on drivers [2]. Recent technological advances have made it possible to develop in-air gesture controls. In-air gesture controls, if supported with appropriate auditory feedback, may limit visual demands and allow drivers to navigate menus and controls without looking away from the road. Research has shown that accuracy of surface gesture movements can be increased with addition of auditory feedback [3]. However, there are many unanswered questions surrounding the development of an auditory supported in-air gesture-controlled infotainment system: What type of auditory feedback do users prefer? How can auditory feedback be displayed to limit cognitive load? What type of menu can offer an easily navigable interface for both beginners and experienced users? More importantly, do these displays reduce the eyes-off-road time and frequency of long off-road glances? Does the system improve driving safety overall when compared to touchscreens or analog interfaces? These are among the many questions that we attempt to address in this research project. Moreover, we want to explore if there is any cultural difference in auditory perception. As a starting point, HMC and MTU will design in-vehicle sounds and conduct an experiment with American populations.References

  • Horrey, and C. Wickens, “In-vehicle glance duration: distributions, tails, and model of crash risk” Transportation Research Record: Journal of the Transportation Research Board, vol. 2018, pp. 22-28, 2007.
  • Green, “Crashes induced by driver information systems and what can be done to reduce them,” In Sae Conf. Proc. SAE; 1999, 2000.
  • Hatfield, W. Wyatt, and J. Shea, “Effects of auditory feedback on movement time in a Fitts task,” Journal of Motor Behavior, vol. 42, no. 5, pp. 289-293, 2010.

Less is More: Investigating Abbreviated Text Input via a Game

Researchers: Keith Vertanen, PI, Assistant Professor, Computer Science

Sponsor: Google Faculty Research Award

Amount of Support: $47,219

Abstract: While there have been significant improvements to text input on touchscreen mobile devices, entry rates still fall far short of allowing the free-flowing conversion of thought into text. Such free-flowing writing has been estimated to require input at around 67 words-per-minute (wpm) [1]. This is far faster than current mobile text entry methods that have entry rates of 20–40 wpm. The approach we investigate in this project is to accelerate entry by allowing users to enter an abbreviated version of their desired text. Users are allowed to drop letters or entire words, relying on a sentence-based decoder to infer the unabbreviated sentence. This project aims to answer four questions: 1) What sorts of abbreviations, a priori, do users think they should use? 2) How do users change the degree and nature of their abbreviations in response to recognition accuracy? 3) Can we train users to drop parts of their text intelligently in order to aid the decoder? 4) Can we leverage the abbreviation behaviors observed to improve decoder accuracy?

To answer these questions, we adopt a data-driven approach; collecting lots of data, from many users, over long-term use. To this end, we will extend our existing multi-player game Text Blaster [2], deploying the game on the Android app store. Text Blaster’s game play encourages players to type sentences both quickly and accurately. Players adopting successful abbreviation strategies will gain a competitive advantage. Text Blaster provides a platform to not only investigate abbreviated input but also a host of other open research questions in text entry. Data collected via Text Blaster will be released as a public research resource.


[1] Kristensson, P.O. and Vertanen, K. (2014). The Inviscid Text Entry Rate and its Application as a Grand Goal for Mobile Text Entry. In MobileHCI 2014, 335-338.

Development of the Safety Assessment Technique for Take‐Over in Automated Vehicles

Empty cockpit of vehicle, HUD (Head Up Display) and digital speedometer, autonomous car

Researcher: Myounghoon “Philart” Jeon, PI

Sponsor: KATRI (Korea Automotive Testing & Research Institute)

Amount of Support: $450,703

Duration of Support: 4 years

Abstract: The goal of the project is to design and evaluate intelligent auditory interactions for improving safety and user experience in the automated vehicles. To this end, we have phased plans for four years. In year 1, we prepare for the driving simulator for the automated driving model. In year 2, we estimate and model driver states in automated vehicles using multiple sensing approaches. In year 3, we design and evaluate discrete auditory alerts for safety purpose, with a focus, specifically, on take-over scenarios. In year 4, we develop real-time sonification systems for overall user experience and make guidelines.

Intellectual Merits: The proposed work will significantly advance theory and practice in the design of natural and intuitive communication between a driver (or occupant) and an automated/autonomous vehicle. The results of the proposed research will not only contribute to design of the current vehicles, but also guide directions of social interaction in the automated vehicles in the near future. We will additionally obtain a theoretical framework to estimate and predict driver state and driving behavior. A more comprehensive understanding of the relationship between multiple sensing (e.g., neurophysiological and behavioral) data and driver states will ultimately be used to construct a more generic driving behavior model capable of combining affective elements and cognitive elements to positively influence safer driving. The proposed work will specifically contribute to the body of current literature in using auditory user interfaces for the automated vehicle contexts. Subsequently, this work will significantly advance theory and practice in interactive sonification design, affective computing, and driving psychology.

Broader Impacts: Applying novel in-vehicle auditory interactions to facilitate take-over process and eco-driving, and to mitigate distractions has the high potential to significantly decrease driving accidents and carbon footprints. Moreover, the entire program of the proposed research can be further developed for other vehicle situations. The proposed work offers an exciting simulated driving platform to integrate research with multidisciplinary STEM (Science, Technology, Engineering & Math) education. The principal investigator of the project will train graduate students, who will mentor undergraduates in the Mind Music Machine Lab at Michigan Tech. Driving simulators will be used for diverse hands-on curricula and courses. The PI will design driving simulation activities to teach courses as part of Michigan Tech’s Summer Youth Programs (SYP). Michigan Tech’s SYP has a strong longitudinal history of recruiting women, rural students from the Upper Peninsula of Michigan, and inner city minority students from the Detroit and Grand Rapids area. They also have a decade of assessment data that demonstrate SYP alumni are more likely to pursue STEM college degrees. The PI will also try to develop close partnerships and collaborations with other universities of the consortium and KATRI in Korea. Students and researchers can come to Michigan Tech and conduct and experience research projects together using cutting edge technologies. Research and education outcomes will be disseminated by the team, through the planned workshop on “New opportunities for in-vehicle auditory interactions in highly automated vehicles” at the International Conference on Auditory Display and AutomotiveUI Conference.

DARPA Research Mentioned in AI Magazine Article

Shane Mueller

This month’s AI magazine includes the article “DARPA’s Explainable Artificial Intelligence Program,” which mentions Michigan Tech’s DARPA research. ICC member Shane Mueller is principal investigator of a 4-year, $255K DARPA XAI project.

The section of the article “Naturalistic Decision-Making Foundations of XAI” reads: “The objective of the IHMC team (which includes researchers from MacroCognition and Michigan Technological University) is to develop and evaluate psychologically plausible models of explanation and develop actionable concepts, methods, measures, and metrics for explanatory reasoning. The IHMC team is investigating the nature of explanation itself.”

Abstract: Dramatic success in machine learning has led to a new wave of AI applications (for example, transportation, security, medicine, finance, defense) that offer tremendous benefits but cannot explain their decisions and actions to human users. DARPA’s explainable artificial intelligence (XAI) program endeavors to create AI systems whose learned models and decisions can be understood and appropriately trusted by end users. Realizing this goal requires methods for learning more explainable models, designing effective explanation interfaces, and understanding the psychologic requirements for effective explanations. The XAI developer teams are addressing the first two challenges by creating ML techniques and developing principles, strategies, and human-computer interaction techniques for generating effective explanations. Another XAI team is addressing the third challenge by summarizing, extending, and applying psychologic theories of explanation to help the XAI evaluator define a suitable evaluation framework, which the developer teams will use to test their systems. The XAI teams completed the first of this 4-year program in May 2018. In a series of ongoing evaluations, the developer teams are assessing how well their XAM systems’ explanations improve user understanding, user trust, and user task performance.

Integrated Research and Education in Physical Design Automation for Nanotechnology and VLSI Technology Co-Design

Researcher: Shiyan Hu, PI, Adjunct Professor, Electrical and Computer Engineering

Sponsor: National Science Foundation

Amount of Support: $222,315

Duration of Support: 4 years

Abstract: This CAREER proposal aims to develop innovative chip layout design and optimization methodologies on nanotechnology interconnect and copper interconnect co-design. As the copper interconnect technology is approaching its fundamental physical limit, novel on-chip interconnect materials such as carbon nanotubes and graphene nano-ribbons have emerged as promising replacement materials due to properties such as superior conductivity and resilience to electromigration that otherwise have plagued copper interconnects. On the other hand, there could also be some issues for using nanotechnology interconnects such as their inferior performance as local interconnect and defects in fabrication. The PI will develop a novel co-design methodology which judiciously integrates nanotechnology into the practical VLSI circuit design together with various enabling techniques on co-design-aware buffering, layer assignment, routing, placement and clocking. The PI also proposes defect-aware techniques for mitigating the impacts due to defects, and improving robustness against faults for nanotechnology interconnects.

The broader impact of the proposed research is to significantly improve the circuit performance of nanoscale circuits. As the interconnect delay is the dominating factor of the circuit delay, the proposed research has the potential to help achieve the design closure for those difficult circuits whose timing cannot be closed even if various traditional physical synthesis optimizations have been stretched to the maximum extent. The proposed research can contribute integrated circuit design methodologies which enable the utilization of nanotechnologies into practical circuit design to defeat the fundamental limit on the prevailing VLSI technology. The PI will also develop a seamless integration of research with education such as developing the new graduate level course, developing lecture series and seminars, recruiting under-represented students, and showcasing the research results at conferences.


Chen Wang, Li Jiang, Shiyan Hu, Tianjian Li, Xiaoyao Liang, Naifeng Jing, and Weikang Qian. “Timing-Driven Placement for Carbon Nanotube Circuits,” in Proceedings of IEEE International System-on-Chip Conference (SOCC), 2015, 2015.

Lin Liu, Yuchen Zhou and Shiyan Hu. “Buffering Carbon Nanotube Interconnects for Timing Optimization,” in Proceedings of IEEE Computer Society Annual Symposium on VLSI (ISVLSI), 2014.

Jia Wang, Lin Liu, Yuchen Zhou, and Shiyan Hu. “Buffering Carbon Nanotube Interconnects Considering Inductive Effects,” Journal of Circuits, Systems and Computers (JCSC), v.25, 2016.

Yang Liu, Lin Liu, Yuchen Zhou, and Shiyan Hu. “Leveraging Carbon Nanotube Technologies in Developing Physically Unclonable Function for Cyber-Physical System Authentication,” in Proceedings of IEEE INFOCOM Cyber-Physical System Security Workshop, 2016.

Jacob Wurm, Yier Jin, Yang Liu, Shiyan Hu, Kenneth Heffner, Fahim Rahman, Mark Tehranipoor. “Introduction to Cyber-Physical System Security: A Cross-Layer Perspective,” IEEE Transactions on Multi-Scale Computing Systems, v.3, 2017, p. 215-227.

More info

VACCS – Visualization and Analysis for C Code Security


Jean Mayo, PI, Associate Professor, Computer Science

Ching-Kuang Shene, co-PI, Professor, Computer Science

Sponsor: National Science Foundation

Amount of Support: $130,001

Duration of Support: 3 years

Abstract: The proposed project will develop Visualization and Analysis of C Code Security (VACCS) tool to assist students with learning secure code programming. The proposal addresses the critical issue of learning secure coding through the development of a system for analyzing and visualizing C code and associated learning materials. VACCS will utilize static and dynamic program analysis to detect security vulnerabilities and warn programmers about the potential errors in their code. The research team has a significant experience in using visualization to teach computer science in such areas as parallel computing, geometric modeling and data encryption. The project will develop visualization and animation of common security vulnerabilities that can be customized for programmers with different level of programming experience. The project will evaluate the effectiveness of VACCS and instructional materials to improve students’ learning of secure coding.

The outcomes of this research will provide a better understanding of the visualization impact on secure programming instruction within a computing curriculum, as well as a deployable VACCS tool for faculty to adopt. This research will inform the broader community on the visualization potential for positive effects on the quality of code developed by future computer scientists. The VACCS tool and educational materials including tutorials, lectures, projects and extensive examples of teaching secure software development will be disseminated to academic computing community. In addition, this project will teach students how to perform software security audits using VACCS and will train graduate students in the art of teaching computer security.

More info

Leveraging Heterogeneous Manycore Systems for Scalable Modeling, Simulation and Verification of Nanoscale Integrated Circuits

Researcher: Zhuo Feng, PI, Associate Professor, Electrical and Computer Engineering

Sponsor: National Science Foundation

Amount of Support: $400,000

Duration of Support: 6 years

Abstract: The goal of this CAREER research project is to best unleash the power of emerging heterogeneous manycore CPU-GPU computing platforms. This will require revolutionizing the next-generation Electronic Design Automation (EDA) tools to deal with unprecedented complexity of circuits involving billions of components, making possible their modeling, analysis and verification tasks which would be prohibitively expensive and even intractable with methods in use today. The experience acquired in this research is also likely to contribute to advances in the use of computing in other areas of science and engineering, thus impacting areas such as complex system modeling and simulation, computational fluid dynamics, social computing, and systems biology. The PI will promote undergraduate and underrepresented student research, as well as K-12 education outreach, to motivate students in pursuing advanced engineering education or a career in STEM areas. Additionally, the PI will integrate the research outcomes into undergraduate and graduate curriculum development, and leverage interdisciplinary, industrial and international collaborations to effectively facilitate the proposed research work and broadly disseminate the results. Future nanoscale Integrated Circuit (IC) subsystems, such as clock distributions, power delivery networks, embedded memory arrays, as well as analog and mixed-signal systems, may reach an unprecedented complexity involving billions of circuit components, making their modeling, analysis and verification tasks prohibitively expensive and intractable with existing EDA tools. On the other hand, emerging heterogeneous manycore computing systems, such as the manycore CPU-GPU computing platforms that integrate a few large yet power-consuming general purpose processors with massive number of much slimmer but more energy-efficient graphics processors, can theoretically delivery teraflops of computing power. The proposal aims to accelerate a paradigm shift in EDA research to more energy-efficient heterogeneous computing regimes. Towards this end, the PI will develop systematic hardware/software approaches to achieve scalable integrated circuit modeling, simulation and verifications by inventing heterogeneous CAD algorithms and data structures, as well as exploiting hardware-specific and domain-specific runtime performance modeling and optimization approaches.


Xueqian Zhao*, Lengfei Han*, and Zhuo Feng. “A Performance-Guided Graph Sparsification Approach to Scalable and Robust SPICE-Accurate Integrated Circuit Simulations,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2015.

Lengfei Han and Zhuo Feng. “Transient-simulation guided graph sparsification approach to scalable harmonic balance (HB) analysis of post-layout RF circuits leveraging heterogeneous CPU-GPU computing systems,” Proceedings of ACM/IEEE Design Automation Conference (DAC), 2015.

Xueqian Zhao, Lengfei Han, and Zhuo Feng. “A Performance-Guided Graph Sparsification Approach to Scalable and Robust SPICE-Accurate Integrated Circuit Simulations,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2015.

Lengfei Han and Zhuo Feng. “TinySPICE Plus: Scaling Up Statistical SPICE Simulations on GPU Leveraging Shared-Memory Based Sparse Matrix Solution Techniques,” IEEE/ACM International Conference on Computer-Aided Design, 2016.

Zhuo Feng. “Spectral Graph Sparsification in Nearly-Linear Time Leveraging Efficient Spectral Perturbation Analysis,” ACM/IEEE Design Automation Conference, 2016.

Zhiqiang Zhao, Yongyu Wang, and Zhuo Feng. “SAMG: Sparsified Algebraic Multigrid for Solving Large Symmetric Diagonally Dominant (SDD) Matrices,” IEEE/ACM International Conference on Computer-Aided Design, 2017.

Zhiqiang Zhao, Zhuo Feng. “A Spectral Graph Sparsication Approach to Scalable Vectorless Power Grid Integrity Verication,” ACM/IEEE Design Automation Conference (DAC), 2017.

More info

Graph Sparsification Approach to Scalable Parallel SPICE-Accurate Simulation of Post-layout Integrated Circuits

Researcher: Zhuo Feng, PI, Associate Professor, Electrical and Computer Engineering

Sponsor: National Science Foundation: SHF: Small

Amount of Support: $250,701

Duration of Support: 4 years

Abstract: Unlike traditional fast SPICE simulation techniques that rely on a variety of approximation approaches to trade off simulation accuracy for greater speed, SPICE-accurate integrated circuit (IC) simulations can truthfully predict circuit electrical behaviors, and therefore become indispensable for design and verification of nanoscale ICs. However, for post-layout nanoscale circuits, using traditional SPICE-accurate simulation techniques to encapsulate multi-million or even multi-billion devices coupled through complex parasitics can be prohibitively expensive, and thus not applicable to large IC designs, since the runtime and memory cost for solving large sparse matrix problems using direct solution methods will increase quickly with the growing circuit sizes and parasitics densities. To achieve greater simulation efficiency and capacity during post-layout simulations, preconditioned iterative solution techniques have been recently proposed to substitute the direct solution methods. However, existing preconditioned methods for post-layout circuit simulations are typically designed with various assumptions and constraints on the circuit and systems to be analyzed, which therefore cannot be effectively and reliably applied to general-purpose SPICE-accurate circuit simulations. In this research project, the PI will study efficient yet robust circuit-oriented preconditioning approaches for scalable SPICE-accurate post-layout IC simulations by leveraging recent graph sparsification research. By systematically sparsifying linear/nonlinear dynamic networks originated from dense parasitics components and complex device elements of post-layout circuits, scalable, and more importantly, parallelizable preconditioned iterative algorithms will be investigated and developed by the PI to enable much greater speed and capacity for SPICE-accurate IC simulations in both time and frequency domains.

The successful completion of this work will immediately benefit the semiconductor industries. The algorithms and methodologies to be developed through this project will be integrated into undergraduate/graduate level VLSI design/CAD courses, while the research results will be broadly disseminated to major semiconductor and EDA companies for potential industrial applications. The CAD tools developed under this research plan will also be exchanged with collaborating industrial partners. The acquired experience in the proposed research plan is also likely to contribute to computing advances in other science and engineering fields, impacting broader research areas that are related to large/complex system modeling and simulation.


Lengfei Han, Xueqian Zhao, and Zhuo Feng. “An Adaptive Graph Sparsification Approach to Scalable Harmonic Balance Analysis of Strongly Nonlinear Post-layout RF Circuits,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2015. doi:DOI:10.1109/TCAD.2014.2376991

Xueqian Zhao, Lengfei Han, and Zhuo Feng. “A Performance-Guided Graph Sparsification Approach to Scalable and Robust SPICE-Accurate Integrated Circuit Simulations,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2015. doi:DOI: 10.1109/TCAD.2015.2424958

Zhuo Feng. “Fast RC Reduction of Flip-Chip Power Grids Using Geometric Templates,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2014. doi:DOI: 10.1109/TVLSI.2013.2290104

Xueqian Zhao, Lengfei Han, and Zhuo Feng . “A Performance-Guided Graph Sparsification Approach to Scalable and Robust SPICE-Accurate Integrated Circuit Simulations. ,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems., 2015.

Zhuo Feng. “Spectral Graph Sparsification in Nearly-Linear Time Leveraging Efficient Spectral Perturbation Analysis,” ACM/IEEE Design Automation Conference, 2016.

Lengfei Han, and Zhuo Feng. “Transient-simulation guided graph sparsification approach to scalable harmonic balance (HB) analysis of post-layout RF circuits leveraging heterogeneous CPU-GPU computing systems,” Proceedings of ACM/IEEE Design Automation Conference (DAC), 2015.

Lengfei Han, Xueqian Zhao, and Zhuo Feng. “An Efficient Graph Sparsification Approach to Scalable Harmonic Balance (HB) Analysis of Strongly Nonlinear RF Circuits,” Proceedings of IEEE/ACM International Conference on Computer-Aided Design (ICCAD), 2013.

Lengfei Han, Xueqian Zhao, and Zhuo Feng. “An Adaptive Graph Sparsification Approach to Scalable Harmonic Balance Analysis of Strongly Nonlinear Post-layout RF Circuits,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2015.

Xueqian Zhao, Lengfei Han, and Zhuo Feng. “A Performance-Guided Graph Sparsification Approach to Scalable and Robust SPICE-Accurate Integrated Circuit Simulations,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2015.

Xueqian Zhao, Zhuo Feng, and Zhuo Cheng. “An Efficient Spectral Graph Sparsification Approach to Scalable Reduction of Large Flip-Chip Power Grids,” Proceedings of IEEE/ACM International Conference on Computer-Aided Design (ICCAD), 2014.

More info.

Effective Sampling-Based Miss Ratio Curves: Theory and Practice

Circuit board

Researcher: Zhenlin Wang, PI, Professor, Computer Science

Sponsor: National Science Foundation

Amount of Support: $390,876

Duration of Support: 4 years

Abstract: Caches, such as distributed in-memory cache for key-value store, often play a key role in overall system performance. Miss ratio curves (MRCs) that relate cache miss ratio to cache size are an effective tool for cache management. This project develops a new cache locality theory that can significantly reduce the time and space overhead of MRC construction and thus makes it suitable for online profiling. The research will influence system design in both software and hardware, as nearly every system involves multiple types of cache. The results can thus benefit a wide range of systems from personal desktops to large scale data centers. We will integrate our results into existing open source infrastructure for the industry to adopt. In addition, this project will offer new course materials that motivate core computer science research and practice.

The project investigates a new cache locality theory, applies it to several caching or memory management systems, and examines the impact of different online random sampling techniques. The theory introduces a concept of average eviction time that facilitates modeling data movement in cache. The new model constructs MRCs with data reuse distribution that can be effectively sampled. This model yields a constant space overhead and linear time complexity. The research is focused on theoretical properties and limitations of this model when compared with other recent MRC models. With this lightweight model, the project seeks to guide hardware cache partitioning, improve memory demand prediction and management in a virtualized system, and optimize key-value memory cache allocation.

More details

Combining Data and Instruction Level Parallelism through Demand Driven Execution of Imperative Programs

Futuristic technology – Cool blue image of a computer cpu

Researcher: Soner Onder, PI, Professor, Computer Science

Sponsor: National Science Foundation

Amount of Support: $113,910

Duration of Support: 2 years

Abstract: This project advances a new execution paradigm, namely, demand-driven execution (DDE) of imperative programs. It studies the feasibility of the paradigm by establishing theoretical performance bounds, and identifying its key scalability aspects. The primary intellectual merit of the proposal is the DDE methodology and its use in removing impediments to parallelism due to data flow and control flow. The project’s broader significance and importance stems from its impact on the design of future processors, and its synergistic use of compilers and microarchitectures. Processors built using the DDE approach can better utilize computing resources and are energy efficient.

The basic idea behind the DDE methodology is to compile C-like programs such that both instruction-level and data-level parallelism can be used through a collaboration between compilers and microarchitectures. The basis for this collaboration is an executable, intermediate program representation known as “Future Gated Single Assignment” (FGSA) form into which a source program is compiled. The FGSA representation not only can be used by an optimizing compiler but also can be used as hardware instructions which can be directly executed by the microarchitecture.

More info.