Timothy Havens, the William and Gloria Jackson Associate Professor of Computer Systems, has co-authored a paper recently published in The Journal of the Acoustical Society of America, Volume 50, Issue 1.
The paper is titled, “Recurrent networks for direction-of-arrival identification of an acoustic source in a shallow water channel using a vector sensor.” Havens’s co-authors are Steven Whitaker (EE graduate student), Andrew Barnard (ME-EM/GLRC), and George D, Anderson, US Naval Undersea Warfare Center (NUWC)-Newport.
The work described in the paper was funded by the United States Naval Undersea Warfare Center and Naval Engineering Education Consortium (NEEC) (Grant No. N00174-19-1-0004) and the Office of Naval Research (ONR) (Grant No. N00014-20-1-2793). This is Contribution No. 76 of the Great Lakes Research Center at Michigan Technological University.
Conventional direction-of-arrival (DOA) estimation algorithms for shallow water environments usually contain high amounts of error due to the presence of many acoustic reflective surfaces and scattering fields. Utilizing data from a single acoustic vector sensor, the magnitude and DOA of an acoustic signature can be estimated; as such, DOA algorithms are used to reduce the error in these estimations.
Three experiments were conducted using a moving boat as an acoustic target in a waterway in Houghton, Michigan. The shallow and narrow waterway is a complex and non-linear environment for DOA estimation. This paper compares minimizing DOA errors using conventional and machine learning algorithms. The conventional algorithm uses frequency-masking averaging, and the machine learning algorithms incorporate two recurrent neural network architectures, one shallow and one deep network.
Results show that the deep neural network models the shallow water environment better than the shallow neural network, and both networks are superior in performance to the frequency-masking average method.
Michigan Tech is among 17 top colleges and universities nationwide that have been selected to compete in the 2021-22 Marine Energy Collegiate Competition: Powering the Blue Economy The event is hosted by the U.S. Department of Energy’s (DOE) Office of Energy Efficiency and Renewable Energy (EERE).
These student competitors are poised to be the next blue economy innovators as they gain real-world experience and make industry connections to prepare for future careers in marine energy, according to the Marine Energy Collegiate Competition.
Administered by the National Renewable Energy Laboratory, on behalf of EERE’s Water Power Technologies Office, the competition challenges interdisciplinary teams of undergraduate and graduate students to explore opportunities for marine energy technologies via real-world concept development experiences, and to propose unique solutions to the burgeoning marine energy industry.
Submissions can run the gamut from concepts that aid in ocean observation and underwater vehicle charging to desalination and more, including—but not limited to—the markets identified in DOE’s Powering the Blue Economy™ report.
The DOE is hosting the challenge to advance one of the most up-and-coming industries: marine energy. Marine energy has the potential to provide reliable power to the blue economy, but further work is needed to optimize designs and reduce costs, according to the competition website.
The “blue economy” describes the sustainable use of ocean resources for economic growth, improved livelihoods and jobs, and ocean ecosystem health.
- Develop a market-research-supported business plan, which will include key aspects of their design of a system that could be commercialized to address power needs for a chosen sector of the blue economy
- Pitch their plan to a panel of judges and hypothetical investors
- Have the option to build and test a device to achieve energy production
- Engage with their community through outreach and educational activities.
- A 20- to 30-page market research-supported business plan and technical design of a marketable device powering any marine energy sector of the blue economy
- A 20-minute public pitch that will be presented to a panel of judges during the competition event at Water Power Week 2022 or virtual followed by a 15-minute Q&A session
- 5 minutes of the public pitch will focus on community engagement and outreach activities the team conducted throughout the year
- A poster summarizing the entire technical and business plan
- Optional: An effective prototype that will be tested for power performance at model scale. Results of the test will be summarized in the written report.
- 5 minutes of the public pitch will focus on community engagement and outreach activities the team conducted throughout the year
Inspiring Blue Economy Ingenuity
“The MECC provides an opportunity for a diversity of experience, education, and perspectives in exploring the possibilities of the blue economy,” said Arielle Cardinal, the MECC operations manager at NREL. “We’re excited to support the 2022 competitors in bringing new ideas and innovations to the forefront of marine energy.”
by Karen S. Johnson, Communications Director, College of Computing
Four Michigan Tech graduate students recently took 6th place in the U.S. Navy’s Artificial Intelligence (AI) Tracks at Sea Challenge, receiving a $6,000 prize.
The Challenge solicited software solutions to automatically generate georeferenced tracks of maritime vessel traffic based on data recorded from a single electro-optical camera imaging the traffic from a moving platform.
Each Challenge team was presented with a dataset of recorded camera imagery of vessel traffic, along with the recorded GPS track of a vessel of interest that is seen in the imagery.
Graduate students involved in the challenge were Zach DeKraker and Nicholas Hamilton, both Computer Science majors advised by Tim Havens; Evan Lucas, Electrical Engineering, advised by Zhaohui Wang; and Steven Whitaker, Electrical Engineering.
Submitted solutions were evaluated against additional camera data not included in the competition testing set in order to verify generalization of the solutions. Judging was based on track accuracy (70%) and overall processing time (30%).
“We never got our final score, but we were the “first runner up” team,” says Lucas. “Based on our testing before sending it, we think it worked well most of the time and occasionally tracked a seagull or the wrong boat.”
The total $200,000 prize was distributed among five winning teams, which submitted full working solutions, and three runners-up, which submitted partial working solutions.
The Challenge was sponsored by the Naval Information Warfare Center (NIWC) Pacific and the Naval Science, Technology, Engineering, and Mathematics (STEM) Coordination Office, and managed by the Office of Naval Research. Its goal was to engage with the workforce of tomorrow on challenging and relevant naval problems, with the immediate need to augment unmanned surface vehicles’ (USVs’) maritime contact tracking capability.
“The problem presented was to find a particular boat in a video taken of a harbor, and track its GPS coordinates.,” says Zach DeKraker. “We were provided with samples of other videos along with the target boat’s GPS coordinates for that video, which we were able to use to come up with a mapping from pixels to GPS coordinates.”
“Basically, we wanted to track boats with a video camera,” adds ECE graduate student Steven Whitaker. “Our team used machine learning and computer vision to do this. At weekly meetings we brainstormed approaches to tackling the problem, and at regular work sessions, together we programmed it all and produced a white paper with the technical details.”
Whitaker says the competition tied in pretty closely to work the students have already done. “We had a good majority of the code already written. We just needed to fit everything together and add in a few more details and specialize it for the AI Tracks at Sea research,” he explains.
Competitions like this one often connect directly or indirectly with a student’s academic and career goals.
“It’s good to not be pigeon-holed, and to use our knowledge in a different scenario,” Steven Whitaker says of these opportunities. “This helps us remember that there are other things in the world other than our small section of research.”
The team knew that there were two primary issues at hand. First, how can the pixel coordinates be translated into GPS coordinates? And second, how can the boat be located so that GPS pixel coordinates can be determined?
“Once we broke it down into these two subproblems, it became pretty clear how to solve each half,” DeKraker says. “Steven had already done a significant amount of work mapping pixel coordinates into GPS coordinates, so we had a pretty quick answer to subproblem one.”
The team met weekly to discuss their ideas for the project and compare and contrast how effective they would be as solutions to the problem at hand. Then, they got together on Fridays or during the weekends to work together on the project.
“Dr. Havens would come in to our weekly meetings and nudge us in the right direction or give tips on what we should do and what we should avoid,” Whitaker adds.
For subproblem two, after some discussion the group decided it was probably best to use a machine learning approach, as that promised the most significant gains for the least amount of effort, which was important given the tight schedule.
“We tried some different sub-projects independently and then worked together to combine the parts we thought worked best,” Evan Lucas says.
To identify the boat and track its movement, the team used a simple neural network and a computer vision technique called optical flow, which made the analysis much faster and cleaner. They used a pre-built algorithm, adding a bit of optical flow so that the boat’s position didn’t have to be verified every time.
“These two tools allowed us to find the pixel coordinates of the boat and turn them into GPS coordinates,” DeKraker says, whose primary role in the project was integrating the two tools and packaging it for testing.
“Part of my PhD is to map out a snowmobile’s GPS coordinates with a camera,” Whitaker says. “This is extremely similar to mapping out a boat’s GPS coordinates. I could even say that it was exactly the same. I don’t believe I’ll add anything new, but I’ve tweaked it to work for my research.”
Whitaker sums up the team’s division of responsibilities like this: “Evan detects all the boats in the picture; Nik detects which of those boats is our boat; Steven takes our boat position and converts it to GPS coordinates, Zach glued all of our pieces together.”
DeKraker says, “One of the things the judges stressed was the ease of implementing the solution. Since that falls under what I would consider user experience (UX) or user interface (UI), it was pretty natural for me to take these tasks on, having studied software engineering for my undergrad,” DeKraker says.
A primary focus was speed. “Using machine learning for object detection tends to be slow, so to mitigate that we used the boat detector only once every 5 seconds,” DeKraker explains.
“Most of the tracking was done using a very fast technique called optical flow, which looks at the difference between two consecutive frames of a video to track motion,” DeKraker says. “It tended to drift from the target though, so we decided on running the boat detector every 5 seconds to keep optical flow on target. “
“The end result is that our solution could run nearly in real-time,” he says. “The accuracy wasn’t the best, but given a little bit more time and more training data, the neural network could be significantly improved.”
DeKraker’s graduate studies focus heavily on various machine learning techniques, He says that this opportunity to integrate machine learning into our solution was a fantastic experience.
“First, it sounded like an interesting challenge. I don’t get to do a lot of software design these days, and this challenge sounded like a great opportunity to do just that,” he explains.
“Second, it looked like a great opportunity to build up my resume a little bit. Saying that you won thousands of dollars for your university in a nationwide competition sounds really good. And finally, I really wanted the chance to see a practical application of machine learning in action.”
DeKraker completed a BS in Software Engineering at Michigan Tech in 2018. He returned to Michigan Tech the next year to complete his master’s degree. He says the biggest reason he did so was to learn more about machine learning.
“Before embarking on this journey, I really didn’t know anything about it,” he says of machine learning. “Having this chance to actually solve a problem, to integrate a neural network into a fully realized boat tracker using nothing but a video helped me see how machine learning can be used practically, rather than merely understanding how it works.”
And although it was a fascinating exploration into the practical side of machine learning and computer vision, DeKraker says it’s rather tangential to his main research focus right now, which is on comparing different network architectures to evaluate which one performs best given particular data and the problem being solved.
DeKraker believes that the culture is the most magnetizing thing about Tech. “Everybody here is cut from the same cloth. We’re all nerds and proud of it,” he explains. “You can have a half-hour conversation with a complete stranger about singularities, the economics of fielding a fleet of star destroyers, or how Sting was forged.”
And the most appealing thing about Michigan Tech was its size. DeKraker says. “When I looked at a ranking of the top universities in Michigan, Tech was number 3, but still extremely small. It was a perfect blend of being a small but very good school.”
And he says the second-best thing about Tech is the location. “The Keweenaw is one of the most beautiful places on earth.”
DeKraker has many ideas about where he’d like to take his career. For instance, he’d love the chance to work for DARPA, Los Alamos National Laboratory, or NASIC. He also intends to commission into the Air Force in the next couple of years, “if they have a place for programmers like me.”
Evan Lucas is a PhD candidate in the Electrical Engineering department., advised by Zhaohui Wang. Lucas completed both a bachelor’s and master’s in Mechanical Engineering at Tech in 2012 and 2014,
Lucas, whose research interests are in applying machine learning methods to underwater acoustic communication systems, worked on developing a classifier to separate the boat of interest from the many other boats in the image. Although the subject of the competition is tangential to Lucas’s graduate studies, as computer vision isn’t his area, there was some overlap in general machine learning concepts. respectively.
“It sounded like a fun challenge to put together an entry and learn more about computer vision,” Lucas says. “Working with the rest of the team was a really good opportunity to learn from people who have experience making software that is used by other people.”
Following completion of his doctoral degree, hopefully in spring 2023, Lucas plans to return to industry in a research focused role that applies some of the work he did in his PhD.
Steven Whitaker’s research interests are in machine learning and acoustics. He tracks and locates the position of on-ice vehicles, like snowmobiles, based on acoustics. He says he has used some of the results from this competition project in his PhD research.
Whitaker’s machine learning research is experiment-based., and that’s why he chose Michigan Tech. “There aren’t many opportunities in academia to do experiment-based research,” he says. “Most machine learning is very software-focused using pre-made datasets. I love doing the experiments myself. Research is fun. I enjoy getting paid to do what I normally would do in my free time.”
In 2019, Whitaker completed his BS in Electrical Engineering at Michigan Tech. He expects to complete his master’s degree in Electrical Engineering at the end of the summer 2021 semester, and his PhD in summer 2022. His advisors are Tim Havens and Andrew Barnard.
Whitaker would love to be a university professor one day, but first he wants to work in industry.
Timothy Havens is associate dean for research, College of Computing; the William and Gloria Jackson Associate Professor of Computer Systems; and director of the Institute of Computing and Cybersystems (ICC). His research interests are in pattern recognition and machine learning, signal and image processing, sensor and data fusion, heterogeneous data mining, and explosive hazard detection.
Michael Roggeman is a professor in the Electrical and Computer Engineering department. His research interests include optics, image reconstruction and processing, pattern recognition, and adaptive and atmospheric optics.
Zhaohui Wang is an associate professor in the Electrical and Computer Engineering department. Her research interests are in communications, signal processing, communication networks, and network security, with an emphasis on underwater acoustic applications.
The Naval Information Warfare Center (NIWC) Pacific and the Naval Science, Technology, Engineering, and Mathematics (STEM) Coordination Office, managed by the Office of Naval Research are conducting the Artificial Intelligence (AI) Tracks at Sea challenge.
View more details about the Challenge competition here: https://www.challenge.gov/challenge/AI-tracks-at-sea/
Watch a Navy webinar about the Challenge here: https://www.youtube.com/watch?v=MjZwvCX4Tx0.
Challenge.gov is a web platform that assists federal agencies with inviting ideas and solutions directly from the public, or “crowd.” This is called crowdsourcing, and it’s a tenet of the Challenge.gov program. The website enables the U.S. government to engage citizen-solvers in prize competitions for top ideas and concepts as well as breakthrough software, scientific and technology solutions that help achieve their agency missions.
This site also provides a comprehensive toolkit, a robust repository of considerations, best practices, and case studies on running public-sector prize competitions as developed with insights from prize experts across government.