Faster apps. More memory. Laura Brown and Zhenlin Wang bring efficiency to Big Data.
What memory resources will be available if applications A, B, and C all run together?
Big companies like Amazon and Google have even bigger data centers. Think 30 data centers each with 50,000 to 80,000 servers. And the underlying computer processors are not all identical; each year new improvements are integrated and added. Brown, Wang, and computer science colleagues from Western Michigan University are digging deep into the management of memory resources in these larger-than-life data centers.
The researchers use machine-learning techniques to create models that predict the cache and memory requirements of an application.
The challenge is how to make accurate predictions with such a massive variety of applications using the data center, and the different computers the application runs on. Applications might include Netflix streaming a movie, Airbnb running database queries, or NASA processing satellite images. Each app is not run in isolation with a dedicated machine. To maximize resources, data centers may have two or more applications all running on a single machine.
“If we learn the memory requirements of application A on computer X, what if the same app runs on machine Y or machine Z? Or, what memory resources will be available if A, B, and C all run together?” Brown asks.