Over the last two years, the COVID-19 pandemic has disrupted many aspects of education and professional life, including academic research. This summer, three undergraduate students at Grand Valley State University directed their own research to understand and examine the pandemic through the lens of their three very different disciplines.

Payton Miloser, a statistics major, tracked the spread of infection through Michigan counties; Ashten Duncan, a marketing major, developed propositions for Design Thinking strategies to help small businesses cope with disruptions like COVID-19; and Seth Ockerman, a computer science major, created and trained a Convolutional Neural Network to track masking rates through images posted to social media.

“Our idea was to see how we could train a model and deploy it on a city, to evaluate the interaction between face mask usage and daily COVID cases,” Ockerman said. “We were using Twitter, so, for example, it would count the number of images with masks in them in all the tweets that were made in Detroit in a given day.”

In order to learn something like how to identify a mask in a photo, a Convolutional Neural Network needs a dataset on which to train. In addition to engineering the ConvNet itself, Ockerman wrote a script that pulled images of people off Twitter. Those images were sent to paid survey takers through the Amazon Mechanical Turk service, a crowdsourcing marketplace, who identified whether or not the people were wearing masks. The resulting dataset can be used by any ConvNet, not just Ockerman’s, to recognize face masks in photos.

“After COVID hit, there were a ton of people who were looking for how to use that broad field of computer vision to help fight the pandemic,” Ockerman said. “It became a collective effort to come up with a large, diverse dataset of people wearing medical masks. The images in datasets tend to be very close-up shots of the face with ideal settings and lighting, and often not that much diversity, which can introduce bias. But our idea was to evaluate mask usage based on the actual pictures being posted to social media, so we wanted data that more accurately mimicked real life.”

To test whether or not their dataset effectively met their use case, Ockerman and his faculty mentor Professor Erin Carrier found other datasets that had been created to train mask recognition and used them to train ConvNets that had been optimized to the best possible fairness. But that dataset they’d created themselves proved to be the most effective teacher for identifying images of masks on social media.

“Now that we have a neural network that’s been trained to detect masks in images on social media, we’ve been deploying it on cities to compare their masking rates with COVID cases,” Ockerman said. “Both the professor and I are excited by the possibility of looking into this, because it does look like there’s something there. We’re also in the process of publishing the dataset itself, so that it can be used by other researchers.”

Ockerman’s study will likely be published in an undergraduate Computer Science journal. His peer Duncan, however, is looking to submit her research to a more practical publication.

“It’s getting exciting, because it feels like something that can be published in a journal that helps small businesses with marketing,” Duncan said. “In writing the paper, we have to have empathy for these businesses and their struggles, to think about the steps they had to take, while simultaneously leading them into thinking through the Design Thinking mindset.”

The broad framework of Duncan’s paper is the “EDIPT” Design Thinking model created at Stanford University. To her, the five stages— Empathize, Define the Problem, Ideate, Prototype and Test— encapsulates all the different strategies they discuss, representing how a business could successfully respond to a problem like COVID-19.

“At the time I was thinking of putting together a research proposal, I was doing social media marketing for my sister-in-law’s small business,” Duncan said. “She told me about how when she was getting supplies for the business, she ended up connecting with a bunch of other small business owners in town, and they were sharing complaints with each other like ‘oh man, post-COVID prices are like double and triple what they were before.’ Especially in the food industry. And I thought, how do we explore why these things happened, while also focusing on what our clients need and stay open?”

From this starting point, Duncan and her mentors, Professor Kevin Lehnert and Seidman Associate State Director Hollie Blagg, examined the literature on how Design Thinking could be used to help people like her sister-in-law. As her research deepened, her paper evolved into more of a framework on how small business owners can implement those strategies to get through disruptions, using COVID as an example. In order to represent this in her Student Scholars Day poster presentation, she used an activity.

“We used sticky notes to walk people through that EDIPT model,” Duncan said. “I asked things like, ‘say you had a bakery. Now your brick-and-mortar store is closed in-person, and you can only sell online. The only way you can continue to sell your product is if you know what your customer wants.’ So I was having people present back to me what their ideas were, and it helped influence my research. What we noticed is that people are very good at thinking of a solution to a problem, but not so good at fully thinking through the ‘why.’ Sometimes we skip immediately to a potential answer and miss out on a lot of ideas that could better suit our needs, if we examined them more carefully.”

While Duncan’s research examined overcoming the economic hurdles of the pandemic, Miloser studied the COVID-19 spread across the state of Michigan.

“When you have a variable and a response, you’re looking for the correlations between them,” Miloser said. “Moran’s I, a measure of spatial correlation, does that with two geographical locations. I focused on Michigan specifically, both because the information has been consistent here and because I’m familiar with the state, which has been super helpful. It’s focusing on how COVID spreads through those counties, using an adjacency matrix— which sounds complicated, but really it’s not.”

The adjacency matrix is simply a square of rows and columns that keeps track of which counties are touching. If two counties are touching, then the box where they intersect will have one inside; if they don’t, it has a zero. This binary representation of which counties are adjacent allows Moran’s I to measure whether COVID outbreaks correlate across borders.

“Once that goes into the equation, we’re able to calculate a relationship focused on case rates,” Miloser said. “That way we can adjust for the population of each county. If we have one hundred cases in one county, but only two hundred people living here, it’s much different than a county that may also have a hundred cases but has a thousand people.”

Miloser is currently working on writing her manuscript, both for her Honors Senior Project and to potentially publish in a journal, but her research during the summer involved a considerable amount of programming.

“My mentor, Professor Paul Stephenson, developed this idea over his sabbatical but didn’t have the time to put it into practice,” Miloser said. “I started by exploring coding maps with him and learning how to use the programming system that he liked, but since the project has kicked into high gear, I’ve done all the coding, all the math, all the Excel work. For example, when the Moran’s I value is closer to one, we get this clustering effect. I’ve figured out how to program a map of Michigan that shows us this cluster, so we can immediately see where it’s happening.”

Like her peers, Miloser’s goal is to contribute to furthering understanding of not just how we dealt with COVID-19 in the past, but how we can better shape our policies and strategies in the future. This year’s undergraduate scholarship represents Grand Valley’s first student research projects that were developed after the global pandemic: as the years progress, it will be interesting to see how subsequent student researchers will join Miloser, Duncan, and Ockerman in applying their own disciplines to studying the virus that shook the world.