Smarter health: How Greece used AI to reopen its borders and curb COVID

For the past few weeks, On Point has aired a four-part special series called Smarter health: Artificial intelligence and the future of American health care.

In the series, we've explored the surge in funding for AI research and health care, the impact it might have on your health, and the ethical, moral and regulatory questions that come with the rapid expansion of powerful technology in the world's most expensive health care system.

MEGHNA CHAKRABARTI: We spent four months reporting this series and spoke on the record with approximately 30 experts, in everything from primary care, to electronic medical records to bioethics. Our work was led by On Point senior editor Dorey Scheimer, and she brought back more stories than we could fit into our radio series, including this one for this special drop.

It's about how the COVID pandemic led to the first ever use of artificial intelligence. Specifically something called reinforcement learning, to manage a massive public health challenge. Dorey takes it from here.

DOREY SCHEIMER: Summer 2020. The worldwide COVID death toll had hit half a million. Countries that relied on billions of summer tourism dollars wondered how long they could stay shut down.

HAMSA BASTANI: Greece had decided that they were going to open up their borders on July 1st, 2020 to let travelers in because they couldn't take the economic hit anymore.

Hamsa Bastani is a professor and researcher at the University of Pennsylvania, focused on algorithms and applications in health care. The Greek government had a decision to make. How could the nation open up to tourists while also keeping COVID under control in Greece?

At the time, many other countries opted for blanket policies like mandatory quarantining for incoming tourists, or testing every traveler upon arrival. Or, using rudimentary color coding systems that classified entire nations according to risk based on publicly reported COVID cases or death rates. Bastani says she and some colleagues thought none of those options were particularly sophisticated. They had a different idea.

BASTANI: We can probably use data science and machine learning to do better.

SCHEIMER: More to the point, Bastani and her colleagues thought AI could do better at catching asymptomatic COVID cases in visiting tourists.

BASTANI: They're getting like 30,000 to 100,000 travelers coming in every day. And they have capacity, even with group testing, to test about 7,500 people. So it's a very, very limited budget. And this is exactly the kind of problem that AI is very useful for. Because basically what you can do is try and predict who is most likely to test positive for COVID, test those people preferentially, because then you'll maximized the number of COVID cases that you catch at the borders.

SCHEIMER: The research group created a screening algorithm. It was called Eva.

BASTANI: They wanted to pick a name that was a single syllable, that was feminine and kind of inspire trust or confidence amongst the population.

SCHEIMER: Eva uses reinforcement learning, a machine learning training method that learns and gets better from trial and error. It was the first time a reinforcement learning algorithm had been deployed anywhere in the world for public health.

BASTANI: Our tool would get the passenger manifest, everybody filled out a passenger locator form for arriving that day.

SCHEIMER: From August to November 2020, every traveler coming to Greece had to fill out information about their place of origin, age and gender on the passenger locator form, 24 hours before arrival. And then the Eva algorithm went to work. For several months, before the algorithm was deployed, Greece randomly tested passengers coming into the country.

This gave Eva an initial dataset to analyze. Then the algorithm used the information in the passenger locator forms, primarily place of origin, along with the earlier testing data to identify which travelers should be flagged for testing.

Let's say a flight was coming to Greece from France. The algorithm determined the risk of French tourists testing positive for COVID based on previous French travelers' positivity rates. If the risk is high, every passenger on the flight might be tested, if the budget allows it. If the risk is low, the algorithm would determine a lower number of passengers should be tested at random.

BASTANI: There was also some work that we did in the background, optimizing and designing the testing supply chain, which lab services, which location, what is the amount of tests that they can handle and things like that. So that was all kind of carefully integrated to the algorithm to make sure that we were testing the right amount of people at each location.

SCHEIMER: After doing its analysis, Eva sent every passenger a QR code. When they arrived in Greece, passengers scanned their QR codes. Greek border control authorities would see if that person had been randomly assigned a test. Bastani says the difference between Eva and conventional border control testing in other countries is that Eva did not use macro public data, like a country's COVID caseload or death counts.

BASTANI: We were testing high risk patients who were symptomatic, who were typically in hospitals. And those are the people we were using our very limited, precious budget of test on. And that's what is getting reported to the public infrastructure. And then if you think about the traveler that you're talking about who is coming to Greece on vacation during the summer of 2020, extremely different kind of person.

SCHEIMER: Bastani says, Eva's more detailed analysis allowed the Greek government to better allocate its limited supply of COVID tests, and best use its test processing facilities.

BASTANI: My goal is to test the riskiest passenger so I can find the most cases today, so that they're not going to beaches, and clubs and infecting people. That's the win for today. But I also want to save some tests to do exploration. So that's this idea that I also want to do the surveillance.

I just want to spend some of my testing budget collecting good data that's spread out across all the populations, so that I can train good models tomorrow that will let me do good to make good decisions tomorrow. How you balance that tradeoff is really what these advanced algorithms are about.

SCHEIMER: Ultimately, the Eva algorithm identified 1.85 times as many asymptomatic, infected travelers as random surveillance testing. With up to 2 to 4 times as many during peak travel. That's according to research released by Bastani's group. In other words, the machine learning algorithm was better than random testing. For Greece to achieve the same effectiveness as Eva, they would have needed to use 85% more tests. That kind of testing and supply chain investment was impossible for Greece.

A Greek government official said at a press conference in July 2020 that using Eva has been an asset both for preparing the opening of the country to visitors from all over the world, as well as allowing for flexibility in decision making regarding our COVID-19 strategy. Eva is the first time reinforcement learning algorithm has been used for public health. In this case, the algorithm determined which travelers should get COVID tested, using minimal personal information.

But what about if the technology is used by other countries, for other purposes? We asked Bastani, Who is going to make sure that the same reinforcement learning technique won't be used to reinforce discrimination and bias at the border?

BASTANI: I definitely share your concerns that you don't want the algorithm exhibiting socioeconomic biases or racial biases and things like that. So I think on one hand, there's a big opportunity to reduce bias. Because these algorithms, they're looking at the outcomes, right?

So I think human decision makers, there's a lot of evidence that we have some biases that aren't actually reflected in reality, that you might think certain populations don't deserve care, or are more risky, for reasons that aren't really supported by the data. So by taking a data driven approach, you're kind of benchmarking it to actual outcomes. So, like high risk populations deemed by the algorithm are potentially, hopefully actually higher risk.

SCHEIMER: Bastani and her team have been approached by Canada and several other European countries that are potentially interested in using Eva. Because each country has different privacy and immigration policies, the algorithm would need to be adjusted to country-specific parameters. Bastani is also currently working on implementing Eva in Sierra Leone to optimize their public health supply chains for vaccinations and essential medications at community hospitals.

BASTANI: I'm excited to see more of these tools actually get deployed to help public health and other social good kind of problems. I think there's a lot of potential, but I think there's a lot of ethical and equity kind of challenges. And, you know, I hope that it goes forward in a responsible way that actually is a win win for society.

SCHEIMER: For On Point, I'm senior editor Dorey Scheimer.

CHAKRABARTI: That's just one of many stories Dorey reported for our special four-part series Smarter health. You can find the series in your podcast feed and we'd be grateful if you subscribed to the On Point podcast if you haven't already. There's a lot more cool stuff like this in the feed, we promise. I'm Meghna Chakrabarti. This is On Point.