• Home
• Contact
• Selected works
• Newsfeed
• About
• Search

Designing Public Health Surveillance

@seabbs started a very thought provoking thread ¹ on mastodon the other day. It only came to my attention today (because I haven’t yet figured out the whole timeline thing). The entire contents of the post are located here in which he asks what should the role of academics be in designing public health surveillance and outbreak response systems. He asks all the right question along the line of are academics (including PhD students, post-docs, early career researchers, etc) the right people to be designing these systems and/or the tools to respond to them. Are the incentives and expertise aligned (e.g., who funds whom and how are careers made). It is a real problem and an open question for public health and academic on how to organize on this topic. It is further compounded by the fact that there are other partners in this dance. There are normal funding government mechanisms, extra/intra-mural funders, NGOs (e.g., like Wellcome, Gates, GAVI, CEPI, among others) and the brass tacks of promotion, compensation, and ego.

In the absense of clear defintions, it seems that governments and large NGOs turn to consultants to figure these things out. Some examples are here, here, here, this one, and had a strong response in vaccination planning. And there are even for profit companies that have sprung up to work on these topics. Consultants can promise results and use their own people and “expertise” to focus on strategy while these agencies might be understaffed/ occupied with keeping the day-to-day going. Consultants are trained to cut through internal bureaucracy. For someone looking for a quick fix, consultants promise a result (and you pay them the big bucks to get it). At the end of the engagement and lots of meets, you get a really fancy presentation, and maybe even an excel spreadsheet “model.” However, as is often case, the major problem is that the consultants don’t stick around to see their recommendation implemented and aren’t on the hook if the recommendations don’t work (because of course they can’t be carried out to the letter for mundane reasons and thus they are never implemented in full giving plausible deniability).

My comments back to @seabbs and current thinking is that the solution to this problem must really be solutions. There is likely no one-size-fits-all solution to the question of outbreak response and surveillance. What works in the UK with the NHS will likely not work in the southern United States which might not work in central Peru or the Gambia. The local dynamics shapes a lot of what can be done. These feelings are shaped by two experiences– leading benchmarking for North America for a multinational company and responding to the COVID-19 pandemic as part of a non-profit health system.

When working with colleagues across the world to benchmark processes and procedures, I quickly learned that there were reasons why certain things could not work. Some of them appeared silly, but they were real obstacles. It could be how people were organized (e.g., management structures), technical (e.g., IT systems), and even legal nuances of a given country or city. While a recommendation appeared reasonable at first glance, when digging into the minutia it would fail miserably without accounting for these differences. This was often the case working with consultants who certainly never dug into these details and didn’t stick around to help with implementation². Similarly, the brilliant and driven stagiaires from the grandes ecoles often had amazing ideas, developed brilliant models, and pushed all of us yet failed most of their projects for a lack of appreciation of the local systems (which to be fair they couldn’t get with their often abbreviated project timelines and ambitious project scopes). While the spirit of different systems and approaches can be shared, very few things are plug and play without modification to the local terrain.

During the COVID-19 pandemic, differences in different systems was very apparent. Following the world-class science and analysis out of the UK, I couldn’t help but be jealous of the surveillance systems available to scientists there. In the United States public health is decentralised to the individual states. Then within a given state, the counties have a large degree of latitude over local health ordinances. Further still, public health departments don’t often provide health care. Providing health care falls to a mixture of non-profit and for-profit hospitals, free clinics, and cash/free clinics. These providers span administrative boundaries (e.g., my system primarily served patients in two states and four to five counties). Data flowed to the state public health department and on to the Centers for Disease Control and Prevention. At the most basic level, there wasn’t any unified reporting of daily incidence across counties, so we had to make that, too. Eventually, the NY Times tracker and the Hopkins datahub came along, but they didn’t correct their records as things evolved or catch the errors that we did.³ By spring of 2021 the CDC started posting cases at lower granularity, but these were often several days behind.

When the COVID-19 pandemic hit, the local health department didn’t have the capacity to run testing operations, so our system did. With our entire data science department we stood up all kinds of sophisticated outbreak detection systems and targetted deployment of mobile testing resources (i.e., a van that could show up in a community and offer testing). All of this was made possible by being able to link testing results, outcomes, and administrative data to very rich electronic health records. Then the state contracted another set of venders to expand testing (which was good, widespread testing and lower result times were great) which led reduced coverage in our surveillance, outbreak detection, and targetted education program because they didn’t share the data and the official government offices only provided aggregated data back. Furthermore, for the targetted outreach to work, we had to meet with community leaders (which included church leaders, leaders of special communities, and civic leaders). We also developed some very sophisticated modelling approaches to inform local leaders to our projects. This included modelling variants starting with Alpha then Delta and Omicron afterwards.⁴ Eventually the CDC started to post projections of cases and hospitalizations, but our models were often (always) better because of more detailed data and were more reflective of how care was actually provided locally. For instance, while the hospitalizations by county were projected by the CDC (again, by spring of 2021), people on the ground knew that the proximity to different major medical centers was important. If you lived in county A and got sick, you were likely going to county B’s major hospital because it was 7 miles away vs the other hospital 25 miles away. Eventually the “case” and “hospitalization” would get rolled up as a hospitalization in the citizen’s home county (and thus continue to propagate garbage in/garbage out to the models).

So back to the question at hand. What do we need and where do academics fit in? I think this might not be the right formulation for the problem. If the question is how does one design and maintain a disease surveillance and outbreak detection system what skills and profiles should be involved? This is more of a functional requirements approach, where if you lay out the functions that you need, the resources will become clearer. Functionally, you will need systems that will operate at the level of action or where interventions and decisions take place. For deployment of surveillance programs, this means you need an approach for everywhere a sample may be taken. For outbreaks this may be similar but also include different sets of people. You will need systems that can plug into the local context and operate within the accepted bounds of the existing infrastructure (e.g., if people only go to be triaged at the emergency department then the emergency departments need to be part of system. Similarly, if free clinics make of the majority of response, then they should be included). This approach could be taken for each level of where decisions are made and resources are allocated (e.g., from city up to national government). Functionally, you want people who can bring experience in running long-term systems. This would include the people who design and maintain them (e.g., IT, and importantly these might be the same people running other public facing systems that have high volume or unite disparate data sources), support them, and run them. Certainly people at the interfaces of all of these operations as well. You would also want people who know the cutting edge approaches as well as the people who have experience deploying and running these types of programs (e.g., public health administrators, contact tracers, etc).

I think the most important consideration is that the people who will run the systems day to day have to be involved. The majority of the team cannot be temporary (even the two year secondments or post-docs)–they have to live with the results and resolve the issues. Then you’ll need expertise. This is where the academics at all levels can come in. How many people do we need to sample? What are the changepoint detection methods we should use? All of this is tempered by the people with experience. Even if we detect a change how does that inform practice? What information is needed in order to make changes? What is truly relevant? What data do we actually have? What information could we get? This also helps with all of the context dependent questions about jurisdiction as well. Furthermore, integration of the people actually providing the care, doing the testing, running the vaccination schemes is important. The “boots on the group” often have additional insight and are critical for operationalizing programs. The next critical piece is that these groups need work on this work for a prolonged period of time measured in years and not months. The problem with temporary “fly-in” solutions is that they “fly-out.” If you have to live with a solution for five years or more, you start thinking differently. I think if we started to think (and fund) along these lines, we might have better results. Rather than trying to develop the next best thing, we could implement and integrate the current best thing, well, with the right people and dedicated resources for the length of time required. When we’re looking for universal solution and the next best algorithm for the next paper or grant proposal, I feel like we are setting ourselves up for a sub-optimum solution.

Update 2022-11-09

As is always the case in further discussion on Mastodon, two key points were brought up by Sam and Noam. Firstly, there is a question about the tooling and said democratisation. In order to have surveillance at the lowest levels, the tools have to be “translated” from research to practice with knowledge diffused through the organization. This is often a key in translational science and is needed for epidemiological practice. I think that this work likely needs to be at the intersection of the “clinical providers” and the “researchers” just as it is with clinical medicine and translation from the bench. Noam brought up university extensions as a place for this transitional work which is really interesting. Much of these relationships exist for agricultural works, but less for everything else. As Noam mentions there aren’t clear linkages with public health in the way that there is for ag, but it is at least a model. Love this discussion and I hope to continue it!

Citation

BibTex citation:

@online{dewitt2022
author = {Michael E. DeWitt},
title = {Designing Public Health Surveillance},
date = 2022-11-06,
url = {https://michaeldewittjr.com/articles/2022-11-06-designing-systems},
langid = {en}
}

For attribution, please cite this work as:

Michael E. DeWitt. 2022. "Designing Public Health Surveillance." November 6, 2022. https://michaeldewittjr.com/articles/2022-11-06-designing-systems