This must have been a ton of work, thanks for this thorough writeup of current discussions in biosecurity. Thank you for the high praise of SecureBio, too! 

I still want to flag three possible misconceptions:

Early in the piece, on the question of motivation for pursuing bioweapons, you ask:

And why wouldn’t those actors opt for much simpler acts of violence that would roughly accomplish the same thing?

This is an objection we hear relatively frequently and it is somewhat missing the whole reason why the current longtermist-flavored scope-sensitive biosecurity that you discuss in your article emerged—global catastrophic risks. If you truly want to cause catastrophic mass harm (from tens of millions of deaths to extinction), you basically only have nukes and pandemics available. Nukes are practically impossible to obtain, and while pandemic-capable viruses are definitely not trivially easy to make and work with, information and methods around them are close to open-source. Thus, if a malicious actor wants to cause global-scale harm, there aren’t any simpler acts of violence that roughly accomplish the same thing.

On the offense–defense balance of AIxBio progress:

AI tools will be just as useful in countermeasure design

I think this claim understates how much disagreement is around the offense–defense balance of biological tools. The landscape of biological tool capabilities seems to be pretty jagged and it’s not obvious to me that this results in balanced progress, especially given (as you correctly point out) the inherent offense-favor of pandemics. I think it would be a pretty big coincidence if a new technology happens to be exactly as useful for offense as it is for defense. Indeed, the consensus of my colleagues and me at SecureBio is that AI tools will accelerate bioweapon design + deployment well before they enable us to have countermeasure design + deployment that is sufficient to render us acceptably safe from such risks.

On weaponization:

Finally, I am not scared of individual actors, because the economics of bioweapons production likely do not work in their favor. Yes, they can rent upstream services—virus production, purification—but the downstream weaponization work requires custom protocols that CROs have no economic incentive to develop. Moreover, given that the weaponization will almost definitely be a bespoke, hands-on R&D project and not one that is easily automated, it feels unlikely that nobody at the CRO will raise an eyebrow.

It is a bit unclear to what extent the concept of “weaponization” applies to many of the pandemic threats many people in scope-sensitive biosecurity care about. Historically, weaponization was indeed a pretty big problem for certain BWs, which required specific conditions for storage in munitions, preparation for aerosolization, etc. But as we’ve seen with COVID, “weaponization” for many pandemic-capable pathogens might be essentially complete at the purification stage and only requires getting it onto somebody’s epithelium via a spray bottle. (And regarding the “economics of bioweapons” point, I’d re-emphasize my first point.)

That being said, I definitely agree with most of the things you wrote about biodefense and reasons for optimism. We have a ton of promising interventions that can form a pretty airtight swiss-cheese stack if we manage to implement them; biohardened environments, PPE stockpiles, and widespread early detection alone would make BWs very unattractive. Again, great to see such a detailed piece on biosecurity.

Great to see the preprint! Kevin Esvelt also explains the approach in his EAG Boston 2023 talk (starting around 20min): 

No wonder Bulby is unhappy; they're missing a good chunk of their head! 

Thanks for digging into the repo 

Does somebody have the image file of the sensitive, sentient lightbulb that visited the forum dropdown menu two days ago? 

That's really sad to hear! Given the number of events hosted there and the renting costs of alternative venues, it seemed like a solid investment (while also being much more picturesque, of course). So, I'm also hoping for a post-mortem with more details. 

I'm personally not that worried about substitution risks. Roughly: The deterrence aspect is strongest for low-resource threat actors and—from a tail-risk perspective—bio is probably the most dangerous thing they can utilize, with that pesky self-replication and whatnot. 

Hi Sean,

Thanks for the write-up and opening the discussion. I agree that material degradation is something that should be thoroughly investigated. Caveat: I'm not a polymer engineer and have read/seen a few papers/talks on the topic, but I'm by no means deeply familiar with all the material types, etc.

Re. the Boeing study (working link, btw): The study used fairly low doses per disinfection cycle but simulated 25 years of service, totalling >108 J/cm² of far-UV exposure. Still just about a third of the bus study exposure, but already in the dose range where the bus study measured only marginal effects of additional far-UV on colour or mechanical properties. 

Still, I don't find myself overly worried (but again, not an expert and I also don't have regulator-brain, so interpret this accordingly).

Far-UV doesn't penetrate deeply and will likely not affect the mechanical properties of solid plastic objects. The bus study found effects in thin sheets of fibre-reinforced materials (in some directions), and I don't doubt that, but in what situations is a 10% decrease in failure strength of thin polymer layers that relevant for consumers or regulators? I genuinely don't know, and there might be specific circumstances in which parts must be replaced more frequently, or the plastic type needs to be switched, but I don't think this will matter a lot in most settings. E.g., Boeing found "no adverse impact on the mechanical properties of thermoplastic and textile materials" in the airplane setting at the dose where the bus study already saw decreased tensile strength. I'm very interested in hearing counterexamples, though! 

But ultimately, both Boeing and the bus study have only tested the materials present in those surroundings. Looking around in my office or the office of friends, I'm not sure how many of the materials overlap. There's some testing behind closed doors from lamp manufacturers, but no public database of common materials and the impact of far-UV on them. We eventually want and need such a database to make the far-UV implementation as pain- and seamless as possible for building operators, etc., but I'm not sure how much the average office worker or regulator cares if the back cover of their monitors starts yellowing faster (like you said, open question: Market research opportunity!). 

And in the beachhead markets for far-UV (long-term care homes, ICUs), the cost–benefit calculus is favouring far-UV so much that a premium for far-UV coatings will happily be paid if yellowing is even something they care about. And the plethora of single-use plastics are not affected. 

That's likely an even smaller issue. Far-UV inactivates the transient microbiome on the upper skin surface but not in the pores where the majority of the bacteria live. It’s also strictly line of sight and the skin is pretty wrinkly and only small areas (e.g., back of the hands) are exposed and you constantly re-seed your microbiome from other parts of the body. There’s prelim data from hairless mice which found no changes to the microbiome.

Everything touched by the immune system is complicated, agree. But I strongly suspect the net balance will be highly positive, still:

a) As you wrote, seasonal respiratory disease is a massive problem, in the order of trillions of dollars of health and economic damage every year

b) There is some cross-reactivity, yes, but the tail events will be too different from what we usually have. Frequent common-cold exposure will not be the thing that protects us from a GCBR. Neither will the original antigenic sin play a role, as that usually requires quite closely related strains to matter. Overall, your immune system will be a bit less-well prepared for this particular pathogen family that you haven't encountered in a while, but that's also the case for any other sufficiently different pathogen.