Chris von Csefalvay is a computational epidemiologist/data scientist working at the intersection of AI, epidemiology and public health.

Chris von Csefalvay

The year is 1959. Eisenhower is on his second term, Castro just kicked Batista out of the country and Ray Charles’s Let the Good Times Roll is topping the charts.

The 95% myth

Over at the work blog, I’m discussing what knowledge means for large language models (LLMs), and the ways in which we can leverage this knowledge dividend for better inference.  Read the full post here.

The knowledge dividend of large language models

The awesome thing about language is that, well, we all mostly speak it, to some extent or another. This gives us an immensely powerful tool to manipulate transformational tasks. For the purposes of this post, I consider a transformational task to be essentially anything that takes an input and is largel intended to return some version of the same thing. This is not a very precise definition, but it will have to do for now.

LAIR - Language As Intermediate Representation

As the new Omicron variant of concern of COVID-19 emerged over Thanksgiving week (of all times!), attention has been turning to Paxlovid (PF-07321332). If, like me, you torment yourself by perusing conspiracy theories, you might have heard it referred to as Pfizermectin.

Peace, love, Paxlovid and ‘Pfizermectin’

Tip  If you’re here for the DOI matching script, the link is here. Quarto is a great tool for reproducible research. It is also a great tool for building websites. After about a decade of Wordpress-based websites, I’ve moved to Quarto primarily for two reasons. First, most of my content is largely static.

Quarto project scripts are awesomeness

There’s a notion in artificial intelligence known as Moravec’s paradox: it’s relatively easy to teach a computer to play chess or checkers at a pretty decent level, but near impossible to teach it something as trivial as bipedal motion. (Hassabis 2017) The sensorimotor tasks that our truly wonderful brains have mastered by our second birthday are much harder to teach a computer than something arguably as ‘complex’ as beating a chess grandmaster.

The hardest AI problem you’ve never heard of.

It appears that in what is clearly a wonderful little PR stunt, a Polish rum company managed to do a Sophia and appoint an ‘AI-driven’ ‘robot’ as its ‘CEO’.  The other guilty party to this pile of steaming bovine excrement is Hanson Robotics, famous for giving us Sophia, the “world’s first robot citizen”. Most of what I’m saying here goes just as well for Sophia.

The skeuomorphic fallacy

Yesterday, I completed a marathon on the SkiErg, with a pretty respectable time (see results here). I know I can do better, and I know I

 will

do better. On the other hand, this was a marathon where I pretty much did everything wrong. And that was sort of an instructive experience in its own right. The title is somewhat misleading. Not much went wrong – the marathon itself was smooth sailing.

How to do a SkiErg marathon entirely the wrong way (but still finish)

There’s something special about language. It is ‘our own’, it is ‘us’, in a profound way, and quite surprisingly, more so than art. I was deeply struck by this when I first saw reactions to large generative language models that created realistic, human-ish prose.

Beyond Broca

There might be valid concerns about the mRNA vaccines. Then there are the clearly insane ones, such as the claims that it involves the injection of a microchip. The argument that mRNA vaccines are a form of ‘gene therapy’ are somewhere in the middle, and that makes responding to it somewhat difficult.

Why the COVID-19 vaccines are not gene therapy

The temple of Asclepios, the Greek god of healing arts and medicine, at Epidaurus was pretty much the ancient Greek world’s equivalent of the Mayo Clinic. It then tells us a lot about the Greek worldview of healing that one of the things the temple complex prominently featured was a theatre. The Greeks believed in drama therapy, but in perhaps a slightly different way of how we think of it today.

The Moral Pulse of the Machine

Every year, the Commandant of the Marine Corps publishes a reading list of books that often only bear on warfighting tangentially at best. The idea behind this is that those entrusted with the lives of servicemembers should have an understanding of the world that goes beyond the profession of arms. In much the same way, I have been advising data scientists to go beyond professional literature.

Five non-data-science books for data scientists

I used to be on Quora. I’m not anymore. But I used to be. And I wrote some answers. Here are some of my favourites, categorised by field… sort of.

 AI

If I wrote an AI and trained it to score 200 on a standardized IQ test, what could a human of 100 IQ still do better? Read the full answer here. Will true AI come before the ability to completely simulate a human brain? Is there a difference? Read the full answer here.

My favourite Quora answers

In my latest post for the Starschema blog, I discuss the end of dashboards, and what comes next:  Read the full post here.

As we may see: the world after dashboards

In the middle of the green and pleasant countryside of England lies a small piece of commons –– land, that is, over which a number of people share certain rights. In this hypothetical scenario, it’s six farmers,  to . Each of them has a hypothetical cow, which differs from ordinary cows in that it grazes a fixed unit, and produces a fixed amount of milk (unlike real cows, which vary wildly in their input and output parameters, or so I’m told).

Ransomware attacks as n-person prisoner’s dilemmas

Say you’re busing tables and you’re trying to pass someone in a wheelchair. What do you do? Do you say “excuse me” and wait for them to move? Do you say “excuse me” and then try to pass them? Do you just try to pass them? Do you say nothing and just try to pass them? All of these are, actually, pretty legitimate answers. Now, say you’re a robot.

What I learned from getting bodied by a robot.

Asemantic Induction of Hallucinations in LLMs

There’s a style of visual design I’m inordinately fond of called Raygun Gothic. It’s hard to describe what the hell exactly one needs to be on to enjoy it, but think of it like the aesthetic from the latter Fallout games with a more optimistic outlook on the future. Gibson described it as “the future that never was”, and I think that’s a pretty apt description. We get these competing futures in technology all the time.

Prompt Engineering: The Art of Yesterday

It’s not every day that you find out you have climbed the exalted heights of another discipline. My work is pretty interdisciplinary, but it shocked me, too, that I’m apparently holding forth on neoliberalism and the epistemic question in African universities (archive link):  This, of course, came at some surprise to me, as I have never written anything on the topic.

Stochastic parrots, cap and gown edition

In the first four entries (1 2 3 4) of this sequence, I have focused primarily on what LLMs aren’t, can’t, won’t, wouldn’t and shouldn’t. It’s probably time to conclude this series by that much awaited moment in all stories, where the darkest night finally turns into a glorious dawn, where we finally arrive at the promised land, where we finally get to talk about what LLMs

 could

be. What I see as the most successful potential model of

Teams of Rivals

My recent post on the Starschema blog discusses the need for better data products to tackle future pandemic challenges:  Read the full post here.

Data for the next pandemic

I have probably spent more time looking at Poussin’s

 Dance to the Music of Time

than any other work of art. Sneaking off to the Wallace Collection in London and just looking at the

 Dance

was my comfort activity while living in London – a time that was not exactly devoid of its trials. It’s not, by any measure,

 great

art, insofar as such judgments can be made with any objectivity.

The Lyre of Hephaestus

In a recent paper that has attracted the interest of popular media as well, Fabio Urbina and colleagues examined the use (or rather, the abuse) of computational chemistry models of toxicity for generating toxic compounds and potential chemical agent candidates.(Urbina et al. 2022) Urbina and colleagues conclude that Urbina, Fabio, Filippa Lentzos, Cédric Invernizzi, and Sean Ekins. 2022.

A different shade of grey

I love posts that allow me to merge some of my addictions.

Auto-DOI for Quarto posts via Rogue Scholar

In a recent paper that has attracted the interest of popular media as
well, Fabio Urbina and colleagues examined the use (or rather, the
abuse) of computational chemistry models of toxicity for generating
toxic compounds and potential chemical agent candidates.[(Urbina et al.
2022)]{.citation cites="urbina2022dual"} Urbina and colleagues conclude
that

::: {.no-row-height .column-margin .column-container}
::: {#ref-urbina2022dual .csl-entry}
Urbina, Fabio, Filippa Lentzos, Cédric Invernizzi, and Sean Ekins. 2022.
'Dual Use of Artificial-Intelligence-Powered Drug Discovery'. *Nature
Machine Intelligence* 4 (3): 189--91.
:::
:::

> By going as close as we dared, we have still crossed a grey moral
> boundary, demonstrating that it is possible to design virtual
> potential toxic molecules without much in the way of effort, time or
> computational resources.

I agree with the conclusion, but for rather different reasons.

::: {#background .section .level1 .page-columns .page-full}
# Background

Computational chemistry is the branch of computational science that
focuses on applications in the chemical field. This includes
pharmacology and rational drug design (RDD) in particular. The purpose
of RDD is to generate drug candidates that show favourable indicators of
effectiveness (such as high binding affinity to a target protein) along
with indicators of biological suitability (such as no or low
interference with other proteins, low toxicity and no inhibition of
metabolic 'bottlenecks' like CYP3A4). The latter part is typically
handled by a toxicity model.

Rather simply put, a toxicity model infers the structural associations
(the chemical structures associated with undesirable effects) from a
library of known compounds with known effects. For instance, the
Toxicology in the 21st Century (Tox21) programme of the US federal
government has performed over sixty different assays (typically, enzyme
inhibition assays) for over 13,000 different compounds. [(Tice et al.
2013)]{.citation cites="tice2013improving"} Using molecular
fingerprinting, which we have discussed in a previous post on this blog
in this very same context, it is possible to build relatively easy
models for toxicity. Where a particular desired toxicity is known, say
mitochondrial toxicity, it's not difficult to build a pipeline that
generates candidate compounds, derives the molecular fingerprint and
evaluates the likelihood that the molecule that is obtained will be an
effective agent. In this sense, I wholeheartedly agree with Urbina: the
cat is very much out of the bag. Even if Tox21's public data does not
include the classical target of modern chemical weapons
(acetylcholinesterase), such data is not exactly hard to come by or, for
a nation-state actor, to generate. A near-peer adversary could create
such assays for cents on the compound.

::: {.no-row-height .column-margin .column-container}
::: {#ref-tice2013improving .csl-entry}
Tice, Raymond R, Christopher P Austin, Robert J Kavlock, and John R
Bucher. 2013. 'Improving the Human Hazard Characterization of Chemicals:
A Tox21 Update'. *Environmental Health Perspectives* 121 (7): 756--65.
:::
:::

Nothing about the above is controversial. In fact, Urbina's paper is an
example of 'trivial genius': just about anyone who has ever done
computational chemistry in the pharmacological/drug design space knows
that any algorithm that is intended to optimise towards lower toxicity
can be inverted to optimise towards higher toxicity, and the same models
used to create effective enzyme inhibitors to treat cancer, depression,
schizophrenia, allergies or autoimmune disease can be repurposed in a
few hours and about \$100 in AWS credits to something that will generate
potent acetylcholinesterase inhibitors (AChEIs). Notably, this is not to
say that all research aimed at acetylcholinesterase inhibition is aimed
at creating a chemical warfare agent. AChEIs are used in a clinical
context, e.g. for myasthenia gravis. They are, however, also the
archetypal "nerve agent". Which leads me to my second point of agreement
with Urbina et al.: the tools of computational pharmacology and RDD are
--- and have been, for a long time! --- open to misuse.
:::

::: {#the-reverse-of-the-medal .section .level1}
# The reverse of the medal

On the other hand, the likelihood of an AI-generated chemical agent ever
posing a threat beyond the theoretical is very, very low. There are a
few reasons for that, and they're inherent partly in computational
chemistry, partly in weapons design.

The computational chemistry part pertains to the fact that molecular
fingerprinting and similar models only give us a narrow view of the
outcomes we may expect. For starters, no model is able to reliably
assess the feasibility and cost-effectiveness of synthesis. There are
plenty of drug candidates that have performed admirably in vitro and
sometimes even in clinical trials, but for which no feasible way of
cost-efficient, large-scale synthesis could be found. Then, there are
the drugs that ought to work, and might even work in vitro, but end up
failing in clinical trials with no effect or an unexpected toxicity.
Effect inference from chemical structure looks only at one side of the
medal, and not even all of that.

The bigger problem is the weapons design part. To avoid a late-night
visit from some mild-mannered federal employees in a dark SUV, I'd like
to point out that anything I discuss here is well in the public domain.
With that said: just as pharmaceutical chemists want some things from
their target compounds (such as relatively little inference, predictable
metabolism, a wide therapeutic margin and few adverse effects),
designers of chemical weapons have their own considerations for which to
optimise. VX, for instance, is immensely popular because its oily
consistency gives it beneficial physical properties. Similarly, a
potential chemical weapon candidate must be stable vis-a-vis e.g. UV
exposure, but not too stable. An example of the latter is the Red Zone
in France, the World War I battlefields that have been bombarded with so
many chemical weapons that to this day, they are heavily contaminated by
arsenic, among others. The preference for binary agents (which contain
two relatively stable and relatively non-toxic chemicals that are mixed,
typically during the flight time of a shell, to form the active agent)
means that a less toxic agent that can be reliably produced from the
simple admixture of two relatively stable agents may be preferred to a
more lethal unary agent. And this, of course, all assumes a state actor
willing and able to violate international law on chemical weapons.

Finally, there is no real need for novel chemical agents, at least not
in the nerve agent category. Not only does using a known agent provide
plausible deniability, there is also no real need to create anything
more lethal than VX. Even relatively old chemical agents, such as
mustard agents, are effective enough. A novel chemical structure may not
guarantee that the agent escapes chemical detection, and functional
antidotes are going to be just as effective against novel agents. To an
atropine/pralidoxime NAAK autoinjector's efficacy, it makes no
difference whether the acetylcholinesterase inhibition comes from sarin,
VX, Tetram or inadvertent exposure to organophosphate pesticides.
Arguably, this becomes somewhat more complex with other agents, where
the biological targets --- and hence the antidotes --- are more
specific. Nevertheless, it is hard to conceive of anyone possessed of a
burning rationale to start creating novel chemical agents.
:::

::: {#a-lighter-shade-of-grey .section .level1}
# A lighter shade of grey

I commend the authors for discussing the moral aspects of this exercise.
It is rather uncomfortable to write about the use of artificial
intelligence to create tools whose predominant use would be to
extinguish human lives (although, as noted, many of these compounds can,
and often do, have a medicinal use).

Where I cannot agree with the authors is the conclusion that this
situation calls for regulation (be it self-regulation or imposed from
above).

> Although MegaSyn is a commercial product and thus we have control over
> who has access to it, going forward, we will implement restrictions or
> an API for any forward-facing models. A reporting structure or hotline
> to authorities, for use if there is a lapse or if we become aware of
> anyone working on developing toxic molecules for non-therapeutic uses,
> may also be valuable. Finally, universities should redouble their
> efforts toward the ethical training of science students and broaden
> the scope to other disciplines, and particularly to computing
> students, so that they are aware of the potential for misuse of AI
> from an early stage of their career, as well as understanding the
> potential for broader impact.

This is all well and good, but --- assuming that there were a realistic
danger of people coming to grief from AI-generated chemical weapons ---
it solves a problem that the authors have failed to substantiate exists.
The tools to do this have been around for a long time. For the reasons
laid out in the previous section, there is no burning desire anywhere in
the world right now, as far as I can see, to develop a successor to VX
purely on a structural basis. Of course, a chemical agent that can be
synthesised without relying on any OPCW listed substances, or which can
have novel effects, or which can escape traditional methods of detection
(GC/MS, typically), would be of interest to potential bad actors, but
current models of computational chemistry do not help with that.
Creating permutational or functional analogues of VX does not,
realistically, put anyone a single step closer to the ability to carry
out an atrocity using such reprehensible weapons.

On the other hand, I am concerned about the moral aspects that derive
from the consequences of Urbina et al.'s paper. Given that the
overwhelming majority of users of computational chemistry and RDD do so
for benign purposes, the public attention garnered by such articles may
create a regulatory push that is not going to make anyone safer, but
will impede scientific inquiry. Urbina et al. point out the reputational
risk, and the risk of a single bad apple spoiling the lot --- I am
rather uncertain whether an attention-grabbing headline in The Verge,
raising the spectre of the scary AI that generates tens of thousands of
killer compounds in hours (virtually all of which are very likely to be
practically useless), is doing our discipline any favours.

The Scythian prince Anacharsis likened laws to cobwebs: strong enough to
catch the weak, but too weak to impede the powerful. In that sense, a
sufficiently determined adversary with the right tools --- scientists,
labs, an AWS account with a few hundred bucks of credit --- will be able
to subvert the art of creating chemistry to save lives and turn it into
a way to destroy them. No amount of regulation, controlled APIs and
lectures on the Hague Ethical Guidelines are going to be any impediment.
To a near-peer adversary or even a well-funded non-state actor, the door
has been open for rational chemical agent discovery (RCAD) for a very
long time. If the past is anything to go by, it has not really yielded
rich fruit. Pretty much the only new thing under the sun for chemical
agents in recent decades was the (possible, theorised) use of binary
agents on Kim Jong-nam --- not exactly a result of assiduous, AI-driven
research into novel agents, but rather the every-day fare of a paranoid,
despotic regime driven by cruelty and ignorance.

The only thing to fear, then, is fear itself. There is a justified
concern in the AI community that while we do indeed need to discuss
ethical use of artificial intelligence in the RDD domain, there is a
time and a place for that. Sensationalism and alarmist headlines of
poison-spewing machine learning models are great clickbait, but they do
not benefit the discipline. Realism, not alarmism, is needed to tackle
these issues.
:::

::: {#quarto-appendix .default}
::: {.section .quarto-appendix-contents}
## Citation {#citation .anchored .quarto-appendix-heading}

<div>

::: {.quarto-appendix-secondary-label}
BibTeX citation:
:::

``` {.sourceCode .code-with-copy .quarto-appendix-bibtex}
@misc{csefalvay2022,
 author = {Chris von Csefalvay},
 title = {A Different Shade of Grey},
 date = {2022-04-13},
 url = {https://chrisvoncsefalvay.com/posts/a-different-shade-of-grey},
 doi = {10.59350/bwee1-hee31},
 langid = {en-GB}
}
```

::: {.quarto-appendix-secondary-label}
For attribution, please cite this work as:
:::

::: {#ref-csefalvay2022 .csl-entry .quarto-appendix-citeas}
Chris von Csefalvay. 2022. "A Different Shade of Grey."
<https://doi.org/10.59350/bwee1-hee31>.
:::

</div>
:::
:::