Chris von Csefalvay is a computational epidemiologist/data scientist working at the intersection of AI, epidemiology and public health.

Chris von Csefalvay

The year is 1959. Eisenhower is on his second term, Castro just kicked Batista out of the country and Ray Charles’s Let the Good Times Roll is topping the charts.

The 95% myth

Over at the work blog, I’m discussing what knowledge means for large language models (LLMs), and the ways in which we can leverage this knowledge dividend for better inference.  Read the full post here.

The knowledge dividend of large language models

The awesome thing about language is that, well, we all mostly speak it, to some extent or another. This gives us an immensely powerful tool to manipulate transformational tasks. For the purposes of this post, I consider a transformational task to be essentially anything that takes an input and is largel intended to return some version of the same thing. This is not a very precise definition, but it will have to do for now.

LAIR - Language As Intermediate Representation

As the new Omicron variant of concern of COVID-19 emerged over Thanksgiving week (of all times!), attention has been turning to Paxlovid (PF-07321332). If, like me, you torment yourself by perusing conspiracy theories, you might have heard it referred to as Pfizermectin.

Peace, love, Paxlovid and ‘Pfizermectin’

Tip  If you’re here for the DOI matching script, the link is here. Quarto is a great tool for reproducible research. It is also a great tool for building websites. After about a decade of Wordpress-based websites, I’ve moved to Quarto primarily for two reasons. First, most of my content is largely static.

Quarto project scripts are awesomeness

There’s a notion in artificial intelligence known as Moravec’s paradox: it’s relatively easy to teach a computer to play chess or checkers at a pretty decent level, but near impossible to teach it something as trivial as bipedal motion. (Hassabis 2017) The sensorimotor tasks that our truly wonderful brains have mastered by our second birthday are much harder to teach a computer than something arguably as ‘complex’ as beating a chess grandmaster.

The hardest AI problem you’ve never heard of.

It appears that in what is clearly a wonderful little PR stunt, a Polish rum company managed to do a Sophia and appoint an ‘AI-driven’ ‘robot’ as its ‘CEO’.  The other guilty party to this pile of steaming bovine excrement is Hanson Robotics, famous for giving us Sophia, the “world’s first robot citizen”. Most of what I’m saying here goes just as well for Sophia.

The skeuomorphic fallacy

Yesterday, I completed a marathon on the SkiErg, with a pretty respectable time (see results here). I know I can do better, and I know I

 will

do better. On the other hand, this was a marathon where I pretty much did everything wrong. And that was sort of an instructive experience in its own right. The title is somewhat misleading. Not much went wrong – the marathon itself was smooth sailing.

How to do a SkiErg marathon entirely the wrong way (but still finish)

There’s something special about language. It is ‘our own’, it is ‘us’, in a profound way, and quite surprisingly, more so than art. I was deeply struck by this when I first saw reactions to large generative language models that created realistic, human-ish prose.

Beyond Broca

There might be valid concerns about the mRNA vaccines. Then there are the clearly insane ones, such as the claims that it involves the injection of a microchip. The argument that mRNA vaccines are a form of ‘gene therapy’ are somewhere in the middle, and that makes responding to it somewhat difficult.

Why the COVID-19 vaccines are not gene therapy

The temple of Asclepios, the Greek god of healing arts and medicine, at Epidaurus was pretty much the ancient Greek world’s equivalent of the Mayo Clinic. It then tells us a lot about the Greek worldview of healing that one of the things the temple complex prominently featured was a theatre. The Greeks believed in drama therapy, but in perhaps a slightly different way of how we think of it today.

The Moral Pulse of the Machine

Every year, the Commandant of the Marine Corps publishes a reading list of books that often only bear on warfighting tangentially at best. The idea behind this is that those entrusted with the lives of servicemembers should have an understanding of the world that goes beyond the profession of arms. In much the same way, I have been advising data scientists to go beyond professional literature.

Five non-data-science books for data scientists

I used to be on Quora. I’m not anymore. But I used to be. And I wrote some answers. Here are some of my favourites, categorised by field… sort of.

 AI

If I wrote an AI and trained it to score 200 on a standardized IQ test, what could a human of 100 IQ still do better? Read the full answer here. Will true AI come before the ability to completely simulate a human brain? Is there a difference? Read the full answer here.

My favourite Quora answers

In my latest post for the Starschema blog, I discuss the end of dashboards, and what comes next:  Read the full post here.

As we may see: the world after dashboards

In the middle of the green and pleasant countryside of England lies a small piece of commons –– land, that is, over which a number of people share certain rights. In this hypothetical scenario, it’s six farmers,  to . Each of them has a hypothetical cow, which differs from ordinary cows in that it grazes a fixed unit, and produces a fixed amount of milk (unlike real cows, which vary wildly in their input and output parameters, or so I’m told).

Ransomware attacks as n-person prisoner’s dilemmas

Say you’re busing tables and you’re trying to pass someone in a wheelchair. What do you do? Do you say “excuse me” and wait for them to move? Do you say “excuse me” and then try to pass them? Do you just try to pass them? Do you say nothing and just try to pass them? All of these are, actually, pretty legitimate answers. Now, say you’re a robot.

What I learned from getting bodied by a robot.

Asemantic Induction of Hallucinations in LLMs

There’s a style of visual design I’m inordinately fond of called Raygun Gothic. It’s hard to describe what the hell exactly one needs to be on to enjoy it, but think of it like the aesthetic from the latter Fallout games with a more optimistic outlook on the future. Gibson described it as “the future that never was”, and I think that’s a pretty apt description. We get these competing futures in technology all the time.

Prompt Engineering: The Art of Yesterday

It’s not every day that you find out you have climbed the exalted heights of another discipline. My work is pretty interdisciplinary, but it shocked me, too, that I’m apparently holding forth on neoliberalism and the epistemic question in African universities (archive link):  This, of course, came at some surprise to me, as I have never written anything on the topic.

Stochastic parrots, cap and gown edition

In the first four entries (1 2 3 4) of this sequence, I have focused primarily on what LLMs aren’t, can’t, won’t, wouldn’t and shouldn’t. It’s probably time to conclude this series by that much awaited moment in all stories, where the darkest night finally turns into a glorious dawn, where we finally arrive at the promised land, where we finally get to talk about what LLMs

 could

be. What I see as the most successful potential model of

Teams of Rivals

My recent post on the Starschema blog discusses the need for better data products to tackle future pandemic challenges:  Read the full post here.

Data for the next pandemic

I have probably spent more time looking at Poussin’s

 Dance to the Music of Time

than any other work of art. Sneaking off to the Wallace Collection in London and just looking at the

 Dance

was my comfort activity while living in London – a time that was not exactly devoid of its trials. It’s not, by any measure,

 great

art, insofar as such judgments can be made with any objectivity.

The Lyre of Hephaestus

In a recent paper that has attracted the interest of popular media as well, Fabio Urbina and colleagues examined the use (or rather, the abuse) of computational chemistry models of toxicity for generating toxic compounds and potential chemical agent candidates.(Urbina et al. 2022) Urbina and colleagues conclude that Urbina, Fabio, Filippa Lentzos, Cédric Invernizzi, and Sean Ekins. 2022.

A different shade of grey

I love posts that allow me to merge some of my addictions.

Auto-DOI for Quarto posts via Rogue Scholar

It's not every day that you find out you have climbed the exalted
heights of another discipline. My work is pretty interdisciplinary, but
it shocked me, too, that I'm apparently holding forth on neoliberalism
and the epistemic question in African universities ([archive
link](https://archive.is/MgSAW)):

::: {#fig-scispace .quarto-figure .quarto-figure-center .anchored}
<figure class="figure">
<img
src="https://chrisvoncsefalvay.com/posts/academic-generative-ai/scispace.png"
class="img-fluid figure-img" />
<figcaption>Figure 1: Apparently, I’m commenting on neoliberalism in
African universities.</figcaption>
</figure>
:::

This, of course, came at some surprise to me, as I have never written
anything on the topic. I have, however, written a lot about AI, and I
have written a thing or two about Africa, so I guess it was only a
matter of time before I was conflated with someone else. This time, the
unwitting victim deprived of his credit was [Prof. Amasa P.
Ndofirepi](https://www.uj.ac.za/members/prof-amasa-philip-ndofirepi/),
who is an educational studies scholar at the University of Johannesburg.
I have no idea how I ended up being credited with his work, but I'm sure
it was an honest mistake.

The problem is, with AI, mistakes compound. So if an unwitting student
were to ask for a quick literature review of neoliberalism on the
subject, they might get something like this from Scispace:

> The literature on the impact of neoliberalism on knowledge production
> and dissemination in African universities has been extensively
> explored by various authors. Qosimova Gulbahor, in her paper "Placing
> Knowledge at the Centre of an Alternative Public Good Imaginary of
> African Universities," discusses the alternative public good mission
> of African universities and the need for them to apply their knowledge
> infrastructure to community development challenges. Chris von
> Csefalvay, in his paper "The Hegemonic Neoliberal Knowledges in the
> African University," examines the pervasive presence of neoliberalism
> in African universities and explores the prospects and opportunities
> to unyoke the trapped knowledge processes. These authors, along with
> others, highlight concerns about the dominance of Western knowledge,
> the commodification of knowledge, and the need for African
> universities to prioritize socially-just knowledges that serve African
> priorities and challenges.

I mean, that's flattering, but I'd really rather be credited mostly for
my own work. I'm sure Prof. Ndofirepi would agree.

::: {#why-i-care .section .level2}
## Why I care {.anchored anchor-id="why-i-care"}

This is, of course, not good for academia. We've generally been coasting
from one crisis to another. We've got a replication crisis, there's
[enough dodgy Western Blots to blot out the
sun](https://retractionwatch.com/2020/09/30/author-says-misguided-efforts-for-the-ideal-western-blot-led-to-the-withdrawal-of-these-studies/),
we've got the [Tessier-Lavigne
mess](https://www.statnews.com/2023/07/19/marc-tessier-lavigne-stanford-president-resignation/),
and that's just what I can think of off the top of my head before my
first coffee. A predatory publishing industry doesn't help this at all.
We need another crisis on top of this like we need a hole in the head,
and yet, here we are.

Now, as far as I'm aware, no serious academic is actually using these
tools to do their research. On the other hand, non-academics *are*. For
journalists, in particular, such tools are a godsend -- literature
reviews are annoying, and if you can get a computer to do it for you,
why not? The problem is, of course, that you're supposed to double-check
this stuff and, well, journalists are known for many things, but
double-checking stuff properly isn't really one of them.

And so, after months of academics fretting about ChatGPT eating their
lunch, we're confronted with the actual problem. AI is not better at
producing decent science, but it is vastly faster and more efficient at
producing *bad* science.

Which we weren't short on to begin with.
:::

::: {#whats-the-problem .section .level2 .page-columns .page-full}
## What's the problem? {.anchored anchor-id="whats-the-problem"}

Language is a tool that works on the basis of some conventions of
meaning. Language models encapsulate these conventions, but they cannot
encapsulate *all* of them -- there are compromises to be made if a
system with limited resources has to contend with nearly unlimited human
imagination. When language models' limits come to blows with
domain-specific language, we get into trouble.

Language models are really weak at one thing: reasoned judgment. As a
scientist, you are trained to exercise this kind of reasoned judgment in
determining what is, and what isn't, worth considering as an authority.
That's why we make our master's students (and hopefully most
undergraduates) write literature reviews until the cows come home. It
teaches them to develop that judgment, and also to know how to explore
the fringes of their research question. I have looked at a few 'academic
AI' tools that claim to be doing some of this, and they're not very good
at it. [Scite](https://scite.ai) is so far one of the better ones, and
the literature reviews it produces are still pretty bad: results are
heavily weighted towards recent publications, towards the specific in
preference to the foundational and often towards meandering
misinterpretations of the research question as long as sources for that
could be found, in preference to actually identifying a gap.

To be quite fair towards these models, they have to deal with academic
literature, which is an abundance of noise with a flicker of signal.
There is, not to put too fine a point on it, a ton of crap out there,
and it's not always easy to tell the difference between the crap and the
good stuff. That's why we have peer review, and that's why we have
literature reviews. The problem is, of course, that these models are not
trained on the literature, but on the internet. And the internet is a
very different place from the academic literature indeed.

::: {.page-columns .page-full}
Academic writing, especially domain specific writing, has a language of
its own. It's not fair to expect a language model trained on English to
also master uses of English that might as well be a different language.
To give a favourite example of mine: in magnetic resonance imaging of
the brain, there's often talk of something called 'flow voids'. Now,
normal human reasoning would interpret a 'flow void' to be the absence
of flow, or something along those lines. In MRI, a flow void is actually
the opposite: it is a 'void' of signal created in a vessel through which
something (usually blood, sometimes CSF) flows.^1^ This is a very
specific term that has a very specific meaning in a very specific
context. A language model, however, would not know that. It would assume
that a 'flow void' is (de)void of flow.

::: {.no-row-height .column-margin .column-container}
^1^ Flow voids happen in the context of spin-echo imaging. These
modalities involve two pulses -- an excitation pulse and a refocusing
pulse. Blood that moves perpendicular to the image plane will be hit by
the excitation pulse but not the refocusing pulse. Therefore, it will
not create a signal, which gives us the 'void' appearance of signal
hypointense vessels.
:::
:::

Just about *all* of science is like that. We have a language of our own,
and it's not always easy to understand. What definitely doesn't make it
easier to work with, however, is when the source material is also wrong.
Which is what we're dealing with all too often, viz. Figure 1.
:::

::: {#what-can-we-do-about-it .section .level2}
## What can we do about it? {.anchored anchor-id="what-can-we-do-about-it"}

Most 'academic AI' applications are riding on the crest of a wave of
high expectations that surround everything AI-related right now. They
offer to be useful aides-de-camp to beleaguered academics who have to
contend with exponentially growing literature, but in reality fall far
short of that promise. And the inherent 'black box' nature of such
models means that it's not always easy to tell when they're wrong.

At this point, perhaps the best we can do is to hold off on using
generative AI tools for academic research until they're better. We're
not there yet. We're not even close. For highly domain specific
applications, retrieval-augmented generation (RAG) approaches utilising
a curated knowledge base of publications in that realm has proven to be
a very useful approach indeed, but those are specialised tools that are
at the present primarily in the purview of private industry. I've seen
some great applications in this field, and I see this as another proof
point for [my assertion that the future belongs to ecosystems of small,
specialised language models rather than one big model that does
everything](../team-of-rivals/).

For the time being, we're going to have to do things the old-fashioned
way: reading the literature and writing our own reviews. I know, it's
not very fun. But it's the only way to do it right, and we've managed
with that for the last few hundred years. Giving it another few years
won't hurt.
:::

::: {#quarto-appendix .default}
::: {.section .quarto-appendix-contents}
## Citation {#citation .anchored .quarto-appendix-heading}

<div>

::: {.quarto-appendix-secondary-label}
BibTeX citation:
:::

``` {.sourceCode .code-with-copy .quarto-appendix-bibtex}
@misc{csefalvay2023,
 author = {Chris von Csefalvay},
 title = {Stochastic Parrots, Cap and Gown Edition},
 date = {2023-12-06},
 url = {https://chrisvoncsefalvay.com/posts/academic-generative-ai},
 doi = {10.59350/412q0-sbn89},
 langid = {en-GB}
}
```

::: {.quarto-appendix-secondary-label}
For attribution, please cite this work as:
:::

::: {#ref-csefalvay2023 .csl-entry .quarto-appendix-citeas}
Chris von Csefalvay. 2023. "Stochastic Parrots, Cap and Gown Edition."
<https://doi.org/10.59350/412q0-sbn89>.
:::

</div>
:::
:::