Chris von Csefalvay is a computational epidemiologist/data scientist working at the intersection of AI, epidemiology and public health.

Chris von Csefalvay

The year is 1959. Eisenhower is on his second term, Castro just kicked Batista out of the country and Ray Charles’s Let the Good Times Roll is topping the charts.

The 95% myth

Over at the work blog, I’m discussing what knowledge means for large language models (LLMs), and the ways in which we can leverage this knowledge dividend for better inference.  Read the full post here.

The knowledge dividend of large language models

The awesome thing about language is that, well, we all mostly speak it, to some extent or another. This gives us an immensely powerful tool to manipulate transformational tasks. For the purposes of this post, I consider a transformational task to be essentially anything that takes an input and is largel intended to return some version of the same thing. This is not a very precise definition, but it will have to do for now.

LAIR - Language As Intermediate Representation

As the new Omicron variant of concern of COVID-19 emerged over Thanksgiving week (of all times!), attention has been turning to Paxlovid (PF-07321332). If, like me, you torment yourself by perusing conspiracy theories, you might have heard it referred to as Pfizermectin.

Peace, love, Paxlovid and ‘Pfizermectin’

Tip  If you’re here for the DOI matching script, the link is here. Quarto is a great tool for reproducible research. It is also a great tool for building websites. After about a decade of Wordpress-based websites, I’ve moved to Quarto primarily for two reasons. First, most of my content is largely static.

Quarto project scripts are awesomeness

There’s a notion in artificial intelligence known as Moravec’s paradox: it’s relatively easy to teach a computer to play chess or checkers at a pretty decent level, but near impossible to teach it something as trivial as bipedal motion. (Hassabis 2017) The sensorimotor tasks that our truly wonderful brains have mastered by our second birthday are much harder to teach a computer than something arguably as ‘complex’ as beating a chess grandmaster.

The hardest AI problem you’ve never heard of.

It appears that in what is clearly a wonderful little PR stunt, a Polish rum company managed to do a Sophia and appoint an ‘AI-driven’ ‘robot’ as its ‘CEO’.  The other guilty party to this pile of steaming bovine excrement is Hanson Robotics, famous for giving us Sophia, the “world’s first robot citizen”. Most of what I’m saying here goes just as well for Sophia.

The skeuomorphic fallacy

Yesterday, I completed a marathon on the SkiErg, with a pretty respectable time (see results here). I know I can do better, and I know I
<em>
 will
</em>
do better. On the other hand, this was a marathon where I pretty much did everything wrong. And that was sort of an instructive experience in its own right. The title is somewhat misleading. Not much went wrong – the marathon itself was smooth sailing.

How to do a SkiErg marathon entirely the wrong way (but still finish)

There’s something special about language. It is ‘our own’, it is ‘us’, in a profound way, and quite surprisingly, more so than art. I was deeply struck by this when I first saw reactions to large generative language models that created realistic, human-ish prose.

Beyond Broca

There might be valid concerns about the mRNA vaccines. Then there are the clearly insane ones, such as the claims that it involves the injection of a microchip. The argument that mRNA vaccines are a form of ‘gene therapy’ are somewhere in the middle, and that makes responding to it somewhat difficult.

Why the COVID-19 vaccines are not gene therapy

The temple of Asclepios, the Greek god of healing arts and medicine, at Epidaurus was pretty much the ancient Greek world’s equivalent of the Mayo Clinic. It then tells us a lot about the Greek worldview of healing that one of the things the temple complex prominently featured was a theatre. The Greeks believed in drama therapy, but in perhaps a slightly different way of how we think of it today.

The Moral Pulse of the Machine

Every year, the Commandant of the Marine Corps publishes a reading list of books that often only bear on warfighting tangentially at best. The idea behind this is that those entrusted with the lives of servicemembers should have an understanding of the world that goes beyond the profession of arms. In much the same way, I have been advising data scientists to go beyond professional literature.

Five non-data-science books for data scientists

I used to be on Quora. I’m not anymore. But I used to be. And I wrote some answers. Here are some of my favourites, categorised by field… sort of.
<strong>
 AI
</strong>
If I wrote an AI and trained it to score 200 on a standardized IQ test, what could a human of 100 IQ still do better?   Read the full answer here. Will true AI come before the ability to completely simulate a human brain? Is there a difference?   Read the full answer here.

My favourite Quora answers

In my latest post for the Starschema blog, I discuss the end of dashboards, and what comes next:  Read the full post here.

As we may see: the world after dashboards

In the middle of the green and pleasant countryside of England lies a small piece of commons –– land, that is, over which a number of people share certain rights. In this hypothetical scenario, it’s six farmers,  to . Each of them has a hypothetical cow, which differs from ordinary cows in that it grazes a fixed unit, and produces a fixed amount of milk (unlike real cows, which vary wildly in their input and output parameters, or so I’m told).

Ransomware attacks as n-person prisoner’s dilemmas

Say you’re busing tables and you’re trying to pass someone in a wheelchair. What do you do? Do you say “excuse me” and wait for them to move? Do you say “excuse me” and then try to pass them? Do you just try to pass them? Do you say nothing and just try to pass them? All of these are, actually, pretty legitimate answers. Now, say you’re a robot.

What I learned from getting bodied by a robot.

Asemantic Induction of Hallucinations in LLMs

There’s a style of visual design I’m inordinately fond of called Raygun Gothic. It’s hard to describe what the hell exactly one needs to be on to enjoy it, but think of it like the aesthetic from the latter Fallout games with a more optimistic outlook on the future. Gibson described it as “the future that never was”, and I think that’s a pretty apt description. We get these competing futures in technology all the time.

Prompt Engineering: The Art of Yesterday

It’s not every day that you find out you have climbed the exalted heights of another discipline. My work is pretty interdisciplinary, but it shocked me, too, that I’m apparently holding forth on neoliberalism and the epistemic question in African universities (archive link):  This, of course, came at some surprise to me, as I have never written anything on the topic.

Stochastic parrots, cap and gown edition

In the first four entries (1 2 3 4) of this sequence, I have focused primarily on what LLMs aren’t, can’t, won’t, wouldn’t and shouldn’t. It’s probably time to conclude this series by that much awaited moment in all stories, where the darkest night finally turns into a glorious dawn, where we finally arrive at the promised land, where we finally get to talk about what LLMs
<em>
 could
</em>
be.  What I see as the most successful potential model of

Teams of Rivals

My recent post on the Starschema blog discusses the need for better data products to tackle future pandemic challenges:  Read the full post here.

Data for the next pandemic

I have probably spent more time looking at Poussin’s
<em>
 Dance to the Music of Time
</em>
than any other work of art. Sneaking off to the Wallace Collection in London and just looking at the
<em>
 Dance
</em>
was my comfort activity while living in London – a time that was not exactly devoid of its trials. It’s not, by any measure,
<em>
 great
</em>
art, insofar as such judgments can be made with any objectivity.

The Lyre of Hephaestus

In a recent paper that has attracted the interest of popular media as well, Fabio Urbina and colleagues examined the use (or rather, the abuse) of computational chemistry models of toxicity for generating toxic compounds and potential chemical agent candidates.(Urbina et al. 2022) Urbina and colleagues conclude that Urbina, Fabio, Filippa Lentzos, Cédric Invernizzi, and Sean Ekins. 2022.

A different shade of grey

I love posts that allow me to merge some of my addictions.

Auto-DOI for Quarto posts via Rogue Scholar

There's a style of visual design I'm inordinately fond of called Raygun
Gothic. It's hard to describe what the hell exactly one needs to be on
to enjoy it, but think of it like the aesthetic from the latter Fallout
games with a more optimistic outlook on the future. Gibson described it
as "the future that never was", and I think that's a pretty apt
description.

::: {.no-row-height .column-margin .column-container}
<div>

::: {.quarto-figure .quarto-figure-center}
<figure class="figure">
<p><img
src="https://chrisvoncsefalvay.com/posts/prompt-engineering/03880260005fa86e78ddb88e645bfcef.jpg"
class="img-fluid figure-img" /></p>
<figcaption>The future we were all promised, in all its Raygun Gothic
glory.</figcaption>
</figure>
:::

</div>
:::

We get these competing futures in technology all the time. Every idea is
a gateway to a possible future, and purely by logic it stands to reason
that most will never come to pass. But oh, does that make them no less
appealing when they first emerge. They rise like the sun, with abundant
promises. They place themselves in our hands, and we can't help but
imagine what they could be. And most of the time, we're left with what
Ronnie Janoff-Bulman so beautifully described as "a little sadder, and a
little wiser". This is not a bad thing. It's just the way things are in
an industry that's still really run by Schumpeterian waves of creative
destruction.

I think prompt engineering is one of those ideas. It's a beautiful idea,
and it's one that I've been thinking about for a while. But I think it's
also one that's been obsolete before it even took off. And like a love
affair that never went anywhere but gave one some temporary joy, we're
better for having had it, but we're also better off moving on.

::: {#prompting-engineering .section .level1 .page-columns .page-full}
# Prompting, engineering

The idea behind prompt engineering is really not all that shockingly
novel. It reminds me of something a friend of mine, who primarily works
in embedded programming, said: most of his work, he said, was about
writing code in a way to get the compiler to produce the kind of
assembly he wanted. In other words, he was writing code to get the
compiler to write code for him. Prompt engineering is the same thing,
but for language models. We've all done this -- all of us have done bits
of compiler hacking by trying to phrase things just the right way for
the compiler to give us what we wanted (because we were too lazy to
actually write the whole thing in assembly, but damn if we didn't want
it our own way).

And so, the idea of taming a complex model by gently perturbing inputs
to get us the outputs we desire is nothing novel. We've been doing this
all the time. What is somewhat novel is that we're 'tricking' a
stochastic system here. The stochasticity and high dimensionality of
LLMs together mean that the responses to queries exhibit the
nonlinearities we like to know and love in chaotic processes. In chaos
theory, we 'quantify' chaotic behaviour using the Lyapunov exponent,
which essentially is the degree to which infinitesimally close starting
conditions result in divergent trajectories. In other words, it's a
measure of how much a system is sensitive to initial conditions. And
LLMs are very sensitive to initial conditions -- they accumulate their
divergent trajectories over time as every token begets the next. This is
why we can get such wildly different outputs from them by just changing
a few words in the prompt.

::: {.no-row-height .column-margin .column-container}
<div>

![](https://latex.codecogs.com/png.latex?%5Clambda%20=%20%5Clim_%7Bt%20%5Cto%20%5Cinfty%7D%20%5Clim_%7B%7C%5Cdelta%20%5Cmathbf%7BZ%7D_0%7C%20%5Cto%200%7D%0A%5Cfrac%7B1%7D%7Bt%7D%20%5Cln%5Cfrac%7B%7C%20%5Cdelta%5Cmathbf%7BZ%7D(t)%7C%7D%7B%7C%5Cdelta%20%5Cmathbf%7BZ%7D_0%7C%7D)

The Lyapunov exponent, a measure of how minor (infinitesimal,
specifically, hence
![](https://latex.codecogs.com/png.latex?%5Cdelta%20%5Cmathbf%7BZ%7D_0%20%5Cto%200))
changes in initial conditions (the input) result in divergent
trajectories (the output). The Lyapunov exponent is a measure of how
sensitive a system is to minor differences in initial conditions.

</div>
:::

But this is also why prompt engineering is so hard. It's hard because
we're trying to tame a chaotic system. And it's hard because we're
trying to tame a chaotic system that's also a black box. And we don't
exactly have a strong evidence-based methodology for this. We don't have
a rigorous way of testing whether our perturbations actually create the
expected effect. In short, we are allegedly 'engineering' our prompts
for a better outcome, when in reality, we have precious few methods for
measuring if we actually get that better outcome. Some of the leading
papers, such as those on tree-of-thought prompting, have rigorous
methods for measuring the effect of prompts, but these are still very
much in their infancy. And so, we're left with a lot of guesswork, and a
lot of trial and error. And that's not engineering. That's just hacking.
And that's fine, but it's not engineering.
:::

::: {#the-art-of-yesterday .section .level1 .page-columns .page-full}
# The art of yesterday

I think the reason why prompt engineering has been obsolete before it
even took off is because it's a solution to a problem that's already
been solved -- by agents and chains.

::: {.no-row-height .column-margin .column-container}
<div>

**Chains** are sequences of queries that proceed in a largely
deterministic manner.

**Agents** are task-driven semi-autonomous entities to perform tasks.
They can operate on their own, making decisions based on their training
and the input they receive. They are focused on reasoning and
decision-making by using the LLM to determine what actions to take next.

</div>
:::

Don't get me wrong -- good prompts are still going to be important. But
it's no longer a one-shot game. The prompt is no longer all that stands
between you and the output you are looking for: there are other prompts,
there are decisions, there is a complex ecosystem of processes,
supervisors and determinators that govern the outcome.

Which makes prompt engineering largely obsolete the way ensemble models
make building the perfect regression model obsolete. No amount of
expertly hand-crafted regression models will ever beat an ensemble of
mediocre at best models. The booster always wins. Clever aggregation of
weak primitives will always beat a single strong solution. And so,
prompt engineering is obsolete because it's a single strong solution in
a world where we can relatively inexpensively generate a lot of weak
solutions that can be aggregated to produce a better outcome.

This makes prompt engineering not the dream job of tomorrow but the art
of yesterday. It's a beautiful art, and it's one that I'm sure will
continue to be practiced. But it's not the future. It's the past. The
future lies in chains and agents.
:::

::: {#bardo .section .level1 .page-columns .page-full}
# Bardo

I think we're in a bardo right now, a liminal state between the old and
the new. The old is the world of prompt engineering, of one-shot
solutions, of the single strong solution. The new is the world of chains
and agents, of weak solutions that are aggregated to produce a better
outcome. Solutions like chain-of-thought prompting occupy this middle
space. They are not fully agentic, but they are beyond simple prompts
seeking to create strong solutions, and closer to building chains and
agentic architectures of weak solutions.

::: {.no-row-height .column-margin .column-container}
<div>

བར་དོ་ (Wylie: *bar do*) is a Tibetan word that describes a liminal state
between life, death and rebirth.

</div>
:::

Prompt engineering is in many ways a human brute force version of
backpropagation: it's a weak experiential form of tweaking inputs to get
a desired kind of output, with the backpropagation algorithm beign
replaced by "hm, this didn't work, let's run something else up the
flagpole". This is not an efficient use of human resources, the human
intellect or the human spirit. But it's all we had before the agentic
shift that we're currently experiencing, and so it attained its spot in
the sun before more considered, better approaches could take its place.
Prompt engineering was a short-lived future that could have been, but it
is not *the* future that *will* be. Everywhere where innovation in this
field happens, nobody is talking about prompt engineering anymore. We're
moving past it as we moved past crawling on all fours (at least on
weekday nights) in our cognitive development towards walking and
running.

Trying to hold on to prompt engineering at this point is holding on to
the past for its own sake. The only parties who have an interest in this
clinging are those who have made economic investments in an intermediate
stage technology and are unwilling to consider those the cost of doing
business and move on. There is, maybe, a future in which the
evolutionary destiny of LLMs would have focused on better singular
solutions. This one isn't that. This world, for better or worse, is
governed by connectionists. The most successful epistemic endeavour in
the realm of machine learning -- deep learning -- is fundamentally
connnectionistic, and relies on the aggregation of weak learners to
produce a strong learner. And so, the future of LLMs is not in prompt
engineering, but in the aggregation of weak solutions. And that's a
future I'm looking forward to.
:::

::: {#quarto-appendix .default}
::: {.section .quarto-appendix-contents}
## Citation {#citation .anchored .quarto-appendix-heading}

<div>

::: {.quarto-appendix-secondary-label}
BibTeX citation:
:::

``` {.sourceCode .code-with-copy .quarto-appendix-bibtex}
@misc{csefalvay2023,
  author = {Chris von Csefalvay},
  title = {Prompt {Engineering:} {The} {Art} of {Yesterday}},
  date = {2023-10-29},
  url = {https://chrisvoncsefalvay.com/posts/prompt-engineering},
  doi = {10.59350/keztr-yx081},
  langid = {en-GB}
}
```

::: {.quarto-appendix-secondary-label}
For attribution, please cite this work as:
:::

::: {#ref-csefalvay2023 .csl-entry .quarto-appendix-citeas}
Chris von Csefalvay. 2023. "Prompt Engineering: The Art of Yesterday."
<https://doi.org/10.59350/keztr-yx081>.
:::

</div>
:::
:::
