Chris von Csefalvay is a computational epidemiologist/data scientist working at the intersection of AI, epidemiology and public health.

Chris von Csefalvay

The year is 1959. Eisenhower is on his second term, Castro just kicked Batista out of the country and Ray Charles’s Let the Good Times Roll is topping the charts.

The 95% myth

Over at the work blog, I’m discussing what knowledge means for large language models (LLMs), and the ways in which we can leverage this knowledge dividend for better inference.  Read the full post here.

The knowledge dividend of large language models

The awesome thing about language is that, well, we all mostly speak it, to some extent or another. This gives us an immensely powerful tool to manipulate transformational tasks. For the purposes of this post, I consider a transformational task to be essentially anything that takes an input and is largel intended to return some version of the same thing. This is not a very precise definition, but it will have to do for now.

LAIR - Language As Intermediate Representation

As the new Omicron variant of concern of COVID-19 emerged over Thanksgiving week (of all times!), attention has been turning to Paxlovid (PF-07321332). If, like me, you torment yourself by perusing conspiracy theories, you might have heard it referred to as Pfizermectin.

Peace, love, Paxlovid and ‘Pfizermectin’

I used to be on Quora. I’m not anymore. But I used to be. And I wrote some answers. Here are some of my favourites, categorised by field… sort of.
<strong>
 AI
</strong>
If I wrote an AI and trained it to score 200 on a standardized IQ test, what could a human of 100 IQ still do better?   Read the full answer here. Will true AI come before the ability to completely simulate a human brain? Is there a difference?   Read the full answer here.

My favourite Quora answers

Tip  If you’re here for the DOI matching script, the link is here. Quarto is a great tool for reproducible research. It is also a great tool for building websites. After about a decade of Wordpress-based websites, I’ve moved to Quarto primarily for two reasons. First, most of my content is largely static.

Quarto project scripts are awesomeness

There’s a notion in artificial intelligence known as Moravec’s paradox: it’s relatively easy to teach a computer to play chess or checkers at a pretty decent level, but near impossible to teach it something as trivial as bipedal motion. (Hassabis 2017) The sensorimotor tasks that our truly wonderful brains have mastered by our second birthday are much harder to teach a computer than something arguably as ‘complex’ as beating a chess grandmaster.

The hardest AI problem you’ve never heard of.

It appears that in what is clearly a wonderful little PR stunt, a Polish rum company managed to do a Sophia and appoint an ‘AI-driven’ ‘robot’ as its ‘CEO’.  The other guilty party to this pile of steaming bovine excrement is Hanson Robotics, famous for giving us Sophia, the “world’s first robot citizen”. Most of what I’m saying here goes just as well for Sophia.

The skeuomorphic fallacy

Yesterday, I completed a marathon on the SkiErg, with a pretty respectable time (see results here). I know I can do better, and I know I
<em>
 will
</em>
do better. On the other hand, this was a marathon where I pretty much did everything wrong. And that was sort of an instructive experience in its own right. The title is somewhat misleading. Not much went wrong – the marathon itself was smooth sailing.

How to do a SkiErg marathon entirely the wrong way (but still finish)

There’s something special about language. It is ‘our own’, it is ‘us’, in a profound way, and quite surprisingly, more so than art. I was deeply struck by this when I first saw reactions to large generative language models that created realistic, human-ish prose.

Beyond Broca

In my latest post for the Starschema blog, I discuss the end of dashboards, and what comes next:  Read the full post here.

As we may see: the world after dashboards

There might be valid concerns about the mRNA vaccines. Then there are the clearly insane ones, such as the claims that it involves the injection of a microchip. The argument that mRNA vaccines are a form of ‘gene therapy’ are somewhere in the middle, and that makes responding to it somewhat difficult.

Why the COVID-19 vaccines are not gene therapy

The temple of Asclepios, the Greek god of healing arts and medicine, at Epidaurus was pretty much the ancient Greek world’s equivalent of the Mayo Clinic. It then tells us a lot about the Greek worldview of healing that one of the things the temple complex prominently featured was a theatre. The Greeks believed in drama therapy, but in perhaps a slightly different way of how we think of it today.

The Moral Pulse of the Machine

Every year, the Commandant of the Marine Corps publishes a reading list of books that often only bear on warfighting tangentially at best. The idea behind this is that those entrusted with the lives of servicemembers should have an understanding of the world that goes beyond the profession of arms. In much the same way, I have been advising data scientists to go beyond professional literature.

Five non-data-science books for data scientists

In the middle of the green and pleasant countryside of England lies a small piece of commons –– land, that is, over which a number of people share certain rights. In this hypothetical scenario, it’s six farmers,  to . Each of them has a hypothetical cow, which differs from ordinary cows in that it grazes a fixed unit, and produces a fixed amount of milk (unlike real cows, which vary wildly in their input and output parameters, or so I’m told).

Ransomware attacks as n-person prisoner’s dilemmas

Say you’re busing tables and you’re trying to pass someone in a wheelchair. What do you do? Do you say “excuse me” and wait for them to move? Do you say “excuse me” and then try to pass them? Do you just try to pass them? Do you say nothing and just try to pass them? All of these are, actually, pretty legitimate answers. Now, say you’re a robot.

What I learned from getting bodied by a robot.

Asemantic Induction of Hallucinations in LLMs

There’s a style of visual design I’m inordinately fond of called Raygun Gothic. It’s hard to describe what the hell exactly one needs to be on to enjoy it, but think of it like the aesthetic from the latter Fallout games with a more optimistic outlook on the future. Gibson described it as “the future that never was”, and I think that’s a pretty apt description. We get these competing futures in technology all the time.

Prompt Engineering: The Art of Yesterday

It’s not every day that you find out you have climbed the exalted heights of another discipline. My work is pretty interdisciplinary, but it shocked me, too, that I’m apparently holding forth on neoliberalism and the epistemic question in African universities (archive link):  This, of course, came at some surprise to me, as I have never written anything on the topic.

Stochastic parrots, cap and gown edition

In the first four entries (1 2 3 4) of this sequence, I have focused primarily on what LLMs aren’t, can’t, won’t, wouldn’t and shouldn’t. It’s probably time to conclude this series by that much awaited moment in all stories, where the darkest night finally turns into a glorious dawn, where we finally arrive at the promised land, where we finally get to talk about what LLMs
<em>
 could
</em>
be.  What I see as the most successful potential model of

Teams of Rivals

My recent post on the Starschema blog discusses the need for better data products to tackle future pandemic challenges:  Read the full post here.

Data for the next pandemic

I have probably spent more time looking at Poussin’s
<em>
 Dance to the Music of Time
</em>
than any other work of art. Sneaking off to the Wallace Collection in London and just looking at the
<em>
 Dance
</em>
was my comfort activity while living in London – a time that was not exactly devoid of its trials. It’s not, by any measure,
<em>
 great
</em>
art, insofar as such judgments can be made with any objectivity.

The Lyre of Hephaestus

In a recent paper that has attracted the interest of popular media as well, Fabio Urbina and colleagues examined the use (or rather, the abuse) of computational chemistry models of toxicity for generating toxic compounds and potential chemical agent candidates.(Urbina et al. 2022) Urbina and colleagues conclude that Urbina, Fabio, Filippa Lentzos, Cédric Invernizzi, and Sean Ekins. 2022.

A different shade of grey

I love posts that allow me to merge some of my addictions.

Auto-DOI for Quarto posts via Rogue Scholar

I love posts that allow me to merge some of my addictions. In this case,
it's my love for Quarto project scripts (which I've [written about
elsewhere](../quarto-project-scripts)), my fondness for [Rogue
Scholar](https://rogue-scholar.org) and the overuse of the word 'mint'
to mean 'generally really quite rather nice'.

[Rogue Scholar](https://rogue-scholar.org) is a fantastic tool for
science bloggers, and while it's a little artisanal (i.e. hand-made much
of the time) at this point, it's got some really cool automated
features. One is that it registers (mints, hence the abundance of lame
peppermint puns across this post) DOIs for your posts.

I've been using Rogue Scholar to mint DOIs for my posts for a while now,
but it's always been a bit of a manual process. I'd have to wait for a
while for the post to go on the Rogue Scholar feed, then copy/paste the
DOI, then copy the DOI into the YAML front matter. It's not a lot of
work, but it's a bit of a pain. I've been meaning to automate it for a
while, but I've been busy with other things.

::: {.callout .callout-style-default .callout-note .callout-titled}
::: {.callout-header .d-flex .align-content-center}
::: {.callout-icon-container}
:::

::: {.callout-title-container .flex-fill}
Note
:::
:::

::: {.callout-body-container .callout-body}
Just after I posted about this solution, Martin Fenner, who runs Rogue
Scholar, [pointed out that there's now an
API](https://github.com/quarto-dev/quarto-cli/discussions/7174#discussioncomment-7557741).
The API is great, and would have spared me the part of having to scrape
the HTML. I will, one of these days, switch over -- if I had to build
it, I'd obviously use the API, and simply parse the JSON result. The
rest, *ceteris paribus*, holds true.
:::
:::

This weekend, I was laid up with being on the receiving end (for once)
of the bounties of a clinical trial, so I've decided to finally build
it. It's a bit of a hack, but it works.

::: {.cell layout-align="center"}
::: {.cell-output-display}
::: {#fig-auto-doi .quarto-figure .quarto-figure-center .anchored}
<figure class="figure">
<div>
<pre class="mermaid mermaid-js" data-label="fig-auto-doi"><code>flowchart TD
    linkStyle default interpolate basis
    A[&quot;Get last 10 posts with DOIs from RS&quot;]
    
    subgraph loop[&quot;Loop through posts&quot;]
    B[&quot;Read YAML preface&quot;] --&gt; C{&quot;Is post a cross-post?&quot;}
    C --&gt;|yes| skip
    C --&gt;|no| D{&quot;Is post citeable?&quot;}
    D --&gt;|no| skip
    D --&gt;|yes| E{&quot;Do we have a DOI?&quot;}
    E --&gt;|no| skip
    E ----&gt;|yes| F[&quot;Get DOI&quot;]    
    end

    A -----&gt; E
    F --&gt; apploop[&quot;Append loop to YAML&quot;]
    apploop --&gt; write[&quot;Write YAML&quot;]
    write --&gt; qr[&quot;Quarto Render&quot;]
    write --&gt; ghc[&quot;Github commit action&quot;]</code></pre>
</div>
<figcaption>Figure 1: Auto-DOI flow chart.</figcaption>
</figure>
:::
:::
:::

First, we scrape Rogue Scholar for titles and DOIs. Rogue Scholar's CSS
isn't really helpful here, as the link isn't a particular class/id of
its own as far as I could discern, so I just grabbed the link by the
fact that only DOI links are formatted like DOI links. Not the most
elegant way, but it does the job.

::: {#lst-getting-dois .listing}
Listing 1: Getting the DOIs from Rogue Scholar.

::: {#lst-getting-dois .sourceCode lst-cap="Getting the DOIs from Rogue Scholar." style="background: #f1f3f5;"}
``` {.sourceCode .python .code-annotation-code .code-with-copy .code-annotated}
def scrape_blog_for_dois(url) -> Dict[str, str]:
    response = requests.get(url)
    soup = BeautifulSoup(response.content, 'html.parser')
1    articles = soup.select('article')[:10]
    
    posts_with_dois = {}
    
    for article in articles:
        title = article.select_one('h3').text
        doi_link = next((a['href'] for a in article.select('a') if a['href'].startswith('https://doi.org')), None)
        if doi_link:
2            doi = doi_link.split('https://doi.org/')[1]
        else:
            doi = None
        posts_with_dois[title] = doi

    logging.info(f"Found {len(posts_with_dois)} posts with DOIs:")
    for title, doi in posts_with_dois.items():
        logging.info(f"{title}: {doi}")
    return posts_with_dois
```
:::
:::

1
:   [Technically unnecessary, as Rogue Scholar currently only displays
    ten links, but hey.]{code-annotation="1"
    code-cell="lst-getting-dois" code-lines="5"}

2
:   [This is where we split the DOI link into the link prefix and the
    DOI. We don't need the prefix, so we just grab the second part of
    the split.]{code-annotation="2" code-cell="lst-getting-dois"
    code-lines="13"}

Next, we iterate through each blog post. This is actually quite fast,
since (1) we have relatively few of them, (2) they're text documents. We
parse the YAML preface at the beginning of each of them. This looks
something like this:

::: {#lst-preamble .listing}
Listing 2: Pre-DOI YAML preamble example.

::: {#lst-preamble .sourceCode lst-cap="Pre-DOI YAML preamble example." style="background: #f1f3f5;"}
``` {.sourceCode .yaml .code-with-copy}
categories:
- Quarto
citation: true
date: 2023-11-13
description: 'Oh, that''s mint. We can finally use Rogue Scholar to mint DOIs for Quarto posts and append them automagically.'
google-scholar: true
title: Auto-DOI for Quarto posts via Rogue Scholar
```
:::
:::

What this tells us is that we do want a citation (someday), which is why
we're doing this in the first place. That, according to our beautiful
flowchart in Figure 1, means this post is eligible to get a DOI
appended. We also know there isn't one -- DOIs are appended as key-value
pairs (with the key being, unsurprisingly, `doi`) to the `citation`
object in the YAML preface. So, we'll see if we can get one by looking
in the dictionary we scraped from Rogue Scholar in Listing 1.

::: {#lst-process-post .listing}
Listing 3: Processing a single post.

::: {#lst-process-post .sourceCode lst-cap="Processing a single post." style="background: #f1f3f5;"}
``` {.sourceCode .python .code-annotation-code .code-with-copy .code-annotated}
def process_qmd_file(file_path: str, posts_with_dois: Dict[str, str]) -> None:
    with open(file_path, 'r') as stream:
        contents = stream.read()

1    delim = re.compile(r'^---$', re.MULTILINE)
    splits = re.split(delim, contents)
    yaml_preamble = splits[1].strip() if len(splits) > 2 else "" #
    rest_of_post = splits[2] if len(splits) > 2 else contents

    yaml_contents = yaml.safe_load(yaml_preamble) if yaml_preamble else None

    if yaml_contents:
        citation = yaml_contents.get('citation')
        google_scholar = yaml_contents.get('google-scholar')
        categories = yaml_contents.get('categories', [])
        title = yaml_contents.get('title')

        # Check files from crosspost categories
2        if any("cross-post" in category.lower() for category in categories):
3            if yaml_contents["citation"] == True or yaml_contents["google-scholar"] == True:
                yaml_contents["citation"] = False
                yaml_contents["google-scholar"] = False
                logging.info(f'Modified crosspost {title} to remove Google Scholar and/or citation reference.')
        else:
            # Ensure that google-scholar is set to true if citation is required
            if citation and not google_scholar:
4                yaml_contents["google-scholar"] = True
                logging.info(f'Setting google-scholar to true for {title}')

            # If citation is true but no DOI, and post exists in scraped posts
            if citation is True and posts_with_dois.get(title):
                yaml_contents['citation'] = {'doi': posts_with_dois[title]}
                logging.info(f'Adding doi for {title}.')

5        new_preamble = yaml.dump(yaml_contents).rstrip()
6        new_yaml_doc = f"---\n{new_preamble}\n---"

        # write the modified YAML document back to file
        with open(file_path, 'w') as yaml_file:
            yaml_file.write(new_yaml_doc + rest_of_post)
        logging.info(f'Updated file {title}')
        
```
:::
:::

1
:   [We have to split the document in two because only the preamble is
    proper, parseable YAML. The rest of the document is just text, so we
    have to recombine it later.]{code-annotation="1"
    code-cell="lst-process-post" code-lines="6"}

2
:   [If it's a cross-post, we don't want it to have a Google Scholar
    link, and we'll definitely not attach a DOI. In theory, we could
    have built this to be overridable in case I'll ever produce a
    cross-post I *do* want to have a DOI, but I don't see that
    happening.]{code-annotation="2" code-cell="lst-process-post"
    code-lines="20"}

3
:   [While we're at it, might as well prune the
    cross-posts.]{code-annotation="3" code-cell="lst-process-post"
    code-lines="21"}

4
:   [And anything with a DOI should also get a Google Scholar
    metadata.]{code-annotation="4" code-cell="lst-process-post"
    code-lines="28"}

5
:   [The `.rstrip()` is pretty useful -- otherwise, every time you run
    this, you'll get another newline appended to the YAML
    preface.]{code-annotation="5" code-cell="lst-process-post"
    code-lines="36"}

6
:   [Don't forget the `\n` before the YAML block's end, otherwise you'll
    end up with a YAML block that's not properly separated from the rest
    of the document and won't parse.]{code-annotation="6"
    code-cell="lst-process-post" code-lines="37"}

Finally, we write the YAML back to the file, and we're done. We can now
declare this as a project script, and we're good:

::: {#lst-project-script .listing}
Listing 4: Declaring project scripts in \_quarto.yml.

::: {#lst-project-script .sourceCode lst-cap="Declaring project scripts in _quarto.yml." style="background: #f1f3f5;"}
``` {.sourceCode .yaml .code-with-copy}
project:
  type: website
  pre-render:
    - scripts/pre_doi_from_rogue_scholar.py
    
```
:::
:::

One thing worth noting is that we're not actually running this on the
Quarto project itself, but on a copy of it. The consequence is that the
changes are made 'on the fly' to the `.qmd` files and do not necessarily
propagate into the repo. This is a pain, because recall that we're only
fetching the last ten posts' DOIs so as to be kind on the server: as
time goes on, that means older posts 'lose' their DOI.

To prevent this, we can simply check our changes back in:

::: {#lst-gh-action .listing}
Listing 5: Github action to commit changes.

::: {#lst-gh-action .sourceCode lst-cap="Github action to commit changes." style="background: #f1f3f5;"}
``` {.sourceCode .yaml .code-annotation-code .code-with-copy .code-annotated}
on:
  workflow_dispatch:
  push:
    branches: main

name: Quarto Publish

jobs:
  build-deploy:
    runs-on: ubuntu-latest
    permissions:
      contents: write
    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - name: Set up Quarto
        uses: quarto-dev/quarto-actions/setup@v2

      - name: Install Python and dependencies
        uses: actions/setup-python@v4
        with:
          python-version: '3.10'
          cache: 'pip'
      - run: pip install jupyter
      - run: pip install -r requirements.txt

      - name: Render and Publish
        uses: quarto-dev/quarto-actions/publish@v2
        with:
          target: gh-pages
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      
      - name: Commit changes to reflect source file  changes
        run: |
          git config --global user.name 'Chris von Csefalvay'
          git config --global user.email 'chrisvoncsefalvay@users.noreply.github.com'
1          git  diff-index --quiet HEAD || git commit -am  "Automated commit of changes to source files"
          git push
```
:::
:::

1
:   [The `diff-index --quiet HEAD` checks if there have been changes to
    the working tree. `git` returns an error if you're trying to commit
    on an empty working tree, so we're checking for that
    first.]{code-annotation="1" code-cell="lst-gh-action"
    code-lines="39"}

And that's it. We can now run this as a Github action, and it'll
automatically append DOIs to our posts.

[As noted](quarto-project-scripts/): Quarto project scripts are pretty
awesome stuff. I'm thinking of setting up an `awesome-` for it on
Github, because way too few of them are shared properly. I'm hoping this
will change.

::: {#quarto-appendix .default}
::: {.section .quarto-appendix-contents}
## Citation {#citation .anchored .quarto-appendix-heading}

<div>

::: {.quarto-appendix-secondary-label}
BibTeX citation:
:::

``` {.sourceCode .code-with-copy .quarto-appendix-bibtex}
@misc{csefalvay2023,
  author = {Chris von Csefalvay},
  title = {Auto-DOI for {Quarto} Posts via {Rogue} {Scholar}},
  date = {2023-11-13},
  url = {https://chrisvoncsefalvay.com/posts/auto-doi},
  doi = {10.59350/5hxdg-fz574},
  langid = {en-GB}
}
```

::: {.quarto-appendix-secondary-label}
For attribution, please cite this work as:
:::

::: {#ref-csefalvay2023 .csl-entry .quarto-appendix-citeas}
Chris von Csefalvay. 2023. "Auto-DOI for Quarto Posts via Rogue
Scholar." <https://doi.org/10.59350/5hxdg-fz574>.
:::

</div>
:::
:::


::: {.callout .callout-style-default .callout-tip .callout-titled}
::: {.callout-header .d-flex .align-content-center}
::: {.callout-icon-container}
:::

::: {.callout-title-container .flex-fill}
Tip
:::
:::

::: {.callout-body-container .callout-body}
If you're here for the DOI matching script, the link is
[here](https://github.com/chrisvoncsefalvay/chrisvoncsefalvay.github.io/blob/main/scripts/pre_doi_from_rogue_scholar.py).
:::
:::

[Quarto](https://quarto.org) is a great tool for reproducible research.
It is also a great tool for building websites. After about a decade of
Wordpress-based websites, I've moved to Quarto primarily for two
reasons. First, most of my content is largely static. I don't quite need
the full power of a CMS, and the performance cost of dynamic generation
I didn't need was quite steep (the economic cost was nothing special,
but I'm a cheapskate). Second, I wanted to be able to write in Markdown
and integrate better with notebooks. Quarto lets me do both of these.

What Quarto also does, however, is something super awesome: project
scripts!

::: {#project-whats .section .level1}
# Project whats?

A project script in Quarto is, basically, [a piece of Python, Lua, R or
Typescript code that is executed at some point during
rendering](https://quarto.org/docs/projects/scripts.html). This is super
useful, because it allows you to do things like, say, automatically
generate a list of publications from a BibTeX file -- which is among
others what I'm using it for. Essentially any programmatic function can
be implemented in a project script.

::: {.cell layout-align="center"}
::: {.cell-output-display}
::: {#fig-pipeline .quarto-figure .quarto-figure-center .anchored}
<figure class="figure">
<div>
<pre class="mermaid mermaid-js" data-label="fig-pipeline"><code>flowchart TD
    A[Rendering pipeline start] --&gt; B(Pre-render scripts)
    B --&gt; C(Page rendering)
    C --&gt; D(Post-render scripts)
    D --&gt; E[Rendering pipeline end]</code></pre>
</div>
<figcaption>Figure 1: Rendering pipeline in Quarto with pre- and
post-render scripts.</figcaption>
</figure>
:::
:::
:::

Quarto distinguishes between pre-render and post-render scripts. No
prizes for guessing the difference, but here it goes: pre-render scripts
are executed before the page is rendered, and post-render scripts are
executed after the page is rendered. Pre-render scripts are particularly
useful as they can modify the source `.qmd` (Quarto markdown) files
before rendering them to HTML or whatever their ultimate evolutionary
destiny is.

To make project scripts work, you need to declare them in your
`_quarto.yml` file:

::: {#cb1 .sourceCode style="background: #f1f3f5;"}
``` {.sourceCode .yaml .code-with-copy}
project:
  type: website
  pre-render:
    - scripts/pre_create_papers_file.py
    - scripts/pre_check_skierg_records.py
```
:::

This declares the scripts `scripts/pre_create_papers_file.py` and
`scripts/pre_check_skierg_records.py` as pre-render scripts. You can
also declare post-render scripts, with the `post-render` key.
:::

::: {#a-poorly-worked-example-generating-a-publication-list .section .level1 .page-columns .page-full}
# A (poorly) worked example: generating a publication list

::: {.page-columns .page-full}
On my website, I use Quarto project scripts to maintain a [list of my
papers](papers). This list is generated from a BibTeX file, which makes
it a little more convenient to maintain my publications, and somewhat
future-proofed in case I want a different format.^1^

::: {.no-row-height .column-margin .column-container}
^1^ I know there's a canonically better solution that uses pandoc and
Lua. I don't care. This works for me, and I don't have to learn Lua.
:::
:::

You can see the full project script
[here](https://github.com/chrisvoncsefalvay/chrisvoncsefalvay.github.io/blob/main/scripts/pre_create_papers_file.py),
but here's the gist of it:

::: {.cell layout-align="center"}
::: {.cell-output-display}
::: {#fig-bibliography .quarto-figure .quarto-figure-center .anchored}
<figure class="figure">
<div>
<pre class="mermaid mermaid-js" data-label="fig-bibliography"><code>flowchart TD
    A[BibTeX file] --&gt; a(&quot;load_bibtex_file()&quot;) 
    a --&gt; B[BibTeX database object]
    B --&gt; b(&quot;render_as_nlm()&quot;) 
    b --&gt; C{book or article?}
    C --&gt; D[article]
    C --&gt; E[book]
    D --&gt; d(&quot;render_article_as_nlm()&quot;) 
    d --&gt; F[article entry]
    E --&gt; e(&quot;render_book_as_nlm()&quot;) 
    e --&gt; G[book entry]
    F --&gt; H[all entries]
    G --&gt; H
    H --&gt; h(&quot;generate_list_by_year()&quot;) 
    h --&gt; J[final set of entries]
    J --&gt; j(&quot;write_into_file()&quot;) 
    j --&gt; K[&quot;papers/index.qmd&quot;]</code></pre>
</div>
<figcaption>Figure 2: My super clumsy pre-render script to build my
bibliography.</figcaption>
</figure>
:::
:::
:::

We load a BibTeX file (`load_bibtex_file()`), parse it, use a function
to determine whether we're rendering a book or an article entry (I don'
have any other types of publications, but if I did, I'd add them here),
and dispatch rendering to the appropriate function
(`render_article_as_nlm()` for articles, `render_book_as_nlm()` for
books). We sort the entries into a Markdown format with years separated
by level 1 headings (`generate_list_by_year()`) and finally write it
into the `papers/index.qmd` file (`write_into_file()`). There's some
minor magic going on behind the scenes, such as capturing my name and
setting it in bold (which is a bit of a convention in academic lists of
publications), but that's the gist of it.

The result is a list of publications that is automatically generated
from a BibTeX file, and is always up to date (as up to date as the
BibTeX file, anyway). I don't have to manually update it... sort of.
:::

::: {#wait-automagically-updated .section .level1 .page-columns .page-full}
# Wait, automagically updated?

The second half of what makes project scripts awesome is that they
integrate with the rendering process. If the rendering process in turn
integrates with some CI that watches for changes, then you can have a
website that is automatically updated whenever you push a change to the
repository. This is what I do with my website: I have a GitHub action
that watches for changes to the `main` branch, and if it detects a
change, it runs the rendering process and pushes the result to the
`gh-pages` branch. This means that if I were to change the BibTeX file,
it would trigger a re-render, and the list of publications would be
automatically updated.

There are some tricks to keep in mind here. Most importantly, because
this requires some custom Python packages, I had to slightly amend my
Github publishing workflow:

::: {.cell layout-align="center"}
::: {.cell-output-display}
::: {#fig-ci-cd .quarto-figure .quarto-figure-center .anchored}
<figure class="figure">
<div>
<pre class="mermaid mermaid-js" data-label="fig-ci-cd"><code>flowchart TD
    A[&quot;actions/checkout@v4&quot;] --&gt; B[&quot;quarto-dev/quarto-actions/setup@v2&quot;]
    B --&gt; C[&quot;actions/setup-python@v4&quot;]
    C --&gt; D(&quot;pip install -r requirements.txt&quot;)
    D --&gt; E[&quot;quarto-dev/quarto-actions/publish@v2&quot;]
    style D stroke-width:8px;
    style C stroke-width:8px;</code></pre>
</div>
<figcaption>Figure 3: Integrating the pre-render script into the Github
Actions CI/CD framework. The parts that need to be integrated into the
publishing workflow are highlighted with bold outlines.</figcaption>
</figure>
:::
:::
:::

In particular, I had to explicitly specify the Python version and
install the requirements. This is because the `quarto-actions/setup`
action does not install the requirements, and the
`quarto-actions/publish` action does not install Python. This is
admittedly a bit of a nuisance to work around, but really not all that
big a deal in the grand scheme of things.

::: {.page-columns .page-full}
One thing to keep in mind when writing your script is the context from
which it will be executed. In particular, if you're using a CI/CD
pipeline, you will need to make sure that the script can find the files
it needs. In my case, I had to explicitly specify the path to the BibTeX
file, as well as the path to the output file. This is because the script
is executed from the root of the repository, but the BibTeX file is in
the `papers` directory, and the output file is in the `papers`
directory, too. Testing locally for deployment in CI/CD is notoriously
hellish, but with a little bit of elbow grease,^2^ you can get it to
work.

::: {.no-row-height .column-margin .column-container}
^2^ And by 'a little bit', I mean absolute tons of it.
:::
:::

One caveat: rendering with pre- and post-render project scripts may be
relatively expensive in terms of time. My silly SkiErg world record
script adds a good 20-25 seconds of processing time to each rendering
run -- not a lot, but it does stack up eventually. The
[documentation](https://quarto.org/docs/projects/scripts.html) helpfully
discloses that Quarto provides an environment variable,
`QUARTO_PROJECT_RENDER_ALL`, which is set to `1` if it's a full render.
It may make sense to do a more manual caching here.
:::

::: {#things-you-can-do-with-project-scripts .section .level1}
# Things you can do with project scripts

Seriously, the possibilities of what you can do with project scripts are
pretty much endless. I have a project script that [checks the world
records list on the official SkiErg website and makes sure my index page
lists my accurate
records](https://github.com/chrisvoncsefalvay/chrisvoncsefalvay.github.io/blob/main/scripts/pre_check_skierg_records.py).
This is completely silly, but it's also a great example of how you can
use project scripts to automate just about everything:

- [A script I
  built](https://github.com/chrisvoncsefalvay/chrisvoncsefalvay.github.io/blob/main/scripts/pre_doi_from_rogue_scholar.py)
  pulls DOIs minted by [Rogue Scholar](https://rogue-scholar.org) and
  appends them to posts.
- A lot of projects use it for housekeeping: [via
  makefiles](https://github.com/CSCAR/workshop-stata-intro/blob/main/Makefile)
- The [bambi](https://github.com/bambinos/bambi) project uses it to
  [build a
  changelog](https://github.com/bambinos/bambi/blob/main/docs/pre_render.py),
  which is a great idea for applications that do need it
- This one cool master's thesis uses pre-render project scripts to [load
  variables from a YAML file into
  envvars](https://github.com/danielvartan/mastersthesis/blob/bebb699af7cab7add37a33768ccd605a60a18ba8/R/quarto-pre-render-html.R#L82),
  and I've [seen the same done with Rprofile
  files](https://github.com/computorg/computo-quarto-extension/blob/main/prerender.R)
  -- obviously, these are much more pertinent to projects where the bulk
  of the project consists of notebooks/code execution rather than
  websites, but again, the idea is the same
- Obviously, I'm not the only one to use it for [reference
  management](https://github.com/nfrerebeau/rarcheo/blob/master/_references.R)
- You can use it to [download static content into your
  project](https://github.com/lnnrtwttkhn/version-control-book/blob/main/Makefile),
  too!

The sky's the limit! I'm currently working on rendering my CV from a
JSON file via a project script, as well as a few other cool things. I
keep my project scripts
[here](https://github.com/chrisvoncsefalvay/chrisvoncsefalvay.github.io/tree/main/scripts)
-- I hope they will inspire you to do something cool with your own
project scripts!
:::

::: {#quarto-appendix .default}
::: {.section .quarto-appendix-contents}
## Citation {#citation .anchored .quarto-appendix-heading}

<div>

::: {.quarto-appendix-secondary-label}
BibTeX citation:
:::

``` {.sourceCode .code-with-copy .quarto-appendix-bibtex}
@misc{csefalvay2023,
  author = {Chris von Csefalvay},
  title = {Quarto Project Scripts Are Awesomeness},
  date = {2023-10-22},
  url = {https://chrisvoncsefalvay.com/posts/quarto-project-scripts},
  doi = {10.59350/b4qyq-70w75},
  langid = {en-GB}
}
```

::: {.quarto-appendix-secondary-label}
For attribution, please cite this work as:
:::

::: {#ref-csefalvay2023 .csl-entry .quarto-appendix-citeas}
Chris von Csefalvay. 2023. "Quarto Project Scripts Are Awesomeness."
<https://doi.org/10.59350/b4qyq-70w75>.
:::

</div>
:::
:::
