Origami: Striped Box

Striped Box

Striped Box, designed by me

Once a year I run a little origami class for kids, for someone I know.  As a self-imposed constraint, I always teach modular origami.  It’s hard to find simple modular origami models that kids can do in a reasonable amount of time!

I’ve wanted to make a modular origami box, and a big one so that it can hold other origami inside.  So I bought some colored A4 paper, and looked around for a simple box design.  None of them were quite to my liking, so I made my own design.  There’s no lid for this box, because we’re keeping it simple.  I have folding diagrams if you’d like to try.

[Read more…]

Let’s Read: Transformer Models, Part 3

This is the final part of my series reading “Attention is all you need”, the foundational paper that invented the Transformer model, used in large language models (LLMs). In the first part, we covered some background, and in the second part we reviewed the architecture of the Transformer model. In this part, we’ll discuss the authors’ arguments in favor of Transformer models.

Why Transformer models?

The authors argue in favor of Transformers in section 4 by comparing them to previously extant options, namely recurrent neural networks (RNNs) and convolutional neural networks (CNNs).

[Read more…]

Let’s Read: Transformer Models, Part 2

This article is a continuation of my series reading “Attention is all you need”, the foundational paper that invented the Transformer model, which is used in large language models (LLMs).

In the first part, I covered general background. This part will discuss Transformer model architecture, basically section 3 of the paper. I aim to make this understandable to non-technical audiences, but this is easily the most difficult section. Feel free to ask for clarifications, and see the TL;DRs for the essential facts.

The encoder and decoder architecture

The first figure of the paper shows the architecture of their Transformer model:

diagram of Transformer architecture

Figure 1 from “Attention is all you need”

[Read more…]

Let’s Read: Transformer Models, Part 1

Large Language Models (LLMs) are a hot topic today, but few people know even the basics of how they work. I work in data science, but I also didn’t really know how they work. In this series, I’d like to go through the foundational paper that defined the Transformer model on which LLMs are based.

“Attention is all you need” by Ashish Vaswani et al. from the Proceedings of the 31st International Conference on Neural Information Processing Systems, December 2017. https://dl.acm.org/doi/10.5555/3295222.3295349 (publicly accessible)

This series aims to be understandable to a non-technical audience, but will discuss at least some of the technical details. If the technical parts are too difficult, please ask for clarification in the comments. You’re also welcome to just read the TL;DR parts, which should contain the essential points.

[Read more…]

Origami: Stars

origami stars

Five- and Six-pointed Stars. Designer unknown.

Back in 2019, we had a small wedding celebration–we didn’t actually hold a wedding reception, and that’s a story that I’ve already told.  As decorations for the celebration, I made a dozen giant paper cranes (actually Tsuru Roses) from wrapping paper, and you can see a photo of those at the bottom of my story.  I also made 50 origami stars from foil paper and holographic paper, seen above.

We’ve officially reached our 5th anniversary!  I am not inclined to be sentimental, but I am grateful for how incredibly fortunate we are.

My history in music

At the end of each year, I have a tradition of sharing a bunch of music. It started out as a little antidote to Christmas music, but then I started to focus on a different theme each year.

Today, I’d like to share music that represents my personal trajectory. I started listening to music in high school–and by that I mean actually choosing music for myself instead of just listening to whatever’s playing on the radio. At first, I gravitated towards electronic and techno. From there I started following a few individual rock bands. Then I went into more niche genres.  There were several different threads running in parallel, and I’d like to share a few examples of each.

[Read more…]

Laudatory categories

Theseus’ ship is a philosophical thought experiment that asks what happens if you take a ship, replace it piece by piece until none of the original pieces are left. Is it the same ship, or is it a different one?

Now imagine the following response: “It depends. Is the ship seaworthy?”

This response is a bit absurd, because clearly the question does not depend on whether the ship is seaworthy. A ship may still be the same ship while falling into disrepair, or perhaps the ship was never seaworthy in the first place. And on the other hand, you could have another ship which is also seaworthy but is nonetheless a different ship. We may disagree on how to answer the question about Theseus’ ship, but surely whether the ship is seaworthy is besides the point.

Nonetheless, this seems to be the way people think about many categories. A laudatory category is one whose definition has become intertwined with the question of “is it good?” A pejorative category is one whose definition has become intertwined with the question “is it bad?” Let’s talk about a few examples.

[Read more…]