Haunted mailbox

We tend to think of writing as a container that carries thought from one mind to another. If I write the word “blueberry” I have planted an image in the reader’s head. But in truth, the reader’s mind is a closed system.

In the book Goodbye, Tsugumi, the titular character tricks her cousin into thinking her dead grandfather is communicating with her through a haunted mailbox. Her cousin later finds papers where Tsugumi practiced imitating her grandfather’s handwriting, and realizes it was an elaborate hoax.

A similar thing happens in the mind. We think the words we read come from a writer, when in fact we have a Tsugumi of our own writing the words and putting them in a haunted mailbox.

Think of it this way. A Large Language Model has no explicit way of distinguishing between the words it wrote, the words the user wrote, and the words in the system prompt other than the tags that mark those sections, and those tags are themselves part of the same stream of words. The LLM only knows to distinguish between those inputs because it has been trained heavily on inputs that make those distinctions.

In the same way I’ve learned to distinguish my thoughts from the words I read and hear, but ultimately it’s all happening in my own head. Once the words have surfaced in my mind they go on to influence my own thoughts, but not in the same way they would have if I had tagged them as my own thoughts.

So in a way, my thoughts are always entirely created and maintained in my own mind, but also my thoughts are never entirely my own thoughts. When I read, sensors in my retina are being stimulated in a pattern that I interpret as words, and the shape of those words have to be tagged as someone else’s thoughts post hoc, just as I tag my own thoughts as my own.