The Magic Prompt Edition

Hey everyone, just a reminder that we’ve moved Wednesday’s edition to be subscriber-only. We have some fun ideas for things to do with WITI, but part of that will be about building up our base of subscribers. As part of being a paid subscriber, you now have access to WITI Subscriber’s Discord, which you should sign up for if you haven’t (you’ll need to verify your sub to get access to the channels). Thanks for reading and subscribing! - Noah & Colin

Noah here. At this point, you’ve surely played with (or at least heard of) the various text-to-image AI services that have popped up over the last few months. Dall-E, Midjourney, and Stable Diffusion all work roughly the same way: you give it a prompt, and it returns an image. Beyond the initial amazement of trying one of these things out, the game is to find the best ways to structure your prompt for the particular tool you’re using. When you log into Dall-E, for instance, you’re greeted with a suggested prompt like “An Impressionist oil painting of sunflowers in a purple vase.” 

Sites like PromptHero are a repository of amazing AI-generated images with their associated prompt. A stroll through the galleries gives you some clues to the kinds of special terms and phrases that truly generate magical results.

Why is this interesting?

Prompt writing truly feels like something new. It’s not prose, as is evident by the jumble of terms in the skull prompt above, but it’s also not code, as there’s no logical flow that connects the whole thing. Recently, software engineer and all-around interesting person Simon Willison called it more like a magical spell, and that feels a lot more right:

When you’re working with these, you’re not a programmer anymore. You’re a wizard, right? I always wanted to be a wizard. We get to be wizards now. And we’re learning these spells. We don’t know why they work. Why does Neuromancer work? Who knows? Nobody knows. But you add it to your spell book and then you combine it with other spells. And if you’re unlucky and combine them in the wrong way, you might get demons coming out at you.

Writing good prompts is about building your book of terms and syntax and trying different combinations until something sticks. There are no API references or even particularly clear expectations. “Instead,” Willison explains, “you have to experiment: try different fragments of prompts and see what works. As you get a feel for these fragments you can then start exploring what happens when you combine them together.”

While I think the metaphor works well, it also has some serious warts (which Willison acknowledges), particularly in its reinforcement of the idea that systems like Dall-E and Midjourney are magical—which they most certainly are not. At the end of the day, all these tools and their various adjacent counterparts (like GPT-3) are reliant on a bunch of math and a massive body of data to train off. They are extraordinary prediction engines. My own experience playing with machine learning in very simple ways mirrors this: I found the result amazing, but also came away convinced they were not magical or even all that intelligent. This isn’t to take anything away from the revolution happening with these tools. They’re great to play with, and people are building some truly useful stuff on the back of these models. I’m excited about what’s to come. 

Why does all this matter? As Emily Bender points out, if we get stuck in magical thinking, we don’t explore technology in the ways we should. Instead, she suggests we demand more answers to questions like, “What will happen to people subjected to the system’s decisions, if the system operators believe them to be accurate? Who benefits from pushing these decisions off to a supposedly objective computer? How would this system further concentrate power and what systems of governance should we demand to oppose that?” 

In the end, many folks are focused on the first-order effects of this stuff as they wonder what the future of writers and artists will be in a world where these tools continue to get better. I’m much more interested in the second-, and third-order cascades as these tools start to expose much broader audiences to a new technology and shape their understanding of how the new world works. But also, we probably shouldn’t freak out too much, as this is a classic example of the way cycles of technology repeat themselves. After all, it was sixty years ago that Arthur C. Clarke wrote, “Any sufficiently advanced technology is indistinguishable from magic.” (NRB)

WITI CLASSIFIEDS

We are experimenting with running some weekly classifieds in WITI. We are giving away some free classifieds to paid subscribers like yourself. If you’re interested, just fill out the form.

Build powerful and composable Elixir applications with a flexible tool-chain. Model your domain, derive the rest. https://ash-hq.org/

The No.1 job board for careers in transportation & auto tech https://mobilityjobs.com/

See why Taur is now the world's most accessible electric vehicle. https://taur.com

The weekly newsletter read by automotive CEOs https://www.trucks.vc/newsletter

New Word of the Day:

Was reading this little New Yorker piece about the art of the bialy, and the term “zizmorcore” jumped out at me:

On a recent afternoon, as the youngest Pastrami Queen customer by about fifty years, I enjoyed a bowl of chicken soup with kreplach and a chocolate egg cream. I took home some health salad and kasha varnishkes, plus a sweatshirt bearing the deli’s logo, the sort of old-school-New York merch that exemplifies Zizmorcore, a recent sartorial phenomenon that reached its logical conclusion earlier this year, when Coach collaborated with Zabar’s on a five-hundred-and-fifty-dollar leather tote emblazoned with a bagel.

This will probably become a full WITI, but if you want some more reading, check out Blackbird Spyplane and NY Mag. (NRB) 

Quick Links:

• If you enjoyed yesterday’s Immer Backstory Edition, here are two more Mayer mixes for your enjoyment: Immer 2 and Fabric 13 (NRB)

• I’ve been reading a bunch about the XYZ Affair recently, a late-1700s Franco-American diplomatic fracas that had huge effects on the United States. (NRB)

• A very good travel tote that fits onto your Rimowa (CJN)

• Good WSJ longread on the state of Xi and China (CJN)

• The plight of expat workers at KPMG in Saudi (CJN)

• Monthly Mortgage Payments Up Record Year-over-year (NRB)

• Understanding Your Michael Myers-Briggs Type (NRB)

—

Thanks for reading,

Noah (NRB) & Colin (CJN)

—

Why is this interesting? is a daily email from Noah Brier & Colin Nagy (and friends!) about interesting things. If you’ve enjoyed this edition, please consider forwarding it to a friend. If you’re reading it for the first time, consider subscribing.