Categories: General

High-Entropy Writing

I read a
post
by Derek Sivers recently that reminded me of Claude Shannon’s concept of
Entropy. The post was about the his opinion that talks should always be
surprising.

People only really learn when they’re surprised. If they’re not
surprised, then what you told them just fits in with what they already know.
No minds were changed. No new perspective. Just more information.

So my main advice to anyone preparing to give a talk on stage is to cut out
everything from your talk that’s not surprising.

Derek Sivers

That pinged my brain super hard because I read a full biography on Claude
Shannon a couple of years ago, and it went deep into his invention of
information theory. Here’s the basic idea.

Shannon entropy quantifies the amount of information in a message. For
example, a message that is always the same contains no new information, so
its entropy is zero. On the other hand, a message that is always different
contains a lot of new information, so its entropy is high.

GPT-4

“New” information

Both Sivers’ post and entropy orbit the same concept, which is “new
information”.

I love that. It makes me happy.

So from now on I’m going to ask myself when I make a piece of content, “Is
this content High-Entropy?” If not, maybe I should do what Sivers says and
cut the stuff that isn’t new.

Surprise is new++

But that’s not quite it. In Shannon’s work, and in Sivers’ post, the idea
went beyond just new. It was about what was surprising.

Maybe surprise is like high-end newness. It’s like new, but with a pop.

Maybe that’s the standard. What’s surprising about this piece of content I’m
creating? What makes a person say “Wow”? What makes them see things
differently?

So maybe I’m looking to make content that’s high in entropy, and
specifically high in surprise.

Wait, what about methodologies?

But there’s a problem with this model, or at least a question. Are the
world’s best talks like this? Are there counter-examples?

Is there any room for something delivered perfectly but that isn’t
surprising? I can’t think of one.

Wait, how about a framework that takes really good ideas and turns them into
a structure that’s useful, transferable, and repeatable? How about a great
checklist?

In that case you have known items just arranged into something useful,
right? Where’s the surprise there?

Maybe the surprise is the elegance of the structure. It’s clarity. It’s
usability. It’s conciseness.

Yeah, I think that’s worth some surprise, even if the content itself is a
set of known items.