Data Is Not Enough: Simpson’s Paradox

The total aggregate can show the opposite trend of the aggregate of its subdivisions. This statistical Gestalt explains how people can justifiably live in alternate realities. Data alone is not enough

Gestalt theory says that that the whole is different from the sum of its parts. In statistics this is called Simpson’s Paradox.

So when we hear people today say “listen to the data” or “I’m data driven” that doesn’t really mean much because data are susceptible to paradoxes, and can tell contradictory stories depending on how they are clustered.  And bad actors manipulate this fact to use data you accept to prove conclusion you do not, trying to persuade you that day is night.

As an example lets take the covid incidence rates for 3 states, and each state shows an increasing trend. But each state show lower overall rates than the preceding. So each individual state show increasing rates, but the total shows decreasing.

This dynamic also explain how one can win the electoral college yes loose the general.   Or vice-versa.  It’s why gerrymandering works.  You can manipulate the cluttering to achieve the effect you want, regardless of the total aggregate.

IOW, you can use the same data to tell contradicting stories. So data alone are not enough.

Economics Isn’t About Money

aerial view of people walking on raod

I think of economics as a set of actors each experimentally discovering the rules of their unevenly shared contexts where discovery destabilizes those contexts in unpredictable and varying ways, forcing all actors to adjust and repeat endlessly. Not limited to financial economies.

In this ways economics is a general dynamic that applies well beyond its familiar financial expression.  Money, status, power, intellectual advances, even computational automata.

Podcast Review: Hidden Forces #202 — A History of the Future

person holding smartphone taking video of a concert near stage with lights during nighttime

Interview with BBC tech writer Rory Cellan-Jones.

I tend to avoid podcasts where the guest is a journalist, because, well, Chrichton’s Gell-Mann Amnesia effect. The stroycraft all too often precedes the insights.

Ironic how many complain about the precedence of narrative at the expense of fact, who then platform so many journalists, whose incentives are to produce and popularize narrative at the expense of fact.

That said, when journalists talk about journalism, things change. They become more conceptually fluid and less rhetorically fluid. A good sign we are moving away from incentivized narratives and toward individual insights. This happens here as Rory moves away from recalling and hyperbolizing tech stories, and toward to his experiences in journalism.


Rory suggests that the iPhone was more significant that assembly line production.

This is an example of why journalists are so hard to trust. Fatuous, wrong claims, if extreme and dramatic enough satisfy their incentives to aggregate attention.

Its an absurd claim. Modern life is not possible without the productivity surpluses produced by assembly line production. In fact the iPhone is not possible with out assembly line production. And we had a modern world, almost indistinguishable from today long before the iPhone. The claim is nonsense.

Furthermore it is heavily biased. First, recency bias. The iPhone is in living memory, while Ford’s assembly line production is not.

Second, egocentric bias. The iPhone announcement was Rory’s first tech story — it was his coming out announcement too. He is projecting the this event’s personal significance on history itself — everything changed for me from that moment, therefore everything changed for everyone for all time from that moment.

Third, you can tell in his recounting of the iPhone event that it’s the story that matters most. Given his journalistic incentives it’s the story that must matter most. And the teller must do everything possible to maximize this. And so he does.

IOW, sober reflection is the goal, the job, of the temperament. That is for historians, not journalists.


“you’ve got to get your hands in that phone”

Look, it was and is a great product. But seriously, it was just an evolution from things we had long had… Palm VII, treo, BlackBerry. It was just incrementally better.

I also held an iPhone in 2007. It was impressive. Like the first time I sat in a Tesla, or used Windows95. But it was certainly not a world defining moment.


“… reflective of the fact that it had been such a struggle to get ahold only one of those things”

This feels like another journalistic lie, because it increases the drama at the expense of truth. You see, there was no struggle — because no one knew about it before the announcement — and recall, that Rory said this was his first tech writing assignment.


“Tik Tok is the ultimate creative experience.”

No. No it is not. It just sounds dramatic and contemporary and ironic to say this. For a journalist things must be the most, the least, the best, the worst, to maximize the narrative drama. Again, everything is subordinated to “The Story”


Shortening attention span.

Rory takes issue with the claim of shortening attention spans, siting long form podcasts. An excellent point. Interestingly both can be true.

The opposite of a profound truth is often also true” -Richard Farson

At this point the interview turns to journalism itself and become much much better. Not much noteworthy. But still a better interview.

The Venn-Inversion Fallacy

A Venn-Inversion is a fallacy where from a small Venn diagram overlap, you posit a complete capture.

The IDW, a favourite wellspring of head scratchers, personifies many fallacies.  This one in particular arises time and again.  But I haven’t seen anyone yet model it and give it a name.  Let Eric be my inspiration.

The pattern works list this… There are innovative people who recognize things that most cannot, and as a result advance their culture.  Dick Fosbury for example.  And there are barking mad people who still believe that the world is flat and vaccines will turn you into Bill Gates’ garden gnome.

Now, not all innovators are crazy.  And not all crazies are innovators.  But there is some overlap between these two sets.  Innovators so visionary, so far ahead of their culture, that their culture cannot understand them and so label them crazy.

The fallacy, come from thinking that because your culture calls you crazy you must be a true visionary, a prophet of the the truth.  The label alone is proof.  If they call you crazy its only because you are right, they are wrong, and they are scared.

From minor overlap, to complete capture and conspiracy thinking.

The IDW folks take every criticism as persecution.  They cultivate this into full persecution complexes and epic Manichean narratives.  And through Venn-Inversions, criticism alone becomes proof positive that they, only they, hold the truth.

Speak Like a Human

total lunar eclipse
Photo by Mark Tegethoff on Unsplash

(Originally Posted 01/04/2007)

In a world run by Google word choice is powerful. Colloquial “vaccination” turns up mostly anti-vaxx material. Clinical “immunization” tuns up mostly sane scholarship. Doctors don’t speak human so humans often don’t hear them.

There’s a post over at medgadget about how searching google for “vaccination” turns up mostly anti-vaccination “moonbats”. One of the reasons is the medical industry’s preference for the word “immunization”.

So doctors say “immunization” while regular people and moonbats use “vaccination”. The lesson here is in a world run by Google, if you don’t speak like regular people, regular people will never hear your voice–even if you’re a doctor.

I wonder how much of our current anti-vaxx problems has been because they never saw the sane information due to the vicissitudes of Google search strings.

Your Don’t Control Your Brand – We Do

(Originally Posted 01/29/2007)

You don’t control your brand.  We do. — Roland Barthes

A couple months ago John over at Brand Autopsy posted a series of simple images from Zag Book that brilliantly illustrates the differences between advertising, marketing, pr and branding.

Your brand is your story, your myth. At its best your brand captivates and fires the imagination, it inspires and seduces people, it marshals markets and makes them hunger for you. A great brand, like a great story, is frighteningly powerful; and maddeningly volatile. Indeed, just like the author of a story, you don’t control your brand–we do.

Sure the post-structuralists and deconstructionists and guys with super cool Serge Gainesbourg accents may have gone a little far with many ideas, but I think they were mostly right with the idea of the author’s death; the idea that after writing a story, the author is no longer relevant and all of the power to make meaning of the text now resides with the reader.

Indeed this is what made the medieval church authorities so terrified to have the bible written in the vernacular: it would allow people to read the text free them from an orthodox intermediary interpreter and construct their own meanings. In this case, the death of the author meant the death of the church, of faith, of God. That is the power of the reader, a power that has terrified thinkers from Aquinas to Darrida, and a power that you need to learn to manage and embrace if you want yours to be a great brand.

So while the reader may have no control over the artifacts of your brand, they have complete control over their interpretation of these artifacts’ meanings and it is the meanings not the artifacts that captivate, fire, inspire and seduce.

Treat your brand as an author does a story, not as an ad-man flogs a soundbite.

No, We Are Not Living In a Computer Simulation

Sock Monkey plush toy on brown panel
Photo by Denisse Leon on Unsplash

We are not living a simulation. The computation required to simulate body momenta increases with number of bodies (N). The speed of action in our universe does not vary with N. Therefore our universe is no the output of computation. Therefore our universe is not a simulation.

I really would like to put an end to the intellectual masterbation of the simulation hypothesis.

The argument seems to go as follows…

First, we make simulations.  Simulations are built on computation.  Our computing power follows something like Moore’s law.  If we extend this in time indefinitely we have computing power increasing as infinitely as we have time.

Second, self-awareness and consciousness are inevitable consequences increased computational fidelity.  Such consciousnesses will eventually start to create their own internal simulations that will follow the same evolution, repeating the pattern fractaly.

We cannot know how many layers of this process there are.  But we can know that the number must be at least 1, and we no reason to think it is not greater.  This means that there is necessaily a greater chance we are living in a some nested layer of a universe simulator, than there is that this universe is the one and only.

The simulation hypothesis is just betting on the probabilities.

However the probabilities are wrong, because they are built on magic.

We can easily compute (or simulate) the momenta of any two bodies at any arbitrary point in the future.  Add a third and this system becomes chaotic and non-computable.  Add 10^82 (estimated number of atoms in the known universe) and, well, things get even trickier.

Let’s say we do eventually develop ways to compute the non-computable (there is no evidence to suggest this could ever be true, but let’s just give this to the simulators).  Even in this case the time to resolve a 10^82 body problem must still be greater than the time to resolve a 2 body problem, because of the differential computation required.

But the time it takes to resolve the motion of N bodies in the real word does not vary with N.  The time it take to compute N bodies in a simulation regardless of computing power, does necessarily vary with N.

In a simulation computing the motion of 2 bodies takes less time than 2+N bodies regardless of computing power.

In the real world resolving the motion of 2 bodies takes the same time as resolving the motion of 2+N bodies because no computation is taking place.  To only way to accomplish this in a simulation is with magic.  And if your hypothesis depends on magic, its fantasy not a hypothesis.

My UI Prototyping Toolkit

From January 2019

We designers love to talk about tools and process. But too many articles are either sloppy laundry lists of usual suspects, idealized manifestos or idiosyncratic case studies.

I want to try something different. I want to help you judge what could work for you and how, by sharing what works for me and how. I will organize things into the prototyping phases that I typically follow.

Read the Full Article…

Craig Vogel’s Reaction to Don Norman’s Articles

(Lazarus – Original 09/22/2005)

Author of “Breakthrough Products”, former IDSA president, and former design professor of mine Craig Vogel shared his thoughts over email on activity and personas in design.

“Experience design and activity design are the same. But knowing a person’s preferences is also important because a functional solution should be complimented with lifestyle attributes.

Norman is a psychologist and not a designer. His focus is on human activity which is fine. I think there is more to products than [just the] action analysis but it is an essential component.”

Sounds like Vogel is saying a study of activity is necessary, but insufficient; where Norman says that a study of activity is not only necessary and sufficient, but other realms of user study (like persons) could ultimately be distracting and therefore result in poorer product designs.

Of course Norman could be showing his phsychology bias here. In the integrated new product development process outlined in “Breakthrough Products” Vogel says that good product design results in products that are useful, usable and desirable. Norman appears to be focused on usable at the expence of both useful and especially desirable. And from a phychological perspective Norman’s may be an entirely appropriate reaction. However, from an iNPD perspective his reaction is a bit narrow.

It is hard to ignore the truth of Normans criticisms, that too often the reality of persona development is that it is improperly done and becomes a resource distraction.

Persona’s, Scenarios and Don Normal 2 of 2

(Lazarus – Original 09/11/2005)

Here is the first part of a discussion I had with Don Norman about his recently published articles on the use of personas and activity-based design…

How does (or should) the thesis of your article, if accepted, affect a group’s use of personas as a design tool? Should we forget about them (except as a communication tool) and concentrate on activities as the driving forces behind product design?

Don: Well, we got along quite well without personas before they became popular. I do not think they are important for the intelligent, observant, designer. As I an d you) said, I think they are useful mainly in communicating the decisions to other people.

I think the emphasis on activities is the key.

Is there perhaps too much growing faith in the power of personas at the expense of in-depth understanding of activities and their associated problems?

Don: Absolutely. The persona still says nothing about how to design.

Is a focus on activities perhaps too mechanistic, and blind to all the nuanced subjectivities of experience that contribute to a product’s success or failure, that are better captured between the lines of a persona narrative? (sorry for the awkward sentence, but I could think of how to better phrase it) .

Don: No.

Any single prescription runs the risk of being accepted mechanically. But if you have only average designers, then mechanical solutions are apt to be pretty good — better than they might produce otherwise.

Is a persona centered design approach even a user centered design approach? Or are many of us simply seduced by ease and economy of them compared with studying actual people?

Don: If you don’t study real people, then you can’t produce sensible personas! A persona is, after all, a distillation of the knowledge gathered about numerous individuals.

What is a comfortable balance between understanding people and activities in terms of designing better products? Your articles hint at an answer here.

Don: In no way can you understand activities without understanding people. An activity is the set of actions ( perceptions, thoughts, decisions, and actions) made within t he context of a set of goals. One cannot separate activities from people. Activities are goal-driven, and goals exist only in the heads of people. A major support need is to handle changing goals, and interrupted goal-driven activity — and this involves people.

Does this help at all?

Thanks for writing, and for the useful set of questions.