to uncover causal relationships, cease making an attempt to invent a time machine and run an experiment as a substitute! Understanding causal relationships affords the information wanted to supply desired outcomes via motion. On this article, I’m going as an instance the ability of experimental design by utilizing a time-machine-based conceptual train. My aim is to persuade you that extra may be realized about causality via experimentation than utilizing a time machine.
Why are time machines helpful in a causal thought experiment?
Utilizing a time machine for a thought experiment feels ridiculous, and in some ways it’s. However it additionally has a attribute that makes it helpful for exploring hypothetical outcomes. Time machines may give us one thing that we, in our time-bound state, can not see – counterfactuals. Because the identify implies, a counterfactual is one thing that didn’t occur. They aren’t observable by definition as a result of they by no means occurred. Counterfactuals what would’ve occurred beneath totally different circumstances. They provide solutions to questions like – “Would I’ve gotten sick if I didn’t eat that gasoline station sushi?” If we had a time machine nevertheless, we may reverse the clock, do one thing totally different and see what occurs. Within the case of the sushi, I may restart the day, not eat the sushi and see if I nonetheless get sick. In different phrases, we may observe the in any other case unobservable counterfactuals.
The counterfactuals realized by the point machine may then be in comparison with what really occurred (we may name it a ‘factual’ I suppose…) to grasp the impression of an intervention. For our unlucky sushi instance, me getting sick is the ‘factual’ – it really occurred. If I had a time machine, I may rewind time, not eat the sushi and observe what would’ve occurred, that is the counterfactual. I may then evaluate the factual with the counter factual to determine causality. Let’s say that I went again in time, saved every little thing in my day the identical besides consuming the sushi. If I nonetheless bought sick (factual = counterfactual), I do know that the sushi didn’t trigger the sickness as a result of I might’ve been sick both manner. If I didn’t get sick nevertheless (factual ≠ counterfactual), then I can conclude that the sushi precipitated my sickness. With a time machine, establishing causality for particular person occasions can be that simple!
At first look, it looks like our time machine can be an superior causality deducing machine! With the ability to observe counterfactuals can be very highly effective, however we are able to really make extra helpful causal deductions utilizing well-designed experiments. Which is nice as a result of, time machines don’t exist, however well-designed experiments do! Let’s get into how designed experiments may be higher than utilizing a time machine.
The causality of particular person occasions will not be generalizable
Whereas a time machine would reply a whole lot of curiosity-driven ‘what if’ causal questions, the learnings we’d achieve from observing counterfactuals wouldn’t be generalizable to different, related (however not the identical) conditions. In my sushi instance, I might fulfill my curiosity by understanding if the sushi made me sick – however the information I gained wouldn’t serve any pragmatic objective for future selections. All I do know is that on that particular day, at that particular gasoline station, at that particular time, that particular serving of sushi made me sick. I don’t know what would occur if I modified any of the bolded circumstances.
We will achieve generalizable information, which we wouldn’t get from the time machine, by designing an experiment. Generalizable information could be very helpful as a result of it will possibly assist us make good selections sooner or later!
Think about that I ran an experiment that randomly assigned a number of courageous souls to eat gasoline station sushi or restaurant sushi. This experiment would inform me if on common, gasoline station sushi makes folks sicker than restaurant sushi. That is already an enchancment from the ‘time machine’ strategy as a result of the outcomes apply to the inhabitants of people who I sampled as a substitute making use of to me solely.
However, I could possibly be smarter concerning the design of the experiment to get much more information! As a substitute of merely assigning folks to gasoline station or restuarant sushi, I may assign folks specifc gasoline stations at particular occasions or the restuarant at particular occasions. By including these two new variables (time and gasoline station location) I can’t solely study if gasoline station sushi makes folks sick extra usually, I can even study if there are variations between the three gasoline stations that serve sushi in my city and if time of day additionally has an impression.
On this experiment, I don’t instantly observe counterfactuals, however the randomized task helps confounders common out so I can estimate the common therapy impact (ATE) virtually as if I may observe counterfactuals.
How do the experiment learnings differ from my time machine learnings? The experiment is (1) utilizing a number of folks, (2) a number of sushi servings, (3) a number of gasoline stations and (4) a number of occasions of day. Consequently, I can take away a whole lot of causal insights that I and different folks can use. For instance, I might perceive if usually, gasoline station sushi makes folks sicker than restaurant sushi in my city. I might additionally study if some gasoline stations make folks extra sick than others and if shopping for sushi at some occasions is worse than others. This data may also help me, and different folks make future selections. It’s rather more helpful than realizing that the sushi from one gasoline station and one time made me sick!
Along with all the variables that we are able to management, we are able to embody covariates in our evaluation. Covariates are components that we can not management however are necessary. On this instance, covariates could possibly be issues like earlier medical situations or age. By together with covariates within the evaluation, we are able to additionally study if there are any interplay results between the covariates and the therapies.
Under is a abstract that compares what we may study with a time machine to what we are able to study with experiments.
Now that we perceive the wealthy depth of causal relationships that we are able to perceive utilizing experimentation, let’s transition to discussing how the number of outcomes beneath an experiment is extra highly effective than a single end result (the one counterfactual) that we’d observe with a time-machine run.
Designed experiments quantify the causal relationships; single counterfactuals don’t
Direct remark of a single counterfactual doesn’t give any concept of the energy of the overall causal relationship. If I am going again in time after I bought sick as soon as to check if the sushi made me sick, I might study that it did, or it didn’t trigger my sickness. I nonetheless wouldn’t have any concept of the likelihood that I’ll get sick if I fulfill my sushi craving at a gasoline station once more sooner or later! Is it deterministic, i.e., will get sick each single time I eat gasoline station sushi? Is it probabilistic, will I get sick fifty % of the time? I simply don’t have sufficient info to know.
The experiment we designed within the earlier part wouldn’t solely assist us perceive if gasoline station sushi makes folks sick, it might additionally assist quantify the connection. For instance, the experiment would possibly discover that on common, consuming gasoline station sushi makes you 5 occasions extra more likely to get sick than restaurant sushi.
Experimental design generalizes higher, and it additionally quantifies the causal relationship higher! If we return in time and take a look at one counterfactual, we are able to’t know the likelihood of observing the identical end result beneath related situations, with experimentation we are able to!
Wrapping it up
My aim in writing this text was to debate why I might nonetheless use experimental design to find out about causal relationships even when I had a time machine that allowed me to look at counterfactuals.
The primary causes experimental design is healthier is as a result of:
- It generates generalizable causal learnings (versus one particular case)
- It offers the energy of relationships to tell future selections
I hope this thought experiment deepened your understanding of the strengths of experimental design!
