Causal inference is not about methods

By on
Testing alt text
Image courtesy of Eleanor Murray - epiellie.com

Solo post: Academic's take

Causal modeling is becoming more popular after some notorious failures of overly optimistic reliance on black-box predictive models to solve business problems (e.g., Zillow's iBuying). This is great. We are also increasingly seeing the introduction of a new method that "solves" causal inference. This is not so good because it misdirects attention.

Causal inference has more to do with data and assumptions than it does with methods. No method can "solve" the causal inference problem (although it can help by reducing bias). If anything, regression is one of the most common methods for causal inference, so regression is an effective causal inference method when all else is in order. This is different from predictive modeling, where brute force bias reduction using the most complex method may succeed.

Price elasticity of demand problem

Simply put, we want to know how demand will change if we change the price by a certain amount in a certain direction. More specifically, we want to know the percentage change in quantity demanded for a one percent increase or decrease in price. This is essentially a causal problem, although it has not always been treated as such. To get the elasticity, we need some bespoke data, ideally from an experiment.

Let's say the product is sold in a physical store, so a fully controlled experiment (an ideal A/B test) is not feasible. Even if it is digital, differential pricing is not the best lever to experiment on (while localized pricing is considered more acceptable). So we need an experimental design where we change the price of the product in one store and estimate the change in demand and compare it to the demand in another comparable store where the price remains the same. Such an experiment would generate the data we need. We could also use historical data if it fits.

Data and assumptions first, methods later

We start with the data that is intended (or generated) for the causal estimation of a target effect in the first place. Then we move on to the assumptions: positivity, consistency, exchangeability, and others like no interference. These assumptions are critical to identifying a causal effect (you can find a one-pager with brief definitions here).

Only when we have the right data and reasonably sound modeling assumptions do we need a method to estimate the effect. So the order is data, assumptions, and method. Does it matter which method we use? Not so much.

We can use regression (or partial correlation, as I saw popularized recently, 118 years after its introduction).1 We can also use a doubly robust estimator with nonparametric estimators. We can use any other method that helps us isolate the effect of price from the confounders (promotions on the item, store traffic, etc.) while avoiding colliders.

Do the latest and greatest methods not help at all?

Using Double ML instead of regression can help with model misspecification bias, but that's about it. For example, when estimating the price elasticity of demand, we need to have the correct model specification, i.e., we need to know exactly how promotions on the product or store traffic affect demand. Is the effect of store traffic non-linear? Does increasing traffic increase demand up to a point, and then level off due to crowding out as the store gets packed? If we do not know and specify the correct functional form in a regression, the regression results will be biased. If we use Double ML instead, we don't have to worry about this as much –the doubly robust estimator takes care of it. However, we still cannot omit variables that may be confounders or violate any of the assumptions.

So modeling price elasticity with Double ML does not magically turn the estimate into a causal effect unless we have the right data and assumptions.  So saying that partial correlation helps causal inference is the same as saying that linear regression helps causal inference. They both help, but so does any other consistent and unbiased estimator.

Bottom line

Causal inference is largely method agnostic. Assumptions about data make it possible to estimate a causal effect. Methods serve the data and assumptions. If the assumptions needed to identify causality hold for the given data and problem, then "Causal inference using Double ML in Python" can easily be translated into "Causal inference using OLS in Stata" while keeping the estimated causal effect intact. As we quote in our work on data centricity:

Assumptions simplify the complex world and make it easier to understand. The art in scientific thinking -whether in physics, biology, or economics- is deciding which assumptions to make.


[1] Apparently, partial correlation was first introduced in a paper by Udny Yule in 1907, building on earlier work in correlation theory by Galton and Pearson. It is basically just another way of standardizing the coefficient in a multiple linear regression.↩

Podcast-style discussion of the article

The raw/unedited podcast discussion produced by NotebookLM (proceed with caution):

Other popular articles

How to (and not to) log transform zero

By on
Image
Image courtesy of the authors: Survey results of the papers with log zero in the American Economic Review Academic's take Log transformation is widely used in linear models for several reasons: Making data "behave" or conform to parametric assumptions, calculating elasticity, etc. The figure above shows that nearly 40% of the empirical papers in a selected journal used a log specification and 36% had the problem of the log of zero. When an outcome variable naturally has zeros, however, log transformation is tricky. In most cases, the solution is to instinctively add a positive constant to each value in the outcome variable. One popular idea is to add 1 so that raw zeros remain as log-transformed zeros. Another is to add a very small constant, especially if the scale of the variable is small. Well, the bad news is that these are all arbitrary choices that bias the resulting estimates. If a model is correlational, a small bias due to the transformation may not be a big con...
Read more

What if parallel trends are not so parallel?

By on
Image
Image courtesy of github.com/asheshrambachan/honestdid Academic's take In difference-in-differences models, parallel trends are often treated as a make-or-break assumption, and the assumption is not even testable. Why not? One misconception is that comparing the treated and control units before treatment is a test of the assumption. In reality, the assumption is not limited to the pretreatment period but for the entire counterfactual. Comparing trends in the pretreatment period is only a plausibility test. That is, "parallel pretreatment trends" and "parallel counterfactual trends" are not the same. We can observe the former, but we need the latter. So, what we have is not what we want, as is usually the case. Even testing for parallel pretreatment trends alone is tricky because of potential power issues (evidence of absence is not the absence of evidence!) and other reasons, including a potential sensitivity to the choice of functional form / transformations ...
Read more

Explaining the unexplainable Part II: SHAP and SAGE

By on
Image
Image courtesy of iancovert.com Academic's take This post is an exception to close the loop we opened with  our post on LIME . Starting in 2025, we're changing the scope of Data Duets to focus on business cases (as opposed to methods). Having discovered an excellent write-up  that explains both SHAP (SHapley Additive exPlanations) and SAGE (Shapley Additive Global importancE), I will focus on the why questions, possible links to counterfactuals and causality, and the implications for data centricity. Why do we need SHAP and SAGE? The need for explainability methods stems clearly from the fact that most ensemble methods are black boxes in nature: they minimize prediction error but they obscure how the predictions are made. This is problematic because some predictions are only as good as the underlying conditions are favorable. For example, Zillow's iBuying was a notorious failure , likely due to a lack of clarity and transparency in how the predictions ...
Read more