Bias Correction For Paid Search In Media Mix Modeling: Paper Review | by Chris Barton | Sep, 2020


Bias Correction For Paid Search In Media Mix Modeling: Linked Paper

Media Mix Modeling makes an attempt to estimate the causal impact of media spend on gross sales, only in keeping with observational knowledge. And, as everyone knows estimating causal results from observational knowledge is fraught with demanding situations.

Over time, two main, and complimentary, frameworks have emerged for coping with causal inference.

  1. Rubin’s Potential Outcome Framework.
  2. Pearl’s Graphical Framework.

This paper explores the usage of Pearl’s graphical framework to keep an eye on for choice bias in media combine modeling, particularly in paid seek advertisements.

Suppose we’re aiming to measure the causal affect of seek promoting (PPC) on gross sales. In a easy regression type, we will regress spend on gross sales and bring a causal estimate:

gross sales= average_sales + roas_estimate * search_spend + error

We can have compatibility the type above with abnormal least squares (OLS), and in a easy international this could produce a correct estimate of ROAS. Unfortunately, the arena isn’t so easy and we all know that there are incessantly confounding variables. For instance, we all know that natural seek may be a motive force of gross sales and that each natural seek and paid seek have an underlying purpose i.e. shopper call for. The graph under illustrates this.

Above, we will see how financial components force shoppers call for, which in flip drives seek queries, which in flip force paid seek and natural seek.

When we use our easy type, outlined above, to type the extra advanced international pictured above we run into the issue of selection-bias or extra extensively endogeneity.

Endogeneity: is a time period utilized in econometrics and refers to an explanatory variable this is correlated with the mistake time period.

Put merely, our type is shooting, or soaking up, the explanatory worth of natural seek in each the mistake time period and the ROAS estimate, thus generating and biased ROAS estimate.

To keep an eye on for this, the paper suggests the usage of Pearl’s backdoor criterion. Pearl’s backdoor criterion is in keeping with the theory of the usage of graphical fashions to explain the causal relationships. Graphical fashions are helpful as they permit us to include area wisdom and concepts from graph concept.

One such concept identity D-separation.

D-separation is brief for directed-separation and it permits us to keep in touch if two nodes in a graph are separated (or conditionally impartial of one another) by a suite 3rd nodes.

For instance, within the graph under, we will say Z is d-separated, or conditionally impartial, from Y given x1 and x2.

Another essential concept is the backdoor criterion.

Backdoor criterion: Given a causal diagram, a suite of variables Z satisfies the
back-door criterion relative to an ordered pair of variables (X, Y ) within the diagram if: 1) no node in Z is a descendant of X; and a pair of) Z “blocks” each trail between X and Y that comprises an arrow into X.

Furthermore, if a suite of nodes, Z, satisfies the backdoor criterion for the directed pair (X →Y) we will discover an independent estimate of X on Y, given a big sufficient knowledge set. This may be outlined as identifiability.

To get yourself up to speed with the idea that of backdoor criterion, I like to recommend taking part in with the next code snippet (check out developing quite a lot of graphs and decide what nodes would fulfill the backdoor criterion) and exploring further assets.

from causalgraphicalmodels import CausalGraphicalTypeeasy = CausalGraphicalType(
nodes=["x1", 'x2', 'z', 'y'],
("z", "x1"),
("z", "x2"),
("x1", "y"),
("x2", "y"),
easy.draw()easy.is_valid_backdoor_adjustment_set("x1", "y", {"z"})

Now that we’ve explored one of the fundamental ideas comparable causal graphical fashions, we will see how they’re implemented to recuperate an independent ROAS estimate for paid seek. To start, the paper illustrates the causal construction of our environment. (noticed under)

The above diagram suggests we’re modeling the affect as:

gross sales= average_sales + roas_estimate * search_spend + error_0 + error_1

Where error_0 and error_1 take in the affect of shopper call for and natural seek, respectively.

Based on our wisdom of graphical fashions, we now know that, if we will recuperate a measure of seek queries, we will fulfill the backdoor criterion for (seek advert X → Sales). To check this, we will use a to hand bundle referred to as: causal graphical models.

from causalgraphicalmodels import CausalGraphicalTypeseek = CausalGraphicalType(
nodes=["economic_factors", "consumer_demand", "search_queriers", "auction", "organic_search", "paid_search", 'organic_search', 'sales'],
("economic_factors", "consumer_demand"),
("consumer_demand", "gross sales"),
("consumer_demand", "search_q"),
("search_q", "public sale"),
("public sale", "paid_search"),
("search_q", "paid_search"),
("search_q", "organic_search"),
("organic_search", "gross sales"),
("paid_search", "gross sales"),
seek.is_valid_backdoor_adjustment_set("paid_search", "gross sales", {"search_q"})# output is True

In the code above, we outline our causal graphical type (DAG) and check if our keep an eye on variables fulfill the backdoor criterion for (paid_search → gross sales).

Next, we generate pattern knowledge and run and OLS regression to check estimates once we fulfill the backdoor criterion and once we don’t.

from causalgraphicalmodels import StructuralCausalModel
from causalgraphicalmodels import CausalGraphicalType
import numpy as np
# create structural causal type. This let's us generate knowledge. search_data = StructuralCausalModel({
"consumer_demand": lambda n_samples: np.random.customary(100,5, measurement=n_samples) ,
"search_q": lambda consumer_demand, n_samples: np.random.customary(consumer_demand*.3, 1, n_samples) ,
"organic_search": lambda search_q, n_samples: np.random.customary(search_q*.6, 1) ,
"paid_search": lambda search_q, n_samples: np.random.customary(search_q*.1, 1) ,

"gross sales": lambda organic_search, paid_search, n_samples: np.random.customary(75 + organic_search*.2 + paid_search*.3, 1 )

knowledge = search_data.pattern(156)# run OLS with out backdoor criterion happy for paid seek --> gross salesX = knowledge[['paid_search' ]].values
X = sm.add_constant(X)
effects = sm.OLS(knowledge.gross sales.values, X).have compatibility()print(effects.abstract())# run OLS with out backdoor criterion happy for paid seek --> gross salesX = knowledge[['paid_search' ]].values
X = sm.add_constant(X)
effects = sm.OLS(knowledge.gross sales.values, X).have compatibility()print(effects.abstract())# with backdoor criterion happy X = knowledge[['paid_search', 'search_q']].values
X = sm.add_constant(X)
effects = sm.OLS(knowledge.gross sales.values, X).have compatibility()print(effects.abstract())

Resulting within the following ROAS estimates:

As we will see, each estimates seize the real parameter with the independent estimator (backdoor criterion happy) coming a lot nearer to the real estimate.

Now, you’ll have spotted, within the code pattern, that we handiest sampled 156 knowledge issues, which might be equivalent to 3 years price of weekly MMM knowledge. This isn’t is lot of information and brings up a very powerful level i.e. how do we all know when our pattern measurement is sufficient?

The paper means that this worry is also alleviated “when pattern measurement is satisfactorily huge to permit for non-parametric estimates”, alternatively huge pattern sizes don’t seem to be not unusual in MMM modeling.

To discover this additional, I’ve created the graphs under display how the ROAS estimates and self belief durations alternate given increasingly more higher pattern sizes.

As we will see, the independent estimator converges to the real parameters, while are biased estimator is overly constructive. Additionally, the graphs above spotlight how the small pattern sizes produce very huge self belief durations. Something to take into accout of if the pattern measurement is small.


Please enter your comment!
Please enter your name here