17 Offsets
The concept of an offset is easy to understand theoretically. Thus far, for each observation \(i\) in our dataset we have input features \(x_{i1}, x_{i2}, \dots x_{ip} \in \mathbb{R}\) and we wish to predict some response \(y_{i}\). In ordinary least squares (OLS), we model the response as
\[y_i = \beta_0 + \beta_1 x_{i1} + \dots + \beta_p x_{ip} + \epsilon_i,\]
With offsets, each observation \(i\) comes with an additional piece of data, \(o_i \in \mathbb{R}\), and the response is modeled as the sum of the offset and the linear predictor, i.e.
\[y_i = o_i + \beta_0 + \beta_1 x_{i1} + \dots + \beta_p x_{ip} + \epsilon_i. \tag{17.1}\]
Notice that the offset has no coefficient; it is included as is. There are a variety of reasons why you might want to do this. It may simply represent the predictions from a different model, perhaps fit on different data with different features, and we are trying to improve on it.
17.1 Why not replace \(y\) with \(y - o\)?
At first glance it seems odd to treat an offset in a special manner. We could rearrange Equation 17.1 to get
\[y_i - o_i = \beta_0 + \beta_1 x_{i1} + \dots + \beta_p x_{ip}.\]
We could then fit our model with the modified response \(y_i - o_i\) instead \(y_i\). Why does glmnet() need to handle offsets via an additional offset argument?
While the argument above certainly applies for OLS, it no longer applies for more complicated models. Take Poisson regression as an example (Section 9.3). For each observation \(i\), we have input and output variables \((\mathbf{x}_i, y_i)\) and exposure lengths \(E_i\), and the model is
\[y_i \sim \text{Poi}(\mu_i), \qquad \log \mu_i = \log E_i + \beta_0 + \beta^\top \mathbf{x}_i,\]
where \(\log E_i\) is the offset. We can’t simply replace \(\log \mu_i\) with \(\log \mu_i - \log E_i\): \(\log \mu_i\) is not passed to glmnet() in the first place! glmnet() only gets the \((\mathbf{x}_i, y_i)\)’s. Hence, to account for offsets correctly in the Poisson model, we need to pass glmnet() the \(\log E_i\) values via a separate offset parameter. glmnet()'s internal routines will ensure that the linear predictor \(\beta_0 + \beta^\top \mathbf{x}\) represents \(\log \mu_i - \log E_i\) instead of \(\log \mu_i\).
17.2 Applications of offsets
As mentioned above and described in Section 9.3, offsets are used in Poisson regression models to handle applications where the response counts are based on different exposure lengths. In this section we mention some other use cases for offsets in supervised learning.
17.2.1 Part of the model is known
Sometimes there are quantities that we want to incorporate directly into the model whose values we already know. For example, consider a web marketing application where we want to model the conversion rate (e.g. probability of click) of a new marketing campaign that spans multiple channels (e.g. email, text, social media). In this setting, we might know the historical conversation rate, \(p_0\), for each of the channels. Instead of trying to model the absolute conversion rate, we could model the improvement over the historical conversation rate instead.
Specifically, instead of modeling the log-odds of conversion directly, we model the improvement in log-odds of conversion over the baseline conversion rate. For each observation \(i\), we have input and output variables \((\mathbf{x}_i, y_i)\) and channel historical conversation rate \(p_{0i}\). The model for the improvement over baseline would be
\[ y_i \sim \text{Bern}(p_i), \qquad \log \left( \frac{p_i}{1-p_i}\right) = \log \left( \frac{p_{0i}}{1-p_{0i}}\right) + \beta_0 + \mathbf{x}_i^\top \beta \]
When supplying offset to glmnet(), note that offset is always on the scale of the linear predictor. Hence, in this case if the baseline conversion rates are stored in the vector p0, offset should be log(p0 / (1-p0), not p0 itself.
Another application where part of the model may already be known is in survival analysis. Recall from Chapter 12 that Cox regression models the hazard rate as
\[h_i(t) = h(t) \exp \left( \mathbf{x}_i^\top \beta \right),\]
where \(h_i(t)\) is the hazard for observation \(i\) at time \(t\), \(h\) is the baseline hazard for the entire population of observations, and \(\beta \in \mathbb{R}^p\) is the vector of coefficients to be estimated. For each individual, we might already have a baseline hazard adjustment \(r_i\) (e.g. a risk score from a different model) which we want to incorporate in the model. We could hence model the hazard rate as
\[h_i(t) = h(t) \exp \left( \mathbf{x}_i^\top \beta + \log(r_i) \right),\]
with \(\log(r_i)\) being the offset.
For Cox regression, the offset vector should be the log of the risk scores \(\log r_i\), not the risk scores themselves.
TODO: Add a section on pre-trained lasso? To balance how much detail here vs. how much detail in the “Other regularized regression models” chapter.