The logit/loglinear equivalence

• For most users of statistics, the loglinear model has one big unaccustomed characteristic: it is an association model; it has no dependent variable.

• There are two other differences from most models which are possibly as important but less disturbing:
  • It works on grouped data (tables);
  • It routinely works with very high order interactions.

• Because it's grouped it is particularly easy to manipulate, but it also becomes clear that including an extra variable changes the data (equally true but less obvious in other models). We could also fit high-order interactions in OLS, for instance, but it's less convenient than adding more variables.

• But despite the absence of a meaningful dependent variable in the loglinear model, we estimate it with a conventional generalised linear model and we can exploit this to reformulate the loglinear model as a logistic regression model for grouped data.

• Logistic regression takes a binary (0/1, yes-no) dependent variable, with a logit link and a binomial error term:
  
  $$\log \frac{y_i}{1 - y_i} = \alpha + \beta x_i + e_b$$
  
  
  or
  
  $$\log \frac{\pi_i}{1 - \pi_i} = \alpha + \beta x_i$$
  
  

• Consider a $n\times 2$ table, where the 2-category variable is to be thought of as dependent.
  
  $$\log F_{ij} = \lambda_0 + \lambda_i^A + \lambda_j^B + \lambda_{ij}^{AB}$$
  
  

• The logit or log-odds are $\log F_{i1}/F_{i2} = \log F_{i1} - \log F_{i2}$ for each row:
  
  $$\begin{array}{rcccc} \log F_{i1}/F_{i2} = & & \lambda_0 + ... ...- \lambda_{i2}^{AB}}\\ = & & \alpha &+& \beta_i \end{array}$$
  
  

  That is, there is a direct relationship between the loglinear parameters and logit parameters.

• Thus, where all the independent variables are categorical, a logistic regression can be estimated using a suitably formulated loglinear model.

• The general rule is the model should contain the $N-1$ and lower order interactions between the independent variables, and for every term in the logistic model formula, an interaction between in and the dependent variable. If Y is the dependent variable, and A, B and C are independent, with an interaction B*C the loglinear model will have the following design:

  Y + A + B + C + A*B + A*C + B*C + A*B*C   <-- Nuisance terms
    + Y*A + Y*B + Y*C + Y*B*C               <-- Model terms
  

  All the initial terms collapse into the intercept or constant term, and all the interactions with Y are interpreted as the effect of that variable on Y.