Background

Explore the algorithm by variational inference

Introduction to SuSiE

Model setting

Covariates \(\textbf X\in \mathbb R^{n\times p}\), of which the rows are observations, and the columns are variables. We assume all variables have a small contribution to the output \(\textbf y\in\mathbb R^n\), and a small proportion of variables have big contributions.

We assume a linear model:

\[ \textbf y = \textbf X\boldsymbol \beta_1 + \textbf X\boldsymbol \beta_2 + \epsilon, \epsilon\sim N(0,\sigma^2I_n), \] where \(\beta_1\) represents the small contribution part

\[ \boldsymbol \beta_1\sim N(0, \sigma^2\sigma_1^2I_p), \]

and \(\boldsymbol \beta_2\) represents the big contribution part, i.e. coefficients are large but sparse, with at most \(L\) non-zero coefficients, where \(L\) is fixed in advance.

\[ \boldsymbol \beta_2 = \sum_{l=1}^L\gamma_l\beta_l,\gamma_l\sim Mult(1,\pi),\beta_l\sim N(0,\sigma^2\sigma^2_2) \]

Normally we assume \(\sigma_2>>\sigma_1\).

The variance \(\sigma^2\sigma_1^2\) s.t. its effect is comparable with the error term effect. An alternative assumption is \(\beta_1\sim N(0, \sigma_1^2I_p)\). It remains to be explored that which assumption has a simpler form, but they do not have a substantial difference.

• Here is one way to solve it by transforming the problem to a vanilla SuSiE problem.

First attempt

Second attempt

Simulation study

Simulation study of “First attempt”

Simulation study of “Second attempt”

Thesis

Abstract

Slide