Linear model gradient

Linear Model Gradient Introduction¶

Currently, we support hetero-lr, homo-lr, hetero-linear regression and hetero-poisson regression. In this folder, we use a unified gradient calculation process template for all hetero linear algorithms.

We also provide a quansi-newton method for hetero-lr and hetero-linear regression.

Stochastic Quansi-Newton¶

When using Newton method, we use the following equation to update gradients.

$w_{k+1}=w_k- \alpha_k* H^{-1}\triangledown F(w_k)$ where H is Hessian matrix of w.

However, getting Hessian matrix is computational expensive. Thus, a more feasible solution is to use quansi-newton methods. We implement a stochastic quansi-newton method whose process can be shown as below.

Figure 1: Stochastic Quasi-Newton Method

Figure 2: Hessian Updating

For more details, please refer to this paper

最后更新: 2021-11-15