Quadratic approximation of the log-sum-exp function

As seen here, the log-sum-exp function \text{lse}: \mathbb{R}^2 \rightarrow \mathbb{R}, with values

    \begin{align*} \text{lse}(x) := \log(e^{x_1} + e^{x_2}). \end{align*}

admits the following gradient and Hessian at a point x:

    \begin{align*} \nabla \text{lse}(x) &= \dfrac{1}{z_1 + z_2}\left(\begin{array}{c} z_1 \\ z_2 \end{array}\right), \quad \nabla^2 \text{lse}(x) = \dfrac{z_1 z_2}{(z_1 +z_2)^2}\left(\begin{array}{cc} 1 & -1 \\ -1 & 1 \end{array}\right), \end{align*}

where z_i: = e^{x_i},  i=1,2.

Hence, the quadratic approximation of the log-sum-exp function at a point x = (x_1, x_2) is given by

    \begin{align*} \text{lse}(x+h) &\approx \text{lse}(x) + h^T \nabla \text{lse}(x) + \frac{1}{2} h^T \nabla^2 \text{lse}(x)h \\ &= \text{lse}(x) + \dfrac{h_1 e^{x_1} + h_2 e^{x_2}}{e^{x_1}+e^{x_2}} + \dfrac{e^{x_1+x_2}}{2(e^{x_1}+e^{x_2})^2} (h_1 - h_2)^2. \end{align*}

License

Icon for the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License

Linear Algebra and Applications Copyright © 2023 by VinUiversity is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, except where otherwise noted.

Share This Book