ENBIS-18 in Nancy

2 – 25 September 2018; Ecoles des Mines, Nancy (France) Abstract submission: 20 December 2017 – 4 June 2018

Handling Error in Variables in Linear and Quadratic Regression Using a Stochastic Gradient Method: Application to State Estimation in Power Grids

4 September 2018, 14:10 – 14:30

Abstract

Submitted by
Stephane Chretien
Authors
Stephane Chretien (National Physical Laboratory), Paul Clarkson (National Physical Laboratory)
Abstract
Linear regression is one of the most basic model in multivariate statistics. Another problem of great importance is the one of quadratic regression, i.e. the estimation problem for the model

y_i = b_0^tX_ib_0 + epsilon_i

where X_i, i=1,...,n are matrices of order p. This type of quadratic measurements are of paramount relevance in many industrial problems, such as e.g. power grid monitoring. Such problems can sometimes be efficiently studied via a convex relaxation based on Semi-Definite programming (SDP), which can be formulated as the following optimisation problem

min sum_{i=1}^n \ (y_i-\text{trace}(X_iB))^2

under the constraint that B is a positive semi-definite matrix of order p. One of the standard ways to look at this problem is to perform the estimation conditionally on the covariates and derive finite sample or asymptotic properties of the estimator.

In many statistical studies, however, practitioners have to take into account the variability of the covariates and provide a consistent estimator of b0 without prior information about the variance of these covariates (Zellner 1970). The corresponding setting is often known as regression with "errors in variables". Various approaches have been proposed for this problem based on the ideal of total least squares minimisation; see van Huffel (2013) for an exhaustive overview of the problem. The Bayesian approach has also been studied by Florens (1974), for instance.

The goal of our work is to address the problem of regression with error in variables using an efficient and scalable stochastic gradient method. In the case of quadratic measurements, we will consider a Semi-Definite Programming relaxation of the quadratic least-squares problem. These problems are reformulated as estimation in a model where expectations are composed with non-linear functionals. We will follow a methodology developed by Wang (2014) based on a new stochastic gradient approach. One of the main advantages of the methods developed by Wang (2014) is their inherent scalability for very large problems, a feature which is not shared by most standard generalised eigenvalue based methods.

Our main contribution is an improved algorithm which extends the method of Wang (2014). Our method is able to handle both the linear setting and the Semi-Definite relaxation of the quadratic setting. Extensive simulation experiments illustrate the efficiency of our algorithm.

Return to programme