Abstract

In this paper, we study asymptotic model change detection in multivariate linear regression by using the Kolmogorov–Smirnov function of the partial sum process of recursive residuals. We approximate the rejection region and also the power function of the test by establishing a functional central limit theorem for the sequence of the partial sum processes of the recursive residuals of the observations. When the assumed model is true, the limit process is given by the standard multivariate Brownian motion which does not depend on the regression functions. However, when the assumed model is not true (some models change), the limit process is represented by a vector of deterministic trend plus the standard multivariate Brownian motion. The finite sample size rejection region and the power of the test are investigated by means of Monte Carlo simulation. The simulation study shows evidence that the proposed test is consistent in the sense that it attains the power larger than the size of the test when the hypothesis is not true. We also demonstrate the application of the proposed test method to Indonesian economic growth data in which we test the adequacy of three-variate low-order polynomial model. The test result shows that the growth of the Indonesian economy is neither simultaneously constant nor linear. The test has successfully detect the appearance of a change in the model which is mainly caused by the COVID-19 pandemic in 2020.

Keywords: economic growth; functional central limit theorem; gross domestic product; Kolmogorov–Smirnov type test; model change detection; multivariate linear regression; multivariate standard Brownian motion; partial sum process; recursive residuals

1. Introduction

As stated in Fujikoshi [1], He et al. [2], Somayasa [3], and Somayasa, Ruslan, and Sutiari [4], multivariate linear regression has been applied in many fields of study including agriculture, economics, geology, biology, and chemistry, among others. One important part of statistical modelling using multivariate linear regression is checking whether each component of the mean of the response vector can be adequately represented by a set of regression functions or whether we need more functions to represent some components of the mean of the response vector. This intention can be realized in practice by conducting simultaneous detection whether there are some changes in regression model. Cotos-Yéñez, Pérez-González, and González-Manteiga [5], Fujikoshi [1], and Zimmerman [6] called this step as model check. In other literatures, it also frequently called goodness of fit test for regression (see Das [7]).

Methods for change detection in multivariate linear regression has been intensively studied. They are commonly developed based on the residuals of the responses. Likelihood ratio test using modified Wilk’s lambda statistic has been documented in the literatures of multivariate linear regression (cf. [1, 2]). This method is applicable only when the response vector is normally distributed. Somayasa et al. [8] developed asymptotic model check method for multivariate linear regression with spatial observations on a compact rectangle based on the partial sum process of the vector of ordinary least squares residuals. The limit process obtained in [8] has been derived analytically by using some transformation theorems and the linear property of the partial sum operator. However, as it can be seen therein, the limit process appeared as an intricate function of the standard multivariate Brownian motion. It depends not only on the design of experiment but also on the assumed regression functions. By this reason, the quantiles of the Kolmogorov–Smirnov as well as the Cramér–von Mises type test statistics can not be computed analytically. This clearly restricted the application of the proposed method in practice. Although the limit process has a complex structure, nevertheless it is geometrically interpretable as a projection of the standard multivariate Brownian motion on its kernel space. This simple structure gives advantage in analyzing the power of the test when the assumed model is not true (see Somayasa [9]).

Partial sum method has been initially investigated for the purpose of quality monitoring in production process. The application of the method has been continuously developed to the problem of monitoring structural change for regression by defining a test based on the partial sum process of recursive residuals. Recent publications related to this context are Jiang and Kurozumi [10], Dao [11], and Otto and Breitung [12], among others. Groen, Kapetanios, and Price [13] pioneered the extension of the application of the method for monitoring structural change in multivariate linear regression based on the partial sum process of the vector of recursive residuals. However, their approaches required a normal distribution assumption attached to the vector of the errors, so that they are not purely asymptotic in the strictly sense. When the vector of random errors in the model is multivariate normally distributed, then the vector of recursive residuals builds a sequence of mutually independent random vectors, so that the limit distribution of the corresponding sequence of the partial sum processes of the vector of recursive residuals can be immediately obtained. This means that the application of the method in the practice must be started with a goodness of fit test or diagnostic check regarding the normality of the vector of recursive residuals.

In this paper, we study the application of the partial sum process of multivariate recursive residuals in asymptotic model check for multivariate linear regression defined on a closed and bounded experimental region. For that, we need to establish a functional central limit theorem for the partial sum process of recursive residuals obtained from multivariate linear regression under more general setting then that defined in Groen, Kapetanios, and Price [13]. It is worth mentioning that the results presented in [10–13] have been derived solely for univariate times series linear regression, where the experimental region is given by the set of positive integers which is not closed and bounded. In contrast to time series, when the experimental region is compact, we have triangular arrays of observations. For that, we need to establish a functional central limit theorem applied to triangular arrays of observations drawn from a multivariate linear regression model. To the knowledge of the authors, this result has not been yet documented elsewhere. It will require more effort on one hand by the existence of the correlation among the components of the response vector, but on the other hand, linear regression is attractive statistical tool which is applied in many areas such as in response surface methodology, chemical industry, and mining industry, among others.

The paper is organized as follows. In Section 2, we discuss literature review. Multivariate regression model with univariate experimental region together with the definition of the test statistic for checking the adequacy of the model is defined in Section 3. The limit process of the sequence of the partial sum processes of recursive residuals is investigated in Section 4 for the situation under as well as under . The proofs of the results are postponed to the appendix. Section 5 discusses the results of numerical simulation. In Section 6, we demonstrate the application of the proposed test method to real data which is the Indonesian economic growth data. Some conclusions and remarks are presented in Section 7.

2. Literature Review

Model check or model change detection plays important role in every serious empirical model building for a random phenomenon. As in geostatistics, Bassani and Costa [14] informed that optimum prediction (kriging) of the spatial process in unobserved geographical positions depends on the adequateness of a proposed regression model. Myers, Montgomery, and Anderson-Cook [15] stated that in response surface methodology, the validity of an assumed regression model determines the level of accuracy of the optimum condition of a production process under study. This means that to be able to obtain accurate prediction result, we have to carry out model change detection before the assumed model is used in the prediction (see also Huang and He [16]).

As documented in Fahrmeir et al. [17] and Zimmerman [6], model check or model change detection in multivariate liner regression can be conducted in several various ways: graphical method, Akaike information criterion (AIC), and likelihood ratio test by making use of Wilk’s lambda statistic. All methods have similar approach in that they are conducted by investigating the residuals of the observations. As a classical method, the decision obtained by graphical method is subjective so that this method is rarely used in the practice. AIC and likelihood ratio test are two inferential methods that have drawback in the application in that they are conducted under the assumption that the observations are normally distributed (see Fujikoshi [1] and He et al. [2]).

Bischoff and Gegg [18] proposed a purely asymptotic test method for detecting change in multivariate linear regression based on the partial sum process of ordinary least squares residuals. This innovative approach successfully incorporates the theory of high-dimensional stochastic process especially high-dimensional Gaussian process in the statistical inference. Many authors proposed test method based on the partial sums of recursive residuals instead of ordinary least squares residuals. Recently, Dao [11] studied the application of this approach for condition monitoring and fault diagnosis of wind turbines. Jiang and Kurozumi [10] investigated the power properties of the test based on the modified partial sum process of recursive residuals. Otto and Breitung [12] applied the partial sum method for testing and monitoring structural change of COVID-19.

To the best knowledge of the authors, the only work that investigated the application of the partial sum process of recursive residuals obtained from multivariate linear regression observed over time is that written by Groen, Kapetanios, and Price [13]. As it has been already mentioned in Section 1, they derived the limit process under multivariate normally distributed random error which clearly restricted the application of the method. Motivated by [13], in this work, we propose asymptotic test method based on the partial sum process of recursive residuals with application to simultaneous model change detection in multivariate linear regression of Indonesian economic growth data.

As defined in Bonokeling et al. [19], economic growth is an increase in the production of economic goods and services in one period of time compared with a previous period. Economic growth of a country reflects and measures the ability of the government in developing the economy of the corresponding country. Economic growth is commonly measured in terms of the increase in aggregated market value of additional goods and services produced which is measured based on the gross domestic product (GDP).

According to the study documented in Sari [20] and Febriyanti [21], there are at least four variables that frequently influence the economic growth of a country, namely, investment, government expenditure, export, and import. Each variable has a different impact on economic growth. While investment, government spending, and export positively affect economic growth, import negatively affects economic growth.

As it has been quoted in [19], other factor that can cause negative influence to the economic growth is disaster, such as COVID-19 outbreaks. The COVID-19 has destroyed world economy in 2020 which has brought the world to worst economic recession. Indonesia is one of more than 210 countries in the world that has been hit by the COVID-19 pandemic. Muhyiddin and Nugroho [22] reported that this situation has caused the Indonesian economic grew negatively in the second, third, and fourth quarters of 2020 after a positive growth achieved in the first quarter of 2020. In this work, we aim to check asymptotically that the COVID-19 causes change in the model of the Indonesian economic growth so that it can not be modelled anymore using low-degree polynomial over time.

3. Model Definition

We consider a nonparametric multivariate regression where is the random response vector, is the true but unknown vector of regression functions whose components are assumed to be continuous and of bounded variation on , and is the random error vector with and . We assume throughout that is a positive definite matrix. Let be linearly independent regression functions in , where is the Lebesgue measure on the measure space . Our goal is to find an asymptotic simultaneous monitoring procedure to check whether or not there are some changes in the regression models. More specifically, we aim to investigate a test procedure for the hypotheses versus , such that , where is a linear subspace in generated by the regression functions . In contrast to the classical inference method for multivariate linear regression, in this work, we do not need normal assumption for the error vector.

Suppose Model 1 is observed independently over an equidistant experimental design on , with triangular array of design points, given by

Correspondingly, let , , and . Then, the triangular array of observations satisfies the model

Next, for , let be the subset of consisting of the first design points. Associated with , we define the following -dimensional vectors as follows: with and , where is the identity matrix. The realization of Model 1 as well as Model 2 when observed on can be written as where with and .

Let be the design matrix of Model 3. That is, is a matrix, defined by whose -th column is given by , where we define , for . If is true, then we have the following multivariate linear regression model where is the matrix of unknown parameters, defined by

The least squares estimator of in Model 4 based on the first vector of observations can be computed by the following formula:

Hence, by the definition of the component-wise projection, we have for ,

We note that index in as well as in presented in Equations (5) and (6) means that the estimation is based on the first observations. By following the univariate case, we define the -dimensional recursive residual of Model 4 as where with .

For the purpose of testing the hypotheses defined above, we investigate the sequence of the partial sum processes of the -dimensional recursive residuals (Equation 7) by transforming the random matrix into a sequence of -dimensional stochastic processes , defined by where and , for . Let us call throughout the paper -dimensional recursive residual partial sum process (RRPSP). By the definition, for every , builds a stochastic process with sample path in the space of -dimensional vector of continuous functions . We define a test using the Kolmogorov–Smirnov functional, given by

It is clear that the wider the dispersion of the assumed model to the true-unknown model, the larger the value of will be. This means that the statistic in Equation (9) measures the discrepancy between the true and the assumed model. By this reason, can be reasonably used as a test statistic in detecting the occurrence of some changes in the model. We will reject when is large. For that, the limit distribution of under as well as under needs to be investigated.

We notice that the partial sum process (Equation 8) differs to those defined in [13] in that they did not include the rest term of Equation (8) in their definition. Consequently, the process defined in [13] has sample path in the space of -dimensional right continuous functions on with left limit, denoted by , instead of the space .

4. Approximation to the Test Statistics

Since the exact distribution of is mathematically not tractable, we investigate their limit distribution. We firstly obtain the limit process of by applying Theorem 7.5 of Billingsley [23] (see also Theorem 1.5.4 of Van der Vaart and Wellner [24]).

By substituting into Equation (7), the vector of the recursive residuals can be expressed as follows:

Since we have and , then by substituting these two equations into the preceding one, we get

Hence, for , it holds

Expression (10) shows that for every , there exists a column vector , defined by where

The column vector satisfies

So the -dimensional vector of recursive residuals can be written as

By Equation (12), we get , and by Equation (11), it holds

Thus, builds a sequence of uncorrelated -dimensional random vectors with and . If the random error vector is distributed as , then is also distributed as . Hence, under such condition, the set constitutes a sequence of independent and identically distributed random vectors.

Furthermore, by Equation (12), there exists an dimensional lower triangular matrix , say, where with the properties , such that

For and , let be the entry of in the -th row and -th column. Then, by the definition, can be concretely written as for , and , for .

Now we are in the position to state the limit distribution of under . The proof is given in the appendix.

Theorem 1. Let the regression functions be linearly independent in , continuous, and have bounded variation on . Suppose that , for , and . If is true, then for , converges in distribution to . Thereby, is the standard -variate Brownian motion on . That is, a centered -variate Gaussian process with the covariance function given by We consult the reader to Durrett [25] for the definition of the standard -variate Brownian motion .

By Theorem 1, it is clear for arbitrary fixed , and the sequence of random vectors converges in distribution to a -variate normal distribution , where

So that by applying Theorem 2.7 in [23] or Theorem 1.3.6 in [24] (continuous mapping theorem), the quadratic form defined by Equation (15) converges in distribution to a chi-square distribution with degrees of freedoms, denoted by . Furthermore, Theorem 1 gives us an approximation to the probability distribution of the Kolmogorov–Smirnov type test statistics when is true allowing us in approximating the rejection region of the test. It is constructed based on the probability distribution of the statistic . For , an asymptotic size -test will reject , if and only if , where is a positive constant that satisfies the condition . In practice, is usually unknown. It is estimated under by a consistent estimator , defined by where for ,

This means that Equation (16) is computed based on the -dimensional ordinary least squares residuals.

To be able to assess the power of the test, we need to find out the limit distribution of the test statistic when is not true. For that, we consider the scaled version of Model 2, defined by

When is not true, the model can be written as

Let be the vector of the recursive residuals when is not true. Then, by recalling Equations (10) and (17), we have

By substituting obtained from Equation (18) into the preceding equation, we get where is the recursive residuals under . It is clear that when is true, defined in Equation (19) coincides with . This can be obtained by replacing the terms and by and , respectively.

The limit process of , for when is not true, is summarized in Theorem 2. The proof is postponed to the appendix.

Theorem 2. Let the regression function be linearly independent in , continuous, and have bounded variation on . Suppose that the model is observed over . When is not true, then under the condition of Theorem 1, converges in distribution to the process , as , where for every ,

The convergence result presented in Theorem 2 provides an approximation to the power function of the test. Let be the power of the test evaluated in a regression function vector . That is,

Then, by Theorem 2 and the well-known continuous mapping theorem, can be approximated by the following boundary crossing probability:

When is true, the power computed in Equations (20) and (21) will reduce to the probability of type I error of the test of size . Conversely, when is not true, both determine the probability of the rejection of provided that is a true regression function vector. In other words, and supply information regarding the ability of the test in detecting the existence of model change. A good test should own the general property stated in Lehmann and Romano [26] and Rasch and Schott [27]. That is, the larger the power under is, the better is the test.