Reformulation of the Gaussian error propagation for a mixture of dependent and independent variables

The Gaussian error propagation is a state of the art expression in error analysis for estimating standard deviation for an expression f(x1,...,xn,z) via its variables. One of its basic assumptions is the independence of the measurable variables in its argument. However, in practice, measurable quantities are correlated somehow, and sometimes, z depends on some of the xi’s. We provide the generalized version of the Gaussian error propagation formula in this case. We will prove this with the formula for total derivative of a general multivariable function for which some of its variables are not independent from the others; a counterpart to the probability approach of this subject.


Introduction
Frequently, the final result of an experiment cannot be measured directly, rather, it is calculated from several measurable physical quantities, each of which has a mean value and an error, and we are interested in the resulting error in the final result of such an experiment.Often, the measurement protocol is very complex and the set of measured physical quantities is a mix of variables in which some are independent of others and some are not.More importantly, selecting only independent physical quantities to be measured is not always possible.These difficulties occur in data analysis after collecting the outcome of measurements, for example: in weather observation or meteorology, astro-or high-energy physics, physical-, chemical-or biological measurements, as well as economics.
Below we discuss a theorem, how the Gaussian error propagation reads if in its x 1 , x 2 , x 3 ,…,x n , z 1 ,…,z m variables, the first n (the x i 's) are independent, but among the z 1 ,…,z m each one depends on some of the x i 's.When all n variables are independent and no such z j exists, the well known Eq.1 (written below) holds, commonly appearing in corresponding text and lab books.However, in many complex and/or large scale measurements, the variables may not be totally independent, and there may not be an alternative way to measure/choose purely independent variables.Statisticians use a procedure commonly called the delta method [1,2,3] to obtain an estimator of the variance when the estimator is not a simple sum of observations.The basic idea is to use a method from calculus called a Taylor series expansion to derive a linear function that approximates the more complicated function.To the best of our knowledge, although this case has been commonly formulated with algorithms using the concept of covariance via probability theory approach, still there is no compact expression formulated via calculus -here we do this.
2(∂f / ∂x 1 )(∂f / ∂x 2 )(Δx 1 )(Δx 2 ).More generally, one ends up with the famous Gaussian error propagation formula [4,5] which states that if f = f(x 1 ,x 2 ,…,x n ) and x 1 , x 2 , …,x n are independent quantities, e.g. of measurement possessing Gaussian distribution, the standard deviation of f (denoted as s f ) is (1) To complete Eq.1 for a measurement in practice, let u denote any of the independent variables among x 1 , x 2 , …,x n , and u j is the j th measured quantity for u, where j = 1,2,…,m(u).The mean of u is u avrg ≡ Σ (j=1…m) u j /m, and the standard deviation of u is On the other hand, to complete Eq.1 for probability variables u, one needs the corresponding unbiased estimate for expected value (E(u)) and its variance (D 2 (u)).
A simple example illustrate what the misapplication of independency or dependency can cause.Let f = x 1 +x 2 , and x 2 = x 1 with the obvious Δx 1 = Δx 2 .Assuming them to be independent variables (although they are not), (Δf 2 , or more simply Δf = (df / dx 1 )(Δx 1 ) = 2(Δx 1 ), i.e. the misapplication underestimates it (√2 < 2).Note: the equation of Gaussian error propagation degrades to the simple estimation of derivatives with the elementary numerical device Δf / (Δx 1 ) ≈ df / dx 1 for one variable (n = 1), given that in numerical analysis the Δx 1 is a small step while in error analysis the Δx 1 is the standard deviation.Similarly, if f = x 1 -x 2 , then the misapplication yields Δf = √2(Δx 1 ) again, but the correct expression yields Δf = 0 (since f = x 1 -x 2 = x 1 -x 1 = 0), i.e. the misapplication overestimates it (√2 > 0).The latter is a warning for a general perspective: in a statistical test for a hypothesis, predicting small positive value instead of zero may mistakenly suggest a statement to be true or false.Now we outline how a measurement can come up with a mix of dependent and independent variables.Let us suppose that one has to calculate a quantity of which dependence is f(x 1 ,x 2 ,x 3 ,x 4 (x 2 ,x 3 ,x 5 )), where x 1 ,x 2 ,x 3 and x 5 are independent variables, and x 4 is not, i.e. dependent as it is indicated.However, x 1 ,x 2 ,x 3 and x 4 can be measured directly, but not so in the case of x 5 .Algebraically it means f(x 1 ,x 2 ,x 3 ,x 4 (x 2 ,x 3 ,x 5 )) ≡ g(x 1 ,x 2 ,x 3 ,x 4 ,x 5 ) ≡ h(x 1 ,x 2 ,x 3 ,x 5 ) with the proper relationship among f, g and h.In other words, x 5 does not show up alone in the argument, but with x 2 and x 3 via x 4 .In this work we call these f-forms and h-forms.In the example mentioned above f = x 1 + x 2 with x 1 = x 2 , so h = 2x 1 .In this way, the general definition of f-and h-forms is obvious.The h-form may have fewer variables than the f-form, but not necessarily.In the particular case above, both have four variables, but in the case of f = x 1 + x 2 with x 1 = x 2 , f has two variables, as opposed to h which has only one.Below, we will need their partial derivatives, and e.g. in the case above ∂f / ∂x 5 = 0, despite that, (∂f / ∂x 4 )(∂x 4 / ∂x 5 ) is generally not zero.This is because x 5 does not appear in the argument of f, but otherwise it is possible.In other words, one has to be careful with the partial derivatives.It is obvious, that the f-form has a mixture of dependent and independent variables in its argument, while the h-form has only independent variables, but both have the same graph.Below, we will consider the general function f(x 1 ,…,x n ,z 1 ,…,z m ), where x 1 ,…,x n are independent variables, and z 1 ,…,z m are dependent variables.The latter means that these depend on at least one of x 1 ,…,x n , e.g.z 1 = z 1 (x 1 , x 2 ), z 2 = z 2 (x 2 , x 3 , x 5 ) with n ≥ 5, m ≥ 2 and so on.Algebraically the f-form can be reduced to h-form, because sometimes the relationship is indeed known, and the latter has only independent variables in its argument.However, sometimes even the exact analytical relationship is unknown, or in practice only the f-form can be used to evaluate that particular measurement and the h-form cannot.We try to enumerate that the effect of "mixture variables" can be positive or negative alike.It clearly shows that the unknown biases committed might be compensated by each other.The correlation of variables has a paradoxical outcome, e.g. the probability of chance correlation is diminished if the variables selected from a large pool are correlated [6].
Next, for the sake of brevity, we will call and use the errors Δf and Δx i , i.e. the standard deviation belonging to their mean or exact values.The measured variables (x i ) obey the Gaussian distribution, so their actual error is smaller than these threshold (Δx i ) values at a certain significance level.Even if f is not known analytically, via the measured or non-explicitly (e.g.recursively, etc.) calculated f(x) at x and x + Δx, the derivative of f can be approximated numerically.On the other hand, if the measured x suffers an error of size as the standard deviation (that is x ± Δx, i.e. the maximal expected deviation on a certain significance level), the error made in f, the Δf (which is also a standard deviation), can be estimated as (∂f / ∂x)Δx, if (∂f / ∂x) is known -that is (Δf) 2 ≈ (∂f / ∂x) 2 Δx 2 , which is Eq.1 for one variable.

The way to the reformulation via calculus
Without losing generality, let us suppose that there is only one dependent z, and we consider the f(x 1 ,…,x n ,z), where x 1 ,…,x n are independent variables and z = z(x 1 ,…,x n ).The latter includes two distinct cases: 1.: z depends on at least one (there exist i s.t.∂f / ∂x i ≠ 0, i=1,…,n), more, or all (for all i, ∂f / ∂x i ≠ 0) variables, 2.: z does not depend on any of the x i (for all i, ∂f / ∂x i = 0).If z does not depend on any x i , that is, the set {x 1 ,…,x n , z} contains only independent variables, the total derivative is df = Σ (i = 1…n) (∂f / ∂x i )dx i +(∂f / ∂z)dz, and the Gaussian error propagation comes from applying Eq.1 with the extension for one more variable (Δf) 2 = Σ (i = 1…n) (∂f / ∂x i ) 2 (Δx i ) 2 + (∂f / ∂z) 2 (Δz) 2 . (3)