Overdispersion at the Binomial and Multinomial Distribution

Overdispersion is a widely discussed phenomenon in case of binomial and Poisson distributed data. We analysed multinomial data with varying multinomial parameters as a part of attribute gauge study. In this situation overdispersion is expected to occur. However in case of nonrepeated observations it is not present. We investigated and proved the “lack” of overdispersion in case of binomial and multinomial variables if there are no repeated observations. The explanation is a “compensation effect”, which only occurs in case of binomial and multinomial variables and does not occur for Poisson distributed data.


Introduction
The most frequent design in medical statistics is the clinical trial.A part of the patients selected for study is exposed to a treatment, to which some of them react, others do not.The other part of patients is exposed to the control (what may be another treatment or lack of treatment), and again some of them react, others do not.The typical question is if the proportion of reacting patients to the treatment significantly differs from that to the control.In hypothesis test context the question is if the binomial parameter of the treated population is equal to that of the control population.Agresti [1] hints that neither the results of the treated patients nor the results of the control patients are from binomial populations.The reason obviously is that e.g. the treated patients are not homogeneous, the distribution of outcomes is a mixture of binomial distributions.This would lead to the wellknown phenomenon of overdispersion.Many literature sources deal with the diagnostics and modelling of overdispersion such as Hinde ,J. and Demetrio, C.G.B. [2], Cox, D. R. [3], Ganio, L. M. and Schaffer D. W. [4] and Barron, D. N. [5].
A multinomial example is the attribute examination of product items.A part of these items is conforming, another part is non-conforming, the outcome of the assignment is that the item is accepted or rejected.This outcome is fourfold: a good item is either accepted or rejected, the same possibilities hold for a bad item.Again the outcomes (items and decisions) do not stem from a single multinomial distribution, one has to face overdispersion here as well.

How we got surprised (the research topic)
Our research topic is the analysis of attribute measurement systems (Gauge R&R).
When using a single limit attribute measurement system, there are two different decision categories: we can reject or accept the measured item.Though the decision is categorical there is often a continuous variable in the background.The probability of acceptance depends on this continuous variable (reference value).E.g. the geometric size of a part must be above the specification limit in order to be conforming.Another example for an attribute type decision is the sensory evaluation of ampoules.
In practice the operator investigates the ampoules visually and decides if there are any solid particles in them.If there are solid particles, the ampoule is not useable.
In both examples there is a continuous characteristic behind the attribute type decision.The geometric size can be measured with a coordinate measurement machine, but in practice this kind of measurement would be too expensive and time consuming for performing it on shop floor.Thus they use a simple go/no-go gauge instead.In the ampoule example the continuous characteristic behind the decision is the size of the solid particles in the ampoule.If they are large enough they can be observed by human eye.In the following the continuous characteristic behind the decision is referred as size or reference value.
The probability of accepting a part depends on the reference value.The "very big parts" will be nearly always accepted, the "very small parts" will be nearly always rejected.There is a reference value interval (grey zone), in which the probability of acceptance strongly depends on the reference value.The connection between the probability of acceptance and the reference value is called gauge performance curve.Its mathematical form is assumed to be given by Eq. (1).
x: reference value p: probability of acceptance α, β: model parameters The graphical form of the gauge performance curve is presented in Fig. 1.The different curves belong to different α and β model parameters.The figure demonstrates that the position and shape (or slope) of the curve depends on α and β.This connection is discussed detailed by Vágo and Kemény [6].The reference value (x) differs from part to part.There are two sources of variation in the outcome: the difference between items and the random fluctuation of the result of the trial on a specific item.The two sources together result in an excess variation as compared with the variation obtained for identical items having parameters of marginal distributions.The phenomenon is termed overdispersion.It is expected that the marginal distribution of outcomes (accept/reject) may not be binomial, that is the variance may not be given by the parameter obtained from the expected value.
Our purpose was to characterize the measurement system, more specifically to quantify the probability of wrong decision.These are that an accepted part is bad or a rejected part is good.
In the typical design of attribute gauge study (Automotive Industry Action Group's measurement systems analysis (AIAG MSA) Manual [7]) several (n: say 50) items are selected in a random way from a large population of products, each of them is rated several (k) times.The outcomes of these ratings form the four categories of the distribution, they are summarized in a 2x2 table (Table 1).
denote the number of occurrences in the appropriate cell.E.g. it happened N 1 times that a good part received "accept" decision.Thus the sum of cell counts (N 1 , N 2 , N 3 , N 4 ) is equal to the total number of decisions (k * n).In Table 1 cells #2 and #3 represent the wrong decisions.Again as items are different, excess variation is expected as compared with the multinomial distribution.
When the variance of estimators is calculated the covariance matrix of N = (N 1 , N 2 , N 3 , N 4 ) is of interest.Simulation studies were performed in order to quantify the covariance matrix with different k number of repeated ratings, compared with the covariance matrix of a single multinomial distribution with average parameters.It was surprisingly found that if k=1 (single rating of all sampled items) there was no excess variation, that is no overdispersion was found.
Mathematical reasoning and proofs are given in the forthcoming sections.

Overdispersion at the binomial and at the Poisson case
The typical example in textbooks is the coin tossing.If y i denotes the outcome of the i-th experiment (1 in case of occurrence, 0 otherwise), and µ is the probability of the event (success, e.g.head), the expected value and variance of number of successes (number of heads obtained) (y In some situations the items on which the experiments are performed are not identical.E.g. when the effect of a medical treatment is investigated, the patients are not the same, the product items qualified are not the same, etc.The corresponding picture is tossing coins taken from a population of different coins (Box,Hunter and Hunter [8]).(When we make a new experiment a new sample is taken from the coins.)In these cases there are two sources of variation: the random fluctuation of the binomial parameters of coins and the random fluctuation of the result of the trial on a specific item.The two sources together result in an excess variation as compared with the variation obtained for identical items termed overdispersion.
According to the theorem of total variance the resulting variance is expressed as ([1, p 8],): where µ µ µ is the vector of size n of binomial parameters.Two cases will be distinguished: all n items in the sample are checked once or with k repetitions.

All Items are checked once
The picture is that a sample is taken from different (biased) coins and the coins are tossed, once each.

The first term
The y i outcomes at µ i binomial parameters are independent, thus The µ i binomial parameters come from the same population: The first product would be the variance if all elements were identical characterized with E (µ) binomial parameter.

The second term
The sum of the two terms: The surprising result is that if a random sample of elements is taken with replacement or from an infinite population, and all exposed once, there is no overdispersion.
Agresti ([1, pp. 493-493]), used this phenomenon implicitly when modelling binary matched pairs.Here the items within each group have different binomial parameters.However, the marginals of each group are considered as binomial with binomial parameter equal to the expected value of the binomial parameters within the group.
It is worth remarking the special case when both the population and the sample contain the same n elements.The corresponding picture is tossing each different coin once.In a new experiment the same coins are tossed.In this case the only source of variation is the random fluctuation of the result of the trial on a specific item.
In this case the second term of Eq. ( 2) is missing as the items are fixed, thus their binomial parameter is not random.The first term: where 2 is the variance of the finite population.The conclusion is that there is underdispersion, as described by Box et al. ([8,), as compared to the variance of a sample of n identical elements having the same E (µ) parameter.
The phenomenon discussed in fact is a compensation effect: the first term of the total variance expression shows underdispersion.The second term gives the overdispersion, and this may compensate or over-compensate the underdispersion.

All items are checked k times
The picture is that a sample is taken from different (biased) coins and the sampled coins are tossed, k times each, with result y i j .
The first term of the total variance expression is The second term The sum of the two terms The result is that overdispersion occurs only when exposition of items is repeated (k >1).This situation may not be typical in most fields of application (including biomedical experiments) but it occurs in case of the attribute gauge R&R study.The typical experiment in that area consists of repeated ratings of different product items by several operators.In the special case when both the population and the sample contain n elements (the binomial parameters are not random) the variance is given with Eq (3): In case of Poisson distribution similar situation can not occur, as there the first term of the total variance does not have a negative part.(λ denotes the Poisson parameter.) The second term The sum of the two terms If the Poisson parameter is not constant the variance also exceeds that in the homogenous case.

Overdispersion at the multinomial case
In some other experimental situations there are more than two outcome categories.
An example of our interest is the attribute gauge R&R study where several (e.g.50) product items are rated by several operators several times.The result of the study is the number of good items classified as good, the number of good items classified as bad etc, altogether 4 categories.
The result is analogous to the binomial case: there is no overdispersion if the rating is not repeated.Moreover, the pooled outcome probabilities are the population averages of the outcome probabilities of individual items, so the outcome frequencies follow multinomial distribution with these averaged probabilities.In the special case of binary outcomes the resulting distribution is binomial.
In the following we calculate the covariance matrix of the frequencies Y Y Y T = (Y 1 , ...Y m ) T of an experiment repeated k times with mpossible outcome categories conducted on N objects selected independently and with replacement.We can suppose without loss of generality that n = 1, for the objects are selected independently.In the general case the covariance matrix is multiplied by n.Denote µ µ µ T = (µ 1 , ..., µ m ) T the individual outcome probabilities of a randomly selected object.We have The second term in the right hand side is the covariance matrix of a multinomial distribution with order nk and parameter E(µ µ µ), hence the first term represents the extra variation due to the variance of the individual outcome probabilities µ µ µ.

Conclusion
Both for binomial and multinomial cases it has been proven that overdispersion occurs only if repeated ratings are performed on the same items.In other words the marginal distribution is binomial or multinomial, if there is no repeated rating, in spite of the fact that the conditional distributions are different from item to item.
The same phenomenon does not occur at Poisson distribution, as here the expression of variance is of different nature (it does not have negative part).