Loading Tree…

Definition

Observed distributional characteristics differ from expected distributional characteristics.

Explanation

Indicators within this domain check observed distributions against reference distributions. Addressed aspects of distributions are location parameters (e.g. mean, median), scale parameters (the spread of a distribution, e.g. standard deviation) or scale parameters (e.g. skewness or kurtosis).

Distribution related checks may be applied to any type of numerical variables.

Checks may be performed against known distribution types (e.g. normal, uniform distribution), parameters. Distributional parameters may also be compared across categories of other variables such es examiners or devices. Observed discrepancies may be interpreted as a sign of examiner or device effects, for example.

Example

An assessment of hip circumference in a study is conducted by five examiners. The assignment of participants to examiners is approximately random. This motivates the assumption that means should be approximately the same across examiners. The number of cases and mean hip circumference per examiner is displayed below:

Examiner ID N observations Mean circumference
342 320 102 cm
333 180 103 cm
231 270 109 cm
123 255 102 cm
345 23 94 cm

The overall mean is 104cm with a standard deviation of 10cm. Key findings are:

  • Observer 342, 333 and 123 have highly comparable results implicating a high degree of standardization between them.

  • Observer 231 has a substantially higher mean . The number of examinations is high, therefore it is concluded that some quality issue exists

  • Observer 345 has a much lower mean of 94cm. However, the number of examinations is low. This introduces uncertainty as to the interpretation of this deviation. it may be possible that a minor series of subjects with a lower circumference has occurred.

Note: Without further information it cannot be told which examiners perform better should some unexpected discrepancy be observed.

Guidance

Checks for implausible distributions of measured variables should be conducted in any study.

Deviations of observed from expected distributions may indicate a wide range of issues which are not necessarily linked to errors. For example an unexpected location parameter may reflect sampling bias rather than information bias.

In a designed study, little effects of study design factors, such as devices or examiners, should be exerted on measurements. Finding associations of relevance between these factors and measurements are commonly indicative of measurement error.

It is highly recommendable to have completed “Consistency” checks before. Extraneous values detected at that step should be excluded before analyzing distributions. This is necessary to limit analyses to plausible but potentially error affected measurements.

Literature

  • Nonnemacher M, Nasseh D, Stausberg J. Datenqualität in der medizinischen Forschung: Leitlinie zum Adaptiven Datenmanagement in Kohortenstudien und Registern. Berlin: TMF e.V..; 2014.