The collection of data as part of reporting systems in healthcare promises insights and the early detection of aberrations. Two applications are the detection of infectious disease outbreaks and the quality assurance in healthcare using quality indicators. In this talk we discuss the impact and consequences of statistical uncertainty when using an algorithmic based approach to identify aberrations in time series of counts. This includes understanding structural biases of the reporting system, knowing the difference between confidence intervals and prediction intervals, distinguishing between variability and causal effects related to case-loads as well as communicating the meaning of uncertainty in situations, where a complete inventory count is analyzed.
Because the algorithmic classification usually just constitutes stage one of a two-stage quantitative-qualitative process, not all answers are of statistical nature. However, a decision theoretic perspective can be useful to make implicit assumptions and the role of uncertainty transparent. We use the automatic monitoring system of infectious diseases by the Robert Koch Institute and the external quality assurance of operational quality in Germany as examples.