Towards more reliable measurements of poverty and social exclusion in Europe

Only the most accurate and reliable methods to measure poverty and social exclusion in Europe should be used to inform policy. By following a set of recommendations developed by the EU-funded AMELI research project, policy makers can now make sure they extract the maximum benefit from estimates of social exclusion, known as Laeken1 or poverty indicators.

Poverty indicators can vary in complexity, by taking into account different variables (i.e. personal income, perception of health), and by estimating poverty in absolute terms or relative to a specific value or point in time. They are calculated using complex methods that estimate a parameter (such as the poverty gap or the at-risk-of-poverty rate) based on the statistical distribution of an ‘actual’ measured variable, such as personal income. Therefore, indicators are approximations of a given parameter and are useful for visualising an extremely large dataset.

The AMELI project compared poverty estimates for Europe based on a range of available state-of–the-art poverty indicators, all calculated from data about income and living conditions in Europe collected during the extensive EU-SILC surveys2. The researchers focused on poverty estimates for population subgroups at the regional scale, known as Small Area Estimation.

As well as optimising the method that should be followed when each indicator is used, the researchers highlighted the importance of understanding the uncertainties linked with their estimation in order for policy makers to apply the results accurately.

Policy recommendations

  • Improve communication between poverty data providers and data users, particularly about estimated errors in the data, in order to lead to more accurate interpretation. Approximations of error should be given in all cases.
  • Take into account how the data are collected and prepared when using indicator data for policy decisions to avoid misinterpretation.
  • Improve robustness of social indicators to identify long-term trends in the data as well as natural variation in the short-term.
  • Consider small area estimations in European policy as well as over a larger area, i.e. global indicators.
  • Increased collaboration across disciplines, including sociology, economics and statistics in the production, interpretation and application of poverty indicator data.

Statistical fine-tuning

One of AMELI’s first objectives was to agree on a common definition of social exclusion, to ensure consistency between estimates for different countries in Europe. This involved compiling results from previous studies using poverty indicators, such as the Luxembourg Income Survey, the ECHP and the EU-SILC surveys.

The researchers examined the sensitivity of each state-of-the-art indicator to variations in the complex sampling designs and statistical methods used. ‘Goodness of fit’ tests against the real data gave an estimate of the performance of each indicator in terms of accuracy, potential bias, how well it represented progress over time and how it was able to deal with contamination of the data (i.e. extreme outliers). By fine-tuning the methods, the researchers found a more appropriate statistical methodology for each indicator under different conditions and survey designs. For example, each indicator should be able to be used reliably to detect a trend in any European country, while still allowing for variability between countries.

Detailed analyses of statistical tests and the outcomes for each indicator are presented in a series of reports available on the website. For some applications, such as estimating the poverty gap, the researchers found that increasing the complexity of the modelling of income did not necessarily improve the accuracy. In others, the default approach generally used yielded results of poor quality, which could be significantly improved by fine-tuning the statistical methods.

The AMELI participants (from Austria, Estonia, Finland, Germany, Slovenia and Switzerland) also analysed the most effective ways to visualise indicator data for use by policy makers. Maps are generally the favoured tool but indicator data can also be illustrated in the form of checker plots, sparklines, mosaic plots and star plots, some of which allow the visualisation of more than one indicator at the same time.

Despite being focused on poverty indicators, many aspects of the AMELI project, particularly the recommendations on statistical analyses, can be applied to other policy areas, therefore providing a model EU framework for all indicator-based studies.

 

1 A set of commonly agreed and defined European statistical indicators on poverty and social exclusion, adopted by the Laeken European Council of December 2001.

2 European Union Statistics on Income and Living Conditions (SILC).
See: http://epp.eurostat.ec.europa.eu/portal/page/portal/microdata/eu_silc

AMELI - Advanced methodology for European Laeken indicators (duration: 1/4/2008 – 31/3/2011). FP7 Socio-economic Sciences and Humanities, Activity 6 “Socio-economic and scientific indicators”, Research area 6.2 “Developing better indicators for policy”. Collaborative project (small and medium scale focused research project).

See: http://www.uni-trier.de/index.php?id=40263&L=2

Contact: Ralf Münnich, muennich@uni-trier.de