The author argued that a large scale expansion in the delivery role of voluntary and community organisations in part reflected a political imperative, rather than any substantive evidence base that such organisations deliver better quality outcomes.

In the context of 25% cuts in central government department budgets, with presumably similar pain inflicted upon local commissioners, such a situation obviously cannot continue. As Andrew Hind, the outgoing Chief Executive of the Charity Commission has remarked “Charities will need to improve their systems of evaluation and will need to invest in demonstrating why they provide the best outcomes… Commissioners certainly won’t be taking their word for it." So, what’s stopping voluntary organisations – and their partners in government - from doing this?

NCVO has undertaken reviews of outcomes measurement systems for voluntary and community organisations in 2002 and more recently in 2010 as part of the Measuring Outcomes for Public Service Users (MOPSU) programme. In each case the messages for commissioners and service delivery organisations have been remarkably similar: an ongoing confusion over terminology that would keep the Plain English Campaign in business for some time; the use of systems or approaches that purport to – but don’t - measures outcomes, or even impact; and a plethora of different systems that appear to share little in terms of common characteristics or methods. Predictably, evidence from reviews by the National Audit Office and New Philanthropy Capital would indicate that the impact of such complexity is inefficiency and confusion. If some of these approaches are not able to attribute outcomes to the interventions paid for by government, then those delivering services, regardless of which sector they are in, are likely to get short shrift from both hard bitten taxpayers and frustrated users.

At this point a sensible retort might be to not bother – to stick with just monitoring service outputs. As the abolition of Comprehensive Area Assessment might suggest, spending less on measurement is politically attractive, particularly if it is top down and centralised. But people do not value services per se, but rather the effect those services have on them. The lack of information on the outcomes of services has sometimes encouraged commissioners to contract to lowest cost providers rather than to those who provide better value for money. This at times may have benefited voluntary and community organisations, yet finding ways to measure the outcomes of service use remains a priority for commissioners and providers.

This is easier said than done - the effects of service use may extend beyond the user themselves, for example parents may benefit from children attending early years education settings by gaining employment and carers may benefit from the person they care for attending a day care centre. But having clear limits on the service delivery system is one of five principles that the MOPSU approach has identified as fundamental to taking forward outcomes measurement. This is perhaps the biggest single challenge for those voluntary organisations that combine service delivery, advocacy and advice and support functions, particularly as outcomes measurement may only relate to one of those functions. But if more harmonisation between different service areas can be achieved this may become less of a problem. Four further principles for taking forwards outcomes measurement have been identified by the MOPSU programme. Any assessment must be based upon the experience of users rather than the interests of commissioners or proiders. The third principle is that outcomes should be directly attributable to the intervention, whilst the fourth principle states that the service should be assessed across different ‘domains’, which in turn are weighted. These two principles ensure that the service is making a demonstrable difference to the user, and that any difference reflects the different dimensions of any service. Finally, the last principle argues that any measures should carry as low a burden as possible, which in practice leads to the usage of regulatory data collected for existing purposes, if possible. More detail can be found at http://bit.ly/mopsu

This approach, which has been tested in two service areas (adult social care and early years education), has resulted in tools that can help commissioners and providers in these and related areas to commission services on the basis of outcomes. Crucially, we believe that these can help to demonstrate value for money: a clear message from the implementation of these measures was that services make a difference to the quality of life of users. Although a step forward, the tools are not however a ‘solution’: further work still needs to be done to extend their usage to areas where the user is harder to identify, whilst not all service areas collect the necessary data to populate such systems. Moreover, some want to go even further, with the notion that much greater use of randomised control trials could really help to identify whether voluntary organisations deliver better services. But that is for another day.  In the meantime, whilst we have high aspirations for the MOPSU tools and approach, we recognise that t is still one of many, many tools. Greater clarity on what to use, when and where is needed and our forthcoming review aims to address this issue.

Looking back, in 2002 we argued that outcome measurement was no longer in its infancy. Eight years later that may well be the case, but it remains a frustration for many that it is not yet an adult either, more of a sulky teenager. If voluntary organisations and commissioners are going to deliver better outcomes for users, based upon the experience of those users, then measurement needs to grow up.