Group of people on a hillside
Group of people on a hillside

press to zoom
Jenny  Walking Sheep Up Street
Jenny Walking Sheep Up Street

press to zoom
Beekeepers examining a frame of bees
Beekeepers examining a frame of bees

press to zoom
Group of people on a hillside
Group of people on a hillside

press to zoom
  • Harriet Bell

Testing Scorecard Version 2 with Maps

Two of the key questions for this ELM Test & Trial were to identify a land management plan which worked for home farms and commons and to explore a payment by results approach.

The answer our Advisory Team of farmers/commoners and landowners gave to both was to use a scorecard system to provide a clear framework for how to deliver public goods and to assess the outcomes. This would then be supported by farmer led mapping and optional advice.

We tested a first iteration of the scorecard earlier in the summer (the results are available in our blog archive). We got a lot of detailed feedback which we brought together, reviewed and then produced version 2 of the scorecard.

Some of the main changes/tweaks were:

  • Based on feedback from farmers testing version 1 we edited the questions to improve them, reduce any repetition and just try and make the whole thing more useable. We also restructured it thematically.

  • James Moran, who has developed a number of scorecards used in Ireland, proposed that a some of our question, more related to management, be pulled out into a separate questionnaire section. A model where farmers/commoners would be paid for their time in answering the questionnaire, the results of which inform the scheme, but not paid for what their answers reveal about their farming practices. This is very different from version 1 when every question answer would be linked to a payment.

  • We introduced negative scoring - taking points off people for actions damaging to the delivery of public goods. With version 1 of the scorecard we'd tried to avoid scoring people negatively as much as possible based on the hypothesis that this would be perceived negatively by participants. In fact feedback suggested that some participants thought negative scoring for damaging activities would be fair so we thought we'd test it out with version 2.

  • We made some visual adjustments, colour coding questions and putting in more images to support answering questions.

You can download scorecard version 2 for the full details:

Scorecard V2
Download XLSX • 1.05MB

We also provided a map to use in conjunction with the scorecard. Funding from this T&T has gone to working with the University of Exeter, through the SWEEP initiative, to develop a satellite based mapping tool for Dartmoor.

One of the issues we identified early in the T&T is the challenge of regularly, affordably and consistently mapping the Dartmoor landscape. The system developed by Exeter makes this possible at a very low cost using a system which is programmed to translate satellite images into UKHab categories. This meant we could easily generate habitat maps for every farm and common involved as well as using other data sets to show information related to the scorecard, such as heritage assets. The slide show below shows an example of the types of map which were produced for each holding in the trial.

Testing both scorecard version 2 and the maps the key findings were:

  • A majority of participants felt version 2 was an improvement.

  • People were happy for some questions to be moved into a 'questionnaire' rather than payment by results section if it helped the scheme identify where to target capital grants. However, almost as many people felt all the questions should remain part of the payment by results element.

  • All but one of the participants favoured the use of negative scoring.

  • A majority felt this was the right approach for ELM and the details just needed fine tuning.

  • A clear majority felt that using the maps, even though they're, at best, 75% accurate, would still help them identify where to locate different habitats on the farm to use the scorecard. Furthermore, 16 out of 17 trial home farms also felt the maps would help them monitor habitat change over time. Participants were entirely positive about the potential to use the maps to pre-populate some of the scorecard answers but 100% of participants felt that they would only want to use that approach if data could be easily corrected where necessary.

We also asked participants about the kind of advice they might like to use with the system, who should provide it and who should pay for it.

Participants valued access to a wide range of different advice models and there was a general preference that advice should be funded entirely by Defra, with some exceptions. Defra was also the preferred provider for the more generic forms of advice, such as online guidance and support. When it came to in person advice the Dartmoor Hill Farm Project was a preferred provider but environmental charities and private consultants were valued for delivering specialist advice.

Download the full survey results and comments:

Scorecard Version 2 Accompanying Questions Results
Download PDF • 53KB

Recent Posts

See All

Following a great deal of hours spent on consultation and meetings, with many individuals in the farming community giving up their time freely towards reaching a consensus, alongside a range of stakeh

One of the key research questions of this test & trial was to develop a 'payment by results' approach which could operate on commons as well as home farms. Originally intended as a smaller element of

Whilst a key objective of this Test & Trial was to look at Land Management Plans and payment methods for commons what was quite clear, as soon as you start engaging with anyone involved in commons agr