Dartmoor Scorecard Version 1 Test Results

We have have pretty much wrapped up testing the first iteration of the scorecard now and updates for version 2 have been underway for a little while.

It's something of a relief that the scorecard appears to have actually gone down quite well. A testament to how well the Advisory Team, who've worked on developing the project, understand what would be the most useful approach to supporting Dartmoor farmers with the delivery of ELM.

For most of the questions we don't seem to have been too far off base about what is attainable and responses to the survey about the scorecard are generally quite positive.

But by all means read for yourself.

Download the anonymous amalgamation of all the scorecard results and the comments on each of the questions about what could be improved/changed:

Scorecard Version 1 Test Results with participant comments organised by public good

.xlsx

Download XLSX • 5.79MB

Download the results of the survey which asked participants about their experience using the scorecard:

Scorecard Survey Results

.pdf

Download PDF • 938KB

Dartmoor ELM Scorecard Version 1

The first iteration of the Dartmoor scorecard is now ready for testing. Whilst we're working specifically with four commons, and their associated home farms, we wanted to give as many Dartmoor farmers as possible the opportunity to have a look and feedback.

Quick refresh on a few things before the link.

Firstly, the objective of the scorecard is to evidence how farmers are delivering Defra's 6 public goods, which is how farmers will be paid under ELM. They are:

• Clean Air

• Clean & Plentiful Water

• Beauty, Heritage & Engagement

• Protection from & Mitigation of Environmental Hazards

• Mitigation of and Adaptation to Climate Change

• Thriving Plants & Wildlife

This approach is very different to how the Basic Payment Scheme worked.

Secondly, our design brief from the farmer Advisory Team included a few things:

• Try and keep it simple

• Try and create something farmers could do by themselves, without need of extra help unless they want it

• Put enough evidence on the scorecard that once its been submitted, to the RPA or whomever, and approved, the evidence should mean that the public goods you've delivered in that year can't be queried in future years e.g. can the scorecard reduce the risk of payments being reclaimed in future

The tension between trying to keep the scorecard clear and simple whilst trying to provide strong enough evidence of the public goods being delivered has been the biggest design challenge in creating the scorecard and its where your feedback will be most helpful.

Finally - expectation management.

We would have loved to produce the all singing all dancing perfect version, which was hosted on a website, a very user friendly website, where the accompanying mapping system, through which you could submit your evidence, was up and running and all the questions are perfect. This is not that version. This is version 1, where you might have to imagine a bit what it could be and mostly where you tell us if you think this is the right path to go down and list all the flaws we need to fix to get to that all singing all dancing perfect version.

Some technical notes for those of you wanting to test the scorecard:

• Look at the questionnaire about the scorecard first. This should give you a sense of what we hope to get out of the testing process.

• We tried very hard to format all the spreadsheets so they were only one page wide but we're not convinced they always open like this for everybody. If, when you open a spreadsheet, there's a button at the top which says 'Enable Editing' click that and hopefully the formatting should come good. If you chose to print it then be aware it's a lot of paper.

• Sometimes the text is pretty small because we tried to make it all fit on one page.

• You don't need to survey your whole farm to test the scorecard. Some questions automatically apply to the whole farm, some might be at a field parcel scale. It's enough to test those questions on one field parcel, that should tell you if the question works and how much effort is involved in answering it.

• We're still working on some species ID cards to help people with the habitat elements but, if you wanted to do some extra testing, you could try using an app for plant identification and then tell us if you think there's a role for apps in the delivery of ELM.

Apps we'd suggest are:

Sward App - Apps on Google Play

PictureThis: Identify Plant, Flower, Weed and More - Apps on Google Play

• We'll have more questions about the kind of mapping you'd like in future but we think something like the LandApp would work well accompanying the scorecard. So if you want to have a look and tell us what you think that would be great.

Download the questionnaire on how you found the scorecard (also available at the bottom of this page online):

Scorecard Accompanying Questions

.xlsx

Download XLSX • 25KB

Download the Dartmoor Scorecard Version 1:

Dartmoor Scorecard Version 1

.xlsx

Download XLSX • 5.78MB

Download the transect sheets for answering the scorecard habitat questions:

ELM Habitat Scorecard Transects

.xlsx

Download XLSX • 49KB

If you fill in the scorecard and the questionnaire on the scorecard in Excel then please email them, when you have finished, to hbell@dartmoor.gov.uk

Answer questions on the scorecard online if you prefer:

Dartmoor scorecard progress

I will hold my hands up and say that the scorecard is not progressing as quickly as we'd hoped and we're a bit behind. However, we're currently, touch wood, on a good run and we thought it was about time we updated everyone on progress.

Cast your minds back to the summer and you may recall that the Advisory Team and Dartmoor farming community at large was asked whether a Land Management Plan for the Commons, which also worked for home farms, should be based on a scorecard approach, a mapping approach or an advisor based approach. To which the answer delivered by the Advisory Team was all three:

- Have an overarching scorecard, covering all 6 of the public goods targets identified by Defra (Clean Air/Clean & Plentiful Water/Thriving Plants and Wildlife/Beauty Heritage and Engagement/ Mitigation and Adaptation to Climate Change/ Protection from and mitigation of Environmental Hazards). This overarching system should identify what the best scenario is and different stages towards achieving that using evidence based indicators.

- The scorecard has an accompanying mapping system.

- Farmers have the option of drawing on advice and support to guide their management in order to perform well against the scorecard or they can rely on their own knowledge and experience or some combination of the two.

We commissioned the Organic Research Centre (ORC) to develop the scorecard on a design brief that it should be as clear, concise and easy to use as possible.

The ORC were given a list of actions/activities related to public good delivery to form the basis of the scorecard. The list was drawn up in conversation with the Advisory Team about each of the public goods and at the few farmer meetings we've managed to squeeze in between lockdowns.

We've then approached a number of organisations working on Dartmoor to help us identify which indicators would provide evidence of public goods delivery. Ideally these indicators are incontrovertible, the way the scorecard would work is that a farmer submits their evidence for a 12 month period and then gets paid accordingly. This annual system takes away any concern of payments for previous years needing to be reclaimed but it all hangs on having quality evidence.

Input was sought from organisations like the Westcountry Rivers Trust, RSPB, Woodland Trust, Forestry Commission, to name some, and the National Park team itself.

The full indicator list in January was just under 300 items in number.

Now our clear brief from the Advisory Team was to keep it simple but we have a couple of issues:

1. Dartmoor, particularly its habitats, is complex and not simple

2. Any indicators we remove are potentially costing a farmer the opportunity to identify an element of public good delivery which, under the system we're proposing, reduces the financial value the scorecard represents to that farmer

Since January we've been trying to work with everyone involved in creating the list to edit it down, so that we only keep indicators of true value.

Our first attempt at this was not particularly successful. Arguably 20 people on one video call going through the full indicator list probably wasn't a great idea (thank you COVID for taking away any chance of an in person workshop). We also needed to recognise that we'd put a lot of time in with the farmer Advisory Team having presentations on this approach, looking at international models and discussing the details and we hadn't put that level of engagement effort in to the non-farmers who've come to our aid. So we were still grappling with some confusion about how the scorecard model would work. Plus, several scorecard models exist but this one will be custom created for Dartmoor so there is a degree of learning by doing which can be uncomfortable for us all. What we're now doing is breaking the scorecard apart thematically and having a series of 2-3 hour mini-meetings looking at indicators for each theme and this seems to work much better. We have three of these meetings left, all on Thriving Plants and Wildlife, taking us to the last week of February.

Then we will draw all the indicators back together again, make a decision about the best scorecard structure and then take it out to test in the real world with a number of Commons who've been invited to take part in the trial, which includes Harford and Ugborough and several other commons.

Understanding how much this change in farming payment means to farm businesses taking the scorecard out is a slightly terrifying prospect, which constantly leads one to think of the quote "Expectations are just resentments waiting to happen".

The first draft, probably the second, will not be good enough. The biggest part of testing it will be in identifying its flaws so we know if we can feedback to Defra that something like this, with more time to improve it, would be a good system, or if actually we've tried, we've tested and it turns out its not for us after all. Both options are valid learning for a Test & Trial.