We use cookies on our website. Some are necessary for the operation of the website. You can also allow cookies for statistical purposes. You can adjust the data protection settings or agree to all cookies directly.
Data Protection Declaration
Imprint
Development of a guideline for the assessment of practical research on organic agriculture based on an analysis of existing evaluation tools for research
Project
Project code: 2806OE307
Contract period: 01.08.2008
- 30.04.2011
Budget: 108,174 Euro
Purpose of research: Inventory & Assessment
The established system of research evaluation (e.g. SCI) does not sufficiently consider the impact of science on practice and society. Therefore the BÖLN-Project (06OE307) focused on the adequate extension of research evaluation beyond scientific impact. The project was carried out exemplarily for organic agricultural science.
Published evaluation concepts for applied, interdisciplinary and transdisciplinary research and inter-views with 22 agricultural scientists shows a high level of agreement that the following achievements of research are relevant for an evaluation of the impact on practice and society:
- Research questions and applicable results are related to relevant problems of practice and society;
- Cooperation with scientists particularly from other disciplines, cooperation and knowledge transfer with actors from practice and society;
- Publications and products of research for non-scientific target groups;
- Relevance and the subsequent impact on practice and society.
Interviewees propose a comprehensive and adaptable criteria set for the evaluation of the impact on practice and society. This corresponds to recommendations in the evaluation literature. Most concepts are tailored for specific evaluation objects and purposes. Accordingly, they differ considerably in focus and extent of criteria sets. Thus, general extension needs adaptability of criteria sets.
Literature concepts also provide a broad spectrum of evaluation methods. Frequently multi-method-approaches are used, which often consider both internal and external perspectives. In most cases evaluations were carried out as “stand-alone-procedures” with high effort for data collection, like interviews or evaluation of project reports. Attempts for a regular data assessment are made for example by the research Councils UK. Indeed, project results about methods point out that:
a) Actors from practice and society have to participate in the evaluation process.
b) A consequent follow-up-assessment for projects is needed to gather later impact on practice and society.
c) Data should be collected in a less time-consuming manner and be available for various evaluations and other purposes to enhance usability and acceptance of complementary assessment procedures.
To improve data collection and usability for evaluation (issue c), we developed a concept to integrate data collection in the application and reporting of third-party funded research projects. We did so, since we found considerable overlapping between data requirements for the evaluation of the impact on practice and society and the documentation requirements and guidelines of many public research funding organisations (e.g. BMBF, BMWi, BMELV, BMU).
To enhance the usability of information provided in proposals and reports for evaluation, it is necessary to
a) structure the data in such a way that makes information useable both for research funding as well as for evaluations,
b) explicitly gather information necessary for the evaluation which is currently not or only implicitly claimed and
c) store it in a database-system.
The latter enables the use of data for various evaluation and research administration purposes, because it can be filtered, aggregated and analysed as needed. Furthermore it can easily be updated with information from follow-up-assessments.
If structured documentation by scientists may substitute considerable parts of research proposals and reports, overall efforts of scientists for documentation would not increase. To further develop this concept in detail it will be implemented and tested in case studies in a follow-up project. Practical implementation would also allow verifying, if the intended benefits for scientists, research funding organisations and evaluation can be achieved. Additionally, a wide discussion within the Scientific Community is needed, to shape an general extension of research evaluation beyond scientific impact.
Section overview
Subjects
- Market Analysis/Marketing Strategy
- Organic Farming
Framework programme
Federal Organic Farming Scheme and other forms of sustainable agriculture (BÖLN)
