Monitoring & evaluation for learning in rural innovation systems

There is no "one suits all" definition of knowledge managment. A fairly wide definition could be as follows: "Knowledge management is the creation and subsequent management of an environment which encourages knowledge to be created, shared, learnt, enhanced, and organization for individual, social, or organizational benefits."

are where people continually expand their capacity to create the results they truly desire, where new and expansive patterns of thinking are nurtured, where collective aspiration is set free, and where people are continually learning how to learn together. The concept originated from Peter Senge's 1990 book The fifth discipline: the art and practice of the learning organization. See also the related terms mental models, team learning, systems thinking and personal mastery.

an approach to analyse and understand how things influence one another within a whole. In nature, systems thinking examples include ecosystems in which various elements such as air, water, movement, plants, and animals work together to survive or perish. In organizations, systems consist of people, institutions, structures, and processes that work together to make an organization effective or not.

The involvement of actors in the process of making decisions that will affect them, including what is to be done and how. In a RAAKS study, planning for the inclusion and meaningful participation of those who have typically been called 'beneficiaries' or 'target groups' is essential. Successful use of participative approaches requires respect for others' knowledge and experience and willingness to involve them in implementing, contributing, sharing and evaluating proposed solutions. Participation may involve individuals, groups or their representatives. The quality of participation has a great impact on the outcome of the process, and can greatly increase the chance that key actors will support the results - so that action results. Decisions in this area are probably the most important of any taken by a RAAKS team.

The expansion of people's capacities and choices; the ability to exercise choice based on freedom from hunger, want and deprivation; and the opportunity to participate in, or endorse, decision making that affects their lives.
Source: UNDP, Governance for sustainable human development. A UNDP policy document (1997)

People are at the core of the knowledge system perspective used in RAAKS - not as passive recipients but as active, knowledgeable participants who can arrive at decisions. 'Actor' as used here may refer to an individual person or to a group, organization or network: all interact, taking and implementing decisions on the basis of their own perceptions, interests, agendas, understandings and the opportunities they are able to see.

In general terms, the purpose of an agricultural knowledge and information system (AKIS) is to facilitate agriculture-related practices to be innovated continuously. The system integrates farmers, agricultural educators, researchers and extensionists to harness knowledge and information from various sources for better farming and improved livelihoods. Even when such a broad intentionality is recognized, the specific mission of an AKIS is permanently debated amongst social actors in one way or the other affected by agricultural performance. AKIS can be improved, e.g. made more sensitive to the needs of poor smallholders, using RAAKS, see http://www.kit.nl/4616?tab=2.

Processes and institutions produce results that meet the needs of society while making the best use of resources at their disposal. It also means sustainable use of natural resources and the protection of the environment.

as far as M&E is concerned, demonstrating to donors, beneficiaries and implementing partners that expenditure, actions and results are as agreed or are as can reasonably be expected in a given situation (Jim Woodhill, 2006).

A network of organizations, enterprises, and individuals that focuses on bringing new products, new processes, and new forms of organization into economic use, together with the institutions and policies that affect their behavior and performance. Innovation systems not only help to create knowledge, but also provide access to knowledge, share knowledge, and foster learning.

People, groups, or institutions that are likely to be affected by a proposed intervention (either negatively or positively), or those which can affect the outcome of the intervention.

SSM as a methodology lends itself particularly well for dealing with complex situations where those involved lack a common agreement on what constitutes the problem that needs to be addressed. The aim of SSM is to bring about improvements in a situation perceived as problematical. As such, it does not seek to solve ‘the problem’ but to facilitate a learning process which allows its users to gradually develop a more comprehensive understanding of the situation under study. In contrast to hard systems, such as computer or transport systems, soft systems are mental constructs for human understanding. When applied to agricultural knowledge and information systems (AKIS), the AKIS could be described as a system to help farmers escape poverty, or a system to increase agricultural production, or a system to develop technologies that have a high theoretical yield potential. SSM supposes a systems approach as described in systems thinking.

building the capacity, self reliance and confidence of beneficiaries and implementing staff and partners to effectively guide, manage and implement development initiatives (Jim Woodhill, 2006). Also known as capacity development, capacity strengthening, or empowerment.

Typically, complex problems are difficult or impossible to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognize. Because of complex interdependencies, the effort to solve one aspect of a such a problem may reveal or create other problems. They often occur in a systems context, where they are also known as “wicked”  or “messy” problems.

Evaluation processes and activities that support program, project, product, personnel and/or organizational development (usually the latter). The evaluator is part of the team whose members collaborate to conceptualize, design, and test new approaches in a long-term, ongoing process of continuous improvement, adaptation and intentional change. The evaluator’s primary function in the team is to elucidate team discussions with evaluative data and logic, and to facilitate data based decision4making in the developmental process” (Patton, 1993).

Double-loop learning is a concept in which an organization or network (or individual) is able to change its approach toward solving a difficult or complex problem. It has close links with “learning how to learn”. Also known as second-order learning.

Relates to the extent to which an intervention induces change and has produced (or is likely to produce) the desired and/or desirable effects (SFOPH, 2005)

Emergent insights or ideas arise out of participants or stakeholders yet are novel (innovative) and irreducible with respect to them. These new and particularly pertinent insights could not have arisen outside the “community of inquiry” of which the participants or stakeholders are part. The pertinence of the innovation derives from the group. See also: social learning.

The use of evaluation concepts,  techniques and findings to foster self- determination. It’s an approach that aims to increase the probability of achieving program success by providing program stakeholders with tools for assessing the planning, implementation, self-evaluation of their program. (Donaldsen et al., 2010)

Evaluation is the process of determining the value or worth of a program, course, or other initiative, toward the ultimate goal of making decisions about adopting, rejecting, or revising the innovation. It should not be confused with assessment, which encompasses methods for measuring or testing performance on a set of competencies. Evaluation is the more inclusive term, often making use of assessment data in addition to many other data sources. See also: Summative evaluation, Formative evaluation.

guidance of a group in a problem-solving process. The facilitator is neutral in regard to the issues or topics under discussion. The facilitator works with the group as a whole and provides procedural help in moving toward a conclusion. (Air University).

evaluation of the evaluand (programme, training) during its development. The focus is on improvement.

the medium-to long term effects of a public measure often in relation to its overall aims/ goals. Such changes can be direct or indirect, intended or unintended, positive or negative.

A programme logic model describes a programme and the assumptions that make the causal link between the planned measures and expected effects. Logical framework (LogFrame) is one of the tools that can be used in this process. A logic model shows what the programme is supposed to do, how (measures and activities), for whom and for what purpose (aim/goal). It can also identify which indicators should be used when judging the results and the factors that may influence the programme’s success or failure.

a set of principles which underpin the action undertaken in a specific situation (Checkland, P., & Poulter, J., 2006).

In classical M&E systems, monitoring is the routine collection of comparable data in relation to the indicators established to track progress against a plan. This is likely to include monitoring of inputs, outputs, processes, outcomes  and impacts. Both evaluation and performance review can use and interpret monitoring data for their respective needs. In an innovation context, monitoring is extended to all those aspect that may contribute to social, multi-stakeholder or organizational learning.

These are the likely or achieved short-to medium term effects of a public

measure in relation to its objectives.

These are the activities, goods and services directly produced by a public measure e.g. brochures, reports, workshops, hotline services, computer programs etc. which can be quantitatively or qualitatively measured.

The outputs, outcomes and/or impacts (intended or unintended, positive or negative, direct or indirect) of a public measure.

learning among participants or stakeholders. In systems thinking, learning is the emergence of new knowledge among stakeholders in the course of a conscious, usually facilitated session or series of learning sessions.

examines the evaluand (programme, training) after it has been finalized. The focus is on data compilation.

definition depends a bit on the methodology, e.g. in SSM a tool is a “rule” for creating conceptualizations. In contrast, RAAKS has about 20 tools that are activities that contribute to innovative learning. Some of the tools existed before a methodology was developed, while others were specifically developed for it.

a participatory, democratic process concerned with developing practical knowing in the pursuit of worthwhile human purposes, grounded in a participatory worldview. (Reason, P., & Bradbury, H. (2001). Handbook of action research : participative inquiry and practice. Retrieved from: http://vlc.bath.ac.uk/carpp/publications/pdf/handbook_of_action_research.pdf).