Module 4
Menu index & Legend
This one slide provides direct links to each section of the document
1.1 Overview Section 1
Title: How to use impact – oriented project management to achieve the
desired results
Format: online
Duration: 60 min
Specifics: learners will use their smartphones
At the end of the module the learner will
have a complete picture of the different stages of the project cycle and how to connect expected outcome with tangible and intangible results
-Tangible
results(physical outputs
– products, services,
infrastructure, etc.
that can be measured)
and
Intangible results (internal outcomes such as improved understanding; changed attitude, enhanced user experience / customer satisfaction, etc.)
During the training module the trainer will present the module in detail, solve questions and discuss with the learners everything that was presented in the module.
1.2. Overview
Title: Impact oriented project management
Format: online
Duration: 90 min
Specifics: learners will use their smartphones
At the end of the module the learner will
have a complete picture of the different approaches to prepare and implement dissemination, exploitation tools, as well as to use different monitoring and evaluation strategies
-How to formulate goals – SMART approach
-Dissemination and exploitation plans – products, services that can be measured)
and
-Evaluation and monitoring plans – what, when and how to measure
During the specific training module the trainer will present the module in detail, solve questions and discuss with the learners everything that was presented in the module.
1.3 Overview
Title: How to guarantee project?
Format: online
Duration: 30 min
Specifics: learners will use their smartphones
EXAMPLE
At the end of the module the learner will be able to apply follow-up methods to assure project sustainability and report impact
EXAMPLE
-Follow-up approach – transfer to new target groups
– Commercialization of technology, products and services
EXAMPLE
During the specific training module the trainer will present the module in detail, solve questions and discuss with the learners everything that was presented in the module.
2. Introduction
A life is not important except in the impact it has on other lives.
~ Jackie Robinson
Measurement is the first step that leads to control
and eventually to improvement.
If you can’t measure something, you can’t understand it.
If you can’t understand it, you can’t control it.
If you can’t control it, you can’t improve it.
~ H. James Harrington –
American businessman, author, and improvement guru
3. The Tool
HOW THE TOOL WORKS?
4.1 Key feature 1 - Project cycle
The project impact assessment is not just counting numbers and checking indicators. Тo achieve impact, a project has to be generated with the concept of making a positive change.
The key steps are:
- Identification of problem area and project idea
- Analysis of the situation – needs, challenges, resources, opportunities
- Design of the project logical framework and project application
- Implementation of the project plan to achieve the desired impact
- Evaluation – ongoing monitoring of quality indicators and impact assessment, which leads to lessons
learned and identification of new opportunities
4.2. Start with WHY
Important questions to ask when starting a project are:
- Why do you start your project? (need)
- What do you want to change in the world? (objective)
- Who’s life is going to improve thanks to this? (target groups)
- How? (outcomes)
- What will you do to achieve this? (activities)
- How do you formulate success? (indicators)
- How do you measure progress? (evaluation methods)
- What lasting changes did you achieve? (impact)
4.3. Impact levels
A project can have impact on different levels, depending on your objectives.
The more complex problem you try to solve, the more levels of impact your project has.
Policy level (structural changes on institutional level that affect all stakeholders in the field or large groups of society – introduce a new national program/ initiative/ law / regulation, establish an agency, adopt budget, sign a bilateral agreement between countries, Brexit, etc.)
Systemic level (outcomes that will make tangible improvements in a sector – economic, social, youth, etc. – and involve policy actors and key decision-makers: change a curriculum, start a new national program for professionals, etc.)
- Geographic scope (local, regional, national, international)
- Capacity (develop sustainable conditions, environment, infrastructure, build ecosystem and resources – establish a new center, train professionals, equip laboratories, set networks, etc.)
- End-users (specific benefits and outcomes for individuals and communities – enhanced access/ householding conditions, developed skills, found jobs, improved quality of resources and services, etc.)
The impact monitoring has to verify the change at each level and validate that this change is a result of your project efforts.
4.4. Type of impact
Project results can be:
- Tangible (physical outputs – products, services, infrastructure, etc.
that can be measured) and - Intangible (internal outcomes such as improved understanding;
changed attitude, enhanced user experience / customer satisfaction, etc.)
Intangible results are:
- short-term outcomes – established contacts, raised awareness;
- medium-term outcomes – improved skills, public image;
- long-term impact (sustainable results) – changed mindset / behaviour,
4.5. Success indicators
Impact can be assessed via:
Quantitative indicators (number of people who participated, % of students who improved their performance, share of graduates who found job 1 year after leaving school, etc.)
Qualitative indicators (improved understanding, willingness to participate, positive attitude towards an innovation introduced, improved business environment, reduced bureaucracy, improved skills, etc.)
It is important to set SMART indicators which can be measured and evaluated. This means, that qualitative indicators should also be quantified.
For example: at least 75% of the participants are “satisfied” and “very satisfied” with the completed course.
4.6. How to measure impact
- Ex-ante – ex-post evaluation: measure the indicators you aim to improve before and after the implementation of the project activities.
- Pilot and control groups: At the beginning of the project make a situation analysis. Present the project objectives and involve a sample of the target population as a pilot group, which will directly take part in the project activities and will benefit from the developed resources and outcomes. Interviews, testimonials and feedback can provide valuable insight for the benefits from the project. Use semi-structured questionnaires that involve open questions: “How did the project help you? / What did the project improve for you?”
- Avoid unnecessarily large assessment scales (from 0 to 10) if you are only interested if the participants are completely satisfied, rather satisfied, rather not satisfied and not satisfied at all (4 scales). Neutral doesn’t give
a meaningful information – it is a way to avoid answer. Better use “I didn’t use / I don’t want to answer.”
4.7. Impact-oriented project management
Project management is the combination of activities, processes and responsibilities, which aim to deliver the planned outcome within the approved timeline and budget. It includes management and coordination of human resources, financial management, time management, administrative and reporting arrangements, risk management, communication and quality management.
Useful resources:
Overview of Process and Outcome Evaluation
Example of a monitoring and evaluation template
4.8. Logical framework
The logical framework (logframe) is a table that lists the project activities, outputs, medium term outcomes, and long term goal. It shows the logic of how the activities will lead to the outputs, which in their turn will lead to the
outcomes, and ultimately the goal – the desired social impact. It will make your proposal more coherent, logical, appropriate and successful.
4.9. Logical framework - example
4.10. Exercises Key Feature 1 Project Cycle
Prepare short logframe matrix for a project, display the key elements of a project design and their relationships to each other in a way that facilitates project analysis.
5.1. Key features 2 - Impact-oriented project management
Project management is the combination of activities, processes and responsibilities, which aim to deliver the planned outcome within the approved timeline and budget. It includes management and coordination of human resources, financial management, time management, administrative and reporting arrangements, risk management, communication and quality management.
Useful resources:
Overview of Process and Outcome Evaluation
Example of a monitoring and evaluation template
5.2. Impact-oriented project management
- Following the rules: Strictly observe the program rules and the project description. Use the planned trainings for beneficiaries, webinars and other forms of communication with the donor.
- Planning: Prepare a detailed project implementation plan with timeline, deadlines and specified tasks and responsibilities. Ensure productive cooperation and smooth communication.
- Keep track of progress: Make a list of steps in your project to monitor your process. Focus on them one by one and move to a new task only after completion of the previous one. Keep daily record of project activities to report staff work and organize it more effectively.
- Organize meetings to plan and review your progress. Envisage face-to-face meeting on the critical points and regular Skypes to keep stakeholders focused and engaged. Keep minutes.
- Document sharing: Keep the project documentation in shared online storage such as Google Drive, where all contributors can easily and quickly access it. Introduce rules how to organize files.
5.3. Impact-oriented project management
- Monitoring and quality management: use piloting, monitoring, quality checks, feedback sessions, etc.) and respective indicators to assess whether the project reaches its objectives.
- Risk management: review and update risks, collect feedback and monitor implementation in order to guarantee that all parties are equally committed, involved and on track.
- Promoting the project. The project management includes also active communication with your target groups and stakeholders to disseminate the results across them.
- Ensuring sustainability. Make necessary provisions that the produced results will remain available and used after the end of the funding period, and to ensure the resources needed to sustain them.
- Finalization. Check deadlines and list of outcomes promised in the project proposal. Summarize the work done and find ways to complete unfinished tasks. Compare actual results with expected ones. Describe your results and expenditures in a final report. Celebrate the work done! Express personal thanks to all contributors and team members.
5.4. Formulate it SMART
Setting SMART objectives helps envisage the impact for the target groups, identify measurable indicators to monitor the progress and achieve relevant and meaningful project results.
5.5. Dissemination and exploitation
The keys to a successful project and for producing social impact is producing relevant results to satisfy the demands of end-users and the main actors in the field – providers, various stakeholders and policy-makers. You may have produced wonderful, innovative, high quality products, but if nobody knows about them and uses them, all efforts are meaningless. This is why information and involving target users and stakeholders is so important part.
- Dissemination is spreading information to ensure that the produced results reach the target audiences. At the beginning of a project we use promotion and awareness-raising to inform target users and relevant stakeholders about the aim of the project and its expected outcomes. When the results are ready, they are spread out across the target users and key actors who may increase the outreach and potential impact of the project.
- Exploitation means putting in use and taking benefit from the produced results.
It may be done through: mainstreaming – transferring the successful results of project to appropriate decision-makers in regulated local, regional, national or European systems; and/or multiplication – convincing individual end users to adopt and/or apply the project results.
5.6. Dissemination plan
- The dissemination plans describes the information channels (traditional, social media, networks, specialized editions, service providers, hotlines, etc.), materials (videos, websites, leaflets, etc.) and activities (public events, seminars, information campaigns, meetings with stakeholders, contests, etc.) to spread out the results of the project.
- The dissemination and exploitation plan should take in mind target users’ needs and characteristics. The proper mix of dissemination channels, activities and messages need to ensure that the results reach the right target audiences and engage them. For example, a flashmob or webinar may not be adequate for elder participants, while a conference will hardly attract young audience.
5.7. Dissemination for better impact
- Involve end users to design your dissemination campaign most effectively. They can advise you of the right dissemination approaches, channels or messages. Use their input during development and piloting to make necessary adaptations. Involve them in your promotional videos/articles – the target group will associate with them. Include them in the public events to present the results, give feedback and act as project ambassadors.
- Networks. Involve in your project key stakeholders, decision-makers and professionals in the fields targeted by the project, as well as partner organizations who are expected to incorporate the project outcomes at national, regional, sectoral or system level.
- Outreach. Plan wider dissemination to guarantee you reach your targets. Spread messages to a broader group of potential users/beneficiaries, and involve key stakeholders as project promoters.
5.8. Dissemination for better impact
- Budget. The project should envisage adequate resources to allow for the proper implementation of your dissemination plans. Depending on your plans, you may need to envisage licenses, software, hosting, design, publication and translation costs, as well as expenditures for organization of public events – hall and equipment rent, coffee breaks and catering, etc.
- Monitoring. List the activities and indicators that will help you measure your effectiveness – for example: number of visitors, number of multipliers attracted, number of publications, commercialisation agreements; quantitative – feedback from seminars, conferences, etc.
- Sustainability. Dissemination and exploitation plans should contribute to the viability of the results after the end of the project, to ensure maximum impact and sustainability.
5.9. Monitoring and evaluation
Monitoring and evaluation are planned activities for measuring the project progress according to the plan for implementation (monitoring) and the accomplished results of project against its objectives (evaluation).
Monitoring is an ongoing process to check results and verify the progress towards the set objectives by documenting the process, collecting data and comparing it with the success indicators.
Impact evaluation / assessment is the process of analyzing the results at a given point (for example – interim and final stage) – what is going on well, what are the challenges, what are the causes of it. It examines more difficult, intangible outcomes.
5.10. Monitoring and evaluation plan
The monitoring and evaluation plan defines all the indicators, data sources, tools and activities for measuring, responsibilities and frequency of checks, and ways of reporting.
Useful resources for monitoring and evaluation plan:
Overview of Process and Outcome Evaluation
Example of a monitoring and evaluation template
5.11. Monitoring and evaluation
Depending on the type of funding program, the monitoring and evaluation can be done internally (by a project partner) or by external experts hired by the partnership or by the donor.
Evaluation can has different focus and purpose in the project:
- Process evaluation is a periodic review which measures how well the project activities and tasks are implemented. It helps monitor performance and progress; get feedback from involved participants and provides an early warning for any problems that may occur.
- Outcome evaluation measures the effect of the project on the target group, and their overall satisfaction with the project results.
- Impact evaluation is done at the end of the project. It shows to what extent the project reaches the desired systemic/long-term effect and provides evidence for further actions (such as mainstreaming, scaling, policy initiatives, funding decisions, etc.)
5.12. Monitoring and Evaluation plan
A Monitoring and Evaluation plan (also called Quality Assurance plan)
is a detailed table that sets out the main evaluation questions, the specific monitoring indicator of success (or quality indicators), and data sources.
It is based on the logical framework of the project and answers the basic questions:
- How we define the success of our project?
- What do we want to know?
- What is the question we will ask?
- How will we know it?
- What indicators will we use?
- What data source and methods will we use?
- Who will collect the data?
- When will the evaluation occur?
- Who will be involved?
- How will it be reported?
(Source: Cabinet Implementation Unit Toolkit)
5.13. Monitoring and Evaluation plan
How to prepare a monitoring and evaluation plan?
- For each indicator listed in the project logical framework table describe precisely what the indicator is and how it will be measured.
- When possible, use standard indicators and benchmarks that are commonly used (for example: such as poverty index/unemployment rate/education attainment rate, etc.) in order to compare the project results with national/European statistics.
- Define each indicator. Specify how the indicators are calculated and the data source you will use, otherwise results can’t be compared.
- Define how frequently the indicators will be measured (monthly, quarterly, annually, etc.)
- Measure the baseline (the initial value of each indicator at the start of the project) and set the target you want to reach within the project.
- Research reliable national sources to collect data about the baseline. Make sure you use the same definitions for calculating the indicator.
- Set realistic targets – research in advance what previous initiatives have accomplished.
5.14. Impact Evaluation
The impact evaluation reports what difference has the project made, what solutions have been brought, what community change or system development has been started in a long-term perspective. It summarizes the main achievements and the lasting effects of the project, particularly related with the problems addressed in the proposal.
5.15. Impact Evaluation
It should explain:
- How many people have been affected by the project and in what way? Are the project results relevant to the needs of the target groups? Are they consistent with the project objectives and impact, set in the proposal?
- Are the project objectives met? Were the project objectives achieved on time and in the most efficient way? What were the major factors influencing the achievement or non-achievement of the objectives and were they adequately addressed?
- Are there any additional (non-planned) impacts (economic, social, policy, environmental) of your project? In what way (positive/negative) do they affect the target group/population?
- Do the benefits of the project continue after the funding period?
- Consider the potential impact of your project in terms of the economy, the environment, society, and culture – the key elements of sustainable community development.
5.16. Impact Evaluation
Instruments for impact evaluation
Tiny tools for impact assessment
How to think about and measure nonprofit impact
World Bank – what is impact evaluation
UNICEF – Building Blocks of Impact Evaluation
UNICEF Impact Evaluation guide
Useful resources for impact evaluation
How to create a monitoring and evaluation (M&E) system
– step-by-step guide
5.17. Monitoring and Evaluation plan
The evaluation is based on collecting relevant data, which may be done with different qualitative and quantitative methods: piloting sessions/ presentations/ trainings, collecting feedback from target groups and stakeholders with structured/semi-structured questionnaires, monitoring of results (Google analytics, number of visitors, trainees, downloads), focus groups and discussions, interviews with professionals, field visits.
5.18 Exercise Key features 2 Monitoring and Evaluation plan
Prepare short Monitoring and evaluation plan, display the key elements of a project that you consider
6.1. Key features 3 Sustainability
Critical parts of the impact evaluation are the plans for ensuring viability and sustainability of results. All donors look for evidence that the funding was invested to produce results, which are relevant, useful and can be scaled to produce maximum value. Your project should ultimately pay for itself and long-term viability of the project must be clearly demonstrated. Explain how the initiative will be sustained after completion, which elements will remain available, how and who will maintain them. A project that requires ongoing long-term support from external sources is considered less viable in long-term perspective.
6.2.Follow-up (sustainability)
Sustainability plans may include:
- Incorporating the project results into systems and practices, for example by having them recognized/certified in the case of training/educational tools. The involvement of key stakeholders and decision makers is essential for mainstreaming the project results.
- Ensuring free access to project results after the completion of the project. In this case envisage the resources (human, technical and financial) for update, support and maintenance.
- Scaling of project results –organizations with genuine interest and capacity should be identified in advance. These could be related institutions which can mainstream the results, or other stakeholders, who can disseminate them across their beneficiaries/members. The transfer of know-how/methodologies is made through training and francizing. In case of transfer of project results, specific agreement should be made on intellectual property rights and sustainability.
6.3.Follow-up (sustainability)
Sustainability plans may include:
- Transfer to new sectors/target groups/countries – in some cases the project results (good practices, methodologies, know-how, innovative services, etc.) could be exploited in different that the original fields and communities of target users. In these case feasibility tests should be made and the necessary adaptations and modifications have to be made.
- Commercialization of technology, methodology, licensing after the funding period. For example: you may offer training for trainers, sell know-how to manufacturers, offer paid consultations, envisage paid access to some section of your platform or publish paid banners, etc. Prepare your marketing plan and intellectual property rights agreement with your project partners.
Explain how feasible each of these options is, what kind of resources and measures have been taken into account to ensure that the results remain available and used. Finally, the impact evaluation should draw conclusions and outline lessons learned, which can be used in future projects.
6.4. Useful resources for sustainability
Useful resources for ensuring projects sustainability
- How 5 Organizations Plan to Scale Social Enterprises
- How to write the sustainability section of a proposal
6.5.How to report impact
Refer to the project priorities and objectives and answer the following questions:
What sustainable results did the project produce for each target group?
What sustainable outcomes did the project deliver?
What did the project change? For whom?
Compare success indicators with achieved results, to quantify your achievements. Analyze the results:
What external factors and resources have contributed to these results?
What external factors have challenged the project?
Check the results for the pilot participants and the control group to validate the impact of your activities.
6.6.Exercise Key features 3 Sustainability
Explain how feasible Transfer to new sectors/target groups/countries and Commercialization of technology, methodology, licensing after the funding period are for a project you are running, what kind of resources and measures have been taken into account to ensure that the results remain available and used. Finally,
Remember! The impact evaluation should draw conclusions and outline lessons learned, which can be used in future projects.
Use the YI+ tool for achieving better results.
7.1. Self-assessment
Referencing
If you have any problem with referencing check the following website:
Quick guide to Harvard referencing (Cite Them Right) | Library Services | Open University
