1.0 Project background and context
The Building Capacity for Resilient Food Security Project (BCfRFSP) in Tanzania is a three-year USAID-funded project aimed at addressing capacity gaps within Tanzania’s Ministry of Agriculture (MoA), Ministry of Livestock and Fisheries (MLF), Ministry of Agriculture, Natural Resources, Livestock and Fisheries (MANRLF) Zanzibar, Tanzania Meteorological Authority (TMA), and the President’s Office Regional Administrative and Local Government (PO-RALG) to effectively respond to the challenges faced by the country’s agriculture sector as a result of unfavorable changes in climatic conditions. MoA, MLF, MANRLF, PO-RALG, and TMA work alongside the three contracted agencies as implementing partners; namely. The United Nations Food and Agriculture Organization (FAO), International Center for Research in Agroforestry (ICRAF), and International Institute of Tropical Agriculture (IITA) to develop and implement specific project capacities geared towards building agricultural resilience and food security. The project aimed at building the capacity of the United Republic of Tanzania (URT) Agencies, Ministries, and institutions to support the adaptation and scaleout of climate-smart agricultural practices and technologies among smallholder farmers in the country.
This is one of the projects implemented under the Agriculture Climate Resilient Plan (2014‒2019) developed by MoA, which aims at strengthening knowledge and systems to target climate actions in the country. A key pillar of the plan is to build the capacity of the relevant agencies to develop and implement policies and projects to address climate variability/change.
The objectives of the project were for USDA to support the capacity building of MoA, MLF, and MANRLF to strengthen implementation of their national climate change and agricultural development policies, such as the ACRP and ASDP. The project has five core capacities prioritized in the project to promote the primary concern of adaptation and resilience actions for Tanzania:
Capacity 1: To determine the potential benefits and tradeoffs of Climate Smart Agriculture (CSA) practices and technologies under different local scenarios by region and production system
Capacity 2: To select sustainable improved production and risk management practices and technologies for specific production systems and regions and develop technical specifications for those practices**
Capacity 3: To develop demonstration and dissemination activities and products (print and electronic) to inform and engage local government authorities, extension staff, academics, students, service providers, agribusiness leaders, and policymakers how to apply practices and technologies and achieve agricultural resilience**
Capacity 4: To ensure that all agriculture extension trainees know the approaches to achieving a resilient agricultural sector, applicable practices and technologies, and how to modify the application for different regions and cropping systems.
Capacity 5: To convert agro-meteorological data and analyses into timely and actionable information available to farmers
The project seeks to engage a local consultant/firm to evaluate the BCFRFSP project under the guidance and supervision of the MoA, MLF, MNRLF, and PORALG.
2.0 Objective of the evaluation
The main objective of this evaluation is to assess the project goal and expected results as defined in the project document. This evaluation will provide information that is credible and useful, enabling the incorporation of lessons into management decision-making more specifically to:
Evaluate the relevance, efficiency, effectiveness, impact, and sustainability of the project and support with evidence the extent to which the project met its objectives.[1]
Document lessons learned—both successes and shortcomings of the project in pursuing its intended objectives to generalize the best practice and strategic recommendations of the future CSA projects.[2] **
3.0 Scope of evaluation and methodology
3.1 Scope of evaluation
The evaluation will be conducted in all project areas in Kongwa, Dodoma City, Mvomero, Kilolo, Ruangwa, Kaskazini B, Kusini Unguja, Wete, and Chake Chake–Pemba in both Tanzania Mainland and Zanzibar. The consultant will adopt a consultative and participatory approach, including desk review, interview, and field visits to project sites to collect first-hand information. Meetings and methods will specifically target key project beneficiaries; Regional and District Government Staff (RAA, DAICOs/DADOs), Extension Officers, Project Implementing Partners (IPs), farmers, and other potential stakeholders.
3.2 Methodology
The project’s evaluation approach will be consultative and participatory . It will deploy qualitative and quantitative data collection methods, including semi-structured and structured interviews with focus groups, key informants, large groups, individuals, and as necessary. The consultant will develop an appropriate theory of change (ToC) and a detailed evaluation methodology that must include, but not be limited to, evaluation design and data collection, units of randomization and analysis, and method of randomization. Concerning impact evaluation criteria, the methodology must describe intermediate and final outcome indicators and how they will be measured, giving a detailed presentation of power calculations with minimum detectable effect size and assumptions used for the estimates. The consultant is expected to employ a Theory-Based Impact Evaluation (TBIE) to answer the question of what worked (by measuring or assessing the changes brought about by interventions), but also why and how it worked (by examining the processes that led to those changes). The consultant will be responsible for performing the following tasks:
Desk review: This will include review and analysis of the following documents: Agricultural Sector Development Program Phase II (ASDP II), Zanzibar Agricultural Sector Development Program, TMA Mid Term Strategic Plan (2020/21-2024/25), District Agricultural Development Plans (DADPs), Medium Term Expenditure Framework (MTF), National Climate Smart Agriculture Guideline, Zanzibar Climate Change Strategy (2014-2019), Agriculture Climate Resilience Plan (ACRP 2014–2019), BCfRFSP Project document, BCfRFSP Baseline Report, BCfRFSP Implementation Reports, BCfRFSP Monitoring Reports, Evaluation and Learning (MEL) Plan, BCfRFSP ICM reports, BCfRFSP-URT Monitoring Reports, and BCfRFSP Work plans, Financial statement reports, Audit report, Project agreement, and any other relevant document and webpage, etc. Stakeholder’s consultation to review relevancy, effectiveness, efficiency, impact, and sustainability of the project.
· Interviews with project beneficiaries: The consultant will establish an appropriate sample size from direct beneficiaries and the control group to be studied regarding evaluation objectives. They will hold interviews using questionnaires (with closed and open questions) that will be administered to sampled farmers participating in the project and non-participants, taking into account social, economic, and demographic profiles. **
· Semi-structured interviews and focus groups: The consultant will conduct semi-structured interviews with MoA, MLF, MANRLF, PO-RALG, IITA, FAO, ICRAF, RS, and LGA officials. Individual meetings and key informant interviews will be held and applied to themes to explore the issues openly, thus allowing the interviewee to bring up new ideas and approaches. FGDs will also be held with project participants who have first-hand information about BCfRFSP implementation processes, achievements, and challenges. The number of FGDs should be determined to validate and triangulate information and findings from the surveys and the document reviews, semi-structured interviews, and focus groups. **
3.3 Data analysis
The consultant is expected to employ a rigorous analytical approach to answer the objectives of this evaluation. A mixed approach of qualitative and quantitative methods is expected.
4.0 Evaluation criteria and questions
This evaluation encompasses the different stages of the given project, including its design, process, results, and impact and is structured around four main criteria: relevance, efficiency, effectiveness, and sustainability. Within each of these criteria, a set of evaluation questions will be applied to guide the analysis. The responses to these questions are intended to explain “the extent to which,” “why,” and “how” specific outcomes were attained.
4.1 Relevance
- Were the project’s objectives aligned with the needs/priorities of the implementing districts?
- Did the design of the project align or integrate with the existing structures?
- Were there any complementarities and synergies with the other work being developed?
4.2 Efficiency
Collaboration and coordination mechanisms between and within the Ministries and agencies (MoA, MANRLF, PORLAG and TMA, TARI, ZARI), Region, and Districts that ensure efficiencies and coherence of response
- Did the governance and management structures of the project contribute to the effective implementation of its operations and coordination of partners?
- How economic resources/inputs were converted into results
- Are there ways this project could have operated more efficiently in regions or districts?
4.3 Effectiveness
- To what extent did the project achieve the intended capacity-building objectives?
- How satisfied were the project’s main beneficiaries with the quality and timeliness of the services they received (to the extent measurable)?
- Has the project made any difference in the behavior/attitude/skills/knowledge and performance of the clients? If yes, how?
- How effective were the project activities in enabling capacities and influencing policy? What capacities were built? What policy influence occurred?
- Are there any tangible policies that have considered the contributions provided by the district in relation to the project under evaluation?
vi. What services did the beneficiaries find most satisfactory and why?
vii. What services did the beneficiaries find less satisfactory and why?
4.4 Sustainability
i. Are there lasting benefits after the intervention has been completed?
ii. What actions would you recommend to enhance the impact of the project?
iii. What is needed to sustain the project outcomes?
4.5 Impact
i. What changes did the project bring about?
ii. Were there any unplanned or unintended changes?
5.0 Deliverables
The consultant is expected to deliver the following:
- Inception report: No later than 10 working days after the engagement, the consultant should deliver the inception report. The inception report should include the theory of change (ToC) and a detailed evaluation methodology that must include, but not be limited to, evaluation design and data collection, units of randomization and analysis, and method of randomization. Concerning impact evaluation criteria, the methodology must describe intermediate and final outcome indicators and how they will be measured, giving a detailed presentation of power calculations with minimum detectable effect size and assumptions used for the estimates. The inception report should include data collection instruments and partners that will be contacted to obtain the evaluation information. A detailed work schedule will be developed and agreed upon between the consultant and the client. The inception report should contain a separate section/paragraph on mitigating risks describing in some detail how the consultant intends to address Covid-19 related constraints in the methods proposed.
- Draft final evaluation report: No later than 5 weeks after the engagement, the consultant should deliver the preliminary report for revision and comments by the URT M&E Team, IPs, and USDA, which should include the main draft results and findings of the evaluation, lessons learned, and recommendations, including its sustainability, and potential improvements in project management and coordination of similar CSA projects;
- Final evaluation report: No later than 8 weeks after the engagement, the consultant should deliver the final evaluation report, which should include the revised version of the preliminary version after making sure all the comments, observations, and recommendations from the URT M&E Team and the implementing partners have been included. Before submitting the final report, the consultant must have received the clearance on this final version from the URT M&E Team, assuring the satisfaction of ICM with the final evaluation report;
- Presentation of the evaluation report: A final evaluation report to be presented to the ICM.
- Submission of evaluation report: The consultant should deliver all documents related to evaluation in its original version, two hard copies and an electronic copy in English.
6.0 Terms and conditions of the consultancy
- The duration of the consultancy will be 60 mandays from mid-April to mid-June 2021. The consultant will be reporting to and be managed by the United Republic of Tanzania-BCfRFSP Monitoring and Evaluation Team under the Environmental Management Unit (MoA-EMU) and (MANRLF–DPPR). The payment will include the consultant’s fees and direct operational costs (for enumerators, transport, stationaries etc.)
7.0 Consultant profile
- The evaluation exercise will be undertaken by a Local Individual Consultant/firm who will collaborate with BCfRFSP coordinators under Environmental Management Unit (MoA-EMU) and (MANRLF–DPPR). The consultant/firm should possess the following skills and qualifications
- The consultant, or team leader, must possess a Master’s Degree in Economics/Agricultural Economics, Rural/Regional Development Planning, Project Management, Monitoring and Evaluation, or related areas.
- Demonstrated experience and hands-on and progressive knowledge of the theory of change (ToC) focused and results-based evaluation methods.
- Experience in managing evaluations and documenting learning for complex cross-sector projects.
- Good command of written and spoken English and Kiswahili with excellent organization and communication skills and demonstrable ability to meet deadlines and work under pressure
- A minimum of three (3) years’ experience in performing similar evaluations (this applies to an individual or people specifically nominated rather than to a firm);
- Demonstrated experience in research design, including designing data collection instruments and processing and analyzing data.
- Strong facilitator with documented experience in conducting participatory qualitative assessments related to behavioral change, conservation, and/or livelihood projects.
- Strong organizational, analytical and reporting skills, presentation skills, attention to detail, ability to meet deadlines, proficiency in Microsoft Office and, potentially, qualitative data analysis software/tools.
- The consultant should be familiar with the local context and have some expertise in at least some of the project’s technical content.
- Proven knowledge and practical ability to understand and utilize, if not design and implement, quantitative data and qualitative data.
- Proven competency in quantitative and qualitative research methods, particularly self-administered surveys, document analysis, and informal and semi-structured interviews
8.0 Structure of the Report
- The final evaluation report shall be written in English (maximum 50 pages excluding Annexes) and contain: Executive summary, introduction, methodology, key results/findings, conclusion, recommendations, lessons learned, and annexes; organized into four sections.
How to apply
9.0 Procedure for application
Qualified applicants must submit an application, which must include: Technical proposal; financial proposal; two-page resumé of Principle Investigator (PI) and the team specifying how they qualify for the assignment. All applications along with all necessary documents must be submitted as a PDF file electronically to: The International Institute of Tropical Agriculture (IITA), IITA-Tanzania, Plot No 25 Mikocheni Light Industrial Area, Mwenge Coca-Cola Road, Mikocheni B. Email: c.njuguna@cgiar.org with the subject line “BCfRFSP End-line Evaluation” before 5.00 pm on 31 March 2021.