Terms of Reference for Evaluation
South Sudan Joint Response 2019-2021 November 2021
This document provides the Terms of Reference for the external evaluation of the Dutch Ministry of Foreign Affairs funded South Sudan Joint Response running from January 2019 to December 2021.
The Dutch Relief Alliance
The South Sudan Joint Response is implemented by the Dutch Relief Alliance (DRA). In response to the challenges of the humanitarian system and the growing gap between humanitarian needs and humanitarian funding, the Dutch Minister for International Trade and Development Cooperation in 2015 set up a Dutch Relief Alliance to increase effectiveness of Dutch humanitarian aid. The DRA is an alliance of 14 Dutch NGOs which respond to chronic crises as well as acute crises, for which they receive funding from Netherlands Ministry of Foreign Affairs (MFA). The main objectives of the Dutch Relief Alliance are to deliver effective, efficient, relevant and timely humanitarian aid through collaboration to more beneficiaries in a better way. DRA partners implement Joint Responses (JRs) to address protracted crises enabling capacity building, localisation and investment in community resilience. The four strategic pillars of the DRA include collaboration, localization, accountability and innovation.
The South Sudan Joint Response
The South Sudan Joint Response is implemented by 7 DRA partner organizations: Save the Children (lead agency), Tearfund, Plan International Netherlands, Dorcas Aid International, CARE Netherlands and Help a Child, who work together with 9 local partner organizations: ACROSS, CEF, CEDS, WDG, MHA, HDC, UNIDOR, WOCO and SAADO to provide lifesaving assistance. The period under review runs from 1 January 2019 to 31 December 2021 during which the partners implemented three annual programs, each phase a continuation of the previous one. The budgets for each consecutive year were EUR 6.050.010, EUR 7.923.698 and EUR 6.352.482. We have reached 151.416 and 467.896 beneficiaries in 2019 and 2020 respectively and plan to reach
167.087 beneficiaries in 2021.
The SSJR program is implemented in counties in South Sudan with highest needs and where DRA partners can work together to generate collaborative impact. In 2019, 2020 and 2021 these locations were: Pibor (Jonglei), Koch (Unity), Malakal and Fashoda (Upper Nile), Aweil East (Northern Bahr el Ghazal), and Wau and Jur River (Western Bahr el Ghazal). SSJR is providing a multi-sectoral integrated response with the aim of holistically meeting the needs of the affected population. In 2019, the response focused on Food Security and Livelihoods (FSL), WASH, Protection, and Multi-Purpose Cash. In 2020 and 2021 the response focused on FSL, WASH, Protection, Multi-Purpose Cash, Nutrition.
The target group consists of Internally Displaced Persons (IDP), returnees and host communities. Among these, vulnerable groups have been identified especially child-headed households, female headed households, Pregnant and Lactating Women (PLW) PLW, elderly, households with children facing Severe Acute Malnutrition (SAM)/Moderate Acute Malnutrition (MAM), and people with disabilities.
The SSJR program is designed as an adaptive and flexible grant structure which encourages a high level of cooperation across partners with the aim of delivering quality programming through continuous learning, adaptive implementation and information sharing. Moreover, the high level of collaboration including joint activity planning, sharing of resources and integrated
programming is leading to increased efficiency and effectiveness of the actions of all agencies in the joint response.
DRA and MFA both value collaboration, gender mainstreaming and disability inclusion, localization, accountability to affected population, protection mainstreaming, and innovation. Therefore, these cross-cutting themes are being addressed in the programming.
SCOPE OF EVALUATION
The SSJR partners have a contractual obligation towards the donor, the Dutch Ministry of Foreign Affairs, to ensure the realization of an external evaluation. The evaluation will cover the period 2019-2021 during which the SSJR partners delivered three annual humanitarian programs in South Sudan
Objective 1 Results achieved: The evaluation report will assess the overall performance of the SSJR against selected OECD DAC criteria and the Core Humanitarian Standards, ensuring accountability towards the Dutch Government, the Dutch public and the beneficiaries of the program. It should be noted that the consolidated logframe with project results on indicator level for 2021 will only be available towards the end of the evaluation (March/April 2022), while project results for 2019 and 2020 are available already.
Objective 2 Learning: Evaluate to what extent the adaptive management system and learning strategy of the project was effective.
Objective 3 Accountability to affected populations: Evaluate to what extent relevant information on the program and partner organisation has been shared with the communities; to what extent communities were able to participate in all phases of the project cycle (from design up to evaluation processes); to what extent communities were aware and have made use of feedback mechanisms and to what extent SSJR partners were able to close the feedback loop.
Objective 4 Collaboration: Evaluate to what extent the collaboration between SSJR partners contributed to reaching beneficiaries and targets in a more effective, efficient, relevant and timely manner, as set in the log frame and in the narrative proposal. In addition, evaluate to what extent the collaboration between SSJR partners has led to increased positive impact on the lives of the beneficiaries.
Objective 5 Localization: Evaluate to what extent the SSJR programme has contributed to increased capacity and ownership of local actors including local NGOs, CSOs, community structures, local authorities etc. To what extent have local partners gained the capacity to continue the delivery of humanitarian aid beyond the SSJR programme.
While the evaluation covers all JR locations and partners, the suggested locations to be visited are Malakal, Wau/Jur River and Koch1.** Malakal was evaluated during the final evaluation of the SSJR 2018. Going back to this location will allow us to determine to what extent the recommendations from the 2018 evaluation have been taken into account in programming going forward. Wau & Jur River and Koch have not been visited before, but are locations continuing as part of the SSJR 2022-2023 program, where collaboration (both between INGOs and NNGOs) has been at the core, and hence could provide interesting opportunity for learning. Naturally,
1 SSJR partners working here are: WCH, Plan, SAADO, WOCO (in Malakal); HAC, Dorcas, MHA, WDG (in Wau/Jur River) and CARE, UNIDOR and HDC (in Koch)
considering the volatile situation in South Sudan, the accessibility and security of these locations will need to be verified prior to the evaluation.
We have listed a range of possible questions below. The consultant in consultation with the SSJR partners is expected to propose a selection of the most relevant questions per objective as part of the inception report, which are realistic within the given scope, time, and budget. The final questions should cover all five objectives and include enabling and hampering factors to achieve progress, and consultant should indicate whether data is gathered on-location or centrally. We envision a ‘quality over quantity’ approach, and consultants should not feel obliged to include all questions. Considering the scope & nature of objectives 2 & 3, it is proposed that the consultant uses a workshop style approach to data gathering.
Objective 1 Results achieved
How relevant have the objectives and activities of the SSJR programme been in addressing humanitarian needs in South Sudan?
To what extent was SSJR able to adapt and provide appropriate response to context changes and evolving needs and capacities, and the priorities of the people, taking into account the specific needs of the most vulnerable groups, including women, children and disabled persons?
To what extent and how was protection mainstreamed across the program activities?
To what extent and how was gender sensitive programming implemented?
To what extent and how was disability inclusion mainstreamed across the program activities?
To what extent have the planned results (outputs, outcomes) been achieved in all the phases of SSJR?
What factors and/or actors were crucial for the achievement or failure to achieve the project objectives?
What value do beneficiaries, stakeholders and communities attach to the results achieved?
How did the project coordinate with and/or complement other similar actions in the field – geographical and thematic?
How did partnership with local NGOs inform DRA partners implementation process?
Was the process of achieving results efficient? To what extent has the SSJR programme been implemented in a timely and cost-effective manner?
Was there good value for money for the activities undertaken?
To what extent has the joint response model of collaboration led to cost-effectiveness?
How has SSJR impacted the lives of crisis affected population (IDPs, returnees and host communities) in sustaining their lives through integrated life-saving humanitarian assistance?
What has changed as a direct result of the SSJR intervention? Are there unintended positive or negative consequences of the SSJR programming on target population and/or on national and international SSJR partners?
Which groups have been affected (or unaffected) by the changes?
To what extent does the intervention reflect and consider factors which have a major influence on sustainability, i.e. economic, ecological, social and cultural aspects?
To what extent have the outcomes and results of the SSJR program been sustained after the previous years of the South Sudan Joint Response and to what extent will they be sustained after the completion of the project?
What are the contributing factors and constraints that require attention in order to improve prospects of sustainability of the project outcomes?
How have localization efforts contributed to sustainability of project activities?
Objective 2 Learning
To what extent has the SSJR collaborative way of working facilitated peer-learning between SSJR partners?
What kind of learning activities have been most effective according to SSJR partners?
Did partners make any changes to their programming as a result of these learnings, leading to better quality program?
Is there any substantial anecdotal evidence on how activities to increase learning have affected the delivery of humanitarian aid by SSJR partners?
How did we learn from previous phases of the joint response? What have we improved compared to phase 1 – 4 and how did we improve between 2019 and 2021? To what extent did we implement the recommendations of the SSJR 2018 final evaluation and the 2019 and 2020 learning exchange visits?
What have been the reasons for (not) implementing the recommendations? What has been the result of the decision to implement or not implement?
Objective 3 Collaboration
Is there any substantial (anecdotal) evidence of how collaboration between SSJR partners, specifically related to complementarity, has led to positive impact on the lives of beneficiaries?
Are there any concrete examples of successful models of collaboration of SSJR partners on geographic level affecting the reach and impact on beneficiaries? What are barriers and/or enablers to this?
To what extent, and how, did SSJR coordinate activities with external parties, i.e. national Government, UN OCHA, non-NJR INGO partners?
How have the comparative advantages2 of different DRA partners contributed to the effectiveness of the joint response (either in individual programming or in collaboration), both at location-level and at a broader level?
Objective 4 Accountability to the affected population
How and to what extent have communities (IDPs, host communities and returnees) been involved in the development of selection criteria to identify the most vulnerable?
To what extent did SSJR partners involve beneficiaries in project design (e.g. development of indicators), implementation (e.g. community committees) and MEAL (e.g. monitoring progress)?
How and to what extent did SSJR partners provide opportunities for community members to provide feedback?
2 Comparative advantage refers to the ability of individual actors to contribute to humanitarian effectiveness where they can clearly and consistently add value. This will increase impact, predictability and quality
- How did the joint accountability system put in place by the SSJR partners (“The Joint AAP system”) influence feedback and complaints reporting by beneficiaries? How inclusive are these feedback channels? Are there any concrete example of adjustments in the program accordingly to beneficiaries’ feedback and preferences?
Objective 5 Localization
- To what extent have capacity strengthening activities fulfilled the needs and priorities of the participants (especially local partners); what influence has this had on local
partners’ way of implementing? Provide concrete examples of how it contributed to
their work and improved program quality benefitting the beneficiaries.
To what extent have SSJR localization efforts contributed to the capacity of local partners to fund, design, and deliver humanitarian actions in South Sudan? What factors played a role in this?
To what extent did local partners feel empowered / felt ownership in the design and implementation of the SSJR programme? What factors played a role in this?
What has been the impact of the SSJR localization efforts on the localization practices of the DRA INGO partners?
What has been the impact of the SSJR localization efforts on the localization agenda of the humanitarian community in South Sudan?
The evaluation will be carried out in a transparent and participatory manner by involving relevant stakeholders (UN, Cluster representatives, SSJR partners and affected people). The COVID-19 pandemic is a key consideration at the time of finalizing the methodology, however a mixed-method approach is anticipated including but not limited to the following methods:
• Desk study and review of all relevant program documentation and monitoring data.
• Key Informant Interviews with key stakeholders (Project teams, senior officials of national and international partners, Cluster leads, Government and UN agencies)
• Questionnaire for SSJR partner staff
• Focus Group discussions with target population
• Household surveys with target population.
The use of creative or participative qualitative methods (to eg. draw out and document learnings) is welcomed. The evaluation should be inclusive taking into account gender, age, disability, and other vulnerability considerations, sensitive of social norms and practices, and be considered of ethical data collection. In addition, the evaluation methodology should consider consultations with child and youth beneficiaries and highlight key approaches for undertaking it:
Safety and ethics considerations for engaging children in evaluation
Data collection methods which are age and gender appropriate
ROLES AND RESPONSIBILITIES
SSJR coordination team
· Starts the evaluation process;
· Prepares and publishes the ToR for the selection of the consultant;
· Participates in the consultant selection process;
· Is the contact person for the consultant;
· Leads and/or coordinates the evaluation
· Makes key documentation available from coordination level
· Make necessary logistical and security arrangements to receive data collection team. This includes organizing and providing a security briefing upon arrival. The JR lead organization however makes final decision on security-related matters;
· Informs key staff, crisis-affected people and other relevant stakeholders of the upcoming visit;
· Makes key documentation and background information available to the consultant;
· Makes sure that key staff, crisis-affected people and other relevant stakeholders are available for participating in interviews and focus group discussions
· Prepares the inception report;
· Prepares the data collection tools;
· Conducts data collection; transcription and data cleaning
· Executes the data analysis;
· Organize validation meeting/workshop with JR partners
· Writes the evaluation report.
Deadline for applications
1 December COB
(Notification to shortlisted candidates will be on 13 December)
Interviews with pre-selected candidates
Selection of evaluation team
17 December (contracting to be finalized by 21 dec)
By 21 January 2022
7 February – 28 February 2022
First draft evaluation report
21 March 2022
Last week March/beginning of April 2022
Final evaluation report
29 April 2022
The timeline detailed above is an indicative timeline.
EXPECTED OUTCOMES AND DELIVERABLES
The assignment shall start in January 2022 with an inception meeting. A first meeting shall be held at the beginning of January before the assignment starts in order to review the ToR and agree on tentative work plan.
An inception report shall be submitted to SSJR Consortium Coordinator detailing the work plan, sampling frame and data collection tools by the 21st of January. The consultant will finalize the
report, incorporating feedback and suggestions coming from the SSJR Coordinator and the SSJR partners.
The consultant will be responsible for data collection including hiring enumerators where required. Data collection and fieldwork by the consultant shall take place in February 2022, possibly extending into March 2022 where needed and the methodology will be mutually agreed upon by SSJR coordination and consultants, keeping the constraints generated by the security situation and the COVID-19 pandemic in mind.
A briefing will be conducted for partners in the beginning of February by the consultant before the start of the data collection.
A draft evaluation report will be ready by end of March 2022. One feedback and verification workshop will be conducted where the main findings are presented to the SSJR partners who will be provided with the opportunity to provide verbal and written feedback.
The final report should be delivered no later than 29 April 2022. The content of the report will be finalized through mutual discussion between SSJR coordination and consultants.
Inception report: The Inception Report will highlight the methodology and the guiding principles of the evaluation. The inception report will include: Objectives and key questions, methodology, evaluation framework/matrix (overview of method and source of information per evaluation sub-question), data collection methods, sampling approach, timeline and logistics, tools to be used for data collection. The report should be no longer than 20 pages.
Data collection and data analysis tools: The evaluator(s) will develop the tools for data collection and data analysis in line with the structure of the tools in the inception report.
Evaluation Final Report (between 30 and 40 pages, annexes excluded, in Microsoft Word format) including tabs and graphs representing the data:
Table of Contents
List of Acronyms
List of Tables
Scope of Evaluation
Methodology (including sampling)
Conclusions and specific recommendations with details how they might be implemented
o Project logframe
o Evaluation ToR
o Objectives and key questions
o Study schedule
o List of people involved
o Bibliography of consulted secondary sources
o Finalised data collection tools
PROFESSIONAL SKILLS AND QUALIFICATIONS
Interested experts/consultancies are required to provide CVs detailing the experience with similar type of assignments completed in the past.
The experts/consultancies should demonstrate the following areas of technical expertise in his/her Expression of Interest:
• Experience in design, planning and implementation of mixed-method evaluation exercises.
• Experience in quantitative data collection and analysis, use of sound statistical methods to identify causal relationships and address threats to internal validity.
• Experience in qualitative data collection and analysis of complex qualitative information, drawing findings from multiple sources and handling potential contradictions between data sets.
• Relevant subject knowledge and prior experience of working on multi-year programming in the emergency and humanitarian sector to ensure that design and research methods are as relevant and meaningful as possible as given in the work scope of this assignment.
• Proven knowledge and experience with using humanitarian sector frameworks for Quality and Accountability (e.g. CHS; SPHERE) in evaluation assignments;
• Statistical analysis: a range of statistical modelling and analysis of impact data; highly proficient user of: SPSS, STATA or equivalent; and qualitative data analysis techniques.
• Language Proficiency: Proficiency in English and the ability to produce good quality written documents in English is a mandatory requirement of this assignment.
Desired expertise and experience
Members of the evaluation team should hold at least a bachelor’s degree in relevant field of study.
The evaluation team should have prior experience of developing research designs that involve remote data collection and management.
Regional experience: it is particularly desirable that the evaluation team has experience working in the South Sudan context.
The consultant will be paid in three instalments with following terms and conditions;
1. 30% advance payment will be released upon acceptance of inception report and tools by assignment focal point.
2. 40% payment will be released upon submission of the draft assessment report
3. 30% payment will be released upon successful completion of assignment in the field and acceptance of deliverables by assignment lead.
Consultant will be paid fixed fee as per signed contract after deduction of all taxes applied by Netherlands Law. Save the Children will cover field travel if required in South Sudan during assignment days. All other costs including international travel, accommodation, life and health insurance, visa and other costs associated for traveling to South Sudan as well as costs for hiring enumerators for data collection, will be borne by the consultant and should be included in the financial proposal.
CHILD SAFEGUARDING AND PSEA
Safeguarding considerations: Ensuring the whole evaluation process adheres to best practice for research with children including the implementation of child safeguarding policy and procedures to ensure safety of participants.
CONFLICT OF INTEREST
The incumbent is required to demonstrate the necessary independence and declare any conflict of interest and potential biases, including bias towards any of the stakeholders, target groups, type of approach etc.
CODE OF CONDUCT
Save the Children’s work is based on deeply held values and principles of child safeguarding, and it is essential that our commitment to children’s rights and humanitarian principles is supported and demonstrated by all members of staff and other people working for and with Save the Children. Save the Children’s Code of Conduct sets out the standards which all staff members must adhere to and the consultant is bound to sign and abide to the Save the Children’s Code of Conduct.
MANDATORY SECURITY TRAINING
The consultant shall be expected to go through mandatory on-line security training and submit the certificate of completion to SC before the commencement of the task. Failure to deliver this will lead to an automatic disqualification. SC will advise on the security plan on appointment.
How to apply:
Save the Children is inviting both individuals as well as teams of consultants to apply for this assignment. Interested candidates who meet the key qualifications and have relevant experience in designing and delivering similar type of assignments must submit their technical and financial proposals to the following email address: email@example.com no later than Wednesday 1 December 2021.
Shortlisted candidates will be notified on the 13th of Decembers. An interview will take place after which the final application will be selected. Interviews are expected to take place on 15 and 16 December.
In case of questions about the assignment or the application process please contact: Anne Nieuwenhuis at Anne.firstname.lastname@example.org
Expression of interest
The Expression of Interest (EoI) should include:
1) A technical proposal (not exceeding 10 pages) that responds to the asks of this ToR. The technical proposal should contain a clear outline of the research methodology (quantitative and qualitative methods), data collection and analysis techniques (and alternatives in case of lack of access), a tentative work plan with clearly defined milestones to achieve within the given timeline of the assignment and a financial proposal.
2) CV for all proposed team members;
3) At least one sample of similar previous work;
4) two references, to be attached as annex to the technical proposal.
Please note that incomplete EoI will not be assessed. Financial Proposal The financial bid must be prepared and submitted with clearly defined breakdown of consultants’ daily fee and other associated costs. Please read carefully section Payment Schedule of the ToR while developing and finalizing financial proposal. Interested applicants should consider that
€25,000 inclusive of VAT is available to conduct this assignment. Please be aware that, along with the quality of the technical proposal and of the sample of previous work submitted, the amount of the financial proposal will also be a criterion for assessing the EoI received.
Save the Children Netherlands reserves the right to revoke the call or stop the process of hiring services without giving any prior reasons to the applicants. Incomplete application, applications submitted after deadline or application without financial proposal or vice versa will not be considered in the selection process.