Anywhere
Posted 3 weeks ago

Organization: World University Service of Canada

Closing date: 26 Sep 2023

Terms of Reference – Endline Evaluation

Adolescent Girls’ Education in Crisis Initiative (AGENCI) Project

1. Introduction

The Adolescent Girls’ Education in Crisis Initiative (AGENCI) is a five-year (2020-2025), $17.97 million project to enhance the empowerment of adolescent girls and female youth (AGFY) pursuing educational pathways in crisis-affected areas of South Sudan, Syria, and Uganda. It targets over 123,000 adolescent girls and female youth aged 10-24. It is a joint initiative of World University Service of Canada (WUSC) and Aga Khan Foundation Canada (AKFC), with financial support from Global Affairs Canada (GAC).

WUSC is a Canadian non-profit organization working to create a better world for all young people. WUSC brings together a diverse network of students, volunteers, schools, governments, and businesses who share this vision. Together, we foster youth-centered solutions for improved education, economic, and empowerment opportunities to overcome inequality and exclusion in over 15 countries across Asia, Africa, and the Americas. WUSC has a project office in Juba, South Sudan and in Moyo and Kampala, Uganda.

AKFC, leveraging resources from the wider Aga Khan Development Network (AKDN), promotes development in Africa and Asia, tackling the root causes of poverty through a holistic and interconnected set of programs. These initiatives address people’s most pressing needs and enable them to unlock their full potential, with a particular focus on marginalized groups like women and girls. AKFC has a country office in Kampala and a presence in Syria.

WUSC and AKFC have entered into a consortium for the implementation of AGENCI, with WUSC as the project lead. Aligned with a consortium agreement, WUSC solely implements the project in South Sudan and AKF solely implements it in Syria. For project activities in Uganda, WUSC and AKFC implement jointly.

2. Background of the AGENCI Project

AGENCI is designed to address the gender, social, cultural and economic barriers faced by the targeted population in accessing education and making life decisions. Rapid gender analyses and baseline studies were conducted in the three countries of implementation in 2020 and 2021 to develop the AGENCI Gender Equality and Social Inclusion Strategy and Project Implementation Plan (PIP). The baseline study reports, as well as the PIP and all annual donor reports, will serve as key documents to inform the design of the endline evaluation.

The project seeks to improve equitable learning outcomes for adolescent girls and female youth in formal and non-formal upper primary school, secondary school, and in skills training programs. AGENCI responds to specific barriers and educational needs identified by adolescent girls, female youth, community members, and education stakeholders through gender analyses conducted in the three countries. AGENCI addresses the root causes of inequality by challenging and helping to transform structural barriers and entrenched social norms, attitudes, and power relations that impact girls’ ability to access and benefit from education.

AGENCI is implementing a range of innovative and tested interventions including media campaigns, community outreach, girls’ and boys’ clubs, flexible response funds, and girls’ mentorship programs. Working with education stakeholders, AGENCI provides training and support to ensure that teachers and educational leaders have the skills, resources, data and information they need to provide girls with safe, quality, gender-responsive and inclusive learning environments. The project addresses two critical moments in girls’ education when they are most at-risk of dropping out of the education system: adolescence (primary to secondary) and transition to adulthood (secondary to vocational training or employment). The specific interventions employed by WUSC and AKFC in South Sudan, Uganda, and Syria are based on the needs, priorities and realities of each context.

3. Purpose and Objectives of the Endline Evaluation

AKF and WUSC are seeking qualified consultant(s) or a consulting firm to lead a multi-country, mixed method endline evaluation. This endline study is a contractual obligation included in the project’s Contribution Agreement (CA) with Global Affairs Canada. The purpose of the AGENCI project’s endline study is to provide key project stakeholders with evidence on the project’s achievements related to a set of expected outcomes and to empower them to collaboratively draw learnings and identify recommendations to foster sustainability after the closure of the activities. Aligned with this purpose, and based on the methodology designed for the baseline study as much as possible, the endline study is expected to achieve four specific objectives:

To establish endline values and analysis for key performance indicators of the AGENCI Performance Management Framework (PMF)across all AGENCI target geographies, as well as measures of the level of achievement of each target. Quantitative data should be triangulated with qualitative data gathered through Focus Group Discussions and Key Informant Interviews;
To assess the validity of the assumptions identified in the project’s Theory of Change (ToC) and validated at baseline;
To provide an overall assessment of the AGENCI project, based on a set of the OECD-DAC evaluation criteria;
To identify lessons learned and recommendations for partners and stakeholders, to foster the sustainability of the achieved results and promote the replicability of successful interventions.

4. The Mandate of the Consultancy

The Consultant (or consultancy firm) will serve as the technical lead for this endline evaluation and will be responsible for revising and updating the methodology and tools developed for the baseline study, coordinating through local AKF and WUSC country units in Uganda, South Sudan, and Syria to facilitate data collection and interpretation, analyzing the collected data, preparing a final consolidated report with individual country annexes, and inserting the data into the Performance Measurement Framework. The Consultant will design and facilitate a plan in close collaboration with the WUSC and AKF project management teams based in Nairobi, Ottawa and Geneva, as well as in the three countries of the study. Specifically, the mandate of the consultancy is to:

Revise/design the endline methodology and tools: Work with designated AKF and WUSC staff during an inception phase to revise the study design developed at baseline (including data collection methodologies, tools, data analysis plans, etc.) for the purposes of the endline study. This includes finalizing all timelines and necessary contingency plans;
Support in-country collaboration/input on endline design: Ensure collaborative adaptation of the overall endline design to the local context through the direct involvement of key project staff and identified implementing partner representatives;
Ensure endline design complements other ongoing AGENCI studies: Coordinatewith AKF and WUSC’s teams leading other studies and research under AGENCI to ensure that the methodology and tools will exploit synergies with those studies and fill information gaps;
Coordinate with country teams on data collection: The consultant will not be responsible for deploying data collection tools on the ground or recruiting/managing enumerators (this will be the responsibility of each AGENCI country team). Rather, the consultant will be responsible for training relevant staff and data collection teams on the use of the data collection tools/methodologies.
Guide data quality control processes: Provide guidance to country teams on quality assurance during sampling, data collection, data entry, and preliminary data cleaning. Conduct data spot-checks and quality assurance on submitted datasets, following up with teams as needed on data quality issues;
Conduct data analysis and compile consolidated findings: Analyze the submitted data from the three different countries in line with the endline data analysis plan, integrate the findings to create a final consolidated endline study report (including country-level annexes) that meets the objectives outlined above in section 3.

The endline study will be jointly managed by WUSC and AKFC, with close collaboration between the two organization-specific Canada-based MERL technical leads, and with country teams in the three target countries. A more detailed consultative and decision-making management structure with clear roles and responsibilities will be finalized during the inception phase of the study.

5. Scope and Methodology of the Endline Evaluation

The following elements should guide the revision of the design and implementation of this endline study:

The study should consciously apply qualitative and quantitative research methods that are gender and youth-sensitive, and engage as much as possible in participatory and consultative approaches;
The geographic scope of the study will be aligned with the targeted areas, schools, households and individuals targeted by the project as laid out in the baseline report and all other relevant documents;
The data collection approaches and specific tools used to establish indicator endline values should be adapted to the local contexts in the three countries, to assure the appropriateness and reliability of the data collected.

The endline study is expected to collect data for all outcome indicators in the PMF. Outcome statements, indicators, sources and data collection methods for each indicator are included in the project PMF and summarized in Table 1 below. While the Consultant, informed by discussions with country-based project teams is encouraged to make suggestions regarding methodologies, the PMF outlines the following data collection methods or evidence streams that were used during the baseline and should ideally be replicated at endline:

Literature and document review, including reference to rapid gender equality analyses conducted in the three countries, baseline reports and annual reports, outcome monitoring reports as available, and extraction of data points from education management information systems, whenever available and appropriate;
Key informant interviews;
Facilitated small group or focus group discussions
1. with a broad target group of girls, boys and adolescent youth, and
2. with narrower group of enrolled female and male learners;
Surveys, at school and/or household level.

The Consultant will be expected to review the methodology/tools used at baseline for collecting data on all AGENCI Project Outcome Indicators and propose detailed refinements as appropriate, using a combination of the above mentioned qualitative and quantitative methods.

Table 1: AGENCI Outcome Indicators and Data Collection Methods

Intended Project Outcome

Indicator

Level / Data Source

Tool and mode of data collection

Frequency of data collection

Household-Level Data

Enhanced empowerment of adolescent girls and female youth pursuing educational pathways in crisis affected areas of South Sudan, Uganda, and Syria

Degree to which adolescent girls/female youth make decisions about their health, education, assets, and finance, and personal domains (by age, refugee/IDP/host status, country)

Household/ AGFY

Primary: Household survey (HHS), Triangulation: Focus Group Discussions (FGDs)

Baseline/ Endline

Reduced social, cultural and economic barriers to accessing safe, secure, gender equitable and quality education by adolescent girls and female youth, including barriers such as SGBV and SRHR that are exacerbated during crisis

Proportion of girls, boys, men and women in the communities reporting/demonstrating concrete support towards girls’ education (by sex, age, refugee/IDP/host status, stakeholder type, country)

Household/ Community

HHS (primary), FGD (triangulation)

Annual

1100.3. Degree to which adolescent girls/female youth state that they are able to overcome barriers to pursuing education, such as COVID-19 (by refugee/IDP/host status, country)

Household/ AGFY

HHS (primary), FGD (triangulation)

Annual

Improved equitable learning outcomes for adolescent girls and female youth benefitting from quality, gender responsive, innovative and inclusive formal and non-formal educational opportunities in crisis situations

% of female youth aged 18-24 reporting to be engaged in paid work (by country, refugee/IDP/host status)

Household/ AGFY

HHS (primary), FGD (triangulation)

Annual

Increased capacity of adolescent girls and female youth to make decisions about their own lives and educational opportunities

Level of self-reported knowledge of adolescent girls and female youth to make decisions about their educational opportunities and personal domains of wellbeing (by country, refugee/IDP/host status)

Household/ AGFY

HHS (primary), FGD (triangulation)

Annual

Improved knowledge and skills of local community members and leaders, including men and boys, to promote education for adolescent girls and female youth

% of girls, men, and women with improved knowledge of barriers for adolescent girls and female youth accessing education (by age group (AGFY vs parent), refugee/IDP/host status, country, gender (parent))

Household/ Community

HHS (primary), FGD (triangulation)

Annual

Level of support adolescent girls and female youth feel they have from women, men and community leaders to access education (by refugee/IDP/host status, country)

Household/ AGFY

HHS (primary), FGD (triangulation)

Annual

Improved capacity of teachers and educational leaders to deliver and support safe, quality, gender

responsive and inclusive education for adolescent girls and female youth,

particularly IDPs, refugees and those with disabilities

% of girls and female youth reporting to feel safe (by country, refugee/IDP/host status)

Household/ AGFY

HHS (primary), FGD (triangulation)

Annual

School Enrolment & Performance Data

Enhanced empowerment of adolescent girls and female youth pursuing educational pathways in crisis affected areas of South Sudan, Uganda, and Syria

% of adolescent girls and female youth who successfully progress and complete learning levels (by age, refugee/IDP/host status, learning level, institution type, country)

School Data

Document review (School enrolment and exam completion data)

Baseline/ Endline

% overall attendance rate of adolescent girls and female youth in project supported schools (by country, refugee/IDP/host status, grade level)

School Data

Document review (School/attendance records)

Annual

% of adolescent girl and female youth learners who achieve minimum proficiency OR at or above grade level in a) reading, b) mathematics and c) leadership (by country, competency, refugee/IDP/host status)

School Data

Learning Assessment Data

Annual

Increased equitable access to resources and services for adolescent girls and female youth to pursue upper-primary, secondary and skills-based education

# of out of school (OOSC) learners newly enrolled in formal/non-formal learning opportunities (by learning level, refugee/IDP/host status, type of institution, country)

School Data

Document Review (School/attendance records)

Annual

Classroom Level Data

Improved capacity of teachers and educational leaders to deliver and support safe, quality, gender responsive and inclusive education for adolescent girls and female youth, particularly IDPs, refugees and those with disabilities

% of targeted formal and non-formal teachers who demonstrate increased knowledge and skills in pedagogical practices (by country, gender)

Schools / Educators

Educational leader survey

Annual

% of targeted formal and non-formal educational leaders who demonstrate increased knowledge and skills in issues related to promoting inclusion and pluralism (by gender, country)

Schools / Educators

Educational leader survey

Annual

Increased access to gender-responsive and inclusive teaching and learning materials, curricula, educational services and learning environments to address gender-related barriers experienced by adolescent girls and female youth

% of targeted learning environments meeting the minimum quality and secure learning environment standards (i.e. that are gender responsive, inclusive, environmentally sound and with appropriate resources and learning materials) (by country)

Schools / education centre

Classroom observation

Annual

Level of satisfaction of educators with the availability of gender responsive and inclusive teaching and learning materials, curricula, educational services and learning environments (by gender, country)

Schools / Educators

Key Informant Interviews

Annual

Education Stakeholder Level Data

Improved capacity of education stakeholders to effectively utilize gender sensitive data and information to inform coordinated school-level, project and policy decisions related to raising learning outcomes

Degree to which education stakeholders use available gender sensitive data and information for school-related decisions (by gender, country)

School leaders / Educators / government / teacher training institutes

Key Informant Interviews (primary), document review (triangulation)

Annual

6. Key Activities and Deliverables of the Consultancy

The endline evaluation will include four key phases:

Contracting
Inception phase
Data collection
Data analysis and report writing

After successfully completing the contracting process, the Consultant will immediately organize an initial round of consultations with key stakeholders, as well as complete an initial document review to better understand the project, its contexts and its informational needs. The Consultant will use the inception phase to assure a consensus on the endline study’s design, outline the details of the revised methodology for the study, and update the data collection tools.

Following approval of the inception report by AKF and WUSC, the Consultant will train the enumerators selected and recruited by the country teams (with the support of the country team as appropriate and agreed upon during inception). This may involve up to 1 week of travel to each AGENCI country to conduct in-person training (or training of trainers). In cases where this is not possible, remote training should be conducted.

Once data collection is completed, the Consultant will focus on the analysis of collected data and drafting of the final report, which will include an updated PMF, as well as recommendations on targets whenever appropriate. During this phase, the Consultant will engage with AKF and WUSC Advisors and Project Managers to refine the final report.

Based on the implementation timeline in the three countries, the initial draft of the final report for South Sudan must be shared with the country team and key stakeholders for validation no later than the 28th of February 2024. The final endline evaluation report for all three countries must be submitted to AKF and WUSC on September 30, 2024.

An overall timeline for the key deliverables of the three phases is laid out in the table below and will be confirmed during the inception phase.

Inception

Data collection, data analysis and report writing

Key Activities, Main Data Collection Tools, and Milestones

Inception meeting to confirm understanding of mandate
Literature and document review
Consultations with project teams (both in countries and their global teams) on the refinement of the methodological details of the study as well as all the data collection tools
Draft and submission of inception report
Submission of the ethical protocols, as required by the relevant national authorities
Identification, recruitment and training of enumerators
Training of data collectors in each country, supervision by the consultant on data quality as needed
Data analysis, leading to the calculation of the endline value for each of the outcome indicators in the PMF
Draft and submission of endline report
Incorporation of feedback from AKFC and WUSC and submission of final report for approval

Phase-Specific Deliverables

Inception report, including: detailed revised methodology and its limitations, all revised data collection tools and consent forms, detailed workplan in each country, development of enumerators training protocol and material, and delivery of enumerator training.

(A detailed table of contents for the inception report will be shared at the beginning of the contract)

Facilitation of a validation workshop in each of the countries of the study, with all project teams, partners and key stakeholders;
Final endline report, including: updated PMF with baseline and endline values for all outcome indicators, limitations to the methodology used for the study, lessons learned and recommendations on the sustainability of achieved results, evaluation insights for future replicability.

(A detailed table of contents for the final report will be shared during the inception phase)

All raw data

Tentative deadline for deliverables

Inception report: 23rd October 2023

South Sudan

Data Collection: Nov 2023 – Jan 2024

Summary report: 28th Feb 2024.

Validation workshop: 20th Mar 2024.

Uganda

Data Collection: May-June 2024

Summary report: July 2024

Validation workshop: August 2024

Syria

Data Collection: March – July 2024

Summary report: July – August 2024

Validation workshop: August 2024

Final Consolidated Report: September 2024

7. Ethical Standards

The research for this endline study must consciously abide AKFC and WUSC’s Codes of Conduct and Child Protection and Safeguarding Policies, as well as by relevant recognized professional and ethical guidelines and codes of conduct for individual researchers. The Consultant must also be sure to obtain the relevant research permits, as required, from all relevant national authorities (i.e. Mildmay Uganda research and Ethics Committee (MUREC). Codes of conduct must be specifically referenced and commitment confirmed in writing by each researcher involved in this study. The design of the study must clearly specify how data collection and use will be undertaken with integrity and honesty, respecting human rights and differences in culture, customs, religious beliefs and practices of all stakeholders. The Consultant must explain how its researchers will be mindful of gender-related needs and roles, ethnicity, ability, age, language and other differences when designing and carrying out the study. The design and implementation of the study must strike an appropriate balance between recognition of the potential benefits of the research, and the protection of participants from potential research-related harms. Safeguarding principles to protect key informants from sexual exploitation and abuse, sexual harassment and bullying, including child protection, must be explicit.

8. Qualifications of Consultant and their team

Lead researcher/firm must demonstrate a minimum of 10 years of experience in administering mixed-method studies, collecting data and producing quality baseline/end line study reports, preferably in education and for international non-profit organizations or multilateral agencies;
Demonstrated experience in designing baseline and end line studies including proven experience in participatory research methods and feminist evaluation approaches;
Demonstrated experience of delivering research in complex settings, with refugee/IDP populations, and in similar contexts to the AGENCI target geographies;
Excellent facilitation skills and ability to recruit and manage enumerators for both the quantitative and qualitative components, in the three countries of the study (either in person or remotely);
Demonstrated experience in quantitative and qualitative data analysis;
Knowledge and experience with education policies in Syria, Uganda and/or South Sudan will be considered an asset;
Knowledge and experience with livelihoods or skills training programs;
Knowledge and experience in gender equality issues in the education sector is mandatory; specific experience in data collection on gender norms is desirable;
Fluency in English is mandatory and other relevant languages (Arabic, Juba Arabic, Madi, Dinka and/or Azande) an asset;
Demonstrated capacity to respect and safeguard vulnerable populations; and
Ability to produce high quality work under tight timeframes.

9. Application Process

The project teams will be in charge of the selection, recruitment and supervision (as appropriate and needed) of the enumerators and all logistical aspects and budget related to the field work. Additional details on this aspect of the process will be discussed during inception.

Qualified and interested parties are asked to submit the following:

Detailed technical proposal (8 to 9 pages maximum) clearly demonstrating a thorough understanding of this ToR and including the following:
1. Description of the capacity and qualifications of the consulting firm, including previous relevant experience (1.5 to 2 pages)
2. Description of the proposed approach and methodology, including sampling strategy, data analysis, integration of gender considerations and ethical standards, quality assurance (3.5 to 4 pages)
3. A proposed schedule/work plan (i.e. a Gantt chart), including contingency planning as appropriate (1 page)
4. Team composition and level of effort of each proposed team member (1 to 1.5 pages pages)
A financial proposal (in CAD) with a detailed breakdown of costs for the study:
Itemized consultancy fees/costs
Itemized administrative expenses
Validity period of quotations
Expected payment plan and method

(Please note that the budgets with itemized field data collection expenses are currently being developed by the country teams and will be shared with the selected consultant/s upon contract signing.)

Curriculum Vitae of all proposed team members outlining relevant experience (max 3 pages per CV)
Names and contact information of three references who can be contacted regarding relevant experience
A copy of a previous report of similar work undertaken

Complete applications should be submitted electronically to: lstanley@wusc.ca with the subject line of: ‘AGENCI Project Endline Study Application’.

Closing date for submission of the application package is September 26th 2023*.*

How to apply

9. Application Process

Qualified and interested parties are asked to submit the following:

Detailed technical proposal (8 to 9 pages maximum) clearly demonstrating a thorough understanding of this ToR and including the following:
1. Description of the capacity and qualifications of the consulting firm, including previous relevant experience (1.5 to 2 pages)
2. Description of the proposed approach and methodology, including sampling strategy, data analysis, integration of gender considerations and ethical standards, quality assurance (3.5 to 4 pages)
3. A proposed schedule/work plan (i.e. a Gantt chart), including contingency planning as appropriate (1 page)
4. Team composition and level of effort of each proposed team member (1 to 1.5 pages pages)
A financial proposal (in CAD) with a detailed breakdown of costs for the study:
Itemized consultancy fees/costs
Itemized administrative expenses
Validity period of quotations
Expected payment plan and method

Curriculum Vitae of all proposed team members outlining relevant experience (max 3 pages per CV)
Names and contact information of three references who can be contacted regarding relevant experience
A copy of a previous report of similar work undertaken

Complete applications should be submitted electronically to: lstanley@wusc.ca with the subject line of: ‘AGENCI Project Endline Study Application’.

Closing date for submission of the application package is September 26th 2023*.*

Source ⇲

To apply, please visit the following URL:

Endline Evaluation – Adolescent Girls’ Education in Crisis Initiative (AGENCI) Project

Weekly Opportunities .