External Evaluation

  • Development
  • Anywhere
Country: Lebanon
Organization: Terre des Hommes Italia
Closing date: 23 Nov 2020

Terms of Reference Final Evaluation – AICS**

Type of service

Final Evaluation

Name of the project

Access and Improvement of Educational and Child Protection Activities in Lebanon through an integrated approach

Project Start and End dates

14 June 2019 – 28 Feb 2021

Project duration

21 months

Project locations

Lebanon

  • Baalbek-Hermel Governorate (Jdeideh & Masharia el Qaa)

  • Mount Lebanon Governorate (Barja)

Thematic areas

Education; Child Protection

Donor

Italian Agency for Development Cooperation (AICS)

Implementing organizations

Ana Aqra Association

Terre des Hommes Italy

A. ORGANIZATION PROFILE

Terre des Hommes Italia

Terre des Hommes Italia is part of Terre des Hommes International Federation, a network of ten national organizations working for the rights of children and for the promotion of equitable development, free from racial, religious, political, cultural, and gender-based discrimination.

Terre des Hommes Italia globally assists over 880,000 children with its 139 projects carried out in 22 countries, focusing on the fulfillment of the child’s rights and working towards its vision. Its action has mainly helped to ensure immediate and emergency support to populations affected by war or by natural disasters. Terre des Hommes Italia operates in Lebanon since 2006, implementing projects in the fields of protection, child protection, education, livelihoods and community support, in different regions including North Lebanon, Mount Lebanon, South Lebanon and North Beqaa.

Today, Terre des Hommes Italia implements Education and Child Protection activities reaching more than 10,000 children across Mount Lebanon, South Lebanon and North Beqaa areas, in collaboration with national and local government authorities, as well as national and international NGOs, and public and private institutions. Terre des Hommes Italia supports formal and non-formal activities for pre-school and school-aged children through innovative education models, both in education centers and at community level. Our education projects are implemented directly by TDH specialized staff, as well as in partnership with local NGOs specialized in the field of education such as Ana Aqra Association.

Ana Aqra Association

Ana Aqra Association is an independent non-profit organization that advances literacy and autonomous learning for vulnerable children in Lebanese public schools as well as out-of-school children. Founded in 1994, Ana Aqra has evolved from providing classroom libraries in public schools to currently providing educational services to public school children and out-of-school children ages 3-14, as well as addressing the cultural and psychosocial needs of communities within and surrounding the public educational system.

B. PROJECT OVERVIEW

The intervention aimed at promoting access to quality pedagogical paths capable of responding to the educational, psychosocial and protection needs of the most vulnerable boys and girls among refugees and host communities in Mount Lebanon and North Beqaa.

The 18 months’ project aims to support 880 children accessing formal and informal educational opportunities. Moreover, 1060 caregivers, educators and members of the community will be involved in activities to raise awareness and strengthen their skills related to the protection and well-being of children.

Between March and June 2020, the project implementation strategy was modified following the announcement of the COVID-19 global pandemic. Although the objectives and key activities of the project remained the same, the modality of service delivery was updated to ensure provision of education activities through online learning modality.

Project Logical Framework

· General objective of the project: Contribute to strengthening protection mechanisms for vulnerable children in Lebanon through an integrated educational approach

· Specific objective of the project: Promote access to quality educational paths able to respond to the educational, psychosocial and protection needs of the most vulnerable children among refugee and host communities in Mount Lebanon and North Beqaa

· Result 1 – Girls and boys not enrolled in school (preschool and school age) are adequately prepared to enter the formal education system

o Activity 1.1: Provision of educational preparedness activities (Early Childhood Education) for out-of-school Syrian refugee girls and boys

o Activity 1.2: Activation of literacy and numeracy courses (BLN) for children aged between 10 and 14 years

· Result 2 – Provision of academic retention activities (homework support sessions) for vulnerable Lebanese and Syrian refugee girls and boys enrolled in public schools but at risk of dropout

o Activity 2.1: Retention activities for minors at risk of dropping out of school

o Activity 2.2: Provide remote well-being and health safety awareness sessions to learners and their parents/caregivers

o Activity 2.3: Provision of structured, long-term capacity-building for education practitioners

· Result 3 – Caregivers actively participate in educational strategies and in the creation of mechanisms to protect vulnerable children

o Activity 3.1: Design and implementation of awareness-raising activities on issues related to the protection of minors aimed at caregivers

o Activity 3.2: Organization of interventions to involve caregivers on educational strategies through an experiential approach

Modifications related to COVID-19 and distance learning modality **

In coordination with the Education Sector and in line with MEHE recommendations, TDH/AAA took the following actions to adapt the planned activities to distance learning modality:

· Temporary suspended in-person education activities in all learning centres and interruption of all retention activities as these are directly linked to the operationality of the Formal Education.

· Implemented a Learning Readiness Rapid Assessment (LeaRA) with caregivers and targeted households

· Training and capacity of teachers and education staff to adapt to distance learning modality

· Ensured continuation of ECE/BLN/HWS activities through adaptation of curriculum and using remote learning approach

· Ensured continuation of child wellbeing-related activities, also including some education-related messages and health promotion activities

C. SCOPE OF THE FINAL EVALUATION

The project final evaluation is being commissioned in order to understand and document the main results and changes brought on by the project in the lives of children and their families.

The purpose of the evaluation is to assess the extent to which the project objectives have been met. It will assess the project against the OECD-DAC criteria (relevance, effectiveness, efficiency, impact, sustainability) and the project logical framework, with a special focus on the transition from regular education activities to distance learning approach and on the related adaptation of curriculum for children, caregivers and facilitators/teachers/partners.

This will inform Terre des Hommes Italia key impact areas and support in understanding improvement needs for future similar programming.

This final evaluation is intended to be independent external reflection on the programme, which will complement internal reflection cycles implemented by Terre des Hommes Italia.

D. METHODOLOGICAL APPROACHES

Evaluation methodology should be participatory, inclusive (consultations with all stakeholders, such as gender, age, disability and other vulnerability considerations), sensitive of social norms and practices, and ethical in data collection practices (safety, informed consent, etc).

Evaluators are encouraged to mix qualitative and quantitative data and methods, as well as to triangulate data sources where available.

E. KEY RESEARCH QUESTIONS

RELEVANCE: IS THE INTERVENTION DOING THE RIGHT THINGS?

For children and caregivers:

· How relevant to the context were the remote learning approaches adopted for ECE/BLN/HWS activities?

· How relevant to the needs of children were the remote learning approaches adopted for ECE/BLN/HWS activities? What needs were not met and what can be improved?

· To which extent were the needs of children with disabilities met through the remote learning modality? Which adaptations are advised?

· To which extent was the PSS sessions delivery method and topics relevant for the children, including children with disabilities? What can be improved?

· How relevant to the needs of caregivers were the remote learning approaches adopted for ECE/BLN/HWS activities? What needs were not met and what can be improved?

· To which extent was the PSS sessions delivery method and topics relevant for caregivers? Which adaptations are advised?

· Did the project respond to the needs of the family to support them staying engaged in remote learning education activities? How much were the equipment and supplies provided by TDH and partner’s relevant to the families’ needs? What kind of additional support is advisable?

· Was the schedule of the remote learning sessions adapted to the priorities of the family? Did it impact on the capacity to engage in the remote learning activities?

For teachers and education staff

· Was the methodology for remote learning appropriate to the needs of the teachers participating in the project? What can be improved?

· Did the facilitators and teachers receive the appropriate support and trainings to transition to remote learning modality?

· Were the appropriate tools provided to the facilitators and teachers?

· What was the impact of the transition to remote learning on facilitators and teachers, in terms of working hours, work-related stress and skills required?

· How did the teachers and facilitators adapt to the new tools required for remote learning?

EFFECTIVENESS: IS THE INTERVENTION ACHIEVING ITS OBJECTIVES?

· How effective were the remote learning approaches adopted for BLN activities?

· How effective were the remote learning approaches adopted for ECE activities?

· How effective were the remote learning approaches for Homework support activities?

· How effective was the outreach methodology implemented?

· How effective was the school selection process for HWS activities?

· Were the remote learning approaches implemented more effective for certain topics compared to others?

· Were there any extracurricular activities provided as part of the distance learning activities? If not, was there an impact on wellbeing of children?

· How did the transition to remote learning impact the identification of at-risk children in need referral to child protection or specialized services?

· What level of involvement was required by caretakers to facilitate the education activities?

· For ECE program, was there any challenges faced by caregivers in supporting children with foreign language courses?

· How did the physical environment of the children impact their capacity to engage in remote learning activities? Any safety issues experienced?

· What kind of experience did children have with smartphones and technology prior to engaging in distance learning? Did they face challenges adapting to new tools and to interact with teachers and facilitators?

· What personal factors influenced the quality of experience and performance during remote learning? (e.g. children with social interaction difficulties; children with high social needs; different communication styles, etc.)

· What family factors influenced the quality of experience and performance during remote learning? (e.g. high level of stress of parents; many children following distance learning within the household)

EFFICIENCY: HOW WELL ARE RESOURCES BEING USED?

· Were activities cost-efficient?

· Were objectives achieved on time?

· Was the project implemented in the most efficient way compared to alternatives?

IMPACT: WHAT DIFFERENCE DOES THE INTERVENTION MAKE?

· Did the education activities provided through remote modality allow children to reach the required skills according to the curriculum? If not, which skills were not reached?

· To what extent did the distance learning modality contribute to improve the access to quality education for the target children?

· To what extent did the interventions contribute to strengthen protection mechanism/to reduce the risk of dropping out from education/to access the FE?

SUSTAINABILITY: WILL THE BENEFITS LAST?

· In case of need to prolong the distance learning modality for a longer period of time, what could ensure commitment of caregivers and children in the activities?

· In case of need to prolong the distance learning modality for a longer period of time, what could support education staff capacity to facilitate the activities and maintain engagement of children/caregivers?

  • The evaluator can expand and detail further the questions in the technical proposal.*

F. DELIVERABLES

The evaluator is expected to provide the following deliverables as a minimum requirement:

1. Inception report

To outline the research framework, including:

  • Detailed methodology

  • Data collection tools

  • Data collection plan

  • Evaluation Report Outline

  • Detailed workplan with timeline

For approval by TDH IT and AAA

2. Validation Workshop (online)

Workshop to be organized by the evaluator based on the first draft of the report. Objective is to share and discuss findings with project team and partner; and discuss findings, recommendations and conclusions.

3. Final Evaluation Report

Should contain the overall analysis of all the collected and analyzed data, including:

  • Executive summary

  • Methodology (incl. limitation)

  • Desk Review of Distance Learning methodologies

  • Main findings (disaggregated by age, gender, location, disability, beneficiary status – Lebanese/non-lebanese)

  • Conclusions and recommendations

No more than 25-30 pages, written in English.

4. Data and data collection tools

Copy of the original and cleaned datasets (both quantitative and qualitative, including transcription of the interviews and FDGs) and of the data collection/research tools adopted

All the reports should be provided in soft copy and in English, data collection tools should be made in English and Arabic when needed. although they could be translated into Arabic by the consultant for data collection purposes. The drafts of the tools and reports shall be shared with TDH IT for comments and inputs to be included by the consultant. E-mail exchanges and any communication related to the consultancy shall happen in English, while all field-related interviews, questionnaires, focus groups etc. shall be conducted in Arabic or in English depending on the interviewee. The evaluator is responsible for all required translation to English.

Data collection and analysis for this evaluation purpose are to be considered as part of the tasks of the consultant.

G. TIMELINE

Deadline

Evaluation TOR publication 06 November 2020

Submission deadline 23 November 2020

Selection of evaluation team 27 November 2020

Inception report 11 December 2020

First draft of Evaluation Report 01 February 2021

Validation Workshop 08 February 2021

Final Evaluation Report 28 February 2021

H. REQUIREMENTS

The main requirement is the experience in the region, and usage of innovative and mixed methods approaches, including the considerations for remote modality due to COVID-19.

· At least a University Degree in Political Science, Social Science, Education or other relevant fields;

· At least 5 years of experience working in Education sector;

· Proven technical experience related to distance learning and innovative education approaches, especially focusing on marginalized children;

· Experience in remote modality data collection methodologies and tools due to COVID-19 adaptations;

· Proven experience (at least 5 years) in survey design, qualitative and quantitative data collection and analysis;

· Excellent knowledge of the Lebanese context and Syrian refugee crisis;

· Excellent command of written and spoken English, Arabic is an asset;

· Proven experience in evaluating education projects in humanitarian context.

Both individual consultants and teams are eligible to apply.

I. PROPOSAL AND FINANCIAL OFFER

The offer should include:

1) Cover letter highlighting how the profile fits with the required specification and including three reference contacts (1-page maximum)

2) Technical proposal outlining (10 pages maximum):

a. Proposed Methodology

b. Work plan presented both as a chronogram – e.g. a Gantt chart – and as a narrative description

3) CV of lead evaluator and evaluation team (when applicable) including list of previous similar experiences

4) Example of previous report or publication in related evaluation or research

5) Financial offer (in USD), which shall include (list is not exhaustive): stationery, internet/data/phone costs for collection of information, transportation, accommodation and meals (if needed and only during the field mission), fees of the Consultant/s (in the field/for distance support).

6) Payment terms: which includes the currency, the method, the schedule and bank account details, local or external.

The financial offer will be evaluated as gross maximum amount of the contract to be signed in case the offer is selected. Any changes to the financial offer may entail a reassessment of all received offers and could lead to awarding the service to a different provider.

J. CRITERIA FOR THE EVALUATION OF APPLICATION

The selection will be based on the following criteria:

Qualifications and Proven Experience of the firm/person 10%

Technical Proposal 15%

Financial offer 70%

Language and Writing Style, Cleanness of the Proposal 5%

K. OTHER

The consultant shall act in full compliance with TDH IT Child Protection Policy, PSEA Policy and Code of Conduct, which will form integral part of the contract.

If applicable, bidders shall guarantee that the Evaluation Team is entitled to obtain entry visa for Lebanon since TDH will not facilitate the process.

All data, information and internal documents collected and consulted during this assignment will not be shared with third parties without the express permission of TDH IT. Data and information provided by the applicants and/or acquired by Terre des Hommes Italia during the selection process, and afterwards in the course of the consultancy, will be solely processed for the purpose of evaluating the proposals, selecting the consultant and facilitating his/her work, according to the principles of correctness, legality, transparency and in full respect of the privacy and individual rights of the applicants. No information shall be transferred to any third party.

Although the information and documents listed herewith should be sufficient to prepare the proposal, should any of the applicants have questions, he/she could submit them via e-mail to Léane Gaumond (Education Program Manager) L.g.Lacerte@tdhitaly.org.

How to apply:

Proposals may be submitted via e-mail to Léane Gaumond (Education Program Manager) L.g.Lacerte@tdhitaly.org; and Wissam Nasr (Supply Chain Manager) w.nasr@tdhitaly.org; quoting in the subject line “RFP LBDEL0017, AICS External Evaluation, *Name of the proposer”* or by sealed envelopes marked, with the same subject line, to TDH IT offices in Baabda, Ain Street, Najem Residence, 4th Floor not later than November 23rd, 2020. Applications received after that date will not be considered.

The decision on responses or provision of additional information belongs solely to Terre des Hommes Italy, yet any feedback provided shall be shared with all applicants.

Source: