Terms of Reference

for

Mid Term Review

Assignment Summary

Name of the assignment	Mid Term Review (Programmatic and Financial analysis) of the Project “Enhancing Access of Local Communities to Good Quality Optical Services”
Districts	Talagang, Layyah, Rahim Yar Khan, Khairpur and Mitiari
Assignment start and end date (tentative)	1 – 20 November 2024
Assignment Purpose	The primary aim of the mid-term review is to assess the progress to date against the planned objectives, provide lessons learnt and strategic recommendations to inform interventions and practical approaches towards achievement of the project goals as well as sustaining of the project impact.
Commissioning Organisation	Brien Holden Foundation
Assignment Duration	20 days

1.     Introduction

Brien Holden Foundation in partnership with public and private organisations is implementing a project to set up optical services for low-income communities in five districts of Pakistan. The project is funded by CBM Christoffel-Blindenmission Christian Blind Mission e.V. It aims to establish sustainable optical services to provide a variety of subsidised and good-quality spectacles to the local communities.

Project Summary:

The project aims to prevent refractive errors related blindness by developing accessible and good-quality refractive services leading to a comprehensive continuum of eye care in Pakistan. The specifically objectifies to enhance the access of local communities to good quality optical services in two provinces of Pakistan, Punjab and Sindh. The project makes the existing package of eye health services comprehensive by integrating optical services. Similarly, the project introduces a range of optical products including spectacles, sunglasses and reading glasses for all ages and diverse genders. The project contributes to developing an eye health workforce through training at least eighteen persons especially those with disabilities to dispense the spectacles and run the optical shops.

Project Results and Activities:

Result 1: Accessible and good quality optical services are established in the project districts

01.01        Situational analysis about the dynamics of the optical market

01.02        Setting up of optical shops - procurement of optical equipment of optical workshops                     

01.03        Setting up of point of sales including desktop computer, barcode scanner, thermal printer, cash drawer                                

01.04        Refurbishment of shops - display counter, reception, minor furniture and up keeping                     

01.05        Disability assessment and making shops accessible                                 

01.06        Training of field staff on dispensing spectacles, safeguarding, disability inclusion, shop management, record keeping and reporting

Result 2: Women, men and children including persons with disabilities are receiving good quality optical services in the project districts

02.01        Procurement of inventory (spectacle frames, lenses, reading glasses, sun glasses)

02.02        Setting up Mechanisms for Uninterrupted Supply Chain, Quality Assurance and Feedback/complaints.

Result 3: The project established and maintained quality standards of implementation                              

03.01        Project Monitoring                                         

03.02        Rent and utilities at optical shops                                                                                                           

03.03        Local travel for monitoring and daily coordination                                  

03.04        Mid Term Review and End Project Evaluation                                          

03.05        Staff and partner organisation training conducted on Safety and safeguarding                                                              

Target Groups:

The customers benefitting from the services of the optical shops are the patients who are diagnosed with Uncorrected Refractive Error and all those who require post-operative correction of distance and near vision after cataract surgery at the project (public, private and charities) hospitals.

2.     The objective of the Mid Term Review (MTR)

MTR aims to:

1. assess the project’s progress against the project log frame and see if the project is running according to the set goals

2. measure the relevance, effectiveness, impact, efficiency and sustainability of the project interventions

3. identify unexpected or unplanned factors that may prevent the project from achieving its objectives and provide mitigation of solving the factors

4. identify the opportunities to promote and speed up the achievement of the project’s positive outcomes

5. document lessons learnt to date and make recommendations on how to improve project delivery and approaches

6. Explore reasons if the project is de-tracked from its original agenda.

7. review whether the budget is spent according to the cost plan.

3.     Scope

The MTR is expected to cover the project period from the start of the project (1 July 2022 until 30^th September 2024, by creating an accurate and comprehensive picture of the project implementation, project budget, target audience and generating findings on MTR criteria and documenting good practices and lessons learned.

MTR Criteria:

- Relevance: This means the project's methods and plans should address the main challenges that local communities face in getting good quality optical services.

ü  How well the project addresses the identified gaps and needs in eye health services in the project interventions area?

ü  To what extent are the project objectives aligned with the priorities and policies of the local health authorities and stakeholders?

ü  How relevant are the interventions to the socio-economic and demographic characteristics of the target population (e.g., rural communities, persons with disabilities)?

ü  How comprehensive and coherent is the project design in achieving its stated objectives (e.g., improving access to inclusive eye health services, and raising awareness about disability rights)?

ü   the strategies and activities outlined in the project plan appropriate for addressing the identified barriers to up taking optical services eye in the project area?

ü  To what extent are the project interventions culturally and socially appropriate for the communities served

ü  Are the resources allocated (financial, human, and technical) adequate and effectively utilized to achieve the project's goals and objectives?

ü  To what extent inclusion is addressed in the project intervention area

- Effectiveness: This is about whether the project is achieving the expected results and benefits for the people it aims to help, including any additional unintended outcomes

ü  What measurable outcomes and outputs are achieved as a result of the project interventions?

ü  To what extent are the targets set for improving access to inclusive eye health services met?

ü  What evidence exists of improvements in eye health outcomes among the targeted population (reduction in visual impairments, increased access to treatments)?

ü  How does the project impact the lives of beneficiaries, particularly women, children, and persons with disabilities, in terms of access to eye health services and overall quality of life?

ü  What changes were observed in the awareness and behaviour of stakeholders (e.g., community members, health staff, local authorities) towards inclusive eye health practices and disability rights?

ü  To what extent did the project contribute to enhancing the capacity and skills of local health staff in delivering inclusive eye health services?

ü  How efficiently were project resources (financial, human, and technical) utilised for the outcomes achieved?

ü  Were there any areas where resource allocation could have been optimized to improve effectiveness?

ü  What were the key factors influencing the cost-effectiveness of the project in delivering eye health services?

ü  What measures were put in place to ensure the sustainability of project outcomes beyond the project's duration? (e.g., institutional capacity building, policy advocacy, community ownership)

ü  How likely are the improvements in eye health services and disability inclusion practices to be maintained in the long term?

ü  What lessons learned from the project could be applied to similar interventions or scaled up in other regions?

ü  How effectively did the project adapt to challenges and unforeseen circumstances during implementation?

ü  What were the key lessons learned from the project that could inform future initiatives in similar contexts?

ü  How did monitoring and evaluation findings contribute to adapting project strategies for better effectiveness?

- Impact: This refers to the actual and potential benefits the target groups receive from the project activities.

ü  What measurable improvements were observed in eye health outcomes among the target population (e.g., reduction in visual impairments, increase in screenings and treatments)?

ü  How did these improvements compare to baseline data or pre-project conditions?

ü  Can specific numerical data be provided to demonstrate the project's impact on eye health indicators?

ü  How has the project positively affected the lives of beneficiaries, especially women, children, and persons with disabilities?

ü  What qualitative feedback or testimonials have been received from beneficiaries regarding their experience with the project's services?

ü  In what ways has the project contributed to improving community attitudes and perceptions towards disability rights and inclusive eye health services?

ü  To what extent has the project raised awareness about eye health issues and disability rights among community members, stakeholders, and local authorities?

ü  What evidence is there of behavioural changes in terms of increased utilization of eye health services or improved treatment-seeking behaviour?

ü  How has the project influenced the capacity of stakeholders to advocate for disability rights and inclusive practices in eye health?

ü  How has the project contributed to strengthening institutional capacities within District Health Authorities or local health facilities to sustain inclusive eye health services?

ü  Have there been any policy changes or institutional reforms influenced by the project that support better integration of disability rights into health services?

ü  What partnerships or collaborations have been forged as a result of the project that are expected to contribute to sustained impact beyond the project's lifespan?

ü  What measures have been put in place to ensure that the improvements in eye health services and disability inclusion practices are sustained in the long term?

ü  How has the project contributed to building local capacity and empowering communities to continue advocating for their rights and accessing services?

ü  What are the anticipated challenges or risks to sustaining the project's impact, and how are they being addressed proactively?

ü  To what extent have the project's stated objectives related to inclusive eye health services and disability rights been achieved?

ü  Were there any unintended or unexpected positive outcomes that emerged from the project's implementation?

ü  What lessons learned from the project can inform future interventions or similar initiatives in other regions or contries?

- Efficiency: This means producing the desired results with the least amount of resources possible and finding ways to make the project more cost-effective.

ü  How effectively were financial resources allocated across different project activities (e.g., screenings, surgeries, training programs)?

ü  Were there any instances of under-utilization or over-utilization of financial resources that could have been optimized?

ü  What proportion of the project budget was spent on direct service delivery versus administrative costs?

ü  How efficiently were human resources deployed and utilized throughout the project lifecycle?

ü  Were the skills and expertise of project staff aligned with the tasks assigned to them?

ü  What were the key challenges or successes in managing and deploying project personnel effectively?

ü  How efficiently were technical resources (e.g., medical equipment, instruments) and material resources (e.g., consumables) managed and utilized?

ü  Were there any instances of equipment downtime or maintenance issues that affected service delivery?

ü  How did the project ensure timely procurement and distribution of necessary resources to support activities?

ü  What was the cost per unit for delivering key project interventions (e.g., cost per screening conducted, cost per surgery performed)?

ü  How did the cost-effectiveness of the project interventions compare to industry standards or similar initiatives?

ü  Were there any cost-saving measures implemented during the project that contributed to overall efficiency?

ü  How was the efficiency of planning and implementation monitored throughout the project duration?

ü  Were there specific indicators or metrics used to track resource utilization and cost-effectiveness?

ü  What adjustments or improvements were made based on monitoring findings to enhance efficiency?

ü  What measures were implemented to ensure that cost-efficiency gains achieved during the project are sustained in the long term?

ü  How did the project contribute to building local capacity in resource management and cost-effective service delivery?

ü  What recommendations would you make for optimizing resource allocation and improving cost-effectiveness in future projects?

- Sustainability: This is about ensuring that optical services continue even after the financial support from donors ends.

ü  What training programs or capacity-building initiatives were implemented to empower local staff in sustainable service delivery?

ü  To what extent have local health authorities or organizations integrated inclusive into their eye care service delivery frameworks?

- Good Practice: This includes strategies, methods, or activities that have been proven through evidence to be effective, efficient, sustainable, and/or transferable, and that consistently achieve the desired results.

- Cross-Cutting Issues: These are important areas that should be considered in the project, such as including people with disabilities, ensuring safety and safeguarding, protecting the environment, and empowering gender equality.

4.     Expected Deliverables

The consultant will be expected to deliver the following;

1. Data collection tools along with the guidelines for the enumerators;

2. A draft report in English with the key findings of the MTR. The draft report will be reviewed, and recommendations will be given for further improvement;

3. A final report with all documents/reports received from stakeholders during the review is attached as annexures;

4. A PowerPoint Presentation (10 slides) on the main findings and recommendations will be presented to the project partners and donors;

5.     Proposed methodology

The goal of the mid-term review is to draw a comprehensive picture of the implementation of the project from the perspectives of project owners and donors built on a set of suitable qualitative and quantitative methods.  The evaluator is free to choose the appropriate set of methods but innovative methods are particularly welcome. The following method will be used as a minimum to collect information:

-        DESK REVIEW: Project proposal, LFA and reports. 

-        FIELD VISITS face-to-face project meeting: Visits of the optical shops and meeting staff and relevant stakeholders

-        IN-DEPTH INTERVIEWS with relevant stakeholders including project implementation partners and beneficiaries

6.     Report Format

See Annex 1

7.     Required Profile of the Consultant(s)

-        Possess proven experience (at least 10 years) in monitoring, evaluation and learning work for NGO development projects preferably of public health and eye health

-        Strong analytical and documentation skills

-        Experience in conducting similar assignments for international NGOs in Pakistan

Proposed MTR Team

• A Master’s degree in Rural Development, Gender studies, Development Studies, Community development, Sociology, or a related field. An ophthalmologist with eye health expertise will be preferred

• An individual or firm should have extensive experience (proven) of similar assignments with International NGOs.

• Other desirables:

ü  Experience in evaluation/ near to the same for non-profit organizations

ü  Medical professionals with eye health knowledge will be an added value

ü  Have experience working with like-minded INGO’s for at least 10 years on senior positions.

ü  Sectors of technical competency: Inclusive Eye Health and disability-inclusive development.

ü  Demonstrated experience in designing and implementing participatory research methodologies and integrated approaches in strengthening or health systems.

ü  Proven Excellent writing and editing skills (Written and oral fluency in English is a requirement

ü  Language skills: English; for field interviews: Urdu, Saraiki, Punjabi

ü  Strong analytical and presentation skills

ü  Own lived experience with a disability will be considered a strong asset

     Note: the team should include local women with disabilities

8.     Time Frame

SN	Description of activity	Number of days	Location	Dates
1	Meeting with Country office team	0.5	Country office
2	Submission of inception report/work plan	01	Consultant place
3	Field work / data collection	10 days	All 05 districts
4	Data analysis	03	Consultant place
5	Submission of First draft	02	Consultant place
6	Final draft	02	Consultant place
7	Presentation	01	Country office

9.     Administrative and logistical support:

Brien Holden Foundation will not pay for the consultant’s travel, hotel accommodation, meals and any incidental costs associated with the assessment. All in-country logistical arrangements will be made by the consultant her/himself.

10.  Submission and selection of proposals

The proposals can electronically be shared at [email protected].

The proposals in paper form can be sent to (please mention “Proposal for MTR” on top of the envelope) at the following address,

Country Manager, Brien Holden Foundation, Plot 5, Basement, Fayyaz Market, Sector G 8/2, Islamabad.

The timeline for submission of proposals is 29^th October 2024.

No expression of interest will be accepted after the deadline.

The following documents should be included in the proposal:

• Technical proposal including proposed methodology

• Financial proposal (detailed budget including taxes according to the rules and regulations of the consultants’ local tax authorities).

• CVS and profiles of the team member and their respective roles.

• At least two recent examples of the same/ relevant assignment.

• An outline of the understanding of these TORs and suggested methodology.

• Work plan and timeline for the entire assignment.

Selection matrix

Criteria	Score
Budget:	40%
Technical proposal:	60%
Experience in the related task	20%
Qualifications of team	15%
Technical proposal and methodology	25%
Total	100%

11.  Proposed payment schedule:

Brien Holden Foundation proposes to pay 20% of the proposed assignment cost upon presentation of a detailed work plan, the next 50% will be paid upon submission of the first draft of the report and the final 30% will be paid as the final settlement of the contractual obligation.

Annex 1:

MTR report template

The OECD DAC Quality Standards for Development Evaluation must be taken into account

(Website OECD DAC)

Table of Contents

-        Table of Content

-        List of figures and tables

-        List of abbreviations

Summary of Evaluation

-        Background information

-        Main findings and conclusions

-        Key recommendations, if any, lessons learned

1.        Introduction

1.1.      Purpose of evaluation (brief description of the project)

1.2.      Reason and objective of evaluation

-        Background and justification for evaluation

-        Objective of evaluation

-        Central questions of the evaluation

1.3.      Mission of Evaluation

-        Period and terms of evaluation

-        Composition and independence of Evaluation team

-        Participation of partners and target groups during evaluation

-        External factors influencing conduction of evaluation and their consequences

2.        Methodological approach

2.1.      Methodology

-        Methodological approach and instruments

-        Measures to ensure the protection of participants

2.2.      Critical opinion

-        Relevance and limitations of the methodical approach

3.        The framework conditions

-        General conditions, problems and potential at the beginning of the project and significant changes during implementation.

-        Presence and activities of other actors

-        Risks to project success

4.        Performance of the implementing organisation

-        Qualification of the staff

-        Other changes regarding the project partner and its implementation partners

5.        Developmental Impact

5.1.      Relevance

-        Alignment of objectives with the needs of the target group, the priorities and policies of the project sponsor and the donor.

-        Adequacy of the project concept

5.2       Coherence

-        Compatibility of the intervention with other interventions in country, sector or institution

5.3       Effectiveness

-        Quality of project planning

-        Quality of the logical framework and indicators

-        Quality of project implementation

-        Motivation, ownership and legitimacy of l´agence d´exécution

-        Quality of project management

-        Achievement of objectives What were the main factors that determined whether or not the objectives were achieved?

5.4       Efficiency and cost-effectiveness

-        Cost-benefit ratio (At least: detailed cost presentation)

5.5       Impact

-        Achievement of main effects

-        Exemplarity, structure building, efficiency on a broad basis,

-        Other effects (positive and possibly also negative) at the level of the main effects

5.6       Sustainability

-        Sustainability of positive effects (after funding ends)

-        Risks and potential for sustainable effects at the level of the target group and at the level of implementing partner

5.7       Gender

5.8       Child-Safeguarding

5.9       Disability Inclusion

6.        Cross-cutting issues

-        Cross-cutting development policy issues

-        Contribution to the organizational objectives of the bottom receiver

7.        Conclusion and recommendations

7.1.      Conclusions and conclusions

7.2.      Recommendations (if any 7.3: general conclusions and lessons learned)

ANNEX:

-        ToR

-        Composition and independence of Evaluation Team

-        Evaluation Matrix

-        Procedure and schedule at evaluation

-        List of interviewees/participants

-        Bibliography

-        Questionnaires and other data/information collection tools

-        Minutes of the validation of evaluation results on site

-        Logical framework and project indicators

-        Others, as required

End of Document