Lead Evaluator (International Consultant) to Conduct Midterm Evaluation of the Subgrantee Portfolio under the USAID-funded Access to Justice and Suppo

TERMS OF REFERENCE

TITLE: Lead Evaluator (International Consultant) to Conduct Midterm Evaluation of the Subgrantee Portfolio under the USAID-funded Access to Justice and Support for the Rule of Law in the Philippines Program (ACCESS)

LOCATION: Conducted remotely, with availability to work during

Philippine business hours

EXPECTED PERIOD OF PERFORMANCE: November 30, 2021- February 15, 2022

ABA ROLI is a non-profit organization that implements legal reform programs in roughly 50 countries around the world. ABA ROLI has nearly 500 professional staff working abroad and in its Washington, D.C. office. ABA ROLI’s host country partners include judges, lawyers, bar associations, law schools, court administrators, legislatures, ministries of justice and a wide array of civil society organizations, including human rights groups.

For more than a decade, ABA ROLI has focused on improving court efficiency, increasing business sector confidence, combating corruption, and expanding access to justice in the Philippines. By continuously working with a broad coalition of local partners — trial and appellate courts, the Supreme Court, the Philippine Judicial Academy, the Integrated Bar of the Philippines, universities, law schools, and a spectrum of civil society representatives — ABA ROLI is building local capacity to champion and uphold the rule of law.

ABA ROLI is seeking a Lead Evaluator (International Consultant) to conduct a midterm performance evaluation of the subgrantee portfolio under its Access to Justice and Support for the Rule of Law in the Philippines Program (ACCESS). The Lead Evaluator may be paired with a local consultant at the direction of ABA ROLI. The Lead Evaluator will work with the Evaluation Team to undertake data collection and evaluation activities remotely due to ongoing COVID-19 travel restrictions.

Background

ABA ROLI is currently implementing the Access to Justice and Support for the Rule of Law in the Philippines Program (ACCESS), a USAID funded five-year Program designed to bolster the rule of law and enhance access to justice in the Philippines. This goal is directed towards the most vulnerable communities in the Philippines, those whose rights are least protected, interests ignored, and ability to access a fair and predictable legal system is most restricted. Through ACCESS, ABA ROLI endeavors to provide support for the efficient and fair administration of justice in the Philippines, in partnership with key institutions, including the Supreme Court of the Philippines.

The ACCESS program is achieved through the following four objectives:

Cross-Cutting Objective 1) Direct support civil society organizations for public awareness-raising and legal services provision.

Objective 2) Support the promotion and institutionalization of pro bono legal service. Increase awareness of citizens’ rights and access to legal aid services, advice, and representation.

Objective 3) Strengthen alternative dispute resolution mechanisms within and outside the court system.

Objective 4) Leverage USAID-supported initiatives to further contribute to the efficient and fair administration of justice.

In pursuit of Cross-cutting Objective 1, ABA ROLI established a competitive subgrant program to support Filipino organizations to reach many of the most poor and marginalized communities through two broad, interconnected themes: (1) to raise awareness of citizens’ basic rights, increase understanding of laws and their impact, and inform vulnerable populations of court and alternative dispute resolution processes; and (2) to provide legal services to specific vulnerable populations. Complementary institutional training and technical assistance is offered to subgrantees, as well as support on monitoring and evaluation. ABA ROLI looked for synergies across subgrant programs and other activities to find ways in which subgrant projects can build upon other work conducted under this grant. In addition, subgrantees have been encouraged to provide cost share as part of their application.

Midterm Subgrantee Portfolio Evaluation Purpose and Scope

In line with USAID’s Evaluation Policy, the objectives of the midterm subgrantee portfolio evaluation relate to both accountability and learning. In particular, the objectives of the evaluation are:

1. To assess progress towards outcomes and outputs committed to under the ACCESS project, and analyze the factors and conditions that facilitate or hinder the achievement of these results (accountability);

2. To identify whether progress made so far is increasing the access to justice of vulnerable populations and, if so, in what ways (accountability).

3. To analyze the theory of change of the program and how it has evolved, as well as the key assumptions, risks, and main challenges and identify what can be learned and its implications to the current program thus far. (learning)

The midterm subgrantee portfolio evaluation will cover the time period between August, 2019 to September, 2021 and will analyze results, progress, and lessons learned during Years 1-3 of the subgrantee portfolio. The evaluation will capture midline results and performance that will inform ongoing implementation in Year 4 and Year 5 of the ACCESS program. Primary users of this evaluation are ABA ROLI, ACCESS program subgrantees, Freedom House, and USAID. The evaluators will utilize and build upon preliminary research compiled from an internal, partially completed, review of the ACCESS program subgrant portfolio.

The evaluation will focus on a performance assessment of the ACCESS program’s subgrant portfolio to date. The evaluation will review and identify performance achievements and shortfalls against agreed subgrantee objectives and targets for 19 subgrantees, identify contributing factors (negative and positive), and identify lessons learned that will be useful in improving programming. This includes a review of the extent to which sub-grantee logic models aligned with the proposed sub-grantee designs as well as alignment with the ACCESS theory of change. The results of the evaluation of the subgrant portfolio will inform medium and long-term strategic decision-making by ABA ROLI and its implementing partners in order to strengthen or refocus program approaches as necessary. Findings will be used to refine and strengthen strategies and activities scheduled for Years 4 and 5, and will be useful for overall “learn and reflect” sessions.

Primarily, the midterm evaluation will address the following questions:

1. To what extent are sub-grantee logic models aligned with the ACCESS theory of change and are there significant deviations in the current logical models from the proposed designs?

2. To what extent is the subgrant portfolio on track to meet its objectives and targets and what are the reasons (positive or negative) behind the current level of performance? What are the key results to date, including from the perspective of beneficiaries?

3. To what extent did activities to increase rights awareness and provide connections to legal assistance providers expand the access to justice of project beneficiaries?

4. How did COVID-19 and other factors that posed challenges and risks affect the subgrantee portfolio’s performance and how did the subgrantees adapt to limitations and constraints?

5. To what extent has subgrantee partner capacity building, both organizational management AND technical/thematic capacity, resulted in gains in achieving subgrantee project objectives? Which training activities and techniques across the two areas have been the most/least successful at developing capacity of stakeholders?

6. Does the subgrantee strategic approach need to be modified in order to reflect the reality of the current operating environment (political, legal, economic, institutional, health) in achieving “access to justice” outcomes? If so, how?

7. What are the lessons learned, good or promising practices, and models of intervention as well as recommendations that will serve to inform programmatic changes and future activities.

8. How sustainable are the subgrantee activities funded under the ACCESS grant and has the ACCESS program been successful in collaborating and leveraging additional interest and investment from other relevant stakeholders, e.g., national institutions; local government/ tribes/ institutions; and other partners.

Methodology

The fieldwork for the evaluation is expected to be conducted remotely. The Lead Evaluator / Evaluation Team, in collaboration with ABA ROLI (Philippines), will finalize the overall evaluation methodology. ABA ROLI expects that, at a minimum, the Lead Evaluator will:

● Review and analyze the existing performance information from ABA ROLI’s documentation and data from ABA ROLI sub-grantees;

● identify potential respondents from and undertake interviews with ABA ROLI partners, beneficiaries, other relevant stakeholders. Given possible COVID-related limitations, the Lead Evaluator will propose a methodology that strategically incorporates the use of key informant interviews and/or focus group discussions in order to maximize the amount of data available to the evaluation process;

● interview ABA ROLI staff and a representative number of experts working in the sector as appropriate;

● collect other evidence using various data collection tools as appropriate.

The evaluation will use a mixed-method evaluation approach. The methodology will combine a review of quantitative data and application of qualitative evaluation techniques to obtain information, opinions, and data from subgrantees, beneficiaries, and other stakeholders as appropriate. The approach will be participatory and involve the use of appropriate data collection tools.

ABA ROLI will provide the Evaluation Team with a core list and/or copies of the subgrantee portfolio’s strategic approach, planning documents, performance reports, relevant assessment reports, and other key documentation. The Evaluation Team will be responsible for expanding this background documentation as appropriate, reviewing, and prioritizing it. ABA ROLI will facilitate introductions to subgrantees and key stakeholders and support scheduling efforts as appropriate.

Tasks and Responsibilities and Deliverables

The Lead Evaluator will lead the evaluation and coordinate with the Program Team for progress reports, review of tools developed, and organization of calls, interviews, and meetings in relation to the evaluation. The Lead Evaluator will develop the evaluation methodology, data collection and data analysis tools, and will draft evaluation reports and summaries of findings, where needed. The Lead Evaluator will lead the development of, and facilitate surveys, key informant interviews and focus group discussions virtually, with an eye toward utilizing digital tools and techniques to ensure inclusivity of marginalized stakeholders and a high-quality of data collection over digital mediums.

Specifically, the Lead Evaluator will lead the production of the following:

1. An inception report / work plan prepared in consultation with the ABA ROLI Monitoring & Evaluation Manager that includes design and methodology, data collection activities and analysis.

2. Relevant tools for data collection and collated data.

3. Draft evaluation report that includes preliminary findings, conclusions and recommendations. ABA ROLI will review and submit comments and suggested revisions.

4. A final evaluation report, including an executive summary, an explanation of the evaluation methodology with its limitations, the findings, conclusions and recommendations, appendices that contain the data analysis collection and analysis tools used.

5. PowerPoint presentation and Q&A session with the ABA ROLI team. The presentation must include findings, conclusions, recommendations and lessons from the evaluation.

6. Databases with machine-readable information that have the quantitative data collected in the evaluation, mostly created from surveys carried out during the evaluation.

Qualifications

• Extensive experience in strategic programming of development assistance within the broader areas of democratic governance, in-depth knowledge of legal reform and rule of law sector capacity building at national and subnational levels.

• At least a graduate degree in public policy or administration, social or political science, community development or other related field with no less than seven years of experience conducting program evaluation OR a bachelor's degree in these same fields with more than 10 years of experience in conducting program evaluations

• Experience conducting evaluations in the Philippines or in similar contexts; experience conducting evaluations in the human rights and access to justice sector is a plus;

• Demonstrated ability to conduct desk research in addition to facilitating surveys, key informant interviews and focus group discussions, either in person or in a virtual setting;

• Experience evaluating USAID or US government-funded programs is a distinct advantage;

• Experience conducting key informant interviews, focus group discussions, or similar activities, over online mediums, with a knowledge of requisite digital tools for the conduct of these activities;

• Knowledge of Philippines justice and civil society sectors strongly preferred;

• Excellent communication (both oral and written) in English.

Duration of the Consultancy

The midterm evaluation is expected to take place over a period of 76 days between November 30, 2021- February 15, 2022.

Payment Schedule

Payment will be tied to successful completion of deliverables as outlined in a contractual agreement and in accordance with acceptable quality standards as determined by ABA ROLI.

How to Apply

Qualified candidates should submit their application to philippines-hr@abaroli.org, specifying “ACCESS Midterm Evaluation Application” in the subject line, by 11:59pm Manila time on November 13, 2021. All applications should include:

1. Cover Letter;

2. Curriculum Vitae, or multiple Curricula Vitae, including a list of evaluations conducted;

3. Proposed approach and methodology (please include as an attachment), including approach to gather information for each evaluation question.

4. Daily rate and proposed budget.

Joint proposal submissions pairing a Lead Evaluator (International Consultant) with an Evaluation Specialist (Local Consultant) are welcome, but not required.

Applications will be reviewed on the basis of experience relevant to the assignment, professional qualifications and competence, approach and methodology, and price. Due to the high volume of applications, ABA ROLI will only be able to contact shortlisted applicants.

How to apply:

How to Apply

Qualified candidates should submit their application to philippines-hr@abaroli.org, specifying “ACCESS Midterm Evaluation Application” in the subject line, by 11:59pm Manila time on November 13, 2021. All applications should include:

1. Cover Letter;

2. Curriculum Vitae, or multiple Curricula Vitae, including a list of evaluations conducted;

3. Proposed approach and methodology (please include as an attachment), including approach to gather information for each evaluation question.

4. Daily rate and proposed budget.

Joint proposal submissions pairing a Lead Evaluator (International Consultant) with an Evaluation Specialist (Local Consultant) are welcome, but not required.

Applications will be reviewed on the basis of experience relevant to the assignment, professional qualifications and competence, approach and methodology, and price. Due to the high volume of applications, ABA ROLI will only be able to contact shortlisted applicants.