An Evaluation Survey of Literacy Interventions

Objectives & scope

• The purpose of the assignment was to: a) access the quantitative and qualitative information regarding validations of the PNFEP and TSKL interventions; and b) monitor progress and achievements against the targets, challenges faced and way forward. FAME Education developed the evaluation framework along with methodology and instruments to cater the above-stated objectives. Consistently, the TPE reported herein examined the extent to which both projects have achieved the intended targets. It also analyzed the viability and effectiveness of the schemes implemented under these projects. Furthermore, the expected impact of these projects on literacy and social development was also examined. The TPE team developed appropriate frameworks in consultation with the major stakeholders (e.g., DG M&E and L&NFBE Department) to map out the evaluation plan (e.g., overall design, data collection instruments) per the stated objectives. A range of research methods were used to collect information from the participants. Several research instruments (e.g. checklists, questionnaires and achievement tests) were developed for different respondents including NFEI teachers, field managers, and community members. In-depth qualitative information was also gathered by using document analysis, semi-structured interviews, focus group discussions and case studies. A stratified random sampling design was used to select the sample of NFEIs. The sampling unit was the NFEI, and strata were type of NFEI (NFBES, NFEFS and ALC), district, and tehsil. The sampling frame was the database of L&NFBE Department. A total of 2669 (which is 98.2% of the agreed sample) NFEIs were covered in the TPE, as 48 NFEIs included in the sample could not be traced. All teachers (i.e. 2669) working in the selected NFEIs were part of the study. All DEOs Literacy and PLCs, as well as 205 LMs who were working with the sampled NFEIs, were also included in the study. In addition, a total of 22,470 (8,510 male and 13,960 female) learners took the assessments under the TPE, which is more than 40% of the average number of learners enrolled in the sampled NFEIs. Furthermore, a total of 276 communities (which is 10% of the surrounding communities of the selected NFEIs) were included in the sample and out of the selected communities, a total of 1250 community members were also surveyed, on a convenience basis. An Android App was developed for real timing monitoring and data collection, along with a computer-based application for live dashboard, M&E and data handling.

Related assignments