Metaevaluation of Programme Evaluation Practice in Kenya

Abstract

This research sought to examine programme evaluation practice in Kenya. This was done by carrying out metaevaluation, that is, evaluation of evaluations that have been carried out on national programmes in Kenya. Metaevaluation is in itself an evaluation process which means that the research is based on the function of evaluation. The impetus of the research was due to the fact that much as evaluation literature is full of discussions on philosophical and theoretical orientations to evaluation, very little has been done or written on evaluation practice as it actually takes place particularly in large settings, more so, within the African context. Also, evaluation has not always lived up to its own noble aspirations as noted by many social programmes worldwide. The researcher therefore thought that it was important to check the quality of the evaluation system in Kenya in order to establish how much is known of the process and to determine areas where better practice is needed. It was also hoped that the research would act as a catalyst to other similar researches. The researcher looked into programme evaluation practice by researching on evaluation as practised in education, focusing on curriculum evaluation in schools in Kenya. The researcher chose this area because curriculum in education is one area where the kenya government has commissioned large national evaluations making it possible to look at programme evaluation practice in diverse aspects.The researcher analysed evaluation practice usmg "The Programme Evaluation Standards" developed by the Joint Committee (1994) as the criteria of performance to determined the acknowledged theory of evaluation practice, as well as an interview guide to determine the context in which evaluation takes place. The research design used was naturalistic inquiry by applying the audit trail content analysis and constant comparative method. The results reveal lack of many desirable qualities related to specific principles of evaluation practice as prescribed by the Standards. Notably, defective principles were utility, serving information needs of intended users; feasibility, being realistic and politically viable; and propriety ethical standards, desiring protection of the rights of individuals. The accuracy standards, that is, technical aspects related to social science research approaches were better addressed. The results also reveal an evaluation context whereby the policy is not mediated or adapted, but, mandated and sanctioned at 'moments that suit interested parties. One therefore, finds a situation that reflects a practice that is guided much more by political and technical aspects of evaluation but less of evaluation principles and methodologies. The research therefore concludes that programme evaluation practice in Kenya does not meet the standards criteria which are based on the acknowledged theory of evaluation. Further, the context does not support or facilitate the evaluation process. The results show that in order for an evaluation task to be completed, an evaluator needs to posses not only social science technical aspects of evaluation butthe three programme evaluation standards, that is, utility, propriety and feasibility to comprehend the evaluation practice. A critical analysis of the outcome of the evaluation research shows that, the Kenyan evaluation process presents a situation that is complex. The situation presents unresolved issues that need to be regarded as possibly new forms of evaluation approaches in a unique context, and that the findings may not always be a result of inappropriate practices but are determined by the situation at hand. The range of practice needed to meet the standards criteria also seems to be out of reach of an evaluator as theoretical evaluation issues that influence practice have not been addressed in the evaluation literature to guide practice, nor does the Joint Committee (1994) address these issues. also knowledge of all