A model for post-implementation evaluation of health information systems: The case of the DHIS 2 in Kenya
Abstract
The Kenyan government through the Ministry of Health (MoH) in its quest to improve
the health outcomes of its citizenry, has embarked on a number of initiatives. One
of these initiatives under the umbrella of eHealth is the District Health Information
Software(DHIS 2). DHIS 2 is ”an integrated, web-based, country owned and managed,
national health information system that integrates quality data used at all levels to improve
health service delivery”Douglis (2012) . The logic behind the emphasis of DHIS
2 is that with quality data, public health decision making will be more efficient and
effective. Despite the importance of DHIS 2, no study has been done to evaluate the
post-implementation of the system, particularly user perspective. DHIS 2 was a large
investment and like any organisation, the MoH would be interested in knowing the impact
the system has on its users. The objectives of this study were: to establish the
user’s perspective in evaluating a health information system (HIS) and present a set of
evaluation factors that can be used in evaluating a HIS, identify a suitable evaluation
model and, test the evaluation model using DHIS 2. We reviewed literature on information
systems, health information systems, the information system(IS) user and IS
success theories and models. The IS-Impact model was used as the theoretical foundation
for this study. The model comprises four dimensions, which are Individual Impact,
Organizational Impact, Information Quality and System Quality. The questionnaire
was the main data collection tool with focus group discussions(FGDs) being used as
complementary tool. A total of 135 reponses from questionnaires and transcribed data
from 2 FGDs were used for analysis. The results indicate that 89% of the respondents
were operational or day-to-day DHIS 2 users with System Quality being the strongest
contributor to the IS-Impact model. In Individual Impact, the individual performance
of the respondents on behalf of their organisation, most respondents stated that DHIS
2 had an impact in their performance. Under Organisation Impact, the impact of DHIS
2 at a broader, organisational level, most respondents felt that DHIS 2 has an impact
particulalrly in achieving the MOH’s objectives. In Information Quality, the quality
of information that the system produces in reports and on screen, most respondents
stated that it was high. However, respondents were divided on whether data from DHIS
2 was unique. Under System Quality where users were asked on their perception of
DHIS 2 performance from a technical and design perspective, there were mixed views
regarding system quality issues. The FGDs revealed that impact of DHIS 2 was due
to comparison with the previous Excel-based system which was time-consuming and
involved a lot of paperwork. On quality the FGDs revealed problems with lack of DHIS
2 integration with other systems and inconsistencies between data collection tools and
the DHIS 2 data entry portal. The conceptual model was tested using tolerance, Variance
Inflation Factor(VIF) and correlation co-efficients. The test results indicate that 30 measures are significant for measuring the impact of health information systems in
Kenya. This demonstrates the validity of the model. Based on these findings, a model
for post-implementation evaluation of health information systems is presented. It is
recommended that the model needs to be extended further to include the context under
which the system is based. For instance, comparative studies can be conducted
to evaluate the success of the HIS in a rural setting as compared to an urban setting.
Also, one of the limitations in this study was getting respondents who were in strategic
management. This study’s respondents were operational users i.e., they interacted with
the system on a day to day basis. Strategic users as compared to operational users
interact less with the system but are important because they decide on the resources
required to achieve organizational objectives. Also, decisions made in the strategic level
have long-term implications. Therefore, this model can be extended further to include
strategic users.
Citation
Master of Science Information SystemsPublisher
University of Nairobi