Design and testing of a demand response q-learning algorithm for a smart home energy management system

Abstract

Growth in energy demand stimulates a need to meet this demand which may be achieved either through wired solutions involving infrastructural investment in generation, transmission and distribution systems or non-wired solutions such as demand response (dr). Dr is a grid load reduction measure in response to supply constraints where consumers voluntarily participate in shifting their energy usage during peak periods in response to a time or price-based incentive. In kenya, residential consumers constitute approximately 33 while 30-40 percent on a global scale which demonstrates an essential fraction for their participation in dr. This research aimed at reviewing smart home energy management systems, reinforcement learning (rl) techniques such as q-learning, designing and testing a single agent q-learning algorithm to objectively determine an optimal policy from a set of load management strategies. The study sought to address the performance of the algorithm by reducing the learning speed of the agent. This was achieved by introducing a continuous knowledge base that updated fuzzy logic rules and setting up a definite state-action space. The algorithm was implemented in matlab and interfaced with the physical environment using the arduino uno kit while adopting serial communication between simulation and physical environment. A graphical user interface developed using the app designer tool in matlab created a provision for integrating consumer feedback which was critical in communicating with the knowledge base to update fuzzy rules. The time of use (tou) tariff plan constituted three major segments which were off-peak, mid-peak and peak tariffs, developed by benchmarking public historical residential tariff data with tou trends for other countries. Load profiles generated from appliance and tou data were used to test the algorithm. The designed algorithm showed an improvement in learning within 500 episodes and net energy savings ranging between 8 and 11 %.