Energy Management of Microgrids Using Deep Q-Network Based Battery Optimization

Microgrid energy management must minimize operating cost while coping with intermittent photovoltaic generation, time varying demand and limited battery capacity under practical constraints on state of charge and grid power quality. Classical rule-based scheduling offers interpretability but often leads to suboptimal battery use and higher cost when prices and profiles change over time. The research contribution is the design of a transparent Deep Q Network controller that optimizes battery charge and discharge through a multi term reward that combines economic cost, state of charge regularity, grid current peaks and approximate frequency and phase quality, implemented in a lightweight NumPy environment suitable for embedded deployment. The microgrid model uses synthetic daily profiles with 5 kW photovoltaic capacity, peak load near 4 kW, a 10kWh battery and a discrete action set that spans strong charge and strong discharge over 48 decision steps per day. The Deep Q Network policy is trained against a deterministic rule baseline and both controllers are evaluated on identical trajectories. Simulations show that the learned controller reduces total daily operating cost by about 35.6%, keeps the state of charge within a tighter band and shifts battery scheduling toward charging in low price hours and discharging at peaks. The learned policy decreases maximum grid current from about 24.28 A to 16.58 A, maintains frequency within roughly 49.89 Hz to 50.08 Hz and pushes phase angles toward a value close to unity power factor while preserving feasible battery operation. Training curves indicate stable convergence with consistent improvement in the long run return. These results indicate that Deep Q Network based energy management can offer a practical and physically interpretable alternative to handcrafted rules and can serve as a foundation for future hardware oriented microgrid controllers.