Cooperative Multi-Agent Reinforcement Learning for Energy-Harvesting Aware Dynamic Clustering and Mobile Sink Path Co-Optimization in Wireless Sensor Networks
DOI:
https://doi.org/10.21928/uhdjst.v10n1y2026.pp130-142Keywords:
Wireless Sensor Network, Multi-Agent Reinforcement Learning, Energy Harvesting, Dynamic Clustering, Mobile Sink, Q-LearningAbstract
Energy efficiency and network lifetime are two critical issues in Wireless Sensor Networks (WSNs). This paper introduces a cooperative multi-agent reinforcement learning solution for energy-harvesting driven, joint dynamic clustering and mobile sink routing. There are two cooperative Q-learning agents; the clustering agent selects the best cluster heads based upon residual energy, geographical location, and energy harvesting rates, whereas the routing agent selects the best routing directions for the sink to maximize nodes under the clustered sink for data collection. A real-world experiment based upon the Intel Berkeley dataset features a validation protocol of 55 nodes and 2.3M+ readings which indicate 99 network lifetime steps (1 episode) with 100% of nodes alive, 2,308 collected packets per episode, and an 82% decrease in reward deviation substantiating stable convergence, as well as a 5.6% improvement over the best comparative (DRL-Routing) baseline from total reward 193,944 to 204,750 from step 1 to 200, confirming effective cooperative multi-agent training for energy-harvesting WSNs.
References
I. F. Akyildiz, Y. Su, Y. Sankarasubramaniam and E. Cayirci. “Wireless sensor networks: A survey”. Computer Networks, vol. 38, no. 4, pp. 393-422, 2002.
J. Yick, B. Mukherjee and D. Ghosal. “Wireless Sensor Network Survey: Computer Networks”. Elsevier, Amsterdam, 2008.
P. Rawat, K. D. Singh, H. Chaouchi and J. M. Bonnin. “Wireless sensor networks: A survey on recent developments and potential synergies”. The Journal of Supercomputing, vol. 68, no. 1, pp. 1-48, 2014.
T. Rault, A. Bouabdallah and Y. Challal. “Energy efficiency in wireless sensor networks: A top-down survey”. Computer Networks, vol. 67, pp. 104-122, 2014.
N. A. Pantazis, S. A. Nikolidakis and D. D. Vergados. “Energy-efficient routing protocols in wireless sensor networks: A survey”. IEEE Communications Surveys and Tutorials, vol. 15, no. 2, pp. 551-591, 2012.
A. A. Abbasi and M. Younis. “A survey on clustering Algorithms for wireless sensor networks”. Computer Communications, vol. 30, no. 14-15, pp. 2826-2841, 2007.
M. M. Afsar and M. H. N. Tayarani. “Clustering in sensor networks: A literature survey”. Journal of Network and Computer Applications, vol. 46, pp. 198-226, 2014.
O. Younis and S. Fahmy. “HEED: A hybrid, energy-efficient, distributed clustering approach for ad hoc sensor networks”. IEEE Transactions on Mobile Computing, vol. 3, no. 4, pp. 366-379, 2004.
C. Tunca, S. Isik, M. Y. Donmez and C. Ersoy. “Distributed mobile sink routing for wireless sensor networks: A survey”. IEEE Communications Surveys and Tutorials, vol. 16, no. 2, pp. 877-897, 2013.
S. Gao, H. Zhang and S. K. Das. “Efficient data collection in wireless sensor networks with path-constrained mobile sinks”. IEEE Transactions on Mobile Computing, vol. 10, no. 4, pp. 592-608, 2010.
F. K. Shaikh and S. Zeadally. “Energy harvesting in wireless sensor networks: A comprehensive review”. Renewable and Sustainable Energy Reviews, vol. 55, pp. 1041-1054, 2016.
Z. A. Eu, H. P. Tan and W. K. Seah. “Design and performance analysis of MAC schemes for wireless sensor networks powered by ambient energy harvesting”. Ad Hoc Networks, vol. 9, no. 3, pp. 300-323, 2011.
J. Wang, Y. Gao, W. Liu, A. K. Sangaiah and H. J. Kim. “An improved routing schema with special clustering using PSO Algorithm for heterogeneous wireless sensor network”. Sensors, vol. 19, no. 3, p. 671, 2019.
C. J. Watkins and P. Dayan. “Q-learning”. Machine Learning, vol. 8, no. 3, pp. 279-292, 1992.
A. Forster and A. L. Murphy. “FROMS: Feedback routing for optimizing multiple sinks in WSN with reinforcement learning”. In: 2007 3rd International Conference on Intelligent Sensors, Sensor Networks and Information. IEEE, 2007.
L. Busoniu, R. Babuska and B. De Schutter. “A comprehensive survey of multiagent reinforcement learning”. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 38, no. 2, pp. 156-172, 2008.
D. Zhang, G. Li, K. Zheng, X. Ming and Z. H. Pan. “An energy-balanced routing method based on forward-aware factor for wireless sensor networks”. IEEE Transactions on Industrial Informatics, vol. 10, no. 1, pp. 766-773, 2013.
W. R. Heinzelman, A. Chandrakasan and H. Balakrishnan. “Energy-efficient communication protocol for wireless microsensor networks”. In: Proceedings of the 33rd Annual Hawaii International Conference on System Sciences. IEEE, 2000.
J. Luo and J. P. Hubaux. “Joint mobility and routing for lifetime elongation in wireless sensor networks”. In: Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies. IEEE, 2005.
A. Kansal, J. Hsu, S. Zahedi and M. B. Srivastava. “Power management in energy harvesting sensor networks”. ACM Transactions on Embedded Computing Systems, vol. 6, no. 4, p. 32-es, 2007.
H. H. El-Sayed, E. M. Abd-Elgaber, E. A. Zanaty, F. S. Alsubaei, A. A. Almazroi and S. S. Bakheet. “An efficient neural network LEACH protocol to extended lifetime of wireless sensor networks”. Scientific Reports, vol. 14, no. 1, p. 26943, 2024.
G. Siamantas, D. Rountos and D. Kandris. “Energy saving in wireless sensor networks via LEACH-based, energy-efficient routing protocols”. Journal of Low Power Electronics and Applications, vol. 15, no. 2, p. 19, 2025.
V. Rajaram, V. Pandimurugan, S. Rajasoundaran, P. Rodrigues, S. V. N. Santhosh Kumar, M. Selvi and V. Loganathan. “Enriched energy optimized LEACH protocol for efficient data transmission in wireless sensor network”. Wireless Networks, vol. 31, no. 1, pp. 825-840, 2025.
A. Senturk. “Artificial neural networks-based LEACH Algorithm for fast and efficient cluster head selection in wireless sensor networks”. International Journal of Communication Systems, vol. 38, no. 3, p. e6127, 2025.
A. Juwaied and L. Jackowska-Strumillo. “DL-HEED: A deep learning approach to energy-efficient clustering in heterogeneous wireless sensor networks”. Applied Sciences, vol. 15, no. 16, p. 8996, 2025.
P. Bekal, P. Kumar, P. R. Mane and G. Prabhu. “A comprehensive review of energy efficient routing protocols for query driven wireless sensor networks”. F1000Research, vol. 12, p. 644, 2024.
H. Li, Y. Dai, Q. Chen, D. Liao and H. Jin. “Energy efficient mobile sink driven data collection in wireless sensor network with nonuniform data”. Scientific Reports, vol. 14, no. 1, p. 28190, 2024.
A. Ben Yagouta, B. B. Gouissem, S. Mnasri, M. Alghamdi, M. Alrashidi, M. A. Alrowaily, I. Alkhazi, R. Gantassi and S. Hasnaoui. “Multiple mobile sinks for quality of service improvement in large-scale wireless sensor networks”. Sensors, vol. 23, no. 20, p. 8534, 2023.
A. Abu Taleb, Q. Abu Al-Haija and A. Odeh. “Efficient mobile sink routing in wireless sensor networks using bipartite graphs”. Future Internet, vol. 15, no. 5, p. 182, 2023.
K. Ramanan and P. Siva Raja. “Mobile sink based efficient data gathering and routing using clustering based hybrid models”. Wireless Networks, vol. 31, pp. 3907-3930, 2025.
M. U. Mushtaq, H. Venter, A. Singh and M. Owais. “Advances in energy harvesting for Sustainable wireless sensor networks: Challenges and opportunities”. Hardware, vol. 3, no. 1, p. 1, 2025.
W. Chen, F. Tang, F. Cui and C. Chen. “Research on energy harvesting mechanism and low power technology in wireless sensor networks”. Sensors, vol. 24, no. 1, p. 47, 2023.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Dlsoz Abdalkarim Rashid
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
