Energy-delay-aware VNF scheduling: a reinforcement learning approach with hierarchical reward enhancement

Energy-delay-aware VNF scheduling: a reinforcement learning approach with hierarchical reward enhancement

연구 분야: Networking

학회: Cluster Computing

Network functions virtualization (NFV) is a technology that virtualizes network functions into virtual network functions (VNF) to deliver communication services. Efficient and flexible VNF scheduling is an important way to improve network resource utilization, reduce costs, and provide better service quality. With the development of artificial intelligence, network equipment is no longer just a forwarding node, but also a computing node. Therefore, energy consumption will become an important indicator that needs to be considered in the VNF scheduling problem. In this paper, we aim to realize VNF scheduling with minimizes idle energy loss (IEL) of NFV nodes and the makespan (i.e., overall completion time) for all services. The problem can be formulated as a Mixed Integer Linear Program (MILP), and the complexity of the problem grows exponentially as the size of the network scale increases. To solve this problem efficiently and flexibly, we treat MILP as a Markov Decision Process (MDP) and design an reinforcement learning (RL) algorithm to solve the MDP problem. Specifically, the algorithm utilizes a hierarchical reward enhancement (HRE) mechanism, called RL-HRE. In addition, a weighted reward function is carefully designed in the proposed algorithm to achieve flexible energy-delay-aware VNF scheduling. The simulation results show that RL-HRE is superior to other comparative algorithms in terms of solution accuracy and time complexity.