Virtual network embedding on multi-cloud infrastructure using hierarchical reinforcement learning and heuristic approaches

Virtual network embedding on multi-cloud infrastructure using hierarchical reinforcement learning and heuristic approaches

연구 분야: Networking

학회: Photonic Network Communications

This paper studies problems related to slice request embedding using optimization techniques in multi-cloud infrastructures for Network Slicing-based 5G/6G networks. Network slicing partitions the network’s physical infrastructure into several logical networks, allowing multiple virtual networks to run on top of the underlying physical network. These logical networks run independently and are required to meet the Quality of Service (QoS) demands of specific user groups. This work considers resource allocation for embedding a network slice’s virtual network functions (VNF) components on a cloud-based physical infrastructure consisting of multiple cloud providers. This virtual network embedding (VNE) involves two steps: selecting a substrate network and determining the substrate nodes for embedding the virtual nodes and the corresponding virtual links on the selected substrate network. This can be modeled as a resource allocation optimization problem. The proposed solution approaches are a greedy strategy, a Reinforcement learning (RL)-based scheme using Proximal Policy Optimization (PPO), followed by a state-of-the-art VNE-Q-HRL solution for performance comparison. The proposed Hierarchical RL model has also been compared with an Integer Linear Programming (ILP) formulation for the VNE problem, and the infeasibility of the exact solution for large-scale VNE is shown. The performance of the algorithms has been studied using a discrete-event simulation model for several performance metrics. The results show that the PPO-based Multi-VNE-HRL algorithm shows an improvement of 8.33%, 53.16% in terms of the number of accepted connections, 3.65%, 20.46% in terms of node utilization, and 6.92%, 53.28% in terms of long-term resource embedded over VNE-Q-HRL and, Greedy algorithms, respectively, for a system of 10 substrate networks with 40 nodes each and an arrival rate of 5 VNRs per 100 units of time.