Anomaly detection framework for highly scattered and dynamic data on large-scale networks using AWS

Anomaly detection framework for highly scattered and dynamic data on large-scale networks using AWS

연구 분야: Safety

학회: International Journal of Information Technology

Network anomaly detection is crucial for securing computer networks and communications. However, handling highly scattered data in large-scale, dynamic networks poses challenges, leading to increased false positives and false negatives in anomaly identification. To address this, a new anomaly detection framework employing ensemble learning techniques was introduced. This framework uses decision trees for scattered data and gradient-boosting algorithms for accuracy in dynamic network behaviors. It successfully identifies known and unknown abnormalities while ensuring scalability and reducing false positives and negatives. The framework was tested on CICIDS2017, NSLKDD, and KDDCUP99 datasets, achieving outstanding accuracy rates of 100% on CICIDS2017 and KDDCUP99 and 99.7% on NSLKDD. Deployed on Amazon Web Services, it accurately detected anomalies in new data inputs. Comparative analysis against existing models highlighted the framework's superiority in detecting anomalies in highly scattered data within complex network behaviors. Its performance measures demonstrate its effectiveness as a leading solution in network anomaly detection.