LACO: A Latency-Constraint Offline Neural Network Scheduler towards Reliable Self-Driving Perception

LACO: A Latency-Constraint Offline Neural Network Scheduler towards Reliable Self-Driving Perception

연구 분야: Verification

학회: ICCAD '24: Proceedings of the 43rd IEEE/ACM International Conference on Computer-Aided Design

Perception is a crucial and compute-intensive self-driving stage where multiple DNN models periodically process image and point-cloud inputs with constant intervals as latency constraints. This scenario is similar to the multistream scenario in the MLPerf benchmark but with more than one query sequence of diverse intervals from different sensors, and each query also evokes more than one model. In this paper, we call this scenario multi-multistream (M2-stream). Because the periodic inputs arrive continuously, the neural network processor should process them in time to avoid information dropping and maintain self-driving safety. To this end, this paper aims to explore throughput optimization under constant deadlines for the M2-stream scenario in self-driving perception. We propose LACO, a novel LAtency-COnstraint scheduler, to optimize the task throughput under the predetermined deadline constraints. Considering that the task arrivals are known with constant intervals, we can employ an offline scheduling method. Specifically, we turn the scheduling space exploration into the Satisfiability problem (SAT) and tackle it using an off-the-shelf SAT solver. Our experiments demonstrate that LACO achieves 1.2×~2.4× throughput improvement against the baselines on an 8×8-core accelerator. LACO also shows consistent superiority in various hardware scales and input resolutions.