Flatdc: Automatic Schema Reverse Engineering of FlatBuffers

Flatdc: Automatic Schema Reverse Engineering of FlatBuffers

연구 분야: Analysis

학회: IFIP International Conference on ICT Systems Security and Privacy Protection

Schema reverse engineering is critical for analyzing closed-source software, including malware. Extracting communication schemas is particularly challenging for binary protocols. This paper presents two approaches for automatically reverse-engineering FlatBuffer schemas, a binary serialization framework by Google that, if exposed, can lead to security vulnerabilities such as size explosion attacks. The first, the binary approach, exploits static analysis and instruction simulation to extract schemas directly from validation logic. The second, the on-the-wire approach, analyzes protocol messages to approximate schemas, effectively narrowing the possible data types per field to an average of 2.3. We evaluate both methods on self-compiled binaries and third-party FlatBuffer schemas. The binary approach consistently extracts full schemas, aiding in vulnerability detection. The on-the-wire approach, though less precise, remains language-agnostic and resilient to binary obfuscation, making it valuable for security-focused protocol analysis.