Stochastic stylization transformer with self-supervision for iris recognition

Stochastic stylization transformer with self-supervision for iris recognition

연구 분야: Verification

학회: Multimedia Systems

Although vision transformer has demonstrated superior performance over convolutional networks in computer vision, its potential in iris recognition remains underexplored, primarily due to the neglect of domain-specific iris structure. To bridge this gap, we propose a second-order transformer called RAformer that consists of R&A transformer blocks and affinity pooling stacked in a pyramid structure, which can effectively capture radial and angular structure of iris texture in an efficient manner. In realistic scenarios, however, the appearance changing due to various uncertain acquisition factors inevitably deteriorates model performance of conventional methods, which heavily rely on less diverse data to handle real-world appearance shifts in iris patterns. To address this challenge, we propose a joint style transfer based normalization and variational prototype learning for appearance-shift robust iris recognition. Concretely, by being the first to treat the appearance-changing as fine-grained style shift, we introduce the identity-aware permuted adaptive instance normalization (pAdaIN) module from the perspective of style transfer, which efficiently swaps the intermediate instance-level feature statistics of samples within a batch to construct diverse appearances. Additionally, to allow the model to bear such great style variations, we introduce stochastic neural network (SNN) classifier based on variational prototype learning, in which each class is represented by a distribution rather than a point in the traditional softmax classifier. They are integrated into a joint learning framework and benefit each other. To further adapt the model for data-scarce scenarios, we are the first to introduce self-supervision as an auxiliary task in a joint learning manner, enabling the model to learn richer and more transferable visual representations even with limited labeled samples. Consequently, with self-supervision, our RAformer can be naturally extended to a more practical scenario called semi-supervised iris recognition, where only a few source data are labeled while the majority of them are unlabeled. We report consistent improvements across four challenging iris benchmarks, with more significant advantages over prior arts in semi-supervised learning settings, establishing new state-of-the-art (SOTA) results. Additionally, even without any fine-tuning, the well-trained model from our framework is promising to work directly in deployment environments covering iris acquisitions with different forms of distribution shifts, which strongly demonstrates the superiority and generalization ability of our proposed method.