Progressive Multitask Learning Network for Online Chinese Signature Segmentation and Recognition

Abstract

Recently, the booming of electronic devices has revolutionized the way we sign in our daily life. The sudden surge of online Chinese signatures calls for need online Chinese signature segmentation as prerequisite for downstream tasks such as building up database of Chinese characters and verifying signatures based upon individual characters. However, common approaches deriving from over-segmentation do not apply well to Chinese signatures, which have little linguistic meanings but instead have flourish, artistic styles, composite structures or even word overlaps. To cope with those difficulties, this paper exploits the benefits of signature recognition to boost segmentation performance and proposes a progressive multitask learning network (PMLNet) for online Chinese signature segmentation and recognition. PMLNet consists of a dual channel stroke feature extraction block (DSF-Block), a stacked transformer encoder block (STE-Block) and a progressive multitask learning block (PML-Block). DSF-block is used to extract stroke-wise spatial features and semantic features through dual channels; STE-block is used to model long-range dependencies among different strokes and enhance their feature representations; PML-block is used to branch interactively and progressively the signature’s final segmentation and recognition. Specifically, we introduce a progressive learning strategy in PML-block to fine-tune segmentation and recognition results by fully leveraging their reciprocal relationship. The experiment results on our private database show that PMLNet achieves 8.77% higher accurate rate (AR), 9.15% higher correct rate (CR) and 9.93% higher sample-level segmentation accurate rate ($\mathrm {ACC_{seg}}$) than SOTAs.