Knowing Where and What: Unified Word Block Pretraining for Document Understanding

1 code implementation • 28 Jul 2022 • Song Tao, Zijian Wang, Tiantian Fan, Canjie Luo, Can Huang

In this paper, we focus on the embedding learning of word blocks containing text and layout information, and propose UTel, a language model with Unified TExt and Layout pre-training.

Contrastive Learning document understanding +3