**SentencePiece** is a subword tokenizer and detokenizer for natural language processing. It performs subword segmentation, supporting the byte-pair-encoding ([BPE](https://paperswithcode.com/method/bpe)) algorithm and unigram language model, and then converts this text into an id sequence guarantee perfect reproducibility of the normalization and subword segmentation.

A **Compact Global Descriptor** is an image model block for modelling interactions between positions across different dimensions (e.g., channels, frames). This descriptor enables subsequent convolutions to access the informative global features. It is a form of attention.

Compact Global Descriptor

Compact Global Descriptor for Neural Networks

SentencePiece

SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing

**SKEP** is a self-supervised pre-training method for sentiment analysis. With the help of automatically-mined knowledge, SKEP conducts sentiment masking and constructs three sentiment knowledge prediction objectives, so as to embed sentiment information at the word, polarity and aspect level into pre-trained sentiment representation. In particular, the prediction of aspect-sentiment pairs is converted into multi-label classification, aiming to capture the dependency between words in a pair.

SKEP contains two parts: (1) Sentiment masking recognizes the sentiment information of an input sequence based on automatically-mined sentiment knowledge, and produces a corrupted version by removing these informations. (2) Sentiment pre-training objectives require the transformer to recover the removed information from the corrupted version. The three prediction objectives on top are jointly optimized: Sentiment Word (SW) prediction (on $\left.\mathrm{x}\_{9}\right)$, Word Polarity (SP) prediction (on $\mathrm{x}\_{6}$ and $\mathbf{x}\_{9}$ ), Aspect-Sentiment pairs (AP) prediction (on $\mathbf{x}\_{1}$ ). Here, the smiley denotes positive polarity. Notably, on $\mathrm{x}\_{6}$, only SP is calculated without SW, as its original word has been predicted in the pair prediction on $\mathbf{x}\_{1}$.

Source	SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com