**ALDEN**, or **Active Learning with DivErse iNterpretations**, is an active learning approach for text classification. With local interpretations in DNNs, ALDEN identifies linearly separable regions of samples. Then, it selects samples according to their diversity of local interpretations and queries their labels.

Specifically, we first calculate the local interpretations in DNN for each sample as the gradient backpropagated from the final
predictions to the input features. Then, we use the most diverse interpretation of words in a sample to measure its diverseness. Accordingly, we select unlabeled samples with the maximally diverse interpretations for labeling and retrain the model with these
labeled samples.

**SKEP** is a self-supervised pre-training method for sentiment analysis. With the help of automatically-mined knowledge, SKEP conducts sentiment masking and constructs three sentiment knowledge prediction objectives, so as to embed sentiment information at the word, polarity and aspect level into pre-trained sentiment representation. In particular, the prediction of aspect-sentiment pairs is converted into multi-label classification, aiming to capture the dependency between words in a pair.

SKEP contains two parts: (1) Sentiment masking recognizes the sentiment information of an input sequence based on automatically-mined sentiment knowledge, and produces a corrupted version by removing these informations. (2) Sentiment pre-training objectives require the transformer to recover the removed information from the corrupted version. The three prediction objectives on top are jointly optimized: Sentiment Word (SW) prediction (on $\left.\mathrm{x}\_{9}\right)$, Word Polarity (SP) prediction (on $\mathrm{x}\_{6}$ and $\mathbf{x}\_{9}$ ), Aspect-Sentiment pairs (AP) prediction (on $\mathbf{x}\_{1}$ ). Here, the smiley denotes positive polarity. Notably, on $\mathrm{x}\_{6}$, only SP is calculated without SW, as its original word has been predicted in the pair prediction on $\mathbf{x}\_{1}$.

SKEP

SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis

ALDEN

Deep Active Learning for Text Classification with Diverse Interpretations

A large amount of information is stored in data tables. Users can search for data tables using a keyword-based query. A table is composed primarily of data values that are organized in rows and columns providing implicit structural information. A table is usually accompanied by secondary information such as the caption, page title, etc., that form the textual information. Understanding the connection between the textual and structural information is an important yet neglected aspect in table retrieval as previous methods treat each source of information independently. In addition, users can search for data tables that are similar to an existing table, and this setting can be seen as a content-based table retrieval. In this paper, we propose StruBERT, a structure-aware BERT model that fuses the textual and structural information of a data table to produce context-aware representations for both textual and tabular content of a data table. StruBERT features are integrated in a new end-to-end neural ranking model to solve three table-related downstream tasks: keyword- and content-based table retrieval, and table similarity. We evaluate our approach using three datasets, and we demonstrate substantial improvements in terms of retrieval and classification metrics over state-of-the-art methods.

Source	Deep Active Learning for Text Classification with Diverse Interpretations
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com