Viet-Anh on Software Logo

What is: SKEP?

SourceSKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

SKEP is a self-supervised pre-training method for sentiment analysis. With the help of automatically-mined knowledge, SKEP conducts sentiment masking and constructs three sentiment knowledge prediction objectives, so as to embed sentiment information at the word, polarity and aspect level into pre-trained sentiment representation. In particular, the prediction of aspect-sentiment pairs is converted into multi-label classification, aiming to capture the dependency between words in a pair.

SKEP contains two parts: (1) Sentiment masking recognizes the sentiment information of an input sequence based on automatically-mined sentiment knowledge, and produces a corrupted version by removing these informations. (2) Sentiment pre-training objectives require the transformer to recover the removed information from the corrupted version. The three prediction objectives on top are jointly optimized: Sentiment Word (SW) prediction (on x_9)\left.\mathrm{x}\_{9}\right), Word Polarity (SP) prediction (on x_6\mathrm{x}\_{6} and x_9\mathbf{x}\_{9} ), Aspect-Sentiment pairs (AP) prediction (on x_1\mathbf{x}\_{1} ). Here, the smiley denotes positive polarity. Notably, on x_6\mathrm{x}\_{6}, only SP is calculated without SW, as its original word has been predicted in the pair prediction on x_1\mathbf{x}\_{1}.