Viet-Anh on Software Logo

What is: Skip-gram Word2Vec?

SourceEfficient Estimation of Word Representations in Vector Space
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Skip-gram Word2Vec is an architecture for computing word embeddings. Instead of using surrounding words to predict the center word, as with CBow Word2Vec, Skip-gram Word2Vec uses the central word to predict the surrounding words.

The skip-gram objective function sums the log probabilities of the surrounding nn words to the left and right of the target word w_tw\_{t} to produce the following objective:

J_θ=1TT_t=1_njn,0logp(w_j+1w_t)J\_\theta = \frac{1}{T}\sum^{T}\_{t=1}\sum\_{-n\leq{j}\leq{n}, \neq{0}}\log{p}\left(w\_{j+1}\mid{w\_{t}}\right)