LINE is a novel network embedding method which is suitable for arbitrary types of information networks: undirected, directed, and/or weighted. The method optimizes a carefully designed objective function that preserves both the local and global network structures.

Source: [Tang et al.](https://arxiv.org/pdf/1503.03578v1.pdf)

Image source: [Tang et al.](https://arxiv.org/pdf/1503.03578v1.pdf)

In the ALIGN method, visual and language representations are jointly trained from noisy image alt-text data. The image and text encoders are learned via contrastive loss (formulated as normalized softmax) that pushes the embeddings of the matched image-text pair together and pushing those of non-matched image-text pair apart. The model learns to align visual and language representations of the image and text pairs using the contrastive loss. The representations can be used for vision-only or vision-language task transfer. Without any fine-tuning, ALIGN powers zero-shot visual classification and cross-modal search including image-to-text search, text-to image search and even search with joint image+text queries.

ALIGN

Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision

LINE

LINE: Large-scale Information Network Embedding

**Unigram Segmentation** is a subword segmentation algorithm based on a unigram language model. It provides multiple segmentations with probabilities. The language model allows for emulating the noise generated during the segmentation of actual data.

The unigram language model makes an assumption that each subword occurs independently, and consequently, the probability of a subword sequence $\mathbf{x} = (x_1,\ldots,x_M)$ is
formulated as the product of the subword occurrence probabilities
$p(x_i)$:

$$
  P(\mathbf{x}) = \prod_{i=1}^{M} p(x_i), \\\\
  \forall i\,\, x_i \in \mathcal{V},\,\,\,
  \sum_{x \in \mathcal{V}} p(x) = 1, \nonumber
$$

where $\mathcal{V}$ is a pre-determined vocabulary.  The most probable
segmentation $\mathbf{x}^*$ for the input sentence $X$ is then given by:

$$
  \mathbf{x}^{*} = \text{argmax}_{\mathbf{x} \in \mathcal{S}(X)} P(\mathbf{x}),
$$

where $\mathcal{S}(X)$ is a set of segmentation candidates built from
the input sentence $X$.  $\mathbf{x}^*$ is obtained with the Viterbi
algorithm.

Source	LINE: Large-scale Information Network Embedding
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com