**StyleGAN** is a type of generative adversarial network. It uses an alternative generator architecture for generative adversarial networks, borrowing from style transfer literature; in particular, the use of [adaptive instance normalization](https://paperswithcode.com/method/adaptive-instance-normalization). Otherwise it follows Progressive [GAN](https://paperswithcode.com/method/gan) in using a progressively growing training regime. Other quirks include the fact it generates from a fixed value tensor not stochastically generated latent variables as in regular GANs. The stochastically generated latent variables are used as style vectors in the adaptive [instance normalization](https://paperswithcode.com/method/instance-normalization) at each resolution after being transformed by an 8-layer [feedforward network](https://paperswithcode.com/method/feedforward-network). Lastly, it employs a form of regularization called mixing regularization, which mixes two style latent variables during training.

**TayPO**, or **Taylor Expansion Policy Optimization**, refers to a set of algorithms that apply the $k$-th order Taylor expansions for policy optimization. This generalizes prior work, including [TRPO](https://paperswithcode.com/method/trpo) as a special case. It can be thought of unifying ideas from trust-region policy optimization and off-policy corrections. Taylor expansions share high-level similarities with both trust region policy search and off-policy corrections. To get high-level intuitions of such similarities, consider a simple 1D example of Taylor expansions. Given a sufficiently smooth real-valued function on the real line $f : \mathbb{R} \rightarrow \mathbb{R}$, the $k$-th order Taylor expansion of $f\left(x\right)$ at $x\_{0}$ is 

$$f\_{k}\left(x\right) = f\left(x\_{0}\right)+\sum^{k}\_{i=1}\left[f^{(i)}\left(x\_{0}\right)/i!\right]\left(x−x\_{0}\right)^{i}$$

where $f^{(i)}\left(x\_{0}\right)$ are the $i$-th order derivatives at $x\_{0}$. First, a common feature shared by Taylor expansions and trust-region policy search is the inherent notion of a trust region constraint. Indeed, in order for convergence to take place, a trust-region constraint is required $|x − x\_{0}| < R\left(f, x\_{0}\right)^{1}$. Second, when using the truncation as an approximation to the original function $f\_{K}\left(x\right) \approx f\left(x\right)$, Taylor expansions satisfy the requirement of off-policy evaluations: evaluate target policy with behavior data. Indeed, to evaluate the truncation $f\_{K}\left(x\right)$ at any $x$ (target policy), we only require the behavior policy "data" at $x\_{0}$ (i.e., derivatives $f^{(i)}\left(x\_{0}\right)$).

TayPO

Taylor Expansion Policy Optimization

StyleGAN

A Style-Based Generator Architecture for Generative Adversarial Networks

** LayoutReader** is a sequence-to-sequence model for reading order detection that uses both textual and layout information, where the layout-aware language model [LayoutLM](https://paperswithcode.com/method/layoutlmv2) is leveraged as an encoder. The generation step in the encoder-decoder structure tis modified to generate the reading order sequence.

In the encoding stage, LayoutReader packs the pair of source and target segments into a contiguous input sequence of LayoutLM and carefully designs the [self-attention mask](https://paperswithcode.com/methods/category/factorized-attention) to control the visibility between tokens. As shown in the Figure, LayoutReader allows the tokens in the source segment to attend to each other while preventing the tokens in the target segment from attending to the rightward context. If 1 means allowing and 0 means preventing, the detail of the mask $M$ is as follows:

$$ M\_{i, j}= \begin{cases}1, & \text { if } i<j \text { or } i, j \in \operatorname{src} \\ 0, & \text { otherwise }\end{cases} $$

where $i, j$ are the indices in the packed input sequence, so they may be from source or target segments; $i, j \in$ src means both tokens are from source segment.

In the decoding stage, since the source and target are reordered sequences, the prediction candidates can be constrained to the source segment. Therefore, we ask the model to predict the indices in the source sequence. The probability is calculated as follows:

$$
\mathcal{P}\left(x_{k}=i \mid x_{<k}\right)=\frac{\exp \left(e_{i}^{T} h\_{k}+b\_{k}\right)}{\sum_{j} \exp \left(e\_{j}^{T} h_{k}+b\_{k}\right)}
$$

where $i$ is an index in the source segment; $e\_{i}$ and $e\_{j}$ are the $\mathrm{i}$-th and $\mathrm{j}$-th input embeddings of the source segment; $h\_{k}$ is the hidden states at the $\mathrm{k}$-th time step; $b\_{k}$ is the bias at the $\mathrm{k}$-th time step.

Source	A Style-Based Generator Architecture for Generative Adversarial Networks
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com