What is: DV3 Attention Block?
Source | Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning |
Year | 2000 |
Data Source | CC BY-SA - https://paperswithcode.com |
DV3 Attention Block is an attention-based module used in the Deep Voice 3 architecture. It uses a dot-product attention mechanism. A query vector (the hidden states of the decoder) and the per-timestep key vectors from the encoder are used to compute attention weights. This then outputs a context vector computed as the weighted average of the value vectors.