Transformer介绍

Transformer Neural Networks - EXPLAINED! (Attention is all you need) ==================================================================== 就喜欢这种不讲公式的。

缩放的点乘注意力ScaledDotProductAttention

缩放的点乘注意力ScaledDotProductAttention 就是Transformer中使用的注意力 ``` import math import torch import torch.nn as nn import torch.nn.functional as F from torch import Tensor import numpy …