正向传播

$$\color{input} x_j \color{black} =$$$$\sum_{i\in in(j)} w_{ij}\color{output} y_i\color{black} +b_j$$

正向传播

$$\color{output} y \color{black} = f(\color{input} x \color{black})$$

正向传播

$$\color{output} y \color{black} = f(\color{input} x \color{black})$$
$$\color{input} x_j \color{black} =$$$$\sum_{i\in in(j)} w_{ij}\color{output} y_i \color{black} + b_j$$

求导数

$$w_{ij} = w_{ij} - \alpha \color{dweight}\frac{dE}{dw_{ij}}$$

[注意] 更新规则非常简单：如果权重在权重增加时下降 ($$\color{dweight}\frac{dE}{dw_{ij}}\color{black} < 0$$)，则增加权重；否则，如果错误在权重增加 ($$\color{dweight}\frac{dE}{dw_{ij}} \color{black} > 0$$) 时增加，则降低权重。

其他导数

• 节点的总输入值 $$\color{dinput}\frac{dE}{dx}$$
• 节点的输出 $$\color{doutput}\frac{dE}{dy}$$。

反向传播

$$\color{doutput} \frac{\partial E}{\partial y_{output}} \color{black} = \color{output} y_{output} \color{black} - \color{output} y_{target}$$

反向传播

$$\color{dinput} \frac{\partial E}{\partial x} \color{black} = \frac{dy}{dx}\color{doutput}\frac{\partial E}{\partial y} \color{black} = \frac{d}{dx}f(\color{input}x\color{black})\color{doutput}\frac{\partial E}{\partial y}$$

反向传播

$$\color{dweight} \frac{\partial E}{\partial w_{ij}} \color{black} = \frac{\partial x_j}{\partial w_{ij}} \color{dinput}\frac{\partial E}{\partial x_j} \color{black} = \color{output}y_i \color{dinput} \frac{\partial E}{\partial x_j}$$

反向传播

$$\color{doutput} \frac{\partial E}{\partial y_i} \color{black} = \sum_{j\in out(i)} \frac{\partial x_j}{\partial y_i} \color{dinput} \frac{\partial E}{\partial x_j} \color{black} = \sum_{j\in out(i)} w_{ij} \color{dinput} \frac{\partial E}{\partial x_j}$$