H 4.3 - Backpropagation.ipynb

Kernel: Julia 1.4

Backpropagation

sigmoid(x)  = 1/(1+exp(-x))
dsigmoid(x) = exp(-x)/(1+exp(-x))^2

x  = 3
w1 = 0.01
w2 = -5
b  = -1
t  = 1

z1 = x*w1
a1 = max(0,z1)
z2 = w2*a1+b
y  = sigmoid(z2)
C = (1/2)*(y-t)^2

dw1 = x * w2 * dsigmoid(z2) * (y-t)

2.080916562170455

dw2 = a1 * dsigmoid(z2) * (y-t)

-0.004161833124340909

db  = 1 * dsigmoid(z2) * (y-t)

-0.13872777081136364