from IPython.display import Image
Image(filename='./pics/Deeplearning 2.png')
Eine Schicht von Neuronen:
Input: $\vec x^T = (x_1, x_2, \dots x_n)$
Output des j-ten Neurons: $$ h_j = \sigma(\sum_{i=1}^n w_{ij} x_i + b_j) $$
Mit der Gewichtsmatrix $W$:
$$ \vec h = \sigma(\vec x \cdot W + \vec b) $$plot_train_data(X_train[t_train==0], X_train[t_train==1])
Im konstruierten Merkmalsraum sind die Daten linear-separabel.
phi_train = X_train**2
plot_train_transformed(phi_train[t_train==0], phi_train[t_train==1])
Mit Neuronalen Netzen können Feature-Transformationen gelernt werden.
Aktivität der (ersten) Hidded-Layer:
$$ \vec h^{(1)} = \sigma_1 \left(\vec x \cdot W^{(1)} + \vec b^{(1)} \right) $$Aktivität des Output-Neurons $o$:
$$ o = h^{(2)}= \sigma_2 \left( \vec h^{(1)} \cdot W^{(2)} + b^{(2)} \right) $$# (first) hidden layer
a = tf.matmul(x, W_h) + b_h
# activity function "rectified linear units"
h = tf.nn.relu(a)
# output neuron:
y = logistic_function(tf.matmul(h, W_o) + b_o)
cross_entropy = -tf.reduce_sum(y_*tf.log(y[:,0]) + (1.-y_)*tf.log(1.-y[:,0]))
l2_reg = tf.reduce_sum(tf.square(W_h)) + tf.reduce_sum(tf.square(W_o))
lambda_ = 0.002
cost = cross_entropy + lambda_ * l2_reg
# Note: there is also a visualization tool called TensorBoard, not used here.
plt.plot(range(epochs), cost_, '-b')
plt.xlabel('Iterations')
plt.ylabel('Cost')
cost_
array([ 44.88797379, 44.88481522, 44.88169098, ..., 6.09101915, 6.09098625, 6.09095526])
plot_contour(X_train[t_train==0], X_train[t_train==1], 'train data')
plot_contour(X_test[t_test==0], X_test[t_test==1], 'test data')
y_e = y.eval(feed_dict={x: X_test})
((y_e>0.5).reshape(-1)==t_test).mean()
0.84999999999999998
plot_hidden_neurons_activities()
siehe auch [Mon14, Pas14]
plot_data() # each point is a training example
plt.plot(range(epochs), cost_, '-b')
plt.xlabel('Iterations')
plt.ylabel('Cost')
<matplotlib.text.Text at 0x11af2e9b0>
#train classification error
y_e = y.eval(feed_dict={x: X})
((y_e>0.5).reshape(-1)==t).mean()
1.0
plot_hidden_space(h=h, x=x, grid_points=grid_points, x_0=x_0, x_1=x_1, W_o=W_o, b_o = b_o, blue=blue, red=red)
plot_spiral_data()
Aktivitäten der Hidded-Layer:
$$ \vec h^{(1)} = \sigma_1 \left(\vec x \cdot W^{(1)} + \vec b^{(1)} \right) $$$$ \vec h^{(2)}= \sigma_2 \left( \vec h^{(1)} \cdot W^{(2)} + \vec b^{(2)} \right) $$Aktivität der Output-Neuronen $\vec o$:
$$ \vec o = \vec h^{(3)}= \sigma_3 \left( \vec h^{(2)} \cdot W^{(3)} + \vec b^{(3)} \right) $$mit
plt.plot(range(epochs), cost_, '-b')
plt.xlabel('Iterations')
plt.ylabel('Cost')
<matplotlib.text.Text at 0x11b37ea20>
plot_spriral_db()
plot_spiral_hdb()
from IPython.display import Image
Image(filename='./pics/Deeplearning 7.png')
from IPython.display import Image
Image(filename='./pics/picture-vs-random-pixels.png')
from IPython.display import Image
Image(filename='./pics/traditional-ml.png')
# adapted from [LeCun]
from IPython.display import Image
Image(filename='./pics/tradional-imageprocessing-speech-approach.png')
# adapted from [LeCun]
from IPython.display import Image
Image(filename='./pics/trainable-feature-transformations.png')
# adapted from [LeCun]
Courses and further readings:
Scientific Papers: