from IPython.display import Image
Image(filename='./pics/Deeplearning 2.png')
Eine Schicht von Neuronen:
Input: $\vec x^T = (x_1, x_2, \dots x_n)$
Output des j-ten Neurons: $$ h_j = \sigma(\sum_{i=1}^n w_{ij} x_i + b_j) $$
Mit der Gewichtsmatrix $W$:
$$ \vec h = \sigma(\vec x \cdot W + \vec b) $$plot_train_data(X_train[t_train==0], X_train[t_train==1])
Im konstruierten Merkmalsraum sind die Daten linear-separabel.
phi_train = X_train**2
plot_train_transformed(phi_train[t_train==0], phi_train[t_train==1])
Mit Neuronalen Netzen können Feature-Transformationen gelernt werden.
Aktivität der (ersten) Hidded-Layer:
$$ \vec h^{(1)} = \sigma_1 \left(\vec x \cdot W^{(1)} + \vec b^{(1)} \right) $$Aktivität des Output-Neurons $o$:
$$ o = h^{(2)}= \sigma_2 \left( \vec h^{(1)} \cdot W^{(2)} + b^{(2)} \right) $$# (first) hidden layer
a = tf.matmul(x, W_h) + b_h
# activity function "rectified linear units"
h = tf.nn.relu(a)
# output neuron:
y = logistic_function(tf.matmul(h, W_o) + b_o)
cross_entropy = -tf.reduce_sum(y_*tf.log(y[:,0]) + (1.-y_)*tf.log(1.-y[:,0]))
l2_reg = tf.reduce_sum(tf.square(W_h)) + tf.reduce_sum(tf.square(W_o))
lambda_ = 0.002
cost = cross_entropy + lambda_ * l2_reg
# Note: there is also a visualization tool called TensorBoard, not used here.
plt.plot(range(epochs), cost_, '-b')
plt.xlabel('Iterations')
plt.ylabel('Cost')
cost_
array([ 44.88797379, 44.88481522, 44.88169098, ..., 6.09101915, 6.09098625, 6.09095526])
plot_contour(X_train[t_train==0], X_train[t_train==1], 'train data')
plot_contour(X_test[t_test==0], X_test[t_test==1], 'test data')