Multinomial Classification
List of Tensorflow 2.0 Tutorials
- TF2.0 - 01.Simple Linear Regression
- TF2.0 - 02.Linear Regression and How to minimize cost
- TF2.0 - 03.Multiple Linear Regression
- TF2.0 - 04.Logistic Regression
- TF2.0 - 05.Multinomial Classification
- TF2.0 - 06.Iris Data Classification
Following the binary classifier, you can also easily implement a multi-category classifier.
(Review) Logistic Regression
Hypothesis
\[{H(X)} = {1 \over {1+ e^{-XW}}}\]Cost Function
\[Cost(W) = {1 \over m} {\sum_{i=1}^m c(H(x_{i}), y_{i})}\]Cross Entropy in Logistic Classification
\[c(H(x), y) = \begin{cases}-log(H(x)) : y=1 \\ -log(1-H(x)) : y=0\end{cases} = -y log(H(x)) - (1-y) log(1-H(x))\]Minimizing Cost
\[W_{new} = W_{old} - \alpha {\partial \over {\partial W}} Cost(W)\]Multinomial Classification
Softmax Function (Hypothesis)
\[S(y_{i}) = {e^{y_i} \over \sum_{j=1}^n e^{y_{j}}}\] \[n: Number\ of\ classes \\ i: i\ class\]Cost Function
\[Cost(W) = {1 \over m} {\sum_{i=1}^m D(S(X_{i}W + b),L_{i})} \\ m: Number\ of\ instances \\ i: i\ instance\]Cross Entropy in Multinomial Classification
\[D(S, L) = - \sum_{j=1}^n L_{j} log(S(y_{j})) \\ \\ n: Number\ of\ classes \\ \\ j: j\ class\]Minimizing Cost
\[W_{new} = W_{old} - \alpha {\partial \over {\partial W}} Cost(W)\]Implement
import tensorflow as tf
import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
print("TensorFlow Version: %s" % (tf.__version__))
TensorFlow Version: 2.0.0
Data
x_data = [[1, 2, 1, 1],
[2, 1, 3, 2],
[3, 1, 3, 4],
[4, 1, 5, 5],
[1, 7, 5, 5],
[1, 2, 5, 6],
[1, 6, 6, 6],
[1, 7, 7, 7]]
y_data = [[0, 0, 1],
[0, 0, 1],
[0, 0, 1],
[0, 1, 0],
[0, 1, 0],
[0, 1, 0],
[1, 0, 0],
[1, 0, 0]]
x_data = np.array(x_data, dtype=np.float32)
y_data = np.array(y_data, dtype=np.float32)
nb_classes = 3
# Weights
tf.random.set_seed(2020)
W = tf.Variable(tf.random.normal([4, nb_classes], mean=0.0))
b = tf.Variable(tf.random.normal([nb_classes], mean=0.0))
print('# Weights: \n', W.numpy(), '\n\n# Bias: \n', b.numpy())
# Weights:
[[-0.10099822 0.6847899 1.6258513 ]
[ 0.88112587 -0.63692456 -0.1427695 ]
[ 0.82411087 -0.91326994 -0.4510184 ]
[ 0.58053356 1.3066356 -0.60428965]]
# Bias:
[ 0.38414612 -0.6159301 -0.5453214 ]
# Learning Rate
learning_rate = 0.01
# Softmax Function
def softmax(X):
sm = tf.nn.softmax(tf.matmul(x_data, W) + b)
return sm
# Training
for i in range(10000+1):
with tf.GradientTape() as tape:
sm = softmax(x_data)
cost = tf.reduce_mean(-tf.reduce_sum(y_data*tf.math.log(sm), axis=1))
W_grad, b_grad = tape.gradient(cost, [W, b])
W.assign_sub(learning_rate * W_grad)
b.assign_sub(learning_rate * b_grad)
if i % 1000 == 0:
print(">>> #%s \n Weights: \n%s \n Bias: \n%s \n cost: %s\n" % (i, W.numpy(), b.numpy(), cost.numpy()))
>>> #0
Weights:
[[-0.11553647 0.6918998 1.6332797 ]
[ 0.8638605 -0.62458277 -0.13784593]
[ 0.797113 -0.8949384 -0.44235206]
[ 0.55236673 1.3261304 -0.5956176 ]]
Bias:
[ 0.37683368 -0.61232066 -0.5416184 ]
cost: 5.2257767
>>> #1000
Weights:
[[-0.8411931 0.75385374 2.2969847 ]
[-0.07680082 -0.16606328 0.34429583]
[ 0.618117 -0.6086176 -0.54967654]
[ 0.24572133 1.326133 -0.28897592]]
Bias:
[-0.13876517 -0.8163229 0.1779835 ]
cost: 0.50721896
>>> #2000
Weights:
[[-1.1914808 0.80435884 2.5967705 ]
[-0.14087628 -0.18363482 0.42594275]
[ 0.8594195 -0.52478886 -0.8748073 ]
[ 0.22371401 1.3248692 -0.26570395]]
Bias:
[-0.40732294 -0.9690039 0.59922254]
cost: 0.43953097
>>> #3000
Weights:
[[-1.464515 0.8519873 2.8221765 ]
[-0.18783872 -0.19453 0.48380077]
[ 1.0863445 -0.49375 -1.1327715 ]
[ 0.16596384 1.3500665 -0.23314708]]
Bias:
[-0.63186496 -1.08178 0.9365396 ]
cost: 0.39596182
>>> #4000
Weights:
[[-1.697922 0.9001363 3.0074346 ]
[-0.22953543 -0.20061918 0.5315868 ]
[ 1.3032706 -0.4934833 -1.3499686 ]
[ 0.09487277 1.3891125 -0.2010959 ]]
Bias:
[-0.83245873 -1.1711738 1.226527 ]
cost: 0.36297297
>>> #5000
Weights:
[[-1.9057775 0.9478083 3.1676147 ]
[-0.26796636 -0.20388351 0.57328254]
[ 1.5109538 -0.5109507 -1.5401874 ]
[ 0.01810047 1.4356134 -0.17081594]]
Bias:
[-1.0164189 -1.2455534 1.4848666]
cost: 0.33603936
>>> #6000
Weights:
[[-2.0948424 0.9941726 3.3103127 ]
[-0.3038853 -0.20546497 0.6107832 ]
[ 1.7097456 -0.53860354 -1.7113296 ]
[-0.06074882 1.485766 -0.1421119 ]]
Bias:
[-1.1872404 -1.3096143 1.7197511]
cost: 0.3131696
>>> #7000
Weights:
[[-2.2689846 1.0387573 3.4398637 ]
[-0.33769488 -0.20603302 0.645161 ]
[ 1.8999859 -0.5720022 -1.8681755 ]
[-0.13977788 1.5373282 -0.11464822]]
Bias:
[-1.3469772 -1.3662182 1.9360921]
cost: 0.29330796
>>> #8000
Weights:
[[-2.4307153 1.0813508 3.5589974 ]
[-0.3696618 -0.2059924 0.67708796]
[ 2.0820625 -0.60849404 -2.0137668 ]
[-0.217931 1.5890023 -0.08818024]]
Bias:
[-1.4970391 -1.417211 2.1371472]
cost: 0.2758096
>>> #9000
Weights:
[[-2.5818226 1.1218913 3.6695557 ]
[-0.39999598 -0.20558535 0.70701504]
[ 2.2564096 -0.64648116 -2.1501348 ]
[-0.29460648 1.6400516 -0.06255731]]
Bias:
[-1.6384954 -1.4638423 2.325234 ]
cost: 0.26023802
>>> #10000
Weights:
[[-2.7236586 1.1604131 3.772864 ]
[-0.4288627 -0.20496319 0.7352611 ]
[ 2.4234724 -0.6849797 -2.278707 ]
[-0.36947724 1.6900569 -0.03769292]]
Bias:
[-1.7722151 -1.5069835 2.5020947]
cost: 0.24627577
Predict
predicted = tf.argmax(softmax(x_data), axis=1)
real = tf.argmax(y_data, axis=1)
def acc(predicted, real):
accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, real), dtype=tf.float32))
return accuracy
accuracy = acc(predicted, real).numpy()
print("Accuracy: %s" % accuracy)
Accuracy: 1.0
predicted.numpy()
array([2, 2, 2, 1, 1, 1, 0, 0])
real.numpy()
array([2, 2, 2, 1, 1, 1, 0, 0])
List of Tensorflow 2.0 Tutorials
- TF2.0 - 01.Simple Linear Regression
- TF2.0 - 02.Linear Regression and How to minimize cost
- TF2.0 - 03.Multiple Linear Regression
- TF2.0 - 04.Logistic Regression
- TF2.0 - 05.Multinomial Classification
- TF2.0 - 06.Iris Data Classification
댓글남기기