Using TensorFlow to Create a Model for Making Predictions
TensorFlow is an open source artificial intelligence framework/library for high performance numerical computation. Its flexible architecture allows easy deployment of computation across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices. Originally developed by researchers and engineers at Google, it comes with strong support for machine learning and deep learning and the flexible numerical computation core is used across many other scientific domains.Tensors are high dimensional generalizations of matrices, where each tensor is a computational unit that can compute based on defined operations. Each tensor takes input, computes, and pass results to next tensor in a continuous flow and hence the whole system is called as TensorFlow. A tensor can also act as a container that can house data in N dimensions, along with its linear operations.
The example below shows how to implement the TensorFlow framework to make deep learning and prediction. It shows an example to predict if an underwater object is rock or a mine based on sensor data from sonar.
For more details on TensorFlow, see this link: https://www.tensorflow.org/tutorials/
TensorFlow to predict if an underwater object is Rock or Mine¶
A naval mine is a self-contained explosive device placed in water to damage or destroy ships or submarines.Situation: You have been hired by US Navy to create a model that can detect the difference between an underwater mine and a rock, because a mine dangerous but a rock is not. And it is difficult for humans to judge if an underwater object is rock or a mine.
Solution: We will write a model in TensorFlow to make the prediction for us if the underwater object is rock or a mine.
Dataset used is Sonar.csv, downloaded from the GitHub loccation:
https://github.com/selva86/datasets/blob/master/Sonar.csv
Creating and Training the Model¶
In [1]:
# Importing the libraries
import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np
import pandas as pd
from sklearn.preprocessing import LabelEncoder
from sklearn.utils import shuffle
from sklearn.model_selection import train_test_split
In [2]:
# Reading the dataset
def read_dataset():
dir_path = "./MLData/00_Misc/"
df = pd.read_csv(dir_path + "Sonar.csv")
X = df[df.columns[0:60]].values
y = df[df.columns[60]]
# Encode the labels
encoder = LabelEncoder()
encoder.fit(y)
y = encoder.transform(y)
Y = one_hot_encode(y)
return (X, Y)
# Define the encoder function
def one_hot_encode(labels):
n_labels = len(labels)
n_unique_lables = len(np.unique(labels))
encode = np.zeros((n_labels, n_unique_lables))
encode[np.arange(n_labels), labels] = 1
return encode
# Read the datset
X, Y = read_dataset()
# Shuffle the dataset to mix up the rows
X, Y = shuffle(X, Y, random_state=1)
In [3]:
# Convert the dataset into train and test sets
train_x, test_x, train_y, test_y = train_test_split(X, Y, test_size=0.20, random_state=415)
# Inspect the shape of the training and testing sets
print('Shape of train_x:', train_x.shape)
print('Shape of train_y:', train_y.shape)
print('Shape of test_x:', test_x.shape)
print('Shape of test_y:', test_y.shape)
In [4]:
# Define the important parameters and variable to work with the tensors
learning_rate = 0.3
training_epochs = 1000
cost_history = np.empty(shape=[1], dtype=float) # loss function
n_dim = X.shape[1]
print('Dimention n_dim =', n_dim)
n_class = 2
# Save model in NMI (Naval Mine Identifier) folder
model_path = './TensorFlow/NMI/'
# Define the number of hidden layers and neurons for each layer
n_hidden_1 = 60
n_hidden_2 = 60
n_hidden_3 = 60
n_hidden_4 = 60
# Inputs and outputs
x = tf.placeholder(tf.float32, [None, n_dim])
y_ = tf.placeholder(tf.float32, [None, n_class])
# Model parameters
W = tf.Variable(tf.zeros([n_dim, n_class]))
b = tf.Variable(tf.zeros([n_class]))
In [5]:
# Define the model
def multilayer_perceptron(x, weights, biases):
# Hidden layer with sigmoid activations
layer_1 = tf.add(tf.matmul(x, weights['h1']), biases['b1'])
layer_1 = tf.nn.sigmoid(layer_1)
# Hidden layer with sigmoid activations
layer_2 = tf.add(tf.matmul(layer_1, weights['h2']), biases['b2'])
layer_2 = tf.nn.sigmoid(layer_2)
# Hidden layer with sigmoid activations
layer_3 = tf.add(tf.matmul(layer_2, weights['h3']), biases['b3'])
layer_3 = tf.nn.sigmoid(layer_3)
# Hidden layer with RELU activations
layer_4 = tf.add(tf.matmul(layer_3, weights['h4']), biases['b4'])
layer_4 = tf.nn.relu(layer_4)
# Output layer with linear activations
out_layer = tf.matmul(layer_4, weights['out']) + biases['out']
return out_layer
# Define the weights and the biases for each layer
weights = {
'h1': tf.Variable(tf.truncated_normal([n_dim, n_hidden_1])),
'h2': tf.Variable(tf.truncated_normal([n_hidden_1, n_hidden_2])),
'h3': tf.Variable(tf.truncated_normal([n_hidden_2, n_hidden_3])),
'h4': tf.Variable(tf.truncated_normal([n_hidden_3, n_hidden_4])),
'out': tf.Variable(tf.truncated_normal([n_hidden_4, n_class])),
}
biases = {
'b1': tf.Variable(tf.truncated_normal([n_hidden_1])),
'b2': tf.Variable(tf.truncated_normal([n_hidden_2])),
'b3': tf.Variable(tf.truncated_normal([n_hidden_3])),
'b4': tf.Variable(tf.truncated_normal([n_hidden_4])),
'out': tf.Variable(tf.truncated_normal([n_class])),
}
In [6]:
# Initialize all the variables
init = tf.global_variables_initializer()
saver = tf.train.Saver()
# Call your model defined
y = multilayer_perceptron(x, weights, biases)
# Define the cost function and optimizer
cost_function = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(logits=y, labels=y_))
training_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost_function)
# Launch the graph
sess = tf.Session()
sess.run(init)
mse_history = []
accuracy_history = []
In [7]:
# Calculate the cost and accuracy of each epoch
for epoch in range(training_epochs):
sess.run(training_step, feed_dict={x: test_x, y_: test_y})
cost = sess.run(cost_function, feed_dict={x: test_x, y_: test_y})
cost_history = np.append(cost_history, cost)
correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
pred_y = sess.run(y, feed_dict={x: test_x})
mse = tf.reduce_mean(tf.square(pred_y - test_y))
mse_ = sess.run(mse)
mse_history.append(mse_)
accuracy = (sess.run(accuracy, feed_dict={x: test_x, y_: test_y}))
accuracy_history.append(accuracy)
print('Epoch: {:<6} Cost: {:<25} MSE: {:<22} Training Accuracy: {:<5}'.format(epoch, cost, mse_, accuracy))
save_path = saver.save(sess, model_path)
print('\nModel saved in folder:', save_path)
In [8]:
# Print the final accuracy
correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print('Test Accuracy: ',(sess.run(accuracy, feed_dict={x: test_x, y_: test_y})))
# Print the final mean square error
pred_y = sess.run(y, feed_dict={x: test_x})
mse = tf.reduce_mean(tf.square(pred_y - test_y))
print('Mean Square Error: %.4f' % sess.run(mse))
In [9]:
# Plot Accuracy graph
fig = plt.figure(figsize=(10,7))
plt.plot(accuracy_history)
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.show()
In [10]:
# Plot Loss graph
fig = plt.figure(figsize=(10,7))
plt.plot(range(len(cost_history)),cost_history)
plt.axis([0,training_epochs,0,np.max(cost_history)/100])
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.show()
Using the Model to make Prediction if Rock or Mine¶
In [11]:
# Reading the new dataset
def read_dataset():
dir_path = "./MLData/00_Misc/"
df = pd.read_csv(dir_path + "Sonar_new.csv")
X_new = df[df.columns[0:60]].values
y_new = df[df.columns[60]]
# Encode the labels
encoder = LabelEncoder()
encoder.fit(y_new)
y_new = encoder.transform(y_new)
Y_new = one_hot_encode(y_new)
return (X_new, Y_new)
# Define the encoder function
def one_hot_encode(labels):
n_labels = len(labels)
n_unique_lables = len(np.unique(labels))
encode = np.zeros((n_labels, n_unique_lables))
encode[np.arange(n_labels), labels] = 1
return encode
# Read the datset
X_new, Y_new = read_dataset()
In [12]:
X_new.shape, Y_new.shape
Out[12]:
In [13]:
saver.restore(sess, model_path)
prediction = tf.argmax(y, 1)
correct_prediction = tf.equal(prediction, tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print("**************************************************************")
print("* 1 stands for M, (i.e. Mine) and 0 stands for R (i.e. Rock) *")
print("**************************************************************")
for i in range(1,15):
prediction_run = sess.run(prediction, feed_dict={x:X_new[i].reshape(1,60)})
accuracy_run = sess.run(accuracy, feed_dict={x:X_new[i].reshape(1,60), y_:Y_new[i].reshape(1,2)})
print(i,"Original Class: ", int(sess.run(y_[i][1],feed_dict={y_:Y_new})), " Predicted Values: ", prediction_run[0] )
print("Accuracy: ",str(accuracy_run*100)+"%")
In [ ]: