Very Simple Intro to Tensorflow, Part 1

This is a very simple intro to using Tensorflow. We will seriously under-utilize it on a basic classification problem - the famous iris dataset. The dataset contains 4 features, and 3 classes of Iris species (Iris setosa, Iris virginica and Iris versicolor). There are only 150 datapoints in the dataset. It can be found here. The source code here is mostly based on, and modified from, the great book, Hands-on Machine Learning with Scikit-Learn and Tensorflow. To do this we need the following Python packages tensorflow scipy numpy sklearn pandas matplotlib ipython jupyter We will do all our work using a Jupyter notebook. After pip installing all the necessary packages, type

jupyter notebook

Go to the newly opened tab in you browser and create a new notebook.

import tensorflow as tf
import numpy as np
import pandas
import matplotlib
import sklearn as sk

Let's grab the dataset (from the local filesystem):

data=pandas.read_csv("/path/to/iris.csv")"

We then need to label the classes. In their raw form they are represented as string literls - the name sof the iris species. But we want some numeric label, well tensorflow does, anyway. Then we need to split our dataset into a training set and a test set. I found the easiest way to split the dataset was just to use scikit-learn.

data=pandas.read_csv("/path/to/iris.csv")
X=data.T[:4].T.as_matrix()
Y=data.T[4:].T
y=pandas.get_dummies(Y)
Xd=X
yd=y.as_matrix()
labels = np.argmax(yd, axis=1)
X_train, X_test, y_train, y_test = train_test_split(X, labels, test_size=0.33)

Now we have our training and test sets, we can get to the fun part. We will create a 3-layered neural network with the aim to classify the who iris dataset, into the correct iris species classes. The below diagram gives a simple outline of the network. We have 4 inputs (the green nodes). The first hidden layer will have 10 nodes. This number is chosen relatively arbitrarily, and is, in all likelihood superfluous. We could probably get away with fewer nodes. But for the purposes of this demonstration we will stick with 10. The activation function for each node will be the sigmoid activation function. Tensorflow allows the choice of several saturating and non-saturating activation functions - sigmoid, relu, atanh for example). The next hidden layer will have 5 nodes, and will also use sigmoid activation functions. Lastly the output layer has three nodes - one node for each class - and will use the softmax activation function, to choose a single node, or single class, for any given input.

Let's create a 3-layer neural network

Xvar = tf.placeholder(tf.float32, shape=(None, 4), name="Xvar")
yvar = tf.placeholder(tf.int64,   shape=(None), name="yvar")
lr=0.01
with tf.name_scope("iris_dnn"):
    hidden1 = fully_connected(Xvar,10,  activation_fn=tf.nn.sigmoid)
    hidden2 = fully_connected(hidden1,5,activation_fn=tf.nn.sigmoid)
    logits= fully_connected(hidden2,3, activation_fn=tf.nn.softmax)
    
with tf.name_scope("loss"):
    xentropy=tf.nn.sparse_softmax_cross_entropy_with_logits(labels= yvar, logits=logits)
    loss = tf.reduce_mean(xentropy, name="loss")

with tf.name_scope("train"):
    optimizer = tf.train.GradientDescentOptimizer(lr)
    training_op = optimizer.minimize(loss)
    
with tf.name_scope("pred"):
    correct = tf.nn.in_top_k(logits,yvar,1)
    accuracy = tf.reduce_mean(tf.cast(correct, tf.float32))

init=tf.global_variables_initializer()
saver=tf.train.Saver()

Now let's run the algorithm. We will run 1000 epochs, and use mini-batches of size 5. For each step we select 5 random datapoints. We will keep track of the training set accuracy and the testing set accuracy so we can plot a graph of how accuracy changes with each epoch.

n_epochs=1000
batch_size=5

tr_acc=[]
te_acc=[]
with tf.Session() as sess:
    init.run()
    for epoch in range(n_epochs):
        for iteration in range(30):
            ind = np.random.permutation(len(X_train))[:batch_size]
            X_batch = X_train[ind] 
            y_batch = y_train[ind] 
            sess.run(training_op, feed_dict={Xvar:X_batch, yvar : y_batch}) 
            acc_train = accuracy.eval(feed_dict={Xvar: X_batch, yvar: y_batch})
            acc_test = accuracy.eval(feed_dict={Xvar: X_test, yvar: y_test}) 
            tr_acc.append(acc_train)
            te_acc.append(acc_test)
            print(epoch, "training acc: ",acc_train, "Test acc: ",acc_test)
            save_path = saver.save(sess,"./irismodel.ckpt")

We can see that after 400 epochs of training, on the admittedly tiny datset, we reach a very high auccracy on both the training set and the test set. This example is a very simple introduction into using Tensorflow, and it really does under-utilize Tensorflow's power, which is more effective for deep neural networks, with recurrent layers, convolutional layers etc, and with GPU integration. We can now look at the accuracy plot. We will use Matplotlib, to plot two line graphs, one for training accuracy, the other for test set accuracy.

import matplotlib.pyplot as plt
r = range(334)
plt.plot(r,tr_acc,r,te_acc)
plt.ylabel('accuracy on training set and test set')
plt.xlabel("epochs")
plt.show()

The graph will look something like this:

That is it for the first part of this introduction. I will add some more simple "use-cases" of Tensorflow later, before moving on to more interesting and powerful uses.