We will specifically build a Convolutional Neural Network (CNN) in this tutorial. CNN is a deep learning algorithm used for image data.<\/p>\n
System and Hardware<\/h2>\n
This tutorial assumes you work on a computer with Ubuntu 16.04 operating system. If you are a Windows user, please refer to this guide<\/a> for installing Ubuntu alongside with Windows. For building a deep learning application, we suggest you to use a computer equipped with a CUDA GPU<\/a>. You are able to run a deep learning application without GPU, but it would be very slow.<\/p>\n Here are the outline of this tutorial: In order to run the codes in this tutorial, we need to make sure that these plugins have been installed:<\/p>\n When you install Ubuntu 16.04, Python 2.7 will also be automatically installed. You can check the installation by running this command in your Ubuntu terminal:<\/p>\n <\/p>\n If Python 2.7 has been installed, you will see a view similar to this image: Type exit()<\/span> to return.<\/p>\n In case that it is not installed, you can run these commands in your terminal:<\/p>\n <\/p>\n To install numpy, run this command in terminal:<\/p>\n <\/p>\n Next, we will install Tensorflow, a deep learning framework developed by Google. Tensorflow comes with two version depending on whether you want to use GPU to accelerate Tensorflow or not. If your computer is equipped with GPU, run this command to install Tensorflow with GPU support:<\/p>\n <\/p>\n Else, run this command:<\/p>\n <\/p>\n Finally, we will install Keras, a high-level deep learning framework built on top of Tensorflow. To install it, run this command:<\/p>\n <\/p>\n Starting from this step, we will start to build the deep learning application using Python. To run the application, you can create a new file named as mnist.py, save it to your home directory, add all the given codes below, and execute the following command in your terminal:<\/p>\n <\/p>\n To build our deep learning application, we need to import Numpy and Keras library to our python codes. You can import the necessary libraries by adding the following codes to mnist.py file:<\/p>\n <\/p>\n In this tutorial, we will use MNIST dataset. MNIST is a publicly available dataset of handwritten digits image. You can see the details of the dataset here<\/a>. Below are samples of image in MNIST dataset:<\/p>\n In this tutorial, we will use a pre-loaded MNIST dataset in Keras. To load MNIST, add these following codes to mnist.py file:<\/p>\n <\/p>\n The loaded data consists of 4 parts: X_train, y_train, X_test, and y_test. X_train contains image data to be used for training CNN model in the form of matrix. y_train contains labels of the image (0 to 9). X_test and y_test are the image data and labels to be used for testing our CNN model performance.<\/p>\n To use the data in our deep learning application, we need to have some adjustments to the data. We will have 3 adjustments: (1) add 1 more dimension to the image data, (2) normalize the data to value range between -1 to 1, and (3) convert the labels to one-hot vector format. We need to add 1 more dimension the image data because CNN model usually deals with RGB image, which shape defined as (width x height x channel) in its matrix format. The shape of pre-loaded MNIST dataset in Keras is only defined as (width x height), so we need to add 1 more dimension as channel. The following codes will reshape the image data to\u00a0 (width x height x channel):<\/p>\n <\/p>\n We also need to normalize the data to value between -1 to 1, as deep learning model usually easier to learn from this input value range. You can normalze the data by adding these following codes:<\/p>\n <\/p>\n Finally, you need to convert the format of labels to one-hot vector format. One-hot vector is a vector which value consists only 0 and 1. To convert the labels, add these following codes to mnist.py file:<\/p>\n <\/p>\n Now, we can start to design our CNN architecture that learns from the loaded MNIST data. The CNN architecture we use in this tutorial is similar to the visualization below:<\/p>\n This architecture is inspired from LeNet [1], the original CNN model published in 1989. We use 2 set of Convolutional – Max Pooling layers in the model. Each of the Convolutional layer uses 3×3 kernel and 1 stride. The Max Pooling layers each uses 2×2 pool size. On top of these two layer set, we will add\u00a0 a fully connected (dense) layer with 128 neurons. The output layer consists of 10 neurons, which each neuron output a probability of the data belongs to class 0 to 9. You can learn more about the detailed operations of each layer in here<\/a>. To use this design into our application, you can add the following codes in our mnist.py file:<\/p>\n <\/p>\n After adding the CNN architecture, we can now compile it to form a CNN model and train the model to do image classification. To compile the architecture, add the following code:<\/p>\n <\/p>\n In this tutorial, we use Adam optimizer [2] and categorical cross-entropy loss. Keras has different optimizer and loss options that can be chosen. You can learn more about Keras optimizers and losses here<\/a> and here<\/a>.<\/p>\n Finally, we can train our CNN model to do image classification. Add this code to train it:<\/p>\n <\/p>\n In this tutorial, we set the maximum epochs of our CNN model to 10 and batch size to 32. You can learn more about epoch and batch size here<\/a> and here<\/a>. When you run mnist.py later, you will see a view similar to the below image when the above code is executed. The acc number in the image means that your CNN model able to classify images in the training set with 99.21%\u00a0 accuracy.<\/p>\n After we have trained the CNN model, we can test its performance using a test dataset. Add the following code to print the accuracy of the model in test dataset:<\/p>\n <\/p>\n If you follow step 2 to step 6, your mnist.py file should look like this:<\/p>\n <\/p>\n To run your mnist.py file, execute this command in your terminal:<\/p>\n <\/p>\n You should see a similar view to the following image after you run your application:<\/p>\nTutorial Outline<\/h2>\n
\n<\/strong><\/p>\n\n
Step 1: Environment Installation<\/h2>\n
\n
$ python<\/code><\/pre>\n<\/div>\n
\n![\"\"](\"https:\/\/research.binus.ac.id\/airnd\/wp-content\/uploads\/sites\/12\/2018\/03\/Screenshot-from-2018-03-13-10-28-39-640x98.png\")
<\/p>\n$ sudo apt-get update\r\n$ sudo apt-get install python-pip python-dev\r\n<\/code><\/pre>\n<\/div>\n$ pip install numpy<\/code><\/pre>\n<\/div>\n$ pip install tensorflow-gpu<\/code><\/pre>\n<\/div>\n$ pip install tensorflow<\/code><\/pre>\n<\/div>\n$ pip install keras<\/code><\/pre>\n<\/div>\nStep 2: Import Numpy and Keras Library<\/h2>\n
$ python mnist.py<\/code><\/pre>\n<\/div>\n#Step 2: Import Numpy and Keras \r\nimport numpy as np\r\n\r\nimport keras\r\nfrom keras.models import Model\r\nfrom keras.layers import Input\r\nfrom keras.layers import Dense\r\nfrom keras.layers import Conv2D\r\nfrom keras.layers import MaxPooling2D\r\nfrom keras.layers import Flatten\r\nfrom keras.layers import Dropout\r\n<\/code><\/pre>\n<\/div>\nStep 3: Load and Preprocess MNIST Data<\/h2>\n
![\"\"](\"https:\/\/research.binus.ac.id\/airdc\/wp-content\/uploads\/sites\/18\/2018\/03\/mnist.jpg\")
<\/p>\n#Step 3: Load and Preprocess MNIST data\r\nfrom keras.datasets import mnist\r\n(X_train, y_train), (X_test, y_test) = mnist.load_data()\r\n<\/code><\/pre>\n<\/div>\nX_train = X_train.reshape(X_train.shape[0], 28, 28, 1)\r\nX_test = X_test.reshape(X_test.shape[0], 28, 28, 1)\r\n<\/code><\/pre>\n<\/div>\nX_train = (X_train\/255) - 0.5\r\nX_test = (X_test\/255) - 0.5\r\n<\/code><\/pre>\n<\/div>\nnum_classes = 10\r\ny_train = keras.utils.to_categorical(y_train, num_classes)\r\ny_test = keras.utils.to_categorical(y_test, num_classes)\r\n<\/code><\/pre>\n<\/div>\nStep 4: Design CNN Architecture<\/h2>\n
![\"\"](\"https:\/\/research.binus.ac.id\/airnd\/wp-content\/uploads\/sites\/12\/2018\/03\/ConvNet-1-640x288.png\")
<\/p>\n#Step 4: Design CNN Architecture\r\ninp = Input(shape=(28,28,1))\r\ni = Conv2D(32, kernel_size=(3,3), activation='relu', name='conv1')(inp)\r\ni = MaxPooling2D(pool_size=(2,2))(i)\r\ni = Conv2D(32, kernel_size=(3,3), activation='relu', name='conv2')(i) \r\ni = MaxPooling2D(pool_size=(2,2))(i)\r\ni = Flatten()(i)\r\ni = Dense(128, activation='relu', name='dense_1')(i) \r\npredictions = Dense(num_classes, activation='softmax', name='dense_last')(i) \r\n\r\nmodel = Model(inputs=inp, outputs=predictions)\r\n<\/code><\/pre>\n<\/div>\nStep 5: Compile and Train CNN Model<\/h2>\n
#Step 5: Compile and Train The Model\r\nmodel.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\r\n<\/code><\/pre>\n<\/div>\nhistory = model.fit(X_train, y_train, epochs=10, batch_size=32)\r\n<\/code><\/pre>\n<\/div>\n![\"\"](\"https:\/\/research.binus.ac.id\/airnd\/wp-content\/uploads\/sites\/12\/2018\/03\/Screenshot-from-2018-03-14-14-26-10-640x98.png\")
<\/p>\nStep 6: Print Result<\/h2>\n
#Step 6: Print Result\r\nscores = model.evaluate(X_test, y_test, verbose=0)\r\nprint(\"Test Accuracy: %.2f%%\" % (scores[1]*100))\r\n<\/code><\/pre>\n<\/div>\nRun The Application<\/h2>\n
#Step 2: Import Numpy and Keras \r\nimport numpy as np\r\n\r\nimport keras\r\nfrom keras.models import Model\r\nfrom keras.layers import Input\r\nfrom keras.layers import Dense\r\nfrom keras.layers import Conv2D\r\nfrom keras.layers import MaxPooling2D\r\nfrom keras.layers import Flatten\r\nfrom keras.layers import Dropout\r\n\r\n#Step 3: Load and Preprocess MNIST data\r\nfrom keras.datasets import mnist\r\n(X_train, y_train), (X_test, y_test) = mnist.load_data()\r\n\r\nX_train = X_train.reshape(X_train.shape[0], 28, 28, 1)\r\nX_test = X_test.reshape(X_test.shape[0], 28, 28, 1)\r\n\r\nX_train = (X_train\/255) - 0.5\r\nX_test = (X_test\/255) - 0.5\r\n\r\nnum_classes = 10\r\ny_train = keras.utils.to_categorical(y_train, num_classes)\r\ny_test = keras.utils.to_categorical(y_test, num_classes)\r\n\r\n#Step 4: Design CNN Architecture\r\ninp = Input(shape=(28,28,1))\r\ni = Conv2D(32, kernel_size=(3,3), activation='relu', name='conv1')(inp)\r\ni = MaxPooling2D(pool_size=(2,2))(i)\r\ni = Conv2D(32, kernel_size=(3,3), activation='relu', name='conv2')(i) \r\ni = MaxPooling2D(pool_size=(2,2))(i)\r\ni = Flatten()(i)\r\ni = Dense(128, activation='relu', name='dense_1')(i) \r\npredictions = Dense(num_classes, activation='softmax', name='dense_last')(i) \r\n\r\nmodel = Model(inputs=inp, outputs=predictions)\r\n\r\n#Step 5: Compile and Train The Model\r\nmodel.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\r\nhistory = model.fit(X_train, y_train, epochs=10, batch_size=32)\r\n\r\n#Step 6: Print Result\r\nscores = model.evaluate(X_test, y_test, verbose=0)\r\nprint(\"Test Accuracy: %.2f%%\" % (scores[1]*100))\r\n<\/code><\/pre>\n<\/div>\n$ python mnist.py\r\n<\/code><\/pre>\n<\/div>\n![\"\"](\"https:\/\/research.binus.ac.id\/airnd\/wp-content\/uploads\/sites\/12\/2018\/03\/Screenshot-from-2018-03-14-14-55-12-640x319.png\")
<\/p>\n