使用tensorflow进行手写数字分类预测的kaggle实战

2018年9月20日笔记

kaggle网站手写数字分类的比赛链接&＃xff1a;https://www.kaggle.com/c/digit-recognizer
注册账号后才能参加kaggle比赛&＃xff0c;本文作者成绩前2%&＃xff0c;如下图所示&＃xff1a;

image.png

0.尝试提交

本文作者提供一份能够获得较好成绩的文件&＃xff0c;读者可以提交该文件熟悉提交流程。
下载链接: https://pan.baidu.com/s/1QKVMmAnW7Ui1104fhfiljg 提取码: mqex
该作答文件的提交成绩有0.99814&＃xff0c;如果读者想提高成绩到0.99985&＃xff0c;请阅读后面的章节。

1.配置环境

使用卷积神经网络模型要求有较高的机器配置&＃xff0c;如果使用CPU版tensorflow会花费大量时间。
读者在有nvidia显卡的情况下&＃xff0c;安装GPU版tensorflow会提高计算速度50倍。
安装教程链接&＃xff1a;https://blog.csdn.net/qq_36556893/article/details/79433298
如果没有nvidia显卡&＃xff0c;但有visa信用卡&＃xff0c;请阅读我的另一篇文章《在谷歌云服务器上搭建深度学习平台》&＃xff0c;链接&＃xff1a;https://www.jianshu.com/p/893d622d1b5a

2.下载并解压数据集

MNIST数据集下载链接: https://pan.baidu.com/s/1fPbgMqsEvk2WyM9hy5Em6w 密码: wa9p
下载压缩文件MNIST_data.rar完成后&＃xff0c;选择解压到当前文件夹&＃xff0c;不要选择解压到MNIST_data。
文件夹结构如下图所示&＃xff1a;

image.png

3.模型训练并保存

本文作者此段代码是在谷歌云服务器上运行&＃xff0c;谷歌云服务器的GPU显存有16G。
因为个人电脑GPU的显存不足&＃xff0c;读者可能无法运行&＃xff0c;解决办法是减少feed_dict中的样本数量。
理解下面一段代码&＃xff0c;请阅读本文作者的另外一篇文章《基于tensorflow&＃43;CNN的MNIST数据集手写数字分类》&＃xff0c;链接&＃xff1a;https://www.jianshu.com/p/a652f1cb95b4

import warnings
warnings.filterwarnings(&＃39;ignore&＃39;)
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
import random
import numpy as npmnist &＃61; input_data.read_data_sets(&＃39;MNIST_data&＃39;, one_hot&＃61;True)
batch_size &＃61; 300
X_holder &＃61; tf.placeholder(tf.float32)
y_holder &＃61; tf.placeholder(tf.float32)
X &＃61; np.vstack([mnist.train.images, mnist.test.images, mnist.validation.images])
y &＃61; np.vstack([mnist.train.labels, mnist.test.labels, mnist.validation.labels])
print(X.shape, y.shape)X_images &＃61; tf.reshape(X_holder, [-1, 28, 28, 1])
#convolutional layer 1
conv1_Weights &＃61; tf.Variable(tf.truncated_normal([5, 5, 1, 32], stddev&＃61;0.1), name&＃61;&＃39;conv1_Weights&＃39;)
conv1_biases &＃61; tf.Variable(tf.constant(0.1, shape&＃61;[32]), name&＃61;&＃39;conv1_biases&＃39;)
conv1_conv2d &＃61; tf.nn.conv2d(X_images, conv1_Weights, strides&＃61;[1, 1, 1, 1], padding&＃61;&＃39;SAME&＃39;) &＃43; conv1_biases
conv1_activated &＃61; tf.nn.relu(conv1_conv2d)
conv1_pooled &＃61; tf.nn.max_pool(conv1_activated, ksize&＃61;[1, 2, 2, 1], strides&＃61;[1, 2, 2, 1], padding&＃61;&＃39;SAME&＃39;)
#convolutional layer 2
conv2_Weights &＃61; tf.Variable(tf.truncated_normal([5, 5, 32, 64], stddev&＃61;0.1), name&＃61;&＃39;conv2_Weights&＃39;)
conv2_biases &＃61; tf.Variable(tf.constant(0.1, shape&＃61;[64]), name&＃61;&＃39;conv2_biases&＃39;)
conv2_conv2d &＃61; tf.nn.conv2d(conv1_pooled, conv2_Weights, strides&＃61;[1, 1, 1, 1], padding&＃61;&＃39;SAME&＃39;) &＃43; conv2_biases
conv2_activated &＃61; tf.nn.relu(conv2_conv2d)
conv2_pooled &＃61; tf.nn.max_pool(conv2_activated, ksize&＃61;[1, 2, 2, 1], strides&＃61;[1, 2, 2, 1], padding&＃61;&＃39;SAME&＃39;)
#full connected layer 1
connect1_flat &＃61; tf.reshape(conv2_pooled, [-1, 7 * 7 * 64])
connect1_Weights &＃61; tf.Variable(tf.truncated_normal([7 * 7 * 64, 1024], stddev&＃61;0.1), name&＃61;&＃39;connect1_Weights&＃39;)
connect1_biases &＃61; tf.Variable(tf.constant(0.1, shape&＃61;[1024]), name&＃61;&＃39;connect1_biases&＃39;)
connect1_Wx_plus_b &＃61; tf.add(tf.matmul(connect1_flat, connect1_Weights), connect1_biases)
connect1_activated &＃61; tf.nn.relu(connect1_Wx_plus_b)
#full connected layer 2
connect2_Weights &＃61; tf.Variable(tf.truncated_normal([1024, 10], stddev&＃61;0.1), name&＃61;&＃39;connect2_Weights&＃39;)
connect2_biases &＃61; tf.Variable(tf.constant(0.1, shape&＃61;[10]), name&＃61;&＃39;connect2_biases&＃39;)
connect2_Wx_plus_b &＃61; tf.add(tf.matmul(connect1_activated, connect2_Weights), connect2_biases)
predict_y &＃61; tf.nn.softmax(connect2_Wx_plus_b)
#loss and train
loss &＃61; tf.reduce_mean(-tf.reduce_sum(y_holder * tf.log(predict_y), 1))
optimizer &＃61; tf.train.AdamOptimizer(0.0001)
train &＃61; optimizer.minimize(loss)init &＃61; tf.global_variables_initializer()
session &＃61; tf.Session()
session.run(init)
saver &＃61; tf.train.Saver()for i in range(20000):selected_index &＃61; random.sample(range(len(y)), k&＃61;batch_size)selected_X &＃61; X[selected_index]selected_y &＃61; y[selected_index]session.run(train, feed_dict&＃61;{X_holder:selected_X, y_holder:selected_y})if i % 100 &＃61;&＃61; 0:correct_prediction &＃61; tf.equal(tf.argmax(predict_y, 1), tf.argmax(y_holder, 1))accuracy &＃61; tf.reduce_mean(tf.cast(correct_prediction, tf.float32))train_accuracy &＃61; session.run(accuracy, feed_dict&＃61;{X_holder:mnist.train.images, y_holder:mnist.train.labels})test_accuracy &＃61; session.run(accuracy, feed_dict&＃61;{X_holder:mnist.test.images, y_holder:mnist.test.labels})validation_accuracy &＃61; session.run(accuracy, feed_dict&＃61;{X_holder:mnist.validation.images, y_holder:mnist.validation.labels})print(&＃39;step:%d train accuracy:%.4f test accuracy:%.4f validation accuracy:%.4f&＃39; %(i, train_accuracy, test_accuracy, validation_accuracy))if train_accuracy &＃61;&＃61; 1 and test_accuracy &＃61;&＃61; 1 and validation_accuracy &＃61;&＃61; 1:save_path &＃61; saver.save(session, &＃39;mnist_cnn_model/mnist_cnn.ckpt&＃39;)print(&＃39;Save to path:&＃39;, save_path)


4.加载模型

本文作者提供获得最佳成绩0.99985的模型&＃xff0c;读者可以加载该模型&＃xff0c;并用此模型预测并提交成绩。
模型下载链接: https://pan.baidu.com/s/1zVLHdGiZflspV9jPWn_ECA 提取码: nktv
如果读者有服务器&＃xff0c;可以尝试获取保存的模型&＃xff0c;下载按钮如下图所示&＃xff1a;

image.png

import warnings
warnings.filterwarnings(&＃39;ignore&＃39;)
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_datamnist &＃61; input_data.read_data_sets(&＃39;MNIST_data&＃39;, one_hot&＃61;True)
batch_size &＃61; 100
X_holder &＃61; tf.placeholder(tf.float32)
y_holder &＃61; tf.placeholder(tf.float32)X_images &＃61; tf.reshape(X_holder, [-1, 28, 28, 1])
#convolutional layer 1
conv1_Weights &＃61; tf.Variable(tf.truncated_normal([5, 5, 1, 32], stddev&＃61;0.1), name&＃61;&＃39;conv1_Weights&＃39;)
conv1_biases &＃61; tf.Variable(tf.constant(0.1, shape&＃61;[32]), name&＃61;&＃39;conv1_biases&＃39;)
conv1_conv2d &＃61; tf.nn.conv2d(X_images, conv1_Weights, strides&＃61;[1, 1, 1, 1], padding&＃61;&＃39;SAME&＃39;) &＃43; conv1_biases
conv1_activated &＃61; tf.nn.relu(conv1_conv2d)
conv1_pooled &＃61; tf.nn.max_pool(conv1_activated, ksize&＃61;[1, 2, 2, 1], strides&＃61;[1, 2, 2, 1], padding&＃61;&＃39;SAME&＃39;)
#convolutional layer 2
conv2_Weights &＃61; tf.Variable(tf.truncated_normal([5, 5, 32, 64], stddev&＃61;0.1), name&＃61;&＃39;conv2_Weights&＃39;)
conv2_biases &＃61; tf.Variable(tf.constant(0.1, shape&＃61;[64]), name&＃61;&＃39;conv2_biases&＃39;)
conv2_conv2d &＃61; tf.nn.conv2d(conv1_pooled, conv2_Weights, strides&＃61;[1, 1, 1, 1], padding&＃61;&＃39;SAME&＃39;) &＃43; conv2_biases
conv2_activated &＃61; tf.nn.relu(conv2_conv2d)
conv2_pooled &＃61; tf.nn.max_pool(conv2_activated, ksize&＃61;[1, 2, 2, 1], strides&＃61;[1, 2, 2, 1], padding&＃61;&＃39;SAME&＃39;)
#full connected layer 1
connect1_flat &＃61; tf.reshape(conv2_pooled, [-1, 7 * 7 * 64])
connect1_Weights &＃61; tf.Variable(tf.truncated_normal([7 * 7 * 64, 1024], stddev&＃61;0.1), name&＃61;&＃39;connect1_Weights&＃39;)
connect1_biases &＃61; tf.Variable(tf.constant(0.1, shape&＃61;[1024]), name&＃61;&＃39;connect1_biases&＃39;)
connect1_Wx_plus_b &＃61; tf.add(tf.matmul(connect1_flat, connect1_Weights), connect1_biases)
connect1_activated &＃61; tf.nn.relu(connect1_Wx_plus_b)
#full connected layer 2
connect2_Weights &＃61; tf.Variable(tf.truncated_normal([1024, 10], stddev&＃61;0.1), name&＃61;&＃39;connect2_Weights&＃39;)
connect2_biases &＃61; tf.Variable(tf.constant(0.1, shape&＃61;[10]), name&＃61;&＃39;connect2_biases&＃39;)
connect2_Wx_plus_b &＃61; tf.add(tf.matmul(connect1_activated, connect2_Weights), connect2_biases)
predict_y &＃61; tf.nn.softmax(connect2_Wx_plus_b)
#loss and train
loss &＃61; tf.reduce_mean(-tf.reduce_sum(y_holder * tf.log(predict_y), 1))
optimizer &＃61; tf.train.AdamOptimizer(0.0001)
train &＃61; optimizer.minimize(loss)session &＃61; tf.Session()
saver &＃61; tf.train.Saver()
saver.restore(session, &＃39;mnist_cnn_model/mnist_cnn.ckpt&＃39;)
correct_prediction &＃61; tf.equal(tf.argmax(predict_y, 1), tf.argmax(y_holder, 1))
accuracy &＃61; tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print(&＃39;load model successful&＃39;)
train_images, train_labels &＃61; mnist.train.next_batch(5000)
test_images, test_labels &＃61; mnist.test.next_batch(5000)
train_accuracy &＃61; session.run(accuracy, feed_dict&＃61;{X_holder:train_images, y_holder:train_labels})
test_accuracy &＃61; session.run(accuracy, feed_dict&＃61;{X_holder:test_images, y_holder:test_labels})
print(&＃39;train accuracy:%.4f test accuracy:%.4f&＃39; %(train_accuracy, test_accuracy))


上面一段代码的运行结果如下&＃xff1a;

Extracting MNIST_data/train-images-idx3-ubyte.gz
Extracting MNIST_data/train-labels-idx1-ubyte.gz
Extracting MNIST_data/t10k-images-idx3-ubyte.gz
Extracting MNIST_data/t10k-labels-idx1-ubyte.gz
INFO:tensorflow:Restoring parameters from mnist_cnn_model/mnist_cnn.ckpt
load model successful
train accuracy:1.0000 test accuracy:1.0000

5.模型预测

此第5张能够成功运行的前提是已经成功运行第4章的代码&＃xff0c;即加载模型成功。
将测试样本分成6份&＃xff0c;可以解决因为显存不足无法运行的问题。

import pandas as pdtest_df &＃61; pd.read_csv(&＃39;test.csv&＃39;)
X &＃61; test_df.values
print(&＃39;特征矩阵的形状&＃xff1a;&＃39;, X.shape)
X1 &＃61; X[:5000]
X2 &＃61; X[5000:10000]
X3 &＃61; X[10000:15000]
X4 &＃61; X[15000:20000]
X5 &＃61; X[20000:25000]
X6 &＃61; X[25000:]
y1 &＃61; session.run(predict_y, feed_dict&＃61;{X_holder:X1})
y2 &＃61; session.run(predict_y, feed_dict&＃61;{X_holder:X2})
y3 &＃61; session.run(predict_y, feed_dict&＃61;{X_holder:X3})
y4 &＃61; session.run(predict_y, feed_dict&＃61;{X_holder:X4})
y5 &＃61; session.run(predict_y, feed_dict&＃61;{X_holder:X5})
y6 &＃61; session.run(predict_y, feed_dict&＃61;{X_holder:X6})import numpy as np
y &＃61; np.vstack([y1, y2, y3, y4, y5, y6])
y_argmax &＃61; np.argmax(y, 1)
y_argmax.shape
print(&＃39;预测值的形状&＃xff1a;&＃39;, y_argmax.shape)
commit_df &＃61; pd.DataFrame({&＃39;ImageId&＃39;: range(1, 1&＃43;len(y_argmax)),&＃39;Label&＃39;: y_argmax})
fileName &＃61; &＃39;kaggle_commit3.csv&＃39;
commit_df.to_csv(fileName, index&＃61;False)
print(&＃39;预测结果已经保存到文件&＃39;, fileName)


上面一段代码的运行结果如下&＃xff1a;

特征矩阵的形状&＃xff1a; (28000, 784)
预测值的形状&＃xff1a; (28000,)
预测结果已经保存到文件 kaggle_commit3.csv

6.提交作答文件

比赛链接&＃xff1a;https://www.kaggle.com/c/digit-recognizer
点击下面的按钮提交作答文件。

image.png

image.png

7.总结

1.自己电脑配置不足&＃xff0c;使用云服务器极大的加快了工程部署和模型训练速度&＃xff1b;
2.在kaggle经典入门赛取得前2%的成绩&＃xff0c;把简单的事做到极致&＃xff1b;
3.本文作者提供可以加载的模型只能取得0.99571的成绩。