文本分类学习笔记（6）贝叶斯

贝叶斯分类器&＃xff1a;
先验概率P(c)&＃61; 类c下单词总数/整个训练样本的单词总数
类条件概率P(tk|c)&＃61;(类c下单词tk在各个文档中出现过的次数之和&＃43;1)/(类c下单词总数&＃43;|V|)
V是训练样本的单词表&＃xff08;即抽取单词&＃xff0c;单词出现多次&＃xff0c;只算一个&＃xff09;&＃xff0c;|V|则表示训练样本包含多少“个”单词。P(tk|c)可以看作是单词tk在证明d属于类c上提供了多大的证据&＃xff0c;而P(c)则可以认为是类别c在整体上占多大比例(有多大可能性)。

注&＃xff1a;在实际计算过程&＃xff0c;特征维度较&＃xff0c;单个概率较小&＃xff0c;连乘的结果会造成精度丢失&＃xff0c;因此采用对数函数对概率进行放大&＃xff0c;而且不需要计算P(Doc)&＃xff0c;即&＃xff1a;

预测算法是根据上述公式计算argmax{P(Ci|Doc)}

#coding&＃61;utf-8
from scipy import sparse,io
from sklearn.linear_model import LogisticRegression
from sklearn.naive_bayes import MultinomialNB
from sklearn import metrics
from numpy import *
import warnings
warnings.filterwarnings("ignore")class Naivebayes:N &＃61; [] #k * n k &＃61; 0 #categoryn &＃61; 0 #samples dimensionsm &＃61; 0 #training samplescategory &＃61; [] # 类别log_probability_c &＃61; [] # 属于每个类的概率log_probability_t_c&＃61;[] # 拉普拉斯平滑后概率def __init__(self, X, Y):if len(X) !&＃61; len(Y):print &＃39;samples\&＃39; length not equal labels\&＃39; length&＃39;elif len(Y) &＃61;&＃61; 0:print &＃39;samples\&＃39; size is zero&＃39;else:self.m &＃61; len(Y)self.n &＃61; len(X[0])D &＃61; []for i in range(self.m):# 对于新的类别添加一个新集合if Y[i] not in self.category:self.category.append(Y[i])D.append([])D[-1].append(X[i])else:D[self.category.index(Y[i])].append(X[i])# 计算每个类的概率for i in range(len(self.category)):self.log_probability_c.append(len(D[i]))self.log_probability_c &＃61; log(self.log_probability_c)# 赋值k、nself.k &＃61; len(self.category)self.N &＃61; zeros((self.k, self.n))for i in range(self.k):self.N[i] &＃61; array(D[i]).sum(0)# &＃43;1平滑self.log_probability_t_c &＃61; self.N &＃43; 1s &＃61; self.log_probability_t_c.sum(1)self.log_probability_t_c &＃61; log(self.log_probability_t_c / s.reshape(len(s), 1))#分类def predict(self, x):p &＃61; self.log_probability_c &＃43; x.dot(self.log_probability_t_c.transpose())i &＃61; p.argmax(1)label &＃61; []for j in range(len(i)):label.append(self.category[i[j]])return labelif __name__ &＃61;&＃61; &＃39;__main__&＃39;:#读取中间数据data &＃61; io.loadmat(&＃39;SetMat1.mat&＃39;)vectormat &＃61; data[&＃39;trainSet&＃39;]labeled_names &＃61; data[&＃39;train_labeled&＃39;][0]labeled_names1 &＃61; data[&＃39;test_labeled&＃39;][0]vectormat1 &＃61; data[&＃39;testSet&＃39;]nb &＃61; Naivebayes(vectormat, labeled_names)labels &＃61; nb.predict(vectormat1)calculate_result(labeled_names1,labels)#print labelsc &＃61; zeros((10,10), dtype&＃61;int)for i in range(len(labels)):c[labeled_names1[i]-1][labels[i]-1] &＃61; c[labeled_names1[i]-1][labels[i]-1] &＃43; 1print c

运行结果
中间数据采用的是TF\IDF值&＃xff0c;依据词频做了简单特征筛选

predict info:
accuracy:0.779
precision:0.759
recall:0.779
f1-score:0.735

使用Bool型特征&＃xff08;one-hot&＃xff09;则有明显提高
count_vec &＃61; CountVectorizer(binary &＃61; True,decode_error&＃61;’replace’)

predict info:
accuracy:0.849
precision:0.835
recall:0.849
f1-score:0.824
[[ 705 0 3 5 0 0 2 0 4 0][ 0 1 0 3 51 0 0 0 1 0][ 13 0 165 0 0 0 0 3 8 0][ 30 0 3 1051 0 0 1 0 2 0][ 0 1 0 5 126 0 1 4 12 0][ 0 0 0 1 1 40 86 0 3 0][ 0 0 1 1 0 8 166 0 3 0][ 3 0 49 1 11 0 0 16 9 0][ 0 0 1 7 3 0 11 0 95 0][ 0 1 0 3 64 0 0 1 2 0]]