Spark中组件Mllib的学习32之朴素贝叶斯分类器（伯努利朴素贝叶斯）*

更多代码请见&＃xff1a;https://github.com/xubo245/SparkLearning
Spark中组件Mllib的学习之分类篇
1解释
&＃xff08;1&＃xff09; 朴素贝叶斯分类器种类

在把训练集中的每个文档向量化的过程中&＃xff0c;存在两个模型。一个是统计词在文档中出现的次数&＃xff08;多项式模型&＃xff09;;一个是统计词是否在文档中出现过&＃xff08;柏努利模型&＃xff09;目前mllib只支持多项式朴素贝叶斯和伯努利贝叶斯&＃xff08;spark-1.5.2&＃xff09;,不支持高斯朴素贝叶斯。 根据&＃xff1a;


/*** Trains a Naive Bayes model given an RDD of &＃96;(label, features)&＃96; pairs.** This is the Multinomial NB ([[http://tinyurl.com/lsdw6p]]) which can handle all kinds of* discrete data. For example, by converting documents into TF-IDF vectors, it can be used for* document classification. By making every vector a 0-1 vector, it can also be used as* Bernoulli NB ([[http://tinyurl.com/p7c96j6]]). The input feature values must be nonnegative.*/
&＃64;Since("0.9.0")
class NaiveBayes private (private var lambda: Double,private var modelType: String) extends Serializable with Logging {import NaiveBayes.{Bernoulli, Multinomial}&＃64;Since("1.4.0")def this(lambda: Double) &＃61; this(lambda, NaiveBayes.Multinomial)


三种朴素贝叶斯分类器都在【4】中有提到

&＃xff08;2&＃xff09;伯努利贝叶斯分类器

参考【5】

2.代码&＃xff1a;

/*** &＃64;author xubo* ref:Spark MlLib机器学习实战* more code:https://github.com/xubo245/SparkLearning* more blog:http://blog.csdn.net/xubo245*/
package org.apache.spark.mllib.learning.classificationimport java.text.SimpleDateFormat
import java.util.Dateimport org.apache.spark.mllib.classification.NaiveBayes._
import org.apache.spark.mllib.classification.{NaiveBayes, NaiveBayesModel}
import org.apache.spark.mllib.linalg.Vectors
import org.apache.spark.mllib.regression.LabeledPoint
import org.apache.spark.{SparkException, SparkConf, SparkContext}/*** Created by xubo on 2016/5/23.* From:NaiveBayesSuite.scala in spark 1.5.2 sources* another examples:NaiveBayesSuite test("Naive Bayes Bernoulli")*/
object BernoulliNaiveBayesLearning {def main(args: Array[String]) {val conf &＃61; new SparkConf().setMaster("local[4]").setAppName(this.getClass().getSimpleName().filter(!_.equals(&＃39;$&＃39;)))val sc &＃61; new SparkContext(conf)val badTrain &＃61; Seq(LabeledPoint(1.0, Vectors.dense(1.0)),// LabeledPoint(0.0, Vectors.dense(2.0)),LabeledPoint(1.0, Vectors.dense(1.0)),LabeledPoint(1.0, Vectors.dense(0.0)))val model1 &＃61; NaiveBayes.train(sc.makeRDD(badTrain, 2), 1.0, Bernoulli)println("model1:")println(model1)sc.makeRDD(badTrain, 2).foreach(println)val okTrain &＃61; Seq(LabeledPoint(1.0, Vectors.dense(1.0)),LabeledPoint(0.0, Vectors.dense(0.0)),LabeledPoint(1.0, Vectors.dense(1.0)),LabeledPoint(1.0, Vectors.dense(1.0)),LabeledPoint(0.0, Vectors.dense(0.0)),LabeledPoint(1.0, Vectors.dense(1.0)),LabeledPoint(1.0, Vectors.dense(1.0)))val badPredict &＃61; Seq(Vectors.dense(1.0),// Vectors.dense(2.0),Vectors.dense(1.0),Vectors.dense(0.0))val model &＃61; NaiveBayes.train(sc.makeRDD(okTrain, 2), 1.0, Bernoulli)// intercept[SparkException] {val pre2 &＃61; model.predict(sc.makeRDD(badPredict, 2)).collect()// }println("model2:")sc.makeRDD(okTrain, 2).foreach(println)println("predict data:")sc.makeRDD(badPredict, 2).foreach(println)println(model)println("predict result:")pre2.foreach(println)sc.stop}
}


3.结果&＃xff1a;

model1:
org.apache.spark.mllib.classification.NaiveBayesModel&＃64;79d63340
(1.0,[1.0])
(1.0,[1.0])
(1.0,[0.0])
model2:
(1.0,[1.0])
(0.0,[0.0])
(1.0,[1.0])
(1.0,[1.0])
(0.0,[0.0])
(1.0,[1.0])
(1.0,[1.0])
predict data:
[1.0]
[0.0]
[1.0]
org.apache.spark.mllib.classification.NaiveBayesModel&＃64;3eda0bed
predict result:
1.0
1.0
0.0

参考
【1】http://spark.apache.org/docs/1.5.2/mllib-guide.html
【2】http://spark.apache.org/docs/1.5.2/programming-guide.html
【3】https://github.com/xubo245/SparkLearning
【4】http://www.letiantian.me/2014-10-12-three-models-of-naive-nayes/
【5】http://blog.csdn.net/xlinsist/article/details/51264829