ICode9
精准搜索请尝试： 精确搜索
首页
编程语言>
数据库
系统相关
互联网
其他分享
Python
Java
JavaScript
android
PHP
首页 > 编程语言> 文章详细

Python+Spark2.0+hadoop学习笔记——Python Spark MLlib决策树二分类

2020-04-02 22:02:27  阅读：234  来源： 互联网

标签：1.0 Python MLlib feature hadoop Predict 0.0 15 Else


 决策树是在数据分类问题中的一种常用且经典的机器学习方法，在本例里使用决策树模型来分析StumbleUpon数据集，来预测网页是暂时的(ephemeral)还是长青的(evergreen)，并且调教参数找出最佳的参数组合，来提高预测准确度。

像这类经典的二分类问题，在python中调包来做的话会非常容易，且分析手段也有很多。但是现在的练习任务是使用Spark来对着类问题进行处理，因此，下面将开始介绍使用Spark进行二分类问题的过程。

第一步：分析数据的特性

我们在本例中要使用的数据来自于Kaggle官网的数据，这份数据的维度很大，有些数据的列是没有意义的，在选择时候要规避。数据虽然说维度很大，但是很多列都涉及到了一些文本的内容，因此在这个问题中不考虑维度灾难，进行初步的筛选之后就进行建模了。

第二步：启动Ipython Notebook

cd ~/pythonwork/ipynotebook

PYSPARK_DRIVER_PYTHON=ipython PYSPARK_DRIVER_PYTHON_OPTS=“notebook" MASTER=local[*] pyspark

第三步：进行数据准备

原始数据需要变换成训练模型所需的数据格式LabeledPoint，并以随机方式按照8:1:1比例把数据分割为3个部分trainData、validationData和testData。

from pyspark.mllib.regression import LabeledPoint

def SetPath(sc):
global Path
if sc.master[0:5]=="local" :
Path="file:/home/jorlinlee/pythonsparkexample/PythonProject/"
else:
Path="hdfs://master:9000/user/jorlinlee/"

def get_mapping(rdd, idx):
return rdd.map(lambda fields: fields[idx]).distinct().zipWithIndex().collectAsMap()

def extract_label(record):
label=(record[-1])
return float(label)

def extract_features(field,categoriesMap,featureEnd):
categoryIdx = categoriesMap[field[3]]
categoryFeatures = np.zeros(len(categoriesMap))
categoryFeatures[categoryIdx] = 1
numericalFeatures=[convert_float(field) for field in field[4: featureEnd]]
return np.concatenate(( categoryFeatures, numericalFeatures))

def convert_float(x):
return (0 if x=="?" else float(x))

def PrepareData(sc):
print("Data loading...")
rawDataWithHeader = sc.textFile(Path+"data/train.tsv")
header = rawDataWithHeader.first()
rawData = rawDataWithHeader.filter(lambda x:x !=header)
rData=rawData.map(lambda x: x.replace("\"", ""))
lines = rData.map(lambda x: x.split("\t"))
print("The number of data" + str(lines.count()))
categoriesMap = lines.map(lambda fields: fields[3]).distinct().zipWithIndex().collectAsMap()
labelpointRDD = lines.map( lambda r:LabeledPoint(
extract_label(r),
extract_features(r,categoriesMap,len(r) - 1)))
(trainData, validationData, testData) = labelpointRDD.randomSplit([8, 1, 1])
print("The number of trainData:" + str(trainData.count()) +
" The number of validationData:" + str(validationData.count()) +
" The number of testData:" + str(testData.count()))
return (trainData, validationData, testData, categoriesMap)

第四步：对模型进行训练

def PredictData(sc,model,categoriesMap):
print("Data loading.....")
rawDataWithHeader = sc.textFile(Path+"data/test.tsv")
header = rawDataWithHeader.first()
rawData = rawDataWithHeader.filter(lambda x:x !=header)
rData=rawData.map(lambda x: x.replace("\"", ""))
lines = rData.map(lambda x: x.split("\t"))
print("The number of data：" + str(lines.count()) )
dataRDD = lines.map(lambda r: ( r[0] ,
extract_features(r,categoriesMap,len(r) )))
DescDict = {
0: "ephemeral",
1: "evergreen"
}
for data in dataRDD.take(10):
predictResult = model.predict(data[1])
print("The web: " +str(data[0])+"\n" +"Predict:"+ str(predictResult)+"Illustration"+DescDict[predictResult] +"\n")

第五步：评估模型的准确率（使用AUC）

def evaluateModel(model, validationData):
score = model.predict(validationData.map(lambda p: p.features))
scoreAndLabels=score.zip(validationData.map(lambda p: p.label))
metrics = BinaryClassificationMetrics(scoreAndLabels)
AUC=metrics.areaUnderROC
return(AUC)

第六步：评估并确定模型的参数（需要确定impurityParm、maxDepthParm和maxBinsParm）

def trainEvaluateModel(trainData,validationData,
impurityParm, maxDepthParm, maxBinsParm):
startTime = time()
model = DecisionTree.trainClassifier(trainData,
numClasses=2, categoricalFeaturesInfo={},
impurity=impurityParm,
maxDepth=maxDepthParm,
maxBins=maxBinsParm)
AUC = evaluateModel(model, validationData)
duration = time() - startTime
print ("Parameter" + \
" impurity="+str(impurityParm) +\
" maxDepth="+str(maxDepthParm) + \
" maxBins="+str(maxBinsParm) +\
" Time="+str(duration) + \
" AUC = " + str(AUC) )
return (AUC,duration, impurityParm, maxDepthParm, maxBinsParm,model)

def evalParameter(trainData, validationData, evalparm,
impurityList, maxDepthList, maxBinsList):
metrics = [trainEvaluateModel(trainData, validationData,
impurity,maxDepth, maxBins )
for impurity in impurityList
for maxDepth in maxDepthList
for maxBins in maxBinsList ]
if evalparm=="impurity":
IndexList=impurityList[:]
elif evalparm=="maxDepth":
IndexList=maxDepthList[:]
elif evalparm=="maxBins":
IndexList=maxBinsList[:]
df = pd.DataFrame(metrics,index=IndexList,
columns=['AUC', 'duration','impurity', 'maxDepth', 'maxBins','model'])
showchart(df,evalparm,'AUC','duration',0.5,0.7 )

使用可视化方法来作为参数选择的参考

def showchart(df,evalparm ,barData,lineData,yMin,yMax):
ax = df[barData].plot(kind='bar', title =evalparm,figsize=(10,6),legend=True, fontsize=12)
ax.set_xlabel(evalparm,fontsize=12)
ax.set_ylim([yMin,yMax])
ax.set_ylabel(barData,fontsize=12)
ax2 = ax.twinx()
ax2.plot(df[[lineData ]].values, linestyle='-', marker='o', linewidth=2.0,color='r')
plt.show()

找出准确率最高的参数组合

def evalAllParameter(trainData, validationData,
impurityList, maxDepthList, maxBinsList):
metrics = [trainEvaluateModel(trainData, validationData,
impurity,maxDepth, maxBins )
for impurity in impurityList
for maxDepth in maxDepthList
for maxBins in maxBinsList ]
Smetrics = sorted(metrics, key=lambda k: k[0], reverse=True)
bestParameter=Smetrics[0]
print("The best parameter：impurity:" + str(bestParameter[2]) +
" ,maxDepth:" + str(bestParameter[3]) +
" ,maxBins:" + str(bestParameter[4]) +
" ,AUC = " + str(bestParameter[0]))
return bestParameter[5]

def parametersEval(trainData, validationData):

print("----- impurity ---------")
evalParameter(trainData, validationData,"impurity",
impurityList=["gini", "entropy"],
maxDepthList=[10],
maxBinsList=[10 ])


给定参数的选择范围
print("----- maxDepth ---------")
evalParameter(trainData, validationData,"maxDepth",
impurityList=["gini"],
maxDepthList=[3, 5, 10, 15, 20, 25],
maxBinsList=[10])
print("----- maxBins ---------")
evalParameter(trainData, validationData,"maxBins",
impurityList=["gini"],
maxDepthList =[10],
maxBinsList=[3, 5, 10, 50, 100, 200 ])

第七步：Spark的相关设置

def SetLogger( sc ):
logger = sc._jvm.org.apache.log4j
logger.LogManager.getLogger("org"). setLevel( logger.Level.ERROR )
logger.LogManager.getLogger("akka").setLevel( logger.Level.ERROR )
logger.LogManager.getRootLogger().setLevel(logger.Level.ERROR)

def SetPath(sc):
global Path
if sc.master[0:5]=="local" :
Path="file:/home/jorlinlee/pythonsparkexample/PythonProject/"
else:
Path="hdfs://master:9000/user/jorlinlee/"

def CreateSparkContext():
conf = SparkConf().setMaster("local[*]").setAppName("DTree")
sc = SparkContext(conf = conf)
print ("master="+sc.master)
SetLogger(sc)
SetPath(sc)
return (sc)

sc.stop()

第八步：运行主程序，得到十个预测结果，并且给出分类规则

if __name__ == "__main__":
print("RunDecisionTreeBinary")
sc=CreateSparkContext()
print("==========Preparing===============")
(trainData, validationData, testData, categoriesMap) =PrepareData(sc)
trainData.persist(); validationData.persist(); testData.persist()
print("==========Evaluating===============")
(AUC,duration, impurityParm, maxDepthParm, maxBinsParm,model)= \
trainEvaluateModel(trainData, validationData, "entropy", 10, 200)
if (len(sys.argv) == 2) and (sys.argv[1]=="-e"):
parametersEval(trainData, validationData)
elif (len(sys.argv) == 2) and (sys.argv[1]=="-a"):
print("-----The best parameter---------")
model=evalAllParameter(trainData, validationData,
["gini", "entropy"],
[3, 5, 10, 15, 20, 25],
[3, 5, 10, 50, 100, 200 ])
print("==========Testing===============")
auc = evaluateModel(model, testData)
print("Test best model, AUC:" + str(auc))
print("==========Predicting===============")
PredictData(sc, model, categoriesMap)
print(model.toDebugString())

 

结果：

RunDecisionTreeBinary
master=local[*]
==========Preparing===============
Data loading...
The number of data7395
The number of trainData:5925 The number of validationData:725 The number of testData:745
==========Evaluating===============
Parameter impurity=entropy maxDepth=10 maxBins=200 Time=6.611667633056641 AUC = 0.6579354553492485
==========Testing===============
Test best model, AUC:0.6323763091368725
==========Predicting===============
Data loading.....
The number of data：3171
The web: http://www.lynnskitchenadventures.com/2009/04/homemade-enchilada-sauce.html
Predict:1.0Illustrationevergreen

The web: http://lolpics.se/18552-stun-grenade-ar
Predict:0.0Illustrationephemeral

The web: http://www.xcelerationfitness.com/treadmills.html
Predict:0.0Illustrationephemeral

The web: http://www.bloomberg.com/news/2012-02-06/syria-s-assad-deploys-tactics-of-father-to-crush-revolt-threatening-reign.html
Predict:0.0Illustrationephemeral

The web: http://www.wired.com/gadgetlab/2011/12/stem-turns-lemons-and-limes-into-juicy-atomizers/
Predict:0.0Illustrationephemeral

The web: http://www.latimes.com/health/boostershots/la-heb-fat-tax-denmark-20111013,0,2603132.story
Predict:0.0Illustrationephemeral

The web: http://www.howlifeworks.com/a/a?AG_ID=1186&cid=7340ci
Predict:1.0Illustrationevergreen

The web: http://romancingthestoveblog.wordpress.com/2010/01/13/sweet-potato-ravioli-with-lemon-sage-brown-butter-sauce/
Predict:1.0Illustrationevergreen

The web: http://www.funniez.net/Funny-Pictures/turn-men-down.html
Predict:0.0Illustrationephemeral

The web: http://youfellasleepwatchingadvd.com/
Predict:1.0Illustrationevergreen

DecisionTreeModel classifier of depth 10 with 419 nodes
If (feature 31 <= 1458.5)
If (feature 2 <= 0.5)
If (feature 23 <= 0.0391787705)
If (feature 16 <= 0.719848485)
If (feature 1 <= 0.5)
If (feature 14 <= 0.8458745000000001)
If (feature 8 <= 0.5)
If (feature 0 <= 0.5)
If (feature 29 <= 34.5)
If (feature 16 <= 0.5433298440000001)
Predict: 0.0
Else (feature 16 > 0.5433298440000001)
Predict: 1.0
Else (feature 29 > 34.5)
Predict: 0.0
Else (feature 0 > 0.5)
If (feature 33 <= 5.5)
If (feature 34 <= 0.2243290195)
Predict: 1.0
Else (feature 34 > 0.2243290195)
Predict: 0.0
Else (feature 33 > 5.5)
Predict: 0.0
Else (feature 8 > 0.5)
If (feature 14 <= 0.4711965)
If (feature 14 <= 0.46250199999999997)
Predict: 1.0
Else (feature 14 > 0.46250199999999997)
Predict: 0.0
Else (feature 14 > 0.4711965)
Predict: 1.0
Else (feature 14 > 0.8458745000000001)
Predict: 0.0
Else (feature 1 > 0.5)
If (feature 18 <= 9.702515E-4)
If (feature 34 <= 4.495E-4)
If (feature 23 <= 0.0197483205)
If (feature 35 <= 0.043529472)
Predict: 1.0
Else (feature 35 > 0.043529472)
Predict: 0.0
Else (feature 23 > 0.0197483205)
Predict: 1.0
Else (feature 34 > 4.495E-4)
Predict: 0.0
Else (feature 18 > 9.702515E-4)
If (feature 16 <= 0.4610159345)
Predict: 1.0
Else (feature 16 > 0.4610159345)
If (feature 19 <= 0.050450614000000005)
If (feature 17 <= 0.07201618500000001)
Predict: 1.0
Else (feature 17 > 0.07201618500000001)
Predict: 0.0
Else (feature 19 > 0.050450614000000005)
Predict: 1.0
Else (feature 16 > 0.719848485)
If (feature 19 <= 0.10272814250000001)
If (feature 23 <= 0.035596906)
If (feature 18 <= 0.3340675475)
If (feature 31 <= 1350.5)
If (feature 6 <= 0.5)
Predict: 1.0
Else (feature 6 > 0.5)
Predict: 0.0
Else (feature 31 > 1350.5)
If (feature 18 <= 0.0885694875)
Predict: 1.0
Else (feature 18 > 0.0885694875)
Predict: 0.0
Else (feature 18 > 0.3340675475)
Predict: 0.0
Else (feature 23 > 0.035596906)
Predict: 0.0
Else (feature 19 > 0.10272814250000001)
Predict: 1.0
Else (feature 23 > 0.0391787705)
If (feature 26 <= 0.614921223)
If (feature 15 <= 1.882617912)
If (feature 18 <= 0.0618079675)
If (feature 25 <= 0.18241650199999998)
If (feature 14 <= 0.88331)
If (feature 15 <= 1.5707610145)
Predict: 0.0
Else (feature 15 > 1.5707610145)
Predict: 1.0
Else (feature 14 > 0.88331)
Predict: 1.0
Else (feature 25 > 0.18241650199999998)
If (feature 26 <= 0.18868168899999999)
Predict: 0.0
Else (feature 26 > 0.18868168899999999)
If (feature 32 <= 184.5)
If (feature 23 <= 0.047017412999999994)
Predict: 1.0
Else (feature 23 > 0.047017412999999994)
Predict: 0.0
Else (feature 32 > 184.5)
Predict: 0.0
Else (feature 18 > 0.0618079675)
If (feature 15 <= 1.813996156)
If (feature 15 <= 1.424541885)
Predict: 1.0
Else (feature 15 > 1.424541885)
If (feature 23 <= 0.105398118)
If (feature 15 <= 1.7089758339999999)
Predict: 0.0
Else (feature 15 > 1.7089758339999999)
Predict: 1.0
Else (feature 23 > 0.105398118)
Predict: 1.0
Else (feature 15 > 1.813996156)
Predict: 1.0
Else (feature 15 > 1.882617912)
If (feature 14 <= 0.48163900000000004)
If (feature 15 <= 2.718090381)
If (feature 17 <= 0.08549820050000001)
If (feature 4 <= 0.5)
Predict: 0.0
Else (feature 4 > 0.5)
If (feature 23 <= 0.0719649975)
Predict: 1.0
Else (feature 23 > 0.0719649975)
Predict: 0.0
Else (feature 17 > 0.08549820050000001)
If (feature 25 <= 0.2354568495)
Predict: 0.0
Else (feature 25 > 0.2354568495)
If (feature 14 <= 0.37942050000000005)
Predict: 0.0
Else (feature 14 > 0.37942050000000005)
Predict: 1.0
Else (feature 15 > 2.718090381)
If (feature 35 <= 0.3323970035)
If (feature 16 <= 0.641086983)
If (feature 16 <= 0.29731728749999997)
Predict: 1.0
Else (feature 16 > 0.29731728749999997)
Predict: 0.0
Else (feature 16 > 0.641086983)
If (feature 15 <= 3.024357284)
Predict: 0.0
Else (feature 15 > 3.024357284)
Predict: 1.0
Else (feature 35 > 0.3323970035)
Predict: 1.0
Else (feature 14 > 0.48163900000000004)
If (feature 20 <= 0.408191065)
If (feature 33 <= 5.5)
If (feature 20 <= 0.392865142)
If (feature 15 <= 3.219436535)
Predict: 0.0
Else (feature 15 > 3.219436535)
Predict: 1.0
Else (feature 20 > 0.392865142)
Predict: 1.0
Else (feature 33 > 5.5)
Predict: 1.0
Else (feature 20 > 0.408191065)
If (feature 20 <= 16.0)
Predict: 0.0
Else (feature 20 > 16.0)
If (feature 34 <= 0.0059084325)
Predict: 0.0
Else (feature 34 > 0.0059084325)
If (feature 25 <= 0.296182712)
Predict: 1.0
Else (feature 25 > 0.296182712)
Predict: 0.0
Else (feature 26 > 0.614921223)
If (feature 17 <= 0.29483385900000003)
If (feature 26 <= 3.845238095)
If (feature 32 <= 90.5)
If (feature 25 <= 0.260248616)
If (feature 20 <= 0.6091436754999999)
If (feature 14 <= 0.651891)
Predict: 0.0
Else (feature 14 > 0.651891)
Predict: 1.0
Else (feature 20 > 0.6091436754999999)
Predict: 0.0
Else (feature 25 > 0.260248616)
If (feature 20 <= 0.6210849915000001)
If (feature 25 <= 0.263034435)
Predict: 0.0
Else (feature 25 > 0.263034435)
Predict: 1.0
Else (feature 20 > 0.6210849915000001)
Predict: 0.0
Else (feature 32 > 90.5)
If (feature 35 <= 0.068156843)
If (feature 15 <= 1.199737533)
If (feature 1 <= 0.5)
Predict: 0.0
Else (feature 1 > 0.5)
Predict: 1.0
Else (feature 15 > 1.199737533)
Predict: 0.0
Else (feature 35 > 0.068156843)
If (feature 31 <= 1059.5)
Predict: 0.0
Else (feature 31 > 1059.5)
Predict: 1.0
Else (feature 26 > 3.845238095)
Predict: 0.0
Else (feature 17 > 0.29483385900000003)
If (feature 15 <= 2.5117194654999997)
Predict: 1.0
Else (feature 15 > 2.5117194654999997)
Predict: 0.0
Else (feature 2 > 0.5)
If (feature 14 <= 0.5029295)
If (feature 33 <= 7.5)
If (feature 32 <= 20.5)
Predict: 0.0
Else (feature 32 > 20.5)
If (feature 32 <= 66.5)
Predict: 1.0
Else (feature 32 > 66.5)
Predict: 0.0
Else (feature 33 > 7.5)
Predict: 1.0
Else (feature 14 > 0.5029295)
If (feature 35 <= 0.053987811499999996)
If (feature 26 <= 0.092158035)
Predict: 0.0
Else (feature 26 > 0.092158035)
Predict: 1.0
Else (feature 35 > 0.053987811499999996)
If (feature 27 <= 0.5)
Predict: 1.0
Else (feature 27 > 0.5)
If (feature 25 <= 0.21417068)
If (feature 16 <= 0.617740727)
Predict: 0.0
Else (feature 16 > 0.617740727)
Predict: 1.0
Else (feature 25 > 0.21417068)
If (feature 24 <= 0.5)
Predict: 1.0
Else (feature 24 > 0.5)
Predict: 0.0
Else (feature 31 > 1458.5)
If (feature 3 <= 0.5)
If (feature 0 <= 0.5)
If (feature 1 <= 0.5)
If (feature 29 <= 25.5)
If (feature 19 <= 0.078501401)
If (feature 23 <= 0.052976552499999996)
If (feature 33 <= 6.5)
If (feature 26 <= 0.1836998015)
Predict: 1.0
Else (feature 26 > 0.1836998015)
If (feature 33 <= 0.5)
Predict: 0.0
Else (feature 33 > 0.5)
Predict: 1.0
Else (feature 33 > 6.5)
If (feature 8 <= 0.5)
If (feature 16 <= 0.387492143)
Predict: 0.0
Else (feature 16 > 0.387492143)
Predict: 1.0
Else (feature 8 > 0.5)
Predict: 0.0
Else (feature 23 > 0.052976552499999996)
If (feature 9 <= 0.5)
If (feature 16 <= 0.762858822)
Predict: 0.0
Else (feature 16 > 0.762858822)
Predict: 1.0
Else (feature 9 > 0.5)
If (feature 14 <= 0.2909735)
Predict: 1.0
Else (feature 14 > 0.2909735)
Predict: 0.0
Else (feature 19 > 0.078501401)
If (feature 15 <= 2.5904545455)
If (feature 32 <= 51.5)
If (feature 15 <= 2.241657397)
Predict: 0.0
Else (feature 15 > 2.241657397)
If (feature 4 <= 0.5)
Predict: 1.0
Else (feature 4 > 0.5)
Predict: 0.0
Else (feature 32 > 51.5)
If (feature 20 <= 0.4474304455)
If (feature 15 <= 2.4902903465)
Predict: 1.0
Else (feature 15 > 2.4902903465)
Predict: 0.0
Else (feature 20 > 0.4474304455)
If (feature 30 <= 0.5)
Predict: 1.0
Else (feature 30 > 0.5)
Predict: 0.0
Else (feature 15 > 2.5904545455)
If (feature 35 <= 0.11862860350000001)
If (feature 20 <= 0.604100313)
If (feature 26 <= 0.0703273525)
Predict: 1.0
Else (feature 26 > 0.0703273525)
Predict: 0.0
Else (feature 20 > 0.604100313)
Predict: 0.0
Else (feature 35 > 0.11862860350000001)
If (feature 26 <= 0.0023895)
If (feature 27 <= 0.5)
Predict: 1.0
Else (feature 27 > 0.5)
Predict: 0.0
Else (feature 26 > 0.0023895)
Predict: 0.0
Else (feature 29 > 25.5)
If (feature 14 <= 0.5597875000000001)
If (feature 17 <= 0.1432107495)
If (feature 32 <= 84.5)
Predict: 0.0
Else (feature 32 > 84.5)
If (feature 32 <= 99.5)
Predict: 1.0
Else (feature 32 > 99.5)
Predict: 0.0
Else (feature 17 > 0.1432107495)
If (feature 15 <= 2.54534632)
If (feature 35 <= 0.1135839195)
If (feature 18 <= 0.071340821)
Predict: 0.0
Else (feature 18 > 0.071340821)
Predict: 1.0
Else (feature 35 > 0.1135839195)
Predict: 0.0
Else (feature 15 > 2.54534632)
If (feature 16 <= 0.842136969)
Predict: 0.0
Else (feature 16 > 0.842136969)
If (feature 35 <= 0.0704363165)
Predict: 0.0
Else (feature 35 > 0.0704363165)
Predict: 1.0
Else (feature 14 > 0.5597875000000001)
If (feature 9 <= 0.5)
If (feature 2 <= 0.5)
Predict: 0.0
Else (feature 2 > 0.5)
If (feature 17 <= 0.2244121985)
If (feature 31 <= 2302.5)
Predict: 1.0
Else (feature 31 > 2302.5)
Predict: 0.0
Else (feature 17 > 0.2244121985)
If (feature 32 <= 368.5)
Predict: 1.0
Else (feature 32 > 368.5)
Predict: 0.0
Else (feature 9 > 0.5)
If (feature 15 <= 2.9814851825)
If (feature 29 <= 54.5)
Predict: 0.0
Else (feature 29 > 54.5)
Predict: 1.0
Else (feature 15 > 2.9814851825)
Predict: 0.0
Else (feature 1 > 0.5)
If (feature 35 <= 0.10557280899999999)
If (feature 15 <= 2.6319293674999997)
If (feature 26 <= 0.0102600085)
If (feature 15 <= 1.982317179)
If (feature 20 <= 0.4708425435)
If (feature 14 <= 0.28213)
Predict: 0.0
Else (feature 14 > 0.28213)
Predict: 1.0
Else (feature 20 > 0.4708425435)
Predict: 0.0
Else (feature 15 > 1.982317179)
If (feature 21 <= 1.065E-4)
Predict: 0.0
Else (feature 21 > 1.065E-4)
Predict: 1.0
Else (feature 26 > 0.0102600085)
If (feature 25 <= 0.33658205350000003)
If (feature 35 <= 0.0833107985)
Predict: 1.0
Else (feature 35 > 0.0833107985)
If (feature 31 <= 1624.5)
Predict: 0.0
Else (feature 31 > 1624.5)
Predict: 1.0
Else (feature 25 > 0.33658205350000003)
Predict: 0.0
Else (feature 15 > 2.6319293674999997)
If (feature 23 <= 0.0459418095)
If (feature 29 <= 18.5)
Predict: 1.0
Else (feature 29 > 18.5)
If (feature 14 <= 0.0799874)
Predict: 0.0
Else (feature 14 > 0.0799874)
Predict: 1.0
Else (feature 23 > 0.0459418095)
If (feature 17 <= 0.4005988025)
If (feature 35 <= 0.09598409050000001)
If (feature 35 <= 0.090486137)
Predict: 0.0
Else (feature 35 > 0.090486137)
Predict: 1.0
Else (feature 35 > 0.09598409050000001)
Predict: 0.0
Else (feature 17 > 0.4005988025)
Predict: 1.0
Else (feature 35 > 0.10557280899999999)
If (feature 20 <= 0.43892298949999997)
If (feature 32 <= 384.5)
If (feature 25 <= 0.248139513)
If (feature 34 <= 0.1917453335)
Predict: 0.0
Else (feature 34 > 0.1917453335)
If (feature 15 <= 2.2038107279999997)
Predict: 1.0
Else (feature 15 > 2.2038107279999997)
Predict: 0.0
Else (feature 25 > 0.248139513)
If (feature 14 <= 0.2909735)
Predict: 1.0
Else (feature 14 > 0.2909735)
If (feature 23 <= 0.051961552)
Predict: 0.0
Else (feature 23 > 0.051961552)
Predict: 1.0
Else (feature 32 > 384.5)
If (feature 18 <= 0.093286713)
Predict: 1.0
Else (feature 18 > 0.093286713)
Predict: 0.0
Else (feature 20 > 0.43892298949999997)
If (feature 26 <= 7.9E-5)
Predict: 0.0
Else (feature 26 > 7.9E-5)
If (feature 20 <= 0.6911990685)
If (feature 23 <= 0.076318364)
If (feature 29 <= 36.5)
Predict: 1.0
Else (feature 29 > 36.5)
Predict: 0.0
Else (feature 23 > 0.076318364)
If (feature 14 <= 0.8499995)
Predict: 1.0
Else (feature 14 > 0.8499995)
Predict: 0.0
Else (feature 20 > 0.6911990685)
Predict: 1.0
Else (feature 0 > 0.5)
If (feature 35 <= 0.123543858)
If (feature 26 <= 0.0398114305)
If (feature 14 <= 0.8499995)
If (feature 14 <= 0.765962)
If (feature 32 <= 184.5)
If (feature 23 <= 0.0221340885)
Predict: 1.0
Else (feature 23 > 0.0221340885)
If (feature 17 <= 0.07201618500000001)
Predict: 0.0
Else (feature 17 > 0.07201618500000001)
Predict: 1.0
Else (feature 32 > 184.5)
If (feature 19 <= 0.0482599545)
If (feature 34 <= 0.092270351)
Predict: 0.0
Else (feature 34 > 0.092270351)
Predict: 1.0
Else (feature 19 > 0.0482599545)
Predict: 0.0
Else (feature 14 > 0.765962)
If (feature 26 <= 0.017958696500000003)
If (feature 14 <= 0.8432904999999999)
Predict: 0.0
Else (feature 14 > 0.8432904999999999)
If (feature 15 <= 1.971547115)
Predict: 1.0
Else (feature 15 > 1.971547115)
Predict: 0.0
Else (feature 26 > 0.017958696500000003)
If (feature 18 <= 4.04E-4)
Predict: 0.0
Else (feature 18 > 4.04E-4)
Predict: 1.0
Else (feature 14 > 0.8499995)
If (feature 25 <= 0.18025862)
Predict: 0.0
Else (feature 25 > 0.18025862)
Predict: 1.0
Else (feature 26 > 0.0398114305)
If (feature 29 <= 33.5)
If (feature 25 <= 0.1375444065)
If (feature 32 <= 214.5)
Predict: 1.0
Else (feature 32 > 214.5)
Predict: 0.0
Else (feature 25 > 0.1375444065)
If (feature 23 <= 0.1289247745)
If (feature 31 <= 1497.0)
Predict: 0.0
Else (feature 31 > 1497.0)
If (feature 35 <= 0.0263854075)
Predict: 0.0
Else (feature 35 > 0.0263854075)
Predict: 1.0
Else (feature 23 > 0.1289247745)
Predict: 0.0
Else (feature 29 > 33.5)
If (feature 19 <= 0.046103239000000004)
If (feature 17 <= 0.1148483135)
Predict: 0.0
Else (feature 17 > 0.1148483135)
Predict: 1.0
Else (feature 19 > 0.046103239000000004)
If (feature 23 <= 0.036442443)
If (feature 31 <= 10718.5)
If (feature 16 <= 0.499097473)
Predict: 0.0
Else (feature 16 > 0.499097473)
Predict: 1.0
Else (feature 31 > 10718.5)
Predict: 0.0
Else (feature 23 > 0.036442443)
If (feature 25 <= 0.24691742700000002)
If (feature 16 <= 0.6619797155)
Predict: 0.0
Else (feature 16 > 0.6619797155)
Predict: 1.0
Else (feature 25 > 0.24691742700000002)
Predict: 1.0
Else (feature 35 > 0.123543858)
If (feature 20 <= 0.494038735)
If (feature 19 <= 0.150085324)
Predict: 0.0
Else (feature 19 > 0.150085324)
Predict: 1.0
Else (feature 20 > 0.494038735)
If (feature 29 <= 33.5)
Predict: 1.0
Else (feature 29 > 33.5)
If (feature 23 <= 0.0285232765)
Predict: 0.0
Else (feature 23 > 0.0285232765)
If (feature 15 <= 2.078819647)
Predict: 1.0
Else (feature 15 > 2.078819647)
Predict: 0.0
Else (feature 3 > 0.5)
If (feature 29 <= 30.5)
If (feature 26 <= 0.0137855585)
If (feature 31 <= 3875.5)
Predict: 1.0
Else (feature 31 > 3875.5)
If (feature 29 <= 5.5)
Predict: 1.0
Else (feature 29 > 5.5)
If (feature 18 <= 0.08599717800000001)
Predict: 0.0
Else (feature 18 > 0.08599717800000001)
If (feature 18 <= 0.13203773600000002)
Predict: 1.0
Else (feature 18 > 0.13203773600000002)
Predict: 0.0
Else (feature 26 > 0.0137855585)
If (feature 14 <= 0.4769935)
If (feature 15 <= 1.5387538305000001)
Predict: 1.0
Else (feature 15 > 1.5387538305000001)
If (feature 19 <= 0.032281749)
Predict: 0.0
Else (feature 19 > 0.032281749)
If (feature 35 <= 0.099369468)
If (feature 27 <= 0.5)
If (feature 14 <= 0.3311615)
Predict: 1.0
Else (feature 14 > 0.3311615)
Predict: 0.0
Else (feature 27 > 0.5)
Predict: 1.0
Else (feature 35 > 0.099369468)
Predict: 0.0
Else (feature 14 > 0.4769935)
If (feature 18 <= 0.0119284765)
Predict: 0.0
Else (feature 18 > 0.0119284765)
If (feature 19 <= 0.0092025405)
If (feature 16 <= 0.350257732)
Predict: 1.0
Else (feature 16 > 0.350257732)
If (feature 29 <= 20.5)
Predict: 0.0
Else (feature 29 > 20.5)
Predict: 1.0
Else (feature 19 > 0.0092025405)
If (feature 29 <= 15.5)
If (feature 15 <= 2.6508928569999997)
If (feature 20 <= 0.338772014)
Predict: 1.0
Else (feature 20 > 0.338772014)
Predict: 0.0
Else (feature 15 > 2.6508928569999997)
If (feature 19 <= 0.0482599545)
Predict: 1.0
Else (feature 19 > 0.0482599545)
Predict: 0.0
Else (feature 29 > 15.5)
Predict: 0.0
Else (feature 29 > 30.5)
If (feature 16 <= 0.572688423)
If (feature 16 <= 0.554178497)
If (feature 26 <= 0.048132494)
If (feature 14 <= 0.7296955)
Predict: 0.0
Else (feature 14 > 0.7296955)
Predict: 1.0
Else (feature 26 > 0.048132494)
Predict: 0.0
Else (feature 16 > 0.554178497)
If (feature 14 <= 0.3016565)
Predict: 0.0
Else (feature 14 > 0.3016565)
Predict: 1.0
Else (feature 16 > 0.572688423)
Predict: 0.0

标签：1.0,Python,MLlib,feature,hadoop,Predict,0.0,15,Else
来源： https://www.cnblogs.com/zhuozige/p/12623301.html

本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享；
2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关；
3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关；
4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除；
5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。

关于我们 | 联系我们 | 留言反馈

专注分享技术，共同学习，共同进步。侵权联系[81616952@qq.com]

Copyright (C)ICode9.com, All Rights Reserved.

ICode9版权所有