基于Python实现的ID3决策树功能示例
作者:leeliyang 时间:2021-10-24 07:57:24
本文实例讲述了基于Python实现的ID3决策树功能。分享给大家供大家参考,具体如下:
ID3算法是决策树的一种,它是基于奥卡姆剃刀原理的,即用尽量用较少的东西做更多的事。ID3算法,即Iterative Dichotomiser 3,迭代二叉树3代,是Ross Quinlan发明的一种决策树算法,这个算法的基础就是上面提到的奥卡姆剃刀原理,越是小型的决策树越优于大的决策树,尽管如此,也不总是生成最小的树型结构,而是一个启发式算法。
如下示例是一个判断海洋生物数据是否是鱼类而构建的基于ID3思想的决策树
# coding=utf-8
import operator
from math import log
import time
def createDataSet():
dataSet = [[1, 1, 'yes'],
[1, 1, 'yes'],
[1, 0, 'no'],
[0, 1, 'no'],
[0, 1, 'no'],
[0,0,'maybe']]
labels = ['no surfaceing', 'flippers']
return dataSet, labels
# 计算香农熵
def calcShannonEnt(dataSet):
numEntries = len(dataSet)
labelCounts = {}
for feaVec in dataSet:
currentLabel = feaVec[-1]
if currentLabel not in labelCounts:
labelCounts[currentLabel] = 0
labelCounts[currentLabel] += 1
shannonEnt = 0.0
for key in labelCounts:
prob = float(labelCounts[key]) / numEntries
shannonEnt -= prob * log(prob, 2)
return shannonEnt
def splitDataSet(dataSet, axis, value):
retDataSet = []
for featVec in dataSet:
if featVec[axis] == value:
reducedFeatVec = featVec[:axis]
reducedFeatVec.extend(featVec[axis + 1:])
retDataSet.append(reducedFeatVec)
return retDataSet
def chooseBestFeatureToSplit(dataSet):
numFeatures = len(dataSet[0]) - 1 # 因为数据集的最后一项是标签
baseEntropy = calcShannonEnt(dataSet)
bestInfoGain = 0.0
bestFeature = -1
for i in range(numFeatures):
featList = [example[i] for example in dataSet]
uniqueVals = set(featList)
newEntropy = 0.0
for value in uniqueVals:
subDataSet = splitDataSet(dataSet, i, value)
prob = len(subDataSet) / float(len(dataSet))
newEntropy += prob * calcShannonEnt(subDataSet)
infoGain = baseEntropy - newEntropy
if infoGain > bestInfoGain:
bestInfoGain = infoGain
bestFeature = i
return bestFeature
# 因为我们递归构建决策树是根据属性的消耗进行计算的,所以可能会存在最后属性用完了,但是分类
# 还是没有算完,这时候就会采用多数表决的方式计算节点分类
def majorityCnt(classList):
classCount = {}
for vote in classList:
if vote not in classCount.keys():
classCount[vote] = 0
classCount[vote] += 1
return max(classCount)
def createTree(dataSet, labels):
classList = [example[-1] for example in dataSet]
if classList.count(classList[0]) == len(classList): # 类别相同则停止划分
return classList[0]
if len(dataSet[0]) == 1: # 所有特征已经用完
return majorityCnt(classList)
bestFeat = chooseBestFeatureToSplit(dataSet)
bestFeatLabel = labels[bestFeat]
myTree = {bestFeatLabel: {}}
del (labels[bestFeat])
featValues = [example[bestFeat] for example in dataSet]
uniqueVals = set(featValues)
for value in uniqueVals:
subLabels = labels[:] # 为了不改变原始列表的内容复制了一下
myTree[bestFeatLabel][value] = createTree(splitDataSet(dataSet,
bestFeat, value), subLabels)
return myTree
def main():
data, label = createDataSet()
t1 = time.clock()
myTree = createTree(data, label)
t2 = time.clock()
print myTree
print 'execute for ', t2 - t1
if __name__ == '__main__':
main()
运行结果如下:
{'no surfaceing': {0: {'flippers': {0: 'maybe', 1: 'no'}}, 1: {'flippers': {0: 'no', 1: 'yes'}}}}
execute for 0.0103958394532
最后我们测试一下这个脚本即可,如果想把这个生成的决策树用图像画出来,也只是在需要在脚本里面定义一个plottree的函数即可。
希望本文所述对大家Python程序设计有所帮助。
来源:http://blog.csdn.net/gentelyang/article/details/75195630
标签:Python,ID3,决策树
![](/images/zang.png)
![](/images/jiucuo.png)
猜你喜欢
松子学ASP—Server篇
2008-04-22 14:14:00
Python基于jieba库进行简单分词及词云功能实现方法
2023-03-21 16:28:47
![](https://img.aspxhome.com/file/2023/7/97597_0s.jpg)
10个值得关注的优秀CSS框架
2009-05-29 18:11:00
![](https://img.aspxhome.com/file/UploadPic/20095/29/blueprint-27s.png)
Python 将 QQ 好友头像生成祝福语的实现代码
2022-01-11 20:29:33
![](https://img.aspxhome.com/file/2023/2/104922_0s.png)
使用python爬取抖音视频列表信息
2023-04-06 12:51:48
![](https://img.aspxhome.com/file/2023/3/107283_0s.jpg)
np.array()函数的使用方法
2022-07-08 08:21:05
PyQt5 QLineEdit输入的子网字符串校验QRegExp实现
2022-05-08 16:47:58
![](https://img.aspxhome.com/file/2023/7/121437_0s.png)
python实现批量解析邮件并下载附件
2023-07-05 02:32:16
找Python安装目录,设置环境路径以及在命令行运行python脚本实例
2022-06-01 23:28:56
![](https://img.aspxhome.com/file/2023/6/96976_0s.jpg)
JavaScript中Object基础内部方法图
2023-08-15 02:36:11
![](https://img.aspxhome.com/file/2023/1/56071_0s.jpg)
Python实现简单猜拳游戏
2022-07-08 04:40:10
![](https://img.aspxhome.com/file/2023/0/124210_0s.jpg)
页面中 CSS 加载方式的优化
2008-03-26 12:36:00
在Django中创建动态视图的教程
2021-09-25 09:17:28
Python 记录日志的灵活性和可配置性介绍
2022-06-05 12:30:15
![](https://img.aspxhome.com/file/2023/7/95857_0s.png)
python中将\\\\uxxxx转换为Unicode字符串的方法
2023-11-04 15:20:07
Python 中 function(#) (X)格式 和 (#)在Python3.*中的注意事项
2021-11-15 18:53:31
交互设计杂七杂八
2010-09-25 18:41:00
![](https://img.aspxhome.com/file/UploadPic/20109/25/02-24s.jpg)
keras中epoch,batch,loss,val_loss用法说明
2021-08-11 10:56:30
![](https://img.aspxhome.com/file/2023/2/83742_0s.jpg)
在VS2017中用C#调用python脚本的实现
2021-09-19 00:59:06
django admin添加数据自动记录user到表中的实现方法
2022-11-17 14:58:24