pandas数据清洗实现删除的项目实践
作者:ACxz 时间:2021-09-02 16:44:17
准备工作(导入库、导入数据)
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import seaborn as sns
sns.set_style("darkgrid")
list_csv = ['Amazon_top_selling_book.csv','breast_cancer_wisconsin.csv','diamonds.csv','insurance.csv','netflix_titles.csv','penguins.csv',
'titanic.csv','winequality-red.csv']
dic_path = r'C:\Users\pandas\Desktop\task\228datasets\datasets'
part_data = pd.read_csv(dic_path+'\\'+list_csv[4])
part_data
show_id | type | title | director | cast | country | date_added | release_year | rating | duration | listed_in | description | |
---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | s1 | Movie | Dick Johnson Is Dead | Kirsten Johnson | NaN | United States | September 25, 2021 | 2020 | PG-13 | 90 min | Documentaries | As her father nears the end of his life, filmm... |
1 | s2 | TV Show | Blood & Water | NaN | Ama Qamata, Khosi Ngema, Gail Mabalane, Thaban... | South Africa | September 24, 2021 | 2021 | TV-MA | 2 Seasons | International TV Shows, TV Dramas, TV Mysteries | After crossing paths at a party, a Cape Town t... |
2 | s3 | TV Show | Ganglands | Julien Leclercq | Sami Bouajila, Tracy Gotoas, Samuel Jouy, Nabi... | NaN | September 24, 2021 | 2021 | TV-MA | 1 Season | Crime TV Shows, International TV Shows, TV Act... | To protect his family from a powerful drug lor... |
3 | s4 | TV Show | Jailbirds New Orleans | NaN | NaN | NaN | September 24, 2021 | 2021 | TV-MA | 1 Season | Docuseries, Reality TV | Feuds, flirtations and toilet talk go down amo... |
4 | s5 | TV Show | Kota Factory | NaN | Mayur More, Jitendra Kumar, Ranjan Raj, Alam K... | India | September 24, 2021 | 2021 | TV-MA | 2 Seasons | International TV Shows, Romantic TV Shows, TV ... | In a city of coaching centers known to train I... |
... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
8807 rows × 12 columns
检测数据情况
Hint:该函数用于检测任意DataFrame中缺失值情况
def missing_values_table(df):
mis_val = df.isnull().sum()
mis_val_percent = 100 * df.isnull().sum() / len(df)
mis_val_table = pd.concat([mis_val, mis_val_percent], axis=1)
mis_val_table_ren_columns = mis_val_table.rename(
columns = {0 : 'Missing Values', 1 : '% of Total Values'})
mis_val_table_ren_columns = mis_val_table_ren_columns[
mis_val_table_ren_columns.iloc[:,1] != 0].sort_values(
'% of Total Values', ascending=False).round(1)
print ("Your selected dataframe has " + str(df.shape[1]) + " columns.\n"
"There are " + str(mis_val_table_ren_columns.shape[0]) +
" columns that have missing values.")
return mis_val_table_ren_columns
missing_values_table(part_data)
Your selected dataframe has 12 columns.
There are 6 columns that have missing values.
Missing Values | % of Total Values | |
---|---|---|
director | 2634 | 29.9 |
country | 831 | 9.4 |
cast | 825 | 9.4 |
date_added | 10 | 0.1 |
rating | 4 | 0.0 |
duration | 3 | 0.0 |
DataFrame.drop(labels=None,axis=0, index=None, columns=None, inplace=False)
参数说明:
labels 就是要删除的行列的名字,用列表给定
axis 默认为0,指删除行,因此删除columns时要指定axis=1;
index 直接指定要删除的行
columns 直接指定要删除的列
inplace=False,默认该删除操作不改变原数据,而是返回一个执行删除操作后的新dataframe;
inplace=True,则会直接在原数据上进行删除操作,删除后无法返回。
方式一:删除指定行或列
labels+axis
demo = part_data.drop(['director'], axis=1)
missing_values_table(demo)
Your selected dataframe has 11 columns.
There are 5 columns that have missing values.
Missing Values | % of Total Values | |
---|---|---|
country | 831 | 9.4 |
cast | 825 | 9.4 |
date_added | 10 | 0.1 |
rating | 4 | 0.0 |
duration | 3 | 0.0 |
方式二:利用boolean删除满足条件元素所在的行
df = df.drop(df[].index)
# 删除release_year年份在2009年之前的行
demo = part_data.drop(part_data[part_data["release_year"]<2009].index)
demo.shape
(7624, 12)
来源:https://blog.csdn.net/pylittlebrat/article/details/124505220
标签:pandas,数据清洗,删除
![](/images/zang.png)
![](/images/jiucuo.png)
猜你喜欢
pytorch 两个GPU同时训练的解决方案
2023-07-04 11:03:55
XHTML 1.0:标记新的开端
2023-08-04 21:59:30
Numpy中如何创建矩阵并等间隔抽取数据
2023-01-14 16:21:16
![](https://img.aspxhome.com/file/2023/0/81780_0s.png)
PyTorch中的squeeze()和unsqueeze()解析与应用案例
2022-09-22 23:04:49
![](https://img.aspxhome.com/file/2023/2/77762_0s.png)
ASP表单验证方法总结
2007-10-06 22:43:00
python中的Elasticsearch操作汇总
2022-01-29 10:44:45
Python通过wordcloud库实现将单词生成词云
2022-02-24 20:17:18
![](https://img.aspxhome.com/file/2023/3/83843_0s.png)
python字典的setdefault的巧妙用法
2021-05-18 02:58:48
通用的下拉菜单__用DL\\DD\\DT解决无法遮住select的问题
2008-07-28 13:28:00
详解Laravel模型事件和模型事件在Trait中的使用
2023-06-17 17:19:28
python调用matlab的m自定义函数方法
2023-11-11 21:40:52
原创一个AJAX类
2008-07-24 13:29:00
Python依赖包迁移到断网环境操作
2021-06-27 06:58:29
vscode 远程调试python的方法
2021-09-07 23:00:59
![](https://img.aspxhome.com/file/2023/7/83717_0s.png)
使用gunicorn部署django项目的问题
2022-07-16 16:39:55
分享我们的select控件设计过程
2009-06-16 18:04:00
![](https://img.aspxhome.com/file/UploadPic/20096/16/01-92s.png)
python dataframe astype 字段类型转换方法
2022-02-19 07:58:50
Python 实现将某一列设置为str类型
2022-07-27 03:20:12
2个asp防刷新程序代码
2008-09-28 21:16:00
如何基于python实现年会抽奖工具
2022-01-28 04:24:42