Pandas is a powerful library in Python for data manipulation and analysis, and its DataFrame is one of the key data structures provided by Pandas. Mastering Pandas DataFrames is essential for anyone working with data in Python, as it offers numerous methods for data manipulation, cleaning, and analysis.
Introduction to Pandas DataFrames
A DataFrame is a two-dimensional labeled data structure with rows and columns, similar to a spreadsheet or SQL table. It allows you to store and manipulate data in a tabular format efficiently. Before diving into methods, let's understand how to create a DataFrame. We can create a DataFrame using various methods, such as reading data from a CSV file or a database, casting a dictionary, or using NumPy arrays.
To create a Pandas DataFrame, we first need to import the Pandas library.
Now, let's create a DataFrame using a dictionary:
Basic Methods for Pandas DataFrame
head() and tail(): the head() and tail() methods of Pandas are used to display the first few rows and the last few rows of a DataFrame, respectively. By default, head() displays the first 5 rows, but we can specify the number of rows (an integer datatype) as an argument to the head() method. Similarly, tail() displays the last 5 rows by default, but we can also specify the number of rows (an integer datatype) as an argument to the tail() method.
print(data_df.head(3))This will display the first 3 rows of the DataFrame.
print(data_df.tail(2))This will display the last 2 rows of the DataFrame.
info(): the info() method displays information about the DataFrame, such as the number of rows and columns, the data type of each column, and the amount of memory used by the DataFrame.
print(data_df.info())The output of info() method results in information about the DataFrame, including the number of rows and columns, the data types of each column, and the number of non-null values in each column, particularly useful for identifying missing or null values, as well as understanding the data types of your columns.
describe(): the describe() method is used to generate descriptive statistics about the DataFrame, such as the count, mean, standard deviation, minimum, maximum values, and percentile levels (25%, 50%, and 75%) for each column.
print(data_df.describe())The output of describe() method is a summary table that provides insights into the distribution of values in each column of your dataset. It is a great tool for getting a quick overview of the data you're working with, and it can help you identify potential issues such as missing values, outliers, or inconsistencies in your data.
sort_values(): the sort_values() method is used to sort the DataFrame by one or more columns. By default, it sorts in ascending order, but we can specify descending order by setting the ascending argument to False (a boolean datatype).
print(data_df.sort_values(by=['column1']))This method sorts the DataFrame by ascending order of 'column1' and results in the sorted DataFrame in the output. The "by" parameter takes a list of column names to sort by. Since we only specified one column ('column1') to sort by, the resulting DataFrame is sorted based on the values in that column.
However, to sort the DataFrame in descending order of 'column1' then:
print(data_df.sort_values(by=['column1'], ascending=False))This sorts the same DataFrame by descending order of 'column1'. By setting the ascending argument to False, we tell sort_values() to sort the DataFrame in descending order. The resulting DataFrame will be sorted in descending order of the values in 'column1'.
fillna(): the fillna() method is used to replace missing or NaN values in a pandas DataFrame or Series with a specified value. This method takes a single parameter, which is the value to be used for replacing the missing values.
data = {'name': ['John', 'Jane', 'Mark', 'Mary', 'Mike'],
'age': [25, 23, None, 29, 31],
'gender': ['M', 'F', 'M', None, 'M']}
df = pd.DataFrame(data)
df.fillna(value=0, inplace=True)This will replace the missing values with 0.
Additionally, the inplace parameter is set to True, which means that the DataFrame is modified in place and the changes are reflected in the original DataFrame "data_df".
It allows us to fill in missing or null values in a DataFrame or Series. It replaces any NaN (Not a Number) values with a specified value or a set of values, particularly useful for dealing with missing data, as it can help to avoid errors in our analysis and modeling by ensuring that our data is complete.
drop(): the drop() method is used to remove rows or columns from a pandas DataFrame based on the axis parameter. It takes one or more parameters, which are the labels of the rows or columns to be removed. The axis parameter is set to 0 for removing rows and 1 for removing columns.
data = {'name': ['John', 'Jane', 'Mark', 'Mary', 'Mike'],
'age': [25, 23, 27, 29, 31],
'gender': ['M', 'F', 'M', 'F', 'M']}
df = pd.DataFrame(data)
df.drop([0, 2], axis=0, inplace=True)
The axis parameter is set to 0, which means that we are removing rows.
replace(): the replace() method is used to replace specified values in a pandas DataFrame or Series with new values. This method takes two parameters; the value to be replaced and the new value to replace it with.
data = {'name': ['John', 'Jane', 'Mark', 'Mary', 'Mike'],
'age': [25, 23, 27, 29, 31],
'gender': ['M', 'F', 'M', 'F', 'M']}
df = pd.DataFrame(data)
df.replace(to_replace='M', value='Male', inplace=True)This will replace the value 'M' in the gender column with 'Male'.
drop_duplicates(): the drop_duplicates() method is used to remove duplicate rows from a pandas DataFrame. This method takes one or more parameters, which are the column labels to use for identifying duplicates. By default, all columns are used to identify duplicates.
data = {'name': ['John', 'Jane', 'Mark', 'Mary', 'Mike
df = pd.DataFrame(data)
df.drop_duplicates(subset=['gender'], inplace=True)This will remove duplicate rows based on the 'gender' column.
sum(): the sum() method is used to calculate the sum of values in a pandas DataFrame or Series. This method takes one argument, which is the axis along which the sum is calculated. By default, the sum is calculated over all columns.
import pandas as pd
data = {'name': ['John', 'Jane', 'Mark', 'Mary', 'Mike'], 'age': [25, 23, 27, 29, 31], 'gender': ['M', 'F', 'M', 'F', 'M']}
df = pd.DataFrame(data)
total_age = df['age'].sum()This will calculate the total age of all the individuals in the DataFrame. It is a quick and useful tool to calculate the sum of values in a DataFrame or Series.
Comments