Learning Pandas: From Clueless to Curious in One Read | by PriyeshShah | Jul, 2025

Welcome to your final beginning information for knowledge evaluation in Python! In the event you’ve ever needed to discover giant datasets and uncover hidden insights, you’re in the suitable place. Immediately, we’re diving deep into Pandas, essentially the most important Python library for knowledge manipulation.

We’ll stroll by means of every part from loading your first dataset to asking complicated questions of it. We’ll use the real-world Stack Overflow Developer Survey as our playground, combining ideas from prime tutorials with hands-on code you possibly can run your self. Let’s get began!

Earlier than we will analyze something, we’d like knowledge. Step one is all the time to load our knowledge right into a Pandas DataFrame. A DataFrame is the core of Pandas — consider it as a sensible spreadsheet or a desk with rows and columns.

First, let’s import the library (the as pd is a normal conference) and cargo our survey knowledge from a CSV file.

import pandas as pd
# Load the primary survey outcomes
df = pd.read_csv('survey_results_public.csv')

Nice! Our knowledge is now in a DataFrame known as df. However what does it appear like? How massive is it? Let’s do some preliminary inspection.

• Test the scale with .form: This attribute exhibits you the size in a (rows, columns) format.

df.form # Output: (88883, 85)

• That’s quite a bit, 88,883 rows and 85 columns!
• Get a technical abstract with .information(): This methodology provides a breakdown of every column, its knowledge kind, and what number of non-null values it accommodates. It is excellent for a fast overview.

df.information()

• Have a look at no matter a part of knowledge you need with .head() and .tail(): You do not wish to print all 88,000 rows. Use .head() to see the first few rows and .tail() to see the final few.

# See the primary 5 rows 
df.head()  
# See the final 10 rows 
df.tail(10)

Professional Tip: With 85 columns, Pandas will cover some from view. To see all of them, you possibly can change the show choices:

pd.set_option('show.max_columns', 85)
pd.set_option('show.max_row', 85)

A DataFrame is only a assortment of Sequence. You may consider a Sequence as a single column of information. More often than not, you’ll wish to work with particular columns or rows.

Deciding on Columns

You may seize a single column (a Sequence) utilizing bracket notation, identical to with a Python dictionary or To pick a number of columns, go an inventory of column names. This can return a brand new, smaller DataFrame.

# Get the 'Hobbyist' column
df['Hobbyist']
# Get the Nation and Schooling Stage columns
df[['Country', 'EdLevel']]

Deciding on Rows with .loc and .iloc

Pandas provides us two main methods to pick rows:

• .iloc (integer location): Selects rows primarily based on their integer place (e.g., the first row, fifth row, and so on.).
• .loc (label location): Selects rows primarily based on their index label.

Let’s see it in motion:

# Get the primary row of information utilizing its integer place
df.iloc[0]
# Get the primary three rows
df.iloc[0:3]
# Get the primary row utilizing its index label (which can also be 0 by default)
df.loc[0]
# Get the primary three rows by label
df.loc[0:3]

Proper now, .loc and .iloc appear to do the identical factor as a result of our default index is simply integers. However what if we had a extra significant index?

The index is the identifier for every row. Whereas the default integer index works, we will make our knowledge a lot simpler to look by setting a extra significant index. The ‘Respondent’ column in our knowledge accommodates a novel ID for every particular person. Let’s make that our index!

You are able to do this proper if you load the information utilizing the index_col argument. That is tremendous environment friendly.

# Load knowledge and set 'Respondent' because the index instantly
df = pd.read_csv('survey_results_public.csv', index_col='QName')

Now, our DataFrame is listed by the respondent’s ID. This makes .loc extremely highly effective as a result of we will now fetch rows by this distinctive ID.

# Get the complete survey response for the particular person with Respondent ID 1
df.loc[1]
df.loc[1,'question'] # We are able to deepen down our search much more

In the event you ever wish to change the index again to the default, you should use reset_index(). To make your index simpler to look, you too can kind it with sort_index().

That is the place knowledge evaluation really begins. Filtering is how we ask questions and pull out particular subsets of information. The method includes making a “filter masks” — a Sequence of True/False values—and making use of it to our DataFrame.

Let’s discover all of the builders from India.

• Create the filter masks: This line doesn’t return the information itself, however a Sequence the place True marks a row the place the ‘Nation’ is ‘India’.

filt = (df['Country'] == 'India')
# Or discovering another particulars you need out of your dataframe
years_code = (df['YearsCode'] > "5")

• Apply the filter with .loc: Now, we use our filter inside .loc to get all of the rows that match.

df.loc[filt]
# You may also mix these filters
combined_filt = years_code & filt 
# And Print these with the particular particulars you need from the dataframe
df.loc[combined_filt,['Age','DevType','LanguageHaveWorkedWith']]

• And identical to that, you’ve a DataFrame containing solely the survey respondents from India!

Combining & Negating Filters

What if in case you have a number of situations?

• Use the AND operator (&) when all situations should be true.
• Use the OR operator (|) when no less than one situation should be true.
• Use the tilde (~) to negate a filter (get every part that does not match).

Let’s discover all of the builders from the United States who’re additionally hobbyist coders.

# Notice the parentheses round every situation
us_hobbyist_filt = (df['Country'] == 'United States') & (df['Hobbyist'] == 'Sure')
df.loc[us_hobbyist_filt]

To get everybody not from the USA, we might do:

# This neglets and thoes the alternative of what we wish so we might get 
# everybody apart from United States
df.loc[~(df['Country'] == 'United States')]

Superior Filtering

• Filtering by an inventory with .isin(): To search out respondents from an inventory of nations (e.g., India, Germany, or the UK), .isin() is way cleaner than an extended OR chain.

international locations = ['India', 'Germany', 'United Kingdom'] 
country_filt = df['Country'].isin(international locations) 
df.loc[country_filt]

• Filtering strings with .str.accommodates(): Wish to discover each respondent who talked about ‘Python’ of their LanguageWorkedWith response? .str.accommodates() is ideal for this.

# na=False handles any lacking values to keep away from errors 
python_filt = df['LanguageWorkedWith'].str.accommodates('Python', na=False) 
df.loc[python_filt]

And there you’ve it! You’ve gone from loading a uncooked CSV file to inspecting it, choosing particular knowledge, creating a robust index, and asking complicated questions with superior filtering. These are the basic constructing blocks of virtually each knowledge evaluation mission you’ll ever encounter.

One of the simplest ways to study is by doing. Attempt asking your individual questions of the Stack Overflow dataset. With Pandas, you now have the instruments to seek out out. Keep tuned for Half 2, the place we’ll cowl modifying knowledge, dealing with lacking values, and rather more. Blissful coding!