Chapter 20 Subsetting Data

20.0.0.1 by columns

selected_cols = ['id', 'name', 'city']

df_csv[selected_cols]

##         id                                        name           city
## 0   102669                   Alaska Pacific University      Anchorage
## 1   101648                   Marion Military Institute         Marion
## 2   100830             Auburn University at Montgomery     Montgomery
## 3   101879                 University of North Alabama       Florence
## 4   100858                           Auburn University         Auburn
## ..     ...                                         ...            ...
## 95  118888                               Mills College        Oakland
## 96  110097                            Biola University      La Mirada
## 97  117140                      University of La Verne       La Verne
## 98  125727                            Westmont College  Santa Barbara
## 99  123651  Vanguard University of Southern California     Costa Mesa
## 
## [100 rows x 3 columns]

20.0.0.2 Label based: df.loc[row_label, column_label]

Selecting Rows and Columns by Labels:

You can use df.loc[] to select rows and columns of a DataFrame using labels. The syntax is df.loc[row_label, column_label].

You can specify a single label or a list of labels for both rows and columns.

row label is the index.

# Select a single row and column
df_csv.loc[1, 'name']

## 'Marion Military Institute'

# Select multiple rows and columns
df_csv.loc[[0, 2, 4], ['name', 'state']]

##                               name state
## 0        Alaska Pacific University    AK
## 2  Auburn University at Montgomery    AL
## 4                Auburn University    AL

# Select rows from 'row_label_start' to 'row_label_end'
df_csv.loc[0:4]

##        id                             name        city state region
## 0  102669        Alaska Pacific University   Anchorage    AK   West
## 1  101648        Marion Military Institute      Marion    AL  South
## 2  100830  Auburn University at Montgomery  Montgomery    AL  South
## 3  101879      University of North Alabama    Florence    AL  South
## 4  100858                Auburn University      Auburn    AL  South

## all same
# df_csv.loc[0:4, ]
# df_csv.loc[0:4, :]

# Select rows where a condition is True
df_csv.loc[df_csv['id'] < 101000]

##         id                                 name        city state region
## 2   100830      Auburn University at Montgomery  Montgomery    AL  South
## 4   100858                    Auburn University      Auburn    AL  South
## 5   100663  University of Alabama at Birmingham  Birmingham    AL  South
## 9   100751            The University of Alabama  Tuscaloosa    AL  South
## 11  100706  University of Alabama in Huntsville  Huntsville    AL  South
## 18  100937          Birmingham Southern College  Birmingham    AL  South
## 21  100724             Alabama State University  Montgomery    AL  South
## 23  100654             Alabama A & M University      Normal    AL  South

df = df_csv.copy()
# Set a value for a specific row and column
df.loc[10, 'name'] = 'My University'

20.0.0.3 Filtering Rows Based on a Single Condition:

df = df_csv.copy()
# Select rows where the 'column_name' equals a specific value
df[df['name'] == 'Auburn University']

##        id               name    city state region
## 4  100858  Auburn University  Auburn    AL  South

df[df.name == 'Auburn University']

##        id               name    city state region
## 4  100858  Auburn University  Auburn    AL  South

20.0.0.4 Filtering Rows Based on Multiple Conditions (AND):

# Select rows where 'column1' equals 'value1' and 'column2' equals 'value2'
df[(df.city == 'Birmingham') & (df.state == 'AL')]

##         id                                 name        city state region
## 5   100663  University of Alabama at Birmingham  Birmingham    AL  South
## 7   102049                   Samford University  Birmingham    AL  South
## 10  102261           Southeastern Bible College  Birmingham    AL  South
## 18  100937          Birmingham Southern College  Birmingham    AL  South

20.0.1 copy

When you use loc/iloc to create a subset of the DataFrame, you are also creating a view into the original DataFrame. Modifications to the subset will reflect in the original DataFrame, and vice versa.

subset = df.loc['row_label1':'row_label5', 'column_labelA':'column_labelB']

subset = df.iloc[0:5, 1:3]  # Select rows 0 to 4 and columns 1 to 2

To create an independent copy of the DataFrame, you can use the copy() method. This ensures that modifications to the copied DataFrame do not affect the original DataFrame.

copy_df = df.copy()