In this blog, we will be looking at the crime data in the city of San Francisco. The data we will be using contains all crimes in San Francisco from the year 2018 to 2020. You can download the data here. Since this dataset is very large (more than 330,000 crimes), we will be considering only a small part of the data for this post.

# import libraries
import numpy as np
import pandas as pd
import folium

# read the data into a pandas dataframe
df = pd.read_csv('SF_Crime_data.csv')
df.head()

# dimensions of the data
df.shape

(330054, 38)

There have been more than 330,000 crimes in San Francisco in the past two years.

df.columns

Index(['Incident Datetime', 'Incident Date', 'Incident Time', 'Incident Year',
       'Incident Day of Week', 'Report Datetime', 'Row ID', 'Incident ID',
       'Incident Number', 'CAD Number', 'Report Type Code',
       'Report Type Description', 'Filed Online', 'Incident Code',
       'Incident Category', 'Incident Subcategory', 'Incident Description',
       'Resolution', 'Intersection', 'CNN', 'Police District',
       'Analysis Neighborhood', 'Supervisor District', 'Latitude', 'Longitude',
       'point', 'SF Find Neighborhoods', 'Current Police Districts',
       'Current Supervisor Districts', 'Analysis Neighborhoods',
       'HSOC Zones as of 2018-06-05', 'OWED Public Spaces',
       'Central Market/Tenderloin Boundary Polygon - Updated',
       'Parks Alliance CPSI (27+TL sites)', 'ESNCAG - Boundary File',
       'Areas of Vulnerability, 2016', 'Unnamed: 36', 'Unnamed: 37'],
      dtype='object')

We do not need all these columns for our analysis. So we will consider only the necessary columns.

df = df[['Incident Datetime', 'Incident Day of Week', 'Incident Number', 'Incident Category', 'Incident Description', 
         'Police District', 'Analysis Neighborhood', 'Resolution', 'Latitude', 'Longitude', 'point']]
df.head()

Now, each row consists of the following 11 features:

• Incident Datetime: The date and time when the incident occurred
• Incident Day of Week: The day of week on which the incident occurred
• Incident Number: The incident or crime number
• Incident Category: The category of the incident or crime
• Incident Desccription: The description of the incident or crime
• Police: The police department district
• Resolution: The resolution of the crime in terms of whether the perpertrator was arrested or not
• Analysis Neighborhoods: The neighborhood where the incident took place
• Latitude: The latitude value of the crime location
• Longitude: The longitude value of the crime location
• point: A tuple of the latitude and logitude values

Let’s drop the missing values from the Latitude and Longitude columns as they will result in an error when creating a map.

df.dropna(subset=['Latitude', 'Longitude'], inplace=True)

Rename the Incident Category column for the sake of simplicity.

df.rename(columns={'Incident Category':'Category'}, inplace=True)
df.head()

limit = 100
df = df.iloc[0:limit, :]

# new dimensions of the data
df.shape

(100, 11)

Visualization

Let’s visualize where these crimes took place in the city of San Francisco.

# San Francisco latitude and longitude values
latitude = 37.7749
longitude = -122.4194

# create a map
sanfran_map = folium.Map(location=[latitude, longitude], zoom_start=12)
sanfran_map

Let’s group the markers into different clusters. Each cluster is then represented by the number of crimes in each neighborhood. These clusters can be thought of as pockets of San Francisco which you can then analyze separately.
We instantiate a MarkerCluster object and add all the data points in the dataframe to this object.

from folium import plugins

# let's start again with a clean copy of the map of San Francisco
sanfran_map = folium.Map(location = [latitude, longitude], zoom_start = 12)

# instantiate a mark cluster object for the incidents in the dataframe
incidents = plugins.MarkerCluster().add_to(sanfran_map)

# loop through the dataframe and add each data point to the mark cluster
for lat, lng, label, in zip(df.Latitude, df.Longitude, df.Category):
    folium.Marker(
        location=[lat, lng],
        icon=None,
        popup=label,
    ).add_to(incidents)

# display map
sanfran_map

If run the above code in a Jupyter notebook, you can zoom out all the way, and you will see that all markers are grouped into one cluster - the global cluster of 100 crimes (which is the total number of crimes we have considered in our dataset). Once you start zooming in, the global cluster will start breaking up into smaller clusters. Zooming in all the way will result in individual markers.