Principal Component Analysis Demo¶

Load Required Libraries¶

Some libraries come preinstalled in the Jupyter environment, others need to be installed.

In [1]:
import warnings
warnings.filterwarnings("ignore") # filter out warnings
import numpy as np
np.set_printoptions(formatter={'float': lambda x: "{0:0.4f}".format(x)}) # sets decimal places
np.set_printoptions(suppress=True) # disables scientific notation
import pandas as pd 
import seaborn as sns
import plotly.express as px
import matplotlib.pyplot as plt
# %matplotlib inline
import pca
#pip install opencv-python
import cv2
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
In [2]:
from sklearn.datasets import load_breast_cancer
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis

# Models
from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.linear_model import LogisticRegression

# Model Performance Evaluators
from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report

Load Breast Cancer Dataset¶

Scikitlearn comes with a breast cancer dataset. Features in this dataset describe cell nuclei characteristics from digitized images of fine needle aspirate (FNA) of breast masses.

In [3]:
data = load_breast_cancer()
df = pd.DataFrame(data=data.data, columns=data.feature_names)
df['target'] = data.target
# Check out the dataset
df.head()
Out[3]:
mean radius mean texture mean perimeter mean area mean smoothness mean compactness mean concavity mean concave points mean symmetry mean fractal dimension ... worst texture worst perimeter worst area worst smoothness worst compactness worst concavity worst concave points worst symmetry worst fractal dimension target
0 17.99 10.38 122.80 1001.0 0.11840 0.27760 0.3001 0.14710 0.2419 0.07871 ... 17.33 184.60 2019.0 0.1622 0.6656 0.7119 0.2654 0.4601 0.11890 0
1 20.57 17.77 132.90 1326.0 0.08474 0.07864 0.0869 0.07017 0.1812 0.05667 ... 23.41 158.80 1956.0 0.1238 0.1866 0.2416 0.1860 0.2750 0.08902 0
2 19.69 21.25 130.00 1203.0 0.10960 0.15990 0.1974 0.12790 0.2069 0.05999 ... 25.53 152.50 1709.0 0.1444 0.4245 0.4504 0.2430 0.3613 0.08758 0
3 11.42 20.38 77.58 386.1 0.14250 0.28390 0.2414 0.10520 0.2597 0.09744 ... 26.50 98.87 567.7 0.2098 0.8663 0.6869 0.2575 0.6638 0.17300 0
4 20.29 14.34 135.10 1297.0 0.10030 0.13280 0.1980 0.10430 0.1809 0.05883 ... 16.67 152.20 1575.0 0.1374 0.2050 0.4000 0.1625 0.2364 0.07678 0

5 rows × 31 columns

In [4]:
# Check out the dimensions of the dataset - dropping last target column
df[list(df.columns[:-1])].shape
Out[4]:
(569, 30)

Data Pre-processing¶

Standardize the dataset so that large ranges do not dominate the analysis

In [5]:
# Define input and target datasets
inputs_df = df[list(df.columns[:-1])]
targets = df[(df.columns[-1])]

# Standarize input dataset
sc = StandardScaler()
inputs_df[df.columns[:-1]] = sc.fit_transform(inputs_df[df.columns[:-1]])

df_scaled = inputs_df[df.columns[:-1]]
inputs_df[df.columns[:-1]].head()
Out[5]:
mean radius mean texture mean perimeter mean area mean smoothness mean compactness mean concavity mean concave points mean symmetry mean fractal dimension ... worst radius worst texture worst perimeter worst area worst smoothness worst compactness worst concavity worst concave points worst symmetry worst fractal dimension
0 1.097064 -2.073335 1.269934 0.984375 1.568466 3.283515 2.652874 2.532475 2.217515 2.255747 ... 1.886690 -1.359293 2.303601 2.001237 1.307686 2.616665 2.109526 2.296076 2.750622 1.937015
1 1.829821 -0.353632 1.685955 1.908708 -0.826962 -0.487072 -0.023846 0.548144 0.001392 -0.868652 ... 1.805927 -0.369203 1.535126 1.890489 -0.375612 -0.430444 -0.146749 1.087084 -0.243890 0.281190
2 1.579888 0.456187 1.566503 1.558884 0.942210 1.052926 1.363478 2.037231 0.939685 -0.398008 ... 1.511870 -0.023974 1.347475 1.456285 0.527407 1.082932 0.854974 1.955000 1.152255 0.201391
3 -0.768909 0.253732 -0.592687 -0.764464 3.283553 3.402909 1.915897 1.451707 2.867383 4.910919 ... -0.281464 0.133984 -0.249939 -0.550021 3.394275 3.893397 1.989588 2.175786 6.046041 4.935010
4 1.750297 -1.151816 1.776573 1.826229 0.280372 0.539340 1.371011 1.428493 -0.009560 -0.562450 ... 1.298575 -1.466770 1.338539 1.220724 0.220556 -0.313395 0.613179 0.729259 -0.868353 -0.397100

5 rows × 30 columns

In [6]:
# Produces a summary table
inputs_df[df.columns[:-1]].describe()
Out[6]:
mean radius mean texture mean perimeter mean area mean smoothness mean compactness mean concavity mean concave points mean symmetry mean fractal dimension ... worst radius worst texture worst perimeter worst area worst smoothness worst compactness worst concavity worst concave points worst symmetry worst fractal dimension
count 5.690000e+02 5.690000e+02 5.690000e+02 5.690000e+02 5.690000e+02 5.690000e+02 5.690000e+02 5.690000e+02 5.690000e+02 5.690000e+02 ... 5.690000e+02 5.690000e+02 5.690000e+02 5.690000e+02 5.690000e+02 5.690000e+02 5.690000e+02 5.690000e+02 5.690000e+02 5.690000e+02
mean -3.153111e-15 -6.568462e-15 -6.993039e-16 -8.553985e-16 6.081447e-15 -1.136369e-15 -2.997017e-16 1.023981e-15 -1.860648e-15 -1.504752e-15 ... -2.297713e-15 1.742016e-15 -1.198807e-15 6.118909e-16 -5.094929e-15 -2.122887e-15 6.118909e-16 -1.998011e-16 -2.422589e-15 2.497514e-15
std 1.000880e+00 1.000880e+00 1.000880e+00 1.000880e+00 1.000880e+00 1.000880e+00 1.000880e+00 1.000880e+00 1.000880e+00 1.000880e+00 ... 1.000880e+00 1.000880e+00 1.000880e+00 1.000880e+00 1.000880e+00 1.000880e+00 1.000880e+00 1.000880e+00 1.000880e+00 1.000880e+00
min -2.029648e+00 -2.229249e+00 -1.984504e+00 -1.454443e+00 -3.112085e+00 -1.610136e+00 -1.114873e+00 -1.261820e+00 -2.744117e+00 -1.819865e+00 ... -1.726901e+00 -2.223994e+00 -1.693361e+00 -1.222423e+00 -2.682695e+00 -1.443878e+00 -1.305831e+00 -1.745063e+00 -2.160960e+00 -1.601839e+00
25% -6.893853e-01 -7.259631e-01 -6.919555e-01 -6.671955e-01 -7.109628e-01 -7.470860e-01 -7.437479e-01 -7.379438e-01 -7.032397e-01 -7.226392e-01 ... -6.749213e-01 -7.486293e-01 -6.895783e-01 -6.421359e-01 -6.912304e-01 -6.810833e-01 -7.565142e-01 -7.563999e-01 -6.418637e-01 -6.919118e-01
50% -2.150816e-01 -1.046362e-01 -2.359800e-01 -2.951869e-01 -3.489108e-02 -2.219405e-01 -3.422399e-01 -3.977212e-01 -7.162650e-02 -1.782793e-01 ... -2.690395e-01 -4.351564e-02 -2.859802e-01 -3.411812e-01 -4.684277e-02 -2.695009e-01 -2.182321e-01 -2.234689e-01 -1.274095e-01 -2.164441e-01
75% 4.693926e-01 5.841756e-01 4.996769e-01 3.635073e-01 6.361990e-01 4.938569e-01 5.260619e-01 6.469351e-01 5.307792e-01 4.709834e-01 ... 5.220158e-01 6.583411e-01 5.402790e-01 3.575891e-01 5.975448e-01 5.396688e-01 5.311411e-01 7.125100e-01 4.501382e-01 4.507624e-01
max 3.971288e+00 4.651889e+00 3.976130e+00 5.250529e+00 4.770911e+00 4.568425e+00 4.243589e+00 3.927930e+00 4.484751e+00 4.910919e+00 ... 4.094189e+00 3.885905e+00 4.287337e+00 5.930172e+00 3.955374e+00 5.112877e+00 4.700669e+00 2.685877e+00 6.046041e+00 6.846856e+00

8 rows × 30 columns

Principal Component Analysis¶

The heatmap will give us some insight into the relationships between features

In [7]:
# Heatmap of correlation coefficient matrix
sns.set(style='ticks', color_codes=True)
plt.rcParams['figure.figsize'] = [20,20]
sns.heatmap(inputs_df[list(inputs_df.columns[:-1])].astype(float).corr(), linewidths=0.1, square=True, linecolor='white', annot=True, cmap="coolwarm",
           cbar_kws={'shrink': 0.6})
plt.show()