Investigate a Dataset

Baseball All-Star Data

Justin Ibanez

Introduction

For my project I decided to investigate the baseball data. The question I asked was, "How are the all-stars different from the entire player base?" I looked at two characteristics to investigate this: birth country (or state for American born players) and handedness for batting and throwing.

Data Wrangling

The data for this project was taken from SeanLahman.com. The CSVs investigated were AllstarFull.csv, Master.csv and Appearances.csv. I loaded the CSVs into dataframes using Pandas and printed their heads to get an idea of their structure.

In [1]:
import pandas as pd

# read CSVs and print their heads
allstar_df = pd.read_csv('AllstarFull.csv')
master_df = pd.read_csv('Master.csv')
appearances_df = pd.read_csv('Appearances.csv')

allstar_df.head()
Out[1]:
playerID yearID gameNum gameID teamID lgID GP startingPos
0 gomezle01 1933 0 ALS193307060 NYA AL 1.0 1.0
1 ferreri01 1933 0 ALS193307060 BOS AL 1.0 2.0
2 gehrilo01 1933 0 ALS193307060 NYA AL 1.0 3.0
3 gehrich01 1933 0 ALS193307060 DET AL 1.0 4.0
4 dykesji01 1933 0 ALS193307060 CHA AL 1.0 5.0
In [2]:
master_df.head()
Out[2]:
playerID birthYear birthMonth birthDay birthCountry birthState birthCity deathYear deathMonth deathDay ... nameLast nameGiven weight height bats throws debut finalGame retroID bbrefID
0 aardsda01 1981.0 12.0 27.0 USA CO Denver NaN NaN NaN ... Aardsma David Allan 220.0 75.0 R R 2004-04-06 2015-08-23 aardd001 aardsda01
1 aaronha01 1934.0 2.0 5.0 USA AL Mobile NaN NaN NaN ... Aaron Henry Louis 180.0 72.0 R R 1954-04-13 1976-10-03 aaroh101 aaronha01
2 aaronto01 1939.0 8.0 5.0 USA AL Mobile 1984.0 8.0 16.0 ... Aaron Tommie Lee 190.0 75.0 R R 1962-04-10 1971-09-26 aarot101 aaronto01
3 aasedo01 1954.0 9.0 8.0 USA CA Orange NaN NaN NaN ... Aase Donald William 190.0 75.0 R R 1977-07-26 1990-10-03 aased001 aasedo01
4 abadan01 1972.0 8.0 25.0 USA FL Palm Beach NaN NaN NaN ... Abad Fausto Andres 184.0 73.0 L L 2001-09-10 2006-04-13 abada001 abadan01

5 rows × 24 columns

In [3]:
appearances_df.head()
Out[3]:
yearID teamID lgID playerID G_all GS G_batting G_defense G_p G_c ... G_2b G_3b G_ss G_lf G_cf G_rf G_of G_dh G_ph G_pr
0 1871 BS1 NaN barnero01 31.0 NaN 31 31.0 0 0 ... 16 0 15 0 0 0 0 NaN NaN NaN
1 1871 BS1 NaN barrofr01 18.0 NaN 18 18.0 0 0 ... 1 0 0 13 0 4 17 NaN NaN NaN
2 1871 BS1 NaN birdsda01 29.0 NaN 29 29.0 0 7 ... 0 0 0 0 0 27 27 NaN NaN NaN
3 1871 BS1 NaN conefr01 19.0 NaN 19 19.0 0 0 ... 0 0 0 18 0 1 18 NaN NaN NaN
4 1871 BS1 NaN gouldch01 31.0 NaN 31 31.0 0 0 ... 0 0 0 0 0 1 1 NaN NaN NaN

5 rows × 21 columns

I started by looking for missing values starting with the all-stars dataframe.

In [4]:
# list number of missing values in allstar_df
allstar_df.isnull().sum()
Out[4]:
playerID          0
yearID            0
gameNum           0
gameID           49
teamID            0
lgID              0
GP               19
startingPos    3489
dtype: int64

The gameID, GP and startingPos columns had missing values. The documentation included with this data said that startingPos would only have a value if the player was a starter, so it makes sense for there to be so many missing values. There was nothing explaining the other two columns, so I investigated them further.

In [5]:
# find entries with missing gameID
allstar_df.loc[pd.isnull(allstar_df['gameID'])]
Out[5]:
playerID yearID gameNum gameID teamID lgID GP startingPos
573 borowha01 1945 0 NaN NYA AL 0.0 NaN
574 boudrlo01 1945 0 NaN CLE AL 0.0 NaN
575 casege01 1945 0 NaN WS1 AL 0.0 NaN
576 chrisru01 1945 0 NaN PHA AL 0.0 NaN
577 cuccito01 1945 0 NaN CHA AL 0.0 NaN
578 ettenni01 1945 0 NaN NYA AL 0.0 NaN
579 ferreri01 1945 0 NaN WS1 AL 0.0 NaN
580 ferrida01 1945 0 NaN BOS AL 0.0 NaN
581 greenha01 1945 0 NaN DET AL 0.0 NaN
582 grimeos01 1945 0 NaN NYA AL 0.0 NaN
583 gromest01 1945 0 NaN CLE AL 0.0 NaN
584 hayesfr01 1945 0 NaN CLE AL 0.0 NaN
585 johnsbo01 1945 0 NaN BOS AL 0.0 NaN
586 krameja01 1945 0 NaN SLA AL 0.0 NaN
587 leeth01 1945 0 NaN CHA AL 0.0 NaN
588 leonadu02 1945 0 NaN WS1 AL 0.0 NaN
589 mayoed01 1945 0 NaN DET AL 0.0 NaN
590 mcquige02 1945 0 NaN SLA AL 0.0 NaN
591 moseswa01 1945 0 NaN CHA AL 0.0 NaN
592 newhoha01 1945 0 NaN DET AL 0.0 NaN
593 reynoal01 1945 0 NaN CLE AL 0.0 NaN
594 stephve01 1945 0 NaN SLA AL 0.0 NaN
595 stirnsn01 1945 0 NaN NYA AL 0.0 NaN
596 treshmi01 1945 0 NaN CHA AL 0.0 NaN
597 barrere01 1945 0 NaN SLN NL 0.0 NaN
598 cavarph01 1945 0 NaN CHN NL 0.0 NaN
599 coopemo01 1945 0 NaN BSN NL 0.0 NaN
600 elliobo01 1945 0 NaN PIT NL 0.0 NaN
601 greggha01 1945 0 NaN BRO NL 0.0 NaN
602 hackst01 1945 0 NaN CHN NL 0.0 NaN
603 holmeto01 1945 0 NaN BSN NL 0.0 NaN
604 johnsdo02 1945 0 NaN CHN NL 0.0 NaN
605 kurowwh01 1945 0 NaN SLN NL 0.0 NaN
606 lombaer01 1945 0 NaN NY1 NL 0.0 NaN
607 marioma01 1945 0 NaN SLN NL 0.0 NaN
608 masiph01 1945 0 NaN BSN NL 0.0 NaN
609 mccorfr01 1945 0 NaN CIN NL 0.0 NaN
610 mungova01 1945 0 NaN NY1 NL 0.0 NaN
611 nichobi01 1945 0 NaN CHN NL 0.0 NaN
612 odeake01 1945 0 NaN SLN NL 0.0 NaN
613 ottme01 1945 0 NaN NY1 NL 0.0 NaN
614 pafkoan01 1945 0 NaN CHN NL 0.0 NaN
615 passecl01 1945 0 NaN CHN NL 0.0 NaN
616 roepr01 1945 0 NaN PIT NL 0.0 NaN
617 rosengo01 1945 0 NaN BRO NL 0.0 NaN
618 sewelri01 1945 0 NaN PIT NL 0.0 NaN
619 verbaem01 1945 0 NaN SLN NL 0.0 NaN
620 walkedi02 1945 0 NaN BRO NL 0.0 NaN
621 wyseha01 1945 0 NaN CHN NL 0.0 NaN

All these entries were for 1945 all-star game. This game was cancelled due to World War II, which explains the missing values. I decided to keep these entries as these players were still chosen to be all-stars even if they did not play the game.

In [6]:
# find entries with missing GP
allstar_df.loc[pd.isnull(allstar_df['GP'])]
Out[6]:
playerID yearID gameNum gameID teamID lgID GP startingPos
5050 santihe01 2015 0 NLS201507141 LAA AL NaN NaN
5051 grayso01 2015 0 NLS201507142 OAK AL NaN NaN
5052 bautijo02 2015 0 NLS201507143 TOR AL NaN NaN
5053 millesh01 2015 0 NLS201507144 ATL NL NaN NaN
5054 martica04 2015 0 NLS201507145 SLN NL NaN NaN
5055 wachami01 2015 0 NLS201507146 SLN NL NaN NaN
5056 rosentr01 2015 0 NLS201507147 SLN NL NaN NaN
5057 hollima01 2015 0 NLS201507148 SLN NL NaN NaN
5058 gordode01 2015 0 NLS201507149 MIA NL NaN NaN
5059 stantmi03 2015 0 NLS201507150 MIA NL NaN NaN
5060 scherma01 2015 0 NLS201507151 WAS NL NaN NaN
5061 odayda01 2015 0 NLS201507152 BAL AL NaN NaN
5062 papeljo01 2015 0 NLS201507153 PHI NL NaN NaN
5063 burneaj01 2015 0 NLS201507154 PIT NL NaN NaN
5064 boxbebr01 2015 0 NLS201507155 TBA AL NaN NaN
5065 gordoal01 2015 0 NLS201507156 KCA AL NaN NaN
5066 herreke01 2015 0 NLS201507157 KCA AL NaN NaN
5067 cabremi01 2015 0 NLS201507158 DET AL NaN NaN
5068 salech01 2015 0 NLS201507159 CHA AL NaN NaN

These entries corresponded to the 2015 all-star game. After doing a little research it seemed like these entries corresponded to players who were selected as all-stars, but did not appear in the game for some reason (such as injuries). I was unable to find a reason for every player (nor was every player who was replaced due to injury on the above list), so I was not completely confident in this explanation.

It was also noteworthy that these entries were the last 18 entries in the dataframe although I could not find a reason for the missing values from that angle.

I decided to keep these entries as all the players are listed on the roster,.

I moved on to the master dataframe.

In [7]:
# list number of missing values in master_df
master_df.isnull().sum()
Out[7]:
playerID           0
birthYear        143
birthMonth       315
birthDay         464
birthCountry      73
birthState       626
birthCity        199
deathYear       9510
deathMonth      9511
deathDay        9512
deathCountry    9517
deathState      9569
deathCity       9521
nameFirst         39
nameLast           0
nameGiven         39
weight           871
height           805
bats            1191
throws           978
debut            193
finalGame        193
retroID           54
bbrefID            1
dtype: int64

Only playerID did not have missing values. Since this data goes back to 1820 the missing data was likely due to the information simply being unavailable. Furthermore, there would be no death time or place information for players who were still alive, which explains why there are significantly more missing values in those columns than the others. There was not much I could do to remedy the missing data, so I decided to continue using it as is.

The variables I were investigating had a relatively small amount of missing data points, but they would be affected by the missing data nonetheless. The most significant was the batting handedness where about 6% of the data was missing. It could also be possible for the missing birth country data to significantly affect the results if those players came from countries with low player contributions.

Finally, I looked for missing data in the appearances dataframe.

In [8]:
# list number of missing values in appearances_df
appearances_df.isnull().sum()
Out[8]:
yearID           0
teamID           0
lgID           737
playerID         0
G_all          203
GS           51921
G_batting        0
G_defense      203
G_p              0
G_c              0
G_1b             0
G_2b             0
G_3b             0
G_ss             0
G_lf             0
G_cf             0
G_rf             0
G_of             0
G_dh         51718
G_ph         51718
G_pr         58006
dtype: int64

The columns lgID (league), G_all (total games played), GS (games started), G_defense (games in which the player appeared on defense), G_dh (games as designated pitcher), G_ph (games as pinch hitter) and G_pr (games as pinch runner) had missing data. Since G_defense, G_dh, G_ph and G_pr referred to positions, it made sense for there to be missing values as not every player would play in those positions in every season. Furthermore, GS would have missing data if a player did not start that season. However, lgID and G_all did not have a clear reason for missing data and needed to be investigated further. I started with with the league information.

In [9]:
# find entries with missing lgID
appearances_df.loc[pd.isnull(appearances_df['lgID'])].head()
Out[9]:
yearID teamID lgID playerID G_all GS G_batting G_defense G_p G_c ... G_2b G_3b G_ss G_lf G_cf G_rf G_of G_dh G_ph G_pr
0 1871 BS1 NaN barnero01 31.0 NaN 31 31.0 0 0 ... 16 0 15 0 0 0 0 NaN NaN NaN
1 1871 BS1 NaN barrofr01 18.0 NaN 18 18.0 0 0 ... 1 0 0 13 0 4 17 NaN NaN NaN
2 1871 BS1 NaN birdsda01 29.0 NaN 29 29.0 0 7 ... 0 0 0 0 0 27 27 NaN NaN NaN
3 1871 BS1 NaN conefr01 19.0 NaN 19 19.0 0 0 ... 0 0 0 18 0 1 18 NaN NaN NaN
4 1871 BS1 NaN gouldch01 31.0 NaN 31 31.0 0 0 ... 0 0 0 0 0 1 1 NaN NaN NaN

5 rows × 21 columns

In [10]:
appearances_df.loc[pd.isnull(appearances_df['lgID'])].tail()
Out[10]:
yearID teamID lgID playerID G_all GS G_batting G_defense G_p G_c ... G_2b G_3b G_ss G_lf G_cf G_rf G_of G_dh G_ph G_pr
732 1875 WS6 NaN stearbi01 21.0 NaN 21 21.0 17 0 ... 0 0 0 2 5 0 7 NaN NaN NaN
733 1875 WS6 NaN stevero01 1.0 NaN 1 1.0 0 0 ... 0 0 0 0 0 1 1 NaN NaN NaN
734 1875 WS6 NaN terry01 6.0 NaN 6 6.0 0 0 ... 0 0 0 0 1 2 3 NaN NaN NaN
735 1875 WS6 NaN thompan01 11.0 NaN 11 11.0 0 11 ... 0 0 0 0 0 1 1 NaN NaN NaN
736 1875 WS6 NaN withech01 1.0 NaN 1 1.0 1 0 ... 0 0 0 0 0 0 0 NaN NaN NaN

5 rows × 21 columns

These entries were from the years 1871 to 1875. Since the American League (the older of the two leagues that comprises the MLB) was founded in 1876, any entries prior to 1876 would not have a league ID. I looked at the games played information next.

In [11]:
# find entries with missing G_all
appearances_df.loc[pd.isnull(appearances_df['G_all'])].head()
Out[11]:
yearID teamID lgID playerID G_all GS G_batting G_defense G_p G_c ... G_2b G_3b G_ss G_lf G_cf G_rf G_of G_dh G_ph G_pr
96757 2013 ARI NL kenneia01 NaN NaN 22 NaN 21 0 ... 0 0 0 0 0 0 0 0.0 1.0 0.0
96758 2013 ARI NL kubelja01 NaN NaN 89 NaN 0 0 ... 0 0 0 53 0 5 57 3.0 30.0 0.0
96759 2013 ARI NL langwma01 NaN NaN 8 NaN 8 0 ... 0 0 0 0 0 0 0 0.0 0.0 0.0
96772 2013 ARI NL reynoma01 NaN NaN 30 NaN 30 0 ... 0 0 0 0 0 0 0 0.0 0.0 0.0
96779 2013 ARI NL thatcjo01 NaN NaN 22 NaN 22 0 ... 0 0 0 0 0 0 0 0.0 0.0 0.0

5 rows × 21 columns

All of these entries were from the 2013 season, but I could not find a reason why this data was missing. I was using the appearances data to help collect information on how handedness changed over time, so none of this missing data affected by analysis. I decided to ignore it.

I then cleaned up the inconsistencies in the country names in the master dataframe.

In [12]:
# make all country names consistent
def country_corrector(country):
    if country == 'CAN':
        return 'Canada'
    elif country == 'D.R.':
        return 'Dominican Republic'
    elif country == 'P.R.':
        return 'Puerto Rico'
    elif country == 'V.I.':
        return 'US Virgin Islands'
    return country

master_df['birthCountry'] = master_df['birthCountry'].apply(country_corrector)

The all-stars dataframe contained an entry for every appearance a player made in an all-star game. This means the same player will have multiple entries if he played in more than one all-star game in his career. I used the all-stars dataframe to create an array of all-star players and then used this list to extract the all-stars from the master dataframe.

In [13]:
# all-stars array
unique_allstars = allstar_df['playerID'].unique()

# all-star players
allstars = master_df.loc[master_df['playerID'].isin(unique_allstars)]

I noticed a discrepancy in this data. The unique_allstars array had 1741 elements. However, the allstars dataframe had 1740 rows. I found the data point in unique_allstars, but not in allstars.

In [14]:
# discrepancy
print len(unique_allstars) # 1741
print allstars.shape[0] # 1740

# find discrepancy
for player in unique_allstars:
    if player not in allstars['playerID'].tolist():
        print player # carperma01
        
# print discrepancy
allstar_df.loc[allstar_df['playerID'] == 'carperma01']
1741
1740
carperma01
Out[14]:
playerID yearID gameNum gameID teamID lgID GP startingPos
4965 carperma01 2014 0 ALS201407150 SLN NL 0.0 NaN

It was not immediately clear to me what was wrong with this data point. After a bit of research I found the answer. All the player IDs were 9 characters long and 'carperma01' contained 10 characters. This was due to a typo as 'carperma01' should have been 'carpema01'. This data point refered to Matt Carpenter's second all-star selection. I corrected this discrepancy and recreated unique_allstars and allstars.

In [15]:
# correct discrepancy
allstar_df.set_value(4965, 'playerID', 'carpema01')

#recreate array and dataframe
unique_allstars = allstar_df['playerID'].unique()
allstars = master_df.loc[master_df['playerID'].isin(unique_allstars)]

# double check that values match
print len(unique_allstars) # 1740
print allstars.shape[0] # 1740
1740
1740

With the discrepancy cleared I organized the player data. Since America had a much larger number of players than any other country they would have dominated most measurements. Therefore, I decided to look at them separately. I wanted to create a dataframe containing all-star and player populations for American and non-American players.

I separated the all-stars and players (from the master dataframe) into American and non-American born players. I grouped each of the resultant dataframes by birth state/country and counted to get the all-star and player populations for each state/country. After renaming some columns for clarity, I concatenated these columns to get my desired dataframes. I also computed the percentage of all-stars in each dataframe for future analysis.

In [16]:
# divide all-stars and players into American and non-American
american_allstars = allstars.loc[allstars['birthCountry'] == 'USA']
non_american_allstars = allstars.loc[allstars['birthCountry'] != 'USA']
american_players = master_df.loc[master_df['birthCountry'] == 'USA']
non_american_players = master_df.loc[master_df['birthCountry'] != 'USA']

# group by state/country
american_allstars_by_state = american_allstars.groupby('birthState').count()
non_american_allstars_by_country = non_american_allstars.groupby('birthCountry').count()
american_players_by_state = american_players.groupby('birthState').count()
non_american_players_by_country = non_american_players.groupby('birthCountry').count()

# clean up column names to prepare for concatenation
american_allstars_by_state['allStars'] = american_allstars_by_state['playerID']
non_american_allstars_by_country['allStars'] = non_american_allstars_by_country['playerID']
american_players_by_state['players'] = american_players_by_state['playerID']
non_american_players_by_country['players'] = non_american_players_by_country['playerID']

# concatenate
american_players_by_state = pd.concat(
    [american_players_by_state['players'], american_allstars_by_state['allStars']], axis=1)
non_american_players_by_country = pd.concat(
    [non_american_players_by_country['players'], non_american_allstars_by_country['allStars']], axis=1)

# replace NaN with 0
american_players_by_state.fillna(0, inplace=True)
non_american_players_by_country.fillna(0, inplace=True)

# add percentAllStars column
american_players_by_state['percentAllStar'] = (american_players_by_state['allStars']
                                               / american_players_by_state['players']) * 100.00
non_american_players_by_country['percentAllStar'] = (non_american_players_by_country['allStars']
                                                     / non_american_players_by_country['players']) * 100.00

# print final dataframes
american_players_by_state.head()
Out[16]:
players allStars percentAllStar
birthState
AK 11 2 18.181818
AL 322 43 13.354037
AR 152 25 16.447368
AZ 99 7 7.070707
CA 2160 295 13.657407
In [17]:
non_american_players_by_country.head()
Out[17]:
players allStars percentAllStar
Afghanistan 1 0.0 0.000000
American Samoa 1 0.0 0.000000
Aruba 5 0.0 0.000000
At Sea 1 0.0 0.000000
Australia 28 2.0 7.142857

The next step was to organize the data for handedness. I started by adding the handedness data from the master dataframe to the all-stars and appearances dataframes. I limited the player database to the same years as the all-stars.

In [18]:
# handedness of all players
handedness = master_df[['playerID', 'bats', 'throws']]

# add handedness to all-stars
allstar_handedness = allstar_df.merge(handedness, on='playerID', how='left')[['playerID', 'yearID', 'bats', 'throws']]

# print result
allstar_handedness.head()
Out[18]:
playerID yearID bats throws
0 gomezle01 1933 L L
1 ferreri01 1933 R R
2 gehrilo01 1933 L L
3 gehrich01 1933 L R
4 dykesji01 1933 R R
In [19]:
# add handedness to player appearances
player_handedness = appearances_df.merge(handedness, on='playerID', how='left')[['playerID', 'yearID', 'bats', 'throws']]
player_handedness = player_handedness.loc[player_handedness['yearID'] >= 1933]

# print result
player_handedness.head()
Out[19]:
playerID yearID bats throws
24990 alexada01 1933 R R
24991 almadme01 1933 L L
24992 andreiv01 1933 R R
24993 brownll01 1933 L L
24994 cookedu01 1933 L R

I grouped the all-stars and players by year and used the value_counts method (as a proportion) to create four series containing different types of handedness: all-star batting, all-star throwing, player batting and player throwing.

In [20]:
# all-star batting
allstar_bat_handedness_by_year = allstar_handedness.groupby('yearID')['bats'].value_counts(normalize=True)
allstar_bat_handedness_by_year = allstar_bat_handedness_by_year.fillna(0)

# all-star throwing
allstar_throw_handedness_by_year = allstar_handedness.groupby('yearID')['throws'].value_counts(normalize=True)
allstar_throw_handedness_by_year = allstar_throw_handedness_by_year.fillna(0)

# player batting
player_bat_handedness_by_year = player_handedness.groupby('yearID')['bats'].value_counts(normalize=True)
player_bat_handedness_by_year = player_bat_handedness_by_year.fillna(0)

# player throwing
player_throw_handedness_by_year = player_handedness.groupby('yearID')['throws'].value_counts(normalize=True)
player_throw_handedness_by_year = player_throw_handedness_by_year.fillna(0)

Exploration

I started by looking at the birth states/countries. I began by plotting the raw data. I decided to use bar graphs, because I was comparing the different amounts of all-stars and the difference in height would easily provide a visual indicator for these differences.

In [21]:
%pylab inline

import matplotlib.pyplot as plt
import seaborn as sns
Populating the interactive namespace from numpy and matplotlib
In [22]:
master_df['birthCountry'].value_counts().plot(kind='bar', figsize=[15,14])
plt.title('Raw Birth Country Statistics')
plt.xlabel('Country')
plt.ylabel('Number of Players')
Out[22]:
<matplotlib.text.Text at 0xcae5f28>

USA completely dominated every other country, which justified my choice in looking at it separately. I plotted the raw birth state data next.

In [23]:
master_df['birthState'].value_counts().plot(kind='barh', figsize=[15,45])
plt.title('Raw Birth State Statistics')
plt.xlabel('Number of Players')
plt.ylabel('State')
Out[23]:
<matplotlib.text.Text at 0xe871898>

Again, American states dominated the data. Furthermore, the sheer amount of states in the master dataframe also justified my choice to group them by the more general category of countries. In particular, most countries contributed players from only a few states within them, so it made sense not to look at them separately. Although the argument could be made to give the other countries with relatively large amounts of player contribution (such as Canada and the Dominican Republic) the same treatment as USA.

I moved on to the American all-stars and their state of birth.

In [24]:
american_players_by_state.describe()
Out[24]:
players allStars percentAllStar
count 51.000000 51.000000 51.000000
mean 323.607843 28.980392 9.294504
std 415.579070 44.908124 4.281587
min 11.000000 1.000000 1.282051
25% 66.000000 5.000000 6.416137
50% 200.000000 17.000000 8.205128
75% 358.500000 35.500000 11.889098
max 2160.000000 295.000000 20.689655

From the description of the data there was an average of about 323 players per state. About 29 or 9.3% of those players were all-stars. There was a large difference between the minimum and maximum values and the 75th percentile was very small compared to the maximum.

In [25]:
american_players_by_state['allStars'].plot(kind='barh', figsize=[15,14])
plt.title('American All-Stars by Birth State', fontsize=18)
plt.xlabel('Number of All-Stars', fontsize=14)
plt.ylabel('Birth State', fontsize=14)
Out[25]:
<matplotlib.text.Text at 0xeeef208>

What stuck out (literally) was that California had many more all-stars than the other states (about three times more than the state with the second most all-stars), which explained the statistics noted above. This could be attributed to their large population, so I also plotted the percentage of all-stars as well to reduce the effect of a large population.

In [26]:
american_players_by_state['percentAllStar'].plot(kind='barh', figsize=[15,14])
plt.title('American All-Stars by Birth State - Percentage', fontsize=16)
plt.xlabel('Percent of Player Base that are All-Stars', fontsize=14)
plt.ylabel('Birth State', fontsize=14)
Out[26]:
<matplotlib.text.Text at 0x10c9fdd8>

This plot looked signficantly different from the previous one. California no longer had the largest value and in general the states' values were a lot closer together. The top three states were Idaho, Wyoming and Alaska. Maine had a low value in both plots.

While looking at the percentage did prevent large populations from dominating the plot, it also had the effect of overvaluing all-stars in states with smaller populations. A state with a small amount of players (such as Idaho, Wyoming and Alaska) only needed a small amount of all-stars to achieve a large percentage. However, there was no way of knowing if these all-stars were outliers and not indicative of the general player base. Therefore, in order to mitigate this effect I removed all states with less than 25 all-stars and plotted the percentages again.

In [27]:
to_plot = american_players_by_state.loc[american_allstars_by_state['allStars'] >= 25]
to_plot['percentAllStar'].plot(kind='barh', figsize=[15,14])
plt.title('American All-Stars by Birth State - Percentage', fontsize=16)
plt.xlabel('Percent of Player Base that are All-Stars', fontsize=14)
plt.ylabel('Birth State', fontsize=14)
Out[27]:
<matplotlib.text.Text at 0x11f01eb8>

Only 21 states remained after imposing this condition. And the percentages of these remaing states were relatively close together compared to the previous plots. The state with the largest percentage here was Arkansas even though it had a relatively low amount of all-stars. This implied that Arkansas born players are disproportionately more likely to become all-stars. New York was the state with the third most all-stars, but it had a relatively low percentage. This implied that the New York born players are disproportionately less likely to become all-stars. The same could also be said for Pennsylvania. However, the relative closeness of the percentages means that birth states did not drastically affect a player's chance at being an all-star. That is to say, California contributes more all-stars due to their larger population and not due to Californians being inherently good at baseball.

Finally, I plotted the number of players per birth state to look at this from another angle.

In [28]:
american_players_by_state['players'].plot(kind='barh', figsize=[15,14])
plt.title('American Players by Birth State', fontsize=18)
plt.xlabel('Number of Players', fontsize=14)
plt.ylabel('Birth State', fontsize=14)
Out[28]:
<matplotlib.text.Text at 0x121a9e48>

Once again, California dominates the plot. Overall, the shape of this plot mimicked the shape of the all-stars plot. States with large amounts of all-stars also had large amounts of players and vice versa. The differences between the proportions were more easily seen in the percentages plot, but in general the all-star plot was a microcosm of the players plot. This reinforced my inital conclusion that birth state does not dratstically affect all-star likelihood.

I moved on to the rest of the countries next starting with a description of the data.

In [29]:
non_american_players_by_country.describe()
Out[29]:
players allStars percentAllStar
count 51.000000 51.000000 51.000000
mean 44.490196 5.137255 7.498157
std 111.524593 14.143577 17.637614
min 1.000000 0.000000 0.000000
25% 1.000000 0.000000 0.000000
50% 4.000000 0.000000 0.000000
75% 17.000000 2.000000 10.427807
max 643.000000 79.000000 100.000000

The other countries contributed an average of about 44 players with an average of 5 all-stars. However, the average percentage of all-stars contributed was 7.5% (although 5/44 is about 11%). This implied a heavily skewed distribution. This is further supported by the 75th percentile for the players and all-stars being extremely low compared to the maximum value.

It would turn out that some countries contributed one player who would also go on to become an all-star leading to a 100% all-star rate.

I plotted the all-stars for each country (for countries with at least one all-star).

In [30]:
to_plot = non_american_players_by_country.loc[non_american_players_by_country['allStars'] > 0]
to_plot['allStars'].plot(kind='barh', figsize=[15,14])
plt.title('Non-American All-Stars by Birth Country', fontsize=18)
plt.xlabel('Number of All-Stars', fontsize=14)
plt.ylabel('Birth Country', fontsize=14)
Out[30]:
<matplotlib.text.Text at 0x126f5048>

Several countried clearly dominated this plot. Again, I decided to look at percentages. This time I limited the results to countries with at least 10 all-stars (many countries had percentages over 50 without this limitation).

In [31]:
to_plot = non_american_players_by_country.loc[non_american_players_by_country['allStars'] > 10]
to_plot['percentAllStar'].plot(kind='barh', figsize=[15,14])
plt.title('Non-American All-Stars by Birth Country - Percentage', fontsize=18)
plt.xlabel('Percentage of Player Base that are All-Stars', fontsize=14)
plt.ylabel('Birth Country', fontsize=14)
Out[31]:
<matplotlib.text.Text at 0x12857eb8>

Many of the countries were eliminated using this criteria and the countries that remained had percentages close to each other despite contributing largey disparate amounts of all-stars. This could be attributed to the data coming from American baseball leagues and so it made sense that foreigners would only compete if they had exceptional skill. A number of countries that contributed a very small number of players (such as two to five) would often have most of those players become all-stars.

Finally, I plotted the players per country.

In [32]:
non_american_players_by_country['players'].plot(kind='barh', figsize=[15,14])
plt.title('Non-American Players by Birth Country', fontsize=18)
plt.xlabel('Number of Players', fontsize=14)
plt.ylabel('Birth Country', fontsize=14)
Out[32]:
<matplotlib.text.Text at 0x12918278>

This plot was consistent with the all-stars plot moreso than the American data. The countries with high amounts of all-stars also had high amounts of players and the countries that contributed few or no all-stars also contributed few players.

I looked at the handedness data next. I started by plotting the raw handedness data from the master database.

In [33]:
master_df['bats'].value_counts().plot(kind='bar')
plt.title('Raw Batting Statistics')
plt.xlabel('Handedness')
plt.ylabel('Number of Players')
Out[33]:
<matplotlib.text.Text at 0x16449470>
In [34]:
master_df['throws'].value_counts().plot(kind='bar')
plt.title('Raw Throwing Statistics')
plt.xlabel('Handedness')
plt.ylabel('Number of Players')
Out[34]:
<matplotlib.text.Text at 0x16507080>

The main distinction between the plots was that players were ambidextrous in batting, but not in throwing. This suggested that players were more versatile at batting. There were more right-handed than left-handed players as well, which was expected.

I moved on to the wrangled data to see how the handedness changed over time.

In [35]:
allstar_bat_handedness_by_year.unstack(level=1).describe()
C:\Users\justin\Anaconda2\lib\site-packages\numpy\lib\function_base.py:3834: RuntimeWarning: Invalid value encountered in percentile
  RuntimeWarning)
Out[35]:
bats B L R
count 81.000000 83.000000 83.000000
mean 0.077125 0.294898 0.629836
std 0.037360 0.048871 0.045201
min 0.018868 0.189655 0.529412
25% NaN 0.264428 0.599065
50% NaN 0.289474 0.633803
75% NaN 0.327200 0.660189
max 0.145161 0.396226 0.740741

The all-star batting handedness could be broken down to about 63% right-handed, 30% left-handed and 7% ambidextrous. The values had a relatively large range. There was a little as 52% right-handed all-stars at one point, which was surprising given the raw data above.

I plotted the batting data for all-stars and players. Since I was dealing with a change over time I decided to use a line graph.

In [36]:
allstar_bat_handedness_by_year.unstack(level=1).plot(figsize=[10,8])
plt.title('All-Star Batting Handedness')
plt.xlabel('Year')
plt.ylabel('Percentage of Players')
Out[36]:
<matplotlib.text.Text at 0x16852f28>
In [37]:
player_bat_handedness_by_year.unstack(level=1).describe()
Out[37]:
bats B L R
count 83.000000 83.000000 83.000000
mean 0.069939 0.286516 0.643545
std 0.026782 0.014535 0.023129
min 0.028571 0.253397 0.599034
25% 0.041688 0.276125 0.627119
50% 0.074521 0.284424 0.644461
75% 0.091829 0.294534 0.658926
max 0.109576 0.323993 0.705036

The averages for the player base were similar to the all-stars, but it seemed the data did not vary as much.

In [38]:
player_bat_handedness_by_year.unstack(level=1).plot(figsize=[10,8])
plt.title('Player Batting Handedness')
plt.xlabel('Year')
plt.ylabel('Percentage of Players')
Out[38]:
<matplotlib.text.Text at 0x16772d30>

What stood out to me was that the all-star plot was much more sporadic than the general player base, which supported the conjectures made above. I suspected that the general player base handedness would be steady due to the population not changing much from year to year. However, the sporadic nature of the all-stars implied that this was not the case for them. This could mean it was hard for someone to beomce an all-star multiple years in a row. There was also a slight increase in the ambidextrous players as time went on, which could mean the player base was becoming more versatile over time.

In [39]:
allstar_throw_handedness_by_year.unstack(level=1).describe()
Out[39]:
throws L R
count 83.000000 83.000000
mean 0.184127 0.815873
std 0.040392 0.040392
min 0.075472 0.721311
25% 0.164057 0.796610
50% 0.180328 0.819672
75% 0.203390 0.835943
max 0.278689 0.924528

About 20% of players were left-handed and 80% were right-handed when it came to throwing. These numbers varied by about 20% over the years, which further indicated the all-star handedness was more sporadic than the general player base.

In [40]:
allstar_throw_handedness_by_year.unstack(level=1).plot(figsize=[10,8])
plt.title('All-Star Throwing Handedness')
plt.xlabel('Year')
plt.ylabel('Percentage of Players')
Out[40]:
<matplotlib.text.Text at 0x16a1ad68>
In [41]:
player_throw_handedness_by_year.unstack(level=1).describe()
Out[41]:
throws L R
count 83.000000 83.000000
mean 0.202861 0.797139
std 0.019360 0.019360
min 0.157168 0.759553
25% 0.189421 0.781822
50% 0.204904 0.795096
75% 0.218178 0.810579
max 0.240447 0.842832

The averages for the player base were similar to the all-stars. However, the varied by half as much as the all-stars, which further supported the trend of more stable statistics for the players compared to the all-stars.

In [42]:
player_throw_handedness_by_year.unstack(level=1).plot(figsize=[10,8])
plt.title('Player Throwing Handedness')
plt.xlabel('Year')
plt.ylabel('Percentage of Players')
Out[42]:
<matplotlib.text.Text at 0x1810aa58>

Again, the sporadic nature of the all-stars relative to the player base further supported that it was hard for all-stars to have repeat showings. Furthermore, the difference in proportions was much greater for throwing than for batting. This was more proof that the players were better at batting than throwing. Overall, the plots reflected the conjectures made on the data's descriptions.

Conclusions

It is important to note that any relations implied by my analysis are not necessarily causal. There are many factors that could affect the results presented below. This project was focused on comparing all-stars to the general player base. However, all-stars are chosen by people whether it be the players, managers or fans rather than a competition or a player's stats. This means any trends in the all-stars' performance could be coincedental. For example, a popular player with average skill might have a better chance of becoming an all-star than a skilled, but unpopular player. Furthermore, being chosen to be an all-star once could affect your future performance. For example, maybe a player picked to be an all-star is more motivated and as a result performs better. Nevertheless, these are the conclusions I arrived to.

  • American all-star distribution by birth state was a microcosm of player distribution by birth state. States with a large number of players had a proportionately large number of all-stars and thus birth state does not drastically affect all-star likelihood.
  • Non-American countries sent their best players to compete in America, so they had a disproportionate amount of all-stars. By that I mean countries that contributed a small number of players would have most of those players become all-stars.
  • Players are more versatile at throwing than batting and their throwing versatility is increasing.
  • It was hard to become an all-star multiple years in a row

Acknowledgements

I would like to thank Caroline Buckey for creating the course on the data analyst process as well as numpy and Pandas, Mat Leonard for creating the course on Anaconda and Jupyter Notebook and Sean Lahman for providing the data.