by Kevin Pelton, 3-28-03
I won’t defend these too much, but I like the results they produce, so I’m willing to accept a little cognitive dissonance.
Offensive Rebounds - .75 (my thinking here is that if you ignore the player in question, the chance of getting an offensive board is about 25%, so the player making an offensive board gives his team an extra .75 possessions over expectation)
Defensive Rebound - .25 (of course, if you follow that thinking, the weight for defensive boards should be more like .4; as I said, I like the results)
Assists - .5 (there’s probably nothing harder in offensive basketball stats than determining the value that should be placed on an assist; 0.5 gave good results *shurg*)
Steals - 1 (no thinking here; obviously a steal is a full possession)
Blocks - .33 (my thinking was that about 1/3 of blocked shots went out-of-bounds and the other 2/3 went evenly to the offense and the defense, meaning the defense gains on average 1/3 of a possession per block; that’s probably too low in hindsight, but as I’ve said, I like the results. I’ve never bought into the whole ‘intimidation’ theory of placing higher values on blocks.)
So together I call this ‘good stuff’. I don’t know why, I just do. Let’s follow a sample player, Reggie Evans, through my Sonics spreadsheet. As of 3-28-03, Evans has 134 offensive rebounds, 221 defensive rebounds, 27 assists, 27 steals, and nine blocks.
134*.75 = 100.5
221*.25 = 55.25
27 * .5 = 13.5
27 * 1 = 27
9 * .33 = 3
So Evans’ ‘good stuff’ total is 199.25.
I designed this formula to work sorta like EqA (Equivalent Average) in baseball, in terms of taking into account a more complete picture while retaining the same form and range as a stat more limited in its scope, in this case field-goal %. We combine the good stuff with scoring efficiency to come up with overall efficiency, found by the following formula:
EFF = (Points + Good Stuff) / (1.5 * (FGA + (.44*FTA) + TO + (Good Stuff/2) + (Min/4)))
Continuing with Evans as our example, he has the following figures:
199.25 ‘good stuff’
EFF = (156 + 199.25) / (1.5 * (100 + (.44*125) + 41 + (199.25/2) + (1069/4)))
EFF = (355.25) / (844.313)
EFF = 0.421
As you can tell because of the relationship to field-goal percentage, Evans’ efficiency is not particularly high. It ranks only 12th on the Sonics (including players they’ve traded).
Efficiency works well for evaluating players when minutes played aren’t important. In particular, I think a good application is to see whether playing time is being distributed correctly on a team. It’s also a good way to identify breakout candidates, players with high efficiency ratings but low minutes who will get more minutes because of a change in their surroundings (see MacCulloch, Todd, at the start of the 2001-02 season).
Many ratings systems (John Hollinger’s PER, for example) would stop there. They only evaluate players on a per-minute basis. Personally, I find this a bit silly. To be blatantly unfair to per-minute systems, looking only at efficiency I would have to conclude that Vince Carter and Paul Pierce have had equally good seasons. Their efficiencies as of press time are separated by only one point (.527 for Carter; .528 for Pierce). Yet obviously Pierce has been much better – he’s played two and a half times as many minutes as Carter.
One of the best ways of illustrating this point I can think of comes from Tony LaRussa baseball. The game was almost entirely stat-based, so to keep users from exploiting players who had only had a handful of at-bats the previous season and been really successful (in theory, that good performance would be duplicated over, say, 600 at-bats), they programmed in injuries. If the player started to play too many games or have too many at-bats, they would get injured. Again, the player was tremendously successful on a per-at-bat basis, but not all that valuable overall because of his limited use.
Value, then, is a function of two things – quantity of play (minutes) and quality of play (efficiency). At the most simplistic level, which I simply call value, we simply multiply minutes by efficiency. We can also look at value on a per-game basis, which tends to look fairly similar to – but is certainly not the same as – the generic version of Tendex, which the NBA calls its ‘exclusive’ efficiency rating. It more or less follows the same distribution as points per game, though while efficiency generally has a wider distribution than FG%, Val/G has a narrower distribution than points per game.
Back to our old friend Reggie, he’s got an efficiency of .421 and has played 1069 minutes over 56 games.
Value = eff * min = 1069 * .421 = 450.
Val/G = value/games = 450/56 = 8.0.
The Final Frontier – VORP
Value is a step forward, but still not complete. The real issue is that the differences in efficiencies aren’t large enough to produce the differences in value we really desire. Minutes tend to play too large a role in determining value, efficiency too small a role. For example, Shaquille O’Neal’s efficiency this year is a league-best .588 (actually, Atlanta’s Chris Crawford has a slightly better mark, but in an insufficient sample size – just 38 minutes). Evans’, as we know, is .421. O’Neal, then, is apparently only 40% better than Evans. That’s certainly too low of a figure in my mind.
The other problem with value is that you can rack up a nice-looking number even if you’re completely ineffective. True value comes not merely from quality and quantity, but from quality above a certain replacement level and quantity.
The answer is to look not at simply value, but Value Over Replacement Player – how much better a player is than the kind of player that could be signed as a free agent from a minor league. This is another concept I got from baseball. It’s prevalent there, but pretty rare in NBA statistical analysis, so far as I know.
At first, I simply set an arbitrary replacement level. This quickly turned out to be a problem, however, when looking at multiple years. Efficiency levels have changed over the years, generally peaking in the early-90s and decreasing ever since (because turnovers must be part of efficiency and cannot simply be ignored as they often are in traditional linear weights systems, I can’t rate seasons before 1979-80). Eventually, my thinking came to be this: After about 10 players, each NBA team’s bench starts to get to approximately replacement level, guys that could easily be swapped out with minor-leaguers without anyone really noticing (statistically, at least).
Thus, I figured that each year’s replacement level could be found by the efficiency level of the player with the nth-highest efficiency, where n is 10 * the number of teams, or 290 in the modern NBA. (I also round to the nearest twentieth; I use .425 instead of .424 or so on).
Finding VORP, then, is only slightly more complicated than finding value:
VORP = (eff – Replacement Level) * minutes
This has produced results I am very comfortable with. Players with a VORP over 300 are stars; over 400 superstars. 200 is an above-average starter, 100 an adequate starter or solid reserve, 50 a decent rotation player, 0 acceptable as a fringe member of the rotation, and below 0 – well, that speaks for itself.
Returning to Evans, his .421 efficiency is slightly below the .425 replacement level I’ve estimated for this season.
VORP – (.421 - .425) * 1069 = -5.
This disappoints me to the degree that I’m a fan of Evans and I believe he is better than replacement level, but the fact is that he – as an undrafted free agent – exemplifies the kind of replacement-level talent the system assumes is easily acquired.
The biggest problem with VORP, like any other NBA statistical rating system, is that it generally ignores the quality of a player’s man defense. Until the NBA adapts more defensive statistics to give a broader picture of defense, all we have to rate players with are blocks and steals, which are clearly inadequate. Defensive specialists who do not generate many blocks and steals, like Bruce Bowen and Michael Curry, are always going to be dramatically underrated by the system, and this is simply something we must mentally adjust for.
I also tend to feel the system rewards players who do a lot of good and bad too much at the expense of those, like Evans, who are offensive non-factors. I don’t feel I could adjust for this without causing bigger problems to the system.
APPENDIX I - Replacement Level
One of the worst things about the VORP system is that replacement level can only be determined on a post hoc basis. I'll post a complete list of past replacment levels at some point in the future.
Year Level 1978 - .435 1979 - .445 1980 - .440 1981 - .440 1982 - .455 1983 - .440 1984 - .445 1985 - .450 1986 - .455 1987 - .455 1988 - .455 1989 - .455 1990 - .440 1991 - .450 1992 - .455 1993 - .455 1994 - .455 1995 - .455 1996 - .445 1997 - .440 1998 - .440 1999 - .425 2000 - .440 2001 - .430 2002 - .435 2003 - .430
APPENDIX II - 2002-03 VORP Leaders
Here are the NBA's top 25 in VORP for the 2002-03 season:
RK Player Team PS VORP 1. Tracy McGrady ORL SG 467 2. Kevin Garnett MIN PF 451 3. Dirk Nowitzki DAL PF 450 4. Kobe Bryant LAL SG 439 5. Shaquille O'Neal LAL C 429 6. Tim Duncan SA PF 413 7. Shawn Marion PHO SF 352 8. Ray Allen SEA SG 310 9. Paul Pierce BOS SG 306 10. Steve Nash DAL PG 306 11. Sam Cassell MIL PG 299 12. Shareef Abdur-Rahim ATL PF 298 13. Karl Malone UTA PF 297 14. Predrag Stojakovic SAC SF 297 15. Pau Gasol MEM PF 295 16. Jermaine O'Neal IND PF 294 17. Michael Redd MIL SG 289 18. Antawn Jamison GS SF 287 19. Matt Harpring UTA SF 280 20. Allen Iverson PHI SG 279 21. Jason Kidd NJ PG 279 22. Steve Francis HOU PG 275 23. Gary Payton MIL PG 260 24. Allan Houston NY SG 254 25. Stephon Marbury PHO PG 250