Explaining VORP

by Kevin Pelton, 3-28-03
(Updated 4-30-03)

Linear Weights
I won’t defend these too much, but I like the results they produce, so I’m willing to accept a little cognitive dissonance.

Offensive Rebounds - .75 (my thinking here is that if you ignore the player in question, the chance of getting an offensive board is about 25%, so the player making an offensive board gives his team an extra .75 possessions over expectation)
Defensive Rebound - .25 (of course, if you follow that thinking, the weight for defensive boards should be more like .4; as I said, I like the results)
Assists - .5 (there’s probably nothing harder in offensive basketball stats than determining the value that should be placed on an assist; 0.5 gave good results *shurg*)
Steals - 1 (no thinking here; obviously a steal is a full possession)
Blocks - .33 (my thinking was that about 1/3 of blocked shots went out-of-bounds and the other 2/3 went evenly to the offense and the defense, meaning the defense gains on average 1/3 of a possession per block; that’s probably too low in hindsight, but as I’ve said, I like the results. I’ve never bought into the whole ‘intimidation’ theory of placing higher values on blocks.)

So together I call this ‘good stuff’. I don’t know why, I just do. Let’s follow a sample player, Reggie Evans, through my Sonics spreadsheet. As of 3-28-03, Evans has 134 offensive rebounds, 221 defensive rebounds, 27 assists, 27 steals, and nine blocks.

134*.75 = 100.5
221*.25 = 55.25
27 * .5 = 13.5
27 * 1 = 27
9 * .33 = 3

So Evans’ ‘good stuff’ total is 199.25.

Efficiency
I designed this formula to work sorta like EqA (Equivalent Average) in baseball, in terms of taking into account a more complete picture while retaining the same form and range as a stat more limited in its scope, in this case field-goal %. We combine the good stuff with scoring efficiency to come up with overall efficiency, found by the following formula:

EFF = (Points + Good Stuff) / (1.5 * (FGA + (.44*FTA) + TO + (Good Stuff/2) + (Min/4)))

Continuing with Evans as our example, he has the following figures:

156 points
199.25 ‘good stuff’
100 FGA
125 FTA
41 TO
1069 Min

EFF = (156 + 199.25) / (1.5 * (100 + (.44*125) + 41 + (199.25/2) + (1069/4)))
EFF = (355.25) / (844.313)
EFF = 0.421

As you can tell because of the relationship to field-goal percentage, Evans’ efficiency is not particularly high. It ranks only 12th on the Sonics (including players they’ve traded).

Efficiency works well for evaluating players when minutes played aren’t important. In particular, I think a good application is to see whether playing time is being distributed correctly on a team. It’s also a good way to identify breakout candidates, players with high efficiency ratings but low minutes who will get more minutes because of a change in their surroundings (see MacCulloch, Todd, at the start of the 2001-02 season).

Value
Many ratings systems (John Hollinger’s PER, for example) would stop there. They only evaluate players on a per-minute basis. Personally, I find this a bit silly. To be blatantly unfair to per-minute systems, looking only at efficiency I would have to conclude that Vince Carter and Paul Pierce have had equally good seasons. Their efficiencies as of press time are separated by only one point (.527 for Carter; .528 for Pierce). Yet obviously Pierce has been much better – he’s played two and a half times as many minutes as Carter.

One of the best ways of illustrating this point I can think of comes from Tony LaRussa baseball. The game was almost entirely stat-based, so to keep users from exploiting players who had only had a handful of at-bats the previous season and been really successful (in theory, that good performance would be duplicated over, say, 600 at-bats), they programmed in injuries. If the player started to play too many games or have too many at-bats, they would get injured. Again, the player was tremendously successful on a per-at-bat basis, but not all that valuable overall because of his limited use.

Value, then, is a function of two things – quantity of play (minutes) and quality of play (efficiency). At the most simplistic level, which I simply call value, we simply multiply minutes by efficiency. We can also look at value on a per-game basis, which tends to look fairly similar to – but is certainly not the same as – the generic version of Tendex, which the NBA calls its ‘exclusive’ efficiency rating. It more or less follows the same distribution as points per game, though while efficiency generally has a wider distribution than FG%, Val/G has a narrower distribution than points per game.

Back to our old friend Reggie, he’s got an efficiency of .421 and has played 1069 minutes over 56 games.

Value = eff * min = 1069 * .421 = 450.
Val/G = value/games = 450/56 = 8.0.

The Final Frontier – VORP
Value is a step forward, but still not complete. The real issue is that the differences in efficiencies aren’t large enough to produce the differences in value we really desire. Minutes tend to play too large a role in determining value, efficiency too small a role. For example, Shaquille O’Neal’s efficiency this year is a league-best .588 (actually, Atlanta’s Chris Crawford has a slightly better mark, but in an insufficient sample size – just 38 minutes). Evans’, as we know, is .421. O’Neal, then, is apparently only 40% better than Evans. That’s certainly too low of a figure in my mind.

The other problem with value is that you can rack up a nice-looking number even if you’re completely ineffective. True value comes not merely from quality and quantity, but from quality above a certain replacement level and quantity.

The answer is to look not at simply value, but Value Over Replacement Player – how much better a player is than the kind of player that could be signed as a free agent from a minor league. This is another concept I got from baseball. It’s prevalent there, but pretty rare in NBA statistical analysis, so far as I know.

At first, I simply set an arbitrary replacement level. This quickly turned out to be a problem, however, when looking at multiple years. Efficiency levels have changed over the years, generally peaking in the early-90s and decreasing ever since (because turnovers must be part of efficiency and cannot simply be ignored as they often are in traditional linear weights systems, I can’t rate seasons before 1979-80). Eventually, my thinking came to be this: After about 10 players, each NBA team’s bench starts to get to approximately replacement level, guys that could easily be swapped out with minor-leaguers without anyone really noticing (statistically, at least).

Thus, I figured that each year’s replacement level could be found by the efficiency level of the player with the nth-highest efficiency, where n is 10 * the number of teams, or 290 in the modern NBA. (I also round to the nearest twentieth; I use .425 instead of .424 or so on).

Finding VORP, then, is only slightly more complicated than finding value:

VORP = (eff – Replacement Level) * minutes

This has produced results I am very comfortable with. Players with a VORP over 300 are stars; over 400 superstars. 200 is an above-average starter, 100 an adequate starter or solid reserve, 50 a decent rotation player, 0 acceptable as a fringe member of the rotation, and below 0 – well, that speaks for itself.

Returning to Evans, his .421 efficiency is slightly below the .425 replacement level I’ve estimated for this season.

VORP – (.421 - .425) * 1069 = -5.

This disappoints me to the degree that I’m a fan of Evans and I believe he is better than replacement level, but the fact is that he – as an undrafted free agent – exemplifies the kind of replacement-level talent the system assumes is easily acquired.

Problems
The biggest problem with VORP, like any other NBA statistical rating system, is that it generally ignores the quality of a player’s man defense. Until the NBA adapts more defensive statistics to give a broader picture of defense, all we have to rate players with are blocks and steals, which are clearly inadequate. Defensive specialists who do not generate many blocks and steals, like Bruce Bowen and Michael Curry, are always going to be dramatically underrated by the system, and this is simply something we must mentally adjust for.

I also tend to feel the system rewards players who do a lot of good and bad too much at the expense of those, like Evans, who are offensive non-factors. I don’t feel I could adjust for this without causing bigger problems to the system.

APPENDIX I - Replacement Level
One of the worst things about the VORP system is that replacement level can only be determined on a post hoc basis. I'll post a complete list of past replacment levels at some point in the future.

Year      Level
1978 - .435
1979 - .445
1980 - .440
1981 - .440
1982 - .455
1983 - .440
1984 - .445
1985 - .450
1986 - .455
1987 - .455
1988 - .455
1989 - .455
1990 - .440
1991 - .450
1992 - .455
1993 - .455
1994 - .455
1995 - .455
1996 - .445
1997 - .440
1998 - .440
1999 - .425
2000 - .440
2001 - .430
2002 - .435
2003 - .430

APPENDIX II - 2002-03 VORP Leaders
Here are the NBA's top 25 in VORP for the 2002-03 season:

RK  Player              Team PS  VORP
 1. Tracy McGrady       ORL  SG  467
 2. Kevin Garnett       MIN  PF  451
 3. Dirk Nowitzki       DAL  PF  450
 4. Kobe Bryant         LAL  SG  439
 5. Shaquille O'Neal    LAL   C  429
 6. Tim Duncan          SA   PF  413
 7. Shawn Marion        PHO  SF  352
 8. Ray Allen           SEA  SG  310
 9. Paul Pierce         BOS  SG  306
10. Steve Nash          DAL  PG  306
11. Sam Cassell         MIL  PG  299
12. Shareef Abdur-Rahim ATL  PF  298
13. Karl Malone         UTA  PF  297
14. Predrag Stojakovic  SAC  SF  297
15. Pau Gasol           MEM  PF  295
16. Jermaine O'Neal     IND  PF  294
17. Michael Redd        MIL  SG  289
18. Antawn Jamison      GS   SF  287
19. Matt Harpring       UTA  SF  280
20. Allen Iverson       PHI  SG  279
21. Jason Kidd          NJ   PG  279
22. Steve Francis       HOU  PG  275
23. Gary Payton         MIL  PG  260
24. Allan Houston       NY   SG  254
25. Stephon Marbury     PHO  PG  250