It can be very tricky evaluating how good a General Manager has been, there are so many factors (payroll, luck, etc.) that come into play that can alter his perception between good and bad. I imagine evaluating something like a scouting director can be even harder since so many more things can get in the way (GM decisions, injury, luck, poor development, etc.). While this might not be the correct way to evaluate such a position, I have tried to do so by comparing drafting results from over the years.
I have compared Brian Sabean's 4 year tenure, Bill Livesey's 5 years, the span of 9 years that apparently had no scouting director employed and finally Damon Oppenheimer's so far 7 year tenure. Obviously it might be too early to compare Oppenheimer's true successes and failures but by the time we have adequate time to judge him it will already be too late. So the criteria i used when searching through draft results on Baseball-Reference.com was that the player had to have played at least 5 years and I have included those with a positive career WAR.
Obviously it is not the Scouting Director's fault who is actually signed or not or who is traded or not but I do give them credit for scouting them and deciding to draft them in the first place because i imagine each draft slot is a grueling decision making process that must give them some kind of credit. I also wonder if there is some kind of talent surge in the last few years that would possibly skew the results of older drafts (does anyone have any proof of this?). Anyway, without further ado, here are my findings, maybe they might not be as conclusive as I hoped but it was fun and i found a lot of interesting names that i didn't expect to see.
Update: I just realized that this might not be the best way to fairly compare them. Instead I'm going to compare results 7 years after they started since players with long careers skew the numbers and we haven't had the time to see Oppenheimer's results
Brian Sabean 1987-1990
1987- Gerald Williams 6.2 WAR, Brad Ausmus (Expansion Draft) 17.2 WAR
1989- JT Snow 10 WAR, Sterling Hitchcock 7.7 WAR
Laughable Mentions: Dave Eiland, Andy Fox (remember him?)
Total WAR: 168.1
Bill Livesey 1991-1995
1991- Brien Taylor (what could have been), Lyle Mouton 1.5 WAR
1993- NO ONE
1994- Jason Grilli 1.7 WAR (Did Not Sign)
Laughable Mentions: Chad Moeller!!
Total WAR: 119.2
The Lost Years 1996-2004
1998- Mark Prior 13.1 (Did Not Sign)
2000- Chris Dickerson 2.6 WAR (better late than never)
2001- Shelley Duncan 1.6 WAR (love you)
2002- Phil Coke 1.3 WAR
Total WAR: 64.8 WAR
Obviously it gets a little tricky for Oppenheimer since we're just seeing the bulk of his contributions see the light of day for the first time, though he's already had successes and failures for us to look at. Hopefully some of these players stick around, but these are the players that are most exciting
Damon Oppenheimer 2005-present
2008- Gerrit Cole (Did Not Sign), Brett Marshall, DJ Mitchell, Ray Kruml, David Phelps,
2010- Cito Culver, Angelo Gumbs, Mason Williams
2011- Dante Bichette
Total WAR: 46.8 WAR
So what can you take out of this? Obviously there are many factors that go into their positives and negatives, but I wanted to put this information together and see what there is to look at. It's very interesting stuff and obviously since I'm not that old, I don't have a lot of knowledge of prospects that just never worked out. Perhaps some of you guys can fill us in.
I'd like to think that the organization is in a better place with Oppenheimer here, but maybe i'm just playing favorites, though a 46.8 WAR already is very promising. I hope he stays, but If he does end up going to the Angels I think one thing is very clear: we need a leader to take over for him and hopefully Cashman has the sense to see that.
Update: Here are the WAR totals for each Scouting Director eras 7 years after starting
Brian Sabean 7 Years Later WAR: .4 WAR out of 19 players in 4 drafts
Bill Livesey 7 Years Later WAR: 8.5 WAR out of 7 players in 5 drafts
No One 7 Years Later WAR: 15.7 WAR out of 19 players 7 drafts
Damon Oppenheimer 7 Years Later WAR: 37.8 WAR out of 19 players 3 drafts
This obviously shows that Oppenheimer has seen more success with less drafts (no one from 2008 has MLB experience yet) so his stats are after only 3 drafts! I also hope to figure out how good the draft class was throughout each era. It's a lot of WAR to calculate so it might take a little while.