Most "scouting services" don't know what they're doing

[Dr_Chimera]

You know the ones I mean. Most of those that sell their product for money, especially the ones written by former HF-ers.

Writing a good scouting report is not random - there has to be some methodology to it. Most folks don't get this. I'm sick and tried of reading crappy scouting reports, written by people who think they've become experts after lurking on hf for a couple of years.

I will tell you what a good scouting report is and there are basically two kinds (it can be broken down further, of course). The worst scouting reports can be exposed for being a confused jumble of the two or in worst cases neither. Some are bad, because they take one side and do an incompetent job.

Scouting report type #1: Ontology.

Folks who see players and write up these kinds of reports believe that they can get across a lot of information about a player simply by breaking down his game into a list of traits. Skating, shooting, vision, stickhandling, etc. It's a belief that scouting is mechanistic, and so are essentially people (in this case, hockey players). This sees an athlete as an abstraction that can be exposed, dissected and studied.

Scouting report type #2: Phenomenology.

This is the most difficult kind of a scouting report to master. It can be done very badly, but in best cases extremely well and those that are really well written are fun to read and personal. The scout in this case doesn't believe that a human being is a mere collage of particulars - rather, every single trait in a player is supported by another. An athlete in this case must be assessed holistically. Not only this, but the scouting report itself has to be written in terms of models and situations. Saying that a player is a good skater with excellent quickness is insufficient, because this doesn't illustrate a situation. A situation illustrates a scenario, in which phenomena occur that can be tied in to the discussed player in which a player performs a certain way holistically in relation to the ongoing play in relation to recalled (by scout) plays. The task of the scout is to abstract the situation and assess it from a personal standpoint. "Here's what the player did, here's how he did it and here's what I think about it. Here's what I can compare it to." "I think it worked because...", "I don't think it worked because..." Often the judgement is made through sheer intuition. The clever scout will incorporate his own experiences and recollections and compare the situation to other situations that he's seen, thus clariying his opinion on the player.

This kind of scouting is "implicate" - it functions on the belief that no player can be properly analyzed or understood without implicating the viewing scout's keen knowledge of hockey. By implicating himself into the scouting report, the scout also implicates the reader to identify and engage with what is written. Insight is fluid. Insight is dialogical. It should paint a picture in such a way that everyone who is exposed to it can react and creatively link the player and situation to other players and situations.

An example of pure scouting ontology is Central Scouting. So much so that it is almost exceedingly dry. Reading a CSS report, one cannot get any kind of handle on a player, because the blurb will merely list one trait after another. The writing isn't personal, the writing barely refers to hockey beyond the mechanistic, the writing often runs out of effective words to get its point across.

The best kind of ontology in scouting is extremely in-depth. I've seen this, but mostly in respect to tennis. I haven't seen a lot of good scouting of this kind in hockey. One could write a whole page about a player's wrist shot or first step, or whatever. Often it's quite fun to read and can be rewarding. In some cases the writing will flow towards analogy and thus become both phenomenology and narratology (the latter, because of the recall of stories; eg. "player x's backhand on this particular play reminded me of the time Bobby Orr scored a wrap-around goal on a backhand").

Close to pure phenomenology of scouting I think is redline report. I haven't read anything of theirs since about 2005, but from what I recall they're low on exact detail and high on 'the situation' stuff. They'll tell you what happened, when and how and how they felt about it. They'll tell you what it may mean in the long run. This kind of writing is hard to do, but they're good at it. It's personal. It doesn't matter whether it offends. Actually, it's better when it does. Sometimes it's too short - the worst offenses is when black humour replaces true insight, thus turning the player into a stereotype to caricature (eg. the sparkplug, the idiot, the dirty rotten bastard).

The important thing is that these two services at least know the difference. Central Scouting takes their explicit approach, because they feel they have to be objective. Or at least they have to appear objective. Writing their reports must be like writing for public service. It is nuanced nothing.

NHL scouts, the ones that do the real stuff, do both. I'm sure that every team has its own methodology, but eventually it breaks down into two things. You have to establish some basics first and that means separating the strengths from the weaknesses. These are, first of foremost, mechanistic. Bad skater? You may as well stop scouting the guy.

But if you keep scouting the guy, the rest of the work involves really heavy abstract work. You have to look for situations, illustrate them for the purposes of visualizing them later. This includes countless and countless examples of how the player handles himself in numerous situations and amounts to a very personal profile. And people are complex, often in ways they don't even themselves understand. Scouts want to know these kids better than these kids know themselves. And very often this requires quite a bit of intuition. Not luck, mind you. Intuition implicates personal experience. Intuition is when you have really studied the person.

So please do yourself a favour. Don't buy cheap shit like FC or whatever other stuff that is written by children with very little ideas or insights. You will know it when you see it - just read the reports and ask yourselves if they're written by a human being who truly cares about the player, or just some guy spewing cliches that have been said countless times.

[RTWAP]

Massive rep!

Can I repost (with link) on our Sens Prospecting Blog?

[Acer]

Phenomenal post! (just to use a reference from your post.)

I'm just curious about where this (I presume) insight is coming from. Care to share details Doc?

[Mr Bugg]

Excellent post. This applies to me, as I essentially have two 'jobs'- the fantasy writing I do for Dobber, and the 'pure' game breakdowns I do for one of the services.

It's often tough to balance the two. On the one hand, my readers at Dobber really don't give a shit about surgical analysis of a player's game. Thus, I often condense that down to one-liners that either a) present a historical comparison b) break down a notable trait or two. The same holds true for projections: I could go on for a paragraph about why I feel the way I do about a player based on what I've seen him do in X situation, but it's not important. What is is whether the guy is worth selecting in a fantasy pool.

On HF, I do find myself slipping into entertainment mode more than I should. The audience there is more attentive to detail, but the responses one receives often makes one feel like it's just not worth the effort.

Here's a page right from my notebook:

4th-leading scorer in the XXXX best player on either team. Just a beautiful skater- came off the bench, went behind his own net and reached the neutral zone in a handful of seconds. Double-shifted due to the absence of forward X, so X saw a ton of ice-time. Tremendous awareness in the offensive zone. Great patience; on the powerplay, he was stationed just below the right face-off dot, and would fake shots to open up passing seams cross-ice. Rarely missed when passing the puck laterally- passes were quick and crisp. Definitely smaller than the 6'0 he's listed at- probably more 5'10- but he wasn't afraid to take- or lay- a hit to make a play. Typical shift: gain the puck in his end, gain speed through the neutral zone, and overwhelm defenders with his speed. Only one instance where he made a more complicated play than required; he typically looked pass first, and if it wasn't available, tried a low wrister five-hole. Shot not very hard, but extremely quick release. The second XXXXX goal was a result of a puck that looked dead coming into X's feet, which he snapped on net to create a rebound. It surprised everyone, including his linemates. Drew a penalty early in the third when he took a pass in his 'office', the right faceoff dot. A defender saw him all alone and jumped on him. Almost had a goal with a few seconds left; made a solo rush through two defenders, burning them with his speed and a quick inside-outside move. Made the goalie bite on a forehand fake, and went backhand but hit the bar.

Recommendation: Passed over in '08 Draft- probably due to playing on the expansion XXXXXX, who were not very good. No college commitment at this point in time. A very, very similar player to Mason Raymond, and would probably post similar numbers if he had better linemates. Is a bit on the small size and needs some muscle, but he'd be a tremendous value pick after the third round.


Okay, I lied. That's the formatted report I draw up from all of the observations I write down. If anyone wants to see those, I could post them, but they're rough.

[PredsFan77]

red line report FTW

[thecrosbyshow]

I agree, even reading through past red line's there is some great insight and pretty bang on predictions.

Also, hell of a post.

[PredsFan77]

One of the more underrated players at any position this year is Mike Green, who rose above an absolutely horrid surrounding cast in Saskatoon to earn the admiration of the scouting community. With no help he managed to post outstanding offensive numbers and refused to give anything less than his best effort at any point during an utterly lost season. No one would have blamed him for the occasional letdown, but the ultra-competitive Green never gave up in numerous lost cause games and continued battling against long odds all season. And despite not being that big, he may be the best open-ice hitter of this year's blue-line class.

[thecrosbyshow]

Sticking with the 2004 class, there were also bang on with Radulov (had him ranked 4th in 2004) and did a glowing write up on Cory Schneider (who is probably the best goalie prospect out there). They also had Meszaros in their top 10.

http://www.redlinereport.com/redlinereport_jun2004DG.html

Super Sleeper: Ondrej Sekera. He is one of Red Line's true favorites. An undersized battler with fine offensive skills, the barely 6-foot Slovak defender weighs in at only 187 pounds — but 186 of that is heart. A big man trapped in a small man's body, Sekera is the fiery, emotional leader for the Slovak Under-18 Team. He throws his body around recklessly and competes hard every shift. He has excellent offensive skills with speed, vision, passing ability and patience under pressure. He also loves to dance and create with the puck and is an effective PP quarterback. A passionate, heart and soul guy we'd love to have on our team any day.

Sekera has struggled down the stretch but is probably the best Sabre defenseman after Rivet.

Also bang on with Mason Raymond in 2005.



SLEEPERS:

Raymond is similar to Greene in that he's a late '85 birthdate who's on the small side and did not opt-in last year. But Raymond did at least get tons of icetime, albeit in the Alberta Junior Hockey League against an inferior competition level. He has great wheels and offensive skills, but the AJHL doesn't attract much scouting attention, so he hadn't gained much notoriety until opening eyes with a tremendous run in the playoffs. Don't be surprised if some club fell in love enough to select him as high as late in the second round.

- taken 51st overall (21st pick of the 2nd round) in 2005.

[Roughneck]

Use of Language: good
Word Choice: excellent
Paragraph structre: Good
Intangibles: off the charts
Overall Grade: A

Summary: A franchise post, look for more from this poster in the years to come.

[Rogers Pancreas]

Very nice insight.

[hockeysense]

Like literature, hockey is a good example of how to turn subjective/abstract material into an objective/tangible explanation. Like an essay, a scouting report is usually divided into three parts - a thesis, the explorative (explanation of thesis), and the conclusion - each part establishes a foundation of universal understanding so that words become a picture.

Part of the problem with scouting is that each scout has a different agenda, a different perception, and most importantly, a different experience for each player scouted. I'm sure the New York Rangers and Ottawa Senators have learned what it takes to scout a player efficiently, conclusively, and incisively (see: Hugh Jessiman, Alexandre Daigle). The most important tool for scouts is transposition and contrast; having the ability to compare and trace scouting profiles for each player establishes an increasingly refined art of scouting subjective material - subjective material can be anything in hockey; progression, maturity, personality, and precedent.

Somewhat unrelated; I was watching a documentary on Quantum Mechanics last night (part of my on-going research into a paper I'm writing) and was impressed by the depth of defining/calculating probability. The perception of reality is usually dissected by experience and self-awareness - turn away from the source of your distraction and it turns into a number of probabilities. For example; the methodology used in explaining this was a basketball. Turn away and there's a limitless probability of where the basketball is being bounced. When you turn away to focus on identifying where the basketball is, your perception is tunnel-visioned. Another good example was a relatively recent experiment on the inner-workings of the brain. A person was asked to focus on an apple. When they did the CAT scan, they found that certain areas of the brain lighted up. When they asked the person to close their eyes and imagine the same apple, the exact same areas of the brain lighted up. This forced scientists to back up and ask themselves, "Whoa! Is our perception of reality defined by what we see or what we've experienced?"

This is why transposition and contrast are such important tools in scouting today; particularly in long-term projections like baseball. A high school player is drafted and allowed to progress in the minors for several years before making the jump. By the time he's done, he's most likely to be around 22-24 years of age - this is a great example of how transposition and contrast in the scouting community can be an extremely powerful tool. Compared to hockey, we've had seven rookies (18 year olds) make the jump from the 2008 NHL Entry Draft to the pros. This is, perhaps, proof of hockey's increasingly refined art of scouting - we're starting to see less and less of a bust and more of a sure-thing in the players drafted. I think we'll start seeing more examples of the 2003 NHL Entry Draft in which all, but two players have managed to make the pros; and shown themselves to be exceptionally great values from where they were picked.

Thusly, NHL scouting has more of a shared community than independent scouting services like Red Line Report, International Scouting Services, etc. I don't rely much on either services to identify a talented player - I just rely on them to expose a name to watch and then come to my own conclusions of the potential impact of a player. Of course, I'm learning that having the ability to transpose and contrast my own conclusions against another's establishes a more refined observation than what I've independently perceived - I'd have a higher rate of failure due to my "tunnel-vision." It's only human.

Ps. This thread is a good example of why I'm happy with this board. Kudos to the OP! Repped!

[hockeysense]

I should add that scouts also take into account of the game's changes, tactics, and suggestively, a team's propensity for drafting certain players - most particularly dependent on management's experience, history, and style. Most of the rules go out the window during draft day.

[Dr_Chimera]

Massive rep!

Can I repost (with link) on our Sens Prospecting Blog?

Repost it any way you like.

[RTWAP]

Repost it any way you like.

Thanks.

http://sensprospects.blogspot.com/2009/04/on-scouting.html

[znk]

Do we have to pay for these posts?

[Fyodor]

Scouting report type #1: Ontology.



Scouting report type #2: Phenomenology.



I've been doing some philosophy of scouting (as a scouting report reader and philosopher :easter_egg: ) and I came to this distinction too. :drunk:

[Beaker]

Redline Report now has John Tavares ranked 3 behind Hedman and Duchene in their rankings.

Sounds to me like they are just throwing something controversial out there to get publicity, and well it made Sportscentre so there you go.

[SensGuy]

Redline Report now has John Tavares ranked 3 behind Hedman and Duchene in their rankings.

Sounds to me like they are just throwing something controversial out there to get publicity, and well it made Sportscentre so there you go.
This isn't the first time Woodlief has gone out on a limb though.

[Rogers Pancreas]

Redline Report now has John Tavares ranked 3 behind Hedman and Duchene in their rankings.

Sounds to me like they are just throwing something controversial out there to get publicity, and well it made Sportscentre so there you go.Woodlief has a tendency to do that, evidence in how low he had Cherepanov ranked last year. And yet, his logic is sound. Tavares is a one-dimensional goal-scorer at the junior level. If he struggles to put in a defensive effort every night playing for the Knights, how long will it take before he becomes the kind of all-around franchise player/leader the Islanders need if at all? It's just something to consider if you're a fan of Avalanche or Leafs.

[Beaker]

This isn't the first time Woodlief has gone out on a limb though.

I know that.... seems to me that sometimes Woodlief's reports need to be read with a grain of salt.

I really like Woodlief, he's one of my favorites, but he always has 1 or 2 guys either way too high... or way too low in the first round. Sometimes i think he goes way out on a limb with those few guys to stand out from the pack and generate some buzz on his rankings. Heck this move got him reported on both Sportscentre and TSN.ca.

It was only one week ago that Bob McKenzie had 9/10 NHL scouts preferring Tavares to Hedman and even had Hedman third in one of those scout's minds.

To see Tavares fall to third based on this playoff is an opinion that is way out there.

He proved his ability to score in the clutch at the WJC for me.

Lets face it Tavares is being targetted by every team in the OHL in both the second half and in these playoffs... he may be playing hurt and we don't know. To discount what he has done in the last 4 years, what he did all season, and what he did at the WJC 2 years in a row now, based on one good but not great playoff performance. Thats a big mistake if the Islanders are doing the same in my book.

[dempsey_k]

I always knew this Chimera guy had the pizzazz to be one of the best Broads.

[RTWAP]

I know that.... seems to me that sometimes Woodlief's reports need to be read with a grain of salt.

I really like Woodlief, he's one of my favorites, but he always has 1 or 2 guys either way too high... or way too low in the first round. Sometimes i think he goes way out on a limb with those few guys to stand out from the pack and generate some buzz on his rankings. Heck this move got him reported on both Sportscentre and TSN.ca.

It was only one week ago that Bob McKenzie had 9/10 NHL scouts preferring Tavares to Hedman and even had Hedman third in one of those scout's minds.

To see Tavares fall to third based on this playoff is an opinion that is way out there.

He proved his ability to score in the clutch at the WJC for me.

Lets face it Tavares is being targetted by every team in the OHL in both the second half and in these playoffs... he may be playing hurt and we don't know. To discount what he has done in the last 4 years, what he did all season, and what he did at the WJC 2 years in a row now, based on one good but not great playoff performance. Thats a big mistake if the Islanders are doing the same in my book.

They interviewed him this afternoon on Team1200. I missed most of it but they did ask him to briefly describe OEL, based on rumours that Ottawa might be interested. He avoided the question with a definitive statement that he would be gone before Ottawa picked, unless they traded up.

[Egil]

I know that.... seems to me that sometimes Woodlief's reports need to be read with a grain of salt.

I really like Woodlief, he's one of my favorites, but he always has 1 or 2 guys either way too high... or way too low in the first round. Sometimes i think he goes way out on a limb with those few guys to stand out from the pack and generate some buzz on his rankings. Heck this move got him reported on both Sportscentre and TSN.ca.

It was only one week ago that Bob McKenzie had 9/10 NHL scouts preferring Tavares to Hedman and even had Hedman third in one of those scout's minds.

To see Tavares fall to third based on this playoff is an opinion that is way out there.

He proved his ability to score in the clutch at the WJC for me.

Lets face it Tavares is being targetted by every team in the OHL in both the second half and in these playoffs... he may be playing hurt and we don't know. To discount what he has done in the last 4 years, what he did all season, and what he did at the WJC 2 years in a row now, based on one good but not great playoff performance. Thats a big mistake if the Islanders are doing the same in my book.

But shouldn't this be normal? I always found these consensus draft picks hard to stomach when if we did the same thing with NHL players the lists would be all over the map even starting with #1. So why should the evaluations of less scouted, not fully grown, inmature be so consistent?

[Nalyd Psycho]

It was only one week ago that Bob McKenzie had 9/10 NHL scouts preferring Tavares to Hedman and even had Hedman third in one of those scout's minds.

To see Tavares fall to third based on this playoff is an opinion that is way out there.


But is he saying Tavares will go 3rd? Or that he'll be the 3rd best player?

Clearly Hedman is the only player that could possibly be taken above him, and even that is unlikely. But could Duchesne be better? Possibly.

[Beaker]

But is he saying Tavares will go 3rd? Or that he'll be the 3rd best player?

Clearly Hedman is the only player that could possibly be taken above him, and even that is unlikely. But could Duchesne be better? Possibly.

I believe he's saying that if he was the one in charge of making the pick, he'd take Duchene over Tavares.

[Dr Love]

But shouldn't this be normal? I always found these consensus draft picks hard to stomach when if we did the same thing with NHL players the lists would be all over the map even starting with #1. So why should the evaluations of less scouted, not fully grown, inmature be so consistent?

Excellent point, and one that I agree with, but there can be drafts where there's a consensus #1. For example Eric Lindros was truly a man among boys when he was drafted. But with a list of current players, you've got quite a few that are on that truly elite level, so which one you would start with is a matter of preference. In an amateur draft however, there can be one player who is head and shoulders above the rest.

[Nalyd Psycho]

I believe he's saying that if he was the one in charge of making the pick, he'd take Duchene over Tavares.

Which is a fair point. Duchene does have less holes in his game, even if he is clearly an inferior scorer.

[pitseleh]

As has been said, Woodlief likes to throw controversy into the mix and is sometimes right and sometimes wrong. Keep in mind that he had Andre Kostitsyn ahead of Eric Staal and Thomas Vanek in 2003, Guillaume Latendresse ahead of Bobby Ryan in 2005, JP Levasseur ahead of Carey Price in 2005, etc. I like the fact that he doesn't feel the need to follow the crowd but he's been wrong on a lot of his bold predictions too.

[Egil]

Excellent point, and one that I agree with, but there can be drafts where there's a consensus #1. For example Eric Lindros was truly a man among boys when he was drafted. But with a list of current players, you've got quite a few that are on that truly elite level, so which one you would start with is a matter of preference. In an amateur draft however, there can be one player who is head and shoulders above the rest.

I agree that #1/2 can have a reasonable consensus, but even then, someone disagrees. For example, I would be shocked if there wasn't a few scouts who didn't have Daigle #1, and he was a "consensus" #1. But certainly passed the first few picks, there is no way that team, or individual lists should be similar.

[Beaker]

I agree that #1/2 can have a reasonable consensus, but even then, someone disagrees. For example, I would be shocked if there wasn't a few scouts who didn't have Daigle #1, and he was a "consensus" #1. But certainly passed the first few picks, there is no way that team, or individual lists should be similar.

They will be similar cause there are usually tiers of players....

Ie in this draft, i see the tiers like this.

Tier 1: 1-2 picks... Hedman and Tavares
Tier 2: 3-7 picks... Duchene, MSP, Schenn, Kane, Cowen
Tier 3: 8-12.... OEL, Kulikov, Schroeder, etc...

Now within a tier guys will move up or down a few spots... and the tiers will get wider as you get further down the list... but IMO not many will have Tavares outside the top 2, and not many will have MSP outside the top 7... and not many will have Schroeder higher than 8 or lower than 12. There is some debate within a tier, but guys don't usually break into the next tier or fall to the one below it, and when someone does, that is where the biggest draft surprises are for me.

When a guy like Hickey breaks out and jumps up two tiers, or a guy like Wheeler jumps three/four tiers, or a guy like Cherapanov falls 2-3 tiers... it makes for an interesting draft.

[Dr Love]

I agree that #1/2 can have a reasonable consensus, but even then, someone disagrees. For example, I would be shocked if there wasn't a few scouts who didn't have Daigle #1, and he was a "consensus" #1. But certainly passed the first few picks, there is no way that team, or individual lists should be similar.

Absolutely, which is one reason the draft is something people chose to watch on TV. If everyone agreed on the first round, it would be pretty boring.

[Roughneck]

Absolutely, which is one reason the draft is something people chose to watch on TV. If everyone agreed on the first round, it would be pretty boring.

I'd love to see the various draft boards for the 2003 draft. No consensus #1, there was hardly a consensus top 5. From a Flames perspective I remember that if Phaneuf was taken Mike Richards was the next player on Sutter's list. I doubt you'd see many matches even with positional rankings all the teams had. Carter, Getzlaf, Parise, Richards, who was the top ranked centre? Suter, Coburn, Phaneuf, everybody had them as the consensus 3 top defensemen, but none would have matched up. etc.

[Chomsky]

As has been said, Woodlief likes to throw controversy into the mix and is sometimes right and sometimes wrong. Keep in mind that he had Andre Kostitsyn ahead of Eric Staal and Thomas Vanek in 2003, Guillaume Latendresse ahead of Bobby Ryan in 2005, JP Levasseur ahead of Carey Price in 2005, etc. I like the fact that he doesn't feel the need to follow the crowd but he's been wrong on a lot of his bold predictions too.

Goddamnit I wish that would have happened... :beat:

[Acer]

http://rapidshare.com/files/232263708/FC_2008_Draft_Guide_Sample_Pack.pdf

Just posting this as evidence for the some points made by Doc in the first post.

[edgar_box]

Love a good tirade.

[Beaker]

Love a good tirade.

Brownman is Alive.... I was worried they put you on global ignore again.

[NewsGuyOne]

Sounds like example one is done by scouts.
And example two by blowhards who think they're special.

I wrote a few things for HF. I covered a u-18 tournament with a ton of good young players. And boy, is Niklas Hjalmarsson making me feel like I should be an NHL scout.
COME ON JACK SKILLE!!!!! You can do it!
A lot of the guys that impressed me have gone onto to become solid young NHLers.
But there were a few guys I really praised who disappeared (Freddie Pettersson, Robert Nyholm).

Still hoping Justin Mercier can come through and become an NHLer.

Ah well, maybe I didn't do so hot.