Sturgeon's Law And The Recommendation Engine

      A modern truism holds the "The Internet drives us apart" or "Social medial encourages extremism."  I think it's at least partially true -- but why does this happen, and how?

      It may be the intersection of two things: recommendation engines and Sturgeon's Law.

      Recommendation engines?  The do great things when you're looking for a film or TV show to stream -- Netflix or Amazon Prime Video or whoever has been keeping track of what you've been watching, and provides a whole category of "things you might like."  The more you watch, the better those suggestions match your tastes. 

      Hooray, right?

      Well--  Musical acts (for instance) get sorted pretty severely.  To get onto the machine's to-be-recommended to people who like X, Y and/or Z list, they've got to be objectively good.  Untalented acts never break that threshold.  Unskilled musicians never break the threshold.

      That "threshold" is where Sturgeon's Law comes in.  The law itself cautions that "90% of everything is crap."  One of the origin tales has Theodore Sturgeon on a panel of judges reading short stories submitted for publication at a science fiction convention; the slush piles are high and the material is, well, not so great.  One of the judges sets down an especially bad example and says, "Most of these stories are crap!"

     Sturgeon agrees, "Sure.  But ninety percent of everything is crap."

      That's one version.  James Gunn remembers something both kinder and more pointed.

      Music is pretty well sorted for quality.  Films and movies, for me it's close but not great; there's a lot of chaff to sift through.  Book recommendations from the smart software are even more hit-or-miss.

      But head off in a less mass-audience direction and things get strange fast -- and that's a problem.  My Hidden Frontier stories rely on playing fast and loose with history; the FTL drive is independently discovered at least three times and stolen twice, and some of that happens during WW II and just after.  It only takes one video or web page about "WW II flying saucers" or the post-war Byrd Antarctic Expedition to end up with some very strange stuff coming up next, entirely ahistorical and often pushing offensive political ideas or worse.  And letting those play just points you at even weirder and crazier stuff. When people say, "It must be true!  I researched it on the Internet," that's the kind of "research" you can end up with: 90% of those recommendations point to utter crap.

      The software can only work with what's there.  Flood the topic with crazy, conspiracy-theory stuff and made-up revisionist history, and where do the recommendation engines aim you?  Yep -- right at it. Get two layers in and you're in Crazytown.

      Be mindful of it. It's a bad neighborhood.
____________________________
* As an experiment, I started with "Six Underground," got Zero7 after that, then a Chet Faker video I'd never heard but enjoyed (not what it first appears -- there's narrative depth there, also considerable degree-of-difficulty points), followed by Talking Heads "Psycho Killer" and White Stripes "Seven Nation Army."  The Rolling Stones are up next.  It's pretty good guessing.