As pointed out by TechDirt’s Mike Masnick, last week Netflix launched a two-part blog post on its recommendations system, one of which was interesting if only for what it revealed about the outcome of this supposed winning formula which it coughed up big bucks for. Indeed, by the time the algorithm was good to go, Netflix as a business had moved on.
Whilst they acknowledge that the work that went into the final product was immense, Xavier Amatriain and Justin Basilico, Personalization Science and Engineering at Netflix, say that the engineering effort required to achieve the accuracy gains they measured, wasn’t entirely justified.
“This is a truly impressive compilation and culmination of years of work, blending hundreds of predictive models to finally cross the finish line,” they say. “We evaluated some of the new methods offline but the additional accuracy gains that we measured did not seem to justify the engineering effort needed to bring them into a production environment. Also, our focus on improving Netflix personalization had shifted to the next level by then.”
Basically, streaming changed the way its members interacted with Netflix, as well as the type of data available in its algorithms. “For DVDs our goal is to help people fill their queue with titles to receive in the mail over the coming days and weeks; selection is distant in time from viewing, people select carefully because exchanging a DVD for another takes more than a day, and we get no feedback during viewing,” they say. “For streaming members who are looking for something great to watch right now; they can sample a few videos before settling on one, they can consume several in one session, and we can observe viewing statistics such as whether a video was watched fully or only partially.”
