Friday, November 24, 2023

A Line is Not a Line: An Analysis of Chessable Course Sizes

Chessable is the current market-leader in digital opening repertoires, and it markets its courses on the number of "trainable lines."  This roughly equals one variation, so 500 trainable lines should mean 500 variations.  Looking at this, you would think that a 250 line course would be half the size ... and you would be wrong.  You see, a line is not always a line.

Like apples and oranges, you cannot easily compare different lines from different courses.  Some authors, like my fav GM Shankland, tend to use many short lines; others use fewer but longer lines.  Multiple this across several chapters and the difference adds up.

In a recent article, I argued that smaller courses are superior to larger courses for most students.  This discrepancy with line sizes, though, poses a problem, as it is not easy to tell if a given course is big or small based on line size alone.  Fortunately, I can offer a solution.

Because I'm slightly insane, I've spent a long time transcribing courses from Chessable into my own personal databases.  As a result, I can calculate the exact number of unique moves inside a given repertoire.  That is, five lines of two moves is the same as one line of 10 moves.  It's the same workload, just spread out differently. 

I have included my own notes and have extended or shortened some variations, so these numbers aren't exact, but they are very close and fine for our general purposes. I have also converted all the annoying "clickables", so this shows the "true" size of these courses.

Inside, I will compare several different repertoires and show just how much similar-sized courses can differ in their moves.

The Complete List

To start, here's a list of courses I have transcribed.  "Moves" are the unique positions within the course; "lines" are the Chessable trainable lines; and "ratio" is the average number of unique moves per line.  This is sorted by moves, so the largest courses are at the top.  Click to enlarge.

I have combined courses together where appropriate, so Shankland's entire 1.d4 trilogy is contained in a single line item.  Same for GM Jones' KID and Sethu's 1.e4 courses.  Obviously, this is not a complete list of the Chessable catalogue, but it's still sizeable and contains many popular courses.  (I apologize to IM Krykun and IM Sielecki fans, as I haven't enjoyed their work and thus haven't transcribed much of it.)

What do we see?  The ratio column really highlights the difference.  A green ratio indicates a smaller workload per variation than a similar course, whereas a red one shows the opposite.  We have an immediate outlier: Colovic's "Simplified KID" has twice the ratio of any other course.  It has the least number of lines but a large chunk of moves.  Anyone who bought the course expecting something simple would have been surely disappointed (ahem, me, for example).

It's such an outlier that I'll repost the list but with it removed:

This shows it much better.  Let's focus on the good green first: Sam's 1.d4 Trilogy.  Despite having the second-most lines, it is 8th in total moves.  And because I've worked with this repertoire so much, adding my own notes and extra lines, that number is larger than the published amount.  That means Sam's course rocks in with the lowest ratio, making it the easiest course to study.

If we sort by ratio instead of moves, we get this.  See if you can find a pattern:

There's a reason I adore Shankland as an author, and it's on full display here: in my sample, he has five of the seven courses with the best moves to lines ratio.  This means his courses are less work that other similar-sized courses, at least by line ratio.  One concrete example, his Semi-slav course has 669 lines, and the Fierce Nimzo course has 525 lines.  Despite this, the Semi-Slav has nearly 400 fewer moves.  You can actually learn Sam's course faster!

Sam's most recent course, on the Neo-Catalan, has a much higher ratio, which largely reflects Chessable's changing publishing standards.  I don't have any ready data on that, but I'll leave a postscript at the end of this post explaining my thoughts.

At the other end of the extreme we have Sethu.  In terms of raw moves, his 1...e5 course is the largest single course, and it's not even close.  All of his courses have a high ratio, and unsurprisingly, many people have said his courses are had to learn, with me included.  I tried, but I gave up.  Part of it comes down to his explanations not being sufficient, but another part is the sheer size.  It's a mammoth workload.

Of course, while all of this interesting, it becomes most enlightening when we start combining courses together. After all, we don't just play openings, we play repertoires.  How do they stack up?

Workloads of Some Sample Repertoires

Let's say you love the King's Indian and you love GM Gawain Jones.  His courses cover a complete Black repertoire against 1.e4 and 1.d4, and total 18k moves.  That's the largest possible workload among my sample.  The lowest?  Yuri Krykun's 1...e5 course combined with Colovic's Simplified KID, amounting to a mere 3k moves.

The difference, 15k, is huge.  Of course, thing's aren't so simple.  Jones's massive courses nevertheless have fairly good ratio numbers, wheres Yuri's sit closer to the bottom (and Colovic is off the chart bad).  It will thus take more effort to learn those 3k moves than it would to learn 3k moves from Jones.

Or how about this: Shankland's original repertoire series (the 1.d4 Trilogy, the Semi-Slav and the Classic Sicilian) comes in at 8,892 moves.  Sethu's 1...e5 course, which only cover one Black response, has 8,310 moves.  Sam's five courses, offering a full repertoire, are essentially the same size as Sethu's one!

You might think that Sethu's course is more complete.  You would be wrong.  Sethu's course is more bloated.  It contains over 100 moves on the position after 1.e4 e5 2.Ne2?!.  Do you know how many moves you need against 2.Ne2?  Zero.  That's where you follow opening principles and reach an equal position without effort.

How about one last comparison: Sam's 1.d4 Trilogy vs the 1.c4 Trilogy.  The d4 trilogy has 600 more lines (1557 vs 942) but the two repertoires have essentially the same total moves (4574 vs 4601).  This is exhibit A for my thesis: the same author, both sets have three courses, one is advertised as having basically 40% fewer lines ... and yet the total workload is identical.

The point: your Chessable repertoire can have an extreme workload difference depending on which courses you use.  This has nothing to do with the line count and everything to do with the move count.

What About the Quickstarter?

Chessable courses now all come with Quickstarter chapters, which are supposed to be the critical lines of the course.  Instead of learning 500 lines, you can just learn the one chapter of 50 or so lines and have a good grasp of the entire course.  Does this change the analysis?

The answer is complicated.  Note that none of the numbers above include Quickstarter variation count.  I really dislike the Quickstarter chapter: most are simply cut-and-paste lines from the respective main chapters and thus suffer from the lack of context.  The workload intends to be very high, with multiple variations of 15+ moves.  Almost all are curated towards GM theoretical soundness, making it less applicable for regular players.  With the exception of GM William's Iron English (not included in my transcriptions), I haven't enjoyed a single Quickstarter that I did and I found them completely ineffective.

And of course, other people love them and ignore the rest of the course.  Personally, I cannot fathom paying full price (at Chessable's inflated price to boot) and only using a part of the course, but that's just me.

That said, I do think a smaller, simplified approach makes sense.  There's no reason why a 2400 IM and a 1400 blitz player need to learn the same lines at the same depth.  I just don't think the Quickstarter does that.  At one point, I created smaller, compact versions of big courses: fewer lines, fewer depth, mainly getting to the major tabiyas and having a few critical or illustrative lines stemming from there.  As a result, I took the Iron English's approximately 5.5k moves and turned that into a simpler 300 move custom course.  That was absolutely perfect for my needs, and probably the needs of most people.  Unfortunately, this is something you have to do yourself.

As to the bigger point: should anyone learn a complete Chessable repertoire?  I would be a hypocrite to say no.  I have learned multiple repertoires, including the entire Shankland corpus (though not the newest Catalan yet).  Whether I could have spent that time better is an open question.

Actually, no it's not.  It's a very easy question: it was a huge waste.  I undoubtedly would be a better chess player today if I didn't spend so much time on Chessable in the last few years.  That, though, is a topic for another time.

Conclusion

A line is not a line: not all 500-line Chessable courses have the same workload.  Unfortunately, this information is hidden behind Chessable's opaque UI, and it's often not possible to know the "true" workload until you buy a course.  When in doubt, buy a Shankland course: the explanations are great and the workload always manageable.  

Post-Script: The Evolution of Chessable Courses

As an early adopter, I have seen the evolution of Chessable.  It has gone through several iterations.

First, as a small indie publisher, they published whatever.  These were generally "slim trees with deep roots," so courses that rarely branched out and certainly didn't try to cover everything.  Small total lines, reasonable workload.

Second, it began attracting talent and converting books into courses.  These were generally bad, and by that, I mean they were straight PGN imports.  If you have ever tried to import PGN into Chessable, you know what I am talking about.  Unweildy variations, extreme depth, inconsistent variations (eg, you might learn a variation with 16 moves and the next one has only 2).  Workload spiked.

Third, Chessable got better.  Basically, it began attracting more talent (Sielecki, Shankland, Krykun stand out) and began formatting the courses better.  Courses got bigger and wider (eg, more lines and more breadth), with depth being hit or miss.  Workload steadily increased, but generally manageable.  IIRC, courses typically had between 300-500 variations, with the marketing copy saying some version of, "You can master this opening in X number of lines." 

Fourth, Chessable invented the Quickstarter.  This changed things substantially, and in my opinion, this is when Chessable turned from offering repertoires (a complete set of lines meant to be learned) to reference works (not meant to be learned).  Within 6 months, courses frequently topped 800 variations, with several (GM Wesley So's being the most prominent) being over 1,000 lines.  The focus had shifted from learning an entire course to learning just this Quickstarter, which somehow justified the rest of the lines gathering dust.  Workload for entire courses increased significantly.

Fifth, the Clickable Revolution.  Chessable noticed that the bigger courses with thousands of variations weren't as well received.  The size was scaring people away.  As such, Chessable started severely cutting down its lines, with many trainables converted into "clickables," little blue text that you can click but cannot train.  As a marketing tactic, this allowed large courses to be presented as much smaller ones. From a user's perspective, this was terrible, as Chessable UI doesn't allow us to do anything constructive with clickables and, honestly, defeats the purpose of using the Move Trainer.  Workload remained steady, but perceived workload lowered.

Sixth, as of November 2023, we are entering the Modern Period.  Chessable went too extreme with shortening courses, and there was outcry with both the Classical Slav and Tarrasch LTRs for having just a few hundred lines but infinite clickables (not to mention being the same price as previous entries).  In the last three months or so, course sizes have increased and clickables appear to have decreased.  Chessable appears to be curating lines better and not automatically converting small tactics into clickables, but it's unclear if this is a trend or just a one-off.

I mentioned GM Shankland's Catalan series having a higher ratio than his other courses.  This is why: the series was released in the Clickable Revolution Era.  As such, it has many clickables, whereas his original works had almost none.  If these clickables were regular trainables, the ratio would be lower, but they aren't, so that pushes the ratio up.

Indeed, in general, clickables mean more work for students.  Clickables are frequently used for smaller lines or to show a certain tactic.  These smaller lines act as a natural break from the mainline.  Because these aren't trained, the entire line, often 10+ moves, needs to be trained in one shot, and the user needs to memorize the clickable lines without the aid of the software.  If these were regular trainable lines instead, there would be more shorter variations with more natural breaks, making it easier.  Not to mention clickables do not show up in position searches on the website.

In my view, since the Quickstarter Revolution, Chessable's presentation has progressively gotten worse even as its content surged in quality.  Great material, but a worse way of interacting with it.  Hopefully this Modern Period is a permanent righting of the ship.

2 comments:

  1. I definitely agree about the clickables. Lots of them are tactics or ways of punishing opponent mistakes, which a player of my (low) level meets more often than many of the trainable variations.

    ReplyDelete
  2. Yeah, Chessable’s quality has gone downhill in my opinion. I prefer older courses more often than not I find.

    ReplyDelete

May 2024: Smithy's Taking A Break

So this is a quick update: the blog will be lying dormant for a month.  I haven't written a new blog post in six weeks and I have exhaus...