How Many Words Can You Learn with These Popular Language Learning Apps And How Quickly?
This is a quantitative analysis of Pimsleur vs DuoLingo, what they teach, and how efficient they are. It’s an evaluation of how efficient their methods are in terms of time spent and content learned. And this post will outline my thoughts on each language learning method, sharing the pros and cons of each.
Before I get into it, though, I want to get a few facts out in the open.
I have a lot of respect for Pimsleur, and I really like their products. My website sells the Assimil product line. But I’m a huge fan (and user) of Pimsleur and Michel Thomas. I should also point out that this post is not intended as a take-down of the Pimsleur vs Duolingo methods. That’d be far too self-serving for a guy who sells a competitive product.
If you’re serious about learning a language, you should use a variety of materials.
Although I’m going to get into some criticisms of Pimsleur vs Duolingo, I should note that no system is perfect. But, my goal in this review is to be as fact-driven and objective as possible. If anyone sees any holes or errors in my methodology or my findings, drop me a line.
An Overview of the Pimsleur Language Programs
For anyone not familiar with Pimsleur Language Programs, they are the creation of Dr. Paul Pimsleur. He created his program in the late 1960s around the same time that he published an academic paper titled “A Memory Schedule”.
It was a pretty short paper, describing Dr. Pimsleur’s observations of how time affected a student’s ability to correctly remember newly acquired language knowledge. The paper is closely related to the concept of the Forgetting Curve and helps make the case for why spaced repetition systems (SRS) can help improve the speed and efficiency of learning.
Interestingly enough, Dr. Pimsleur never actually published the underlying data that guided his writing. That’s not a knock on his work or the efficacy of its outcome. I think that the Pimsleur method was novel and effective when it was first introduced. That’s a fact. Sadly, Dr. Paul Pimsleur passed away in 1976. Then, Simon & Schuster acquired his intellectual property in 1995.
How Does the Pimsleur Method Work?
Dr. Paul Pimsleur pioneered the Pimsleur Method in the 1960s, long before computers were widely available (let alone affordable). The simplest way to describe his method is the combination of traditional “listen and repeat” language learning methods combined with a simple form of spaced repetition. The Pimsleur Method also makes use of a technique called back-chaining. It’s not a technique that Pimsleur invented. However, it’s one he helped to popularize.
Side Note: Pimsleur’s spaced repetition system was “simple” due to the technology available when he created it. It’s not an insult to his work.
This Is How the Pimsleur Method Works:
- The method is audio-only with a series of quickly spoken dialogues read aloud in the target language.
- A narrator then speaks, providing context, instruction, and explanation.
- You hear the dialogue spoken again but more slowly.
- You listen and repeat words, phrases, and sentences.
- For multi-syllable words (or words with sounds that are complex to an English speaker) the platform incorporates back-chaining.
- The back-chaining technique reverses the order of syllables, from back to front, and for some odd reason, that helps people with pronunciation.
- As each lesson goes on, the amount of time between each review of learned materials grows until it cements into the student’s mind.
The description above is a simple summary. My intention here is not to review the Pimsleur Method. There is no shortage of reviews or in-depth evaluations on the Internet. My intention here is to provide a quantitative analysis of the method. Afterward, I intend to evaluate how efficient it is (in terms of time spent and content learned).
There’s no question that the Pimsleur Method works well. I used Pimsleur Language Programs to learn Russian. And I’m dabbling with Turkish and Arabic. That said, as many reviewers have pointed out that the lessons can be a bit long in the tooth. And I don’t know how often they update their dialogues.
In the modern age of apps, with computer-based spaced repetition systems (SRS) that react to the student’s triumphs and errors, it’s important to see how efficient the Pimsleur Method is.
Here’s My Experience with the Pimsleur Program
I purchased the Pimsleur Russian series, levels I – III as MP3 files. Over the course of 3 – 4 months, I listened to each lesson nearly every day. In total, there are 90 lessons, and each one is about 30 minutes in length. If I listened to every Pimsleur lesson only once, I would have invested 45 hours into the complete program. In reality, I listened to about half the lessons more than once. As a result, I’d say that my total time invested is around 60 hours.
That said, let’s give Pimsleur the maximum benefit and say that I wasn’t a great student. Other people would only have to listen to each lesson once. (This is a generous assumption since Pimsleur’s narration advises to listen to each lesson as often as necessary to get a level of comfort with the content covered.)
How Much Content Does Pimsleur Cover?
I think that Pimsleur did a great job forcing me to speak the Russian language and practice my pronunciation. However, after I completed Pimsleur, I did Michel Thomas’ Russian program. Also, I’m now about one-third of the way through with Assimil’s Russian program. With these additional resources, I came to understand how little grammar Pimsleur addresses. And I became really curious as to what, exactly, I picked up in those 45 – 60 hours.
So, I went back and had every single Russian Pimsleur lesson transcribed. I know that sounds insane. But, I hired an English-Russian transcriptionist to type out everything that said in every lesson. Interestingly, I found that Pimsleur Russian covers 101 dialogues over the course of the 90 lessons.
In retrospect, I think that makes sense. There were several cases where a program introduced a related dialogue at the end of one lesson and reinforced it at the start of the next lesson.
I should also add that many reviewers note that the structure of lessons and dialogues across Pimsleur programs is pretty consistent. And while I limited my review to Pimsleur Russian, the general findings are pretty consistent for most, if not all, Pimsleur Language Programs.
So, over the course of 45 hours, I learned 101 dialogues. But how many actual unique pieces of information did I learn?
What Pimsleur Dialogues Look Like
I took all of the dialogues and de-duped them by speaking line. (“De-duped” means removing duplicate entries.) If one speaking line in dialogue was: “Yes, thank you,” and if that speaking line appeared more than once, I de-duped it. However, if one line read: “Yes, thank you,” and another line read: “Yes, thanks,” I created two unique lines.
Using this approach, I had 693 unique spoken lines. That means that I learned 15.4 lines per hour of time invested. That seems a bit inefficient. However, a “spoken line” is not really a fair unit of measure. You’ll understand why when you see examples of the uniquely spoken lines learned:
- You need to go to the pharmacy? To the pharmacy on Pushkin Street?
- You understand Russian well.
- You understand Russian very well.
- You owe fifty rubles.
- Tomorrow I’m supposed to go to the city. Would you like to go there with me?
- A-a-ah! Hello, Mister Jones. Please, come in.
As you can see from the real examples above, there’s little consistency in the length or diversity of spoken lines. Some consist of simple phrases. Other lines are sentences with more complex clauses. And 3 of the 6 examples above are two full sentences.
How Many Unique Words Does Pimsleur Cover?
A better analysis would be to evaluate how many unique words the course covered. This isn’t the fairest way to evaluate the quality of a course. However, it’s the most consistent way to evaluate the efficiency of language learned for the time invested.
Fortunately, when I had all of the transcriptions done, I also had all of the Russian phrases translated into English. This means that I can look at all the unique words through the lenses of both English and Russian. Using English translations of the Russian content, there are a total of 4,881 words in those 693 spoken lines. That means that, on average, each line is about seven words long.
Using the actual Russian content, a total of 4,091 words were used in those 693 spoken lines, leading to an average of about six words per spoken line.
The Difficulties with Word Counting and Language Learning
In case you’re wondering why there’s a discrepancy, you should understand that there are many differences between any two languages. There are many differences between English and Russian, but one of the most notable is that the verb “to be” (or “быть” in Russian) is not used in the present tense. This is just one example, but a notable one, that illustrates why and how there could be discrepancies in word usages between the Russian content and its English translation.
That’s all pretty interesting, but let’s get back to the issue of Pimsleur’s efficiency…
How Many Unique Words Did Pimsleur Teach Me?
That’s a pretty easy question to answer: around 500 – 600. What do I mean by that? Why is the upper end of my range 20% higher than the lower end of my range? First, I’ll explain using the English data set. Then, I’ll explain using the Russian data set.
When looking at translations of the Russian content that Pimsleur teaches, my analysis shows that there are 490 unique words taught in those 45 hours of instruction. That said, this figure is somewhat misleadingly high. There are two reasons: words and lemmas.
- One of the unique English “words” taught is “Aleksandrovich”, which is actually a Russian surname.
- As for lemmas, here are three examples of words that are actually all derivatives of the same lemma (arrive): arrived, arrives, arriving.
Pimsleur doesn’t teach much formal grammar. However, one could argue that there’s only one word there. And grammar rules (verb conjugation) dictate that its different uses.
How Pimsleur Covers Russian Grammar
In the actual Russian course content, there are a total of 628 unique words taught. The issue of names is the same in Russian as in English. If anything, the issue of lemmas is more acute. This owes to the fact that Russian grammar is more complex than English. (That is to say that Russian has more formal grammar rules than does English.)
For example, Russian has a formal and informal pronoun for “you” (“вы” and “ты”, respectively). Additionally, like many European languages, Russian nouns have gender – masculine, feminine or neuter. In some verb tenses, you conjugate verbs differently based on the gender of their direct objects. For example, the Russian verb for “was” is “был”, but can appear as “была” (for feminine nouns), “были” (for plural nouns) and “было” (for neuter nouns). In case you’re curious, you use “был” for masculine nouns.
Additionally, in all tenses, Russian adjectives take suffixes that agree in number and gender with the nouns they modify. For example, the Russian adjective for “good” can appear as “хорошо” for masculine or neuter nouns and “хорошая” for feminine nouns. These complexities lead to a higher word count for the Russian content when compared to its English translations.
A Summary of My Pimsleur Experience
Where does that leave us? I’ll be as generous as possible and say that all 628 “words” taught in the Russian Pimsleur Language Program constitute “useful things”. That means that Pimsleur taught me 628 “useful things” in the Russian language over the course of 45 hours. So, I learned about 14 “useful things” (words) per hour for each of the 45 hours that I spent on the course.
(Actually, since I listened to many of the lessons more than once, I’d say I probably invested more like 60 hours of time. That means that I learned a little less than 10.5 “useful things” per hour of time invested. But like I said, I wanted to be as generous to Pimsleur as possible in this analysis.)
Before I go any further, I should be the first to point out that while this sounds disturbingly inefficient, it’s not a fair statement. There is a huge difference between learning words and learning actual phrases. Also, since I modified my question to evaluate how many “useful things” Pimsleur taught me, I should factor in those spoken lines. Let’s do that now.
How Much Does Pimsleur Cover
As I said before, the Pimsleur Russian Language Program is comprised of 693 unique spoken lines. However, many lines are more than one sentence long. Breaking them out into individual sentences and then de-duping any repeated sentences left 823 unique sentences. Now, I should point out that in some of these “sentences” there are statements like, “Yes”, and “Oh, yes!”. But, that’s ok. These are fully grammatically correct statements.
In the spirit of generosity, I’ll even add unique sentences to all the unique words and use only the Russian course content (since the word count is higher). As a result, Pimsleur offers
- 628 “Useful things” (words)
- 823 “Sentences” (grammatically correct statements)
- 1,451 Total things that Pimsleur taught me
In the end, over the course of 45 hours, that works out to 32.2 things learned per hour. (That’d be 24.18 things learned per hour using the more realistic 60-hour estimate I referenced earlier.)
How Does the Pimsleur Method Compare to Other Courses?
Quite honestly, I’m not sure. I thought it’d be interesting to look at the words that Pimsleur teaches and compare them to a frequency data set. There are many word frequency data sets floating around out there. And all of them come with various trade-offs and caveats. In order to keep things as apples-to-apples as possible, I decided to compare the Pimsleur data sets with an index of word frequencies for both Russian and English lemmas as published by Rutledge.
I chose Rutledge for two simple reasons:
- It’s is a well-respected academic publisher that puts a lot of time and care into developing its word frequency lists.
- The word frequency lists are available for both English and Russian.
In case you’re curious, here are the Amazon links to the published sources of the data from Rutledge for English and Russian. (These are straight links, no affiliate stuff. I just want folks to know where my data come from.)
It took a little bit of data massaging to get the data sets as comparable as possible. I had to manually go back into the English and Russian data sets and assign a lemma to each word from the Pimsleur data set. This allowed me to do a straight comparison across Pimsleur and Rutledge.
The Results of My Pimsleur and Rutledge Comparison
All the words in the data set are Russian lemmas from Pimsleur’s Russian course, levels I – III. Words marked in blue are those words from the Pimsleur course that also appear in the top 500 words from the Rutledge data set.
I find it very interesting that among Rutledge’s top 500 words, there are relatively few Pimsleur words. Now, in some ways, I think this can make sense. For example, the word “help” is one of Pimsleur’s top words. However, the word “help” isn’t even in the top 200 most frequent words from the Rutledge data set. This makes a lot of sense because they measure the word frequency lists across native speakers while helpful words for a new learner of a language might vary.
But getting back to my original point, how efficient is the Pimsleur method? Efficiency and effectiveness are two different things. Sadly, there’s been very little research done on effectiveness, and almost no work done on efficiency. That said, Duolingo carried out one of the few studies that looked at efficiency.
Duolingo vs Pimsleur: How the Two Compare
When it comes to the analysis of Pimsleur vs Duolingo, it’s important to look at how long it would take you to learn a language with each platform. According to this post, it takes 34 hours of Duolingo to learn the equivalent of one semester of college on average. Since a semester of college generally takes a lot more than 34 hours of work, this suggests that Duolingo is more effective than a college course.
An external research team that previously evaluated the effectiveness of other methods such as Rosetta Stone carried out the study. It is of note that it took 55 hours of study with Rosetta Stone to reach the equivalent of one semester of college. In conclusion, not only is Duolingo free-er than Rosetta Stone, but the study suggests it’s also better.
But There’s More to Comparing Pimsleur vs Duolingo
The previous finding sounds pretty impressive. However, having read the entire 25-page paper published by Duolingo, there are a few things to bear in mind:
- The study provided native-English speakers of non-Hispanic origin a college-level test on Spanish before and after completing the Duolingo Spanish course.
- Overall improvement was 91.4 “points”. This was statistically significant. However, it’s not clear what “points” are or what the total possible points could have been. (I have trouble believing that the scale consisted of 100 points because that would imply that on average, everyone in the study would have scored an A- on the Spanish test post-study.)
- Out of an initial population of 556 people, the report excluded 170 for a variety of statistical reasons.
- Of the remaining 386 participants in the study: The average age of eligible participants was 32.0 years with 48.4% females, and 99.0% of them were novice to intermediate (self-report) users of Spanish.
Can You Learn a Semester of Any Language with Duolingo?
That’s interesting. The paper doesn’t get into much detail on defining “novice to intermediate”. However, I suspect the definition is not “speaks absolutely zero Spanish”. That would imply that the study participants had a bit of a head start. And that means that comparing them to college students starting from zero may not be a fair comparison. (Of course, that assumes that all college students who study Spanish start from zero, which is also probably a flawed assumption.)
The study goes on to state that 11.0% of study participants’ spouses, partners or close friends spoke Spanish. And that 4.5% of participants’ parents, grandparents, or great grandparents spoke Spanish. Also, more than a quarter (28.5%) of participants knew at least one other foreign language. Finally, the paper concludes that the average Duolingo user could learn the equivalent of one college semester of Spanish with 26 – 49 hours of time spent on Duolingo.
Is Duolingo as Effective as They Say?
Now don’t get me wrong. I think Duolingo is a pretty interesting app, and I applaud them for using an outside research firm to evaluate the efficacy of their approach to language learning, but:
- The range of 26 – 49 hours is pretty broad; it’s nearly a 2x multiple.
- There’s a big gap between efficiency as measured by a test and efficiency as measured by the ability to speak to another human being in a foreign language.
Since I’m using Duolingo on a semi-regular basis to practice my Russian, I decided to grab a list of the Duolingo vocabulary set for Russian. (Duolingo makes all vocabulary courses for all its languages available through its Tiny Cards app. And that’s where I got the data set.)
With the Duolingo data, I wasn’t able to form a word cloud because I don’t have all the unique sentences that the platform uses for its Russian lessons. All I have is a list of words. As was the case with the Pimsleur data set, I did my best to normalize the data and account for lemmas.
This time, I thought I’d mix things up a bit and put together some pie charts. So here’s what happens when we compare all the words in the Duolingo Russian course with the top 500 words from the Rutledge data set:
Overall Impression of Duolingo’s Content
What does this mean? It means that 67% of the top 500 words in the Rutledge data set are not found in the Duolingo Russian course. In case you’re curious, the green, yellow and orange data shows the percentage covered in the top 10, 25, and 100 words. Now, this does NOT mean that there are tons of words that Duolingo doesn’t teach. It just means that the emphasis on those words, as defined by their presence in Rutledge’s top 500 most frequent words, is prioritized differently.
By the way, after playing around with this for a while, I did a Pimsleur infographic to compare, and this is what I came up with: As you can see, in terms of coverage between Pimsleur vs Duolingo, Pimsleur’s is superior. This makes sense, too, since Pimsleur has been around for so long, and it was designed by language teachers vs Duolingo’s app developers.
Pimsleur vs Duolingo: Which Platform Works Best?
When it comes to Pimsleur vs Duolingo, they both have their strengths and weaknesses. The short answer: we collectively need more research done to evaluate the efficiency and effectiveness of language learning programs. It needs to avoid the standard, written college tests. But that’s a problem with language education in general. Instead, those studies need to look at verbs, written and reading comprehension of a language.
For my own part, whether we go with the 45-hour estimate of the 60-hour estimate, I’m pretty happy with the fact that Pimsleur taught me around 30 “useful things” per hour. That’s a new nugget of knowledge every 2 minutes, on average, of time spent invested with the program.
That feels like a pretty good use of my time. Moreover, the success of Pimsleur vs Duolingo comes from how much you can learn. While I’ve dabbled with Duolingo’s Russian course, Pimsleur’s course actually taught me how to speak. (It didn’t teach me how to read, which Duolingo does fairly well, but it got my jaw yapping, which is important.) Overall, I’d say that Pimsleur is a very solid language program. I have my personal views on the underlying content it teaches, but I’ll keep them to myself.
