The case for using DUOLINGO as part of the language classroom experience

This article explores the idea of using an already existing language learning app, Duolingo, to complement traditional college level Spanish as second language courses. These types of apps use adaptive learning technologies, which are able to tailor the tasks to the level of each student. In the case of this study, Duolingo was used as part of the program of studies in two Spanish university courses, one a beginner’s Spanish course (level A1) and the other an advanced intermediate course (B2). The students used the app online, either in its mobile version or in their web browser. I will describe how Duolingo operates, what kind of activities can be done, and how learning is achieved. Preliminary results suggest that Duolingo is an easy-to-use app that is useful and has potential, although its main lessons are not based on communicative competence. It is usually enjoyed by students because of several elements, such as the accessibility on a mobile device, its gamification aspect, and the variety of tasks. I will discuss possible ways to incorporate Duolingo into foreign language courses, always considering it as a complement to the curriculum, but considering its value to reinforce vocabulary and grammar acquisition through spaced repetition, interleaving of different skills and variety of activities.

Nowadays, most of our students own a smartphone. According to data from the report "Worldwide Mobile Phone Users: H1 2014 Forecast and Comparative Estimates," by the end of 2015, almost 66% of the world population will have one of these phones. These mobile devices can be equipped with numerous applications that can be used to learn a foreign language. Some of these applications are normally used for other objectives, but can also be useful in this sense, such as watching YouTube videos, recording voice memos, having online conversations with Facetime or Google Hangouts or even playing with gaming apps like Trivia Crack (a game similar to Trivial Pursuit) in languages rather than your own. But there are also an increasing number of applications that have been created with the main objective of learning a foreign language. Examples of these apps are Lingua.Ly (to augment reading comprehension), HelloTalk (to find people with whom to practice languages), Memrise (vocabulary acquisition) or Duolingo, the app we will be exploring in this article.
According to Ramírez Montoya (2009), a definition of mobile language learning can include several components: It is the direct descendant of e-learning, which is any type of learning supported by electronic tools and resources, and m-learning, which uses online resources that can be accessed through a mobile device. M-learning allows students to augment the classroom learning by providing a flexible type of learning that can enrich the classroom experience. Montoya mentions that authors like Sharples (2005) go a bit further, including in their definition the idea that m-learning is a process of coming to know, where students, collaborating with their peers and instructors build that knowledge together. A more current definition is the one offered by Crompton (2013, p. 4), which states that m-learning is "learning across multiple contexts, through social and content interactions, using personal electronic devices".
With this meaning in mind, this article will be studying one particular app, Duolingo, which can be accessed either through the web on a desktop computer or through a cell phone or a tablet. Through a series of exercises, divided into lessons and skills, this app's goal is to teach vocabulary and grammar, and to bring students to an A2 level by having them complete all the required nodes. One of its 85 RIED v. 19: 1, 2016, pp 83-101 AIESAD I.S.S.N.: 1138-2783 main features is the use of spaced repetition, in which its algorithms detect when a user needs to review words or chunks that may have faded from memory. Spaced repetition has proven to be very effective for acquiring vocabulary in particular, since repetition is essential to acquire new skills. In a very interesting experiment with cell phones, Cavus and Ibrahim (2009) showed that students using the devices enjoyed and learned new words, and the students themselves suggested in their survey that other educators should augment their teaching with the use of these devices.
In addition to spaced repetition, the activities presented in Duolingo's lessons interleave different skills (from pronunciation to translation, for example), and provide a variety of activities (recognizing a word from a picture, writing down what you hear, etc.). It is these three elements, spaced-out practice, interleaving, and variety, which have been shown to produce better mastery as well as longer retention and versatility in very recent research on learning. (Brown, Roediger and McDaniel, 2014, p. 46). Duolingo, then, seems to be able to tap into these three modes.
Duolingo also offers instant feedback to its users after each task. As García (2013, p. 21) points out in his review of the app, this feedback goes beyond basic multiple choice and includes other appropriate versions. In many instances, it also includes a discussion area where users can negotiate the exact meaning or possible problems with the activity or the offered solution.
Another aim of this study is to observe whether using Duolingo as part of online homework produces a shift towards self-directed learning by students. Self-directed learning in this case is understood as "learning in which the conceptualization, design, conduct and evaluation of a learning project are directed by the learner" (Brookfield, 2009(Brookfield, , p. 2615. One important aspect of this study is the idea that Duolingo can serve well as an addition to a formal language course offered through a school or university, whether face-to-face or online. Thus, here we are understanding the use of the app as a way to augment or substitute more classical types of homework rather than replacing the class itself. A non-peer reviewed study by Vesselinov & Grego (2012), which was commissioned by Duolingo itself, estimated than an average of 34 hours spent on Duolingo were the equivalent of a college beginner-level course. It is not the aim of this study to discuss the validity of that study, as I believe that Duolingo has a good potential for use in a course of studies as one more tool in the instructor's arsenal.

Duolingo: definition of the app
Duolingo is a free app created by Luis Von Ahn and Severin Hacker in November 2011. Its slogan is "Free language education for the world." According to its website, it has more than 30 million registered users. It offers several languages for English speakers as well as others for non-English speakers.
In the desktop version, the app has different areas: I.S.S.N.: 1138-2783 AIESAD • The "tree", which contains skills, each represented by a node that changes color from grey (indicating that a skill has not been started), to a color like red, blue or green (you have started the lessons within the skill), to gold (you have mastered all the lessons and vocabulary for that specific skill). Note that the gold color can "turn" back to another color if the algorithm of Duolingo establishes that you need to go back and review those nodes because you have forgotten the vocabulary or because enough time has elapsed. Duolingo has not, up to now, published the content of the tree. The only way to find out is to complete the tree yourself. • Words: A list of the words the user has studied or already knew. Each word is accompanied by a strength bar, which indicates if the word is still strong in your memory or if it is time to practice again, according to Duolingo's algorithm. You can also review these words through flashcards.
• Activity: Duolingo works similarly to a social network site. You can follow other students, and other people can follow you. The activity stream shows not only what you and the people you follow accomplish in Duolingo (levels opened) but also if you or they have made a comment in the discussion board. All the people you follow and you yourself appear on a "leaderboard" list, which shows the XP (Duolingo points) that you obtain weekly. This is reset every Sunday at 7:00 pm Eastern Standard Time (United States).
• Discussion: In this area, users can post discussions about topics that interest them. A discussion will also be created if there are questions about any of the activities completed during the lessons, in which case users negotiate meaning among themselves. The discussions are divided according to the language you study, and there are also general discussion areas, like the one recently opened for educators (this will be discussed in more detailed later). Discussions are voted up or down (similar to places like Reddit), so the more popular ones are more visible. It is also the area where the creators or moderators of Duolingo post important messages to the community.
• Immersion: This area is for more advanced students. Anyone can upload a public document in any language and the community can start translating it. Users can offer suggestions for better translations or can just make general comments. Users get XP points for translating either their own documents or helping with other documents already uploaded to the site. They also get lingots (see below for a definition) if their translations are up-voted by other users.

87
• The Lingot Store: "Lingots" are the currency that Duolingo uses to reward its users. You get lingots when you complete a skill node or maintain a streak of days using the site. Users can also "reward" other users with lingots in the Discussion area. With lingots you can buy several things, like a "freeze streak" which will allow you to miss a day and not lose your streak, some silly outfits for the Duolingo owl (the program's mascot), or, more important in my opinion, a progress quiz, which shows you on a scale of 0 to 5 where you currently stand in your studies.
The mobile version of Duolingo does not have these many areas. It is limited to the tree and the Lingot Store, although you still can see there the leaderboard with information about the people you follow. The app also informs you of your streak. You can set up daily goals (the minimum being 10 XP per day), and the app will send you reminders and notifications if your goal has not been completed that day.
For this study, I have only used the tree area with the skills and lessons. The following description applies to this area exclusively: students can perform different types of activities on Duolingo. Once you click on a skill, you are presented with the available number of lessons for that particular skill. Each lesson lists the words (up to eight) that it will review. In addition to the lessons, each skill can be just reviewed in general, once you have completed all the lessons or have tested out of that skill. That is called "practice" or "strengthening skills" within the app in order to differentiate it from regular lessons. You can chose general practice in order to review areas that the program considers have not been practiced, rather than just one specific skill. Or you can choose to do practice within one skill once you have completed all the lessons. There is a symbol in each lesson that indicates the "strength" of that skill with a maximum "grade" of 5. Once you reach five, the symbol for the skill becomes gold.
Following is a list of the most common activities in each lesson (it may not be comprehensive, since the app is constantly being modified): • Write a vocabulary word after seeing a picture that represents it. • Translate a sentence into your native language. When words are first presented, the user can hover over the word to see its meaning.
• Translate a sentence into the language being studied. • Dictation: write a sentence that you hear. There are two speeds, normal and slow, which you can click to hear the sentence more clearly.
• Pronouncing a sentence. Through voice recognition software, the app can detect whether your pronunciation is correct.
• Match pairs of words. I.S.S.N.: 1138-2783 AIESAD • Put a series of scrambled words in order. • Choose from three sentences in the target language to see which ones fit the sentence in the native language.
Activities are presented sequentially, and the lesson "extends" itself if you get wrong answers, as indicated by the strength bar at the top of the lesson. If there are no mistakes, it takes seventeen short activities like the ones described above to complete a lesson. This generally takes five to ten minutes. This time may vary, however, since new activities are added if you make mistakes.
Duolingo incorporates some gamification elements to motivate and engage learners. Some examples of this are the lingots as awards given when you complete a skill; the inclusion of a weekly leaderboard, where you can "compete" against friends to see who has the highest XP; a symbol of a flame next to your name with the numbers of days of your streak on the site; the aforementioned strength bar, which appears when a user is completing a lesson, to indicate how close they are to finishing it, etc. These elements make the app more enjoyable, although the exercises themselves are quite traditional, as we have seen.

Educator's Area in Duolingo
Recognizing that many educators were using this tool with their students, as evidenced by the numerous discussion boards posting to that effect, Duolingo opened a new area in its website in January 2015, dedicated to teachers who want to use the platform with their classes. The new area, https://dashboard.duolingo.com/, allows educators to create "groups" or classes. This generates a link, which can be sent to students so that they can join the group. In the dashboard, the teacher can monitor students' progress in several ways. The dashboard shows the time the student logged in, the lesson or practice he/she completed and the XP he/she obtained. A weekly report is sent by email, which includes all the work students have done during that time. In addition, there is now a dedicated area in the Discussion Forums just for educators' concerns and ideas.
This dashboard did not exist when the study described here was conducted, and the instructor had to follow each student to see how many lessons they had completed. This new dashboard should facilitate the work of educators immensely and make this tool much more useful, as suggested in this study.

Duolingo as an educational tool for online and face-to-face courses
As we can see, even though Duolingo is presented as a very modern-looking gaming app, in reality most of the activities necessary to complete the lessons are very traditional and are heavily based on translation, dictation, and pronunciation. In fact, there is already some criticism of the app within the language learning research community. Krashen (2014) points out in his rebuke of the Vesselinov and Grego (2012) study, in which they equated 34 hours of Duolingo to a university language course, that language learning is different from language acquisition. Krashen states that the activities presented in Duolingo (or in similar tutoring programs such as Rosetta Stone) involve conscious learning. In his opinion, conscious learning does not lead to language competence. He presents a summary of the research done by Mason, 2004;2011;and Mason, Vanata, Jander, Borsch, and Krashen, 2009, noting that in his view, acquisition-oriented methods, which use subconscious learning, are superior to skill-based methods such as those used by Duolingo. I do not dispute his statements, but, as already mentioned in the introduction, the proposal presented here aims not to replace language study with this app, but rather to complement any regular course with it. In fact, as we will see, for both college groups in which the app was tried as part of their syllabus, Duolingo represented only 10% of their final grade, hopefully having the rest use acquisition-oriented tasks.
It is also interesting to point out that although the translation and dictation exercises used by Duolingo harken back to the old days of the grammar-translation method, recent research shows that they do have value. Hall, Graham and Cook, (2012) point out that there is in fact a revival in the role of translating as part of Second Language Acquisition (SLA) research. According to them, "the argument is that in many contexts, translation is a natural and effective means of language learning, develops an important skill, answers students' needs and preferences, and protects students' linguistic and cultural identity." (Hall, Graham and Cook, 2012, p. 283). In their review of the literature, they point out that Laufer & Girsai (2008) "make the case for explicit contrastive analysis and translation as part of formfocused instruction after finding that learners taught unfamiliar vocabulary items via translation fared better in a subsequent retention test than those taught solely through meaning-focused instruction" (Hall, Graham and Cook, 2012, p. 289).
In the case of dictation, recent research in SLA shows that it may also be beneficial. In an interesting study, Rahimi (2008) conducted an experiment in which a group of students used dictation in class practice in addition to their regular assignments, while another group skipped the dictations. The results of his study showed that the group with additional dictation exercises made more gains in grammar, vocabulary, reading, and listening comprehension. In another experiment, Kuo (2010) used dictation to help students decode words that they heard on the radio. The dictation exercises helped students facilitate their listening and enhanced their comprehension. Nation and Newton (2009) also consider dictation a valid language learning tool. They observe that "dictation helps language learning by making learners focus on the language form of phrase and clause level constructions, and by providing feedback on the accuracy of their perception." (Nation and Newton, 2009: 59). They also believe that dictation is most useful if it involves familiar vocabulary presented in different constructions, and, importantly, when "there is opportunity for the repetition of the I.S.S.N.: 1138-2783 AIESAD material." (p. 60). According to them, it becomes even more valuable if learners are aware of the mistakes that they made. In our observations of the Duolingo app, all these characteristics seem to be present. The dictation activities consist of very short sentences in which familiar vocabulary is placed in different structures, repetition is constant (according to the algorithms used, which emphasize words that may have been forgotten), and each sentence always comes with the right response at the end so that students are informed of their mistakes.
Another aspect that has been traditionally neglected in language courses has been that of pronunciation exercises. Here also, new research indicates the validity of such exercises. Trofimovich & Gatbonton (2006) examined the role of repetition exercises focusing on form with several experiments in a Spanish course. They state that "both repetition and focus on form have measurable benefits for L2 speech processing, lending validity to those approaches to teaching pronunciation that include repetition and involve focus on form." (Trofimovich & Gatbonton, 2006, p. 532). Although they include pronunciation exercises that have a more communicative approach, even those without it were still effective. Jensen & Vinther, (2003) also found that repetition of utterances led to improvements in comprehension, phonological decoding strategies, and grammatical accuracy.

Aims of this study
The aim of this study has been to see whether Duolingo can be used efficiently as part of a language course, with the idea that students can practice anywhere and anytime, in a manner adapted to their level and needs, so as to ideally complement and augment what is covered in class. With this aim, the instructor wanted to answer these questions: • Is Duolingo an easy to use, helpful, and enjoyable app to practice Spanish? • How does it compare to regular, book based, homework? • Does Duolingo promote self-directed learning that moves beyond the course's requirements?
• Will Duolingo be useful even after the course is completed? • What are the students' suggestions for improving the use of Duolingo as part of a Spanish course curriculum?

METHOD
Forty six students from a first-year Spanish course (level A1) and sixteen from a more advanced course (level B2) used Duolingo for one university semester (from 91 RIED v. 19: 1, 2016, pp 83-101 AIESAD I.S.S.N.: 1138-2783 the end of August to the middle of December). Students were free to use the mobile or the desktop version of the app. For both groups, Duolingo represented 10% of their final grade. Both groups took the free placement test offered by Duolingo when you sign up for a new course. This meant that each student started at a different point, depending on where they placed in the test. As expected, most of the first-year students started at zero. Starting levels varied considerably for students in the level B2 course.
These were the instructions given to both groups: First-year Spanish course: Students needed to complete five Duolingo lessons per week. Points were given based on when students did the lesson, according to this As evident from the table, doing a little on more days had more value than doing a big chunk in only one sitting. The reasoning for this is that the instructor wanted to emphasize spacing learning, since research on learning how to learn suggests it is best for memory retention to study a little every day, rather than doing a big chunk the day before a task is due (Brown, Roediger and McDaniel, 2014). Students were not required to complete any one set of lessons or skills. Instead, each student worked at his/her own level, completing five lessons wherever they were. They were also encouraged by the instructor to turn the past nodes gold if they became another color (an indication that they needed to refresh those skills, according to the Duolingo algorithm). The main goal was not so much to advance far in the tree as to maintain a consistent practice.
For the B2 course, the learning objectives were very different. Students come into this course from many different backgrounds, and their levels vary widely. The goal was to have students complete all of the lessons offered by Duolingo (as mentioned above, at the time of this writing there are 329 lessons, divided into 66 skills, with a total of 1571 words reviewed). Students were required to complete five skills per week. As noted previously, skills, that is, each node in the tree, vary in the 92 RIED v. 19: 1, 2016, pp 83-101 I.S.S.N.: 1138-2783 AIESAD number of lessons they have, which can range from just one lesson to up to eleven. Although at the beginning, it was very easy for students to complete the nodes in the tree and turn them into gold, as the semester progressed it became more and more difficult, particularly for students with a lower level of Spanish. Thus, by the end of the semester, many students in this group were frustrated with Duolingo, as evident from the results of the survey.
All students in both groups completed surveys at the end of the semester. Two of the questions in the survey were the same as the one used in the Vesselinov and Grego (2012) study of Duolingo. Question #1 concerned user satisfaction with the app and question #2 asked, "How likely are you to recommend Duolingo to a colleague or 'friend?" (All the survey questions can be found in Appendix 1).

RESULTS
Regarding our first question from the aims of this study, "Is Duolingo an easy to use, helpful, and enjoyable app to practice Spanish?". Tables 2 and 3 show the results for groups A1 and B2. If we combine the Strongly Agree with the Agree results we obtained, we observe that 91% found Duolingo easy to use, 82% found it helpful, 80.4% enjoyed using it and 78.3% were satisfied with the app. In addition, not too many students seem to disagree with these statements.  The results were a bit different from those seen for the beginner's group. Although students in both classes found the app easy to use and helpful in studying Spanish, clearly the students in this group did not enjoy using the app as much (only 43.8% said they Strongly Agree or Agree). Only half the class was actually satisfied with Duolingo (when we again combine the results from Strongly Agree and Agree). It is still encouraging that over 80% of students still found the app helpful.

93
For question number two, "How does it compare to regular, book based, homework?" we can look at the answers students gave in the following tables: As we can see, students in the A1 group did indeed like it much better than regular homework. The questionnaire presented a follow-up question to this one, i.e. "Why?" Some of the responses given in this group were evidence that (1) students liked the variety of activities provided by Duolingo ("Because it is better. Includes hearing and writing to practice more".), (2) it is an easy and simple app to use ("Easy app that I could do whenever I wanted.";"It was better because it was easy to do the lessons on the app".), (3) they like both the fact that it is on the phone and its gamification aspect ("Since college students like to use their phones, it makes it feel like we are playing a game rather than doing homework;" "It was like playing a game while learning".), and (4) it gives them instant feedback ("Because instead of simply doing the homework and not knowing if you were right or wrong, this program helped you understand if you were wrong or right, and if you were wrong it would correct you and show you your mistake;" "Duolingo will tell you right then and there if you are wrong and help you correct your mistakes".). For the students in this group who answered that they didn't like it, one person wrote: "I like doing my homework from a workbook with paper and a pen. I learn better if I handwrite the information myself".
For group B2, we can see again that the results were very different. But even though not that many students liked Duolingo better, if we combine the "Yes" answers with those of "The Same," the percentage is 87.6%. This indicates to this researcher that, although there were aspects they may not have liked about the app, they still thought it could be considered as valid as regular homework. The answers from this group as to why they liked it more are similar to the ones we saw for the other group ("Because it is easy to do on the go, and I honestly feel that it helps me remember the language well;" "it was easier to access and it also kept me motivated to keep turning the circles gold;" "It was easy to complete and convenient that it was on my phone;" "Because it is easy to do on the go, and I honestly feel that it helps me remember the language well".). As to why they did not like it, many students expressed frustration not so much with the app, but with the requirement of turning five skill circles gold every week. They also complained that often, the translations in Duolingo have to be specific and the program marks them wrong even if there is another option.
Questions 3 and 4 of our aims for this study: "Does Duolingo promote selfdirected learning that moves beyond the course's requirements?" and "Will Duolingo be useful even after the course is completed?" were answered with the following questions in our survey, tables 5 and 6. We observe here the same differences than before. Students in the A1 group seemed much more inclined to do more lessons than the required ones (some of them even did lessons in other languages as well), and most of them believe that they may use Duolingo in the future without any prompt from a course. Most students in group B2, on the other hand, did not complete extra lessons and do not think they will continue using Duolingo, although if we combine the Yes with the Maybe, it indicates that more than 50% of the class thinks it is a possibility.
The survey to the students also included the question "How did you access Duolingo?", since I wanted to gauge the popularity of the mobile options, the phone or tablet, as opposed to the browser version. Students could click on all that applied. The following chart presents the results. Chart 1. Access modality From this chart, we see that more than half of students in both groups used their mobile phones or tablets at some point. The B2 group appeared to also use the web desk version more often, but that may have had to do with the fact that in the web version, Duolingo allows you to do a quick practice to turn a skill back to gold, and this was what they needed to do for this class. In any case, based upon their written responses and these results, it is obvious that having the ability to do homework on your mobile devices is something preferred by both groups.
Lastly, all of our students were asked how the use of Duolingo in class could be improved. In group A1, many students said that they liked it the way it was. Some mentioned that the skills they were reviewing in Duolingo did not correspond with what was seen in class and thought that they should be better aligned. Others also suggested doing the lessons in class, maybe for five minutes every day.
For our B2 students, many suggested that the requirement be made less stringent and count XPs rather than requiring five skills turned to gold per week, so that students could enjoy it more.

DISCUSSION
In this paper, I have tried to see if adding Duolingo to a Spanish course can improve the course and also can give students a new tool that they can use after the course is completed. I studied the use of Duolingo in two different Spanish level university courses with different pedagogical goals. In one course, the aim was simply to complement the course (A1 group), while in the more advanced group, the goal was to have students review basic vocabulary and grammar that they should know at that level.
Based upon the results in the survey and the aims of this study, students appear to find Duolingo an easy-to-use, helpful, and enjoyable app to practice Spanish; they seem to like it more than regular, book based homework because of the convenience it provides; they like the fact that it can be accessed in different formats, particularly through mobile access; and lastly, they also enjoy its gamification aspects. I believe that this app is also successful because of the way the lessons are presented, with short prompts that are varied and with different skills interleaved. As we saw with the research from Brown, Roediger and McDaniel (2014), these methods, together with spaced repetition, make learning more efficient.
There are some drawbacks to Duolingo, which were also noticed by students. One of them is the accuracy of its translations, which may not always be exact or which sometimes do not accept other versions. In the browser app, you can discuss your answer with other learners to try to negotiate meaning in a way, but this is not possible in the mobile version. The writer of the article has tried to do all of the Duolingo tree in Spanish, and has experienced this only rarely and mostly in more advanced nodes, thus it does not represent an important obstacle.