Here’s what I mean when I say that using memorized content is a bad idea.  Check out this bad body paragraph, in response to a question like “Do you prefer to read books or to watch movies based on the books?”

“To begin with, watching movies based on books will let us know if reading the books themselves is a good idea. As we are very busy, it is a real challenge to read even one or two books in a month. If we watch movies, however, we can get an idea of which books we should read in our limited free time. My personal experience is a compelling example of what I mean. When I was young, the book “Harry Potter” seemed interesting to me, but I didn’t read it because it had too many pages. Additionally, I didn’t have enough time to finish it since I practiced basketball almost every day when I was a junior high school student.  I went to the basketball court in my neighborhood every weekend and practiced passing, shooting and dribbling with my friend, Jim, who I had known since I was in elementary school.  We practiced really hard, and, as a result, we were eventually invited to join a local team.  At the end of that year, the team won a local championship.  Had we not practiced every day, we would not have achieved such success. Several years later, a film based on the novel was released. I went to the local cinema and watched the move because it was only ninety minutes long. After I saw the film, I bought a copy of the novel because the movie showed me that it would be a good use of my limited free time.”

 

Note the stuff in bold, which is just stuff the student memorized before going to the test center.  They have inserted it into the paragraph to increase the word count and to add some slightly more complicated sentence structures.  It’s got parenthetical commas, the past perfect tense and a conditional.  Even a transitional phrase!  How nice!  Sadly, it sticks out like a sore thumb.  The grader can tell it is off-topic memorized junk.  This is an essay about books and movies, not an essay about how to win a basketball championship. 

I constantly get paragraphs that have been stretched out with irrelevant digressions like this.  Sometimes you can get away with using them, but generally it is a bad idea.  My impression is that ETS is working harder than ever nowadays to crack down on this junk.

If you see a warning at the test center saying “do not use memorized examples” this is what the warning is about.

 

One of the most frequent questions I get is about the “maximum word count” in the TOEFL essays.  A lot of students think they must not write more than the suggested word count in both essays.  Even some teachers think it is a problem.

Hopefully this blog post will put that rumor to rest.

Here’s what ETS says about the word count for TOEFL writing task 1 (the integrated essay):

(Official Guide to the TOEFL Test, 6th edition, page 192. Published 2021)

And here’s what ETS says about the word count for TOEFL writing task 2 (the independent essay):

(Official Guide to the TOEFL Test, 6th edition, page 199.  Published 2021).

Meanwhile, when ETS is training teachers at their regular “Propell Workshops” they provide sample essays of each score level.  To represent the 5.0 level (the highest possible score) they provide the following 305-word integrated essay:

sample integrated essay

(Propell Workshop for the iBT Test, Page 42.  Published 2019)

And to represent an independent essay scoring 5.0, they provide this 466-word monster:

(TOEFL Teacher Workshop Manual, Page 49)

So there you go.  Write as much as you want.  Don’t worry about being penalized.

The other day, someone asked:

I’ve got twelve months to prepare for the TOEFL, and I need 100 points.  What should I do?

The good news for that student is that they have time to really improve their English fluency instead of just learning TOEFL tricks and strategies.  I know it sounds crazy, but the best way to increase your TOEFL score is to become more fluent in English.

 

Here’s how I responded:

  1.  Get a good grammar book like “English Grammar in Use” (also called “Grammar in Use – Intermediate” in some countries).  I read about a dozen TOEFL essays every day, and I see that most students suffer from grammar and language use problems.   Reduce your error rate and your writing score will go up.
  2. Find someone to practice speaking with.  To improve your score you need to speak fluidly.  You need to eliminate pauses, “umm breaks”, and repetitions.  You need to pronounce vowels and consonants properly.  You need to reduce the effort required to understand what you are saying.  Regular practice will help with this.  You don’t necessarily have t pay big bucks for a special TOEFL teacher to do this.  You can probably find an affordable tutor on a service like italki for this.
  3. Take accurate practice TOEFL tests.  There are 15 official ETS practice tests available (Official Guide x 4, Official iBT Tests x 10, website x 1) plus some PDF junk on the website.  You should work through all of those.  Fortunately, you have time to buy all of the books!  Switch to unofficial material only when you run out.
  4. If you have a year to prepare you can also improve your reading and listening skills in a general sense.  Spend some time reading good non-fiction books and articles (I like Science News, and National Geographic).  Make use of your local library, if they have an English section.  For listening, try Khan Academy, or podcasts like 60 Second Science.
  5. Towards the end of your preparation period take one of the scored practice tests from ETS to gauge your current level and see how to use the last few months most effectively.

 

And, yes, along the way you should devote some time to becoming familiar with the test.  Read the Official Guide cover to cover (a few times).  Read some of the guides on this website and watch some Youtube videos.  Review sample writing and speaking responses.  Just don’t get bogged down in “strategies” if the test is still a year away.

Editor’s Note: This is part three in my series on the e-rater.  You might want to  read part one  and part two.

Les Perelman has a new article in the Journal of Writing Assessment about current problems associated with automatic essay scoring in general, and the ETS e-rater in particular.  This is stuff Perelman has written about before, but I do encourage you to read the article, even if you are familiar with his work.  It is illuminating.

I want to touch on one observation made by Perelman in the article.  He notes:

“Indeed, ETS researchers themselves acknowledge the susceptibility of e-rater to both coaching and gaming when discussing e-rater’s scoring mainland Chinese on average over a half a point higher than human raters (d = 0.60) on the GRE issue essay”

Perelman doesn’t mention it, but the effect is similarly pronounced in the TOEFL , (d = 0.25 on a 5 point scale) according to research from ETS.

Note, also, that Korean students experience a similar benefit, though it is not as large as the one experienced by Chinese students.

How Do Teachers “Game the System”?

In case you are curious, here’s how this kind of TOEFL prep works.

If we assume that almost every prompt can be supported with an argument about health, we teach the student to begin their first body paragraph with a topic sentence like this:

“To begin with, _________ can improve our overall physical condition.”  

They just need to fill in the blank with their choice from the prompt.

Looking at some of my sample questions, this works quite often:

  • To begin with, the widespread use of the Internet can improve our overall physical condition.”
  • To begin with, technologies in the modern world can improve our overall physical condition.”
  • To begin with, living in the country can improve our overall physical condition.”
  • To begin with, modern diets can improve our overall physical condition.”

Of course, not EVERY question can be answered with a comment about health, but if you teach the student five or six different sentences, you can cover most of the prompts.  Like:

  • To begin with, ________ can improve our career prospects.
  • To begin with, ________ can improve our creativity.
  • To begin with, ________ can improve our relationships with our loved ones.

And so on.  These will all sound a bit clunky when used (like the four above do) but they will be grammatically correct and will please the e-rater.

After that, we teach the student a totally memorized sentence to function as “explanatory” content immediately after that sentence.  These just make some generalized sentences about the topic.  Something like:

  • “Most educated people agree that we cannot achieve anything in life without a body that is strong and healthy.”
  • “It is undeniable that our overall quality of life is strongly affected by how much success we enjoy in our career.”
  • “It is undeniable that we will live longer and more prosperous lives if we are imaginative.”
  • “In my culture, everyone feels that maintaining close connections with loved ones is more important than anything else.”

Just one blandly generic sentence isn’t enough to get the student flagged for being off-topic.  The student must depend on their own ability to write the rest of the essay, but if they use this technique in both body paragraphs, 15 or 20 percent of their essay will be written in perfect English.  That’s a nice start.

Some students might use even more memorized content, but of course that increases the risk of being flagged as off-topic.

Does ETS Know?

Yeah.  They say:

“Another possible explanation for the greater discrepancy between human and machine scores for essays from mainland China may be the dominance of coaching schools in mainland China that emphasize memorizing large chunks of text that can be recalled verbatim on the test. Human raters may assign a relatively low score if they recognize this memorized text as being somewhat off topic, though not so far off topic as to generate a score of 0. On the other hand, this grammatical and well-structured memorized text would receive a high score from e-rater. Although automated scoring engines can be trained to identify text that is not at all related to the assigned topic, they may not yet be sensitive enough to recognize this slightly off topic text.”

No shit. What surprises me, as a teacher, is that after saying this they just leave it hanging.  Neither a solution nor a response is really offered.  

They do say that:

“each essay is scored by at least one human plus e-rater. Second, if there is a discrepancy of 1.5 or more points (on the 0–5 score scale) between the human score and the e-rater score, an additional human score is obtained. The item score is then the mean of the three scores (2 human plus e-rater) unless one score is an outlier (more than 1.5 points discrepant), in which case the outlier is discarded and the remaining two scores are averaged.

But this is meaningless, as a 1.5 difference is huge.  That’s a nine point difference on the 30-point scale.  By using memorized content only in the independent task, a student could get a bonus of 2.25 points overall in the writing section without any alarm bells being sounded. 

This is weird, because ETS uses a much better system to prevent this from being a problem in the GRE:

The GRE uses a highly conservative approach in which the machine is used only to flag discrepant human scores to signal the need for a second human rating. Specifically, if the e-rater score rounded to the nearest whole number does not agree exactly with the first human score, then a second human score is obtained; the e-rater score is never averaged with a human score
It is also worth mentioning that ETS has licensed their e-rater (and SpeechRater) tech to the Chinese test prep website KMF which provides very low-cost access to the tech only to Chinese students.  This gives those students the opportunity to experiment with the e-rater and fine-tune the way their write their essays in order to please it.

What about the Plagiarism Warning?

No, you cannot use the exact  sentences mentioned above.  Presumably ETS can write some software that will detect that you’ve copied from this website.  However, wealthy students will just hire a teacher to write personalized content only for them.  That content will not appear anywhere online, and it won’t even be used in anyone else’s tests.  That’s how they can avoid being penalized by automatic plagiarism detection.

This is a Really Bad Thing

It is probably a bad thing that students from some ethnicities do better with the e-rater.  Especially since some ethnicities, particularly African-Americans (on the GRE) and Arabic and Hindi speaking students (on the TOEFL), do worse.

There aren’t just racial implications, but class implications as well.  As Perelman indicates (emphasis is mine):

“It is the following paragraph, however, that contains the most egregious instance of misinformation. “The primary emphasis in scoring the Analytical Writing section is on your critical thinking and analytical writing skills rather than on grammar and mechanics.” (Educational Testing Service, n.d.-a, n.d.-b). E-rater provides half the final score. Yet, e-rater does not emphasize “critical thinking and analytic writing skills.”   Indeed, it is completely oblivious to them. Its closest approximation is its highly reductive feature of development, which is calculated by the number of sentences in each paragraph and the number of paragraphs in the essay. Furthermore, grammar and mechanics compose a significant portion of the features included in e-rater’s calculations. Low-income students will believe these statements and focus on critical thinking and analytic skills. Affluent students who have taken test preparation classes, on the other hand, will be coached to provide e-rater with the proxies that will inflate their scores.

Anyways.  That’s all I have to report. Thank you for listening to my TED Talk.

 

 

I got another question from a student.  They asked:

“I have a question about the independent writing task I hope you can answer. What is the most common type of question that most test takers get in this task?  Is it multiple choice?  Advantages and Disadvantages?  Something else?”

Here’s the answer I gave:

The most common type of independent writing question is the “Agree/Disagree” style, which seems to come up about 50% of the time.  This type presents a single-sentence statement and you must state if you agree or disagree with it.

The “paired choice” question comes up about 25% of the time.  In this one, you must choose between two opposing or related options.  Like whether it is best to study in a group or to study alone.  Sometimes this takes the form of a hypothetical situation, like whether it is better for a company to donate to a museum or to donate to a children’s sports team.

The “multiple choice” question comes up about 25% of the time.  This is a lot like the paired choice, but with three options (sometimes four or five).  It is a bit tricky because with all those options the prompt can take a long time to read.

Finally, in rare cases (or maybe never) you might get a “good idea” prompt, where you are presented with a hypothetical situation, and should state whether you think it is a good idea.  For instance, you might be told that a company will forbid employees from answering emails on the weekend.

I must mention that “advantages and disadvantages” questions are not used on the test.

I heard from a student today who was confused about why their TOEFL writing score was so low.  They got 30 points in the reading section, 29 points in the listening section, 30 points in the speaking section… but just 19 points in the writing section.  I could tell just by reading their e-mail that their grammar was far above the “19 point level,” so clearly that wasn’t the cause of the low score.

So what caused their weird (and disappointing) writing score?  Here’s how I responded:

It is impossible for me to say what caused your low score, since I cannot read the essays you wrote on test day.

However, I have a few ideas:

  1. One or both of your body paragraphs in the independent essay could have been flagged as off-topic. This will result in a major penalty, even if your grammar is perfect.  This is the sort of thing that a score review might help you with.
  2. You might have written really short essays. If you follow the recommended word counts stated on the screen during the test it is hard (but not impossible) to score in the high 20s.
  3. You may have misunderstood some key lecture details in the integrated essay. Again, even with perfect grammar you can get a low score if your details are wrong.
  4. Nowadays ETS seems to be penalizing students who use made-up research or news stories to support their reasons in the independent essay.

 

These are just some of the reasons why a student with an advanced command of the English language might get a low score in the writing section. 

It is important to note that number four is a recent development.  Some students have suggested that they’ve been penalized for this, but obviously this is just speculation on their part.  That said, a recent update to the Official Guide to the TOEFL (August 2020) supports this theory.  Below is that update.

 

Hey, I finally uploaded the 2020 version of my guide to the independent TOEFL essay.  This is the fifth annual installment of this video!  In the next few days, I will update the website version to match the content here.

The new video has a slightly different template, a new sample essay and a few different ideas about grammar.  I also (finally) added an “FAQ” section to the end in order to avoid answering the same questions in the comments again and again!

I’ll create and film a video about the integrated writing task before the end of the month (I hope).

I was able to ask a few more questions at an ETS webinar. Here’s what I learned (the answers are not direct quotes):

Q: Will results come back in six calendar days or six business days now?
A: Six calendar days.

Q: How significant are pauses when students are answering questions in the speaking section?
A: They can be very significant and can affect the score a lot.

Q: Could the same human grader score all four speaking responses?
A: No.

Q: Will a new Official Guide be published in 2019?
A: No. That has not been prioritized.

Q: Could students get only NINE reading questions with a specific reading passage?
A: Yes. This will happen if a fill-in-a-table question is given.

Q: Is it okay to mention the reading first in integrated essay body paragraphs?
A: The order “does not matter.” The scoring rubric is “not that structured.”

I get this question a lot. Here’s what I think you should do:

  • Get some templates for each essay type.
  • Watch videos lectures on each question type.
  • Read as many sample essays as possible.
  • Take accurate practice tests.
  • Improve your grammar.
  • Get your practice essays graded.

Details about how to do these things are below!

Read My Guides (and Templates) for Each Writing Task

Everything I know about the writing section 

can be found in my guides to the two tasks. Here they are:

Not only will these guides show you how to write the essays, but they will tell you how the questions are structured and the patterns to look out for.

Check out My Video Series

I have also made a bunch of YouTube videos about this section. Here’s the first:

There are quite a few good grammar books available nowadays, but English Grammar in Use is still my favorite. A new fifth edition was published this year, and I will continue recommend it to students. Though it is advertised as a book for intermediate learners, I usually recommend it even to upper-intermediate and advanced students.

The 145 units in the book (and related appendices and extra exercises) make it a fairly comprehensive look at English grammar. With 34 years worth of revisions over the course of the book’s five editions some obvious thought has been put into how the information in the book is presented. It is remarkably easy to follow.

A moderate level of revision has occurred since the fourth edition was published in 2012. This seems to mostly have affected the organization of the exercises in the book, but the explanatory parts have been revised to some extent as well. On a shallower note, I’m happy to report that the illustrations in the book finally reflect a modern aesthetic – those in the fourth edition were not particularly attractive.

Overall, though, the book maintains the organization used since at least the second edition (the earliest I have in my collection). Each unit is two facing pages long. The left-hand page explains a specific grammar point, and the right-hand page includes exercises that students can work through. Answers are included at the end, though Cambridge does sell a version with those omitted. The seven appendices list verb forms, contractions, tricky spelling rules and notes about American English usage. There is also a study guide which might help students discover specific grammar points they should focus on.

Note that the units are grouped into logical chapters (present and past, modals, adjectives and adverbs, etc) rather than from “easy to difficult.” This isn’t a book that students work through from the beginning to end, but rather one where they focus on units covering their specific needs. The study guide might help, but they really need a teacher to show them the way. While the cover bills this as a “self-study guide,” it is better used as a supplement to classroom instruction or personal tutoring.

These activities pair well with Cambridge’s companion text English Grammar in Use: Supplementary Exercises. That book was updated in 2019 to match the new edition of the main text. Obviously, it provides additional practice which matches the units in the main book. Keep in mind, though, that it book lumps units together, and not always in chronological order.

Continue reading “Book Review: English Grammar in Use (Fifth Edition)”

I’m making a writing course!

First, the good news: the course will be free. It will always be free.

Next, the bad news: the course isn’t finished. It will take some time to finish.

I understand that might be a bit annoying, but as you will see, I’m going to upload the course here as I build it. That means you’ll benefit at least a little bit right away, and you can also tell me what you want from the course. If some of my lessons are unclear you will have an opportunity to let me know.

The Lessons – The Independent Essay

Here are the lessons that make up the course. Links are to those completed at this time:

  • Question types and topics
  • Basic essay templates
  • Writing a great introduction
  • Writing great body paragraphs
  • Writing a great conclusion
  • Better essays: Vocabulary strategies
  • Better essays: Discourse phrases
  • Better essays: Grammar strategies
  • Better essays: Coming up with ideas
  • Using your time wisely
  • How is the essay scored?

Resources

This course will be more rewarding if you have a copy of the Official Guide to the TOEFL. That link is to the 5th edition of the book (from 2017). You can also use the 4th edition, which is cheaper.

You might also want to pick up at least one of the two Official Test Collection books.

For additional practice tests, I recommend the ten test collection from TSTPrep. Use the coupon code “goodine10off” for a 10% discount.

Basically, the course is more rewarding if you have some real questions to practice with. The three aforementioned sources are worthwhile in this regard!

Most students have a pretty good understanding of what the two TOEFL writing questions require them to do. Everyone knows that the TOEFL independent writing question asks them to write an essay that describes their personal opinion about a given topic. Moreover, they all know that the integrated writing question asks them to compare an article about an academic topic to a lecture about the same topic. If they are smart, they even know that the lecture always opposes the reading.

But do you know that ETS (the makers of the test) use some really common patterns when designing the questions every week? That these questions have really predictable forms and structures? Today I want to talk about some of the forms that ETS uses when designing the independent essay question.

If you can learn these structures before taking the test your job might be a bit easier on test day. At the very least you will avoid being surprised by what you get. Oh, and don’t worry. I’ll talk about the integrated essay in a few days time. It is also super predictable.

Obsolete Question Styles

Before I actually describe the question styles you will get on the test, I want to talk about a few styles that aren’t actually used anymore. A problem a lot of students face is that even the Official Guide to the TOEFL mentions several styles that ETS has eliminated in recent years.

First of all, you likely won’t get a “compare and contrast” question in the independent writing section. Here’s one such question right from the Official Guide:

“Compare and contrast your life with that of your parents. Which do you think will be more satisfying for future generations.”

Yes, that is in the Official Guide (and some poorly written textbooks). However, you aren’t going to be asked to compare and contrast anything on the real test.

Next up, you won’t get a “characteristics of” question on the real test. Here’s one from the Official Guide:

“Neighbors are the people who live near us. In your opinion, what are the characteristics of a good neighbor.”

Nope! You’ll never be asked to mention the characteristics, traits of features of anything!

Third, you won’t get a “causes of” question. Here’s one of those… again, from the official guide:

“In general, people are living longer these days. Describe the causes of this phenomenon.”

That sounds like a really interesting question, but it won’t show up on the test. You will never be asked to talk about the “causes” of anything.

Okay, those are the most popular “obsolete” styles. So what will you get on the actual test?

Current Question Style #1: Agree/Disagree

The most common question type in the independent writing section is the “agree/disagree” style question. In this type you are presented with a short statement and asked if you agree or disagree with it. Here’s one:

“Do you agree or disagree with the following statement? Children should be required to follow strict rules set by their parents. Use specific reasons and examples to support your answer.”

Here’s another:

“Do you agree or disagree with the following statement? The widespread use of the Internet today has a positive effect on society. Use details and examples to support your opinion.”

Heck, here’s one more:

“Do you agree or disagree with the following statement? Sometimes it is better to not tell the truth. Use details and examples to support your opinion, dude.”

Simple, right? You are given a statement and asked if you agree or disagree with it. Just remember to actually pick a side. Don’t support both sides!

Current Question Style #2: Preference

The second most common question type is the “preference style” question. This type requires you to choose from between two opinions regarding some topic. Here’s one:

“Some people feel that high school students should be required to do volunteer work in their free time. Others think that they should spend their free time studying and preparing for classes. Which do you think is better? Use specific reasons and examples to support your answer.”

Here’s another:

“Some students prefer to study for tests alone. Others prefer to study with some of their classmates. Which do you prefer? Use specific reasons and examples to support your answer.”

Do you see what I mean? You are presented with two opinions and you must choose which one you “prefer” or “think is better.”

Current Question Style #3: Three Choices

The last question style is the “three choices” style. This one is pretty straightforward. You must choose from between three given choices. Here’s one such question:

In your opinion, which of the following strategies is the best way for a student to prepare for a successful career?

-Studying specific academic subjects
-Taking a part-time job
-Getting to know other students

Use specific reasons and examples to support your answer.

Here’s one more:

When a person is deciding what subject to focus on at college or university, which do you think is the best source of advice?

-their current classmates
-their teachers and professors
-their parents

Use specific reasons and examples to support your answer.

Final Thoughts

And that’s it. As far as I know, ETS only really uses those three question styles. If you get something different, though, please let me know. If you want to know how to actually write this essay you can check out my guide to the independent task.

You got this.