According to last week’s SEC filings, revenue for the Duolingo English Test in Q1 2022 was $8,080,000. That’s almost exactly the same as Q4 2021. It is up about three million dollars from Q1 of 2021.
At $49 a pop, we might extrapolate that the test was taken 165,000 times in the quarter. The actual number is probably a little bit higher than that due to discounts and free tests.
The list of countries for which ETS will collect local sales taxes on TOEFL registrations and fees is now: Australia, Canada, Colombia, Nigeria, Turkey, United States, Vietnam, and Singapore.
Canada, Vietnam and Singapore are new to the list this month.
Reports indicate that ETS has suspended TOEFL and GRE tests in Iran until further notice. The reasons are unclear. Does anyone have information about this?
Students were notified about cancellations about ten hours ago.
Update: According to reports, ETS is awaiting the renewal of its OFAC license. When that has been renewed, testing will resume in Iran. I don’t know when that will happen.
I can’t believe I’ve been writing this column for twenty months! Remember, I’m always happy to receive your recommendations. I’ve tracked down a few books recommended by readers over the past year and a half.
I’ll get started with a few article recommendations. First up, I read the January 31 issue of The New Yorker. A few things stood out:
America’s Favorite Pickup Truck Goes Electric is about the release of a new electric vehicle in the United States. In the usual New Yorker fashion it meanders through a rough history of the Ford Motor Company and about the automobile industry in general. I found it fascinating, and it is about a topic that is certainly attracting quite a lot of attention nowadays.
Invasion of the Pacific Football Fish! is about a sighting of a peculiar type of anglerfish. Unlike most New Yorker articles, this one is similar in length to a real TOEFL article. Actually, I think the anglerfish has made a few appearances on the TOEFL over the past year. I might have created a few questions about it myself.
Next, I peeked at the February 7 issue of the same magazine. One article stood out:
Can Germany Show Us How to Leave Coal Behind? is another long article that mixes a technical topic with a social history. There is a lot to dig into here. The New Yorker articles I’ve linked to over the past couple months are quite long, so you might want to read them over the course of a day or two.
I also read the April 2022 issue of Reader’s Digest (Asia). One article seems relevant:
The Farmer Trying to Save Italy’s Ancient Olive Trees is about efforts to prevent the spread of a bacteria that is killing olive trees. TOEFL veterans will know that this sort of topic is really common on the test, especially in the integrated writing section. If you read only one article from today’s column, this one should be it. Note that I’ve linked to the original source of the article, which is Atlas Obscura.
Lastly, I read the memoir “Hillbilly Elegy” by J.D Vance. In it, Vance talks about growing up in poverty in rural America and how he improved his life through education and personal growth. This one attracted a ton of attention five years ago when people were seeking answers to why America went crazy in 2016. People claim this book has some profound insights into why that happened, but I didn’t find them. It is a good book, though, and if ETS wasn’t so conservative they could do an amazing reading about the “Hillbilly Migration.” The subject matter here overlaps a little with that of “Educated” which I wrote about a few months ago. You can borrow a copy from the Open Library or buy your own from Amazon.
After a very long delay, the IRS finally processed the 990 form submitted by the Educational Testing Service for the year ending September 2020. If you want to know where all the money goes (and comes from) you can check it out via ProPublica. A few highlights:
Revenue was about 1.02 billion dollars, a 20% decrease from the previous year.
Revenue less expenses was about -70 million dollars.
Total assets under control of ETS were valued at about 1.8 billion dollars.
The top earner was president Walt Macdonald, with a total compensation of about 1.4 million dollars.
The Chauncey Hotel and Conference center brought in revenues of 1.5 million dollars.
About $47,000 was spent on grassroots lobbying.
The value of ETS’s land (not counting the buildings) was listed at $1.9 million.
The value of ETS’s investments in Central America and the Caribbean is listed at about 171 million dollars. I chuckled. You bet I did.
As always, I don’t mean to make light of anything. I just feel it is critically important that we pay close attention to our nation’s wonderful non-profits and the valuable work they do.
The ETS research department has a report about whether essays rated on desktop computers get different scores than those rated on iPads. The result of the research: they get the same scores.
This is why I love ETS. Most organizations would just tell the raters to use whatever they want. ETS, though, studied the matter very carefully. I respect that.
As always, readers will find the most interesting details buried deep in the report.
In this case, we learn that the raters participating in the study scored GRE essays. We also learn that they scored twenty essays in one hour. That’s three minutes per essay… including the time needed to queue up each essay, read it, click a button to submit the score and blink a few times before moving on to the next one. The report also indicates:
The 20 essay ratings per device were only a fraction of the number of essays a typical rater would score in a day
That tracks with what I’ve heard from former ETS raters, but I don’t think I’ve ever seen it in print. Obviously, the time taken to score TOEFL essays could be longer, but I suspect the workflow is similar.
I don’t know if this is useful information, but it is always nice to get a peek behind the curtain. Now and then.
E2 Test prep has licensed an old TOEFL test for distribution on YouTube. You can find the video and opt-in for some “bonus content” at this link.
I recognize parts of the old test from operational use a long time ago, but I don’t think it has been licensed to anyone else. It isn’t part of the big TPO collection ETS sells to Chinese schools.
Be wary if you download the bonus content. Some of it is out of date stuff that ETS doesn’t use on the real TOEFL anymore. I think it is content that was cut from the test so that it matched the revisions in 2019. You won’t get an open-ended speaking question, nor a campus problem speaking question. You probably won’t get a “fill in a table” reading question.
As I mentioned earlier, the TOEFL writing rubrics are notoriously difficult to understand. Perhaps the most difficult part is the requirement that score-five and score-four independent essays demonstrate “syntactic variety” and that score-three essays include a “limited range of syntactic structures.”
What the heck is syntactic variety? What is a syntactic structure?
Here’s what you should know:
Often I see essays that are quite long and have perfect grammar. But I still can’t give them a perfect score. This is because the sentences and clauses are all very similar. Sometimes the student just uses simple sentences. Sometimes they use too many compound sentences. Sometimes every sentence starts with a transitional adverb. Sometimes every sentence starts with a pronoun. That kind of writing is boring and lacks variety.
Syntax is the arrangement of words into sentences, clauses and phrases. We don’t just put words anywhere. They have to be arranged properly to convey meaning, and for our sentences to be considered correct. Of course you know that.
“Syntactic variety” refers to the use of various types of sentences, clauses and phrases.
Sentence Types
The best way to ensure that your TOEFL essay has syntactic variety is to use the three main sentence types in English: simple, compound, and complex sentences. You may already be familiar with these. If not, start studying.
Simple sentences look like this:
Simon took the math test. He was totally unprepared for it.
Compound sentences look like this:
Simon took the math test, but he was totally unprepared for it.
Complex sentences look like this:
Even though Simon took the math test, he was totally unprepared for it.
Note that complex sentences seem to be most important for the purposes of establishing syntactic variety and complexity.
You can further increase your syntactic variety through the use of noun, adverb and adjective clauses.
Noun Clauses
A noun clause is a group of words that functions like a noun. They often start with “how” or a “wh-” word. Like:
Why she didn’t call me is a mystery.
What I did that day surprised my family.
She listened to whatever I suggested.
These demonstrate more variety and complexity than writing:
That is a mystery.
This surprised my family.
She listened to my ideas.
Placing a noun clause in the subject position of a sentence may be considered a sign of more mature and complex writing.
Adverb Clauses
An adverb clause is a group of words that functions as an adverb. Like adverbs, they usually describe how we do things. Like:
With great enthusiasm, I finished the project.
Before doing anything else, Matthew turned on his computer.
These are a bit more impressive than:
“Quickly, I finished the project.”
“Eagerly, Matthew turned on his computer.”
Adjective Clauses
An adjective clause (also called a relative clause) is a group of words that functions like an adjective. It describes a noun in a sentence. Like:
“The test, which I have taken five times, is extremely difficult.”
“My friend Simone, who is three years older than me, is currently a university freshman.”
Don’t go Crazy
Remember that your essay might only be 20 sentences in total. You don’t have to do all of these things. Just include a few compound sentences and a few complex sentences. Try to work in a few of the above clauses along the way.
Other Things
There are other ways to achieve syntactic variety. Standardized tests that have a more human touch explicitly mention some of them in their grading rubrics. Consider the ALP Essay Test from Colombia University, which specifically mentions such techniques as:
The TOEFL writing rubrics are famously difficult to understand. Even experienced teachers have a hard time turning them into something that students can actually make use of. Today’s blog post will kick off a series that attempts to explain what the rubrics actually refer to. Starting with…
Idiomaticity
References in the rubric to “idiomaticity” and “idiomatic language” are particularly difficult to grasp. The rubric says that a score-five independent essay should “display appropriate word choice and idiomaticity.” Meanwhile, it notes that a score-four essay should have only “minor errors” in its “use of idiomatic language.”
But what does this actually mean?
Many students (and teachers) think that ETS wants test-takers to use idioms like “it was raining cats and dogs last week” or “I won’t beat around the bush.” That is not correct. That’s a different matter.
“Idiomaticity” is tough to define, but the dictionary definition is best. It says that idiomaticity is “the extent to which a learner’s language resembles that of a native speaker.”
This is what your teachers are hinting at when they change one of your sentences not because of a specific grammar error, but because they think some of your word choices don’t seem natural.
Here’s a sentence I recently read:
“Business owners want employees to make quick decisions, which renders stress for those who take their time.”
There aren’t any grammar errors in that sentence. But “renders” sounds weird to me. Changing that to “causes” or “creates” will increase the idiomaticity of the sentence.
Here’s another one:
“When the shopping mall opened, many local shops ceased their business.”
That’s a lot more subtle. “Ceased their business” is pretty good, but it is a little bit awkward. A native speaker would probably say something like “went out of business.”
You might think I’m being needlessly picky, but to get a perfect score (5 on the rubric, 30 scaled) you need to use the best possible words at all times.
In TOEFL essays, problems related to idiomaticity seem to come from two sources:
Inexperience with the language.
A desire to shove a lot of fancy words into the essays to get a higher score.
The second source is not normal. Ignore advice from inexperienced teachers who think that using obscure words will help you. They won’t. Some of the essays I’ve read come pretty close to Noam Chomsky’s famous “colorless green ideas sleep furiously”. That’s a beautiful sentence, but no meaning can be derived from it.
This month’s issue of “Language Testing” includes a review of IELTS. Like… the whole test. Test reviews are the best part of that journal, and this one is no exception. Check it out for an overview of the test, its history and its validity argument.
It isn’t exactly a glowing review, as the author raises some interesting questions regarding the validity of the IELTS as a test for immigration in four countries with multiple classes of immigrants which sometimes experience abrupt changes in immigration policy. The author also highlights some of the compromises that have been made in allowing the academic and general training modules to share speaking and listening sections.
The review also highlights how testing organizations “can lose control of how their test is used” by discussing how Australian immigration officials have gradually cranked up score requirements mostly to control the number of visa granted (instead of attempting to reflect the level of English needed to participate in society).
The author seems to have a mixed opinion of the validity of each of the specific sections of the test.
It isn’t all negative. I don’t want you to think that. There are some kind words about the potential for positive washback here.
–
What I find most striking is that the IELTS is, to some extent, a test stuck in the late 1980s. Consider how ETS managed to modernize itself by introducing the TOEFL CBT in 1998 and the TOEFL iBT in 2005. Consider also how they are currently (and successfully) trying to do the same thing with at-home testing. IDP, in contrast, seems much slower to adapt to the changing world.
As I reported yesterday, ETS (formerly the Educational Testing Service) is seeking a new executive director for the Office of Testing Integrity. If I was to advise the incoming director I would recommend the following changes.
1. Staff up. Staff way up. Administrative review for TOEFL tests is supposed to finish in 2-4 weeks. I often hear from students who have waited for much longer. One student who spoke to me recently waited for 102 days.
2. Help test-takers help themselves. I often hear from students who have experienced score cancellations due to unauthorized software running in the background. Remember that in the Windows 10+ era it is a lot harder to control what goes on in the background of our systems than it used to be. Needless to say, modern versions of Windows are built in a way that makes remote proctoring a challenge. Duolingo recently produced a little video showing students a few ways to avoid such problems. The OTI should have made the same sort of content two years ago.
3. Reconsider the use of statistical data as a justification for score cancellations. There are very valid reasons why a student might, for example, have a speaking score much lower than their listening score. Some of those reasons are cultural. Think about that for a moment.
As always, ETS, you know how to reach me. In lieu of a consulting fee I’m willing to accept meal vouchers for the ETS cafeteria.
ETS is seeking to recruit a new Executive Director for the Office of Testing Integrity. For many years (decades?) the OTI was a little-known office within ETS that only attracted notice once every few years when a cheating scandal popped up, usually on the SAT. Now, in the era of remote proctoring and at-home testing, the OTI is known to a lot of people. It is home to the staff that send out reports of TOEFL scores being “on hold.” Perhaps some innovation within the OTI is called for.
Ray Nicosia has held the position of executive director of the OTI for 27 years and he did something else at ETS for six years prior to getting the job. The New York Times once referred to him as “the SAT’s top cop.” I don’t know if Ray is retiring from ETS or just moving to another office. I met him once, very briefly, at a presentation he gave in Seoul some years ago. Actually, I recall that on his way to talk to much more important people he actually stopped and asked me where I was from! As readers of the blog know, I like to keep a low profile… but the top cop surely had a sixth sense!
I kid, of course. It is worth mentioning that Ray seemed like a cool dude. And his presentation about test security was the best part of the whole seminar. For real.
A pharmacist sent me a message like: “I’ve taken 30 hours of classes with Mr. Smith, Ms. Jones and Dr. Ford. But my reading score hasn’t improved. Can you teach me?”
I said that I couldn’t teach her because there was nothing left to teach. After 30 hours of lessons, she probably got everything that a teacher could impart. There aren’t any secrets or magic tricks to getting a high score. What could I possibly tell her that Smith, Jones and Ford hadn’t already said?
I suggested, instead, that she build her comprehension through individual self-study.
Try this:
Every day, pick one TOEFL reading from a reliable source (just one article, not a whole section). Read it very slowly and closely. Every time you find a new word, write it down in a notebook with its definition. Use a good dictionary (COBUILD Advanced Learner’s Dictionary is great). Read it a few times until you totally understand the content.
When you finish, note the subject of the article: history, biology, zoology, anthropology, whatever.
After that, look at the ten questions really closely. Examine each one and all of the answer choices. Even if you can easily find the answer, study all of the incorrect choices and make notes about why each one is incorrect. Maybe a choice is incorrect because it includes a detail not mentioned. Maybe it is an issue of chronology. Whatever it is… just make a note of it and then move to the next one. This will encourage even closer and even deeper readings of the articles.
Do that for an hour every day for the next six months and your overall reading comprehension will likely improve. So will your score. You don’t need a teacher to do this, and you likely have access to a near-infinite supply of TOEFL reading articles.
