I can’t believe I’ve been writing this column for twenty months!  Remember, I’m always happy to receive your recommendations.  I’ve tracked down a few books recommended by readers over the past year and a half.

I’ll get started with a few article recommendations.  First up, I read the January 31 issue of The New Yorker.  A few things stood out:

  • America’s Favorite Pickup Truck Goes Electric is about the release of a new electric vehicle in the United States.  In the usual New Yorker fashion it meanders through a rough history of the Ford Motor Company and about the automobile industry in general.  I found it fascinating, and it is about a topic that is certainly attracting quite a lot of attention nowadays.
  • Invasion of the Pacific Football Fish! is about a sighting of a peculiar type of anglerfish.   Unlike most New Yorker articles, this one is similar in length to a real TOEFL article.  Actually, I think the anglerfish has made a few appearances on the TOEFL over the past year.  I might have created a few questions about it myself.

Next, I peeked at the February 7 issue of the same magazine.  One article stood out:

  • Can Germany Show Us How to Leave Coal Behind? is another long article that mixes a technical topic with a social history.  There is a lot to dig into here.  The New Yorker articles I’ve linked to over the past couple months are quite long, so you might want to read them over the course of a day or two.

I also read the April 2022 issue of Reader’s Digest (Asia). One article seems relevant:

  • The Farmer Trying to Save Italy’s Ancient Olive Trees is about efforts to prevent the spread of a bacteria that is killing olive trees.  TOEFL veterans will know that this sort of topic is really common on the test, especially in the integrated writing section.  If you read only one article from today’s column, this one should be it. Note that I’ve linked to the original source of the article, which is Atlas Obscura.

Lastly, I read the memoir “Hillbilly Elegy” by J.D Vance.  In it, Vance talks about growing up in poverty in rural America and how he improved his life through education and personal growth. This one attracted a ton of attention five years ago when people were seeking answers to why America went crazy in 2016.  People claim this book  has some profound insights into why that happened, but I didn’t find them.  It is a good book, though, and if ETS wasn’t so conservative they could do an amazing reading about the “Hillbilly Migration.”  The subject matter here overlaps a little with that of “Educated” which I wrote about a few months ago.  You can borrow a copy from the Open Library or buy your own from Amazon

After a very long delay, the IRS finally processed the 990 form submitted by the Educational Testing Service for the year ending September 2020.  If you want to know where all the money goes (and comes from) you can check it out via ProPublica.  A few highlights:

  • Revenue was about 1.02 billion dollars, a 20% decrease from the previous year.
  • Revenue less expenses was about -70 million dollars.
  • Total assets under control of ETS were valued at about 1.8 billion dollars.
  • The top earner was president Walt Macdonald, with a total compensation of about 1.4 million dollars.
  • The Chauncey Hotel and Conference center brought in revenues of 1.5 million dollars.
  • About $47,000 was spent on grassroots lobbying.
  • The value of ETS’s land (not counting the buildings) was listed at $1.9 million.
  • The value of ETS’s investments in Central America and the Caribbean is listed at about 171 million dollars.  I chuckled.  You bet I did.

As always, I don’t mean to make light of anything. I just feel it is critically important that we pay close attention to our nation’s wonderful non-profits and the valuable work they do.

Meanwhile, if you want a spoiler for the 2021 forms, you can already read the independent audit of ETS for that year

The ETS research department has a report about whether essays rated on desktop computers get different scores than those rated on iPads.  The result of the research: they get the same scores.

This is why I love ETS.  Most organizations would just tell the raters to use whatever they want.  ETS, though, studied the matter very carefully.  I respect that.

As always, readers will find the most interesting details buried deep in the report. 

In this case, we learn that the raters participating in the study scored GRE essays.  We also learn that they scored twenty essays in one hour.  That’s three minutes per essay… including the time needed to queue up each essay, read it, click a button to submit the score and blink a few times before moving on to the next one. The report also indicates:

The 20 essay ratings per device were only a fraction of the number of essays a typical rater would score in a day

That tracks with what I’ve heard from former ETS raters, but I don’t think I’ve ever seen it in print. Obviously, the time taken to score TOEFL essays could be longer, but I suspect the workflow is similar.

I don’t know if this is useful information, but it is always nice to get a peek behind the curtain.  Now and then.

 

E2 Test prep has licensed an old TOEFL test for distribution on YouTube.  You can find the video and opt-in for some “bonus content” at this link.

I recognize parts of the old test from operational use a long time ago, but I don’t think it has been licensed to anyone else.  It isn’t part of the big TPO collection ETS sells to Chinese schools.

Be wary if you download the bonus content.  Some of it is out of date stuff that ETS doesn’t use on the real TOEFL anymore.  I think it is content that was cut from the test so that it matched the revisions in 2019.  You won’t get an open-ended speaking question, nor a campus problem speaking question.  You probably won’t get a “fill in a table” reading question.

 

As I mentioned earlier, the TOEFL writing rubrics are notoriously difficult to understand. Perhaps the most difficult part is the requirement that score-five and score-four independent essays demonstrate “syntactic variety” and that score-three essays include a “limited range of syntactic structures.”

What the heck is syntactic variety?  What is a syntactic structure?

Here’s what you should know:

Often I see essays that are quite long and have perfect grammar.  But I still can’t give them a perfect score.  This is because the sentences and clauses are all very similar.  Sometimes the student just uses simple sentences.  Sometimes they use too many compound sentences. Sometimes every sentence starts with a transitional adverb.  Sometimes every sentence starts with a pronoun.  That kind of writing is boring and lacks variety.

Syntax is the arrangement of words into sentences, clauses and phrases.  We don’t just put words anywhere.  They have to be arranged properly to convey meaning, and for our sentences to be considered correct.  Of course you know that.

Syntactic variety” refers to the use of various types of sentences, clauses and phrases. 

Sentence Types

The best way to ensure that your TOEFL essay has syntactic variety is to use the three main sentence types in English: simple, compound, and complex sentences.  You may already be familiar with these.  If not, start studying.

Simple sentences look like this:

Simon took the math test.  He was totally unprepared for it.

Compound sentences look like this:

Simon took the math test, but he was totally unprepared for it.

Complex sentences look like this:

Even though Simon took the math test, he was totally unprepared for it.

Note that complex sentences seem to be most important for the purposes of establishing syntactic variety and complexity.

Beyond Sentence Types – Noun Clauses, Adverb Clauses and Adjective Clauses

You can further increase your syntactic variety through the use of noun, adverb and adjective clauses.

Noun Clauses

A noun clause is a group of words that functions like a noun. They often start with “how” or a “wh-” word.  Like:

Why she didn’t call me is a mystery.

What I did that day surprised my family.

She listened to whatever I suggested.

These demonstrate more variety and complexity than writing:

That is a mystery.

This surprised my family.

She listened to my ideas.

Placing a noun clause in the subject position of a sentence may be considered a sign of more mature and complex writing.

Adverb Clauses

An adverb clause is a group of words that functions as an adverb.  Like adverbs, they usually describe how we do things.  Like:

With great enthusiasm, I finished the project.

Before doing anything else, Matthew turned on his computer.

These are a bit more impressive than:

“Quickly, I finished the project.”

“Eagerly, Matthew turned on his computer.”

Adjective Clauses

An adjective clause (also called a relative clause) is a group of words that functions like an adjective.  It describes a noun in a sentence.  Like:

“The test, which I have taken five times, is extremely difficult.”

“My friend Simone, who is three years older than me, is currently a university freshman.”

Don’t go Crazy

Remember that your essay might only be 20 sentences in total.  You don’t have to do all of these things.  Just include a few compound sentences and a few complex sentences.  Try to work in a few of the above clauses along the way.  

Other Things

There are other ways to achieve syntactic variety. Standardized tests that have a more human touch explicitly mention some of them in their grading rubrics.  Consider the ALP Essay Test from Colombia University, which specifically mentions such techniques as:

  • Inversion
  • Noun clauses in subject position
  • adverb/adjective/noun clauses
  • Appositives
  • Parallelism 

 

The TOEFL writing rubrics are famously difficult to understand. Even experienced teachers have a hard time turning them into something that students can actually make use of. Today’s blog post will kick off a series that attempts to explain what the rubrics actually refer to.  Starting with…

Idiomaticity

References in the rubric to “idiomaticity” and “idiomatic language” are particularly difficult to grasp. The rubric says that a score-five independent essay should “display appropriate word choice and idiomaticity.” Meanwhile, it notes that a score-four essay should have only “minor errors” in its “use of idiomatic language.”

But what does this actually mean?

Many students (and teachers) think that ETS wants test-takers to use idioms like “it was raining cats and dogs last week” or “I won’t beat around the bush.” That is not correct. That’s a different matter.

“Idiomaticity” is tough to define, but the dictionary definition is best. It says that idiomaticity is “the extent to which a learner’s language resembles that of a native speaker.”

This is what your teachers are hinting at when they change one of your sentences not because of a specific grammar error, but because they think some of your word choices don’t seem natural.

Here’s a sentence I recently read:

“Business owners want employees to make quick decisions, which renders stress for those who take their time.”

There aren’t any grammar errors in that sentence. But “renders” sounds weird to me. Changing that to “causes” or “creates” will increase the idiomaticity of the sentence.

Here’s another one:

“When the shopping mall opened, many local shops ceased their business.”

That’s a lot more subtle. “Ceased their business” is pretty good, but it is a little bit awkward. A native speaker would probably say something like “went out of business.”

I would even complain about something like:

“I strongly think that children should attend all of their classes”

My preferred phrasing would be something like “I strongly believe…”.

You might think I’m being needlessly picky, but to get a perfect score (5 on the rubric, 30 scaled) you need to use the best possible words at all times.

In TOEFL essays, problems related to idiomaticity seem to come from two sources:

  1. Inexperience with the language.
  2. A desire to shove a lot of fancy words into the essays to get a higher score.

The first source is normal. No one is perfect. You can overcome this by studying. Read sample essays. Get feedback on your own writing. Try studying with a collocations book like “Collocations in Use.” Consider using a learner’s dictionary.

The second source is not normal. Ignore advice from inexperienced teachers who think that using obscure words will help you. They won’t. Some of the essays I’ve read come pretty close to Noam Chomsky’s famous “colorless green ideas sleep furiously”. That’s a beautiful sentence, but no meaning can be derived from it.

This month’s issue of “Language Testing” includes a review of IELTS.  Like… the whole test.  Test reviews are the best part of that journal, and this one is no exception.  Check it out for an overview of the test, its history and its validity argument.

It isn’t exactly a glowing review, as the author raises some interesting questions regarding the validity of the IELTS as a test for immigration in four countries with multiple classes of immigrants which sometimes experience abrupt changes in immigration policy.  The author also highlights some of the compromises that have been made in allowing the academic and general training modules to share speaking and listening sections.

The review also highlights how testing organizations “can lose control of how their test is used” by discussing how Australian immigration officials have gradually cranked up score requirements mostly to control the number of visa granted (instead of attempting to reflect the level of English needed to participate in society).

The author seems to have a mixed opinion of the validity of each of the specific sections of the test.

It isn’t all negative.  I don’t want you to think that. There are some kind words about the potential for positive washback here.

What I find most striking is that the IELTS is, to some extent, a test stuck in the late 1980s.  Consider how ETS managed to modernize itself by introducing the TOEFL CBT in 1998 and the TOEFL iBT in 2005. Consider also how they are currently (and successfully) trying to do the same thing with at-home testing.  IDP, in contrast, seems much slower to adapt to the changing world.

As I reported yesterday, ETS (formerly the Educational Testing Service) is seeking a new executive director for the Office of Testing Integrity.  If I was to advise the incoming director I would recommend the following changes.

1. Staff up. Staff way up. Administrative review for TOEFL tests is supposed to finish in 2-4 weeks. I often hear from students who have waited for much longer. One student who spoke to me recently waited for 102 days.  Update (September, 2022): the student is still waiting.  197 days now.

2. Help test-takers help themselves. I often hear from students who have experienced score cancellations due to unauthorized software running in the background. Remember that in the Windows 10+ era it is a lot harder to control what goes on in the background of our systems than it used to be. Needless to say, modern versions of Windows are built in a way that makes remote proctoring a challenge. Duolingo recently produced a little video showing students a few ways to avoid such problems. The OTI should have made the same sort of content two years ago.

3. Reconsider the use of statistical data as a justification for score cancellations. There are very valid reasons why a student might, for example, have a speaking score much lower than their listening score. Some of those reasons are cultural. Think about that for a moment.

As always, ETS, you know how to reach me. In lieu of a consulting fee I’m willing to accept meal vouchers for the ETS cafeteria.

ETS is seeking to recruit a new Executive Director for the Office of Testing Integrity.  For many years (decades?) the OTI was a little-known office within ETS that only attracted notice once every few years when a cheating scandal popped up, usually on the SAT.  Now, in the era of remote proctoring and at-home testing, the OTI is known to a lot of people.  It is home to the staff that send out reports of TOEFL scores being “on hold.”  Perhaps some innovation within the OTI is called for.

Ray Nicosia has held the position of executive director of the OTI for 27 years and he did something else at ETS for six years prior to getting the job.  The New York Times once referred to him as “the SAT’s top cop.”  I don’t know if Ray is retiring from ETS or just moving to another office.   I met him once, very briefly, at a presentation he gave in Seoul some years ago.  Actually, I recall that on his way to talk to much more important people he actually stopped and asked me where I was from!  As readers of the blog know, I like to keep a low profile… but the top cop surely had a sixth sense!

I kid, of course.  It is worth mentioning that Ray seemed like a cool dude.  And his presentation about test security was the best part of the whole seminar.  For real.

A pharmacist sent me a message like: “I’ve taken 30 hours of classes with Mr. Smith, Ms. Jones and Dr. Ford. But my reading score hasn’t improved. Can you teach me?”

I said that I couldn’t teach her because there was nothing left to teach. After 30 hours of lessons, she probably got everything that a teacher could impart. There aren’t any secrets or magic tricks to getting a high score. What could I possibly tell her that Smith, Jones and Ford hadn’t already said?

I suggested, instead, that she build her comprehension through individual self-study.

Try this:

Every day, pick one TOEFL reading from a reliable source (just one article, not a whole section). Read it very slowly and closely. Every time you find a new word, write it down in a notebook with its definition. Use a good dictionary (COBUILD Advanced Learner’s Dictionary is great). Read it a few times until you totally understand the content.

When you finish, note the subject of the article: history, biology, zoology, anthropology, whatever.

After that, look at the ten questions really closely. Examine each one and all of the answer choices. Even if you can easily find the answer, study all of the incorrect choices and make notes about why each one is incorrect. Maybe a choice is incorrect because it includes a detail not mentioned. Maybe it is an issue of chronology. Whatever it is… just make a note of it and then move to the next one. This will encourage even closer and even deeper readings of the articles.

Do that for an hour every day for the next six months and your overall reading comprehension will likely improve. So will your score. You don’t need a teacher to do this, and you likely have access to a near-infinite supply of TOEFL reading articles.

Hey, so I learned how the “Mentor’s Guide to the TOEFL” works.  Here’s what you should know:

  • If a teacher emails ETS and supplies the names and email addresses of ten students, ETS will set up a course the students can complete via the edX platform.  The teacher will be added to the course as an admin.
  • The course is almost identical to the existing edX course that students can do on their own.
  • It does include a new practice test.  That appears to be one of the “Test Practice Online” sets that ETS has licensed to schools in China, but it will be new to everyone outside of China.
  • If the students pay a $50 fee (each) to edX, the teacher can track their progress through the course.  If they do not pay that fee, the teacher cannot track their progress.
  • The platform includes a message board for course participants to you.

That’s about it.  I tried to get some students to enroll, but they had already completed the edX content and weren’t interested in doing it again.

An update for those waiting for the IELTS Online (aka the home edition). The IELTS website has been updated and now saysIELTS Online will be available in 2022 in limited countries.”

It used to say “IELTS Online will be available from early 2022.

I think we can read this as a delay to the roll-out of the new test.

Also notable is that the website now says “you will receive your results 3-6 days after taking IELTS Online.

The original version said “you will receive your results 3-5 days after taking IELTS Online.”