I watched a webinar provided by ETS Global called “TOEFL Writing Without Secrets.”
It was, overall, a very useful webinar. It contained insights that would be useful to both test-takers and test-prep people. But look at this reading passage used to illustrate the first TOEFL writing task (the integrated essay):
Eagle-eyed readers will recognize that as the question used to illustrate this task in the Official Guide to the TOEFL. Long-time readers will know that this isn’t what the TOEFL integrated essay actually looks like. I am too tired to once again explain how the integrated question is designed, but basically the reading has four paragraphs (not two). It has an introductory paragraph and it has three supporting body paragraphs, each with a concise argument.
This faulty question has been in the Official Guide since it was first published back in 2005. One imagines that the writers of the book were working from prototype versions of the TOEFL and didn’t have access to samples of actual test forms (’cause the test hadn’t actually been given at that point). The original practice test in the book has a similarly faulty question, by the way. It has also appeared in every new edition of the book.
What really gets my goat is that this has been sort of catastrophic for TOEFL test prep. Countless third-party books have been published that also include crappy practice questions, seemingly influenced by the above content. Overall, this makes the TOEFL a less attractive test than the IELTS. IELTS test-takers have a crystal clear picture of what that test is like, partially because of all the amazing and accurate official test collections that have been published.
I feel that had the Official Guide been a bit more accurate the third-party books would also be more accurate. And now the faulty question has even influenced the quality of an actual ETS webinar! Oh the humanity.
Apparently an updated edition of the Official Guide will soon be published (to match the new version of the TOEFL). If anyone from ETS is reading this, I implore you to touch up these sections.
ETS has published ten new sample writing for academic discussion questionson their website. That’s really wonderful news for people who plan to take the TOEFL when it is revised on July 26.
But that’s not all! Users can submit their responses to the questions and get an AI score from 0 to 5! That will help everyone predict how they will do on the real test.
The folks have ETS have indicated on LinkedIN that the collection of questions will be expanded in the future. And, as I have already reported here, ETS plans to release a test-prep app which includes new questions of every type along with AI grading for both the writing and speaking sections.
When preparing for standardized tests, people are often forced to struggle with bad study tools. Books and courses often contain inaccurate questions. People teaching those tests also struggle with this problem, obviously.
To nudge publishers and course designers in the right direction as they update their books for the revised TOEFL test, I recently teamed up with Jaimie Miller to analyze existing samples of the new TOEFL writing question and produce some guidelines for good question creation. The results of our work follow.
How to Write “Authentic” Academic Discussion Prompts for the new TOEFL iBT Writing Test
If you are producing sample activities for TOEFL iBT test-takers who need to prepare for the Academic Discussion portion of the new TOEFL iBT Writing test (added to the test starting July 26, 2023) we strongly encourage you to base your activities on the analysis that we have done of the sample activities that ETS has made available. By doing so, you’re creating material that test-takers are likely to recommend to their friends.
Test-takers have this view of the information when it is time to write their Academic Discussion response:
(You may also want to download this side-by-side comparison and breakdown of the 3 samples that ETS has released so far as a reference to guide)
Section One: Instructions
Section 1 contains the instructions, which are always the same. Only the academic department (sociology, business, political science, etc) changes. Subjects in the liberal arts seem most common, but anything could be used as long as the question is accessible. The questions are unlikely to require any specific technical knowledge. For instance, there are likely to be questions about the use of social media (which most people are broadly familiar with) but questions about something like genetically modified plants (which some people aren’t familiar with) are unlikely. There is an image of the professor.
Section Two: The Question
Section 2 includes the actual question, and some context. In sample questions now available, it ranges from 39 to 56 words. It establishes the general theme of the question and provides background information to activate schemata and give the test-taker time to adjust. There is commonly a reference to “the discussion board” to make it look realistic.
After a clear line break, the professor presents 1 or 2 questions that are visually set apart in a block. The questions are academic in tone, but not challenging. Just a slight step up from the banality of the Independent writing task. Nothing technical, nothing complicated, nothing culturally or demographically inaccessible.
When they ask an open-ended question (“What do you think is the most significant effect…?” or “Which issue would you argue is more important…?”), they follow with a simple “Why?” question that encourages the test-taker to dig into reasons and examples.
When they ask a YES/NO question (“Is advertising just a way of manipulating people…?”), they follow with a second YES/NO question that takes an opposing perspective (“… or is it an important source of information…?”).
It’s common to find a comparative or superlative adjective in the question. For examples, review ETS’s 3 samples here.
The total word count for the questions ranges from 19 to 30 words in the samples currently available.
The total word count for Section 2 probably ranges from 69 to 75 words total in the samples currently available.
Section 3: The First Response
In Section 3, the first student responds to the question. Responses in the currently available samples are 39 to 59 words. In samples currently published by ETS, the first student’s responses seem to have the following characteristics:
✅ contractions (“don’t” as opposed to “do not”) occur with limited frequency
✅ “I know / I don’t think / I think”
❌ limited use of personal examples or personal stories
❌ no abbreviations (“television” not “TV”)
✅ 1 example with a generalization that is based on EITHER:
blending plausible 2nd Person generalizations with 3rd Person generalizations (“When you are watching television, you are not moving around or exercising. This is especially true for children. When children spend a lot of time watching television, they have a greater tendency to be overweight”)
OR plausible statistics with specific details and multiple numbers (“I read that in just one year, from 2018 to 2019, the number of computers, tablets and mobile phones using ad blockers increased from 142 million to 615 million”)
✅ realistic use of capital letters for emphasis in strategic places (“I think the REAL question is…”)
(The above features may or may not appear in other items of this type)
Section Four: The Second Response
In Section 4, the second student responds to the question in 53 to 59 words. In open-ended questions (“What do you think is the most significant effect…” or “Which issue would you argue is more important…?”), they mention a new idea that Student #1 didn’t mention. For YES/NO questions (“Is advertising just manipulation… or is it a source of information?”), Student #2 argues against whatever Student #1 said.
In samples currently published by ETS, student #2’s responses seem to have the following characteristics:
✅ “I think / I disagree with…”
✅ contractions (“I’m” and “There’s” as opposed to “I am” and “There is”) occur with limited frequency
❌ abbreviations (“television” not “TV”)
✅ occasional use of adverbs (“actually”)
✅ 1 example with a generalization that is based on EITHER:
blending plausible 2nd Person generalizations with 1st Person personal stories (“Think of all the different places in the world you can experience through television! Last night, I watched a program about life in Antarctica, and it was fascinating!”)
OR plausible statistics with specific details and multiple numbers (“People can find out a lot about products from advertising. There’s plenty of evidence that people usually begin the process of making a big purchase by looking at ads and reviews… I’m going to post later about an advertisement that gave me a lot of useful information.”)
(Again, note that the above features appear in the materials currently published. They may or may not appear in future items of this type.)
The total number of words that test-takers are exposed to is probably in the range of 165 to 193.
Section Five: Participants
Section 5 is simply to note that one student is male and one student is female. Images of each student are presented along with their responses.
Section 6: The Test-Taker’s Response
The test-taker’s response is typed in Section 6. A word count is displayed on the screen.
—
If you use these guidelines to create your own content, I will be happy to link to it here. So far, decent questions can be found at:
I think the most frequently asked question about the new TOEFL writing task (“writing for an academic discussion”) is about the timing. Test-takers want to know if the ten minutes provided for the task is for both reading and writing. The answer to that comes from a Tweet by ETS:
The second task requires a limited amount of reading to provide a context for writing and help the writer form a response. The reading is part of the task, so it is included in the 10-minute response time.
So there ya go. You’ve got ten minutes to read the question, read the student responses and write your own response. That will require effective time-management. I recommend quickly skimming the responses so that you can spend most of your time writing your own answer.
I do hope that ETS recognizes that a lot of people want to know this information, and that a Tweet (a reply no less) is probably not the best way to disseminate it!
I spent a few hours fixing up my sample questions for the new TOEFL writing task. I somewhat expanded the discussions and added pictures to each speaker, just like the real test will have. I also revised the recommended template a bit so that it begins with a clear thesis statement. I am also thinking about using short paragraph breaks for fun.
Students preparing for the TOEFL often ask something like:
Should I mention the lecture first in each body paragraph? ETS wants me to mention the lecture first, right? That’s what the instructions say!
My answer is always no. You can do that if you want, but it isn’t mandatory.
I’m not sure where this misconception came from, but the truth is that ETS doesn’t prefer that test-takers mention the lecture first in each body paragraph. In the past, I’ve suggested that students check out the Official Guide to the TOEFL and the two Official TOEFL iBT Tests books where the examples usually mention the reading first in each body paragraph.
Note how the ETS material specifically recommends mentioning the reading first in each body paragraph. Note how the template suggested by ETS does the same.
You can mention the lecture first if you like (as is noted at the end of the second screenshot), but don’t assume that ETS prefers it that way.
If you can afford it, the ETS course is a helpful product. It dispels some other weird urban legends, which I might highlight here in the days ahead.
The other day, I woke up to TWO emails from random strangers letting me know that their TOEFL scores were cancelled due to something resembling an accusation of plagiarism. The emails themselves weren’t strange – I get weird emails all the time since I am the only a few people writing about the minutiae of standardized language tests online.
What’s fascinating is that back in the day (say, 2021 and earlier) I would get ONE such report each year. Now I get multiple reports of score cancellations due to plagiarism each month. Sometimes multiple reports in a single week.
I don’t have access to the data, but I suspect something is different than before.
In each case, the test-taker gets the exact same un-specific notification:
“In the quality control process, the ETS Writing staff noticed that your response(s) to the integrated/independent Writing task did not reflect a response to the assigned task. This was noticeable since the responses for which you receive a score should be your own original and independent work. Further reviews determined that a portion of your Writing response(s) contains ideas, language and/or examples found in other test taker responses or from published sources.”
No further information is provided, even when specifically requested.
In all but one case, the students have denied (to me) committing actions along these lines.
There have been suggestions that AI is used to detect plagiarism nowadays, but I haven’t gotten a confirmation of that.
I don’t know if any of this matters, but it might be interesting to test-watchers.
I recently cleaned up another one of the AI-generated TOEFL integrated writing questions and uploaded it to YouTube with a complete reading, lecture and sample essay. Enjoy!
In a few days, I will give a presentation about the TOEFL independent writing task at an event hosted by ETS Global. It is designed for teachers, but students can certainly benefit as well. Sign up over here.
Okay, everyone is probably sick of reading these… but I’ve got a couple more writing questions I made with ChatGPT. I’ll post them both here, and then start dumping my creations on YouTube or something. But I will use the blog to share whatever I can come up with for the other section. Sorry!
Here’s a reading about the colonization of asteroids:
The idea of colonizing asteroids has long been a topic of fascination and speculation in science fiction and popular culture. In recent years, however, the idea of asteroid colonization has become more realistic and feasible, thanks to advances in technology and space exploration. There are many potential benefits to colonizing asteroids, and these benefits make the pursuit of asteroid colonization a worthwhile endeavor.
One of the most obvious benefits of asteroid colonization is the scientific potential. Asteroids provide a unique environment for scientific research due to their small size, low gravity, and lack of atmosphere. Research conducted in these environments could provide valuable insights related to many different academic fields. Not only that, but asteroids could also serve as stepping stones for future missions to other destinations in the solar system, such as Mars or the moons of Jupiter and Saturn.
Another benefit of asteroid colonization is the potential for economic growth and development. Natural resources mined on asteroids could create new industries and job opportunities, which would generate significant revenue for both governments and private companies. Asteroid mining could also reduce the need for resource extraction on Earth, which could help to preserve our planet’s natural environment.
In addition to economic benefits, asteroid colonization could also have important implications for the long-term survival of humanity. Asteroids could serve as potential refuge for humans in the event of a large-scale disaster on Earth, such as an asteroid impact or a nuclear war. Even if such a disaster doesn’t occur, asteroids could provide valuable information regarding the origins and evolution of the solar system, and how life emerged here in the first place.
Here is me reading the corresponding lecture:
And here is a transcript of the lecture:
While some people argue that starting colonies on asteroids could have many benefits, this claim is not necessarily supported by evidence. In fact, there are several reasons why starting colonies on asteroids may not provide the advantages that are often claimed.
One reason why asteroid colonization may not provide significant scientific benefits is our lack of knowledge about asteroids. Despite decades of space exploration and study, our knowledge of asteroids is still limited. We don’t know how many asteroids exist, what resources they contain, or what conditions are like on their surfaces. Until we have a better understanding of asteroids, it’s very difficult to predict what scientific benefits colonization might provide.
Next, we can’t exactly predict the economic benefits of settling on asteroids. While asteroids are known to contain valuable resources, such as metals and minerals, it’s not clear how much of these resources exist on asteroids, or how much they would be worth when transported back to Earth. It’s not clear whether mining for resources in space would be more cost-effective or efficient than simply getting them through conventional methods here on earth.
Finally, asteroids are just not suitable for long-term human settlement right now. Most asteroids are small and do not have enough mass to generate significant gravitational pull. This means that any human settlements on asteroids would need to be designed to cope with the challenges of low gravity, such as serious difficulty moving around and possible damage to our bones and muscles. These challenges could make living on asteroids extremely uncomfortable and dangerous in the long run.
