More Q&A with ETS

Posted on December 7, 2019 by TOEFL Resources

This week I was lucky enough to again have an opportunity to attend a workshop hosted by ETS for TOEFL teachers. Here is a quick summary of some of the questions that were asked by attendees of the workshop. Note that the answers are not direct quotes, unless indicated.

Q: Are scores adjusted statistically for difficulty each time the test is given?

A: Yes. This means that there is no direct conversion from raw to scaled scores in the reading and listening section. The conversion depends on the performance of all students that week.

Q: Do all the individual reading and listening questions have equal weight?

A: Yes.

Q: When will new editions of the Official Guide and Official iBT Test books be published?

A: There is no timeline.

Q: Are accents from outside of North America now used when the question directions are given on the test?

A: Yes.

Q: How are the scores from the human raters and the SpeechRater combined?

A: “Human scores and machines scores are optimally weighted to produce raw scores.” This means ETS isn’t really going to answer this question.

Q: Can the human rater override the SpeechRater if he disagrees with its score?

A: Yes.

Q: How many different human raters will judge a single student’s speaking section?

A: Each question will be judged by a different human.

Q: Will students get a penalty for using the same templates as many other students?

A: Templates “are not a problem at all.”

Q: Why were the question-specific levels removed from the score reports?

A: That information was deemed unnecessary.

Q: Is there a “maximum” word count in the writing section?

A: No.

Q: Is it always okay to pick more than one choice in multiple choice writing prompts?

A: Yes.

Some Q & A with ETS

Posted on October 25, 2019December 1, 2019 by Michael

I was able to ask a few more questions at an ETS webinar. Here’s what I learned (the answers are not direct quotes):

Q: Will results come back in six calendar days or six business days now?
A: Six calendar days.

Q: How significant are pauses when students are answering questions in the speaking section?
A: They can be very significant and can affect the score a lot.

Q: Could the same human grader score all four speaking responses?
A: No.

Q: Will a new Official Guide be published in 2019?
A: No. That has not been prioritized.

Q: Could students get only NINE reading questions with a specific reading passage?
A: Yes. This will happen if a fill-in-a-table question is given.

Q: Is it okay to mention the reading first in integrated essay body paragraphs?
A: The order “does not matter.” The scoring rubric is “not that structured.”

Fill in A Table Questions are Back

Posted on October 24, 2019December 4, 2019 by Michael

Well, I’ve had enough reports from students to confirm that fill in a table questions are again being used in the reading section of the TOEFL.

This is interesting because before the TOEFL changed in August of this year they had mostly disappeared.

It is important to note that when a fill in a table question is used with a reading passage, that passage will only have nine questions in total. This means that ETS’s statement that each reading passage has ten questions is not exactly correct.

There are a few things to note:

You will not get both a prose summary question and a fill in a table question with the same passage.
The prose summary question is still more common that the fill in a table question.
You can earn from zero to three points from the fill in a table question.

Regarding scoring, here’s what the Official Guide to the TOEFL says:

“You can earn up to a total of 3 points, depending on how many correct answers you select and correctly place. For zero, one, or two correct answers you will receive no points. For three correct answers you will receive 1 point; for four correct answers you will receive 2 points; and for all five correct answers you will receive the entire 3 points.”

Overall, I think this is a positive development. The fill in a table question is challenging, but it is a bit more reasonable that the other question types which sometimes seem intentionally obtuse.

How Do I Improve My TOEFL Reading Score?

Posted on October 14, 2019December 7, 2019 by Michael

Introduction

I get this question a lot. Broadly speaking, here’s what I think you should do to prepare for the TOEFL reading section:

Learn your current level
Learn how the reading section is designed
Get some accurate practice tests
Improve your reading comprehension
Get some strategies for solving questions
Hire a good tutor

Details about how to do these things are below!

Learn Your Current Level

If you haven’t taken the test already, make sure you know your current level in the TOEFL reading section. The easiest way to do this is to take the free sample test from ETS. You can also take one of the tests in the Official Guide to the TOEFL. Once you have done this you will know how much you need to improve.

Learn How the Reading Section is Designed

Okay, this might be obvious, but you need to know how the TOEFL reading section is designed. If you understand how the test is designed, you will have fewer surprises on test day. Start by checking out the practice reading set from ETS. Read that set very carefully. Pay attention to the length of the passages and the number of questions included with each passage.

Pay special attention to the types of reading questions used by ETS. Briefly, the main types are:

Factual Information
Negative Factual Information
Rhetorical Purpose
Vocabulary
Sentence Simplification
Insert a Sentence
Inference
Reference
Summary
Fill in a table

The best descriptions of these question types is found in the Official Guide to the TOEFL. Note that you don’t need to pay for the 5th edition, as every edition of the guide has pretty much the same descriptions. Read them carefully.

You can get the same descriptions and advice in the TOEFL Insider’s Guide course on edX. This is free, and is mostly video. I like it.

I recently analyzed the most recent practice materials. Read my blog post for an indication of how frequently each question type will appear on test day.

I should mention a few things before we go on:

Since August 1, ETS has used fewer vocabulary questions than before. Expect just one or two per article. In the past, you would get three to five per article.
“Fill in a table” and “reference” questions seem to be used much less than before. I used to think they were gone forever, but since posting the original version of this guide, I’ve gotten some reports that they have reappeared. Be prepared.
You might get an unexpected question like “how does paragraph 1 relate to paragraph 2” or “what function does paragraph 2 serve in the organization of the passage as a whole.” These types are not mentioned in most popular study guides. Sorry.

Continue reading “How Do I Improve My TOEFL Reading Score?” →

Analysis of New TOEFL Preparation Materials

Posted on September 22, 2019December 5, 2019 by Michael

To get a better sense of the distribution of questions on the new version of the TOEFL, I have compared the new versions of the TOEFL Reading Practice Sets released by ETS to their old versions. Note that the three sets in the above link are modified versions of the old TPO 7 and 8 sets. The articles are the same, but certain questions have been removed. Here’s what I found out about the question types on the new version of the test.

	Old Set 1	Old Set 2	Old Set 3	New Set 1	New Set 2	New Set 3
Factual Information	4	4	3	3 (-1)	3 (-1)	3
Negative Factual Information	2	2	1	1 (-1)	1 (-1)	1
Rhetorical Purpose	1	1	2	1	1	2
Vocabulary	4	4	3	2 (-2)	2 (-2)	1 (-2)
Sentence Simplification	1	1	1	1	1	1
Insert a Sentence	1	1	1	1	1	1
Inference	0	0	1	0	0	0 (-1)
Summary	1	1	1	1	1	1

This confirms my earlier reports that the new test has far fewer vocabulary questions. Factual and Negative Factual questions have also been reduced, it would seem.

This also confirms that Reference and “Fill in a Table” questions will probably not appear on the test much nowadays, as they are totally absent from the practice materials. Note that even though the single Inference question has been removed from the test, it is still being used quite frequently, according to reports.

Next up, I’ve done the same analysis of the Free Practice Test provided by ETS. The results are as follows.

	Old Set 1	Old Set 2	Old Set 3	New Set 1	New Set 2	New Set 3
Factual Information	3	1	4	2 (-1)	1	3 (-1)
Negative Factual Information	2	2	1	2	2	2
Rhetorical Purpose	0	1	1	0	1	1
Vocabulary	4	4	3	2 (-2)	2 (-2)	1 (-2)
Sentence Simplification	1	1	1	1	0 (-1)	0 (-1)
Insert a Sentence	1	1	1	1	1	1
Function of Paragraph	1	0	0	0 (-1)	0	0
Inference	1	3	1	1	2 (-1)	1
Summary	1	1	1	1	1	1

Again, we can see that there are far fewer vocabulary questions. But we can also see that all of the question types are affected, except for the Insert Sentence and Summary types.

The odd “function of paragraph” entry refers to a non-standard question that isn’t mentioned in the Official Guide or any other ETS resources. On the original set it was phrased as “What function does paragraph 3 serve in the organization of the passage as a whole?”. I guess this is sort of like a rhetorical purpose question, but it really surprises students when it comes up. Note that although it has been removed from the practice test, I have had reports that it has appeared on the real test since August 1.