Grammar and Usage

What is it?

Grammar and usage mistakes are often easy to spot. This category of errors includes missing words, extra words, and incorrect or out of order words. These should be marked as Grammar and Usage errors.

Examples

A PhD student from the University of Kent in the UK, claims to have discovered a clever way to explain the positive emoticons in cats

• Explanation: The word should probably be "emotions", not "emoticons".

A couple is facing criticism for their extravagant gender reveal party. The bewitching pair had first stripped down to fishnets and backward.

• Explanation: This sentence seems to be missing a noun after the word "backward", e.g. "backward hats". Note that we select the whole phrase "and backward" because it is the smallest span that we could delete to make the sentence sensible.

Exercise

Please mark the Grammar and Usage error or errors in the following text:

Redudant

What is it?

Redundant text repeats itself. Sometimes, you will see the exact word or phrase repeated. Other times, the same idea is repeated using different words. To annotate Redundant text, first select the extra repeating text and choose the "Redundant" error type. An additional section of the annotation box will pop up asking you to select the antecedents (earlier spans of text) that are being repeated.

Examples

Many merchants worry about the possibility of poor service or service for certain categories of customers.

• Explanation: This text includes the extra phrase “or service” that is repetitive. Note that when highlighting redundant information, please select the whole section that could be cut to make the text correct. After selecting the redundant text, please select the text that is being repeated:

Many merchants worry about the possibility of poor service or service for certain categories of customers.

• Explanation: In this example, the word "service" is the antecedent of the Redundant text.

They then made decisions based on Kondo’s instructions, to the extent that they created de-cluttered spaces and got rid of clutter and clutter-filled spaces.

• Explanation: Here, the antecedent phrase "created de-cluttered spaces" is repeated by the redundant phrase "and got rid of clutter and clutter-filled spaces". Note again that we select "and" as part of the redundant phrase because we could delete all this and the sentence would make sense.

The soap production center, which was used during Roman times (around 1,200 years ago), once produced large quantities of a quality soap that was exported throughout the empire. The remains of an ancient Roman factory that produced high-quality soap have been unearthed in Israel.

• Explanation: This is an example of sentence-level redundancy. The words of the second sentece are different from the first, but they repeat the same content.

Notice: If there are multiple redundant spans in text, mark them as multiple redundant errors. Here is an example: https://yao-dou.github.io/redundant_example/.

Exercise

Please mark the Redundant error or errors in the following text:

Off-prompt

What is it?

In this task, every example you annotate will come with a "prompt", or a piece of text written by a human that the AI is supposed to continue. Sometimes, however, the AI will write a phrase or sentence that is completely unrelated to the prompt. Other times, the text might be related, but it contradicts the prompt. In both of these instances, you should label the span Off-prompt.

Examples

Prompt: Dogs are the new kids.

Text: Statistics suggest that most Americans would be happier with dogs than children. In fact, four out of five don't even visit the dentist annually, much less every six months. Dog owners report much higher rates of happiness than non-dog owners.

• Explanation: The prompt is about dogs, but the second sentence about dentists is unrelated to the prompt.

Prompt: China sets new record for Economic Growth

Text: The Chinese economy fell 10% this month, the third such loss this year.

• Explanation: The prompt claims that China's economy is growing, but the text states the opposite. Important: even though this is a contradiction, because it contradicts the prompt, we mark it as Off-prompt.

Prompt: Increased awareness of Anti-Doping regulations in Sport

Text: The practice of doping is banned by sports federations throughout the world. Athletes need to know which substances are banned in sport. The use of drugs during music festivals is widespread. Furthermore, they must make sure that any product or medication they take does not contain a prohibited substance.

• Explanation: Here, the prompt deals with the use of drugs in sports. The third sentence of the text does talk about drugs, so the whole sentence is not off-prompt. However, the part about music festivals is unrelated to the prompt about sports. This is why we select only the span "music festivals" as Off-prompt.

Exercise

Please mark the Off-prompt error or errors in the following text:

Self-Contradiction

What is it?

Self-Contradiction errors occur when the AI writes something that contradicts another piece of text that the AI had previously written.

When labeling Self-Contradiction errors, you will make two selections, similar to how Redundant errors are annotated. First, select the text that is contradictory. Then, you will be prompted to select the text that is being contradicted (the antecedent).

Watch out

The Self-Contradiction label is only used when the contradiction occurs within the text. If a span contradicts the prompt, that should be labeled as an Off-prompt error instead.

Examples

McDonald's is considering a design which will replace the cardboard packaging. Mr Gore-Cotter said: "We recognise the concern around waste. We are now looking at a new design that minimises the plastic bag."

• Explanation: Here, the goal of minimizing the "plastic bag" contradicts the stated goal of replacing cardboard packaging. After selecting the contradictory text, please select the text that is being contradicted:

McDonald's is considering a design which will replace the cardboard packaging. Mr Gore-Cotter said: "We recognise the concern around waste. We are now looking at a new design that minimises the plastic bag."

• Explanation: In this example, the phrase "cardboard packaging" is the antecedent of the Self-Contradiction that occurs at "plastic bag".

Mall of America plans to lay off and furlough hundreds of its employees. It has no plans to restrict the number of hours workers can work.

• Explanation: Workers who are laid off or furloughed are explicitly restricted from working at the Mall, so this is a self-contradiction. First you should highlight "no plans to restrict the number of hours workers can work", and then when you are prompted to select the antecedent choose "lay off and furlough hundreds of its employees".

Exercise

Please mark the Self-Contradiction and its antecedent in the following text:

Incoherent

What is it?

Incoherent text is text that doesn't fit into the above categories, but it still just doesn’t make any sense all.

Use this label for text that is grammatical, not redundant, on prompt, not contradictory, but still confusing.

Examples

Melody Mitsugi, 28, had never given her kids cheese toast before her husband drew a map of it on her toast.

• Explanation: There are so many things wrong here: you can’t draw a map of Cheese Toast, and you wouldn’t draw it on a piece of toast if you could! However, note that the grammar is correct and there are no redundancies. This is why it gets the Incoherent label.

Cats naturally show anxiety and fear by at times breaking apart different parts of the brain in an attempt to keep the others from escaping.

• Explanation: It's hard to imagine what is happening in this passage, but the text is grammatical and non-contradictory and so we use the Incoherent label.

Exercise

Please mark the Incoherent portion of this text:

Technical Jargon

What is it?

It can be hard to understand writing simply because it uses jargon or specific words from a field you’re not familiar with. When this happens please mark it as Technical Jargon.

Examples

Due to the large size of the heavy s-block elements, including strontium, a vast range of coordination numbers is known, from 2, 3, or 4 all the way to 22 or 24 in SrCd₁₁ and SrZn₁₃.

• Explanation: Basically every part of this sentence is confusing to me because I don't understand the words. (I think I would need to know a lot more about chemistry).

In Chile, an 800-megawatt photovoltaic plant was built for a record low cost of $129 per megawatt-hour last year.

• Explanation: This word is too technical for me to know what it means without looking it up. (Though if it was explained in more detail in the text, then I wouldn't mark it.)

He uses a spirit mash made from white corn and malted barley and a neutral grain, which he describes as a "whiskey grain.”

• Explanation: I don’t know exactly what these terms mean without looking them up. (I think I would need to know more about making spirits.)

A wedding gown draped over one shoulder is not a "corset," it's a "camisole," according to fashion maven Diane von Furstenberg.

• Explanation: It’s sort of embarrassing, but I don’t really know what a corset or a camisole are. If you did, and you knew that this was wrong, then you would mark that whole phrase as Wrong: Commonsense. But for me, I just have to mark the individual words as Technical Jargon.

Exercise

Please mark the Technical Jargon error or errors in the following text:

Needs Google

What is it?

When there’s a fact or figure that you suspect might be true, but you would need to Google it to be sure, don’t google it! Instead, mark the Needs Google error type.

Here are the kinds of things you'll typically use the Needs Google tag for:

• Specific people and their roles (e.g., does X have Y job?)
• Specific events (e.g., did X win award Y on year Z?)
• Specific dates (e.g., did X happen in year Z?)
• Specific numbers (e.g., did X actually cost Y?)

Examples

It was promoted by Dr. Michael Fanning, the Executive Director of the Foundation for Mental Health Awareness, Inc. 

• Explanation: Do you know off the top of your head whether Dr. Michael Fanning is actually the Executive Director of the Foundation for Mental Health Awareness, Inc? I don’t, so I would mark this Needs Google

Paul Farmer, who was Chief Executive of the International Fund for Agricultural Development (IFAD) in 2010 when it won the Nobel Peace Prize

• Explanation: There’s actually two things I would need to Google here:
  1. Was Paul Farmer the Chief Executive of the International Fund for Agricultural Development (IFAD) in 2010?
  2. Did the International Fund for Agricultural Development (IFAD) win the Nobel Peace Prize in 2010?

an 800-megawatt photovoltaic plant was built for a record low cost of $129 per megawatt-hour last year. 

• Explanation: Again, I would mark two Needs Google errors here (along with a Technical Jargon error for the word “photovoltaic”). Here are the two Needs Google errors I would use:
  1. Can a photovoltaic plant be roughly 800-megawatt?
  2. Is $129 per megawatt-hour an actual reasonable cost for a power plant?

Watch out

You don't need to rigorously fact-check every single sentence. Instead, just mark Needs Google for text that makes a specific claim that you're not sure is true.

Exercise

Please mark the Needs Google error or errors in the following text:

Bad Math

What is it?

Sometimes the text will simply have Bad Math. This includes:

• problems with basic math (+ - ✖️ ÷) of known quantities (e.g., "half of 10 apples is 4," or "half of $5 is $3")
• problems converting fixed units (m, in, cm, kg, lbs, oz, $ → ¢, etc.)
• problems converting currencies that are wildly impossible (e.g., 1$ = 10£)
  • for reference 1 US Dollar ($) has varied between roughly 1.2 and 2.2 Great British Pounds (£) over the last 30 years.

Examples

One account, @Iain_Rowling1, had over 500,000 followers at one point, but in just four days they fell by around half – some 4,000. 

• Explanation: Half of 500,000 is not 4,000

Her doctor said that while her weight of 125 lbs (45 kg) was normal, the bizarre creatures...

• Explanation: 125 lbs = 56.7 kg

... compared with just over £1,000 ($18,868) for previous versions of Samsung’s flagship phone.

• Explanation: This conversion assumes that at some point when Samsung was making phones, £1.00 = $18.9, which is way outside the expected range of $1 – $2.5 per £.

Watch Out

Not all problems with numbers are Bad Math! When you encounter unbelievable numbers, this is usually a Wrong: Commonsense problem. Prefer to use the Wrong: Commonsense error when you think a number is unreasonable.

Here are some examples of numerical errors that fall into other categories:

... compared with just over £10,000 ($18,868) for previous versions of Samsung’s flagship phone.

• Explanation: The problem here isn't the currency conversion, which claims 1£ = $1.89 (which is believable for recent history). The problem is that this claims a Samsung phone costs £10,000 ($18,868), which is way too high for a phone. Since this isn't a specific phone, we instead rely on our commonsense knowledge of how much phones cost, and mark Wrong: Commonsense.

... compared with just over £1,000 ($1,868) for previous versions of Samsung’s flagship phone.

• Explanation: There's no error here! The price for a phone (£1,000) is believable, and the currency conversion (1£ = $1.89) is also believable.

The picture is from high above the South Pole, where close to 100,000 Astronauts live and work.

• Explanation: It's unbelievable that there are 100,000 astronauts. This is Wrong: Commonsense. This isn't a result of bad math (there's no math being done). The problem is that the number seems unreasonable.

Exercise

Please mark the Bad Math error or errors in the following text:

Wrong: Commonsense

What is it?

The AI sometimes writes text that violates our everyday basic understanding of the world. We mark these kinds of errors with Wrong: Commonsense. Commonsense errors come in many forms. Let’s look at some examples!

Examples

The picture is from high above the South Pole, where close to 100,000 Astronauts live and work.

• Explanation: There's no way that there are 100,000 astronauts currently in space! Even if we don't know the exact number, we can use our common sense to know that's way too high.

The thinness of women's bodies isn't an answer to all common human health problems like obesity or diabetes

• Explanation: While “thinness” may not be an answer, it’s impossible to be "thin" and "obese" at the same time. The AI confuses this commonsense fact.

Every person who holds a high school- or college-level diploma is unhappy.

• Explanation: This doesn't make sense. We know that a diploma doesn't make it impossible for someone to be happy.

You can get the dress custom-made and stitched at your favorite spa.

• Explanation: Spas don't offer stitching.

Now in 2020, NASA is measuring California wildfire temperatures using an instrument on the International Space Station. This year's record-shattering heat has had global repercussions in 2017, forcing sea level rise on California and increasing the risk of deadly wildfires.

• Explanation: It doesn't make sense that something in 2020 could affect something in 2017. That's backwards in time!

Exercise

Please mark the Wrong: Commonsense error or errors in the following text:

Wrong: Encyclopedic

What is it?

The AI sometimes writes things that are just plain factually wrong. We use Wrong: Encyclopedic to mark errors where the correct information is written down in a fact table somewhere, like a textbook, a wikipedia sidebar, or an encyclopedia.

Examples

Japanese Prime Minister Justin Trudeau said he will be halting all imports and exports until the current situation can be contained.

• Explanation: Justin Trudeau is the Prime Minister of Canada, not Japan. This is a clear fact that's written down in numerous places.

The gas contains something known as phyto-romatic acid, a common chemical element in the periodic table .

• Explanation: Phyto-romatic acid is not an element on the periodic table. (If you didn't know this, but you suspected it might be wrong, you would use Needs Google instead.)

Watch out

For Wrong: Encyclopedic, we want to mark errors that we know are factually wrong. These are things we could look up in Wikipedia, but we don’t have to, because we already know it.

Here is how to distinguish this from other kinds of errors:

• Pick Wrong: Commonsense if you know something is wrong, but it would probably be awkward to find it written down in Wikipedia. For example, knowing that a toaster is smaller than a person, or that doctors usually help patients, would both fall under commonsense knowledge.
• Pick Needs Google if you suspect something might be wrong, but you would need to check online to be sure. (Remember, no need to actually Google anything!)

Exercise

Please mark the Wrong: Encyclopedic error or errors in the following text: