Discrete Reasoning Over Paragraphs (DROP)
Intro
Il benchmark DROP (acronimo di "Discrete Reasoning Over the content of Paragraphs") è un insieme di dati creato per valutare le capacità di comprensione del testo e ragionamento di sistemi di intelligenza artificiale. Questo benchmark è particolarmente focalizzato sulla capacità di manipolare informazioni numeriche e di effettuare inferenze basate su testi. DROP si distingue per la sua enfasi sulle domande che richiedono una comprensione approfondita del testo e la capacità di effettuare operazioni discrete, come il calcolo di somme, differenze, o la determinazione di date e numeri specifici all'interno del testo.
Esempi
To start the season, the Lions traveled south to Tampa, Florida to take on the Tampa Bay Buccaneers. The Lions scored first in the first quarter with a 23-yard field goal by Jason Hanson. The Buccaneers tied it up with a 38-yard field goal by Connor Barth, then took the lead when Aqib Talib intercepted a pass from Matthew Stafford and ran it in 28 yards. The Lions responded with a 28-yard field goal. In the second quarter, Detroit took the lead with a 36-yard touchdown catch by Calvin Johnson, and later added more points when Tony Scheffler caught an 11-yard TD pass. Tampa Bay responded with a 31-yard field goal just before halftime. The second half was relatively quiet, with each team only scoring one touchdown. First, Detroit's Calvin Johnson caught a 1-yard pass in the third quarter. The game's final points came when Mike Williams of Tampa Bay caught a 5-yard pass. The Lions won their regular season opener for the first time since 2007
How many points did the buccaneers need to tie in the first?
{ "spans": [ "3" ], "types": [ "number" ] }
To start the season, the Lions traveled south to Tampa, Florida to take on the Tampa Bay Buccaneers. The Lions scored first in the first quarter with a 23-yard field goal by Jason Hanson. The Buccaneers tied it up with a 38-yard field goal by Connor Barth, then took the lead when Aqib Talib intercepted a pass from Matthew Stafford and ran it in 28 yards. The Lions responded with a 28-yard field goal. In the second quarter, Detroit took the lead with a 36-yard touchdown catch by Calvin Johnson, and later added more points when Tony Scheffler caught an 11-yard TD pass. Tampa Bay responded with a 31-yard field goal just before halftime. The second half was relatively quiet, with each team only scoring one touchdown. First, Detroit's Calvin Johnson caught a 1-yard pass in the third quarter. The game's final points came when Mike Williams of Tampa Bay caught a 5-yard pass. The Lions won their regular season opener for the first time since 2007
How many field goals did the Lions score?
{ "spans": [ "2" ], "types": [ "number" ] }