HellaSwag

Intro

È stato progettato per testare la capacità di un modello di prevedere la conclusione logica di una narrazione, spesso in contesti che richiedono un'elevata capacità di ragionamento e comprensione del contesto. Il benchmark HellaSwag include due tipi principali di contesti: descrizioni di video e situazioni di tipo Wikipedia. Per ogni contesto, vengono fornite diverse scelte di continuazione, e il compito del modello è selezionare quella più logica e coerente con il contesto fornito.

Esempi

[header] How to overcome communication apprehension [title] Pinpoint your fears. [step] Before you can learn to overcome your communication apprehension, you must figure out exactly what it is that is scary or nerve-wrecking. Construct an anxiety hierarchy to help you better understand your fears.

Possibilità

['Consider several factors before making a list of potential fears related to communicating inaccurately. [substeps] You may be scared of being on the receiving end, not being able to talk to someone, being overwhelmed by personal details, not being able to stop worrying, or being loud and demanding in public.' '[substeps] Constructing an anxiety hierarchy starts with a list of situations where the anxiety is felt. After the list is complete, you then rank them from the lowest level of anxiety all the way to the one situation with the highest level of stress.' '[substeps] Your hierarchy might include things like " fear of airplanes " or " fear of the unknown. " oftentimes fear comes from alone or with others.' "[substeps] For example, research has shown that it's perfectly normal to feel nervous when you are getting called back at work or meeting work deadlines. But, anxiety arises when you are feeling afraid or unable to perform the tasks that you need to do."]

Risposta esatta: 1 (seconda)

Link

HellaSwag: Can a Machine Really Finish Your Sentence?