Quickstart: grade existing outputs

Score a JSONL file of question/answer records with built-in evaluators.

Source: examples/quickstart. Read the guide for the concepts behind it.

qa.jsonl

{"id": "capital-fr", "input": "What is the capital of France?", "output": "Paris", "reference": "Paris", "metadata": {"topic": "geography"}, "usage": {"input_tokens": 12, "output_tokens": 2, "latency_ms": 340}}
{"id": "capital-jp", "input": "What is the capital of Japan?", "output": "Kyoto", "reference": "Tokyo", "metadata": {"topic": "geography"}, "usage": {"input_tokens": 12, "output_tokens": 2, "latency_ms": 310}}
{"id": "capital-au", "input": "What is the capital of Australia?", "output": "canberra", "reference": ["Canberra", "Canberra, ACT"], "metadata": {"topic": "geography"}, "usage": {"input_tokens": 13, "output_tokens": 2, "latency_ms": 295}}
{"id": "apples", "input": "Sam has 3 bags of 12 apples and eats 4. How many are left?", "output": "3 x 12 = 36, minus 4 leaves 32 apples.", "reference": "32", "metadata": {"topic": "arithmetic"}, "usage": {"input_tokens": 21, "output_tokens": 14, "latency_ms": 820}}
{"id": "train", "input": "A train travels 120 km in 1.5 hours. What is its average speed in km/h?", "output": "The average speed is 80 km/h.", "reference": "80", "metadata": {"topic": "arithmetic"}, "usage": {"input_tokens": 24, "output_tokens": 9, "latency_ms": 640}}
{"id": "budget", "input": "A project costs $1,200 per month. What does it cost for a year?", "output": "That comes to $14,000 per year.", "reference": "14400", "metadata": {"topic": "arithmetic"}, "usage": {"input_tokens": 20, "output_tokens": 9, "latency_ms": 700}}