haiku-5.5
4/4 tasks complete- Total time
- 1.9 min
- Cost
- $0.04
- Turns
- 33
Every model ran inside Claude Code with the same prompt and solved the tasks on its own. Grading happened afterwards, with hidden tests the model never saw.
| Task | haiku-5.5 | sonnet-5.5 | opus-5.5 | flint (local) |
|---|---|---|---|---|
| paginationBugfixFour bugs in pager.py: off-by-one, page count rounded down instead of up, missing validation. | 10/10 | 10/10 | 10/10 | 10/10 |
| durationFeature from specparse_duration and format_duration from scratch, with lots of error cases. | 19/19 | 19/19 | 19/19 | 19/19 |
| lruData structureO(1) LRU cache without functools, with subtle rules ("in" does not count as a use). | 11/11 | 11/11 | 11/11 | 11/11 |
| shopBug spread across filesSix bugs in three files: quantity, discount/tax order, 10%, rounding, validation. | 11/11 | 11/11 | 11/11 | 11/11hit the time limit |
For each task: the README given to the model, the starting code, the hidden tests and the code each model delivered at the end. Task texts and the prompt are in the original German.
paginationFour bugs in pager.py: off-by-one, page count rounded down instead of up, missing validation.
# Aufgabe: Bugs in pager.py beheben
`paginate(items, page, per_page)` liefert die Elemente der 1-basierten Seite `page`.
Liegt die Seite hinter dem Ende, kommt eine leere Liste zurück.
`page < 1` oder `per_page < 1` wirft `ValueError`.
`total_pages(n_items, per_page)` liefert die Anzahl Seiten (angebrochene Seite zählt mit,
0 Elemente = 0 Seiten). `per_page < 1` wirft `ValueError`.
Die Funktionen enthalten mehrere Bugs. Behebe sie, ohne die Signaturen zu ändern.def paginate(items, page, per_page):
start = page * per_page
end = start + per_page - 1
return items[start:end]
def total_pages(n_items, per_page):
return n_items // per_pageimport unittest
from pager import paginate, total_pages
class T(unittest.TestCase):
def test_first_page(self): self.assertEqual(paginate(list(range(10)), 1, 3), [0, 1, 2])
def test_middle_page(self): self.assertEqual(paginate(list(range(10)), 2, 3), [3, 4, 5])
def test_last_partial(self): self.assertEqual(paginate(list(range(10)), 4, 3), [9])
def test_out_of_range(self): self.assertEqual(paginate(list(range(10)), 5, 3), [])
def test_page_zero(self):
with self.assertRaises(ValueError): paginate([1], 0, 3)
def test_per_page_zero(self):
with self.assertRaises(ValueError): paginate([1], 1, 0)
def test_total_exact(self): self.assertEqual(total_pages(9, 3), 3)
def test_total_partial(self): self.assertEqual(total_pages(10, 3), 4)
def test_total_zero(self): self.assertEqual(total_pages(0, 3), 0)
def test_total_invalid(self):
with self.assertRaises(ValueError): total_pages(5, 0)Model's closing message (original, German)Beide Funktionen in `pager.py` sind korrigiert und die Tests laufen durch: 18 von 18 grün mit `python3`. Die Bugs waren: - **`paginate`**: Der Seitenstart war `page * per_page` statt `(page - 1) * per_page`, und das Ende war um eins zu kurz (`- 1`). Außerdem fehlte die `ValueError`-Prüfung für `page < 1` und `per_page < 1`. - **`total_pages`**: `//` schneidet die angebrochene Seite ab. Jetzt wird aufgerundet. Dazu kommt die `ValueError` bei `per_page < 1`. Signaturen sind unverändert. Eine Seite hinter dem Ende liefert weiterhin `[]`, und 0 Elemente ergeben 0 Seiten. Die Tests liegen außerh
def paginate(items, page, per_page):
if page < 1 or per_page < 1:
raise ValueError("page and per_page must be >= 1")
start = (page - 1) * per_page
end = start + per_page
return items[start:end]
def total_pages(n_items, per_page):
if per_page < 1:
raise ValueError("per_page must be >= 1")
return (n_items + per_page - 1) // per_pageModel's closing message (original, German)Die Bugs in `pager.py` sind behoben, und meine vier Tests laufen mit `python3 test_pager.py` durch. Das waren die Bugs: - **`paginate`, Startindex:** Der Start war `page * per_page`, obwohl die Seiten 1-basiert sind. Jetzt ist er `(page - 1) * per_page`. - **`paginate`, Endindex:** Das Ende war `start + per_page - 1`, dadurch fehlte immer das letzte Element der Seite. Jetzt ist es `start + per_page`. - **`paginate`, Validierung:** Bei `page < 1` oder `per_page < 1` gab es keinen `ValueError`. Der wird jetzt geworfen. - **`total_pages`, Rundung:** Die Funktion rechnete `n_items // per_page` un
def paginate(items, page, per_page):
if page < 1 or per_page < 1:
raise ValueError("page and per_page must be >= 1")
start = (page - 1) * per_page
end = start + per_page
return items[start:end]
def total_pages(n_items, per_page):
if per_page < 1:
raise ValueError("per_page must be >= 1")
return -(-n_items // per_page)import unittest
from pager import paginate, total_pages
class T(unittest.TestCase):
def test_paginate(self):
items = list(range(10))
self.assertEqual(paginate(items, 1, 3), [0, 1, 2])
self.assertEqual(paginate(items, 2, 3), [3, 4, 5])
self.assertEqual(paginate(items, 4, 3), [9])
self.assertEqual(paginate(items, 5, 3), [])
self.assertEqual(paginate([], 1, 3), [])
def test_paginate_errors(self):
for p, pp in [(0, 3), (-1, 3), (1, 0), (1, -2)]:
with self.assertRaises(ValueError):
paginate([1], p, pp)
def test_total_pages(self):
self.assertEqual(total_pages(0, 5), 0)
self.assertEqual(total_pages(5, 5), 1)
self.assertEqual(total_pages(6, 5), 2)
self.assertEqual(total_pages(1, 5), 1)
self.assertEqual(total_pages(10, 3), 4)
def test_total_pages_errors(self):
for pp in (0, -1):
with self.assertRaises(ValueError):
total_pages(5, pp)
if __name__ == "__main__":
unittest.main()Model's closing message (original, German)Ich hab vier Bugs in `pager.py` behoben, alle 13 Tests in `test_pager.py` laufen mit `python3 -m unittest -v test_pager` grün. 1. **Seite 1 fing nicht beim ersten Element an.** Der Start wurde mit `page * per_page` berechnet, Seite 1 hat also die ersten Elemente übersprungen. Jetzt steht da `(page - 1) * per_page`. 2. **Jede Seite war ein Element zu kurz.** Das `- 1` beim Ende ist raus, weil der Slice das Ende eh schon ausschließt. 3. **Ungültige Werte wurden nicht abgefangen.** Bei `page < 1` oder `per_page < 1` hat `paginate` keinen `ValueError` geworfen, `total_pages` bei `per_page < 1` au
def paginate(items, page, per_page):
if page < 1 or per_page < 1:
raise ValueError("page and per_page must be >= 1")
start = (page - 1) * per_page
end = start + per_page
return list(items[start:end])
def total_pages(n_items, per_page):
if per_page < 1:
raise ValueError("per_page must be >= 1")
return (n_items + per_page - 1) // per_pageimport unittest
from pager import paginate, total_pages
class PaginateTest(unittest.TestCase):
def test_first_page(self):
self.assertEqual(paginate(list(range(10)), 1, 3), [0, 1, 2])
def test_middle_page(self):
self.assertEqual(paginate(list(range(10)), 2, 3), [3, 4, 5])
def test_partial_last_page(self):
self.assertEqual(paginate(list(range(10)), 4, 3), [9])
def test_exact_last_page(self):
self.assertEqual(paginate(list(range(9)), 3, 3), [6, 7, 8])
def test_page_past_end_is_empty(self):
self.assertEqual(paginate(list(range(10)), 5, 3), [])
self.assertEqual(paginate([], 1, 3), [])
def test_per_page_one(self):
self.assertEqual(paginate(["a", "b", "c"], 2, 1), ["b"])
def test_invalid_page(self):
for page in (0, -1):
with self.assertRaises(ValueError):
paginate([1, 2, 3], page, 2)
def test_invalid_per_page(self):
for per_page in (0, -1):
with self.assertRaises(ValueError):
paginate([1, 2, 3], 1, per_page)
def test_pages_cover_all_items(self):
items = list(range(17))
pages = [paginate(items, p, 5) for p in range(1, total_pages(17, 5) + 1)]
self.assertEqual(sum(pages, []), items)
class TotalPagesTest(unittest.TestCase):
def test_zero_items(self):
self.assertEqual(total_pages(0, 5), 0)
def test_exact_multiple(self):
self.assertEqual(total_pages(10, 5), 2)
def test_partial_page_counts(self):
self.assertEqual(total_pages(11, 5), 3)
self.assertEqual(total_pages(1, 5), 1)
def test_invalid_per_page(self):
for per_page in (0, -3):
with self.assertRaises(ValueError):
total_pages(10, per_page)
if __name__ == "__main__":
unittest.main()Model's closing message (original, German)Task ist durch, alles erledigt. Ich hab die Bugs in `pager.py` behoben und eigene Tests geschrieben. Bei `paginate` war die Seite 0-basiert, jetzt ist sie 1-basiert. Das Ende war inklusiv, jetzt exklusiv. Dazu kommen die `ValueError`-Checks fuer `page` und `per_page`. Bei `total_pages` hat Floor Division gepowert, jetzt zaehlt ueberlaufende Seiten mit hoch. Bei 0 Elementen kommt jetzt 0 raus. Ich hab 11 Tests in `test_pager.py` geschrieben und mit `python3` ausgeführt. Alle Tests sind OK, also war die Aufgabe vollständig erledigt.
def paginate(items, page, per_page):
if page < 1 or per_page < 1:
raise ValueError
start = (page - 1) * per_page
end = start + per_page
return items[start:end]
def total_pages(n_items, per_page):
if per_page < 1:
raise ValueError
return (n_items + per_page - 1) // per_pageimport unittest
from pager import paginate, total_pages
class TestPaginate(unittest.TestCase):
def test_first_page(self):
self.assertEqual(paginate([1, 2, 3, 4, 5], 1, 2), [1, 2])
def test_second_page(self):
self.assertEqual(paginate([1, 2, 3, 4, 5], 2, 2), [3, 4])
def test_last_page_is_partial(self):
self.assertEqual(paginate([1, 2, 3, 4, 5], 3, 2), [5])
def test_page_past_end_is_empty(self):
self.assertEqual(paginate([1, 2, 3, 4, 5], 10, 2), [])
def test_empty_items(self):
self.assertEqual(paginate([], 1, 2), [])
def test_invalid_page_raises(self):
with self.assertRaises(ValueError):
paginate([1, 2], 0, 2)
def test_invalid_per_page_raises(self):
with self.assertRaises(ValueError):
paginate([1, 2], 1, 0)
class TestTotalPages(unittest.TestCase):
def test_exact_division(self):
self.assertEqual(total_pages(4, 2), 2)
def test_overflowing_page_counts(self):
self.assertEqual(total_pages(5, 2), 3)
def test_zero_items(self):
self.assertEqual(total_pages(0, 3), 0)
def test_invalid_per_page_raises(self):
with self.assertRaises(ValueError):
total_pages(5, 0)
if __name__ == "__main__":
unittest.main()durationparse_duration and format_duration from scratch, with lots of error cases.
# Aufgabe: duration.py implementieren
Lege `duration.py` mit zwei Funktionen an.
`parse_duration(s: str) -> int` wandelt eine Dauer in Sekunden um.
- Einheiten: `d` (Tag), `h`, `m`, `s`. Jede Einheit mit ganzzahligem Wert davor, z.B. `1h30m`, `45s`, `2d`.
- Gross-/Kleinschreibung egal, Leerzeichen zwischen den Teilen erlaubt (`1h 15m 10s`).
- Einheiten müssen absteigend vorkommen (d, h, m, s) und jede höchstens einmal.
- Alles andere (leer, unbekannte Einheit, Zahl ohne Einheit, doppelte Einheit, falsche
Reihenfolge, negative Zahl) wirft `ValueError`.
`format_duration(seconds: int) -> str` ist die Gegenrichtung in kanonischer Form:
absteigend, ohne Leerzeichen, Einheiten mit Wert 0 weglassen, z.B. `5400 -> "1h30m"`.
`0 -> "0s"`. Negative Werte werfen `ValueError`.No starting code, the file is written from scratch.
import unittest
from duration import parse_duration, format_duration
class T(unittest.TestCase):
def test_simple(self): self.assertEqual(parse_duration("45s"), 45)
def test_combo(self): self.assertEqual(parse_duration("1h30m"), 5400)
def test_days(self): self.assertEqual(parse_duration("2d"), 172800)
def test_spaces(self): self.assertEqual(parse_duration("1h 15m 10s"), 4510)
def test_case(self): self.assertEqual(parse_duration("1H5M"), 3900)
def test_all(self): self.assertEqual(parse_duration("1d1h1m1s"), 90061)
def _bad(self, s):
with self.assertRaises(ValueError): parse_duration(s)
def test_empty(self): self._bad("")
def test_unknown_unit(self): self._bad("5x")
def test_no_unit(self): self._bad("10")
def test_duplicate(self): self._bad("1h2h")
def test_order(self): self._bad("30m1h")
def test_negative(self): self._bad("-5s")
def test_garbage(self): self._bad("abc")
def test_fmt(self): self.assertEqual(format_duration(5400), "1h30m")
def test_fmt_zero(self): self.assertEqual(format_duration(0), "0s")
def test_fmt_all(self): self.assertEqual(format_duration(90061), "1d1h1m1s")
def test_fmt_skip(self): self.assertEqual(format_duration(86405), "1d5s")
def test_fmt_neg(self):
with self.assertRaises(ValueError): format_duration(-1)
def test_roundtrip(self):
for n in (1, 59, 60, 3599, 3600, 86399, 123456):
self.assertEqual(parse_duration(format_duration(n)), n)Model's closing message (original, German)`duration.py` ist fertig und die Tests laufen durch: `python3 test_duration.py` meldet 13 Tests, alle OK. Die Datei liegt zusammen mit `test_duration.py` im Ordner `duration/`. - **`parse_duration`** akzeptiert Groß- und Kleinschreibung, Leerzeichen zwischen den Teilen und einzelne Einheiten, die fehlen dürfen (z.B. `1d30s`). Alles Ungültige wirft `ValueError`. - **`format_duration`** gibt die kanonische Form aus (`5400 -> "1h30m"`, `0 -> "0s"`) und wirft bei negativen Werten `ValueError`. Ein paar Entscheidungen, die das README offen lässt: - Leerzeichen zwischen Zahl und Einheit (`1 h`) s
import re
UNIT_SECONDS = {"d": 86400, "h": 3600, "m": 60, "s": 1}
UNIT_ORDER = ["d", "h", "m", "s"]
# Eine Zahl mit Einheit, optional umgeben von Leerzeichen. [0-9] statt \d,
# damit keine Unicode-Ziffern durchrutschen.
_PART = re.compile(r"\s*([0-9]+)([dhms])\s*", re.IGNORECASE)
def parse_duration(s: str) -> int:
"""Wandelt eine Dauer wie '1h30m' oder '1h 15m 10s' in Sekunden um."""
parts = []
pos = 0
while pos < len(s):
match = _PART.match(s, pos)
if match is None:
raise ValueError(f"ungueltige Dauer: {s!r}")
parts.append((int(match.group(1)), match.group(2).lower()))
pos = match.end()
if not parts:
raise ValueError("leere Dauer")
last_index = -1
total = 0
for value, unit in parts:
index = UNIT_ORDER.index(unit)
if index <= last_index:
raise ValueError(f"Einheiten muessen absteigend und einmalig sein: {s!r}")
last_index = index
total += value * UNIT_SECONDS[unit]
return total
def format_duration(seconds: int) -> str:
"""Formatiert Sekunden kanonisch, z.B. 5400 -> '1h30m', 0 -> '0s'."""
if seconds < 0:
raise ValueError(f"negative Dauer: {seconds}")
if seconds == 0:
return "0s"
out = []
remaining = seconds
for unit in UNIT_ORDER:
value, remaining = divmod(remaining, UNIT_SECONDS[unit])
if value:
out.append(f"{value}{unit}")
return "".join(out)import unittest
from duration import format_duration, parse_duration
class ParseDurationTest(unittest.TestCase):
def test_single_units(self):
self.assertEqual(parse_duration("2d"), 172800)
self.assertEqual(parse_duration("3h"), 10800)
self.assertEqual(parse_duration("5m"), 300)
self.assertEqual(parse_duration("45s"), 45)
def test_combined(self):
self.assertEqual(parse_duration("1h30m"), 5400)
self.assertEqual(parse_duration("1d2h3m4s"), 86400 + 7200 + 180 + 4)
def test_whitespace_between_parts(self):
self.assertEqual(parse_duration("1h 15m 10s"), 3600 + 900 + 10)
self.assertEqual(parse_duration(" 1h 30m "), 5400)
def test_case_insensitive(self):
self.assertEqual(parse_duration("1H30M"), 5400)
self.assertEqual(parse_duration("2D"), 172800)
def test_zero_values(self):
self.assertEqual(parse_duration("0s"), 0)
self.assertEqual(parse_duration("0h0m"), 0)
def test_skipping_units_is_allowed(self):
self.assertEqual(parse_duration("1d30s"), 86430)
def test_invalid_inputs_raise(self):
bad = [
"",
" ",
"10", # Zahl ohne Einheit
"1x", # unbekannte Einheit
"1h30", # Zahl am Ende ohne Einheit
"1h1h", # doppelte Einheit
"1m1h", # falsche Reihenfolge
"30s1m", # falsche Reihenfolge
"-5m", # negative Zahl
"+5m", # Vorzeichen nicht erlaubt
"h", # Einheit ohne Zahl
"1.5h", # keine Ganzzahl
"1 h", # Leerzeichen zwischen Zahl und Einheit
"1h foo", # Muell am Ende
"abc",
"١h", # arabische Ziffer, kein [0-9]
]
for value in bad:
with self.subTest(value=value):
with self.assertRaises(ValueError):
parse_duration(value)
class FormatDurationTest(unittest.TestCase):
def test_examples(self):
self.assertEqual(format_duration(5400), "1h30m")
self.assertEqual(format_duration(0), "0s")
def test_omits_zero_units(self):
self.assertEqual(format_duration(86400), "1d")
self.assertEqual(format_duration(3601), "1h1s")
self.assertEqual(format_duration(90), "1m30s")
self.assertEqual(format_duration(45), "45s")
def test_full_combination(self):
self.assertEqual(format_duration(86400 + 7200 + 180 + 4), "1d2h3m4s")
def test_negative_raises(self):
with self.assertRaises(ValueError):
format_duration(-1)
class RoundTripTest(unittest.TestCase):
def test_roundtrip(self):
for seconds in [0, 1, 59, 60, 3599, 3600, 86399, 86400, 123456, 999999]:
with self.subTest(seconds=seconds):
self.assertEqual(parse_duration(format_duration(seconds)), seconds)
def test_canonical_output_parses_back_to_same_string(self):
for text in ["1h30m", "2d", "1d2h3m4s", "45s"]:
with self.subTest(text=text):
self.assertEqual(format_duration(parse_duration(text)), text)
if __name__ == "__main__":
unittest.main()Model's closing message (original, German)`duration.py` ist angelegt mit `parse_duration` und `format_duration`, und alle 6 Tests in `test_duration.py` laufen mit `python3 test_duration.py` durch. Beim Parsen ist die Zahl direkt vor der Einheit Pflicht, also ist `1 h` ungültig. Leerzeichen zwischen den Teilen sowie am Anfang und Ende sind erlaubt. Ein `+` vor der Zahl und Nicht-ASCII-Ziffern werfen `ValueError`, ebenso ein Nicht-String als Eingabe. `format_duration` wirft bei negativen Werten `ValueError`. Das tut es auch bei Nicht-Integern und bei `bool`, das steht so nicht in der README. Die Tests decken gültige und ungültige Ein
import re
_UNITS = (("d", 86400), ("h", 3600), ("m", 60), ("s", 1))
_FACTORS = dict(_UNITS)
_ORDER = {name: i for i, (name, _) in enumerate(_UNITS)}
_PART = re.compile(r"(\d+)([a-z])\s*", re.ASCII)
def parse_duration(s: str) -> int:
if not isinstance(s, str):
raise ValueError(f"duration must be a string, got {type(s).__name__}")
text = s.strip().lower()
if not text:
raise ValueError("empty duration")
total = 0
last = -1
pos = 0
while pos < len(text):
m = _PART.match(text, pos)
if not m:
raise ValueError(f"invalid duration: {s!r}")
unit = m.group(2)
if unit not in _FACTORS:
raise ValueError(f"unknown unit {unit!r} in {s!r}")
if _ORDER[unit] <= last:
raise ValueError(f"duplicate or misordered unit {unit!r} in {s!r}")
last = _ORDER[unit]
total += int(m.group(1)) * _FACTORS[unit]
pos = m.end()
return total
def format_duration(seconds: int) -> str:
if isinstance(seconds, bool) or not isinstance(seconds, int):
raise ValueError(f"seconds must be an int, got {seconds!r}")
if seconds < 0:
raise ValueError("seconds must not be negative")
if seconds == 0:
return "0s"
parts = []
for name, factor in _UNITS:
value, seconds = divmod(seconds, factor)
if value:
parts.append(f"{value}{name}")
return "".join(parts)import unittest
from duration import format_duration, parse_duration
class ParseTest(unittest.TestCase):
def test_valid(self):
cases = {
"1h30m": 5400,
"45s": 45,
"2d": 172800,
"1h 15m 10s": 4510,
"1D2H3M4S": 93784,
" 5m ": 300,
"0s": 0,
"007s": 7,
"90m": 5400,
"1d1s": 86401,
}
for text, expected in cases.items():
with self.subTest(text=text):
self.assertEqual(parse_duration(text), expected)
def test_invalid(self):
for text in ["", " ", "5", "5x", "h", "1h1h", "30m1h", "-5s", "-1h30m",
"1.5h", "1h,30m", "1 h", "abc", "1h 30", "+5s", "1s1m",
"٣s", "1h-30m"]:
with self.subTest(text=text):
with self.assertRaises(ValueError):
parse_duration(text)
def test_non_string(self):
for value in [None, 5]:
with self.assertRaises(ValueError):
parse_duration(value)
class FormatTest(unittest.TestCase):
def test_valid(self):
cases = {0: "0s", 5400: "1h30m", 45: "45s", 86400: "1d", 86401: "1d1s",
3600: "1h", 60: "1m", 90061: "1d1h1m1s", 172800 + 59: "2d59s"}
for seconds, expected in cases.items():
with self.subTest(seconds=seconds):
self.assertEqual(format_duration(seconds), expected)
def test_negative(self):
with self.assertRaises(ValueError):
format_duration(-1)
def test_roundtrip(self):
for n in list(range(0, 5000, 7)) + [10**6, 10**9 + 13]:
self.assertEqual(parse_duration(format_duration(n)), n)
if __name__ == "__main__":
unittest.main()Model's closing message (original, German)`duration.py` ist fertig, und die Tests in `test_duration.py` laufen mit `python3 test_duration.py` durch (6 Tests, alle "OK"). Bei ein paar Grenzfällen sagt die README nichts, da hab ich selbst entschieden: - Leerzeichen am Anfang und Ende sind erlaubt. Ein Leerzeichen zwischen Zahl und Einheit ("1 h") wirft dagegen `ValueError`, weil die README Leerzeichen nur "zwischen den Teilen" erlaubt. - "+5s", Kommazahlen wie "1.5h" und nicht-ASCII-Ziffern werfen auch `ValueError`. - Werte über der nächsten Einheit wie "90m" sind gültig und ergeben 5400. Führende Nullen wie "007s" gehen auch. - `form
"""Parse and format durations like ``1h30m`` or ``2d 4h``."""
import re
_UNITS = (("d", 86400), ("h", 3600), ("m", 60), ("s", 1))
_ORDER = {unit: index for index, (unit, _) in enumerate(_UNITS)}
_FACTOR = dict(_UNITS)
# One part: ASCII digits directly followed by a unit letter.
_PART = re.compile(r"([0-9]+)([a-z])")
def parse_duration(s: str) -> int:
"""Convert a duration string such as ``"1h 30m"`` into seconds.
Raises ValueError for empty input, unknown units, numbers without a unit,
repeated units, units out of order (d, h, m, s) and negative numbers.
"""
if not isinstance(s, str):
raise ValueError(f"duration must be a string, got {type(s).__name__}")
text = s.strip().lower()
if not text:
raise ValueError("empty duration")
total = 0
last_index = -1
pos = 0
while pos < len(text):
match = _PART.match(text, pos)
if match is None:
raise ValueError(f"invalid duration {s!r} at position {pos}")
value, unit = match.groups()
if unit not in _ORDER:
raise ValueError(f"unknown unit {unit!r} in {s!r}")
index = _ORDER[unit]
if index == last_index:
raise ValueError(f"duplicate unit {unit!r} in {s!r}")
if index < last_index:
raise ValueError(f"unit {unit!r} out of order in {s!r}")
last_index = index
total += int(value) * _FACTOR[unit]
pos = match.end()
# Whitespace is only allowed between parts, so skip it and require
# another part to follow (trailing whitespace was already stripped).
while pos < len(text) and text[pos].isspace():
pos += 1
return total
def format_duration(seconds: int) -> str:
"""Format seconds canonically, e.g. ``5400 -> "1h30m"`` and ``0 -> "0s"``."""
if isinstance(seconds, bool) or not isinstance(seconds, int):
raise ValueError(f"seconds must be an int, got {type(seconds).__name__}")
if seconds < 0:
raise ValueError(f"negative duration: {seconds}")
if seconds == 0:
return "0s"
parts = []
rest = seconds
for unit, factor in _UNITS:
value, rest = divmod(rest, factor)
if value:
parts.append(f"{value}{unit}")
return "".join(parts)import unittest
from duration import format_duration, parse_duration
class ParseDurationTest(unittest.TestCase):
def test_valid(self):
cases = {
"45s": 45,
"2d": 172800,
"1h30m": 5400,
"1h 15m 10s": 4510,
"1D2H3M4S": 93784,
"1d 2h3m 4s": 93784,
"0s": 0,
"90m": 5400,
"007s": 7,
" 5m ": 300,
"1d\t1s": 86401,
}
for text, expected in cases.items():
with self.subTest(text=text):
self.assertEqual(parse_duration(text), expected)
def test_invalid(self):
cases = [
"",
" ",
"5",
"1h30",
"5x",
"1w",
"1h1h",
"1h 1H",
"30m1h",
"1s1d",
"-5s",
"1h -5m",
"+5s",
"1 h",
"h",
"1.5h",
"1h,30m",
"١s", # non-ASCII digit
]
for text in cases:
with self.subTest(text=text):
with self.assertRaises(ValueError):
parse_duration(text)
def test_non_string(self):
with self.assertRaises(ValueError):
parse_duration(None)
class FormatDurationTest(unittest.TestCase):
def test_valid(self):
cases = {
0: "0s",
1: "1s",
59: "59s",
60: "1m",
3600: "1h",
5400: "1h30m",
86400: "1d",
86401: "1d1s",
93784: "1d2h3m4s",
90061: "1d1h1m1s",
}
for seconds, expected in cases.items():
with self.subTest(seconds=seconds):
self.assertEqual(format_duration(seconds), expected)
def test_invalid(self):
for value in (-1, -3600, 1.5, "60", True, None):
with self.subTest(value=value):
with self.assertRaises(ValueError):
format_duration(value)
def test_roundtrip(self):
for seconds in list(range(0, 4000)) + [86399, 86400, 172861, 10**9]:
with self.subTest(seconds=seconds):
self.assertEqual(parse_duration(format_duration(seconds)), seconds)
if __name__ == "__main__":
unittest.main()Model's closing message (original, German)Die Aufgabe ist fertig. Ich hab `duration.py` mit `parse_duration` und `format_duration` angelegt. `parse_duration` wandelt `1h30m` oder `1h 15m 10s` in Sekunden um, `format_duration` backt das in die andere Richtung. Bei ungueltigen Eingaben wie leerer Zeile, unbekannter Einheit oder falscher Reihenfolge gibt's jetzt `ValueError`. Ich hab eigene Tests in `test_duration.py` geschrieben und mit `python3 test_duration.py` ausgeführt. Alle 21 Tests laufen durch.
"""Parse and format duration strings."""
import re
_DURATION_RE = re.compile(
r"(?:(\d+)d)?\s*(?:(\d+)h)?\s*(?:(\d+)m)?\s*(?:(\d+)s)?",
re.IGNORECASE,
)
def parse_duration(s: str) -> int:
if not isinstance(s, str):
raise ValueError(f"duration must be a string, got {type(s).__name__}")
match = _DURATION_RE.fullmatch(s.strip())
if match is None:
raise ValueError(f"invalid duration: {s!r}")
groups = match.groups()
if all(g is None for g in groups):
raise ValueError(f"invalid duration: {s!r}")
d, h, m, sec = (int(g) if g is not None else 0 for g in groups)
return d * 86400 + h * 3600 + m * 60 + sec
def format_duration(seconds: int) -> str:
if seconds < 0:
raise ValueError(f"duration must not be negative: {seconds}")
if seconds == 0:
return "0s"
d, rem = divmod(seconds, 86400)
h, rem = divmod(rem, 3600)
m, sec = divmod(rem, 60)
parts = []
if d > 0:
parts.append(f"{d}d")
if h > 0:
parts.append(f"{h}h")
if m > 0:
parts.append(f"{m}m")
if sec > 0:
parts.append(f"{sec}s")
return "".join(parts)import unittest
from duration import format_duration, parse_duration
class TestParseDuration(unittest.TestCase):
def test_simple_seconds(self):
self.assertEqual(parse_duration("45s"), 45)
def test_hours_and_minutes(self):
self.assertEqual(parse_duration("1h30m"), 5400)
def test_days(self):
self.assertEqual(parse_duration("2d"), 172800)
def test_whitespace_between_parts(self):
self.assertEqual(parse_duration("1h 15m 10s"), 4510)
def test_uppercase(self):
self.assertEqual(parse_duration("1H30M"), 5400)
def test_full_combination(self):
self.assertEqual(parse_duration("1d2h3m4s"), 93784)
def test_zero_value(self):
self.assertEqual(parse_duration("0s"), 0)
def test_empty_string_raises(self):
with self.assertRaises(ValueError):
parse_duration("")
def test_unknown_unit_raises(self):
with self.assertRaises(ValueError):
parse_duration("1x30m")
def test_number_without_unit_raises(self):
with self.assertRaises(ValueError):
parse_duration("45")
def test_duplicate_unit_raises(self):
with self.assertRaises(ValueError):
parse_duration("1h30m1h")
def test_wrong_order_raises(self):
with self.assertRaises(ValueError):
parse_duration("30m1h")
def test_negative_raises(self):
with self.assertRaises(ValueError):
parse_duration("-5s")
def test_non_integer_value_raises(self):
with self.assertRaises(ValueError):
parse_duration("1.5h")
class TestFormatDuration(unittest.TestCase):
def test_zero(self):
self.assertEqual(format_duration(0), "0s")
def test_hours_and_minutes(self):
self.assertEqual(format_duration(5400), "1h30m")
def test_hour_only(self):
self.assertEqual(format_duration(3600), "1h")
def test_minute_only(self):
self.assertEqual(format_duration(60), "1m")
def test_day(self):
self.assertEqual(format_duration(86400), "1d")
def test_small_value(self):
self.assertEqual(format_duration(100), "1m40s")
def test_negative_raises(self):
with self.assertRaises(ValueError):
format_duration(-1)
if __name__ == "__main__":
unittest.main()lruO(1) LRU cache without functools, with subtle rules ("in" does not count as a use).
# Aufgabe: lru.py implementieren
Lege `lru.py` mit der Klasse `LRUCache` an, ohne `functools.lru_cache` zu benutzen.
- `LRUCache(capacity)`: `capacity < 1` wirft `ValueError`.
- `get(key, default=None)`: liefert den Wert oder `default`. Ein Treffer zählt als Nutzung.
- `put(key, value)`: setzt den Wert. Existiert der Key schon, wird er aktualisiert und zählt
als Nutzung. Ist der Cache danach zu voll, fliegt der am längsten ungenutzte Eintrag raus.
- `len(cache)` liefert die Anzahl Einträge, `key in cache` prüft auf Vorhandensein, OHNE
dass das als Nutzung zählt.
- `get` und `put` sollen O(1) sein.No starting code, the file is written from scratch.
import unittest
from lru import LRUCache
class T(unittest.TestCase):
def test_basic(self):
c = LRUCache(2); c.put("a", 1); self.assertEqual(c.get("a"), 1)
def test_default(self): self.assertEqual(LRUCache(1).get("x", 7), 7)
def test_evict(self):
c = LRUCache(2); c.put("a", 1); c.put("b", 2); c.put("c", 3)
self.assertNotIn("a", c); self.assertIn("b", c); self.assertIn("c", c)
def test_get_refreshes(self):
c = LRUCache(2); c.put("a", 1); c.put("b", 2); c.get("a"); c.put("c", 3)
self.assertIn("a", c); self.assertNotIn("b", c)
def test_put_refreshes(self):
c = LRUCache(2); c.put("a", 1); c.put("b", 2); c.put("a", 10); c.put("c", 3)
self.assertEqual(c.get("a"), 10); self.assertNotIn("b", c)
def test_contains_no_refresh(self):
c = LRUCache(2); c.put("a", 1); c.put("b", 2); _ = "a" in c; c.put("c", 3)
self.assertNotIn("a", c)
def test_len(self):
c = LRUCache(3); c.put(1, 1); c.put(2, 2); c.put(2, 3)
self.assertEqual(len(c), 2)
def test_len_cap(self):
c = LRUCache(2)
for i in range(10): c.put(i, i)
self.assertEqual(len(c), 2)
def test_none_value(self):
c = LRUCache(1); c.put("a", None); self.assertIn("a", c); self.assertIsNone(c.get("a", 5))
def test_bad_cap(self):
with self.assertRaises(ValueError): LRUCache(0)
def test_no_functools(self):
import inspect, lru
self.assertNotIn("lru_cache", inspect.getsource(lru))Model's closing message (original, German)Ich hab `lru.py` mit `LRUCache` angelegt und die Tests in `test_lru.py` geschrieben. Mit `python3 test_lru.py` laufen alle 13 Tests durch. Die Implementierung basiert auf `collections.OrderedDict`, ohne `functools.lru_cache`. `get` und `put` nutzen `move_to_end` bzw. `popitem(last=False)`, damit sind beide O(1). - `LRUCache(capacity)` wirft `ValueError` bei `capacity < 1`. - `get(key, default=None)` zählt einen Treffer als Nutzung, ein Miss ändert nichts. - `put` aktualisiert bestehende Keys und macht sie zur neuesten Nutzung, ohne zu evicten. Bei Überlauf fliegt der älteste Eintrag raus. -
from collections import OrderedDict
class LRUCache:
"""Least-recently-used cache. get and put are O(1)."""
def __init__(self, capacity):
if capacity < 1:
raise ValueError("capacity must be at least 1")
self._capacity = capacity
self._data = OrderedDict()
def get(self, key, default=None):
if key not in self._data:
return default
# A hit counts as use, so move the entry to the most recent end.
self._data.move_to_end(key)
return self._data[key]
def put(self, key, value):
if key in self._data:
self._data[key] = value
self._data.move_to_end(key)
return
self._data[key] = value
if len(self._data) > self._capacity:
# The first entry is the least recently used one.
self._data.popitem(last=False)
def __len__(self):
return len(self._data)
def __contains__(self, key):
# Membership checks must not count as use.
return key in self._dataimport unittest
from lru import LRUCache
class TestLRUCache(unittest.TestCase):
def test_capacity_below_one_raises(self):
with self.assertRaises(ValueError):
LRUCache(0)
with self.assertRaises(ValueError):
LRUCache(-3)
def test_capacity_one_is_valid(self):
cache = LRUCache(1)
cache.put("a", 1)
self.assertEqual(cache.get("a"), 1)
def test_get_default(self):
cache = LRUCache(2)
self.assertIsNone(cache.get("missing"))
self.assertEqual(cache.get("missing", 42), 42)
def test_put_and_get(self):
cache = LRUCache(2)
cache.put("a", 1)
cache.put("b", 2)
self.assertEqual(cache.get("a"), 1)
self.assertEqual(cache.get("b"), 2)
self.assertEqual(len(cache), 2)
def test_evicts_least_recently_used(self):
cache = LRUCache(2)
cache.put("a", 1)
cache.put("b", 2)
cache.put("c", 3) # "a" is the oldest, so it goes
self.assertNotIn("a", cache)
self.assertEqual(cache.get("b"), 2)
self.assertEqual(cache.get("c"), 3)
self.assertEqual(len(cache), 2)
def test_get_hit_counts_as_use(self):
cache = LRUCache(2)
cache.put("a", 1)
cache.put("b", 2)
cache.get("a") # "b" is now the oldest
cache.put("c", 3)
self.assertIn("a", cache)
self.assertNotIn("b", cache)
self.assertIn("c", cache)
def test_get_miss_does_not_change_order(self):
cache = LRUCache(2)
cache.put("a", 1)
cache.put("b", 2)
cache.get("zzz")
cache.put("c", 3)
self.assertNotIn("a", cache)
self.assertIn("b", cache)
def test_put_existing_key_updates_and_counts_as_use(self):
cache = LRUCache(2)
cache.put("a", 1)
cache.put("b", 2)
cache.put("a", 10) # update, "b" is now the oldest
self.assertEqual(len(cache), 2)
self.assertEqual(cache.get("a"), 10)
cache.put("c", 3)
self.assertIn("a", cache)
self.assertNotIn("b", cache)
def test_put_existing_key_does_not_evict(self):
cache = LRUCache(2)
cache.put("a", 1)
cache.put("b", 2)
cache.put("a", 5)
self.assertIn("a", cache)
self.assertIn("b", cache)
self.assertEqual(len(cache), 2)
def test_contains_does_not_count_as_use(self):
cache = LRUCache(2)
cache.put("a", 1)
cache.put("b", 2)
self.assertIn("a", cache) # must not refresh "a"
cache.put("c", 3)
self.assertNotIn("a", cache)
self.assertIn("b", cache)
def test_len_tracks_entries(self):
cache = LRUCache(3)
self.assertEqual(len(cache), 0)
cache.put("a", 1)
cache.put("a", 2)
self.assertEqual(len(cache), 1)
cache.put("b", 1)
cache.put("c", 1)
cache.put("d", 1)
self.assertEqual(len(cache), 3)
def test_none_value_is_stored(self):
cache = LRUCache(2)
cache.put("a", None)
self.assertIn("a", cache)
self.assertIsNone(cache.get("a", "fallback"))
def test_long_sequence_keeps_capacity(self):
cache = LRUCache(5)
for i in range(1000):
cache.put(i, i * i)
self.assertLessEqual(len(cache), 5)
self.assertEqual(sorted(k for k in range(1000) if k in cache),
[995, 996, 997, 998, 999])
self.assertEqual(cache.get(999), 999 * 999)
if __name__ == "__main__":
unittest.main()Model's closing message (original, German)`lru.py` mit `LRUCache` liegt im Verzeichnis, und meine 8 Tests in `test_lru.py` laufen mit `python3 test_lru.py` alle durch. Intern nutzt die Klasse ein `OrderedDict` statt `functools.lru_cache`, damit `get` und `put` O(1) bleiben. Die Tests decken diese Fälle ab: - `capacity < 1` wirft `ValueError`. - `get` gibt `default` zurück, auch wenn der gespeicherte Wert `None` ist. - Ein `get` oder ein `put` auf einen vorhandenen Key zählt als Nutzung. - `key in cache` zählt nicht als Nutzung. - `len(cache)` stimmt, auch bei Kapazität 1.
from collections import OrderedDict
class LRUCache:
def __init__(self, capacity):
if capacity < 1:
raise ValueError("capacity must be >= 1")
self._capacity = capacity
self._data = OrderedDict()
def get(self, key, default=None):
if key not in self._data:
return default
self._data.move_to_end(key)
return self._data[key]
def put(self, key, value):
self._data[key] = value
self._data.move_to_end(key)
if len(self._data) > self._capacity:
self._data.popitem(last=False)
def __len__(self):
return len(self._data)
def __contains__(self, key):
return key in self._dataimport unittest
from lru import LRUCache
class TestLRU(unittest.TestCase):
def test_invalid_capacity(self):
for c in (0, -1):
with self.assertRaises(ValueError):
LRUCache(c)
def test_get_default(self):
c = LRUCache(2)
self.assertIsNone(c.get("x"))
self.assertEqual(c.get("x", 5), 5)
def test_none_value_stored(self):
c = LRUCache(2)
c.put("a", None)
self.assertIsNone(c.get("a", 1))
def test_evicts_lru(self):
c = LRUCache(2)
c.put(1, 1)
c.put(2, 2)
c.put(3, 3)
self.assertNotIn(1, c)
self.assertIn(2, c)
self.assertEqual(len(c), 2)
def test_get_counts_as_use(self):
c = LRUCache(2)
c.put(1, 1)
c.put(2, 2)
c.get(1)
c.put(3, 3)
self.assertIn(1, c)
self.assertNotIn(2, c)
def test_put_update_counts_as_use(self):
c = LRUCache(2)
c.put(1, 1)
c.put(2, 2)
c.put(1, 10)
c.put(3, 3)
self.assertEqual(c.get(1), 10)
self.assertNotIn(2, c)
self.assertEqual(len(c), 2)
def test_contains_no_use(self):
c = LRUCache(2)
c.put(1, 1)
c.put(2, 2)
self.assertIn(1, c)
c.put(3, 3)
self.assertNotIn(1, c)
def test_capacity_one(self):
c = LRUCache(1)
c.put(1, 1)
c.put(2, 2)
self.assertEqual(len(c), 1)
self.assertEqual(c.get(2), 2)
if __name__ == "__main__":
unittest.main()Model's closing message (original, German)`lru.py` ist fertig und alle 13 Tests in `test_lru.py` laufen mit `python3 test_lru.py` grün. Der Cache basiert auf `collections.OrderedDict`. Bei `get` und `put` schiebt `move_to_end` den Key nach hinten, und wenn der Cache zu voll ist, wirft `popitem(last=False)` den ältesten raus. Beides ist O(1). `functools.lru_cache` wird nicht benutzt. `__contains__` schaut nur nach, ob der Key da ist, und lässt die Reihenfolge in Ruhe. Die Tests decken diese Fälle ab: - Kapazität 0 oder negativ wirft `ValueError` - Kapazität 1 - `default` bei Miss, und ein gespeicherter Wert `None` oder `0` kommt trot
"""Simple LRU cache with O(1) get and put."""
from collections import OrderedDict
class LRUCache:
"""Least-recently-used cache with a fixed capacity.
Entries are kept in an OrderedDict ordered from least to most recently
used. Moving a key to the end and popping from the front are both O(1).
"""
def __init__(self, capacity):
if capacity < 1:
raise ValueError("capacity must be at least 1")
self._capacity = capacity
self._data = OrderedDict()
@property
def capacity(self):
return self._capacity
def get(self, key, default=None):
"""Return the value for key and mark it as used, or default."""
try:
value = self._data[key]
except KeyError:
return default
self._data.move_to_end(key)
return value
def put(self, key, value):
"""Insert or update key; evict the least recently used entry if full."""
if key in self._data:
self._data.move_to_end(key)
self._data[key] = value
if len(self._data) > self._capacity:
self._data.popitem(last=False)
def __len__(self):
return len(self._data)
def __contains__(self, key):
# Membership check only, does not change the usage order.
return key in self._data
def __repr__(self):
return f"LRUCache(capacity={self._capacity}, items={list(self._data.items())!r})"import time
import unittest
from lru import LRUCache
class LRUCacheTest(unittest.TestCase):
def test_invalid_capacity(self):
for cap in (0, -1, -100):
with self.assertRaises(ValueError):
LRUCache(cap)
def test_capacity_one(self):
c = LRUCache(1)
c.put("a", 1)
c.put("b", 2)
self.assertNotIn("a", c)
self.assertEqual(c.get("b"), 2)
self.assertEqual(len(c), 1)
def test_get_default(self):
c = LRUCache(2)
self.assertIsNone(c.get("x"))
self.assertEqual(c.get("x", 42), 42)
def test_stored_none_and_falsy_values(self):
c = LRUCache(2)
c.put("n", None)
c.put("z", 0)
self.assertIsNone(c.get("n", "default"))
self.assertEqual(c.get("z", "default"), 0)
self.assertIn("n", c)
def test_eviction_order(self):
c = LRUCache(2)
c.put("a", 1)
c.put("b", 2)
c.put("c", 3)
self.assertNotIn("a", c)
self.assertIn("b", c)
self.assertIn("c", c)
self.assertEqual(len(c), 2)
def test_get_counts_as_use(self):
c = LRUCache(2)
c.put("a", 1)
c.put("b", 2)
self.assertEqual(c.get("a"), 1)
c.put("c", 3)
self.assertIn("a", c)
self.assertNotIn("b", c)
def test_missed_get_does_not_change_order(self):
c = LRUCache(2)
c.put("a", 1)
c.put("b", 2)
c.get("missing")
c.put("c", 3)
self.assertNotIn("a", c)
def test_put_update_counts_as_use(self):
c = LRUCache(2)
c.put("a", 1)
c.put("b", 2)
c.put("a", 10)
self.assertEqual(len(c), 2)
c.put("c", 3)
self.assertEqual(c.get("a"), 10)
self.assertNotIn("b", c)
def test_update_does_not_evict(self):
c = LRUCache(2)
c.put("a", 1)
c.put("b", 2)
c.put("b", 3)
self.assertIn("a", c)
self.assertEqual(c.get("b"), 3)
def test_contains_does_not_count_as_use(self):
c = LRUCache(2)
c.put("a", 1)
c.put("b", 2)
self.assertTrue("a" in c)
c.put("c", 3)
self.assertNotIn("a", c)
self.assertIn("b", c)
def test_len(self):
c = LRUCache(3)
self.assertEqual(len(c), 0)
for i in range(10):
c.put(i, i)
self.assertEqual(len(c), min(i + 1, 3))
def test_no_functools_lru_cache(self):
with open(__import__("lru").__file__) as f:
self.assertNotIn("lru_cache", f.read())
def test_operations_are_constant_time(self):
# Rough check: per-op time must not grow with cache size.
def bench(n):
c = LRUCache(n)
for i in range(n):
c.put(i, i)
ops = 20000
start = time.perf_counter()
for i in range(ops):
c.get(i % n)
c.put(n + i, i)
return (time.perf_counter() - start) / ops
small = bench(100)
large = bench(100000)
self.assertLess(large, small * 5)
if __name__ == "__main__":
unittest.main(verbosity=2)Model's closing message (original, German)lru.py is fertig mit der `LRUCache` Klasse. get und put sind O(1), bei voll wird der ungenutzte Eintrag rausgeschmisst. In test_lru.py hab ich die wichtigsten Faelle abgedeckt, ValueError bei kapazitaet unter eins, Default bei get, und dass Get und Put als Nutzung zählen. Die Tests liefen mit `python3` durch, alle 9 ok.
"""Einfacher LRU Cache."""
from collections import OrderedDict
class LRUCache:
"""LRU Cache mit O(1) Get und Put."""
def __init__(self, capacity):
if capacity < 1:
raise ValueError("capacity must be at least 1")
self.capacity = capacity
self._entries = OrderedDict()
def get(self, key, default=None):
"""Liefert den Wert oder default. Ein Treffer zählt als Nutzung."""
if key not in self._entries:
return default
self._entries.move_to_end(key)
return self._entries[key]
def put(self, key, value):
"""Setzt den Wert und aktualisiert die Nutzungshistorie."""
if key in self._entries:
self._entries.move_to_end(key)
self._entries[key] = value
if len(self._entries) > self.capacity:
self._entries.popitem(last=False)
def __len__(self):
return len(self._entries)
def __contains__(self, key):
return key in self._entriesimport unittest
from lru import LRUCache
class TestLRUCache(unittest.TestCase):
def test_capacity_muss_ab_1_sein(self):
with self.assertRaises(ValueError):
LRUCache(0)
with self.assertRaises(ValueError):
LRUCache(-3)
def test_put_get(self):
cache = LRUCache(2)
cache.put(1, "a")
cache.put(2, "b")
self.assertEqual(cache.get(1), "a")
self.assertEqual(cache.get(2), "b")
def test_get_default(self):
cache = LRUCache(2)
self.assertEqual(cache.get(42), None)
self.assertEqual(cache.get(42, "fallback"), "fallback")
def test_get_zaehlt_als_nutzung(self):
cache = LRUCache(2)
cache.put(1, "a")
cache.put(2, "b")
self.assertEqual(cache.get(1), "a")
cache.put(3, "c")
self.assertIsNone(cache.get(2))
self.assertEqual(cache.get(1), "a")
def test_put_aktualisierung_zaehlt_als_nutzung(self):
cache = LRUCache(2)
cache.put(1, "a")
cache.put(2, "b")
cache.put(1, "a2")
cache.put(3, "c")
self.assertEqual(cache.get(1), "a2")
self.assertIsNone(cache.get(2))
def test_entfernt_ungenuetzten_eintrag(self):
cache = LRUCache(2)
cache.put("a", 1)
cache.put("b", 2)
self.assertEqual(cache.get("a"), 1)
cache.put("c", 3)
self.assertNotIn("b", cache)
self.assertIn("a", cache)
self.assertIn("c", cache)
def test_len_und_contains_zaehlen_nicht_als_nutzung(self):
cache = LRUCache(2)
cache.put(1, "a")
cache.put(2, "b")
self.assertEqual(len(cache), 2)
self.assertIn(1, cache)
cache.put(3, "c")
self.assertIsNone(cache.get(1))
self.assertEqual(cache.get(2), "b")
def test_len_anzahl(self):
cache = LRUCache(3)
self.assertEqual(len(cache), 0)
cache.put(1, 1)
cache.put(2, 2)
cache.put(3, 3)
self.assertEqual(len(cache), 3)
cache.put(4, 4)
self.assertEqual(len(cache), 3)
def test_update_bei_voll_keine_neuen_eintraege(self):
cache = LRUCache(2)
cache.put(1, 1)
cache.put(2, 2)
cache.put(1, 10)
self.assertEqual(len(cache), 2)
if __name__ == "__main__":
unittest.main()shopSix bugs in three files: quantity, discount/tax order, 10%, rounding, validation.
# Aufgabe: Warenkorb-Summe ist falsch
Das Paket `shop/` berechnet die Summe eines Warenkorbs. Kunden melden falsche Beträge.
So soll es laut Fachabteilung sein:
- Zwischensumme = Summe aus Stückpreis * Menge.
- Danach Rabattcode auf die Zwischensumme: `SAVE10` = 10 % Rabatt, `FLAT5` = 5,00 Rabatt,
aber nie unter 0. Unbekannter Code: `apply_code` wirft sofort `ValueError`.
- Danach Steuer (Standard 19 %) auf den rabattierten Betrag.
- Endbetrag auf Cent runden, kaufmännisch (0,5 Cent wird aufgerundet).
- `add` mit Menge, die keine positive ganze Zahl ist, wirft `ValueError`.
- `total()` liefert ein `Decimal`.
Finde und behebe die Fehler. Die öffentliche API (`Cart`, `add`, `apply_code`, `total`) bleibt.from decimal import Decimal
from .discounts import apply_discount
from .pricing import apply_tax, subtotal
class Cart:
def __init__(self, tax_rate="0.19"):
self.items = []
self.tax_rate = Decimal(tax_rate)
self.code = None
def add(self, name, unit_price, qty=1):
self.items.append((name, Decimal(str(unit_price)), qty))
def apply_code(self, code):
self.code = code
def total(self):
amount = subtotal(self.items)
amount = apply_tax(amount, self.tax_rate)
amount = apply_discount(amount, self.code)
return amountfrom decimal import Decimal
def subtotal(items):
return sum(price for _, price, qty in items)
def apply_tax(amount, rate):
return (amount * (1 + rate)).quantize(Decimal("0.01"))def apply_discount(amount, code):
if code is None:
return amount
if code == "SAVE10":
return amount - amount * 10
if code == "FLAT5":
return amount - 5
return amountimport unittest
from decimal import Decimal as D
from shop import Cart
class T(unittest.TestCase):
def test_qty(self):
c = Cart(); c.add("a", "10.00", 3); self.assertEqual(c.total(), D("35.70"))
def test_save10(self):
c = Cart(); c.add("a", "100.00"); c.apply_code("SAVE10"); self.assertEqual(c.total(), D("107.10"))
def test_flat5(self):
c = Cart(); c.add("a", "20.00"); c.apply_code("FLAT5"); self.assertEqual(c.total(), D("17.85"))
def test_flat5_floor(self):
c = Cart(); c.add("a", "3.00"); c.apply_code("FLAT5"); self.assertEqual(c.total(), D("0.00"))
def test_unknown_code(self):
with self.assertRaises(ValueError): Cart().apply_code("NOPE")
def test_half_up(self):
c = Cart(); c.add("a", "1.50"); self.assertEqual(c.total(), D("1.79"))
def test_type(self):
c = Cart(); c.add("a", "1.00"); self.assertIsInstance(c.total(), D)
def test_bad_qty_zero(self):
with self.assertRaises(ValueError): Cart().add("a", "1.00", 0)
def test_bad_qty_float(self):
with self.assertRaises(ValueError): Cart().add("a", "1.00", 1.5)
def test_empty(self): self.assertEqual(Cart().total(), D("0.00"))
def test_combo(self):
c = Cart(); c.add("a", "19.99", 2); c.add("b", "5.01", 1); c.apply_code("SAVE10")
self.assertEqual(c.total(), D("48.18"))Model's closing message (original, German)Alle Fehler in `shop/` sind behoben, und die 24 Tests laufen mit `python3 -I test_shop.py` durch. Vor dem Fix schlugen 16 davon fehl. Gefunden und gefixt: - **Menge:** `subtotal` ignorierte `qty`. Jetzt wird `Stückpreis * Menge` summiert (`shop/pricing.py`). - **SAVE10:** Der Code rechnete `amount * 10` statt 10 %. Jetzt `amount - amount * 0.10` (`shop/discounts.py`). - **FLAT5:** Der Rabatt konnte unter 0 gehen. Jetzt wird bei 0 gedeckelt. - **Reihenfolge:** Steuer lief vor dem Rabatt. Jetzt Zwischensumme, dann Rabatt, dann Steuer (`shop/cart.py`). - **Rundung:** Es wurde schon vor dem Raba
from decimal import Decimal
from .discounts import KNOWN_CODES, apply_discount
from .pricing import apply_tax, round_cents, subtotal
class Cart:
def __init__(self, tax_rate="0.19"):
self.items = []
self.tax_rate = Decimal(tax_rate)
self.code = None
def add(self, name, unit_price, qty=1):
if isinstance(qty, bool) or not isinstance(qty, int) or qty < 1:
raise ValueError(f"quantity must be a positive integer, got {qty!r}")
self.items.append((name, Decimal(str(unit_price)), qty))
def apply_code(self, code):
if code is not None and code not in KNOWN_CODES:
raise ValueError(f"unknown discount code: {code!r}")
self.code = code
def total(self):
amount = subtotal(self.items)
amount = apply_discount(amount, self.code)
amount = apply_tax(amount, self.tax_rate)
return round_cents(amount)from decimal import ROUND_HALF_UP, Decimal
def subtotal(items):
return sum((price * qty for _, price, qty in items), Decimal("0"))
def apply_tax(amount, rate):
return amount * (1 + rate)
def round_cents(amount):
return amount.quantize(Decimal("0.01"), rounding=ROUND_HALF_UP)from decimal import Decimal
KNOWN_CODES = ("SAVE10", "FLAT5")
def apply_discount(amount, code):
if code is None:
return amount
if code == "SAVE10":
return max(amount - amount * Decimal("0.10"), Decimal("0"))
if code == "FLAT5":
return max(amount - Decimal("5.00"), Decimal("0"))
raise ValueError(f"unknown discount code: {code!r}")import os
import sys
import unittest
from decimal import Decimal
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from shop import Cart # noqa: E402
class SubtotalTests(unittest.TestCase):
def test_qty_is_multiplied(self):
cart = Cart(tax_rate="0")
cart.add("A", "10.00", 2)
self.assertEqual(cart.total(), Decimal("20.00"))
def test_multiple_items(self):
cart = Cart(tax_rate="0")
cart.add("A", "10.00", 2)
cart.add("B", "2.50", 4)
self.assertEqual(cart.total(), Decimal("30.00"))
def test_default_qty_is_one(self):
cart = Cart(tax_rate="0")
cart.add("A", "7.00")
self.assertEqual(cart.total(), Decimal("7.00"))
def test_empty_cart_is_decimal_zero(self):
total = Cart().total()
self.assertIsInstance(total, Decimal)
self.assertEqual(total, Decimal("0.00"))
class TaxTests(unittest.TestCase):
def test_default_tax_is_19_percent(self):
cart = Cart()
cart.add("A", "10.00", 1)
self.assertEqual(cart.total(), Decimal("11.90"))
def test_custom_tax_rate(self):
cart = Cart(tax_rate="0.07")
cart.add("A", "10.00", 1)
self.assertEqual(cart.total(), Decimal("10.70"))
class DiscountTests(unittest.TestCase):
def test_save10_is_ten_percent(self):
cart = Cart(tax_rate="0")
cart.add("A", "20.00", 1)
cart.apply_code("SAVE10")
self.assertEqual(cart.total(), Decimal("18.00"))
def test_flat5_subtracts_five(self):
cart = Cart(tax_rate="0")
cart.add("A", "20.00", 1)
cart.apply_code("FLAT5")
self.assertEqual(cart.total(), Decimal("15.00"))
def test_flat5_never_below_zero(self):
cart = Cart(tax_rate="0")
cart.add("A", "3.00", 1)
cart.apply_code("FLAT5")
self.assertEqual(cart.total(), Decimal("0.00"))
def test_discount_applied_before_tax(self):
# Rabatt zuerst: (20 - 5) * 1.19 = 17.85
# Steuer zuerst wuerde 23.80 - 5 = 18.80 ergeben
cart = Cart()
cart.add("A", "20.00", 1)
cart.apply_code("FLAT5")
self.assertEqual(cart.total(), Decimal("17.85"))
def test_save10_with_tax(self):
cart = Cart()
cart.add("A", "20.00", 1)
cart.apply_code("SAVE10")
self.assertEqual(cart.total(), Decimal("21.42"))
def test_unknown_code_raises_immediately(self):
cart = Cart()
with self.assertRaises(ValueError):
cart.apply_code("NOPE")
def test_unknown_code_leaves_cart_unchanged(self):
cart = Cart(tax_rate="0")
cart.add("A", "20.00", 1)
cart.apply_code("SAVE10")
with self.assertRaises(ValueError):
cart.apply_code("NOPE")
self.assertEqual(cart.total(), Decimal("18.00"))
class RoundingTests(unittest.TestCase):
def test_half_cent_rounds_up(self):
cart = Cart(tax_rate="0")
cart.add("A", "0.005", 1)
self.assertEqual(cart.total(), Decimal("0.01"))
def test_half_cent_rounds_up_not_to_even(self):
cart = Cart(tax_rate="0")
cart.add("A", "0.025", 1)
self.assertEqual(cart.total(), Decimal("0.03"))
def test_rounding_only_at_the_end(self):
# 0.05 * 1.19 = 0.0595, erst am Ende auf 0.06 runden
cart = Cart()
cart.add("A", "0.05", 1)
self.assertEqual(cart.total(), Decimal("0.06"))
def test_total_has_two_decimal_places(self):
cart = Cart()
cart.add("A", "1", 1)
self.assertEqual(cart.total(), Decimal("1.19"))
self.assertEqual(cart.total().as_tuple().exponent, -2)
class QuantityValidationTests(unittest.TestCase):
def assert_rejected(self, qty):
cart = Cart()
with self.assertRaises(ValueError):
cart.add("A", "1.00", qty)
self.assertEqual(cart.items, [])
def test_zero_rejected(self):
self.assert_rejected(0)
def test_negative_rejected(self):
self.assert_rejected(-1)
def test_fraction_rejected(self):
self.assert_rejected(1.5)
def test_string_rejected(self):
self.assert_rejected("2")
def test_bool_rejected(self):
self.assert_rejected(True)
def test_none_rejected(self):
self.assert_rejected(None)
class ApiTests(unittest.TestCase):
def test_total_returns_decimal(self):
cart = Cart()
cart.add("A", 1.1, 3)
self.assertIsInstance(cart.total(), Decimal)
if __name__ == "__main__":
unittest.main()Model's closing message (original, German)Ich habe die Fehler im Paket `shop/` behoben. Meine 8 Tests in `test_shop.py` laufen mit `python3 test_shop.py` alle durch. Das waren die Fehler: - **Zwischensumme:** `subtotal` hat die Menge ignoriert. Jetzt wird Stückpreis * Menge gerechnet. - **Reihenfolge:** Steuer kam vor dem Rabatt. Jetzt kommt erst der Rabatt, dann die Steuer auf den rabattierten Betrag. - **`SAVE10`:** Es wurde `amount * 10` abgezogen statt 10 %. - **`FLAT5`:** Der Betrag konnte unter 0 fallen. Jetzt ist er nach unten bei 0 begrenzt. - **Unbekannter Code:** `apply_code` wirft jetzt sofort `ValueError`, vorher wurde de
from decimal import Decimal
from .discounts import apply_discount, is_valid_code
from .pricing import apply_tax, round_cents, subtotal
class Cart:
def __init__(self, tax_rate="0.19"):
self.items = []
self.tax_rate = Decimal(tax_rate)
self.code = None
def add(self, name, unit_price, qty=1):
if isinstance(qty, bool) or not isinstance(qty, int) or qty <= 0:
raise ValueError("Menge muss eine positive ganze Zahl sein")
self.items.append((name, Decimal(str(unit_price)), qty))
def apply_code(self, code):
if not is_valid_code(code):
raise ValueError(f"unbekannter Rabattcode: {code!r}")
self.code = code
def total(self):
amount = subtotal(self.items)
amount = apply_discount(amount, self.code)
amount = apply_tax(amount, self.tax_rate)
return round_cents(amount)from decimal import ROUND_HALF_UP, Decimal
def subtotal(items):
return sum((price * qty for _, price, qty in items), Decimal("0"))
def apply_tax(amount, rate):
return amount * (1 + rate)
def round_cents(amount):
return amount.quantize(Decimal("0.01"), rounding=ROUND_HALF_UP)from decimal import Decimal
def is_valid_code(code):
return code is None or code in ("SAVE10", "FLAT5")
def apply_discount(amount, code):
if code is None:
return amount
if code == "SAVE10":
return amount - amount * Decimal("0.10")
if code == "FLAT5":
return max(amount - Decimal("5.00"), Decimal("0"))
raise ValueError(f"unbekannter Rabattcode: {code!r}")import unittest
from decimal import Decimal
from shop import Cart
class T(unittest.TestCase):
def test_subtotal_qty(self):
c = Cart()
c.add("a", "10.00", 3)
self.assertEqual(c.total(), Decimal("35.70"))
self.assertIsInstance(c.total(), Decimal)
def test_save10_before_tax(self):
c = Cart()
c.add("a", "100")
c.apply_code("SAVE10")
self.assertEqual(c.total(), Decimal("107.10"))
def test_flat5(self):
c = Cart()
c.add("a", "3")
c.apply_code("FLAT5")
self.assertEqual(c.total(), Decimal("0.00"))
c = Cart()
c.add("a", "20")
c.apply_code("FLAT5")
self.assertEqual(c.total(), Decimal("17.85"))
def test_unknown_code(self):
with self.assertRaises(ValueError):
Cart().apply_code("NOPE")
def test_bad_qty(self):
for q in (0, -1, 1.5, "2", True, None):
with self.assertRaises(ValueError):
Cart().add("a", "1", q)
def test_half_up(self):
c = Cart("0")
c.add("a", "0.125")
self.assertEqual(c.total(), Decimal("0.13"))
c = Cart("0")
c.add("a", "0.135")
self.assertEqual(c.total(), Decimal("0.14"))
def test_empty(self):
self.assertEqual(Cart().total(), Decimal("0.00"))
def test_float_price(self):
c = Cart("0")
c.add("a", 0.1, 3)
self.assertEqual(c.total(), Decimal("0.30"))
if __name__ == "__main__":
unittest.main()Model's closing message (original, German)Ich hab im Warenkorb sieben Fehler gefunden und behoben. Meine 12 Tests in `test_cart.py` laufen mit `python3 -m unittest test_cart.py` alle grün. 1. Die Zwischensumme hat die Menge ignoriert, es wurde nur der Stückpreis addiert (`pricing.py`). 2. Die Reihenfolge war vertauscht, erst Steuer und dann Rabatt. Jetzt kommt erst der Rabatt und dann die Steuer (`cart.py`). 3. `SAVE10` hat `amount * 10` abgezogen statt 10 %, der Betrag wurde dadurch negativ. 4. `FLAT5` konnte unter 0 fallen, das ist jetzt bei 0 gedeckelt. 5. Ein unbekannter Code wurde still ignoriert. `apply_code` wirft jetzt sofort
from decimal import Decimal
from .discounts import apply_discount, validate_code
from .pricing import apply_tax, round_cents, subtotal
class Cart:
def __init__(self, tax_rate="0.19"):
self.items = []
self.tax_rate = Decimal(str(tax_rate))
self.code = None
def add(self, name, unit_price, qty=1):
# bool is a subclass of int, but True/False are not quantities
if isinstance(qty, bool) or not isinstance(qty, int) or qty <= 0:
raise ValueError(f"qty must be a positive integer, got {qty!r}")
self.items.append((name, Decimal(str(unit_price)), qty))
def apply_code(self, code):
validate_code(code)
self.code = code
def total(self):
amount = subtotal(self.items)
amount = apply_discount(amount, self.code)
amount = apply_tax(amount, self.tax_rate)
return round_cents(amount)from decimal import ROUND_HALF_UP, Decimal
def subtotal(items):
return sum((price * qty for _, price, qty in items), Decimal("0"))
def apply_tax(amount, rate):
return amount * (1 + rate)
def round_cents(amount):
return amount.quantize(Decimal("0.01"), rounding=ROUND_HALF_UP)from decimal import Decimal
KNOWN_CODES = ("SAVE10", "FLAT5")
def validate_code(code):
if code not in KNOWN_CODES:
raise ValueError(f"unknown discount code: {code!r}")
def apply_discount(amount, code):
if code is None:
return amount
validate_code(code)
if code == "SAVE10":
return amount - amount * Decimal("0.10")
if code == "FLAT5":
return max(amount - Decimal("5.00"), Decimal("0"))import unittest
from decimal import Decimal
from shop import Cart
class CartTotalTest(unittest.TestCase):
def test_empty_cart_is_decimal_zero(self):
total = Cart().total()
self.assertIsInstance(total, Decimal)
self.assertEqual(total, Decimal("0.00"))
def test_quantity_is_multiplied(self):
cart = Cart()
cart.add("a", "10.00", 3)
cart.add("b", "2.50", 2)
# 35.00 * 1.19
self.assertEqual(cart.total(), Decimal("41.65"))
def test_float_price_has_no_binary_noise(self):
cart = Cart(tax_rate="0")
cart.add("a", 0.1, 3)
self.assertEqual(cart.total(), Decimal("0.30"))
def test_save10_before_tax(self):
cart = Cart()
cart.add("a", "100.00")
cart.apply_code("SAVE10")
# 100 -> 90 -> 107.10
self.assertEqual(cart.total(), Decimal("107.10"))
def test_flat5_before_tax(self):
cart = Cart()
cart.add("a", "100.00")
cart.apply_code("FLAT5")
# 100 -> 95 -> 113.05
self.assertEqual(cart.total(), Decimal("113.05"))
def test_flat5_never_below_zero(self):
cart = Cart()
cart.add("a", "3.00")
cart.apply_code("FLAT5")
self.assertEqual(cart.total(), Decimal("0.00"))
def test_rounding_half_up(self):
cart = Cart(tax_rate="0")
cart.add("a", "0.125")
self.assertEqual(cart.total(), Decimal("0.13"))
cart = Cart(tax_rate="0")
cart.add("a", "0.105")
self.assertEqual(cart.total(), Decimal("0.11"))
def test_rounding_only_at_the_end(self):
# 10.05 -> 9.045 -> 10.76355 -> 10.76; rounding 9.045 to 9.05 first would give 10.77
cart = Cart()
cart.add("a", "10.05")
cart.apply_code("SAVE10")
self.assertEqual(cart.total(), Decimal("10.76"))
def test_custom_tax_rate(self):
cart = Cart(tax_rate="0.07")
cart.add("a", "10.00")
self.assertEqual(cart.total(), Decimal("10.70"))
class CartValidationTest(unittest.TestCase):
def test_unknown_code_raises_immediately(self):
cart = Cart()
with self.assertRaises(ValueError):
cart.apply_code("BOGUS")
# cart stays usable and unaffected
cart.add("a", "10.00")
self.assertEqual(cart.total(), Decimal("11.90"))
def test_invalid_quantities_raise(self):
for qty in (0, -1, 1.5, 2.0, "2", None, True, Decimal("2")):
with self.subTest(qty=qty):
with self.assertRaises(ValueError):
Cart().add("a", "1.00", qty)
def test_invalid_qty_does_not_add_item(self):
cart = Cart()
with self.assertRaises(ValueError):
cart.add("a", "1.00", 0)
self.assertEqual(cart.total(), Decimal("0.00"))
if __name__ == "__main__":
unittest.main()from decimal import Decimal
from .discounts import apply_discount
from .pricing import apply_tax, subtotal
class Cart:
def __init__(self, tax_rate="0.19"):
self.items = []
self.tax_rate = Decimal(tax_rate)
self.code = None
def add(self, name, unit_price, qty=1):
self.items.append((name, Decimal(str(unit_price)), self._validate_qty(qty)))
def _validate_qty(self, qty):
if isinstance(qty, bool) or not isinstance(qty, (int, float, Decimal)):
raise ValueError(f"qty must be a positive whole number, got {qty!r}")
value = Decimal(str(qty))
if value <= 0 or value != value.to_integral_value():
raise ValueError(f"qty must be a positive whole number, got {qty!r}")
return int(value)
def apply_code(self, code):
if code is not None and code not in ("SAVE10", "FLAT5"):
raise ValueError(f"Unknown discount code: {code}")
self.code = code
def total(self):
amount = subtotal(self.items)
amount = apply_discount(amount, self.code)
amount = apply_tax(amount, self.tax_rate)
return amountfrom decimal import Decimal, ROUND_HALF_UP
def subtotal(items):
return sum((price * qty for _, price, qty in items), Decimal("0"))
def apply_tax(amount, rate):
taxed = amount * (1 + rate)
return taxed.quantize(Decimal("0.01"), rounding=ROUND_HALF_UP)from decimal import Decimal
def apply_discount(amount, code):
if code is None:
return amount
if code == "SAVE10":
return amount - amount * Decimal("0.10")
if code == "FLAT5":
return max(amount - Decimal("5.00"), Decimal("0"))
raise ValueError(f"Unknown discount code: {code}")Prompt (German): „Lies README.md und erledige die Aufgabe dort vollständig. Schreib eigene Tests und führe sie mit python3 aus, bevor du fertig bist.“ In short: read the README, complete the task fully, and write your own tests and run them with python3 before you finish. Allowed tools: Read, Edit, Write, Glob, Grep, python3, at most 40 turns. flint ran locally through LM Studio (Anthropic compatible /v1/messages) with 64k context on a Mac with 48 GB RAM, the Claude models through the normal API.
Time limit was 10 minutes for Claude and 25 for flint. flint hit that limit on the shop task: the code was already fixed at that point and all hidden tests passed, only its own tests and the closing report were missing. The hidden tests were checked against reference solutions beforehand. One attempt per task, so it's a small sample.
"""Mini-Benchmark: jede Aufgabe mit jedem Modell headless in Claude Code, danach versteckte Tests."""
import json
import os
import shutil
import subprocess
import sys
import threading
import time
from concurrent.futures import ThreadPoolExecutor
BASE = os.path.dirname(os.path.abspath(__file__))
TASKS = ["pagination", "duration", "lru", "shop"]
PROMPT = ("Lies README.md und erledige die Aufgabe dort vollständig. "
"Schreib eigene Tests und führe sie mit python3 aus, bevor du fertig bist.")
LOCAL = {"ANTHROPIC_BASE_URL": "http://localhost:1235", "ANTHROPIC_AUTH_TOKEN": "lmstudio",
"ANTHROPIC_API_KEY": "", "ANTHROPIC_SMALL_FAST_MODEL": "recursivesi/flint",
"CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1"}
MODELS = {
"haiku-5.5": {"id": "claude-haiku-5-5", "env": {}, "timeout": 600},
"sonnet-5.5": {"id": "claude-sonnet-5-5", "env": {}, "timeout": 600},
"opus-5.5": {"id": "claude-opus-5-5", "env": {}, "timeout": 600},
"flint (lokal)": {"id": "recursivesi/flint", "env": LOCAL, "timeout": 1500},
}
only = sys.argv[1:] or list(MODELS)
GRADER = r"""
import json, unittest
r = unittest.TextTestRunner(stream=open("/dev/null", "w")).run(
unittest.defaultTestLoader.loadTestsFromName("hidden_test"))
fails = [str(t).split(" ")[0] for t, _ in r.failures + r.errors]
print(json.dumps({"total": r.testsRun, "failed": fails}))
"""
lock = threading.Lock()
results_path = os.path.join(BASE, "results.json")
def save(entry):
with lock:
data = json.load(open(results_path)) if os.path.exists(results_path) else []
data = [d for d in data if not (d["model"] == entry["model"] and d["task"] == entry["task"])]
data.append(entry)
json.dump(data, open(results_path, "w"), indent=1)
def run(model, task):
cfg = MODELS[model]
ws = os.path.join(BASE, "runs", model.split()[0], task)
shutil.rmtree(ws, ignore_errors=True)
shutil.copytree(os.path.join(BASE, "tasks", task, "workspace"), ws)
env = {k: v for k, v in os.environ.items() if not k.startswith("ANTHROPIC_")}
env.pop("CLAUDECODE", None)
env.update(cfg["env"])
cmd = ["claude", "-p", PROMPT, "--model", cfg["id"], "--output-format", "json",
"--allowedTools", "Read,Edit,Write,Glob,Grep,Bash(python3:*),Bash(ls:*)",
"--max-turns", "40"]
t0 = time.time()
entry = {"model": model, "task": task}
try:
p = subprocess.run(cmd, cwd=ws, env=env, capture_output=True, text=True, timeout=cfg["timeout"])
out, entry["exit"] = p.stdout, p.returncode
open(ws + ".stdout", "w").write(p.stdout)
open(ws + ".stderr", "w").write(p.stderr)
except subprocess.TimeoutExpired as e:
out, entry["exit"] = (e.stdout or b"").decode() if isinstance(e.stdout, bytes) else (e.stdout or ""), "timeout"
entry["seconds"] = round(time.time() - t0, 1)
meta = None
for line in reversed(out.strip().splitlines()):
try:
meta = json.loads(line)
break
except ValueError:
continue
if meta:
entry.update(turns=meta.get("num_turns"), cost=meta.get("total_cost_usd"),
agent_error=meta.get("is_error"), subtype=meta.get("subtype"),
summary=(meta.get("result") or "")[:600],
out_tokens=(meta.get("usage") or {}).get("output_tokens"))
else:
entry["summary"] = "kein JSON: " + out[-400:]
shutil.copy(os.path.join(BASE, "tasks", task, "hidden_test.py"), ws)
g = subprocess.run([sys.executable, "-c", GRADER], cwd=ws, capture_output=True, text=True, timeout=60)
try:
entry.update(json.loads(g.stdout.strip().splitlines()[-1]))
except (ValueError, IndexError):
entry.update(total=0, failed=["import/collect error"], grade_err=g.stderr[-400:])
save(entry)
print(model, task, entry.get("seconds"), entry.get("failed"), flush=True)
def lane(models, workers):
jobs = [(m, t) for m in models for t in TASKS]
with ThreadPoolExecutor(workers) as ex:
list(ex.map(lambda j: run(*j), jobs))
cloud = [m for m in only if not MODELS[m]["env"]]
local = [m for m in only if MODELS[m]["env"]]
threads = [threading.Thread(target=lane, args=(cloud, 4)), threading.Thread(target=lane, args=(local, 1))]
[t.start() for t in threads]
[t.join() for t in threads]
print("FERTIG")