Meta's new AI chat sucks at coding
Yesterday, Zuck got on stage to announce Meta's ChatGPT killer, Llama 3, apparently making this bold claim:
Meta says that Llama 3 outperforms competing models of its class on key benchmarks and that it's better across the board at tasks like coding
Coding? You sure about that?
I've been pairing with ChatGPT (using GPT-4) every day for the last few months and it is demonstrably terrible 80% of the time, but 20% of the time it saves me an hour of headaches, so I put up with it anyway. Nevertheless, my experience with Llama 2 was so miserable, I figured Zuck's claim about Llama 3 outperforming GPT-4 was bullshit, so I put it to the test this morning.
TLDR: I asked three questions and Llama 3 whiffed. Badly.
Question 1
Here's the first question I asked, pondering a less messy way to generate URL paths (secretly knowing how hard this is, given that Rails models and controllers are intentionally decoupled):