Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models.

The world’s most popular programming language is losing market share to more specialized languages such as R and Perl, Tiobe ...

Oh, sure, I can “code.” That is, I can flail my way through a block of (relatively simple) pseudocode and follow the flow. I ...