Skip to content

feat: Improved support for evals#167

Merged
jcarver989 merged 9 commits into
mainfrom
refactor/eval-onmessage
Jun 18, 2026
Merged

feat: Improved support for evals#167
jcarver989 merged 9 commits into
mainfrom
refactor/eval-onmessage

Conversation

@jcarver989

Copy link
Copy Markdown
Contributor

This PR makes a number of improvements to Aether's evals feature, including:

  1. Evals output streaming messages for observability / debugging
  2. Consumers of the TS SDK can now hook into this via a onmessage callback
  3. aether-cli gets a new generate command for a single call and response (llm as a judge uses this)
  4. llm as a judge gets better support via a judge method for building prompts with criteria and context built in.

@jcarver989 jcarver989 enabled auto-merge (squash) June 18, 2026 22:47
@jcarver989 jcarver989 disabled auto-merge June 18, 2026 22:47
@jcarver989 jcarver989 enabled auto-merge (squash) June 18, 2026 22:48
@jcarver989 jcarver989 merged commit 8942ab1 into main Jun 18, 2026
13 checks passed
@jcarver989 jcarver989 deleted the refactor/eval-onmessage branch June 18, 2026 23:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant