Skip to content

Initial Support for Remote Evals Devserver#23

Merged
Andrew Kent (realark) merged 7 commits intomainfrom
ark/remote-evals
Dec 29, 2025
Merged

Initial Support for Remote Evals Devserver#23
Andrew Kent (realark) merged 7 commits intomainfrom
ark/remote-evals

Conversation

@realark
Copy link
Copy Markdown
Collaborator

No description provided.

@realark Andrew Kent (realark) added the enhancement New feature or request label Dec 17, 2025
@realark Andrew Kent (realark) force-pushed the ark/remote-evals branch 7 times, most recently from 28b1576 to 6404057 Compare December 19, 2025 01:16
@realark Andrew Kent (realark) force-pushed the ark/remote-evals branch 2 times, most recently from e234cc2 to 5646de4 Compare December 19, 2025 21:00
@BeforeAll
static void setUp() throws Exception {
// Set up mock Braintrust API server
mockApiServer = HttpServer.create(new InetSocketAddress("localhost", MOCK_API_PORT), 0);
Copy link
Copy Markdown
Collaborator Author

@realark Andrew Kent (realark) Dec 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a few a mock http servers in the SDK unit tests. I plan to switch everything over to VCR pretty soon, but this PR is already a beast so I want to do that later

Copy link
Copy Markdown
Contributor

@delner David Elner (delner) left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you give us a brief description of the feature, the design you went with, and any particular decisions you made? Would be helpful for contextualizing the rest of my review!

* @param <V> Value type
*/
@ThreadSafe
class LRUCache<K, V> {
Copy link
Copy Markdown
Contributor

@delner David Elner (delner) Dec 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. What's this used for? Didn't see any references to it.

Is this generic enough (given the use of templates) that its considered a "Util" as opposed to "devserver" specific? Unfamiliar with Java's patterns for common use components like this.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's used in devserver. It caches the auth info (on my phone rn but I think the name is authCache).

This could totally be a generic util. It's package-private right now, so if/when another module needs it I can move it up a package without any breaking changes.

@realark
Copy link
Copy Markdown
Collaborator Author

Can you give us a brief description of the feature, the design you went with, and any particular decisions you made? Would be helpful for contextualizing the rest of my review!

For sure! I'll give you a demo Monday. The context helps

Copy link
Copy Markdown
Contributor

@delner David Elner (delner) left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Andrew gave me a demo of this early last week (thanks!) Changes make sense overall!

@realark Andrew Kent (realark) merged commit 1db1ece into main Dec 29, 2025
1 check passed
@realark Andrew Kent (realark) deleted the ark/remote-evals branch December 29, 2025 03:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants