How to win a hackathon with

NexHacks Workshop 2026-01-14

Thank you for showing up

What are we talking about today?

What is observability?
What are evaluations?
How do I use evals to optimize my agent?
Demo

Arize Prize

$1,000 cash for Best Use of Arize

Assumption:

you're building

an agent

What is observability?

Observability = rich, structured logging

Automatic integrations

TypeScript

BeeAI
LangChain
Mastra
Vercel AI

Python

Agno
Autogen
CrewAI
DSPy
Google ADK
Graphite
Haystack
Instructor
LlamaIndex
LangChain
LangGraph
NVIDIA
Portkey
Pydantic AI

Our example agent

tracer_provider = register(
	project_name="crewai-tracing-quickstart",
    auto_instrument=True
)

bit.ly/nexhacks-example

Phoenix Cloud

app.phoenix.arize.com

Tracing

Congratulations!

You can win $1k now

What are evaluations?

Unit testing for nondeterministic outputs

Solution:

LLM as a judge

Code evaluations

are also possible

Evaluations

Evaluation annotations

Evaluations

save time

Datasets

Experiments

Experiment results

Demo time!

Thank you!

These slides:

slides.com/seldo/nexhacks

The code again:

bit.ly/nexhacks-example

I'm on BlueSky:

🦋 @seldo.com