Applied AI Labs

Join Applied AI Collective for a practical study group on evals in AI systems — how teams measure, test, and improve the quality, reliability, and safety of AI products before and after deployment.
As AI systems move from demos to real-world products, evaluation becomes one of the most important parts of the stack. It is not enough for an AI system to “look good” in a demo. Builders need ways to understand whether the system is accurate, reliable, safe, useful, and improving over time.
This session is designed as a builder-friendly discussion for engineers, founders, researchers, product builders, and AI enthusiasts who want to better understand how evals work in practical AI systems.
What we’ll cover:
• What AI evals are and why they matter
• How evals are used in LLM apps, RAG systems, and AI agents
• Offline evals vs. online evals
• Human evals, automated evals, and LLM-as-judge approaches
• Common failure modes: hallucinations, wrong tool use, bad retrieval, unsafe outputs, and poor user experience
• How evals connect to reliability, guardrails, product quality, and launch readiness
• Open discussion around tools, frameworks, examples, and real-world challenges
Who should attend:
AI builders, software engineers, ML engineers, founders, product managers, researchers, students, and anyone interested in building more reliable AI systems.
No prior expertise is required. Bring your questions, examples, tools you have tried, or AI systems you are currently building.
This is a study group, not a formal lecture. The goal is to learn together, discuss practical approaches, and help each other understand how to evaluate AI systems in the real world.

Let’s build a stronger community around practical, reliable, real-world AI systems.

Applied AI Labs: AI Agent Hack
Join Applied AI Labs for a free, beginner-friendly AI hackathon-style build session.
This is a hands-on afternoon event where you’ll build a small practical AI agent or AI app in a few hours. You can work solo or team up with others at the event.
The goal is not to build something huge. The goal is to leave with a small working prototype, a clearer understanding of AI agents, and a few new builder connections.

Judging and prizes:
At the end of the event, projects will be reviewed by a panel of 3 expert judges.
The top 2 implementations will receive a small surprise gift/prize.
Judging will focus on practical usefulness, working demo/prototype, creativity, clear explanation of the problem and solution, and simplicity of execution within the time limit.
Who this is for:
Beginners, developers, students, product-minded builders, and anyone curious about practical AI agents.
You do not need prior hackathon experience. Basic Python or JavaScript experience is helpful but not required.
What to bring:
• Laptop
• Charger
• OpenAI / Anthropic / Gemini / other API key if you have one
• A simple idea, or just come and join a team
Cost:
Free and open to the public.

Find Your Co-Founder: Founder Networking & Builder Collaboration Meetup

Finding the right co-founder, technical partner, or early collaborator can be one of the most important steps in building a successful AI startup.
Join Applied AI Labs for a founder-focused networking meetup designed to bring together AI founders, aspiring entrepreneurs, engineers, product builders, designers, researchers, and startup-minded professionals who are looking to connect with potential co-founders and collaborators.
Whether you already have an AI startup idea, are actively building a product, or are still exploring opportunities, this meetup is a space to meet like-minded builders, share what you are working on, discuss ideas, and discover strong collaboration opportunities.
For those in the ideation stage, this is also a chance to introduce your idea, receive feedback from other builders, validate early concepts, and learn from people working across the AI and startup ecosystem.
Who should attend:
• AI founders looking for co-founders or early teammates
• Engineers interested in joining or starting AI projects
• Product builders, designers, and operators exploring startup ideas
• Researchers or students interested in applied AI ventures
• Anyone looking to build, collaborate, or join an early-stage AI project
What to expect:
• Founder and builder networking
• Short intros from attendees
• Open discussion around AI startup ideas
• Opportunities to find technical and non-technical collaborators
• Peer feedback on early-stage concepts
• Connections with people interested in AI agents, GenAI, automation, RAG, AI tools, and real-world AI products
Come with an idea, a skill set, a project, or simply the desire to build. Leave with new connections — and possibly your future AI co-founder, technical partner, or startup collaborator.
Let’s bring together the Bay Area AI builder community for meaningful conversations, collaboration, and new startup momentum.