Let’s talk

The R&D lab that takes your AI from “what if” to clear answers.

We help teams figure out which models, methods, and bets are worth making. That means experiments, benchmarking, failure analysis, and technical recommendations grounded in evidence.

Bring us a question, a dataset, or a rough prototype. Most projects go from question to answer in 4 to 12 weeks.

Discuss your AI problem See the work

What you get

  • Working models: trained on your data, with the code and pipelines to rerun everything.
  • Experiment results: baselines and head-to-head comparisons you can reproduce end to end.
  • Evaluation reports: benchmark numbers and failure analysis for LLMs, search, ranking, and forecasting.
  • Red-team findings: where the system breaks, how often, and what it takes to fix.
  • A decision memo: what to build and what to skip, plus a walkthrough with your team.

Selected clients and collaborators

Alongside Allen Institute for AI Gates Foundation Actively Learn Algonomy Preva Group Alzheimer’s Drug Discovery Foundation / Gates Ventures Combustion Global Health Labs Alzheimer’s Disease Data Initiative

What we do

Machine learning and AI consulting, run as research sprints with clear answers backed by experiments.

R&D sprints

Rapid exploration, baseline comparisons, and hard-nosed feasibility checks. See our work.

LLM evaluation & guardrails

Prompting, rubric design, reliability testing, and failure analysis. Learn about our LLM chatbot work.

Search & ranking research

Offline/online metrics, ranking experiments, and user-impact analysis. Read our search engine case study.

Messy data science

We clean messy data, find meaningful patterns, and test hypotheses, even on small datasets. Explore our small-dataset benchmarks.

Forecasting & anomaly detection

We compare models and quantify uncertainty.

Technical due diligence

Independent evaluation of models and methods before you commit.

How we work

We run 4–12 week research sprints. You get clear answers backed by experiments, benchmarks, and documented methods.

  1. 1–2 weeks

    Discovery & data audit

    We start with the decision you need to make, the data you have, and your constraints.

    • Problem framing & success metrics
    • Data quality review
    • Analysis plan & experiment design

  2. 2–8 weeks

    Experiments & evaluation

    We compare baselines and test approaches head-to-head.

    • Model development & prototyping
    • Evaluation design & benchmarking
    • Failure analysis & safety testing

  3. 1–2 weeks

    Findings & handoff

    We write up what worked, what didn’t, and what to do next.

    • Written findings & recommendations
    • Benchmark results & methods documentation
    • Team walkthrough

The team

Sergey Feldman is a Principal Research Scientist at the Allen Institute for AI (Ai2). Previously, he served as Head of AI at Alongside, a youth well-being platform acquired by TimelyCare in 2026. He has led research and development for large-scale AI, retrieval, and language systems, including work behind Semantic Scholar and Asta. His expertise spans LLMs, machine learning, natural language processing, and scientific document analysis.

Sergey founded Data Cowboys to bring research-grade thinking, honest evaluation, and clear communication to organizations solving hard problems. He holds a PhD in machine learning from the University of Washington and has been building AI systems since 2007.

Seattle

Ilya Barshai is a Principal Data Scientist at McGraw Hill, where he applies AI and machine learning to educational technology. He brings over a decade of hands-on data science experience.

Ilya joined Data Cowboys in 2016, bringing a practitioner’s eye for what actually works: rigorous analysis, careful modeling, and straight answers. Before moving into data science, he spent eight years in risk and failure analysis of electromechanical product designs.

Chicago

What clients say

Sergey and Ilya grasped what we do the fastest of anyone I’ve explained our startup to. They are professional, timely, and so fast with their thinking and output.

— Jay Goyal, Co-Founder, Actively Learn & Alongside

Sergey has a wonderful knack for quickly scoping a problem, making efficient decisions on model choice and parameters, and clearly communicating throughout the process.

— Apu Mishra, Lead Data Scientist, RichRelevance

Sergey Feldman is brilliant, collaborative, candid, and lightning-fast. He has a special gift for explaining complex analytical concepts in terms anyone can understand.

— Deb Arnold, Principal, Deb Arnold, Inc.

Sergey provides well-considered results quickly with in-depth knowledge communicated clearly to readers at every level.

— Jenny Dearborn, author of Data Driven

Data Cowboys’ work has been incredible. We understand our data, verify our impact, and keep people safe from exploitation.

— Robert Beiser, Seattle Against Slavery

From the blog

AUROC comparison across models on 108 datasets.
Fig. 1 Mean AUROC of each model across 108 small datasets.

Which ML Classifiers Work Best for Small Datasets?

We tested 108 small datasets to see what really works.

Schematic slope chart: six models with nearly equal held-out scores spread far apart once deployed.
Fig. 2 Schematic: models with equal held-out scores can diverge once deployed.

Underspecification in Machine Learning

Why two models with the same held-out score can still behave very differently in production.
A/B test chart of average clicks per query.
Fig. 3 A/B test of the new ranker: clicks on the results page.

Building Search for 240M+ Papers

How we improved Semantic Scholar search using three years of logs.
Sum of absolute SHAP values vs. Bernoulli probability p.
Fig. 4 Total SHAP importance of a Bernoulli feature as p varies.

SHAP Values & Feature Variance

Making ML results understandable to collaborators.

Talks & interviews

Interview · Authority Magazine

Five Things for a Successful AI Career

On building an AI career, evaluation, and lessons from research.
Podcast · YouTube

The Data Scientist Show

Discussion on machine learning for search, consulting for the Gates Foundation, AI for mental health, and career lessons.
Talk · WACh / Global Health

Machine Learning for Global Health

Predicting child mortality with the CHAIN network using XGBoost and SHAP values to identify key physiological drivers for targeted interventions.
Conference · PyData Miami 2019

You Should Probably Be Doing Nested Cross-Validation

Nested cross-validation for model selection and performance estimation on small datasets.

Got an AI or data question with real uncertainty?

Let’s figure out what the evidence says before you commit more time and budget.

Want to email instead? Reach us at ilya@data-cowboys.com.