Data Duets

Image – Data scientist steering AI agents || Around this time last year, agentic AI took a major leap. It is no longer just a tool for generating code, rewriting prose, or summarizing documents. In our own work, we are watching agent collectives conduct literature sweeps, extract and clean data, build plausible models, and synthesize insights, all orchestrated under the supervision of a lead agent. While an individual agent remains imperfect, an orchestrated collective is genuinely impressive. Given these capabilities, what does the future of data science hold, and what will modern data science teams look like? We spoke with five data science leaders across retail and tech to assess the state of the field: the impact of LLMs and agentic workflows on day-to-day work (via automation or augmentation), the limitations that persist, shifts in the market's demand for data science talent, and ways data scientists can stay ahead as the role evolves. Our key takeaways include the...

Image – Synthetic customers ranking different promotion cues tl;dr (never ai;dr) LLMs can simulate shoppers and help data science teams prescreen options before running experiments. One good use case is ranking options, not estimating effect sizes, if the model's bias is order-preserving. Simulation quality depends on methodological details such as persona construction and representation, clear experimental instructions, response elicitation, and robustness checks. Recent evidence shows a major robustness problem: even when persona content stays the same, changing only the textual representation can shift simulated agent behavior by more than 70 percentage points. At this point, synthetic panels are safest as exploratory playgrounds unless results survive serious robustness audits across prompts, personas, and model choices. Podcast-style summary by NotebookLM We are taking a break in our augmented data science se...

Image – Augmented Data Science: Data cleaning agent tl;dr (never ai;dr) Data cleaning is an execution-heavy task, but only after the data scientist defines the modeling objective. We developed and tested a data-cleaning agent skill that profiles the data, separates method-invariant cleaning from method-dependent cleaning, and labels each step as diagnose , act , or ask . On a large, rich dataset with Amazon purchases and purchaser demographics, the agent produced a modeling-ready table compared to a baseline prompt that generically applied data cleaning. The skill made decisions explicit. This helped detect leakage in our use case: the skill excluded same-period purchase-derived features such as price, spend, and frequency because those features leak the target. Our analysis here supported the Augmented Data Science framework we introduced earlier: the LLM can execute cleaning, but the data scientist must set the target, method...

Image – Augmented Data Science: Method selection agent tl;dr (never ai;dr) Method selection is a stage where human judgment matters most: the hard part is often not fitting a model, but choosing a method whose assumptions match the business question and the data-generating process. We built a four-step agent skill ( context gathering , causal, predictive, prescriptive, or descriptive framing , branch-specific clarifying questions , and an assumption-aware evidence table ) to structure that choice. Testing the skill on price elasticity showed how the agent's questions shape the identification strategy. We tested the skill using two recent models: Claude Opus 4.7 emphasized instruments and design choices, while GPT-5.5 emphasized dose-response and machine-learning approaches. The skill provides directions, not decisions. The data scientist must decide which assumptions are plausible and which claims stakeholders actually need...

Image – Augmented Data Science: Hypothesis search agent tl;dr (never ai;dr) Generating testable hypotheses has mostly relied on the data scientist's experience and a literature review (in line with guidance from the business team). An LLM agent skill can structure and expand that search. We built a three-step skill ( context gathering , causal vs. predictive framing , and evidence-backed hypothesis table ) and tested it on two retail business questions using Claude Opus 4.6 and GPT 5.4. Framing the problem and providing the agent with available variables did most of the work: with minimal context, the causal/predictive distinction produced useful, literature-backed hypotheses. 86% of references checked out; directional claims were reliable, but effect sizes were not. Consistent with prior work, we found that LLMs exaggerate the findings in existing research. This confirms our earlier conclusion: data science teams need to e...

Image: Synergy between human intent and AI execution in data science tl;dr (never ai;dr) LLMs are now part of data science workflows. The question is no longer if we use them, but how and where . We reviewed emerging research to create a framework defining the synergy between human and machine: human intent followed by (and bounding) LLM execution . Effective AI-assisted workflows successfully decouple intent (the "what" and "why" ) from execution (the "how" ). In this model, the data scientist acts as the orchestrator , while the LLM serves as the execution engine . Success hinges on assigning intent to the correct tasks. The data scientist sets goals and validates assumptions . The LLM searches well-defined spaces and executes code subject to human validation. This post sets the stage for a new series where we hope to introduce a task-level guide for data science leaders embedding LLMs int...

Image: Distributions of the real vs. synthetic data for selected variables tl;dr (never ai;dr) One underappreciated use case of generative models is effectively creating realistic tabular datasets that preserve the underlying statistical properties of the original data. Leading libraries for data synthesis include Synthetic Data Vault, YData-Synthetic, and Synthcity. Practical applications include navigating the bottlenecks of sharing sensitive data with vendors or augmenting datasets for rare events, such as product recalls. Ultimately, this approach enables a data-centric workflow even when data is scarce or biased, ensuring models are trained on a high-fidelity representation of reality. Podcast-style summary by NotebookLM Introduction How can we use generative models beyond ...