Projects

Copilot in Excel

An Excel sheet with a race-geography dashboard built by Copilot, next to the chat pane showing the request to analyze races by continent over the decades and Copilot's summary of the trends it found
Copilot in Excel analyzing a dataset of races by continent over the decades and building a dashboard of the trends it finds.

Over the last couple of years I have worked with the Excel team on the agent behind Copilot in Excel. Spreadsheets are a demanding setting for an agent. The data is messy, requests are often under-specified, and a result that is almost right can be worse than none at all. At the same time, high-stakes decisions, from budgets to financial reports, rest on spreadsheet analysis. Even a one-line request, like the one in the screenshot above, turns into many steps: exploring the data, deciding what is worth analyzing, and building something the user can act on.

One large part of this work is the agent itself: the harness that drives it, the tools it uses to read and change a workbook, and the prompts and skills that guide it. For example, should the agent work with a spreadsheet by writing programs, by reading and writing cells directly, or by looking at screenshots? Each choice has its own strengths and weaknesses for output quality and cost, and each has a large effect on what the user experiences.

The other large part is measurement. We build benchmarks and graders from realistic spreadsheet tasks, so that we can tell whether a new prompt, skill, or model actually helps; ConDABench (SIGMOD Companion 2026) is one example, for open-ended data analysis in conversation. Measurement in turn makes self-improvement possible, and the ideas behind SkillAxe and DUET came directly from trying to improve a shipping agent.

This project has been a very enriching experience for me. Beyond the research, I have been involved in product and engineering decisions, deployment strategy, and collecting and acting on user feedback. It has taught me, up close, what it takes to drive a complex agentic project end to end.

Related Papers

    Evaluation and Self-Improvement for Agents

    Overview of DUET: adaptive task selection, solver and grader agents, and an update step that alternates between revising the grader and the solver
    DUET co-evolves a solver agent and its grader, updating one while holding the other fixed. Figure from the DUET paper.

    Judging how well an agent is doing is hard, especially when its task involves back-and-forth with a person. Should an agent ask many questions to pin down every detail of a task, or make reasonable assumptions and proceed? ConDABench (SIGMOD Companion 2026) evaluates open-ended data analysis in interactive settings, where questions like this come up constantly. RUBICON (AIware 2024) generates rubrics for judging domain-specific conversations. In both cases, a good evaluation has to judge the whole interaction, not just whether the final answer is right.

    Once we can measure how good an agent is, we can try to have it improve itself automatically. Self-improvement matters because agents need to keep up with a fast-changing landscape of new models and tools. MetaReflection (EMNLP 2024) improves an agent's instructions, StackFeed (EMNLP 2025, industry track) its knowledge base, and SkillAxe its skills. DUET ties evaluation and self-improvement together by evolving the agent and its grader side by side, so the agent cannot simply learn to exploit the grader's blind spots. Ultimately, we want agents that keep getting better on their own, while people stay in control of what changes.

    Related Papers

      Blue Pencil and IntelliCode

      The Visual Studio lightbulb menu offering an IntelliCode suggestion based on recent edits, with a preview of the change
      An IntelliCode suggestion based on recent edits in Visual Studio. Figure from the ICSE-SEIP 2023 paper.

      Blue Pencil started from a simple observation: a lot of programming is making the same small change over and over. You rename a parameter and then fix every call site, or add a null check in one place and then in the five other places that need it. Blue Pencil watches the edits you make, and once you have made a change once or twice, it works out the pattern and offers to make the rest of the changes for you. You just keep coding, and Blue Pencil automatically starts producing suggestions.

      The core on-the-fly synthesis algorithm was introduced in our OOPSLA 2019 paper and first shipped in Visual Studio 2019. With sustained research and iteration based on user feedback, it eventually grew to automate over a million repeated edits each day. Our OOPSLA 2020 paper extended the technique to generalize from a single edit, without needing repetitions, and our OOPSLA 2022 paper to learn from sequences of common edits. Our ICSE SEIP 2023 paper laid out the HCI principles behind presenting suggestions, and its recommendations are still followed in Visual Studio today.

      This line of work spans program synthesis, human-computer interaction, and machine learning. One of the most rewarding moments was learning how much it helped developers with disabilities.

      When every click hurts, every click you can save means all the more.

      For me, this is huge. It took fewer gestures to edit 10 lines than it would take to just get the cursor onto the next line without IntelliCode.

      Anna White, software engineer at Microsoft, in Inclusive Coding Unlocked (her repeated-edits demo starts at 3:52)

      Video: Blue-Pencil, modeless program synthesis · OOPSLA 2020 talk · Visual Studio blog: repeated edits · Visual Studio blog: suggestions in the completion list

      Related Papers

        Program Synthesis

        A loop in which annotations train an ML model, the model proposes candidate programs, these are combined into a disjunctive program, and a semi-automated annotator produces new annotations
        Combining synthesis and machine learning for data extraction: an ML model proposes candidate programs, which are combined into a disjunctive program that helps annotate more data. Adapted from our PLDI 2019 paper.

        Program synthesis is the task of automatically generating a program from a specification of what it should do, such as a logical formula or a few input-output examples. Program synthesis was the primary area of my research for many years, and I have worked on both new synthesis algorithms and applications of synthesis to real-world problems.

        My most influential work has been EUSolver (TACAS 2017), which decomposes a large synthesis task into smaller programs that are each correct on a subset of the inputs, and then combines them using decision trees. EUSolver won the general track of the SyGuS competition in 2016 and 2017. Another key contribution is cut functions from FlashFill++ (POPL 2023), a new way to efficiently search large DSLs and produce readable programs. FlashFill++ is deployed in products ranging from spreadsheets to notebooks and business intelligence tools. I have also worked on bringing machine learning into synthesis. In our PLDI 2019 work, an ML model and a synthesizer bootstrap each other to extract data from heterogeneous documents, and more recent work uses pre-trained language models to guide the search for programs (OOPSLA 2021, arXiv 2025).

        Related Papers