Zach Flynn

Zach Flynn

Info

• Website: https://zflynn.com
• Email: zlflynn@gmail.com
• LinkedIn: https://www.linkedin.com/in/zlflynn
• Github: https://github.com/flynnzac
• Skills: Economics (Industrial Organization, Econometrics), Machine Learning, Data Science
• Current and Prior Employers: Udemy (an EdTech company), Compass Lexecon (an Economic Consulting company), Afiniti (an AI company), Amazon (an internet company)

Education

PhD in Economics

• The University of Wisconsin - Madison (2012-2017)
• Fields: Industrial Organization and Econometrics.
• Dissertation title: Measuring productivity and market power.

BS in Economics and Mathematics

• Tulane University (2008-2012)

Employment

Udemy

• Principal Data Scientist, Chicago, IL
• August 2022 to present
• Developed from the ground-up a new analysis framework for experiments at Udemy, incorporating:
  • Variance reduction methods.
  • Sample sizing methods that adjust for "peeking" as well as methods to choose the "optimal" level of peeking from the perspective of expected experiment duration.
  • Novel methods to reduce variance in quantile treatment effects.
• Developed a pre-experiment tool for metric selection based on composing multiple metrics into an ideal blended metric.
• Developed an estimator that is robust to "sample ratio mismatch" issues, allowing the system to use a correct-not-test approach.
• Wrote educational material on how to make decisions about experiments, interpret results as well as did short video tutorials.
• Developed an improved pricing model that:
  • Unified structural econometric techniques and causal inference methods into a unique pricing approach.
  • Modeled substitution between various products and product types.
  • Delivered pricing wins from fine-tuning the model for different markets.
• Developed a long-run, downstream impact model for experiments to give a bottom-line measured of impact across different product types (transactional and subscription purchases).

Compass Lexecon

• Senior Economist, Chicago, IL
• February 2020 to August 2022
• Assisted economic experts testifying on a variety of matters in antitrust (mergers, price-fixing, output restriction) and financial cases (allegations of "spoofing", alleged unlawful securities) by providing data analysis and economic analysis.

Afiniti

• Research Scientist, Research and Development, Washington, DC.
• July 1, 2018 to January 2020.
• Applied and developed models and built the tools to operationalize those models. A hybrid research scientist/engineer role.
• The company builds models to decide how best to pair callers and call center agents. It bills based on how much it increases, say, the sales rate or reduces the cancellation rate.
• I developed
  • an internal R package and strategy to predict the performance of models in production before they are deployed.
  • models that determined which call center agents and callers should be paired to maximize our client’s objectives.
  • a main component of the application we use to decide how to route calls between callers and agents (a production application in C++).
  • metrics to monitor how well different strategies for pairing callers and agents do in production once they are deployed.
  • research on new, next-generation matching algorithms.

Amazon

• Economist, Amazon (Prime), Seattle, WA.
• May 1, 2017 to June 30, 2018
• Designed, developed, and used a discrete choice demand model to help make pricing and product design decisions that was used in a wide variety of applications at the company. For example, it was used
  • to inform major Prime pricing decisions;
  • to separately value the various benefits of being a Prime member (free shipping, streaming video and music, etc)
  • to estimate the effect of modifying Prime benefits on Prime membership;
  • to estimate the effect of potential video content investment decisions on Prime membership.
• Developed a censored quantile regression approach to model what factors make customers buy a product or sign up for Prime sooner.
• Worked on many smaller projects, including analysis of which cities had the greatest opportunity size for advertising Prime during NFL games and which lower-income markets had the greatest opportunity size for advertising a lower-priced Prime for lower-income customers.
• Promoted to Economist II in April 2018.

University of Wisconsin - Madison, Economics Department

• Research Assistant to Jesse Gregory
  • Fall 2014 to August 2016
  • Implemented dynamic discrete choice models to understand housing decision problems for two projects: one on post-Katrina New Orleans and the other on the national housing market.
  • Used Condor at the CHTC at the University of Wisconsin Madison to estimate the large scale dynamic discrete choice models.
• Teaching Assistant
  • Introduction to Macroeconomics (Spring 2013, Spring 2014), Economic Statistics (Fall 2013), Introduction to Microeconomics (Fall 2012)

Programming languages and computing

Environments and programs: AWS, Linux, Databricks, Hive, Spark.

Programming languages: C/C++, R, Python, SQL (MySQL, Oracle, Redshift, PostgreSQL, etc), SAS, InfluxDB, Fortran, Matlab/Octave, Ada, Perl, Stata, Julia, Shell Scripting, Lisp, and Scheme.

For some examples of projects I have done/jobs where I used the above languages regularly:

• C/C++ — I used C++ for building on the core product at Afiniti and I have written C code for personal projects (see Github).
• R — All of my academic papers were mostly written in R (and Fortran). The censored quantile regression project at Amazon mentioned above was done in R as well. I used it frequently for projects at Compass Lexecon and Afiniti, including developing a widely-used internal R package for validating models.
• Python — I used Python for various internal projects at Afiniti, a few cases at Compass Lexecon, and for essentially all projects at Udemy, including the experiment analysis system mentioned above.
• Julia — I worked in Julia for an internal project at Afiniti.
• I used various SQL flavors daily at Afiniti, Amazon, and Udemy.
• Stata — Some of my RA work at Wisconsin was in Stata, the paper with Lenadro Magnusson developed a Stata package, and I used Stata at Amazon and Compass Lexecon for many projects.
• RA work with Jesse Gregory used both Matlab and Fortran.
• I use Perl in personal scripts, and I used it to automate certain tasks at Amazon.
• I used SAS in cases at Compass Lexecon.

Writing

The latest versions of all papers are also linked at my website: https://zflynn.com. I have (mainly) studied econometric issues, how to measure productivity, and how productivity affects market structure and economic growth.

Published Papers

• Identifying productivity when it is a factor of production (RAND Journal of Economics, Summer 2020)
• Parametric Inference Using Structural Breaks (with Leandro Magnusson). Stata Journal (2013).

Short writing/Blogs

• Losing Is The Value Of Experimentation
• Peeking Not Considered Harmful
• Stop the Count! Why Putting A Time Limit on Metrics is Critical for Fast and Accurate Experiments
• In Defense of Statistical Significance
• Scale Experiment Decision-Making with Programmatic Decision Rules

Working Papers

• Measuring markups with production data (with Amit Gandhi and James Traina)
• Inference based on continuous linear inequalities via semi-infinite programming
• Ex-ante evaluation of a policy’s effect on productivity: the substitution and scale effects of carbon taxation on power plant productivity
• Unproductive by choice: substitution and the slowdown in aggregate productivity growth in the United States
• Partial identification of production functions with flexible inputs (with Amit Gandhi)
• Identifying the elasticity of experience and its effect on market structure

Awards and Fellowships

• Who's Who in Experimentation 2025 — Kameleoon
• Alice S. Gengler Fellowship — University of Wisconsin Madison — awarded May 2016
• Culbertson Prize for Top Field Paper — University of Wisconsin Madison — awarded May 2015
• Graduate Research Fellowship — University of Wisconsin Madison — awarded May 2015