In law’s new frontiers, data may be as important as precedent

Are there greater racial disparities in criminal sentences now that federal sentencing guidelines are no longer mandatory? If health insurance companies are required to provide coverage for fertility treatments, will fewer children be adopted? Do staggered corporate boards provide more or less value for shareholders? For the growing number of empiricists at HLS, there is nothing quite so satisfying—or unimpeachable—as resolving a thorny, often contentious, legal policy question through rigorous analysis of cold, hard data.

Credit: Webb Chappell Crystal Yang

As a student at HLS, Crystal Yang '13—appointed last year as an HLS assistant professor—set out to determine whether the mandatory application of federal sentencing guidelines resulted in more or less racial disparity in sentencing. In 2005, in United States v. Booker, the U.S. Supreme Court held that the U.S. Sentencing Guidelines, by which federal judges impose criminal sentences, were to be advisory only. Comparing data from courts, Yang, who holds a master's in statistics and a Ph.D. in economics from Harvard, found that greater judicial discretion post-Booker has resulted in African-American defendants receiving longer sentences than comparable white defendants.

"So many policies have assumptions embedded within them, that this is how people respond," says Yang, whose article about her findings will be published in The Journal of Legal Studies. "But if we show through empirical work that that is not how something works, that's very important for lawmakers to know," so that laws and policies have their intended consequences.

Empirical legal studies is the application of data-based research methodologies, both quantitative and qualitative, to legal and law-related policy questions. Long relied upon in medicine, economics, psychology and social sciences, empirical research can take any number of forms, from analyzing huge sets of existing data (tens of thousands of records of tort cases in Massachusetts, say) to running field experiments to generate data, an approach in which HLS Professor D. James Greiner has taken the lead.

While empirical legal studies is not at all new, it is experiencing enormous growth and is arguably the hottest area of legal thought today.

For the empiricists at HLS, there is nothing quite so satisfying as resolving a thorny legal or policy question through the analysis of cold, hard data.

"Lawyers often make claims about policy changes they think would be good or a legal problem that needs to be solved, but often those adjustments are made from an armchair," says HLS Professor I. Glenn Cohen '03. "But things in the real world don't always act as we think they should. Instead of just arguing and assuming this or that is true," he said, "let's actually find out the answer." Faculty director of the Petrie-Flom Center for Health Law Policy, Biotechnology, and Bioethics, Cohen has done empirical research that has revealed, for instance, that insurance coverage for fertility treatments does not lead to a reduction in adoptions.

In corporate law and governance, an empirical approach has been prominent for at least 10 years, with Lucian Bebchuk LL.M. '80 S.J.D. '84, Allen Ferrell '95, John Coates (see Page 8), Guhan Subramanian J.D./M.B.A. '98 and Holger Spamann S.J.D. '09 among the HLS faculty producing influential research. Spamann has also used empirical methods in other areas, including to assess criminal sentencing practices.

There is also a significant intersection between ELS and the field of law and economics, which, at HLS, is centered at the John M. Olin Center for Law, Economics, and Business under the direction of Professor Steven Shavell. Other HLS programs, including the Center on the Legal Profession, also take an empirical approach to research.

Courses involving empirical legal reseach have long been offered at the school. For example, Elizabeth Warren, now a U.S. senator and HLS professor emerita, taught Empirical Analysis of Law, drawing on her own experience as a scholar. For years, Professor Kathryn E. Spier has taught the popular Analytical Methods for Lawyers, as has Lecturer David Cope.

With a growing recognition that an empirical approach could be invaluable to virtually any type of legal question, HLS is giving the field even greater emphasis, reflecting not just academic trends but demand from students and even law firms.

Credit: Webb Chappell Alma Cohen and Oren Bar-Gill, who both hold Ph.D.s in economics, joined the HLS faculty last year. Cohen's research includes empirical studies on risk, regulation and corporate governance. Bar-Gill, an expert on contracts, is currently working on a study that relies on research subjects to test how different legal protections affect how people bargain over entitlements.

Last year, in addition to Yang, HLS appointed two other empirically focused faculty, both with Ph.D.s in economics: Professor Oren Bar-Gill LL.M. '01 S.J.D. '05, whose field is contracts, and Alma Cohen, HLS Professor of Empirical Practice. Cohen, an empirical economist, has analyzed data that found that there is decreased value to shareholders of staggered corporate boards, that different groups vary in their responsiveness to governmental subsidies for childbearing, and that mandatory seat belt laws do reduce fatal accidents—but not as much as the government claims.

"An empirical revolution has been taking place in all fields of social science as well as in law," Alma Cohen says. "In many areas of legal scholarship, the share of work that is empirical or informed by empirical work has substantially increased." It's also increasingly prominent in litigation, regulatory rulemaking and legal policy discussions—a shift due, in part, to the "vast increase of data that is available and the wide accessibility that all of us now have to the computational power that is needed for most of the empirical work," she adds.

Credit: Michal Fattal / Getty Images Netta Barak-Corren S.J.D. '16 in Israel, where she is conducting empirical research exploring the circumstances in which people comply with the law when it conflicts with their religious beliefs.

HLS now has a statistician on the library staff to assist with faculty and student research, and an Empirical Legal Studies group, known as HELS and launched two years ago by Netta Barak-Corren S.J.D. '16, which brings in top empirical scholars, offers training workshops, and serves as a community for students who conduct empirical research themselves. And there are now a range of new courses on empirical methodology aimed at budding litigators as well as scholars.

"In terms of the practice of law, empirical analysis—including valuation and event study analyses—has proved enormously important in complex business and securities litigation," says Ferrell, who recently won a prestigious prize for outstanding quantitative research in socially responsible investing and who teaches an empirical legal methods course with Alma Cohen. "Litigators must deal with empirical issues in terms of how the markets work, how to think about causation, and damages and settlement value on a regular basis."

Many empiricists agree that there are long-standing assumptions in policy and law that likely rest on faulty premises, since they were not derived from actual data-based research.

"I think that's why there's more recognition of the need for having empirical legal studies, especially in law schools," says Yang. Law is at the center of so much that "it is vitally important" to have lawyers who also have the methodological tools to do empirical research.

No one is suggesting that a law school course or two would prepare someone to conduct statistically valid research. Rather, the HLS courses, and the workshops through the Empirical Legal Studies group, are intended to give students fluency in understanding empirical research whether they are practicing law or are interested in getting further education in empirical methods.

"In my courses, I try to teach students how to assess the reliability and limitations of empirical findings."

—Alma Cohen

"While empirical analysis is nowa­days often easy to conduct, it often produces results that should be used with caution and with keen awareness of their limitations," Alma Cohen says. "In my courses, I try to teach students how to assess the reliability and limitations of empirical findings."

Even lawyers with statistical backgrounds find that teaming up with social scientists or economists is essential to producing reliable results that can be defended. Bar-Gill is working with a colleague at the Max Planck Institute in Germany on an experiment—his first—that relies on research subjects to test how different types of legal protections affect how people bargain over reallocation of entitlements. While he expects to do more of such experiments, he acknowledges that "there's a big question of external validity: If you find something in [such experiments], does that mean you'd get the same results in the real world?"

That's why James Greiner is conducting a different kind of experiment, a "field experiment," studying people in actual legal settings. These randomized controlled trials, unusual in legal studies, have been established as the "gold standards" of empirical research in other fields. People are divided into groups, with some receiving a particular intervention and a control group that is not. In one experiment, he is working with the Boston Municipal Court to see if the default rate by defendants sued for consumer debt can be reduced by sending low-cost mailers that explain the legal process and encourage people to show up for court. The answer: Yes. According to preliminary results, the mailers have at least doubled the number of defendants who avoid default.

Andrea Matthews '15, who did empirical studies at a think tank before coming to law school, chose HLS because of Greiner, and has worked with him on the Boston court study and other empirical projects. "I'd been searching for someone doing randomized experiments in law," says Matthews, who next year will be a lawyer in the honors program at the federal Consumer Financial Protection Bureau in Washington, D.C. "I think the empirical standpoint is something the CFPB prizes and seeks," she adds.

But empirical research costs money, and it's unlikely that there will ever be enough resources to apply this approach everywhere it could be useful. Still, says Glenn Cohen, recognizing its importance means "at least you can prioritize important questions." And although conducting rigorously devised and executed studies is expensive, "the social harm of not aggressively investigating what works and what doesn't" must be weighed against the cost, insists Matthews.

As the value of this research becomes increasingly apparent to policymakers, litigators and others, empiricists expect their approach to continue to grow in all areas of the law. After all, says Barak-Corren, "Data can be quite a convincing tool."