The Big Short is a fabulous film about the global crash of 2007–2008. It stars Steve Carell as Mark, a financial dealer who is obnoxious, loud and utterly correct in his analysis of a financial market that was white-ant riddled with fraud. As a piece of drama it’s great watching, not least because it depicts—in Panavision and at 100 decibels—the incentives that motivate private sector operators.

Who would have thought that money motivates companies and individuals to act in sensible ways—because markets do work—but also in weird and despicable ways? Ways that cause tangible harm to others? Who would have thought that we should not only think about the costs of regulation but also the benefits?

In one scene, Mark and his partner, Vinny, visit a ratings agency, whose purpose in commercial life is to rate the ability of debtor companies to meet financial liabilities as they come due.

Now a credit agency is not a regulator; rather it is a private service that provides markets with information about the financial standing of target companies. In theory, credit agencies perform an important function in that they improve knowledge in the market and reduce transaction costs. Not having expert knowledge ourselves about financial product X and company Y, we rely on an expert, a ratings agency. As a result, the market works better. Regulators very often perform a similar function by ensuring that buyers and sellers in a market are well informed.

In theory, rating agencies protect their long term interests in the market by rating target companies accurately. In theory, the product they sell is integrity of information, not just information. The Big Short, however, depicts something different: rating companies that were dependent on the revenues they received from the debtor companies they were rating. A conflict of interest existed because the issuer of a particular financial instrument paid the rating agency to rate that instrument.

Accordingly, junk bonds, by process of alchemy, were rated AAA— ratings agencies were, in the words of Vinny, ‘Holy shit, selling ratings for fees.’

Speaking of things that fall short of the ideal, we turn to the oversight and regulation of property and apartment development and construction in Australia. This is a sector defined by a similar corrupt structure of incentives. Compliance with building codes and standards was put, by deliberate action of state governments, into the hands of private sector operators who were effectively selling compliance for fees.

It’s hard to think of a worse incentive structure than one that:

• hands over the responsibilities of compliance to a privatised entity
• makes this entity financially dependent on the companies whose products they are required to certify as being compliant
• sits back and watch as dangerous or substandard work was waved through as being fully compliant
• sits back and watch as owners saw the value of their apartments trashed
• sits back and watch as owners were driven from the apartments they scrimped and saved to purchase.

Even the worst scoundrel would find it hard to fabricate a set of structures that were more compromised, less independent and less conducive to compliance.

To regulate properly is hard. But a few simple rules help us think clearly and regulate effectively.

First, focus on outcomes—not outputs or inputs. An additional $10 million in resources for a regulator is not an outcome. Additional resources may be necessary but are not sufficient.

Second, focus on the outcome that reduces harms to the greatest extent. Not all harms are equal; not all harms are worthy of regulatory attention. What is the regulatory response that generates the greatest net social benefit?

Third, make sure that you measure what you are regulating. You can’t manage what you can’t measure. Regulation only achieves an ends if behaviour is measured and breaches identified and enforced. A classic case is the extraction of surface water—e.g. pumping water from rivers for irrigation. Regulating the use of water through a licence and allocation regime is pointless if the amount being taken by users is not actually measured. Accurate, tamper-proof meters must provide real time information, which enables the monitoring of individuals and catchments.

Fourth, consult certainly, but resist capture. Understand that regulated entities have a very clear set of incentives to reduce the impact of regulation on their activities—because by doing so they either increase revenue or reduce costs. Some regulated entities are relentless in searching out ways to game a regulatory regime.

Fifth, build expertise. Regulation is a complex business and technical expertise in the business of regulation, for example, aircraft design, catchment systems, therapeutic drugs, and alcohol and health, underpins effective regulation.

Sixth, enforce and punish to realise outcomes. This is tricky. Sometimes enforceable undertakings are exactly what are required because they actually maximise net social benefit. Other times punitive sanctions may be called for. Don’t be formulaic. The questions here are: What reduces harms now and what sanction will reduce harms in the future?

Finally, understand that regulation imposes costs on society. So, just because you can regulate an industry or sector doesn’t mean you should. See rule number two.

If you’re not a subscriber to our newsletter then you’re missing out and you should join up here