Private equity investors have ridden a roller-coaster of returns over the decades. But less visible among the upward and downward gyrations has been a longer-term secular decline in returns, as the industry has matured and competition intensified.

Private equity returns through the first half of 2011 were strong as portfolio valuations continued to rise, powering short-term gains. But while a healthy pop in returns is always welcome, the conditions that have steadily driven them up over the past two years are now coming to an end and are not likely to be repeated.

As we reported in Bain’s Global Private Equity Report 2012, the consistent quarterly gains private equity investors have seen were the by-product of the confluence of the equity market implosion following the 2008 financial crisis and the PE industry’s transition to mark-to-market accounting rules. When equities took their hard hit that autumn, PE firms promptly and aggressively wrote down their portfolio company net asset valuations (NAVs).

As the public markets recovered, GPs continued to value portfolio companies conservatively with a result that private valuations have lagged the recovery of public valuations. Subsequent restatements since the second quarter of 2009 have gradually and smoothly lifted the depressed private equity NAVs out of their deep hole.

With private equity assets now appraised close to their intrinsic value, returns will become more volatile as they more closely track the ups and downs of the public markets. But it is far from clear what external market forces, or “beta,” will power the next wave of PE returns. Certainly, the familiar combination of factors that PE funds relied on in the past—GDP growth, multiple expansion and leverage—do not look nearly as favorable in the current recovery as they had in past ones.

With weakness apt to continue in the traditional sources of market beta, returns that depend on them do not look poised for a great leap forward. To continue to produce returns that will sustain private equity’s edge over other asset classes, GPs will need to dig deeper to generate alpha, the returns that rely on the superior management talent of PE firms to lift the performance of the assets they control above that of comparable public companies.

Evidence of the power of alpha in private equity is compelling. Oliver Gottschalg, a professor at the École des Hautes Études Commerciales (HEC) in Paris, demonstrates the alpha effect by comparing actual private equity returns to stock market returns of similar investments in company shares (see figure). In his most recent research with Golding Capital Partners, Gottschalg estimates that return margin to be worth an additional 5 percentage points. The PE alpha effect has been exceptionally strong in times of economic turbulence. By Gottschalg’s calculation, PE returns generated through alpha added 18 percentage points to PE performance during periods of economic recession.

This post was written by Graham Elton, Bill Halloran, Hugh MacArthur and Suvir Varma, leaders of Bain & Company’s Private Equity Group.