January 9, 2002
Updated: November 3, 2011
Up to my Finance page.
The Arguments for Investing in Total Markets
Efficient Reasons for Differing from the Market
Behavioral Reasons for Differing from the Market
Conclusions and Advice
The author thanks the following people who reviewed drafts of this article and made major contributions: Victoria Fineberg, Taylor Larimore, Mel Lindauer, and Richard Pritz.
Financial academics and other experts often recommend that investors should use index mutual funds to invest in entire markets. This advice is common with both stocks and bonds and with both domestic and international investing.
Why do we so often hear this advice? When is it appropriate for an individual investor to not follow it or to use it as a starting point but deviate from it? These are the questions we will address in this article.
People who give this advice include leading academics like Eugene Fama, Burton Malkiel, Robert Merton, Paul Samuelson, and William Sharpe. (Merton, Samuelson, and Sharpe are Nobel Prize recipients.) John Bogle is a major proponent of this philosophy. Mr. Bogle is the founder of the Vanguard family of mutual funds and the creator of the first retail index mutual fund, the Vanguard 500 Index Fund. (This fund, once called "Bogle's Folly," grew to become the largest mutual fund in the world.) Many highly respected and popular financial professionals, columnists, analysts, and book authors also give this advice, including Bob Brinker, Scott Burns, Jonathan Clements, Charles Ellis, Sheldon Jacobs, Jane Bryant Quinn, John Rekenthaler, Gus Sauter, and Jason Zweig.
The following Vanguard index mutual funds are often used to follow this advice.
This article discusses the advantages of total-market indexing, not just any kind of indexing. We also require that the indices and mutual funds be capitalization-weighted, not equal-weighted. (In a capitalization-weighted index, each security is weighted in proportion to its total current market value. Thus a large company comprises a larger percentage of the index than does a small company.) It is important to keep these distinctions in mind.
Our discussions in this article apply to all of the major individual markets in the world, including the US stock market, the US bond market, and the non-US markets. They also apply to the combination of these markets, the entire global financial market considered as a single entity encompassing all of the world's investable assets.
Before we begin, there are two basic principles of investing which are important to remember as we develop our arguments.
The first basic principle is that there are no "sure bets" in the financial markets. Even safe instruments like US Treasury bills, bank savings accounts, and money market funds can lose ground to the ravages of inflation. Any kind of investment which has a higher expected return than these "risk-free" investments can fail (lose money) even over long time horizons. The "expected return" for an investment is an estimate of the average return for the investment depending on what happens in an uncertain future. It is by no means a guaranteed return.
The second basic principle is that with one exception there is no "free lunch" in investing. Risk and reward (expected return) always go hand-in-hand. The only way to increase reward is to increase risk, and the only way to decrease risk is to decrease reward. Changing a portfolio to get a higher expected return (e.g., by increasing or decreasing the percentage weight of a sector or style or asset class) always increases its risk. Diversification is the only exception to this rule. Making a portfolio more diversified by investing in a broader range of securities is the only way to simultaneously increase its expected return and decrease its risk. We will talk more about diversification in the next section.
The arguments for investing in total markets are based on market history, simple arithmetic, and financial theory.
Many people feel that the strongest arguments for investing in total markets are based on market history. Innumerable studies have shown that individual investors and even professionals like institutional fund managers and mutual fund managers have terrible track records trying to beat the market. For example, many studies have shown that total-market index mutual funds consistently outperform most actively managed mutual funds.
It is easy to see why most actively managed mutual funds must fail to beat their passive total-market index fund counterparts. The performance of "the market" is nothing more or less than the capitalization-weighted average of the performance of all the investors in the market. If some active managers outperform, other active managers must underperform. For every winner (a performer above the average), there must be a loser (a performer below the average). This is a matter of simple arithmetic. Index funds match the market's performance minus their low expenses. Actively managed funds have considerably higher expenses than index funds because they must pay for expensive research into individual securities and sectors, and they must pay extra costs for frequent trading of individual securities. As a consequence, the average performance of all actively managed mutual funds must trail the performance of total-market index mutual funds by the significant difference in their expenses. John Bogle has done many studies that confirm that this is exactly what happens in practice.
As an example, in his recent speech The Wisdom of Investment - The Folly of Speculation, Bogle compares long-term performance figures for indexing versus active management for US stock and bond investing.
|US Stocks, August 1976 - October 2001|
|Vanguard 500 Index Fund||13.2%||$22.7 million|
|Average US stock mutual fund||11.1%||$14.1 million|
|US Bonds, 1986 - October 2001|
|Vanguard Total Bond Market Index Fund||8.0%||$3.1 million|
|Average US bond mutual fund||7.0%||$2.7 million|
The differences in the total returns over these periods match the differences between the low expenses of the index funds and the high expenses of the managed funds, as expected.
Notice how these seemingly small differences in annualized total returns (2.1% for stocks, 1.0% for bonds) result in much larger differences in total wealth over long time periods. In the US stocks example the ending wealth after 25 years is 61% higher with indexing, a difference of $8,600,000. In the US bonds example the ending wealth after 15 years is 15% higher with indexing, a difference of $400,000. This is due to the effects of compounding. Over long time periods the extra expenses of active management make an enormous difference.
(Bogle used the Vanguard 500 Index Fund in his example because it has existed much longer than Vanguard's Total Stock Market Index Fund. Results for the Total Stock Market Index Fund since its inception are similar.)
Bogle's numbers are just averages. Over any given time period some active managers do in fact manage to outperform the market even if most of them do not. Is this luck or skill? If it is skill, we would expect those active managers who have outperformed in the past to continue to outperform in the future. Many studies have been done to test this hypothesis. The general conclusion is that there is almost no persistence in mutual fund performance. Past performance is generally unrelated to future performance. Overperformance and underperformance appear to be nearly random. It is interesting that one of the few cases of persistence in performance that has been discovered is that mutual funds with very high expenses that have underperformed in the past tend to continue to underperform in the future.
A few active managers like Warren Buffett and Peter Lynch have astounding track records over long periods of time. How is it possible to claim they were just lucky? Perhaps their success was indeed due to skill. Then again, perhaps even they were just lucky. After all, if you have 1,000 chimpanzees flip coins ten times in a row, it is likely that one of them will get heads all ten times. We call the chimpanzee lucky. If the chimpanzee were a mutual fund manager we would call it talented. In any case, whether it is luck or skill, these few long-term success stories are in the past. How is an investor supposed to identify the next Buffett or Lynch in advance?
The financial theory used to support investing in total markets is called the "Efficient Market Hypothesis," or "EMH" for short. (It is also known as EMT, or Efficient Market Theory.) This theory was first proposed by Eugene Fama at the University of Chicago in the 1970s. It has since become a cornerstone of modern academic Finance, although it is by no means uncontroversial.
The Efficient Market Hypothesis states that modern financial markets are "efficient." This means that they quickly react so that prices reflect all available information. For example, new information can often cause prices to rise or fall on the world's major stock and bond exchanges to adjust appropriately within only minutes. Prices of individual securities, market sectors and style segments, and entire stock and bond markets are therefore always "correct" in the sense that they always reflect the collective beliefs of all investors taken together as a whole about their future prospects.
One major consequence of the EMH is that unless an investor is just plain lucky, it is impossible to exploit the market to make an abnormal profit by using any information that the market already knows. Another consequence is that for someone without any such private information, it does not make any sense to talk about "undervalued" or "overvalued" individual securities, sectors, styles, or markets.
Another consequence of the EMH is that the total market is always perfectly diversified. Other portfolios that deviate from the total-market portfolio are never "more diversified" than the total-market portfolio. In other words, it is impossible for any other portfolio to have both a higher expected return and lower risk than the total-market portfolio. (The technical way to say this is that the total-market portfolio is always located on the "efficient frontier" in academic risk/return models. This is true in both the classical single-factor risk model of Markowitz and Sharpe and in the newer three-factor risk model of Fama and French.)
The EMH does not require that investors behave rationally. This is a common misconception. When faced with new information, some investors may overreact and some may underreact. Indeed, this behavior is expected and common. Markets would not behave the way they do in the real world if everyone always reacted in the same perfectly rational way to new information. All that is required by the EMH is that these overreactions and underreactions be random enough and cancel each other out enough so that the net effect on market prices cannot be exploited to make an abnormal profit. Stated another way, irrationality is irrelevant as long as it is unpredictable and unexploitable. Even the entire market can behave irrationally for a long period of time and still be consistent with the EMH, again as long as this irrational behavior is not predictable or exploitable. Thus crashes, panics, bubbles, and depressions are all consistent with a belief that markets are efficient.
Note that the first two arguments, the ones based on market history and simple arithmetic, do not depend on the EMH. Those two arguments remain valid whether markets are efficient or not.
To summarize, we first observed that history demonstrates that it is difficult even for professionals to beat the total market. Second, simple arithmetic proves that it is impossible for every investor or even a majority of investors to beat the total market. Third, the theory of how markets work tells us that in order to beat the total market investors either have to be lucky or they have to know something that the market does not already know.
Finally, there are several more benefits to total-market index fund investing:
The Efficient Market Hypothesis states that the market always incorporates the collective beliefs of all investors taken as a whole about the future prospects of the market, and that without private information, current prices for the market as a whole and all of its component securities, sectors, and styles are correct. For this reason, the only prudent strategy for an average investor who believes that the market is efficient is to invest in the total market.
There is no such thing as "an average investor," of course. All investors have individual circumstances which make them different from the average. Each person usually has good reasons for having an investment portfolio which is different from the market average. All of these reasons involve balancing a person's investment portfolio with the non-market aspects of his or her financial life. Examples include the person's risk preferences (need and ability to take risk), home country, trade or profession, sources of income, tax situation, and age.
Here are a few examples of efficient reasons to differ from the total market:
All of these examples represent good reasons for differing from the total market. These investors are all making adjustments to a total-market portfolio to take into account their individual sets of circumstances. Their behaviors are all consistent with a belief that markets are efficient.
Notice how these examples are in pairs. In an efficient market, for every set of investors with good reasons to deviate from the average in one direction, there is some other set of investors with equally good reasons to deviate in the opposite direction. These investors are in effect using the markets to make trades of risks which benefit both sides of the trade. Many academic theorists say that this is in fact what efficient markets are all about.
Examples 1 through 4 in this list are the most important ones. The other ones are less important for most investors. We will discuss this in more detail in the conclusion.
In recent years a new field of "Behavioral Finance" has emerged. Some researchers in this field claim to have discovered consistent, persistent, and predictable irrational behavior by both institutional and individual investors that makes the markets inefficient. A few researchers and practitioners believe that they have identified particular inefficiencies which can be exploited by smart investors to make abnormal profits.
In addition, it seems that the overwhelming majority of investors simply cannot resist the temptation to "outsmart the market." Nearly everyone has a theory about which stocks and sectors and styles are currently "overvalued" or "undervalued," and there seems to be an endless supply of different strategies to take advantage of these theories to make abnormal profits, or at least to increase expected return while decreasing risk. All of these strategies also amount to "behavioral" reasons for differing from the total market, because their proponents all feel that other investors' behavior is incorrect, and their strategies are designed to take advantage of that misbehavior.
It would be impossible to present an exhaustive catalog of all of these arguments, theories, and strategies. If one can dream it up, no matter how outlandish it might be, someone probably already believes in it and is most likely selling a newsletter outlining a strategy to implement it. We will just give a sampling of some of the behavioral theories that currently seem to be popular. To be fair, we have included only the most logical ones, and we have omitted the truly outrageous ones.
Efficient market believers counter by saying that the aging population and the growing need for additional health care in the future are common knowledge and that this information is already incorporated into market prices for this sector. They say "Your point is valid, but you need to tell me what you know about the health care sector that the market does not already know." They do not advise that all or most people should either overweight or underweight this sector.
An efficient market believer first notices that the US stock market makes up less than half of the total global stock market, and it is not wise to ignore such a huge part of the world's wealth when investing. Second, while foreign stock returns have certainly been disappointing in the 90s, they dramatically outperformed US stocks in the 70s and the 80s, and nobody knows what they might do in the future.
Academic researchers say that adding foreign stocks to a US stock portfolio significantly improves its diversification and hence lowers its risk.
Investing only in what is familiar is dangerous advice. Ignoring foreign stocks for a US investor makes no more sense than ignoring stocks of companies located in states other than the one in which the investor happens to live.
While it is true that many non-US countries have unstable governments and inferior business practices, especially in emerging market countries, this knowledge is common and public and is hence already incorporated into market prices in those countries. This is in fact why the expected returns in those countries are so high.
Emerging markets are indeed risky, but the risk of an investor's total portfolio is what matters, not the risk of its individual pieces. Both the historical evidence and academic theory show that adding an appropriate amount of emerging markets stock to an already diversified international stock portfolio actually decreases its risk while at the same time it increases its expected return. This is the "free lunch" benefit of diversification.
People who believe that the global stock market is efficient or mostly efficient allocate a significant portion of their stock portfolio outside the country in which they live. They invest in every major country and region in the world, including emerging markets.
Eugene Fama, the same academic who first proposed the EMH and remains its staunchest defender, is one of the major researchers responsible for this discovery. Fama says that this "value premium" is due to higher risk for value investing. He continues to recommend total-market investing as the best starting point for most people. The total market already incorporates average exposure to both the risks and the rewards of value investing, so there is no benefit to either overweighting or underweighting value stocks for all or most investors.
Others disagree with Fama's analysis. In particular, some behaviorists argue that most investors, institutional ones and individual ones alike, consistently, irrationally, and persistently underprice value stocks and overprice growth stocks. They argue that value stocks are actually less risky than growth stocks, and that investing in value stocks is a "free lunch" that achieves the holy grail of investing - higher expected return with less risk. For people who side with these behaviorists, the obvious strategy is to heavily overweight value stocks. Robert Haugen is a champion of this point of view.
Some behavioral practitioners, however, prefer to ignore or deny the risk and they overweight small-cap stocks. Other behavioral practitioners are afraid of the risk and they underweight small-cap stocks.
People who believe that the markets are efficient start by holding total-market portfolios which have average exposure to the small-cap style. They then adjust their portfolio one way or the other only if there is an efficient reason for them to do so.
For example, some analysts (but by no means all) have the opinion that the P/E and P/B ratios for large growth stocks are still much too high. Sometimes people even say things like "the total stock market has too many large growth stocks," or "the total stock market is not diversified enough," which is just nonsense for someone who believes that the US stock market is efficient. As another example, some analysts claim that US stocks are currently overvalued while some kinds of foreign stocks have more attractive P/E ratios. They advocate overweighting their favorite kinds of foreign stocks.
In both of these examples, other analysts who look at the same numbers disagree and recommend other strategies based on what they think the numbers say. For example, some analysts say that after the recent crash in tech prices, some large growth tech companies are beginning to look attractive again and they are giving their favorite tech stocks "buy" recommendations.
In general, with this kind of "fundamental analysis," the markets appear to be perversely efficient in yet another way - for every analyst, advisor, expert, or pundit who holds one opinion, there seems to be another one who holds the opposite opinion. When we take the average of all these opinions, we invariably get, not unexpectedly, the same opinion as that currently represented by the total market. Indeed, when one thinks about it, this phenomenon is an obvious consequence of the EMH.
People who believe that markets are efficient call this "noise." The noise often causes erratic movements in prices, but it tends to cancel itself out eventually, so it can and should be ignored. Efficient market believers may tune in to "Wall Street Week" every Friday night or read these kinds of arguments in magazines or on the Internet for amusement, but they do not take them seriously.
Some academics like Robert Shiller and Jeremy Siegel also enjoy this pastime, although their arguments are much more dignified and at a higher level than what we see in the popular press and on the Internet, and their focus is on trying to estimate long-term future returns for the market as a whole rather than identifying mispriced securities, sectors, or styles. There is some academic evidence that long-term expected returns for major markets (10 years or more) may be predictable to some degree, although there is by no means complete agreement that this is true. Even these two highly respected experts, however, cannot seem to agree on what the numbers mean. Shiller argues that the long-term outlook for the US stock market is rather dismal, while Siegel is considerably more optimistic, and they continue to carry on their debate. Efficient market believers enjoy reading their books and listening to their arguments but have a difficult time figuring out which one is right and which one is wrong, if either one of them is making any sense at all. Some people tentatively agree with Shiller and several other experts (including John Bogle) who forecast lower than average long-term returns for US stocks, and they modestly decrease their US stock allocation accordingly. Others prefer to make no judgement about this issue and do not adjust their allocation. This issue is certainly open to debate even among efficient market believers.
In the case of bubbles and panics, efficient market believers point out that identifying bubbles and panics may be easy in hindsight, but market history shows that it is nearly impossible to identify when or where they will occur with enough foresight or precision to take advantage of them or protect against them. Market history is littered with the corpses of failed market timing strategies that tried to do this. They argue that the best strategy is to hold a total-market portfolio (domestic or international) and trust to its complete diversification to provide as much protection as possible against the inevitable market ups and downs in general, including unpredictable bubbles and panics in individual sectors, styles, and countries.
In the case of equal-weighting countries, people who believe that markets are efficient counter by noticing once again that if the cap-weighted international market percentage for some country is, for example, 15%, and if some behavioral analyst says that the proper weighting is some other percentage, say 10% or 20%, then the question to ask the analyst is "The international market thinks the proper weighting for this country is 15%, but you disagree - what do you know about this country's prospects that the international market does not already know?" To an efficient market believer, equal-weighting countries or regions in international investing makes no more sense than equal-weighting sectors, styles, or even individual stocks in US investing. None of these equal-weighting strategies have anything at all to recommend themselves if markets are efficient.
Once again, the efficient market believer says "The market seems to feel otherwise - if it did not, small growth prices would be much lower. What do you know that the market does not already know about small growth and its prospects for the future?"
Unlike the efficient reasons for differing from the market that we presented in the previous section, here in each case there are winners and losers instead of everyone being a winner. The winners are the smart people who follow the winning strategy. The losers are the people who are making mistakes and do not follow the strategy. In each case the winners exploit the losers to make an abnormal profit, or to increase expected return without increasing risk.
Note another difference between the efficient examples and the behavioral examples. Investors who use efficient reasons for differing from the market are making no judgment about the market or about other investors. They are simply adjusting a total-market portfolio one way or the other based on the personal sets of circumstances which make them different from the mythical "average investor." They are not making any bets that the market is wrong or making any forecasts about what they think might happen to the market in the future. On the other hand, investors who use behavioral reasons are saying "I know something the market does not already know," and are making a bet that they are right and the market is wrong. Charles Ellis calls this "the loser's game."
Some people mistakenly think that there is a difference between recommending that all or most investors should overweight an individual security or a market sector and recommending that all or most investors should overweight a market style. They argue that styles are "risk factor asset classes" in the "Fama-French three-factor model," and hence their strategy is consistent with a belief in efficient markets. This argument is incorrect. The Efficient Market Hypothesis is much simpler and more direct than these people make it out to be. In particular, it is the same in the new three-factor model of Fama and French as it is in the old single-factor model of Markowitz and Sharpe. The arguments are the same. Recommending that all or most investors should overweight a style is just as much a bet that the market is wrong as recommending that all or most investors should overweight an individual security or a sector. In both cases the proponents of these strategies are saying that they know something that the market does not already know. Both strategies are behavioral, not efficient.
There is a simple test that can be used to distinguish between efficient and behavioral reasons or strategies for differing from the total market. If a strategy recommends that all or most investors should deviate in the same direction, then it is behavioral. Proponents of such a strategy are saying that they know something the market does not know - the market is overweighted too far in the other direction.
Examples of currently popular behavioral strategies include overweighting the health care sector, avoiding foreign stocks, overweighting the small and value styles, and underweighting Japan in international investing.
An investor considering some kind of behavioral strategy should ask the following questions:
There is a considerable body of evidence that modern financial markets are largely efficient, at least efficient enough so that it is difficult to exploit any inefficiencies which may exist. In any case, both market history and simple arithmetic clearly demonstrate the wisdom of total-market investing, whether or not markets are efficient.
Despite the interesting arguments of the new Behavioral Finance academics, there is not yet enough evidence to abandon a belief in efficient markets. Buying into the behavioral story would mean we would have to abandon just about everything we think we know about how markets work and start all over from scratch. Revolutionary new theories that overturn old ones have happened in the history of science, of course, but they are rare, and the behaviorists are not yet anywhere near accomplishing such a revolution.
Our conclusion is that total-market investing is the right starting point for most people. If an investor deviates from the total market, he or she should do it only for efficient reasons. As tempting as the behavioral reasons can be, they should be avoided.
The two most important efficient reasons for differing from the market average are adjusting the stock/bond ratio to match the investor's risk preferences (the investor's need and ability to take risk) and adjusting the domestic/foreign ratio. Unfortunately, there are no hard-and-fast rules for doing this. Developing this "asset allocation policy" is the first and most important decision an investor must make.
Nearly everyone should have both stocks and bonds. As a general rule of thumb, 40% or less stocks is considered very conservative, a 50/50 stock/bond allocation is conservative, 60/40 is moderate or average, 70/30 is aggressive, and 80/20 or above is very aggressive.
Many advisors recommend that at least 20% to 30% of a US investor's stocks should be foreign. Some say even more is appropriate, perhaps up to as much as 40%. The historical evidence is consistent with these recommendations. Most studies that have mathematically analyzed the diversification benefits of global investing say that percentages in this range are optimal for US investors.
To summarize, the first and most important decision an investor must make is the broad asset allocation policy decision. This is the percentage allocation of the investor's total portfolio among the three major asset classes: US stocks, foreign stocks, and bonds.
The next step is to examine the other efficient reasons for differing from the total-market portfolio. For most people these reasons are less important than the big stock/bond and domestic/foreign decisions. For people who are not significantly different from average in the other respects, there is no practical need to make further adjustments. There is a great deal to be said for simplicity in investing, and ignoring details is a perfectly reasonable thing to do. For some people who are significantly different from the average, however, the other adjustments can be important. One way to implement an efficient deviation from the total market for one of these other reasons is to add a small amount of some other mutual fund, preferably an indexed fund. Vanguard offers a wide variety of funds which can be used for this purpose. For example, investors with significantly above average exposure to inflation risk might like to allocate 5% or 10% of their portfolios to inflation-protected securities using Vanguard's Inflation-Protected Securities Fund, or simply buy TIPS and/or I Bonds directly from the US government.
Once the asset allocation policy decision has been made, we strongly recommend using the three Vanguard total-market index funds listed in the introduction to implement it. If these Vanguard funds are not available in the retirement plan offered by the investor's employer, the best possible approximation should be constructed. This is often difficult and usually compromises are necessary. Our most important advice (after the asset allocation decision) is to use index funds whenever possible to minimize expenses and manager risk and to get the largest possible amount of diversification. Total-market index funds are of course the most desirable if they are available.
As time goes on and different market segments go up and down, an investor's portfolio eventually strays from the target asset allocation policy. It is important to periodically rebalance a portfolio to correct this drift. Doing this once per year should be more than enough, perhaps less often in a taxable account if rebalancing would trigger capital gains taxes. In a taxable account using already scheduled periodic additions or withdrawals to rebalance is a good strategy.
After the policy has been implemented, it is important to stay the course and avoid market timing. "Market timing" is the misguided practice of adjusting the asset allocation policy based on current market conditions or forecasts of future market movements. Market timers invariably become overconfident during bull markets and underconfident during bear markets, which leads to buying high and selling low and seriously underperforming the market.
The asset allocation policy should be changed only if there is a change in the investor's efficient reasons for differing from the market. For example, for some investors (but by no means all), it is reasonable to gradually decrease the stock allocation and increase the bond allocation as the investor gets older and approaches retirement.
These are the four basic principles for using a total-market approach to investing: Design an asset allocation policy, implement it using total-market index funds, rebalance periodically, and stay the course.
Maintaining a total-market portfolio with only efficient changes requires fortitude, patience, and humility. Investors must have fortitude when the total market is suffering one of its inevitable bad periods while some sectors or styles or competing behavioral strategies are doing well. They also need fortitude to ignore the overwhelming amount of seductive noise generated by the analysts and other pundits. They need patience while they stay the course and wait for years and decades for their strategy to pay off. They need humility to admit that they are not smarter than the market or other investors and to resist the temptation to try to exploit them.
Wise investors cultivate these three virtues of fortitude, patience, and humility. For total-market investors, the three disciplines of history, arithmetic, and reason all say that they will succeed in the end. As John Bogle says, "In my view, owning the market and holding it forever is the ultimate strategy for winners."
The following references are to books and to resources on the world-wide web. For books the links are to Amazon.com. For web resources the links are directly to the web pages.
Up to my Finance page.