A Critique of Monte Carlo Retirement Calculators

Monte Carlo retirement calculators have become a cause celebre for the popular financial press [5][8]. Deterministic retirement calculators are dismissed out of hand without serious consideration.

In fact if retirement calculators were regulated by the Federal Food and Drug Administration then Monte Carlo Simulators would be banned because they lack efficacy and have potentially dangerous side effects.

In his paper Finance and Monte Carlo Simulation Nawrocki [9] denounces the use of Monte Carlo methods is all matters of finance. His paper is a must read before you bet your retirement portfolio.

Retirement Calculators

A retirement calculator is computer software that uses input parameters reflecting the user's age, financial situation and retirement plans to chart how the retirement savings will be spent over time. A comprehensive retirement calculator will have the input parameters shown in Appendix A:

Ockham's Razor needs to kept at hand during parameter list definition. Parameter and features that apply to only a tiny minority of users or that have very little impact on the models results need to be culled.

Efficacy

The retirement calculator should be both a strategic planner and an educational tool.

• As a strategic retirement planner it should compute results that accurately reflect the real world situation at hand, meaning the computed age when the money runs out or the computed amount of money available for spending should be realistic. From this the strategic outline of retirement can be derived.
• As an instructional tool the retirement calculator illuminates issues that effect every retirement. This may be on the input form and help documents where the usefulness of a reverse mortgage is brought to the users attention. Or it may be in the computed results where non conventional withdrawals are made from more than one account in a year or an IRA to Roth IRA partial rollover is indicated.

Dangerous Side Effects

The appearance of the parameters shown in Appendix A on the retirement calculator's input parameter form lead to the conclusion that the retirement calculator is using them in its computer modeling. Parameters listed here but absent from a particular retirement calculator indicate that the retirement calculator is missing required features and will not produce results that are truly representative of the user's situation. For example, most users of retirement calculators do not live in rental housing and their home will have a substantial equity buildup by the time they decide to sell it. When they decide to sell their home will have a huge impact on their retirement picture.

An incomplete model representation leads unpredictably to one of two unwanted side effects:

• Overly optimistic results will fail to accurately predict when the money will run out leaving the retirees to move in with their heirs.
• Overly pessimistic results will understate the amount of money in the estate leaving the heirs with a windfall while the retirees had lived unnecessarily frugally.

Monte Carlo Simulation

Wikipedia defines Monte Carlo simulation as follows[1]:

Monte Carlo methods are a class of computational algorithms that rely on repeated random sampling to compute their results. Monte Carlo methods are often used when simulating physical and mathematical systems. There is no single Monte Carlo method; instead, the term describes a large and widely-used class of approaches. However, these approaches tend to follow a particular pattern:

1. Define a domain of possible inputs.
2. Generate inputs randomly from the domain, and perform a deterministic computation on them.
3. Aggregate the results of the individual computations into the final result.

The name Monte Carlo was popularized by physics researchers Stanislaw Ulam, Enrico Fermi, John von Neumann, and Nicholas Metropolis, among others; the name is a reference to a famous casino in Monaco Carlo where Ulam's uncle would borrow money to gamble.[2] The use of randomness and the repetitive nature of the process are analogous to the activities conducted at a casino.[1]

Monte Carlo Retirement Calculator

A Monte Carlo Retirement Calculator assumes that the two exogenous parameters, rate of inflation and return on asset investment, are beyond the retiree's influence. Therefore these two parameters ((Domain of possible inputs) are omitted from the user settable list. The Monte Carlo Simulator uses a random number generator or some other randomizing process to generate values for these parameters for each trial of the retirement calculator.

There are at least four different ways of computing the random numbers.

1. Uniform Distribution: Any number within a specified range has an equal chance of being used. Except for games of chance the uniform distribution generator has no know use in financial modeling.
2. Normal Distribution: The domain of selection is bell shaped about some selected average value and selected width. Some calculators allow the user to select the average and width. Use of the normal distribution requires that the distribution of numbers being generated do in fact look like a normal distribution and that when two or more values are being generated that the two generated values do not correlate with each other. Nawrocki [9] writes that investment returns are not normally distributed. Further, there are many situation where there will be trends in the data being simulated that are not captured by the normal distribution. For example in a bear market investment returns after a down year typically may have another down year. In a bull market investment returns may show several up years in a row. The chosen average and width has a big impact on the calculator's results.
3. Random Historical Data: The values are chosen randomly from a table of historical data going back to some date or other. A particular year is selected and the historical values for all generated variables are chosen for that year. This deals with the problem of correlated values, e.g. investment returns and inflation. 1920s to the present is a popular time period. A more realistic choice would be 1946 to the present. The.Full Employment Act of 1946 [6] put the Federal government, and particularly the Federal Reserve, directly into the business of stabilizing the U.S. economy. The economic environment has been an awful lot different since 1946 so this domain of asset returns and inflation rates is more representative of what will occur in the future than those from the Great Depression.
4. Random Interval: A starting and ending year are randomly selected and then the values supplied to the calculator are the same values in the same order as the selected historical period. Random intervals overcomes the trend problem and correlation problem noted with normal distributions.

Evensky [10] says it is vital to select a random number generator that will replicate the distribution of values in the future Evensky. This is of course impossible. The best that can be done is to replicate the past which carries no guarantee for the future. Evensky and Nawrocki make the case that the distribution of the random numbers are the Achilles' Heal of the Monte Carlo Method. Figure 1 illustrates the problem.

A conventional retirement calculator is used to solve the model with a combination of user inputs and random values (the deterministic computation). The user specifies an amount that she hopes will be available for spending during retirement and the model is solved anywhere from 2,000 to 10,000 times. Each trial is done with new random numbers. Each solution is recorded at the conclusion of each trial. After all trials are completed the final result is displayed.

The final number is a probability that the retiree will not run out of money before the end of the plan. Probabilities between 90% and 99% are deemed safe.

A far more unpredictable exogenous variable is the United States Congress meddling with the US Personal Income Tax Code. This is an infrequent but completely unpredictable event. The resultant tax tables are completely unknown until the legislation is passed. This random event is not modeled by any retirement calculator, Monte Carlo simulator or otherwise.

Computational Survey

To get some idea of how Monte Carlo Retirement Calculators compare to each other a test scenario was constructed and then run against several Internet based calculators and the results recorded. Three kinds of Monte Carlo calculators were excluded:

1. Calculators that were cloaked behind some sort of membership requirement.
2. Calculators that were available only for purchase. This criteria excluded ESPlanner [4] which is known to have a comprehensive set of parameters and uses dynamic programming to optimize its results.
3. Calculators that had to be download to the client computer. This is an open door to all kinds of viruses and Trojan horses. ESPlanner has this problem as well.

The parameters for the test are:

1. Single retiree, age 55.
2. $1,000,000 Tax-deferred savings.
3. $50,000 After-tax savings
4. $7,000 annual contribution to the Tax-deferred Account.
5. $3,000 annual savings to the After-tax Account.
6. Retire at age 65.
7. Social Security benefits begin at age 65
8. Life expectancy age 85.

The results of this informal and completely uncontrolled survey are shown in Appendix B.

Conclusion

The implementation of a retirement calculator as a Monte Carlo Simulator is an intuitively appealing idea, particularly to those with an aversion to the linear estimates of conventional retirement models[5]. Furthermore, it is based on solid mathematical concepts. Even more appealing is the certainty that seems to reside in the probability of success value. This probability can be forced into a range of safety by reducing the annual spending requirement input variable and running the model again.

Monte Carlo Retirement Calculators are only as good as their components:

1. Domain of random numbers. Nawrocki states that Monte Carlo simulation techniques are not valid for financial modeling because realistic random values cannot be generated.
2. Deterministic computation: Most of the underlying calculators feature no where near the parameters and features listed in Appendix A. Therefore the deterministic results are unlikely to realistically represent the retiree's true retirement situation. Most calculators will error by being overly pessimistic, under estimating the amount of money available for spending.[7]
3. Aggregated results: Monte Carlo retirement calculators do not produce meaningful reports. The probability of a success result is neither particularly useful nor instructional. The calculators offer no insight into how their results are arrived at.

Monte Carlo retirement calculators are based on the use of random numbers and as such if you run one of them three times in a row you will get three different success rate projections. In many cases the differences will be significant. The question is: which do you base your financial decisions on? Do you pick the lowest, the highest or an average?

The most serious problem is that Monte Carlo retirement calculators answer the wrong question. The retirement question is not:

Monte Carlo retirement calculators are irrelevant because they lack instructional reporting and are, from a modeling perspective, inaccurate. Since their developers do not reveal their assumptions using their results to make retirement financial decisions is purely faith based.

Appendix A: Important Retirement Calculator Input Parameters

• Retiree and Spouse:
  These accounts that have to be manage separately because age differences between the retiree and spouse combined with the tax law require that the accounts be treated individually.
  • Current Age of Retiree and Spouse
  • Tax-deferred Account Balances (401K IRA, 403B, Profit Sharing, etc.)
  • Roth IRA Balances
  • Tax-deferred Account Annual Contributions
  • Roth IRA Account Annual Contributions.
  • Annual Social Security Benefits
  • Annual Pension
  • Post Retirement Earned Income.
  • Age When Earned Income is to stop.
• As a Couple:
  These are parameters apply to the family and don't need individual treatment.
  • After-tax Account Balance - conventional brokerage account.
  • After-tax Account Annual Savings.
  • Value of Illiquid Asset, e.g. the couples home.
  • Age for Sale of the Illiquid Asset
  • Reverse Mortgage Parameters
  • State income tax code specifications..
• Optional Parameters:
  Depending upon the calculator these parameters will be specified by the user or computed by the retirement calculator.
  • Annual Spending in Retirement.
  • Age That Plan is to End -- Life expectancy
• Exogenous Parameters
  • Estimated Rate of Inflation
  • Estimated Investment Rate of Return.
• Built in features
  • Federal Progressive Income Taxes
  • Required Minimum Distribution for Tax-deferred Account.

Appendix B: Survey

References

1. Monte Carlo Method; Wikipedia, the free encyclopedia
2. Hubbard, Douglas; How to Measure Anything: Finding the Value of Intangibles in Business; pg. 46, John Wiley & Sons, 2007
3. Metropolis, N.; The beginning of the Monte Carlo method , Los Alamos Science Special Issue, 1987
4. Kotlikoff, Laurence J. and Gokhale, Jagadeesh;ESPlanner
5. Retirement Calculator Review; by July 12, 2007; Sharpe Investing
6. Full Employment Act of 1946 Wikipedia, the free encyclopedia
7. Welch, Jr., James S.; Optimal Retirement Withdrawal Strategy; June 2008.
8. Carnahan, Ira The Best of the Web Retirement Planning Tools ; Forbes, June 6, 2005
9. Nawrocki, David Finance and Monte Carlo Simulation; Journal of Financial Planning/November 2001; Dr.Nawrocki is professor of finance, Villanova University
10. Evensky,H.; Heading for Disaster; Financial Advisor, April 2001, pp. 64-69

Last Update: July 31, 2008		© 1998-2008, James S. Welch, Jr