海角大神

As financial services become increasingly automated,聽retirement-spending apps聽have emerged that enable you聽to enter your聽income needs and portfolio information, ostensibly to get a reasonable prediction of whether or how long your聽nest eggs may last in retirement.

Many of these apps are on the market 鈥� some developed by firms such as Betterment, Vanguard, T. Rowe Price and Schwab, and others sold as subscription services to financial advisors for use with their clients. The problem is that users are led to believe they should make important life decisions with the aid of these apps, even though the underlying probabilities are based on inherently unpredictable outcomes.

In truth, applying probability software to retirement-planning analysis is folly. Even the most sophisticated retirement-planning software used by financial professionals is a far cry from a crystal ball.

The problem with probabilities

The failings of probability-based retirement software, specifically those apps that apply so-called聽Monte Carlo聽simulation techniques, are reasonably well-known in professional circles. One of the first academic papers to raise the issue was a聽2006 article聽written by renowned retirement researcher and York University of Toronto professor Moshe Milevsky, who noted in his introduction:

鈥淥f course, as most investment advisors have known for years, a retirement number 鈥斅爄f it actually exists 鈥斅爄s vague and imprecise, as聽it depends on many economic unknowns, especially future equity market returns. After all, this number must be invested somewhere in order to produce income,聽and the portfolio return process is inherently random.鈥�

In addition to the unpredictability of future returns, Milevsky goes on to document how 鈥減robabilities鈥� produced by popular retirement software applications vary from one app to the next, depending upon the applications鈥� internal assumptions and design parameters.

Another聽academic study, published in February, concluded that 鈥渢he advice provided from a majority of these tools is extremely misleading to households.鈥�

These publications have caused some to question whether retirement-planning software offers any value to consumers at all. So what alternatives are there?

鈥楤ack-testing鈥� software

Financial advisors who use Monte Carlo simulation software often express their clients鈥� results in terms of the likelihood of a positive outcome. Instead of attempting to predict 鈥減robabilities of success,鈥� perhaps a better way to approach retirement planning is from a glass-half-empty perspective.

What you聽really need to know is not how you聽may fare if things go well, but what will happen to you聽if a 10% possibility聽of rain turns into a 100% probability of a thunderstorm.聽You聽desperately need and want to know, 鈥淚f things go badly in the investment markets, will I still be OK?鈥�

Traditionally, historical 鈥渂ack-testing鈥� software has been used for this purpose. By entering your聽retirement profile into a back-testing app, you can test how your聽portfolio may have fared if you聽had retired prior to previous economic downturns. While such information is useful and interesting to consumers, back-testing also has significant limitations.

Specifically, past returns are unlikely to be repeated in the exact same sequence again, and it is entirely possible聽that future returns will聽be worse than historical experience.

Further, suppose you聽wanted to test how your聽portfolio might hold up over a 30-year retirement horizon if you聽had retired at the end of 1999 (just before the 2000-鈥�02 and 2007-鈥�09 bear markets). Because聽we are only in 2016, it isn鈥檛 possible to play out the analysis over the full 30-year horizon. You can鈥檛 back-test the future.

Bootstrapping technique

One solution to the limitations of back-testing is to apply a simulation technique called bootstrapping. While the simulation engine under the hood of many retirement apps requires the program designer to make assumptions about expected mean rates of return and volatility for various asset classes, bootstrapping requires no such assumptions. Simulations are produced instead by randomly sampling historical returns.

If enough simulations are generated 鈥� typically a minimum of 5,000 鈥� the median result may be expected to be roughly in line with historical averages. By considering the range of results below the median, bootstrapping programs may illustrate scenarios showing聽below-average investment returns, with the value-at-risk statistics (the bottom 1%, 5% and 10% results) representing scenarios that may be as bad as or worse than the historical record.

For example, the following table shows聽the bootstrapping simulation results for a 65-year-old investor with a 25-year retirement horizon, a $1 million initial portfolio value and a 70-to-30 stock-bond retirement allocation. In this example, the investor requires a $50,000 (5%) first-year withdrawal rate and a 3% annual cost-of-living increase thereafter. He estimates his annual investment expense at 1% and has stated that he expects to withdraw proportionately from each asset class each year and rebalance to maintain his 70-to-30 allocation.

In the chart below, the percentages in the left column are simulation percentiles, and the columns on the right indicate how much in savings would remain after 5, 10, 15, 20 and 25 years for each simulation percentile.

By focusing on the bottom half of the results and displaying the simulation range in five-year increments over the time period, you can gain a much more tangible sense of whether and how long your savings may last. What鈥檚 more, by presenting the data in this format, it is easy to then test how changing factors that are within your聽control (spending amount, withdrawal strategy, asset allocation, investment expenses) may affect聽the outcomes.

To be clear, there is absolutely nothing predictive in these simulation results, and the simulation percentiles should not be viewed as probabilities. Instead, the worst聽results merely represent potential scenarios that may be used to give you聽a clearer picture of what may happen if things go badly.

While bootstrapping offers a neat way to illustrate these聽data, it is also not without its flaws and limitations. In this example, bootstrapping was applied only to historical stock market data from 1970 to 2014. The bond portion of the portfolio was assumed to be a constant 2% per year, which reasonably reflects the return an investor might earn today on a five-year CD or 10-year Treasury. The fact that bootstrapping simulations were not applied to historical bond data reflects a limitation seen in聽most retirement apps in that the yields on bonds today are near the bottom of the historic extreme. As a result, any Monte Carlo application that is generating numbers聽based on mean historical bond returns or any bootstrapping simulation that is randomly sampling historical bond index returns may produce overly optimistic results.

With any retirement-planning app, the devil is in the details. Consumers and advisors alike would do well to take the time to understand the assumptions and limitations inherent in any retirement-planning application.

John H. Robinson聽is the owner of聽Financial Planning Hawaii聽and a co-founder of聽Nest Egg Guru, a retirement-planning software application for financial professionals.聽Learn more about J.R.聽on NerdWallet鈥檚聽Ask An Advisor.

This article first appeared in NerdWallet.