Just before Christmas past, I was hoping to interview David Samborsky from Compuvision (the maker of TradeSim). But, to cut a long story short, things didn’t go to plan and we never did get to record that interview.

Instead, I emailed Dave a few questions including those submitted by my clients and he’s kindly answered them below. Check them out now and if you’ve got any further questions be sure to post them on the blog – I can then follow up with Dave and get you the answers you need.

Why is backtesting your trading system so important?

It’s important for a number of reasons:

• Backtesting should be used as a good estimator of future performance but should not be used as a guarantee of future performance.
• Backtesting can be used to stress test your system over periods of adverse market conditions in order to see how your system holds up.
• Backtesting can be used to quickly find the conditions under which your system breaks down.
• Backtesting can be used to quickly test a system that would otherwise take years to paper trade.
• Use back testing to quickly test what-if scenarios.

What are your thoughts on mechanical trading vs intuitive?

Mechanical trading takes away human emotional as well as the temptation to change the system you started with. Because humans have a tendency to always want to be right this tendency works counter intuitively to mechanical trading which is better explained by the law of averages rather than being able to predict a specific outcome with any great accuracy.

One of the most important aspects of back testing results is a smooth equity curve with a minimum drawdown especially over periods of adverse market conditions. Some people try to minimize their string of losses because this can have a dramatic impact on the trader’s emotion, psyche and stress levels.

However with a portfolio trading system a one off back test can be misleading because there are many ways to trade the system using available capital and trading opportunities. Compared to a single security system, back testing a portfolio trading system requires a different approach that makes use of advanced statistical analysis techniques that takes advantage of the variance in the system.

For a portfolio analysis a thorough statistical analysis is imperative. TradeSim has a Monte Carlo analysis feature which tests a portfolio system in much the same way you would if you got 100 or a 1000 traders to trade the same system using the same stocks and then analysing all of the results using statistical techniques.

What are the limitations of backtesting ?

The main limitation of any back testing is that no back testing can guarantee future performance so in essence it should be used as a tool to estimate future performance and test the robustness of a system rather than look for a guaranteed future performance. My recommendation is to thoroughly test your systems over periods of adverse market conditions such as corrections and crashes. Seeing how your system would hold up in those conditions is a good indication of how it will fare in the future.

The other limitation of backtesting is actually not being able to replicate exactly what would have happened if you did actually trade the system at the time. Such things as order entry and exit slippage etc cannot be duplicated exactly with back testing. However when statistically analysing a system the effect of these discrepancies can more or less be minimized.

What is the biggest lesson you’ve learn over the years with your involvement in TradeSim?

The biggest lesson with back testing is that it is a mistake to assume that a one off back test that would be applied to a single security trading system is adequate to characterise and quantify a portfolio trading system. It is not !!

Now for some questions from my clients…
How do you manage your trading system for slippages? (backtesting vs actual trading) – Rajarshi

There are two ways of emulating slippage.

The first method includes the ability to model slippage through order control within TradeSim itself. You can choose to control the amount of slippage at trade entry and trade exit by selecting different entry and exit order type mechanisms. My favourite technique is to use worst case slippage which always enters and exits the trade at the worst possible price thus applying more stress testing to the system.

The second  method is through the use of provisional trades where you treat trades as prospective trades and an algorithm of your choosing determines whether or not a trade is filled or not.

Again, both techniques use some sort of algorithm to approximate the real life trading scenario and that should be born in mind with any back test.

David, with all the backtesting Mr. Samborski is involved with, he must have detected the elliot wave/fib levels over time even if he were not looking for these cycles or patterns. Did he? I’ve been looking at Tony Plummer, Robert Miner, and Bill Williams lately (I’ll pick up Gann somwhere down the line) – Sammy

I have not done any testing of Elliot Wave or Gann but I do believe there are some plugins to MetaStock which emulate Elliot Wave. The problem is due to the somewhat subjective nature of counting the waves where I don’t think you would find two Elliot Wave packages produce the same results. Having said that, if your Elliot Wave plugin provides you with entry and exit triggers as well as the prices then it is possible to back test it using TradeSim.

As Darly Guppy pointed out, markets change over time and systems need to change with them in order to stay profitable. How do you recommend keeping a system current? If you wait until you have a number of loosing trades to let you know things have changed, you’re going to give back a lot of profit. “How do you keep a system current without giving back a lot of your profit when the markets change?” – Joe Nemecek

Some people like to re-test their systems on a regular basis and to make minor tweaks along the way. Other people prefer to stress test their system and see how it performs over long periods of time which also includes periods of adverse market conditions. At the end of the day it is really up to your personal preferences as to what suits you as an individual trader. Because back testing is no guarantee of future performance it should be used as a tool rather than a means to an end.

I have the Mean Reversal system… giving me 20 odd trades aday… with limit orders… but not nececesary to to fillup all. How TradeSim is going to handle such sistuation… I mean can I place the limit order in some priority. If the day I place an order, the market turns around , like last week, Dubai got the rescue package, how Tradesim can handle such turnaround situation at 3:15 pm.

Yes in later versions of TradeSim we included provisional trades. Using Provisional Trades along with Trade Ranking allows complete signal ranking although ranking trades completely restricts proper statistical analysis by removing the variance of the system. This defeats the whole purpose of trading a diverse portfolio.

Signal Ranking is discussed further in document AN-6 “Provisional Trades and Signal Ranking using TradeSim” which is freely available from the articles section of our website or by clicking on the following link.

As a new trader to CFD’s can you tell me how to handle the stress as soon as I put a trade on I get stressed. – Elaine

Firstly you need to ascertain what risk profile you are willing to allow yourself to be exposed to before you start trading.  You should also limit the risk of each trade by employing a suitable stop loss strategy.

Also bear in mind that because of leverage, the profit and downside risk with a CFD trade applies to all of the stock and not to the amount you contribute to. For example if you enter a CFD trade for $1000 and the margin is 10% then you are essentially in control of $10,000 worth of stock and not the $1,000 contribution that you make. You should therefore adjust your risk according to the leveraged amount of stock and not the contribution that you make to the trade.

As well as settling on so much risk per trade you should be looking at the worst case heat or maximum risk.

Planning all of this before you trade means that you will be aware of the maximum losses before you actually place the trades. This is what is referred to as risk management and this should be an important part of any trader’s strategy.

Why do most technical analysts fail to comprehend the transient responses of various types of first order low pass digital filters?

Most technical analysts do not have an engineering background and therefore do not have any understanding of digital signal processing concepts such as z-transform and finite and infinite impulse response filters.

Why do most technical analysts seem to favour exponential moving averages (EMA) over simple moving averages (SMA)?

Probably because EMA is used more in TA text books than the SMA, and the EMA provides a better smoothing for a given filter period but has greater lag.

End of Q&A… If you’d like learn more about TradeSim, click the link below.