Hi Chuck, those are very good questions. Your final question is a good starting point: “don’t you think P&F is better than nothing for the individual investor. Even just for entry and stop points?”

If we are to answer that question with truth and logic, then we necessarily have to provide evidence supporting the belief. As I said in the post, we can believe something just because we want to believe. For example, you may hear someone quote from a book “The abc pattern is a good one”. You may see a problem with that. How do you know its good? Does the author provide quantitative evidence such as actual performance? That is, if you are applying buy or sell signal to a securities or universe of securities without knowing how much profit it earns when it’s right and how much it looses when it’s wrong, then you have no basis behind it.

For a good system, we generally need a few things. First, its methods should be based on some sound logic. There should be some real reason behind it. For example, the signals may be exploiting some fundamental market condition that causes inefficiency. Second, the signal should be tested historically to determine what its profit potential would have been and the risk it look (as measured by drawdowns) to earn that profit. Third, you realize that you are intentionally data mining and doing so with perfect hindsight. If you are testing different indicators and parameters to determines those that worked in the past, you are snooping through the data with hindsight bias. Fourth, a complete system is required, not just a buy and sell signal. Even if you have designed, developed, and tested a complete system and factored in all the ways you may be fooling yourself, then you have to do the fifth thing. You have to execute the system. If you have all of those things, then you have all kinds of testing results and an actual track record. Those are the things that evidence whether or not it’s a repeatable systematic process and if its risk and reward profile are within your objectives.

What I have described is the scientific method. It’s the way to discover statistical significance. It’s a similar process as found in other industries like medicine. You have a theory something may work and it’s based on good logic. You test it historically on sample data. If you have included enough of the same kind of data you’ll experience in the future, then your research is statistically significant. If the future does indeed look similar to what you tested, then your results should too- if you are able to execute it. You’ll know if their testing and development process is robust and they execute it if their actual performance in real time looks similar to their back-test. If it doesn’t, they probably over-optimized in their development process. Or, they are unable to execute it. Either way, sometime is wrong.

Now, think about your question: “Don't you think P&F is better than nothing for the individual investor. Even just for entry and stop points?”. I am very familiar with Point & Figure charting and intuitively I want to say “sure”, but that’s not fully true. Of all the people I know who use Point & Figure signals, they do so subjectively. They don’t arrive at the same conclusions. They change box sizes to make the chart look the way they want. One uses a large box to make it a positive trend, the other uses a small box to make it a sell. It seems they use it to confirm what they already believe. If it were totally objective and a repeatable process, then we could answer the question with evidence. We could then say: yes or no based on the expectation. I can’t say that I have such information I will share.

If we had the quantitative data for probability and expectation, then we could answer the question. Such data could be either a hypothetical back-test on the various Point & Figure buy and sell signals. However, simulations and back-testing has many issues. We have been running several systems in real time for nearly a decade now and I can tell you the forward-walk can be quite different than a backward look. So, we could really only state the robustness if we have some actual performance from portfolio managers who apply it systematically and repeatedly. You see, we need to know the mathematical expectation. That is, the percentage of time it is profitable and the size of the profits vs. the percentage of time it is wrong and the size of the losses. The percentage of time it’s profit is only the probability. High probability doesn’t mean anything. It could be very accurate, but lose so much when it’s not that it wipes out its gains. The magnitude of the profits and losses is what matters. The probability is only for your ego. It seems many people attach their self worth to being right or wrong.

Your other question: “Why can't point and figure charting be tested?” Some chart patterns are programmable, others aren’t. For a signal to be objective, we should be able to quantify it into a mathematical equation. People have to agree on what the pattern is exactly. Otherwise, it is necessarily subjective is we are unable to do make it a definitive equation that is testable. “It just works” doesn’t do it. The only way to separate luck from skill (a real edge) is to quantify it. If we can do that, we can program it and test it to discover its potential profit and loss. We have tested some of the signals and I know others who have. I am unaware of anyone that is providing the mathematical expectation of the many Point & Figure signals. Money managers like myself have no incentive to do so. Those who do have the incentive are those who promote the method. For example, Investors Intelligence, formerly known as “Chartcraft” says they “pioneered the use of point & figure charting in the 1950s”. However, I am not aware that they provide any data to quantify the signals. I’m not saying you can’t create good results with the method, but I’m saying the quantification is necessary to determine if it’s an edge.

Below is the definition of a pattern in the Investors Intelligence glossary;

"Buying Climax: Investors Intelligence defines a buying climax as when within one week, a new 52 week high is made followed by the stock falling and closing below the previous week’s close. This can be an important reversal signal for the stock. Also the total number of buying climaxes across the whole stock market can provide clues as to the timing of market tops."

Source: http://www.investorsintelligence.com/x/glossary.html

We can maybe agree on an equation that would allow us to test the signal to learn its historic profit and risk. However, without doing that, we can’t prove the statement “This can be an important reversal signal for the stock”. How do you know it is? Or, the statement “the total number of buying climaxes across the whole stock market can provide clues as to the timing of market tops”. I am personally well ware of the Buying Climax. I can tell you that from my experience, these statements seem to match my own empirical evidence – what I believe I have observed in my experience. However, I may not recall every Buying Climax I’ve witnessed. I may not have observed the outcome afterwards. Its my recollection, but it may not be accurate or include all the needed data. I therefore wouldn't' rely on it. I've proven myself wrong many times after applying our quantitative methods to test for the true answer.

Below is another link to Point & Figure chart patterns and the definition of one.

"The Triple Top

The triple top is similar to the double top but the area of resistance above the up columns is stronger and so takes two attempts to breakout. These signals work best when the main medium term trend is bullish and combined with positive relative strength.

Stops are placed below the previous down column"

Source:
http://www.investorsintelligence.com/x/classic_point_and_figure_formations.html\

We may be able to quantify that into code if we can agree precisely on the definition in mathematical terms. However, if we have 10 people attempt it, I bet we’ll have some disagreement. What box size would we use? How many boxes will reverse it? Which chart patters would we use, all of them? Or, just a few?

If you know of any available evidence that provides the mathematical expectation of the signals or even better: actual performance results using the signals we’d love to see it.

By the way, the expectation of the signals themselves still wouldn’t determine if “P&F is better than nothing”. You see, you could be using every Point & Figure chart pattern and publish all of your buy and sells and we would end up with the same results. Even if you told us exactly when you buy and when you sell and made sure we get the same buy and sell price: You didn’t tell us how much…

You may see what I mean by a “complete system”. It’s more than just the entry and exit. It’s even more than how much. dot-dot-dot.