Aug

31

I just had some code written to setup and process a workflow. The code is all Python so I can maintain/expand it. The problem, of course, is that it's taken me a week of learning obscure (chmod, source, PATH) Unix commands just to test the darn thing. It is kind of cool, though.

I subbed out the code work on Guru.com. It's a beautiful bunch of code modules, lots of features, all documented and well-written. Even has file locks. Will work locally or on a server. My cost? $120 (and some Unix pain).

Aug

19

For those interested in an overview of the process of mastery, I'd recommend The Cambridge Handbook of Expertise and Expert Performance, edited by Ericsson, and also his earlier work, The Road To Excellence: The Acquisition of Expert Performance in the Arts and Sciences, Sports, and Games.

Also worth reading, in summary form, is the overview of Mihaly Csikszentmihalyi's Flow: The Psychology of Optimal Experience. I say summary form because the book contains a handful of ideas that could have been more concisely expounded in an article. A major problem with the book is the lack of showing how to acquire the flow state, so the book is merely descriptive of the state, not a manual on how to achieve it (though still worth musing).

Finally, "mastery" is surely a misnomer since it implies one has reached some final state. To quote Bohr, one has already made "all the mistakes there are to be made" in the field.

In keeping with Vic's continual reminders about humility, I often think of the famous director Akira Kurosawa, one of the giants, who, when accepting an honorary Oscar for lifetime achievement at the age of 80 said that he would not accept the award for lifetime achievement, but rather for future work, because he felt he was only just beginning to master his craft. 

Henry Carstens adds:

Five Principles for Growth is an interesting article on this subject, as is the book by Josh Waitzkin, The Art of Learning.

Jul

19

Beautiful Code got me thinking about "How did these guys get better? How did they get to the place where we wanted to listen to their answers in the first place? How does Jon Bentley know he wants to write 'the algorithm he never wrote'?"

I think a book on "How I got better" would be much, much more interesting.

Jul

15

For the past week I've been running Microsoft's Live Search and Google side-by-side and I found the comparative search results fascinating — Peter Norvig appears to have been a very good hire.

I've switched and am now comparing Yahoo Search and Google. At first glance the results are much more equal although the Yahoo results are taking about twice as long to be returned.

As the relationship between the value of the search results for Microsoft, Yahoo and Google change, comparing the relative value of the search results should make for some nice systematic trading opportunities.

Here's the pseudo code for a search engine trading system:

  1. Each week, download the top search terms from the web.
  2. Run each search term through each search engine, grab the results and compare identical items on the first page.
  3. Score each engine on its closeness to the Google results.
  4. As (if?) the gap with Google narrows, adjust the relative weighting of the two/three stocks, pairs trade, etc.

Philip McDonnell extends:

PoindexterGoogle has many advantages over its competitors. They are not standing still, but are expending more resources on their core search engine technology than the competition is. Google is getting better all the time. The competition is shooting at a rapidly moving target.

One of Big G's advantages is its large number of personal customized users. They can tailor your search query to your personal information if you have given them permission. The benefit is greatly enhanced search results. If you queried for 'movie schedules' you won't have to sift through the movie theater listings from Zimbabwe and Outer Mongolia to see what's playing. The top listings will all be in your area - no additional user input is required.

About a year ago they hired an intern who, within 90 days, was able to show the core engineering group a technique to improve search engine performance ten-fold. Initially there had been no interest in the proposal but when it was demonstrated by an actual prototype implementation, the new technology immediately became the top development priority in the company. It is impressive to see a big company that can turn on a dime when a good idea comes along.

Let me give another imaginative example where the company has an edge. Suppose Cramer comes on the air and recommends his latest Turkey Inc. (symbol: TURK) stock. Millions race to their favorite search engine to check it out. Because of their large sample and rapid search ability they can actually alter their search rankings based upon recent interest as correlated with user personal information. The investing parent will get the stock page, the cook will see a turkey recipe, and the school child will get a page full of information on the country Turkey. It is truly impressive technology that is advancing rapidly, perhaps faster than the competition can imagine.

It is often said that imitation is the sincerest form of flattery. It certainly applies in this case as there is a growing trend for competing search engines to simply scrape off Google's top results as their own. 

Alston Mabry adds:

Interesting essay on Norvig's site: Teach Yourself Programming in Ten Years

Researchers have shown it takes about ten years to develop expertise in any of a wide variety of areas, including chess playing, music composition, painting, piano playing, swimming, tennis, and research in neuropsychology and topology. There appear to be no real shortcuts: even Mozart, who was a musical prodigy at age 4, took 13 more years before he began to produce world-class music.

May

19

The point of having 'will to win' is not to intimidate the opponent but rather heighten one's own motivation, awareness, speed of thought (pattern synthesis) and be able to withstand the pain. I don't think someone can do this well when they're a cold fish. You've got to be 'on fire' to achieve these higher levels of brain function. This is very clear to the fraternity of chess Grandmasters, who tend to discover all sorts of errors in their homework when they're actually sitting at the board in combat.

What enables people to achieve such levels of concentration will vary from person to person; Korchnoi found it was good to hate his opponents. For Fischer I believe it was some kind of sadistic impulse ('I like to crush the other guy's ego').

Not everyone accepts the idea of day trading being a cold intellectual exercise, and quite rightly in my view. Maybe long-term traders and investors can afford to be more aloof. But for good day traders (and chess players) I believe 'aloofness' will be much more the exception than the rule. This is certainly the case amongst people from both fields that I know. The whole brain is needed for tough challenges like day trading, not some lobotomized part of it.

Adam Robinson writes:

I couldn't agree more with Nigel's point.

That heightened emotions can enhance one's cognitive awareness and "power" is well known. As organisms, we are Darwinian beneficiaries whose higher functioning is galvanized by the presence of danger, when adrenaline kicks off a cascade of cognitive and physiological responses.

Moreover, the pure absence of emotion in thinking, as in the complete detachment epitomized by Spock in the original Star Trek series, is demonstrated to be a mistaken ideal by Damasio in his book "Descartes Error" (highly recommended). But. . .

  1. Although a mammal's (trader's) neural and synaptic functioning are accelerated by adrenaline, the field of cognitive awareness is narrowed, leading to potential errors. (In the presence of a potential danger, a mammal focuses on the perceived threat, to the practical exclusion of everything else.)
  2. Emotions may better serve a trader in the discovery of a trading system than in its application. Is it not a goal to trade by a system?
  3. It is well-documented in the history of science that, while heightened emotions are necessary in the "incubation" stage of scientific discovery, that the actual discoveries come when one is relaxed, usually out of the blue (the famous BBB phenomenon is bed, bath, bus — the last being an alliterative proxy for walking or travel).
  4. In my former field, in which I dedicated several decades to pondering optimal brain functioning, I have yet to work out the extent to which intelligence tests measure intelligence versus measure "the will to win" (i.e., solve a problem or answer a question).

From Henry Carstens:

Hmmm….

Trading Firm 1 is filled with discretionary traders. They have back-tested their ideas and have a defined edge which they trade each day with discretion. Traders do additional research on nights and weekends.

Trading Firm 2 is filled with discretionary traders and mechanical systems derived from each trader. Each trader's ideas have been back-tested and their edges defined. Firm 2 uses each trader's mechanical edge as a baseline to monitor their performance, to forecast the performance of the firm, and to create realistic risk and capital allocation controls. When a trader goes on vacation, has a new idea to test, or retires, that trader's mechanical system is substituted for the trader so the firm suffers no downtime.

Trading Firm 3 is filled with computers and researchers. Each researcher builds out all the viable trades in a time frame, mechanizes the rules, and moves onto a new time frame in the same or a new market. Incoming researchers are handed existing systems to maintain and improve before getting their own timeframes to build out. Each system has built in monitoring code that alerts the risk department when it has moved outside normal parameters.

Then,

Which trading firm has the least overall risk?

Which trading firm is most likely to evolve with the markets?

Which trading firm is least subject to ever-changing cycles?

What is the growth curve likely to look like for each of these firms? 

Nigel Davies adds: 

In my mid 20s I learned some methods for systematically relaxing during a chess game because my 'nerves' were often getting the better of me and leading to errors during the later stages of a game. This became more or less automatic over time.

Whilst trading I find that a similar 'controlled intensity' works best, though usually at lower levels than in chess, depending on the state of the market. I think that in both fields the focus may well be on 'perceived threats', but I think that one can learn what to look for via research or experience.

It may be just me but I find any attempt to conduct research whilst trading to be an exercise in futility, not to mention very distracting. It is much better to do it beforehand and have an armory of trading patterns ready prepared.

Research during a chess game would of course be illegal, but I think the effect would be the same - no benefit and plenty of distraction. I even found the Blumenfeld rule (writing down the move before playing it) to be very distracting, though now I've been saved from the temptation because they've made that illegal too (relying on 'notes' apparently).

So this paragraph kind of tallies with my own experience. But my take on it is that adrenaline levels can be controlled by the combat veteran and that any narrowing of cognitive awareness isn't a problem as long as someone has the most important things 'hard wired' via training.

George Zachar writes:

Did Einstein have the will to win? Feynman? What about Mises and Hayek? Palindrome? Chair?

My barstool analysis is that humans have too many personality facets, too many parameters of success, and too many life tracks to allow for broad, empirical rules in this sphere.

Looking in the mirror, I see aspects of my life where I am a complete failure and aspects where I have succeeded beyond my wildest dreams, having marshaled the same intellect and will throughout.

From Stefan Jovanovich:  

Ian Deary's book Intelligence is the best thing I have read on a subject. Deary's position is that (1) yes, there are measurable qualities in human beings that relate to the brain's ability to calculate, (2) the word "intelligence" is the shorthand description people have decided to use to describe this trait (hence I.Q. tests), and (3) as a measure of human capacity I.Q. tests alone are relatively feeble because living is far more complex than calculation.

I continue to trust my Dad's opinion about this. He was shrewd enough to foresee that standardized testing (MCAT, SAT, etc.) and test cramming (Princeton Review, etc.) would be growth businesses well before any other American or European publisher did. He was also someone who understood the limitations of what he did to make money. He thought there were two problems with intelligence testing. The first was that everyone who fell below the 95th percentile - and their parents - ended up hating the test. The second problem was that neither schools nor teachers nor publishers were honest about telling the students and teachers the truth about how limited standardized tests were.

His solution to both problems was to expand testing so that it measured the entire range of human capabilities - spatial orientation, dexterity, honesty (yes, honesty), mechanical aptitude, and all the other skills and abilities for which actual quantitative testing had been developed. That would confirm what people intuitively knew - that straight A students did not automatically rule the world; and allow schooling to be of value to all students.

It would identify each student's strengths and weaknesses so that - ideally - the student, his or her parents, and teachers, could all work towards diminishing the weaknesses and improving on the strengths. Dad believed that everyone could learn to do better. It was not just a matter of faith. He knew that, even in the area of I.Q. testing, repeated drill and practice worked. Students could improve their scores by almost an entire standard deviation simply by taking an I.Q. test every 3 months instead of only once or twice in their entire schooling. Drill and practice worked. The reason it fell out of disfavor was simple; it was hard work for both the teachers and the students.

Nigel Davies adds: 

I'm sure discretionary traders would do badly. But a key factor here is the state of the human material the firms start out with, and even the ancient Hagakure notes falling standards in manhood:

"… [W]hen one comes to speak of kaishaku, it has become and age of men who are prudent and clever at making excuses. Forty or fifty years ago, when such things as matanuki were mainly considered manly, a man wouldn't show an unscarred thigh to his fellows, so he would pierce it himself.

"All of man's work is a bloody business. That fact, today, is considered foolish, affairs are finished cleverly with words alone, and jobs that require effort are avoided. I would like young men to have some understanding of this."

Apr

20

When we do a study based on historical data and find a statistically significant result at the 5% level, we really are saying that there is less than a 5% chance that this study is completely attributable to chance. But if we observe some pattern in recent market action and then study it, that can be a problem: the multiple hypotheses problem.

One might think that if only one test is done that only one hypothesis was tested. Sometimes this is true. Other times traders will be intense students of the markets and notice a recurrent pattern. The trader then forms a hypothesis based on this pattern. It is properly tested on the most recent data and shows itself to be statistically significant.

There are two problems with this approach. First, if "the most recent data" include the same patterns that were observed and used to form the hypothesis then we are subject to the multiple hypothesis issue. This is true because that exquisite pattern-matching machine called the human mind continually looks for non-randomness and meaning in everything it sees. The mind tries out incredibly many hypotheses all the time. Most of us cannot even guess how many hypotheses our mind tries out before we identify one as interesting. So including the data, which formed the hypothesis, implicitly includes an element of multiple hypothesis testing.

The other problem is that we already know that the data will validate our study because it was used to help form the hypothesis. So it is not independent data but inherently biased. Thus our significance tests will be biased toward acceptance.

The best way to do these kinds of studies is to form the hypothesis on one data set and to test it on another completely different data set from another period.

Bruno Ombreux adds:

Or consider the same period but another market. For instance, if some phenomenon shows up in US stocks, test it on French and German stocks, too. There must be a reason for the putative phenomenon, either microstructural, behavioral, or economic. If so, it should show up in several markets. This extends the amount of testable data. One must be cautious with microstructure however, because it can differ. 

Philip J. McDonnell responds:

I do not agree with the idea of testing on data from different markets during the same time period, because many markets are highly correlated on a coterminal basis, sometimes as much as 90%. So it is really not an independent test on independent data.

But when one uses different time periods the correlations drop to near zero. So we can conclude that the data are truly out of sample.

Bruno Ombreux replies:

Dr. McDonnell is 100% right, but I still think it is not completely worthless to extend the sample to other markets. If you test a hypothesis on the US market, you'll be interested in the cases when you reject the null. Now, you test the German market and you still reject the null. You're right — not very useful. But if you fail to reject it on the German market, you need to come up with a very good explanation why it would work in the USA and not in Germany.

This is not nearly as good as different time periods, but it can be useful and increase understanding. 

Yishen Kuik adds:

I like to take an idea that has demonstrated its worthiness in actual trading in the US, then port it to other countries to see whether it works or not. If one has a group of countries for which the idea works and another for which it does not, it becomes interesting to try to figure out what members of each group have in common.

Nigel Davies remarks:

Presumably you're also taking account of time zones here. I've noticed that other markets tend to be led by the US during the day session (and even a couple of hours before its open) and have their measure of independence at other times. China is probably leading the overnight action now and Europe dominates during its morning. So perhaps it's not so much cultural as different time snapshots showing a certain similarity.

Martin Lindkvist extends:

Like the human flus that originate in Asia, many market ones seem to come from there too. Now, last night's Chinese flu seems to be of the same strain as that of late February. And as such, the market's immune system should be better prepared now. Perhaps a bit of coughing, and some sneezing for a little while, but not much of a fever this time? 

Henry Carstens adduces:

From a book recently recommended to me: "Routine design involves solving familiar problems, reusing large portions of prior solutions. Innovative design, on the other hand, involves finding novel solutions to unfamiliar problems." To borrow a quote from a friend, "Better necessarily means different." 

Apr

12

I'm looking for a book or paper that will help me think about trading and building trading systems from a fresh perspective. I am not looking for a trading book. I am looking for something that tackles a big question in a big way.

Three examples:

  1. In his notebooks, Da Vinci tackles learning to draw by thinking about and exploring straight lines (linear solution).
  2. In The Timeless Art of Building, the author integrates art, flow and aesthetics into architecture (gestalt solution).
  3. In Notes on Programming, Alexander Stepanov talks about knowing when a program/function/algorithm is correct (correct solution).

I have found that the way to get better at what I do is to choose a path, incrementally improve until the delta improvements become too small too matter (or be interesting), and then find a new path and start the process all over again. The book I'm looking for will help me find a new path.

Russ Herrold writes: 

In scanning this piece, it refers to TAOCP by Knuth,

1. Knuth, Donald E. "The Art of Computer Programming," Volumes 1-3 Boxed Set, 1998 Addison-Wesley Professional (1998), Edition: 2, Hardcover, which I too have used for years (decades) as my polestar (I have a set for the office and a set for home); but times change, and my coding partner has convinced me that I also needed to look more broadly, and see more modern approaches. As he spent over a decade attaining his Computer Science PhD, and teaching along the way, I tend to listen to him in such matters. Also the code inside are an expression of "the software engineering techniques [which he, Bill Pippin] used to control program complexity [as Stepanov also mentions early on]. Those techniques extend the implementation work done as part of [his] doctoral dissertation, "Optimizing Threads of Computation in Constraint Logic Programs," in particular by demonstrating a non-trivial instance of the single-tree pattern, whereby all singleton types are parameterized and then stratified by their binding pattern.

If you think: "wow, that sounds dense", and you read C++, take a moment and read the headers and the code. Bill recently wrote a roadmap to reading it.

The 'single tree' and its (relentless) application to the problem and space we are addressing (exploring the conflicts between the theories trusted by fundamentals investors, and the practical results observed by technical traders in reality [a favorite topic of this list] — Bill and I each started as Nixon Era Economics wonks in Washington DC, in the era of the now forgotten religion of Chicago School monetarism) is a really _big_ and non-trivial system. But perhaps not a formal work per se. Yet…

Each of the following either looks at a 'big question' area, or apply a method to solve a non-trivial (big) question. Reference to trading and investing are tangential.

2. Skiena, Steve S. "The Algorithm Design Manual" 1998,and 3. Skiena, Steven "Calculated Bets" 2001. The first is a more contemporary yet sound algorithms (tying to the mention of provably correct" solutions) work (with a fine bibliography), and the latter just plain thoughtful and fun.

4. Cormen, Thomas H "Introduction to Algorithms, Second Edition" 2001. This is the modern leading work on algorithms, but appallingly dense; I recommended the Skiena works first, as I find them more approachable.

5. Hofstadter, Douglas R. "Godel, Escher, Bach: An Eternal Golden Braid." It is one of those books one should take a month to read, and which has delighted me with new insights for twenty-five years each time I re-read it (another delightful bibliography).

6. Aronson, David R "Evidence-Based Technical Analysis: Applying the Scientific Method and Statistical Inference to Trading Signals" 2006. Not as to trading, but it applies scientific method in thinking about our trading beliefs.

7. Mehrling, Perry "Fischer Black and the Revolutionary Idea of Finance" and 8. Black, Fischer "Exploring General Equilibrium" 1995. This delightful pair being what I feel will be the reference biography, and the last work, as to unanswered questions, of this major 'counter' taken from us too soon.

This personal library inventorying tool has finally solved the desire I had for a tool to feed an ISBN, letting it gather and retain the rest. Also recommended.

Sam Humbert adds:

Perhaps too obvious to mention are the Tufte books. I got a lot out of his first book, The Visual Display of Quantitative Information, and sequentially less from the later volumes (much as I found food for thought in the Expert's first book Dynamic Hedging, but less in his later writings).

Mar

6

Some of Kent Osband's ideas in Iceberg Risk: An Adventure in Portfolio Theory, such as dividing strategies into "teams," are quite interesting. And I find Ralph Vince's, The New Money Management useful for finding maximums and helping to rank systems and strategies. He also has a new book coming out, The Handbook of Portfolio Mathematics: Formulas for Optimal Allocation & Leverage.

Jan

7

Is this the best trading book of the past year?

No, but Blink by Malcolm Gladwell generated more ponderable and testable trading ideas for me than any other book in recent memory.

Blink is about how intuitive decisions are made. The book is composed of a series of scientific case studies, each of which brought an 'Aha! Trading' moment for me. The cumulative sum of these ideas easily filled a couple of notebook pages, the study of which will fill and influence months of work.

One example the book shows is how a simple, small factor algorithm surpassed ER doctors in determining if a patient was actually having a heart attack. The conclusion was that the judgment of ER doctors was affected too much by information.

As traders and market researchers, we are continuously confronted with too much information, and we usually end up going down paths like 'If (this and this) Then…' or 'If (this or this) Then…' but we rarely go down paths like 'If (this is not present) Then…'

That type of twist, from 'and/or' to 'not' is precisely what made Blink so interesting for me: the ideas it generated were more revolutionary and perspective-changing than evolutionary.

Sam Humbert comments:

…a simple, small factor algorithm surpassed ER doctors in determining if a patient was actually having a heart attack. The conclusion was that the judgment of ER doctors was affected too much by information.

I've been thinking about this lately. Since this fall/winter has been warm in New England, I've been out on my bike at least once a week. And I need to dress properly, given the winter temperatures and the self-generated windchill from riding reasonably fast.

What I've found through trial-and-error is that I'm better off going to weather.com and dressing based on a mechanical system (40s = jersey + 2 fleeces, 50s = jersey + 1 fleece, low 60s = jersey + windbreaker, high 60s = long sleeve jersey etc., adjusted for unusual wind or rain). Then I am by standing in my driveway to "see how it feels."

My subjective markings, it turns out, are prejudiced by ephemeral factors (sun is behind a cloud, a gust of wind blows through) and also by preconceptions ("it's winter, so it should be cold and windy," "it was warm yesterday").

I've sometimes gotten darned hot or cold by dressing by "how it feels," but I'm never too far off dressing by weather.com.

Rod Fitzsimmons Frey adds: 

I'm glad that others got good things out of Blink! I thought it was one of the most deceitful books I have ever read. Perhaps I judged too quickly (blink!) and read the rest through a negative filter, but I thought it was an anti-intellectual defense of emotional intuition over careful rationality.

As a remedy I suggest Think!: Why Crucial Decisions Can't Be Made in the Blink of an Eye by Michael LeGault, as a fast-and-dirty response, or The Closing of the American Mind by Allan Bloom as a much deeper criticism.

Dr. Aronson addresses the ER physicians example (or something like it) in Evidence Based Technical Analysis (around p.42). He cites many studies that show that human decision making is very effective for linear and sequential problems and hopeless for configural thinking. When faced with configural problems, humans tend to reframe them into linear or sequential problems. Often this works, but for some things (like medical diagnoses), it is disasterous. It was a much more satisfying analysis of the issue than given by Gladwell.

Nigel Davies adds:

One of Bent Larsen's favorite expressions was 'long think, wrong think.' I think there's a lot of truth in this. Many people seem to tie themselves in knots by thinking too deeply and by considering so much information that they simply confuse themselves. But there's a paradox here in that good intuition requires mastery of the medium concerned, and that requires extensive testing, revising and doubting of one's conclusions.

I'd suggest that it's easy to play a blinker, but it's hard to play a master who can blink.

Nov

30

Dr. Brett Steenbarger has written a new and wonderful book, Enhancing Trader Performance, that has kept me thinking about trading, performance and niches since I got it.

The book starts with four composite traders built up by personality type. It’s just so fascinating to see oneself illuminated that way. Next, the book takes each composite trader through a series of hurdles before he discovers his trading niche. Again, quite illuminating and it sets the stage for the second part of the book which provides sound, proven tools and techniques for performance enhancement.

The book is beautifully written, easy to read and worth orders of magnitude more than the price of admission. A psych book for quants — imagine that!

Archives

Resources & Links

Search