Daily Speculations The Web Site of Victor Niederhoffer and Laurel Kenner


The Chairman
Victor Niederhoffer



About Victor Niederhoffer

2006 All content on site protected by copyright



Write to us at:(address is not clickable)

Operant Conditioning, by Victor Niederhoffer

The Hawaiian polymath James Sogi recommends Coercive Family Processes by Gerald R. Patterson. The book discusses how to measure and study aggressive behavior, and has already lead to great controversy in my family, as it recommends an authoritarian approach to raising children by removing what kids value, e.g. attention, when they are bad. Don't give them attention when they cry. Removing the attention is called negative reinforcement. The whole subject of how we behave when faced with stimuli of various kinds, with selling and buying being the behavior, and the environment, e.g. an economic announcement, a vivid change in a related market, or a backdrop of staged conditioning by the Fed Commissioners, would seem to call out for study and testing. This introduction to operant conditioning provides a nice summary of the kinds of things that behavioral psychologists study and might open up some fruitful lines of inquiry. A good reference to Patterson's work can be found here.

In examining the diverse bodies of stimulus and response schedules covered by behavioral psychologists, one comes away with the impression that the grass is always greener on the other side and that if instead of following the promiscuous theories of cognitive psychology, that have a hypothesis for any seemingly irrational behavior, (albeit most of them are completely rational and based on rules of thumb that people in real life as opposed to college students for a buck an hour would choose), the often validated and completely specified studies of operant conditioning would be a much more fruitful line of inquiry for market people. One feels he is one the right track here as "Operant Conditioning" and "Stock Market " is almost a Google whack at 337 mentions but "Operant Conditioning" "Cognitive Psychology" has a promiscuous 38,700 mentions.

It would be good to take the basic two by two table of operant conditioning and classify it by fixed ratio, fixed interval, variable ratio, variable interval, and see how these relate to predictive patterns. For example: bonds up/ stocks down, a positive reinforcer when it occurs at a steady rate with little variation (fixed interval) versus when it occurs with great variability (variable ratio). But bonds up/ stocks down, if it occurs at an unsteady state, it is an example of a positive punishment variable ratio. All the predictions of operant conditioning could be tested in the real world of humans with prices in markets, instead of on rats.

(behavior increases)
(behavior decreases)
(something added)
Positive Reinforcement:
Something added increases behavior
Positive Punishment:
Something added decreases behavior
(something removed)
Negative Reinforcement:
Something removed increases behavior
Negative Punishment:
Something removed decreases behavior

Source: "An Animal Trainer's Introduction To Operant and Classical Conditioning"

Alston Mabry Replies:

As I understand it, in animal learning trials, if you put the rat in the cage with the little lever, eventually, in the process of exploring the cage, the rat pushes on the lever, and there is some possibility that a bit of food plops out. The process repeats, and the rat learns to associate pushing the lever with getting food.

Interestingly, if what you want is for the rat to push the lever a lot, you provide the food reward only intermittently and randomly. If the food is provided each time the rat pushes the lever, the rat will push the lever only when it is hungry. However, if the food appears only occasionally when the lever is pressed, the rat will press the lever over and over, brimming with anticipation.

Now let's assume the Mistress is a master trainer, to her own benefit. She places the rat (trader) in it's cage (home office with high-speed internet access, TradeStation account, etc.) and waits until the rat discovers the plastic keys on the keyboard and starts tapping them. Then she provides the rat with a food pellet (profitable trade).

If the Mistress wants the trader/rat to trade as often as possible, she will reward the trader/rat with a profit (food pellet) only intermittently and randomly. If the trader/rat could get profit/food any time it pleased just by tapping the keys on the keyboard, then it would tap the keys only when it needed money. But because it is actually the Mistress who is in control, and she wants to maximize trading behavior from each rat, she keeps the rewards as random and unexpected as possible.

In fact, "unexpectedness" is one of her most important tools. By the Rescorla-Wagner model of conditioning, the greater the unexpectedness of the reward, the higher the associative strength of the learning. This is why it is so effective for the Mistress, after a rat has tapped the keys many, many times with no reward at all and become convinced in bleak despair that no further reward is possible, to toss a nice food pellet into the cage and provoke the rat to even greater efforts.

Russell Sears responds:

This is of course the opposite of what is recommended for a baby totally dependent on the parent. I find this one of the greatest challenges of parenting, determining when to use negative reinforcement to cut off the dependency. And looking around to family and friends, especially with young adults, it seems many have never truthfully acknowledged this.

Steve Leslie adds:

This is exactly the foundation of slot machines. Intermittent rewards promote more activity on behalf of the participant. The theory is that if one gets rewarded on equal installments the activity is seen as work, whereas if one receives an intermittent reward then it is seen more as recreation.

This is also how companies motivate their salesmen and saleswomen. They conduct sales contests but they do it randomly. It is one way that the company keeps the salespersons attention. Brokerage firms were famous for offering sales contests during the summer months, typically the slowest months for commissions to keep the brokers working and keep the revenue flowing.

Here is a sidebar to this discussion. In Las Vegas, if a casino advertises that they give a 99% payout on their slots, then they must pay out on average the machines that they have posted to pay out that amount.

This does not mean that every slot machine in the casino pays out 99%. It applies only to the bank of machines that are listed as paying out this amount and the patron has to look long and hard inside the facility to find those.

What this does mean is that if you took a large enough sample size for example a $1 slot machine and played this machine forever and each individual were to put $100 in and no more, taken collectively they would receive back $99 on average.

Now statisticians will tell you that everyone who plays slots will eventually go broke. The reason for this is that people continually take their reward and plow it back into the machine until eventually they have spent their full bankroll. Therefore the machine will collect everything, it just takes longer if the payouts are higher.

This applies to all other games as well including roulette baccarat and dice. Even though you can approach almost even money odds such as betting the color on a roulette wheel, the player only on the baccarat table, and the line on the craps table, if you keep playing them long enough you will lose your entire bankroll.

Jay Pasch replies:

Markets are authoritarian, nature is authoritarian, society is authoritarian, the world they're going to live in is authoritarian, "ya gotta serve somebody" as Dylan would say. Of course there is great benefit to self and others in going against at times, i.e. Thoreau's Civil Disobedience, the rebel call, et al. But on the battlefield of child-rearing, relieving one's self of authority is like dropping one's arms on the field, and pants, and waiting to take one between the... eyes. What works best for the young warriors is that they have 'contracted' to decency and respect with all of the ensuing benefits and luxuries given their meritorious behavior; but break the contract and it is they that surrender their benefits, rather than the mindset that some sort of entitlement has been 'taken away'. Under this arrangement the kids have buy-in, they feel important, creative, their ideas beneficial, because they were asked to help create their world in the first place. They see clearly the reality of their own behavior, understanding it was they that surrendered their privileges rather than the big bad general removing their stripes...

Daniel Flam replies:

It would seem to me that all education revolves around pain. So you say we can't "flik" the kids? Ok let's give them a mental pain Like take away something they like, put them in the corner, its like the way the intelligence interrogators in the western world operate under the democratic laws, we just find a better way of inflicting pain in confines of the law...

I find the same with the market... which bring an old adage... "No pain, no gain"

How would we go about studying pain in the market?

Steve Leslie replies:

First let me say that "No Pain No Gain" is a very dangerous statement. Physical pain while training is an indication that one is approaching a physical limit. By going too far, one can instill permanent damage. Only a fool would feel a muscle tearing during a set of lifting weights and continue to lift weights. Now there are minor aches and pains that an athlete must endure however there are limits that the body can withstand. An athlete who is in touch with their body is well aware of the difference.

I am sure my good friends Dr. Goulston and Dr. Dorn are much more qualified than myself to comment on this subject matter and I hope that they do weigh in.

However, there are three distinct subjects here.

Giving a child an iPod for excellent grades is positive reinforcement. Withholding a reward from a child or taking away privileges would be negative reinforcement. Yelling and/or corporal punishment would be forms of punishment

They are very different.

The problem with punishment is that it has a very short term result. And repeated punishment eventually will result in no positive result whatsoever.

Please forgive me for probably misrepresenting this study but here goes: There was a famous study performed where an electric grid was installed in an enclosed box. Mice were placed in the box and half of the box was shocked. The mice went over to the other side away from the pain. Then a barrier was installed so they could not move from one side of the box to the other. Then the mice were shocked. They initially tried to escape to the other side. However the barrier would not allow them to move over. After repeated shocking, the barrier was removed. The mice were shocked yet they did not move over to the safe side. In effect, they were conditioned to just sit and take the pain.

Think about this: When your dog runs away and you beat it. That is punishment. If the dog runs away and you beat it again it will be trained to stay away. If you beat a dog long enough eventually it will just lie there and allow itself to be beaten.

This is shown dramatically in abused wives. They become beaten physically and/or mentally and that if this occurs long enough that eventually they just sit there and continue to be beaten. And should someone come along and offer them sanctuary, the abused wife will chose to stay with the abuser.

Someone once said you train animals but you teach children.

If you really want to go into deeper understanding of this, I recommend an exceptional person Dr. James Dobson either in his numerous books on this subject most notably Love Must Be Tough. He also hosts an extremely informative radio show entitled Focus on the Family. My church radio station broadcasts this as do many Christian radio stations around the corner. He is seen very regularly on Fox shows such as Hannity and Colmes.

Daniel Flam adds:

Having spoiled brats that everyone in the room hates to be around because you don't want to put them in their spot, Will just delay the point in time where someone that is not a family member will put him in place in a most unpleasant way.

Bringing up Children is like painting a work of art. You must use all the colors of the spectrum, although some colors should be used a very small dose, or you might get an ugly result.

I see additional factors to the one suggested:

Today we find names for anyone who doesn't behave like a sedated rabbit.

This reminds me of that shirt "I hate it when people think I have ADD! Oh look, a chicken!"

James Sogi replies:

Rather than 'greed' and 'fear', counting, like behaviorism, is more scientific. Quantify to predict. The market trains everyone to do the wrong thing. When one is trained to go long, the market goes south. When one is trained to play the range, it breaks out. Of course it trains one in the just the most intermittent and thus most powerful manner, like slots, to go the wrong way. It is called variable reinforcement. Counting gives the clue that the training is in play and not to follow the masses and to stay a step ahead of the market. Be the trainer not the trainee. Who is in control here after all.

Little babies train their parents. It is the brat in public that has the haggard parent running around like a chicken. Both are miserable. Proper training involves the use of love attention and affection. It is not the rats-in-a-box syndrome. The natural reaction is to run to the crying baby. That merely reinforces the crying. The natural crying pattern has variations. When there is a break in the first few moments of crying, use that moment variation to sooth the child. The reinforces the calm not the cry. Inconsistent parents give mixed signals can cause confused children, unhappiness. Consistency give certainty and clearness to the child. I tried to see how many days we could et my kids without crying. How many times per day would they cry? Why did they cry, what were the operant conditions? Quantify the responses. Forget the mumbo cognitive jive.

In the market, the public rushes to the upsurge, but is this the correct response? When the market tanks, the public trained panics. Again, scientists, is this the right response? Quantify one's own responses to get an idea of what works, what doesn't. consistency brings profit.

J. T. Holley reminisces:

My PaPa would espouse to me "the grass might be greener on the other side but someone has to mow and rake it too" whenever I would act like those cognitive psychologists! I think the operant conditioning like B. F. Skinner is appropriate for those dealing with the markets. The classic philosophy (shortened and brief) is that Plato felt to "know the good was to do the good", whereas Aristotle had a more operant conditioning belief in that "to do the good was to know the good".

Russell Sears suggests exercise:

What the kid needs is an outlet for his energy. Have the kid run a few lapse, go a few miles on his bike, or even shoot some hoops. I would suggest, that what Lackey encourages his kids to do has more to do with his kids well adjusted behavior . Lackey little league, and coaching wouldn't see these kids. Kids with no competitive outlet, takes it out on the adults. Exercise generally works better than any drug for mild depression. But what Doctor will prescribe 2-3 miles run everyday for 2 months to a single Mom for her kid. Its called "child abuse". But giving him mind altering drugs, to a developing growing brain, is called "therapeutic care."

Pamela Van Giessen laments the loss of just being a kid:

This seems to be part of a larger issue where every single moment of childrens' days are being structured and moderated by adults. There is school, soccer practice, swim lessons, judo, music lessons, play dates, etc. It's kind of like jail. Even worse because at every turn there are adults loitering, supervising, and otherwise keeping a watchful eye. I call them helicopter parents. They mean well, but I can't help but be eternally grateful for my parent's lack of vigilance. I read an excerpt from John Dickerson's book about his mother, Nancy (first female TV news star), where he noted how absent his parents were and that he and his siblings were often left to their own devices, and how, in the long run, that turned out to not be an entirely bad thing. My American nephews are supervised 24/7 and while they are smart and adorable children, I notice that they are more prone to temper tantrums and the like. My Dutch nephews roam free; they rarely have a baby spell. And, honestly, the Dutch kids seem more creative and amusingly naughty. I like children who stick carrots up their nose at the dinner table, provided they are stealthy and quiet about it. Kids don't put up with other kid's temper tantrums and so children who hang out with children stop behaving like brats -- at least if they want to have friends.

At the age of seven, I was biking a mile to go get candy. I rarely see children about my 'hood without adults. Can't they even go to the bodega without Mom? At what point will they not be supervised and watched over?

I've also noticed that the young women (oh, how I hate saying that) that work for me seem to approach their jobs, careers, and even daily to-do list like a school exam that they must ace. They miss the larger point about spontaneity, about creating, about doing as you go and it all becomes about getting an A and moving on to the next "test." They also seem to structure their lives accordingly. From x-time to y-time is work time, from z-time to a-time is not work time. One hopes that romance isn't scheduled so rigidly.

When I think of all the wonderful experiences and successes (and even some failures) I've had by being spontaneous, by looking in rooms I wasn't due to be in, by not scheduling my life with much structure it makes me sad to see us creating a society of automatons.

Nat Stewart adds:

One of the most worrisome trends in my view is the "bans" on student organized, spontaneous recess games, which for me were always the highlight of the day in the early grades. The spontaneity and sense of it being "ours" and not a teacher/instructor lead activity also increased the value and fun of these activities.

I think for many kids this type of vigorous exercise is almost a need or requirement, It certainly was for me.

Kids who are naturally curious, such as this kid in the article who is a "gifted reader" need independent outlets to exercise their own curiosity, and opportunities for individual study and thought.

I think many of these kids are just bored stiff! The extreme bureaucratic environment is not a good learning environment for many children.

Kid can use logic, and I believe many start to rebel and have trouble when they are repeatedly asked to do things that they do not find logical. "Johnny has a problem..." Well, maybe he is mad that so much of his day is wasted in useless, pointless, mind numbing activities? Maybe he would rather be off on his own, reading a book. Kids can be sensitive to injustice, and little things over time poison can poison ones attitude to the entire process or system, which is unfortunate.

All kids are different. Labeling children with 1000 different Disorders is only a smokescreen that hides our severely dysfunctional system.

Profess Gordon Haave replies:

I would suggest that what is wrong with the children is nothing... except a total lack of discipline and their learning at 5 when taken to a psychiatrist that being crazy is normal and they can do whatever they want because they are not being bad, they are "sick".

Another good thing about Oklahoma: I don't know anyone who sends their kid to a psychiatrist. Kids get discipline, hard work, and an ass-whupping if they do something particulalry egregious.

November 11, 2006 Troubled Children What's Wrong With a Child? Psychiatrists Often Disagree By Benedict Carey
Paul Williams, 13, has had almost as many psychiatric diagnoses as birthdays.
The first psychiatrist he saw, at age 7, decided after a 20-minute visit that the boy was suffering from depression.
A grave looking child, quiet and instinctively suspicious of others, he looked depressed, said his mother, Kasan Williams. Yet it soon became clear that the boy was too restless, too explosive, to be suffering from chronic depression.
Paul was a gifted reader, curious, independent. But in fourth grade, after a screaming match with a school counselor, he walked out of the building and disappeared, riding the F train for most of the night through Brooklyn, alone, while his family searched frantically.
It was the second time in two years that he had disappeared for the night, and his mother was determined to find some answers, some guidance.

Steve Wisdom responds:

The long-time sense of the word "discipline" was to instruct, educate, train. It somehow became twisted (as has the word "liberal") to mean, in common usage, Prof. H's "ass-whupping."

What does an "ass-whupping" instruct or educate? Well, it teaches that if you're frustrated, angry, tired or stressed, and have the advantage of being bigger and stronger than the other guy, then it's OK to indicate your frustration with verbal or physical violence.

Is this the what a parent wants to teach?

"Discipline", in the bastardized sense of the word, means the parent has failed. Failed to authentically instruct, educate, train. And is now lashing out, motivated by frustration, not by a desire to educate or improve the child. The parent's reptile brain is in charge.

And what becomes of kids who are beaten into submission for 12, 14 years.. But then become teenagers? How will they conduct themselves "out of eyeshot" of their parents, when their parents are around to "control" them with "discipline"?

What actually does work in parenting -- since "discipline" doesn't -- is spending time with kids, and most especially, meeting them at their level, not at your own. Becoming engaged in their lives, their interests, their hopes, fears, dreams. Really hearing them, rather than lecturing them.

My kids have never been "disciplined", and many parents in our town have commented to us that there are -- far from being "undisciplined" -- among the kindest, most thoughtful little boys they've met. The proof is in the pudding.

Profess Gordon Haave replies:

Although, as I have said, I don't believe in Ass whupping, I don't think what you are stating is correct. In its simplest form, it is the most crude way of stating "actions have consequences". Most of this on this list know that there are better ways of teaching that then ass-whupping, therefore they don't do it. Around here in Oklahoma, it is probably not very common, but was even just 15 or 20 years ago.

Now, what goes on in NYC is simply the opposite message, that actions don't have consequences, that nothing is your fault, that if you look out the window during class or talk back to your mother you have a problem that needs to be medicated.

Mr. Wiz suggests that those who receive an ass-whupping grow up having learned the wrong lessons, etc. I submit that it is better than the weirdos who grow up thinking that actions don't have consequences. They are more prone to destroying families and societies, in my opinion.

So, I will restate: Ass-whupping is preferable to the NYC psychobabble approach, even if it is crude in its own right.

Stefan Jovanovich replies:

The "ass-whupping" meme seems to me more than a bit overdone. Striking a small child is like beating a cat. Children are small creatures compared to us adults, and they spend most of the years up to the age of puberty navigating around us comparative giants. Simply restraining them physically - holding them still - is enough physical punishment for "acting out". What was notable in the article about poor Paul Williams is that his father - the person most likely to have the physical strength to be able to hold him still - is nowhere mentioned.

You can step on a cat's tail, and she will instantly forgive you even though the pain was excruciating. Intentionally strike the same animal with one-tenth the same force, and she will view you as an enemy until the day one of you dies.

I agree with Gordon's skepticism about psychiatric diagnoses. Since they almost always have no clinical basis in blood chemistry or any other quantifiable physical symptom, they are usually like visits before the parole board. The patient - i.e. prisoner - has to reassure everyone that he is "sorry" and will make a sincere effort towards "rehabilitation" - i.e. sitting still in school.

My Dad's theory was that compulsory education was invented so that the adults could find somewhere to warehouse the children during working hours. In his darker moments he also speculated that it was an expression of society's underlying belief that poverty was a crime. Since almost all children were destitute, society was simply doing what it did with other criminals - locking them up and then pretending that incarceration had some useful purpose.

GM Nigel Davies responds:

I agree. And given that one of the tenets of libertarianism is to remove physical force and coercion from human affairs, this seems to be given quite the wrong message. I strongly suspect that kids who get beaten will tend towards an authoritarian attitude to life.

There are more creative ways to instill discipline, such as gaining a child's attention by showing them something that actualky interests them and using a system of reward and punishment based on what they like to do. If good behaviour is rewarded it represents a trade and fosters an attitude to life based on exchange rather than force.

The President of the Old Speculators Club replies:

I recently read an article with a darker view -- suggesting that Americans who send their children to public schools are allowing the "state" to "kidnap" their children for 8 hours a day. Hours in which they are taught what it is believed they should be taught, and shielded from those things that might make them less than docile, cooperative citizens. The goal is to produce individuals who will view governments the provider of all solutions.

Roger Arnold replies:

When I was a boy, getting a butt tannin from time to time was a part of growing up, as it was for everyone else I knew. I can still hear the sound of my father's belt as it is pulled through his belt loops. My mother would send me and my brother to our room with a pronouncement of "wait til your father gets home", and we would sit in there laughing and joking until we heard the front door open -- and oh my god that's when the terror began.

Nowadays we joke about it at family get togethers and, although I have never raised a hand to my own child, I can understand the utility of the spanking as a tool of nurturing.