Dopamine: the reason you pursue reward

It was a beautiful fall afternoon. The breeze sent leaves of yellow, orange and brown sailing through the air. As my unleashed companion and I walked briskly through the park, the wind also carried something far more troubling straight into the sniffing nostrils of my faithful friend, the scent of a young female canine. I watched as my dog’s attention suddenly turned to the Collie and her master who were also enjoying the nice day at the other end of the park. By observing the subtle changes in his posture I knew where this was going. His head lifted and turned to the unaware pair, the tension spread through each muscle in his body starting with his head and ending in his hind legs which were now fully primed to spring forward. Not wanting him to disrupt the plans of innocent bystanders I responded in the only way I knew how. “Come!” I said as clearly and seriously as I could manage. His head turned toward me, and his ears perked to attention. But instead of immediately bounding over to me as was usual, I could see signs of the conflict raging in his brain. What to do? Explore that interesting-smelling female or obey the call of his friend and companion?

But wait, let me interrupt the story right there. Why would the sound of my voice have even a small chance of changing his plans to go explore this other dog? At first, the answer to this question seems obvious; because I spent years training him by giving him a bacon-flavored treat each and every time he responded appropriately. Indeed, humans have been using this trick of rewarding good behaviors to train animals – and other humans – for thousands of years. But upon closer examination my rigorous training program is a perfectly good answer to how I got my command to be so important to my little friend, but it doesn’t answer why it works so well that a single word from my lips now has the potential of overriding the instinctual urge to barrel full speed towards this possible mate. Decades of neuroscience research have finally filled in this question, and the answer it turns out is a microscopic, chemical neurotransmitter floating around in the brain called dopamine.

120px-Dopamine-3D-struct

A lot of our understanding of what dopamine does in the brain comes from research trying to figure out what causes addiction. There are many, many drugs available that change your brain’s chemistry in profound ways: antipsychotics, antidepressants, stimulants, opiates, and so on. But just because a drug affects your brain doesn’t mean it has addictive properties. In fact, there are some drugs with very similar effects in which one drug is highly addictive and the other not so much. Think about cocaine and caffeine for example. Both are stimulants, but cocaine can be extremely addictive whereas caffeine isn’t really. For a long time addiction neuroscientists wanted to know what’s the difference? What do heroin, cocaine, nicotine, and alcohol have in common that aspirin or Prozac don’t?

In the 70s and 80s a series of studies figured out that the one thing every addictive drug has in common is that they all increase dopamine in the brain. Some of them do it directly, like cocaine, and others do it indirectly like alcohol. But all of them do it one way or the other.

So we figured out that dopamine is the common factor in addictive drugs, but that didn’t tell us why these dopamine-increasing drugs were addictive. That part was worked out by a series of experiments done by Dr. Schultz in the 90s. These studies started out by simply recording the activity of neurons that release dopamine to see what they did. Each time a neuron fires it shoots an extremely fast electrical impulse down a long extension of the neuron (called an axon). By placing an electrode that is sensitive to electrical impulses very close to the neuron, you can detect each time it fires.

At first it seemed all too obvious what dopamine neurons were doing. Researchers found that if you gave an animal some kind of reward – like juice for example – while you were recording the activity of dopamine neurons, they fired like crazy as soon as the animal got the reward. This is shown in the figure below. Each “dot” on the lower section represents a neuron firing. The line above R (for reward) is when the animal got some juice. The upper section shows overall activity of this neuron by adding up all of the trials below.

Schultz fig 1

 

It seemed pretty obvious that these were the “reward neurons”. They fired when something good happened. That made sense too, because we know that addictive drugs increase dopamine. If these drugs increase the activity of “reward neurons” every time you take them it makes sense that you will find them very rewarding.

But of course nothing in neuroscience is ever this simple. Researchers were repeating the experiment over and over to get lots of recordings from these neurons. And as the experiment continued they began to notice something strange was happening. As time went on, the dopamine neurons stopped responding to the juice completely. That seemed to suggest that the animal no longer found the juice rewarding, but that was confusing because they still drank the juice enthusiastically. To make things stranger, the dopamine neurons weren’t being silent. They began responding to an image that flashed on the screen a few seconds before the juice just to let the animal know that the reward was about to come. So every time this image turned on indicating that juice would be coming out of the spigot the dopamine neurons increased firing, but when the juice actually came they fired at a normal rate as if nothing had happened. In the graph below you can see that now the neuron is firing to the image just before the juice (CS stands for conditioned stimulus) but not to the juice itself (R).

Schultz fig 1b

So if dopamine neurons represent reward or pleasure in the brain, these results suggested that the animal didn’t like juice anymore but loved seeing a square pop up on a screen. At first this was very confusing, but eventually neuroscientists began to work out what was going on. Every time you gave an animal a new reward, the dopamine neurons activated. If there was anything that predicted the arrival of that reward, eventually the dopamine neurons would start activating to the prediction instead of the reward. Cool! This seemed to suggest that maybe the dopamine neurons were not “reward” neurons, but instead were “reward prediction” neurons. In fact, in a later study researchers found that the dopamine neurons would give bigger responses to images that predicted bigger rewards, and smaller responses to images that predicted smaller rewards.

Tobler fig 1

 

The reason they were seeing dopamine responses to the juice itself at first turns out to be because there are two dopamine signal components. The first is the prediction like we’ve seen above. The second is the error signal. After the dopamine neurons make a prediction about how rewarding something will be, they appear to wait around to see if they got it right or not. If the reward is just about what they predicted it would be – prediction: some delicious juice is coming; outcome: we got some delicious juice – then there is no dopamine activity during the juice drinking. But if the reward ends up being bigger or smaller than they predicted, the dopamine neurons increase or decrease their activity to show that there was an error in their prediction.

When scientists were first giving the reward to the animal, the dopamine neurons were giving a big error signal because they had not made any prediction that a reward was coming but suddenly a delicious sweet beverage was delivered. Prediction: nothing is going on right now; outcome: sweet nectar of the gods!

Since addictive drugs artificially increase dopamine every time you take them, they are sending an error signal to your brain that the experience you just had was much more rewarding than predicted. Therefore the next time you encounter a situation with that drug you will feel more motivated to take the drugs again. This will continue each and every time you take the drug regardless of how big the prediction signal was from your dopamine neurons. It becomes a vicious cycle in which your dopamine system keeps increasing how rewarding it thinks the drug is, making it more and more compelling to take it. Eventually this cycle can get completely out of control where people start making very bad choices to take the drug.

This system is also why giving the dog a treat every time he obeys your command works. The first time you give the command the dog’s dopamine system has no reason to expect a reward, so there is no prediction. But when the dog comes and you give him a treat, the reward causes a surge of dopamine indicating that the experience resulted in a big reward even though no reward was predicted. If you keep pairing the dog’s obedience with a reward, eventually the dopamine system will begin responding every time you give the command indicating that obeying has a high probability of producing a reward. That’s why training works and why training has to be repeated over and over and over. You’re teaching his dopamine neurons to respond to your command.

And so here we are, having come full circle. My dog sees the female across the street. Instinctively his dopamine neurons surge promising possible social or sexual reward if he approaches her. The sound of my voice sends another surge of dopamine promising a high probability of food reward if he obeys and goes to me instead. That moment when the dog looks back and forth torn between choosing the two options is a visible display of these two competing reward predictions fighting it out inside the dog’s brain.

Thankfully, in my situation one dopamine signal eventually overcame the other. My partner in crime came bounding towards me, leaving the female dog and her human alone to enjoy their walk. From years of training, the dopamine signal predicting reward if he followed my command was so strong that even the potential of a young female could not compete. And you better believe that as soon as he made it to my open arms, I rewarded him with a delicious treat.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s