Context-Dependent Choice: Modeling, Prediction, and Remediation

Normalized value coding provides insight into both the physiological mechanism of decision-making and the efficiency of choice behavior. Using computational modeling and human choice experiments, we are exploring how normalization and stochastic variability in neural value coding can explain many context-dependent violations of rational choice theory. Ongoing projects include the algorithmic modeling of context-dependent choice behavior, the characterization of novel forms of contextual choice inefficiencies in human subjects, the development of normalization-based compensatory behavioral strategies, and the examination of context-dependent value coding using neuroimaging techniques.

Louie, K.L. & Glimcher, P.W. (2012). Efficient coding and the neural representation of value. Ann. N.Y. Acad. Sci., (1251): 13-32. [pdf]

Fooling the System: Reassigning Value Through Exogenous Dopamine Activation

The current project seeks to extend upon the work on response properties of dopamine neurons previously conducted by Schultz et al. (1997), Nakahara et al. (2004) and Bayer & Glimcher (2005). Their experiments clearly demonstrated that dopamine neurons increase their firing rate to an unexpected reward in a way that aligns with the reinforcement-learning theorem and suggests that dopamine neurons encode a reward prediction error. This error term teaches subjects about the value of objects in the environment around them. For the experiment we engage non-human primates in a dynamic learning task where they are choosing between two stimuli, one has a smaller reward associated with it than the other. We electrically stimulate in the ventral tegmental area (thereby activating dopamine neurons and creating a positive reward prediction error) directly following reward of the stimulus with the smaller reward associated with it. We are able to show that the subjects’ preferences switch to the smaller rewarded stimulus in a way that can be modeled with the reinforcement-learning theorem. Our hope is that this experiment helps describe in some ways how psychostimulant drugs work to hijack the reward circuitry.

Can Deep Brain Stimulation Reduce Preference for Cocaine?

One possible cause of addiction is that drugs of abuse inflate the value of actions above their natural value – its not so much that addicts like drugs as that they want them. Many drugs of abuse, like cocaine, disrupt the normal functioning of the midbrain dopamine system, which is thought to be responsible for how the values of actions are learned and updated. It may be possible to inhibit the midbrain dopamine system at the precise moment at which these values are updated, and therefore reduce or negate the effects of cocaine. In this line of research, we use the magnitude matching task of Herrnstein to assess a thirsty monkey’s relative preference for cocaine over water on a trial-by-trial basis. Then, we use deep brain stimulation of the Lateral Habenula to inhibit the dopamine system precisely when the monkey chooses cocaine. The monkey should reduce its preference for cocaine, or even prefer the water.

Neural Random Utility: A Theoretical Framework Linking Neuroscience to Stochastic Behavior

An important step in our research is creating a theoretical framework which can bridge between the detailed constraints and processes at the level of neuroscience, and the more abstract (but flexible) modeling of choice behavior at the level of economics. One project, the Neural Random Utility Model development, attempts to lay out such a theoretical framework, and allows us to relate the stochasticity in networks of spiking neurons to the stochasticity found in choice behavior. Related projects attempt to impose more direct neural constraints on the model, exploring how (evolutionarily adaptive) information processing constraints can lead to seemingly sub-optimal choice behavior, and the formation of a reference point.

Webb, R., Glimcher, P.W, Levy, I., Lazzarr, S., Rutledge, R. (2012) Neural Random Utility. Social Science Research Network. [pdf]

Decision-Making Across the Life Span

Characterizing behavioral changes in decision-making across the life span and understanding why they occur has significant implications for behavioral problems associated with poor decision-making at different stages of life - such as careless driving in adolescents and disadvantageous medical or financial decision-making in the elderly. Scientists in many disciplines have long observed that age seems to be a significant determinant of decision-making under risk and ambiguity. There has, however, been significant disagreement about how and why preferences toward risk and ambiguity change with age. Research on risk attitudes and age has so far focused separately on adult or minor populations, making it hard or impossible to assess the differences in risk attitudes between adolescents or older adults and other age groups. The most important result of this controversy has been the reliance, by policy makers, on a set of stylized facts about the decision-making behavior of mid-life representative agents. But whether those stylized facts about the representative agent are robustly true, what those stylized facts mean for decision-makers of different ages and how those representative mid-life agents relate to individual decision-makers has never been exhaustively examined. The goal of this project is to provide a comprehensive study of decision-making in a population that ranges in age from 12 to 90 years old using both behavioral and fMRI research techniques.

Additionally, in cooperation with the Museum of the National Academy of Sciences in Washington, we designed an interactive exhibit in which museum visitors can estimate their own risk and ambiguity attitudes by participating in a simple, incentivized choice task. Over the next several years, we will use this data set to assess the impact of (1) age and other individual-specific factors: marital status, birth order, birth cohort, gender or culture and (2) individual-independent factors such as macroeconomic shocks, weather or context, on attitudes toward known and unknown risks.

Tymula, A., Rosenberg Belmaker, L.A, Roy, A.K, Ruderman L.,Manson, K., Glimcher, P.W, and Levy, I. (2012) Adolescents' Risk Taking Behavior is Driven by Tolerance to Ambiguity. Proceedings of the National Academy of Sciences of the United States of America, 109 (42): 17135-17140 [pdf]

Temporal Context Effects in Decision-Making

People make thousands of choices every day. Reference dependent preferences have a huge and detrimental economic impact. Recent price history and related phenomena lead to increases in crime, distortions of the labor market and inefficiencies in investor behavior. To date, no model has been developed which can capture these effects completely. The goal of this project is to advance our understanding of why and how a decision makers’ evaluation of outcome quality changes when the benchmark against which these outcomes are compared, the reference point, changes. As a final product of this research project we propose to construct a novel model of reference dependent preferences, based on temporal normalization models from neuroscience that describe the biophysical calculations hypothesized to underlie the physical instantiation of the reference point. This model should be able to account for a wide range of choice inefficiencies/irrationalities that are observed in everyday life, as well as offering a mechanism for suggesting and modeling remedies. The unique strength of this project is its joint application of tools from economics and neural science in an effort to build a complete model of the reference-dependent preferences which plague both consumer and investor choice behavior.

Wealth Effects in Decision-Making Under Risk

Standard economic techniques allow us to evaluate human risk-attitudes, although it has been technically difficult to relate these measurements to the overall wealth levels standard models employ as a critical variable. Previous work has, however, applied these techniques to animals to answer two questions: 1) Do our close evolutionary relatives share both our risk attitudes and our economic rationality? 2) How does satiety state (or wealth level in the language of economics) change risk-attitudes? Previous studies have provided conflicting answers to these questions. To address these issues, we employed standard techniques from human experimental economics to measure monkey risk-attitudes (utility function curvature) for water rewards in captive rhesus macaques as a function of blood osmolality (an objective measure of water wealth). Overall, our monkey subjects were slightly risk-averse in a manner reminiscent of human choosers, but only after significant training. Monkeys consistently violated expected utility theory (violating first order stochastic dominance) early in training, indicating that traditional economic models cannot be used to describe their behavior at that stage. Once these choosers were rational, measured risk-attitudes were thirst-dependent. But unexpectedly, as the animals became thirstier risk-aversion actually increased, a finding that may be incompatible with some standard economic models..

Flexible Valuations for Consumer Goods as Measured by the Becker-DeGroot-Marschak Mechanism

In this project we experimentally investigate whether valuations elicited by the commonly used Becker-DeGroot-Marschak (BDM) procedure depend on the distribution of prices that are used in the elicitation mechanism. To answer this question we created a novel within-subject design that allowed us to observe an individual's bid for a given product repeatedly while varying the price distribution. Our data clearly show that subjects do not bid constantly the same amount for each good on each offer as would be predicted by EU theory, but rather show a mass-seeking bias. This bias is strongest when the mass of the price distribution is close to the average bid that the subject places on the good. Bids are influenced by the mass to a lesser extent when the mass of the distribution is further away from the mean bid. We characterize preference structures that are consistent with the observed behavior.