Electrode Position and Current Amplitude Modulate Impulsivity after Subthalamic Stimulation in Parkinsons Disease—A Computational Study

Background: Subthalamic Nucleus Deep Brain Stimulation (STN-DBS) is highly effective in alleviating motor symptoms of Parkinson's disease (PD) which are not optimally controlled by dopamine replacement therapy. Clinical studies and reports suggest that STN-DBS may result in increased impulsivity and de novo impulse control disorders (ICD).

Objective/Hypothesis: We aimed to compare performance on a decision making task, the Iowa Gambling Task (IGT), in healthy conditions (HC), untreated and medically-treated PD conditions with and without STN stimulation. We hypothesized that the position of electrode and stimulation current modulate impulsivity after STN-DBS.

Methods: We built a computational spiking network model of basal ganglia (BG) and compared the model's STN output with STN activity in PD. Reinforcement learning methodology was applied to simulate IGT performance under various conditions of dopaminergic and STN stimulation where IGT total and bin scores were compared among various conditions.

Results: The computational model reproduced neural activity observed in normal and PD conditions. Untreated and medically-treated PD conditions had lower total IGT scores (higher impulsivity) compared to HC (P < 0.0001). The electrode position that happens to selectively stimulate the part of the STN corresponding to an advantageous panel on IGT resulted in de-selection of that panel and worsening of performance (P < 0.0001). Supratherapeutic stimulation amplitudes also worsened IGT performance (P < 0.001).

Conclusion(s): In our computational model, STN stimulation led to impulsive decision making in IGT in PD condition. Electrode position and stimulation current influenced impulsivity which may explain the variable effects of STN-DBS reported in patients.

Introduction

Deep brain stimulation (DBS) of the subthalamic nucleus (STN), is a surgical technique now widely applied for the treatment of Parkinson's disease (PD) when dopamine replacement therapy fails to provide sustained relief of motor symptoms or induces drug-induced dyskinesias (Benabid, 2003). Though the exact mechanism of action of DBS is not well-established, it is known that stimulation disrupts (Rosa et al., 2012) the increased synchrony and bursting activity in the β band (8–30 Hz; Kühn et al., 2006) of the STN neurons.

Several reports have highlighted the development of new onset, often transient, impulse control disorders (ICDs) following STN stimulation (Hershey et al., 2004; Smeding et al., 2007; Combs et al., 2015). This was thought to be, due to stimulation of the cognitive sub territory of STN or the spread of stimulation to adjacent parts of the cortico-limbic circuits. In support of this theory, stimulation parameters such as current spread and electrode position were shown to affect the outcome in cognitive tasks in PD patients (Sudhyadhom et al., 2007; York et al., 2009; Witt et al., 2013). Stimulation of the ventral STN decreased the number of correct hits and increased the number of errors on commission in Go-No Go task, when compared to stimulation to dorsal STN (Hershey et al., 2010). STN stimulation can also increase risk taking behavior in Iowa Gambling Task (IGT; Evens et al., 2015). Patients with STN-DBS tended to overestimate their performance with a preference toward competitive environments (Florin et al., 2013). On the other hand, pre-existing ICDs were reported to resolve following STN-DBS, as a result of reduction in dopaminergic medication (Castrioto et al., 2015). Thus STN-DBS may lead to varying net effects on impulsivity in PD through different mechanisms. Clinical studies aimed at dissecting out the effect of STN stimulation on impulsivity have been limited by the confounding effects of therapeutic reduction of dopaminergic medication following STN-DBS.

Computational models provide an opportunity to overcome this limitation by simulating the effect of variations in stimulation and medication protocols individually which cannot be easily applied in human subjects. We hypothesized that electrode position and stimulation parameters affect the decision making ability of PD patients who received STN-DBS. We used a spiking network model of basal ganglia (BG; Mandali et al., 2015) to test the performance under various conditions on the standard gambling task, IGT (Evens et al., 2015). It is known that IGT captures certain impulsive features such as the risk taking ability (Fukui et al., 2005) and lack of premeditation (Zermatten et al., 2005) during decision making. PD patients are known to have poor IGT performance, especially during dopaminergic medication (Poletti et al., 2011; Gescheidt et al., 2012).

Simulating IGT requires learning, which was incorporated in the proposed model using Reinforcement learning (RL; Chakravarthy et al., 2010). RL describes the manner in which an agent learns stimulus-response (S-R) relations based on action outcomes: S-R pairs associated with rewarding outcomes are reinforced while those associated with punishment are attenuated (Dayan and Abbott, 2001). Experimental evidence shows that dopamine (DA) codes for reward prediction error or the temporal difference error term (“δ”) in RL (Niv, 2009).

Using the spiking network model of BG (described in Section Materials and Methods), we studied the performance of the model in IGT in normal, PD with and without medication [L-DOPA and Dopamine Agonists (DAA)] and STN-DBS conditions. Our results show that model in normal condition was able to learn from bad choices during the initial trials and improved its performance as the trials progressed. This was observed to be absent in PD with medication (both L-DOPA and DAA) condition.

We then studied the effect of STN stimulation alone on learning and performance by comparing it PD with and without medication. The simulation results show that during the initial trials, the stimulation current interferes with the learning which is reflected as poor performance. We then proceeded to study the factors of stimulation such as electrode position, amplitude of the current and spread that are specific to a patient. The simulation results indicate that electrode position played a significant role in altering performance in the model. We also observed that the model's performance improved for a narrow band of current amplitude.

Materials and Methods

Spiking Neuron Model of the Basal Ganglia

The network model of BG (Mandali et al., 2015; Figure 1A) was built using 2-variable Izhikevich spiking neurons (Izhikevich, 2003) where each nucleus was modeled as a 2D array (=50 × 50). Parameters for each of the nuclei [STN, Globus Pallidus internus (GPi) and externus (GPe) were chosen such that the model neurons display firing patterns (in terms of firing rate and firing patterns such as rebound firing) of their biological counterparts (Mandali et al., 2015). STN and GPe neurons were bi-directionally connected (Plenz and Kital, 1999) where projections from GPe (STN) are inhibitory (excitatory). Each GPi neuron received both glutamatergic projections from STN and GABAergic projections from D1R-expressing medium spiny neurons (MSNs) of the striatum (Gerfen and Surmeier, 2011). The final action selection was done at thalamus which was simulated as a race model (Vickers, 1970). The activities of both D1- and D2-receptor expressing, striatal MSNs that receive input from cortex (Tritsch and Sabatini, 2012) were modeled as Poisson spike trains. The full set of equations and module sizes related to the model are described in Appendix A and Table A.I (Datasheet in Supplementary Material). The input from cortex to STN, also known as hyper-direct pathway and the GABAergic projection from GPe to GPi, were not included in the model as their functional significance has not been fully explored. The list of acronyms and parameters used in the model are listed in Table 1 and Tables A.II, IV (Supplementary).

$\begin{array}{rc}\frac{d{v}_{ij}^x}{dt}=0.04{(v_{ij}^x)}^2+5v_{ij}^x-u_{ij}^x+140+I_{ij}^x+I_{ij}^{syn}& (1)\end{array}$

$\begin{array}{rc}\frac{d{u}_{ij}^x}{dt}=a(b{v}_{ij}^x-u_{ij}^x)& (2)\end{array}$

$\begin{array}{ll}if v_{ij}^x\ge v_{peak}\left\{\begin{array}{l}{v}_{ij}^x\leftarrow c\hfill \\ u_{ij}^x\leftarrow u_{ij}^x+d\hfill \end{array}\right\}& (3)\end{array}$

where, $v_{ij}^x$ = membrane potential, $u_{ij}^x$ = membrane recovery variable, $I_{ij}^{Syn}$ = total synaptic current received, $I_{ij}^x$ = external current applied to neuron “x” at location (i, j), v_peak = maximum voltage (+30 mv) with x being STN/GPe/GPi neuron.

FIGURE 1

Figure 1. (A) Shows the computational spiking basal ganglia model with key nuclei such as striatum (D1, D2), STN, GPe, GPi, and thalamus. Excitatory/inhibitory/modulatory glutamatergic/GABAergic/dopaminergic projections are shown by green/red/violet arrows. (B) Shows the BG model and the regions within each nuclei corresponding to the 4 decks are indicated.

TABLE 1

Table 1. Lists the Acronyms and parameters used in this article.

Behavioral Task-IGT

The task involved presentation of four decks of cards wherein each of the decks A/B/C/D is associated with a combination of reward and penalty. IGT was conducted for a total of 100 trials (5 bins of 20 trials each).The net outcome of a certain card selection (reward + penalty) in each trial was calculated. The probability and amount of penalty varied from deck to deck as explained in Table A.III (Datasheet in Supplementary Material). Over a few trials, one can observe that cards from the decks A and B (C and D) were disadvantageous (advantageous) as the corresponding expected value is negative (positive) The performance was measured as IGT total score (number of selections from “C,” “D”—number of selections from “A,” “B”)

Simulating IGT Using Spiking Neuron Network Model

Since IGT consists of 4 decks, each nucleus [STN/GPe/GPi/ Striatum (both D1 and D2)] in the network was divided equally into 4 quadrants, where each quadrant received input from one of the decks (Figure 1B).The expected value of each card was represented by the cortico-striatal weight which was modulated by DA term “δ.” The input to GPe and GPi (i.e., the output of D2 and D1 striatum) was modeled as Poisson spike train (Reti, 2015), whose frequency was proportional to the cortico-striatal weight ($w_{i,k}^{D1}$, $w_{i,k}^{D2}$) of the corresponding card (i) and trial (k). The striatal neuronal firing rate was restricted to 2–40 Hz as per experimental literature (Kravitz et al., 2010). Since DA is known to modulate plasticity in cortico-striatal conditions, the error term “δ” (in the model) was used to update the cortico-striatal synapses (Surmeier et al., 2007). DA also modulated the synaptic strengths within various BG nuclei such as STN (Cragg et al., 2004), GPe (Smith and Kieval, 2000).

Cortico Striatal Weight Update and Temporal Difference Error

Each deck was associated with 2 cortico-striatal weights ($w_{i,0}^{D1}$, $w_{i,0}^{D2}$) which were initialized with random values from a uniform distribution over (0, 1). The two cortico-striatal weights were trained as,

$\begin{array}{rc}\Delta w_{i,k+1}^{D1}=\eta {\delta }_k{x}_{i,k}^{inp}& (4)\end{array}$

$\begin{array}{rc}\Delta w_{i,k+1}^{D2}=-\eta {\delta }_k{x}_{i,k}^{inp}& (5)\end{array}$

The expected value (V_k) for kth trial was calculated as,

$\begin{array}{ll}{V}_{\mathrm{\text{k}}}={\displaystyle \sum _{i=1}^4}{w}_{i,k}^{D1}*x_{i,k}^{sel}& (6)\end{array}$

The reward (Re_k) for kth trial was calculated as,

$\begin{array}{ll}\mathrm{\text{R}}{\mathrm{\text{e}}}_{\mathrm{\text{k}}}={\displaystyle \sum _{i=1}^4}{r}_{i,k}*x_{i,k}^{sel}& (7)\end{array}$

The loss (L_k) for the kth trial was calculated as,

$\begin{array}{ll}{L}_{\mathrm{\text{k}}}={\displaystyle \sum _{i=1}^4}{l}_{i,k}*x_{i,k}^{sel}& (8)\end{array}$

The error (δ) for kth trial was defined as,

$\begin{array}{ll}{\delta }_k={\text{Re}}_k+{\mathit{\text{L}}}_k-{\mathit{\text{V}}}_k& (9)\end{array}$

where, $w_{i,k+1}^{D1}(w_{i,k+1}^{D2}),w_{i,k}^{D1}(w_{i,k}^{D2})$ were the cortico-striatal weights of D1 (D2) striatum for ith card in k+1th and kth trial, r_{i, k} and l_{i, k} were the reward and loss obtained for the selected ith card in kth trial, x^inp was the input binary vector representing the 4 decks, x^sel was the binary vector representing the selected card e.g., if the card “A” is selected x^sel = [1 0 0 0].

Simulating Untreated PD and Medically Treated PD Conditions

Bearing in mind that “δ” is similar to DA activity (Schultz, 1998; Niv, 2009) and there is loss of DA neurons in PD, we simulated PD condition by clamping the “δ” value (Equation 9) to a low limit (δ_lim) which resembles the untreated PD condition (Equation 10).

$\begin{array}{rc}{\delta }_{lim}=min(\delta ,D{A}_{ceil})& (10)\end{array}$

Where min (y,a) is defined as $\begin{array}{l}z=yify<a\\ aify>a\end{array}$ and DA_ceil is the upper limit of “δ.” Medically treated PD condition clinically involves external intake of dopamine precursors such as L-DOPA which was simulated by adding a positive “δ_med” term to the δ_lim (Mandali and Chakravarthy, 2016) (Equation 10).

$\begin{array}{ll}{\delta }_{new}={\delta }_{lim}+{\delta }_{med}& (11)\end{array}$

Another class of medication prescribed to PD patients is DAA, which has differential affinity for dopamine receptors. We simulated DAA with preferential affinity for D2 receptors, also known to be linked to impulsivity (MacMahon and Macphee, 2008). The δ_new in the Equation (11) was used to update only D2 cortico-striatal weight (w^D2) unlike for L-DOPA where both w^D1 and w^D2 were updated.

DBS Current

An external current which mimics the clinically delivered DBS current was applied to the STN neurons in the model. The parameters (frequency, pulse duration, and amplitude) of the stimulation current were chosen to be similar to the typical values used in a clinical setting (Garcia et al., 2005) [Appendix A (Datasheet in Supplementary Material)]. The spread of current over network of neurons spatially is known to follow a Gaussian distribution (Lukasiewicz and Werblin, 1990). Apart from that, Foutz and McIntyre (2010) have simulated various stimulation waveforms and observed that non-rectangular waveforms are more efficient (Foutz and McIntyre, 2010). The stimulation current was applied to the entire/part of STN module in the form of Gaussian distribution (Foutz and McIntyre, 2010). The mean of the Gaussian coincides with the lattice position (i_c, j_c) which was assumed to be the center of the electrode and extent of current spread was controlled by the variance parameter (σ).

$\begin{array}{ll}{I}_{ij}^{DBS}=A_{DBS}*e^{\frac{-({(i-i_c)}^2+{(j-j_c)}^2)}{{\sigma }^2}}& (12)\end{array}$

where $I_{ij}^{DBS}$ is the current received by the neuron at position (i, j) added to equation 1 of STN neurons, A_DBS is the amplitude of the current (pA), σ controls the current spread and (i_c, jc) is the mean/center point of the electrode. The effect of electrode position (i_c, j_c) and stimulation parameters A_DBS and σ on STN activity and on behavior was explored.

Values of the model parameters used for simulating various conditions (PD, medication and stimulation) are given in Appendix A (Datasheet in Supplementary Material).

Statistical Analysis

The IGT score obtained from each of the conditions [Healthy Controls (HC), untreated PD, medically treated PD, with and without STN stimulation] were compared using repeated measures of ANOVA. ANOVA (which stands for analysis of variance) is a popular statistical technique used to check if the means of two or more populations are equal and come from same distribution. It calculates the variance of means between the groups rather than intra group variance to determine whether the groups come from the same distribution.

Once the statistical test such as ANOVA has been performed, it is important to determine the underlying patterns in the data i.e., groups which could be representatives of different populations and the detects used are generally called as the a-posteriori tests. One such simple yet powerful test is the Bonferroni test which uses the “p-value” to determine the significance of the result. We have used the post-hoc Bonferroni test to study the effect of stimulation, medication on performance. All statistical tests were performed using IBM SPSS Statistics for Windows, Version 21.0, and Armonk, NY: IBM Corp., USA.

Results

De-Synchronization by DBS Current

The membrane potential of STN neurons PD untreated condition (Figures 2B,D) showed bursting activity and frequency content showed a peak at around 4 Hz with high synchrony level (=0.67; Mandali et al., 2015) which was absent in healthy condition (Figures 2A,D). On stimulating the STN neurons in PD condition, the peak around tremor frequency (=4 Hz) was significantly reduced (Figure 2D; P < 0.00001). Similarly the bursting activity in Figure 2B was overridden and suppressed by the stimulating current (Figure 2C). The synchrony level, R^sync (Equation A.13) in the presence of DBS current (Figure 2E) decreased from 0.67 (in PD condition and stimulation-OFF) to 0.42 (stimulation-ON; Figure 2F) but increased at higher current amplitudes.

FIGURE 2

Figure 2. The activity of STN neuron healthy, with and without DBS in PD. (A) Shows the activity of STN neurons in healthy condition, (B) shows the bursting activity of STN neurons in PD condition. (C) STN neurons resume to tonic firing after DBS, (D) the reduction in the frequency content at tremor frequency (4 Hz) in STN neurons in mentioned conditions, (E) shows the DBS current in biphasic mode (frequency = 130 Hz with amplitude of 200 pA), (F) shows the synchronization levels in STN neurons with increase in DBS current.

IGT-Healthy vs. PD Condition

Total IGT scores for HC, untreated PD, medically treated PD (L-DOPA and DAA) were significantly different [F_{(3, 36)} = 9.813, P < 0.0001] [Figure 3A, Table A.V (Datasheet in Supplementary Material)]. Post-hoc analysis indicated a significant difference between HC and the other three conditions: untreated PD (P = 0.003), medically treated PD L-DOPA (P = 0.001) and DAA (P < 0.0001). No statistically significant difference was observed between the two medically treated PD (L-DOPA and DAA) conditions.

FIGURE 3

Figure 3. Mean IGT score obtained from spiking model with standard error (SE) for four conditions, HC, PD OFF, PD ON (L-DOPA), and PD-ON (DAA) (A) Total IGT score (B) bin score.

In HC condition, the score was negative in the 1st bin, but changed to positive at the 2nd bin and continued to increase, a trend that was absent in both untreated and medically treated PD (both L-DOPA and DAA) conditions (Figure 3B). The mean scores [Table A.V Datasheet in Supplementary Material)] for 1st bin of HC, untreated PD, medically treated PD L-DOPA and DAA were similar (F = 0.684, P = 0.568). For both 2nd and 3rd bins medically treated PD condition performed worse than healthy controls [bin 2 showed significant difference between HC, medically treated PD L-DOPA and medically treated PD DAA (P < 0.005); bin 3: The scores were significantly different between HC, medically treated PD L-DOPA (P = 0.01) and medically treated PD DAA, P = 0.025)]. Though untreated PD had lower scores compared to HC and higher scores compared to treated PD in both the bins, these differences did not reach statistical significance. The mean scores for 4th bin among the four conditions were not significantly different. For the 5th bin, except HC, all other conditions showed poor performance (F = 8.744, P < 0.0001). The individual bin scores obtained in each of the above described condition are given in Table 2.

TABLE 2

Table 2. The mean IGT scores with standard deviation for HC, untreated PD, medically treated PD (L-DOPA and DAA) at total and individual bin levels and total IGT score.

The mean IGT score values (Figure 4A) for HC, obtained from experiment (Gescheidt et al., 2012) and simulation (Figure 4B) were not significantly different (P = 0.19). Similarly the mean IGT score values obtained from medically treated PD subjects (Gescheidt et al., 2012) and simulation were not significantly different (P = 0.74).

FIGURE 4

Figure 4. IGT performance results were redrawn from Gescheidt et al. (2012). (A) HC from experiment and simulation. (B) Medically treated PD patients from experiment and simulation (L-DOPA).

IGT-PD Condition with and without Stimulation

Total IGT score was negative for untreated PD, untreated PD with stimulation, medically treated PD and medically treated PD with stimulation (Figure 5A) with a significant difference among them [F_{(3, 36)} = 7.24, P = 0.001]. Post-hoc analysis revealed that medically treated PD (P = 0.001) and medically treated PD with stimulation (P = 0.004) had worse performance compared to untreated PD. No significant difference was observed between medically treated PD with and without stimulation.

FIGURE 5

Figure 5. The mean IGT score with SE obtained from spiking model calculated for four conditions; PD OFF, PD OFF+DBS, PD ON, PD ON+DBS (A) Total IGT score and (B) bin score.

A significant difference was observed among the 4 conditions (Figure 5B) for bin 1[F_{(3, 36)} = 3.24, P = 0.033] [Tables A.VI, A.VII (Datasheet in Supplementary Material)]. Post-hoc analysis indicated a significant difference only between untreated PD and untreated PD with stimulation (P = 0.039). For the 2nd bin [F_{(3, 36)} = 5.58, P = 0.003], medically treated PD performed worse compared to untreated PD (P < 0.008), no significant effect of stimulation was noted. No significant difference was observed for IGT score for the last three bins. The individual bin scores obtained in each of the above described condition are given in Table 3.

TABLE 3

Table 3. The mean IGT scores with standard deviation for untreated PD, medically treated PD (L-DOPA) with and without stimulation at total and individual bin levels.

The mean IGT score values obtained from medically treated PD (PD “ON”) subjects from experiment (Oyama et al., 2011) and simulation (Figures 6A,B) were statistically similar (P = 0.42). Similarly the mean IGT score values for PD with STN-DBS (DBS “ON”) experiment and simulation were similar (P = 0.55).

FIGURE 6

Figure 6. Plots of IGT scores calculated for PD “ON” and PD ON+DBS condition. The experimental results are redrawn from Oyama et al. (2011). (A) PD “ON” controls from experiment and simulation for 3 sessions (B) DBS subjects for baseline and DBS ON.

Effect of DBS Electrode Parameters on IGT Score

When the electrode position (positions explained in Figure 7 legend) was changed such that stimulation (for untreated PD) is given selectively to a part of the STN module corresponding to each deck in IGT, we observed a significant variation in the IGT score (Figure 7B) (P < 0.0001).On changing the spread of DBS current (Figure 7C), there was a trend toward better performance with lower radius of spread (σ = 10), which, however, did not reach statistical significance (P = 0.67). We also observed a lower IGT score at higher (=300 pA) and lower (=70 pA) currents compared to that obtained from optimal current (=100 pA) (P < 0.001) (Figure 8A). Interestingly, the current range where the highest IGT score was obtained was nearly the same range where the synchrony in STN neurons was observed to be lowest (Figure 8B). The underlying cause for such an effect (Figures 7D, 8A,B), was investigated by observing the spiking activity of STN for both optimal (=100 pA) and high (=300 pA) current scenarios. At optimal current levels, DBS desynchronized the activity of the STN neurons (Figure 2F) that received the stimulation, leading to the selection of the stimulated panel (Figure 2D; Figure A1; purple line) and a positive IGT score. On the contrary, at non-optimal (=300 pA) current amplitudes, the corresponding STN neuron activity is increased leading to de-selection of the panel “D” (Figure A1).

FIGURE 7

Figure 7. The IGT score. (A) Shows the STN network (=50 × 50) with quadrants that receive input from each of the corresponding decks (A–D) (B) The IGT score for three electrode positions [Position 1—in first quadrant with electrode center at lattice point (13,13), Position 2—center of the electrode at the lattice point (25,25), and Position 3- center of the electrode at the lattice point (38,38) in the fourth quadrant]. (C) For the electrode at position-2, the spread of the current (σ) was varied (D) the effect of DBS current amplitude (70, 100, and 300 pA) on IGT scores when the electrode is placed in position 3.

FIGURE 8

Figure 8. Shows (A) The IGT score obtained when the DBS current was increased from 0 (only medication) to up to 150 pA with frequency 130 Hz, biphasic mode. (B) The corresponding synchrony levels observed in STN neurons for that stimulation current.

Discussion

We built a computational spiking basal ganglia network model to understand the effects of STN stimulation on impulsivity. We first tested the spiking network model, by comparing its simulated activity with the known pathophysiological alterations in PD. It has been observed that desynchronized, irregular STN activity observed in HC changes into synchronized bursting behavior in PD (Wilson and Bevan, 2011), which is also observed in STN neurons of MPTP treated primates (Bergman et al., 1994). This bursting oscillatory activity (Plenz and Kital, 1999) was also observed in our spiking model which correlated with tremor frequency, suggesting that the spiking model has the ability to reflect pathological STN activity.

Effect of PD and Dopaminergic Therapy on IGT Performance

Overall IGT performance was poor in untreated and medically treated PD conditions compared to HC. Medically treated PD condition did worse than HC and untreated PD, regardless of the type of medication used. Analysis of bin scores revealed that learning of the task was poor in all PD conditions compared to HC. Medically treated PD condition resulted in lower bin scores from the 2nd bin onwards, suggesting significantly impaired learning of the task, while in untreated PD; a significantly low bin score compared to HC was seen only in the last bin. The model in treated PD condition does not learn from its action outcomes (rewards/punishments) and wanders among the decks, which is reflected in the negative IGT score (Figure 3). This behavior is similar to that previously reported in a probabilistic action selection paradigm where PD “ON” subjects fail to avoid punitive choices (Frank et al., 2007). Physiologically this behavior is attributed to excess DA levels in striatum (Frank et al., 2007). In the model, striatal weights were positively updated even in punishment situation due to dopaminergic medication (δ_med), leading to the selection of wrong choice. Clinical studies have identified dopamine agonists to be associated with higher risk of impulsivity, mediated through their D3 receptor affinity. We did not specifically model the D3 receptor activity and did not observe a higher risk with DAA (which selectively increased the D2 weight) compared to L-DOPA in our model.

Effect of STN Stimulation

Stimulation applied to the entire STN module did not result in a significant deterioration of overall IGT performance (Figure 5A). However, stimulation significantly impaired performance in the 1st bin when applied to untreated PD condition (Figure 5B). It is during these early trials that most of the learning regarding deck's reward pattern happens and this learnt information is used for future card selection. Stimulation seems to affect this learning ability, making the model performance worse as trials progress. Similar behavior is also reported in clinical experiments, where it was observed that PD patients with stimulation tend to overestimate their choices (Florin et al., 2013). No significant change was observed between medically treated PD with and without stimulation. This could be due to an overriding effect of dopaminergic medication over stimulation.

A few aspects of DBS that are specific to a PD patient who receives stimulation are the active contact point in the electrode, the amplitude of the current and the spread due to the current. Keeping this mind, we first changed the position of electrode within the STN nucleus (Figures 7A,B) and observed a significant change in IGT score. Physiologically, it might be possible that stimulation of different active points (which are 0.5–1 mm apart in the electrode) could lead to differential activation of neurons. This activation not only leads to overall difference in the current spread (McIntyre et al., 2004) but also in the activity of neurons that receive input from a different sources. This controlled activation might eventually changes the behavior (Witt et al., 2013).

Apart from position, another DBS parameter thought to significantly influence cognition is current amplitude. Various computational and experimental studies showed that the volume of tissue activated is dependent on the stimulating current amplitude (McIntyre et al., 2004; Arle et al., 2007; Yousif et al., 2010). We observed that a high stimulation current can increase the firing rate of STN neurons that received the stimulation whereas sufficiently low amplitude current would just desynchronize the activity. This change in STN, in terms of spiking activity during optimal and non-optimal (Figure A1; for a fixed position) stimulation currents might be the reason behind conflicting results observed at behavioral level (Figure 7D). For instance, de-selection of panel “D” increases probability of selection of other panels. In Figure A1, the selected panel was “A” (blue line) is a highly rewarding panel when viewed at a shorter time scale but punitive in long term. The model selects this panel due to its inability to learn the punishments associated with it. This selection of other panels (A and B are disadvantageous, and only C is rewarding) gave rise to a negative IGT score with high current.

With the above results one can consider the possibility that stimulation current when applied to the corresponding topographical areas of the panels within STN might lead to inhibition/facilitation of the corresponding panel selection depending on the current amplitude. To relate the above results physiologically, we suggest a role for the parallel functional loops within BG nuclei (Alexander and Crutcher, 1990) (motor, cognitive, and the limbic loops) and topographical mapping in impulsive behavior. A coarse functional organization is also observed within STN nucleus with motor and cognitive areas being adjacent to each other (Temel et al., 2005). We suspect that the highly variable cognitive outcomes in experimental studies could be correlated to electrode position and current spread (Figures 7B–D, 8A). These results are similar to those observed in the clinical study where a decrease in performance (hit rate) was observed when the position of the electrode was changed (Hershey et al., 2010).

Limitations of our model include the connectivity pattern within BG nuclei (GPe-GPi connection and hyper direct pathway is not included) and neuronal number. More elaborate modeling studies are required to further explore the effect of electrode position and stimulus waveform on motor and cognitive aspects of the PD patients. Although the human STN is known to be organized into motor and cognitive sub territories, a further topographical division into separate areas for various choices is yet unconfirmed. Nevertheless, our model reflects the pathophysiology of STN in PD and predicts behavioral changes similar to clinical data. Our results yield valuable information on the effect of electrode position and current amplitude on behavioral and cognitive outcomes of STN stimulation in PD that may help in the development of optimal stimulation protocols in a clinical setting.

Author Contributions

AM: Designed the study, built the computational model and performed the simulations, Manuscript Preparation and Analysis. VSC: Designed the study, built the computational model and performed the simulations, Manuscript Preparation and Analysis. RR: Designed the study, Manuscript Preparation and Analysis. AK: Designed the study, Manuscript Preparation and Analysis. SS: Analysis.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: http://journal.frontiersin.org/article/10.3389/fphys.2016.00585/full#supplementary-material

References

Alexander, G. E., and Crutcher, M. D. (1990). Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends Neurosci. 13, 266–271. doi: 10.1016/0166-2236(90)90107-L

PubMed Abstract | CrossRef Full Text | Google Scholar

Arle, J., Mei, L., and Shils, J. (2007). Modeling Parkinsonian circuitry and the DBS electrode. Stereotact. Funct. Neurosurg. 86, 1–15. doi: 10.1159/000108584

PubMed Abstract | CrossRef Full Text | Google Scholar

Benabid, A. L. (2003). Deep brain stimulation for Parkinson's disease. Curr. Opin. Neurobiol. 13, 696–706. doi: 10.1016/j.conb.2003.11.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Bergman, H., Wichmann, T., Karmon, B., and Delong, M. (1994). The primate subthalamic nucleus. II. Neuronal activity in the MPTP model of parkinsonism. J. Neurophysiol. 72, 507–520.

PubMed Abstract | Google Scholar

Castrioto, A., Funkiewiez, A., Debû, B., Cools, R., Lhommée, E., Ardouin, C., et al. (2015). Iowa gambling task impairment in Parkinson's disease can be normalised by reduction of dopaminergic medication after subthalamic stimulation. J. Neurol. Neurosurg. Psychiatry 86, 186–190. doi: 10.1136/jnnp-2013-307146

PubMed Abstract | CrossRef Full Text | Google Scholar

Chakravarthy, V., Joseph, D., and Bapi, R. S. (2010). What do the basal ganglia do? A modeling perspective. Biol. Cybern. 103, 237–253. doi: 10.1007/s00422-010-0401-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Combs, H. L., Folley, B. S., Berry, D. T., Segerstrom, S. C., Han, D. Y., Anderson-Mooney, A. J., et al. (2015). Cognition and depression following deep brain stimulation of the subthalamic nucleus and globus pallidus pars internus in Parkinson's disease: a meta-analysis. Neuropsychol. Rev. 25, 439–454. doi: 10.1007/s11065-015-9302-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Cragg, S. J., Baufreton, J., Xue, Y., Bolam, J. P., and Bevan, M. D. (2004). Synaptic release of dopamine in the subthalamic nucleus. Eur. J. Neurosci. 20, 1788–1802. doi: 10.1111/j.1460-9568.2004.03629.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Dayan, P., and Abbott, L. F. (2001). Theoretical Neuroscience. Cambridge, MA: MIT Press.

Google Scholar

Evens, R., Stankevich, Y., Dshemuchadse, M., Storch, A., Wolz, M., Reichmann, H., et al. (2015). The impact of Parkinson's disease and subthalamic deep brain stimulation on reward processing. Neuropsychologia. 75, 11–19. doi: 10.1016/j.neuropsychologia.2015.05.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Florin, E., Müller, D., Pfeifer, J., Barbe, M. T., Fink, G. R., and Timmermann, L. (2013). Subthalamic stimulation modulates self-estimation of patients with Parkinson's disease and induces risk-seeking behaviour. Brain. 136, 3271–3281. doi: 10.1093/brain/awt241

PubMed Abstract | CrossRef Full Text | Google Scholar

Foutz, T. J., and McIntyre, C. C. (2010). Evaluation of novel stimulus waveforms for deep brain stimulation. J. Neural Eng. 7:066008. doi: 10.1088/1741-2560/7/6/066008

PubMed Abstract | CrossRef Full Text | Google Scholar

Frank, M. J., Samanta, J., Moustafa, A. A., and Sherman, S. J. (2007). Hold your horses: impulsivity, deep brain stimulation, and medication in parkinsonism. Science 318, 1309–1312. doi: 10.1126/science.1146157

PubMed Abstract | CrossRef Full Text | Google Scholar

Fukui, H., Murai, T., Fukuyama, H., Hayashi, T., and Hanakawa, T. (2005). Functional activity related to risk anticipation during performance of the Iowa Gambling Task. Neuroimage 24, 253–259. doi: 10.1016/j.neuroimage.2004.08.028

PubMed Abstract | CrossRef Full Text | Google Scholar

Garcia, L., D'alessandro, G., Bioulac, B., and Hammond, C. (2005). High-frequency stimulation in Parkinson's disease: more or less? Trends Neurosci. 28, 209–216. doi: 10.1016/j.tins.2005.02.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Gerfen, C. R., and Surmeier, D. J. (2011). Modulation of striatal projection systems by dopamine. Annu. Rev. Neurosci. 34, 441. doi: 10.1146/annurev-neuro-061010-113641

PubMed Abstract | CrossRef Full Text | Google Scholar

Gescheidt, T., Czekóová, K., Urbánek, T., Mareček, R., Mikl, M., Kubíková, R., et al. (2012). Iowa Gambling Task in patients with early-onset Parkinson's disease: strategy analysis. Neurol. Sci. 33, 1329–1335. doi: 10.1007/s10072-012-1086-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Hershey, T., Campbell, M. C., Videen, T. O., Lugar, H. M., Weaver, P. M., Hartlein, J., et al. (2010). Mapping Go–No-Go performance within the subthalamic nucleus region. Brain 133, 3625–3634. doi: 10.1093/brain/awq256

PubMed Abstract | CrossRef Full Text | Google Scholar

Hershey, T., Revilla, F. J., Wernle, A., Gibson, P. S., Dowling, J. L., and Perlmutter, J. S. (2004). Stimulation of STN impairs aspects of cognitive control in PD. Neurology 62, 1110–1114. doi: 10.1212/01.WNL.0000118202.19098.10

PubMed Abstract | CrossRef Full Text | Google Scholar

Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Trans. Neural Netw. 14, 1569–1572. doi: 10.1109/TNN.2003.820440

PubMed Abstract | CrossRef Full Text | Google Scholar

Kravitz, A. V., Freeze, B. S., Parker, P. R., Kay, K., Thwin, M. T., Deisseroth, K., et al. (2010). Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature 466, 622–626. doi: 10.1038/nature09159

PubMed Abstract | CrossRef Full Text | Google Scholar

Kühn, A. A., Kupsch, A., Schneider, G. H., and Brown, P. (2006). Reduction in subthalamic 8–35 Hz oscillatory activity correlates with clinical improvement in Parkinson's disease. Eur. J. Neurosci. 23, 1956–1960. doi: 10.1111/j.1460-9568.2006.04717.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Lukasiewicz, P. D., and Werblin, F. S. (1990). The spatial distribution of excitatory and inhibitory inputs to ganglion cell dendrites in the tiger salamander retina. J. Neurosci. 10, 210–221.

PubMed Abstract | Google Scholar

MacMahon, D. G., and Macphee, G. J. (2008). Dopamine agonists and impulse control disorders in Parkinson's disease. Prog. Neurol. Psychiatry 12, 5–9. doi: 10.1002/pnp.100

CrossRef Full Text | Google Scholar

Mandali, A., and Chakravarthy, V. (2016). Probing the role of medication, DBS electrode position and antidromic activation on impulsivity using a computational model of Basal Ganglia. Front. Hum. Neurosci. 10:450. doi: 10.3389/fnhum.2016.00450

PubMed Abstract | CrossRef Full Text | Google Scholar

Mandali, A., Rengaswamy, M., Chakravarthy, V. S., and Moustafa, A. A. (2015). A spiking Basal Ganglia model of synchrony, exploration and decision making. Front. Neurosci. 9:191. doi: 10.3389/fnins.2015.00191

PubMed Abstract | CrossRef Full Text | Google Scholar

McIntyre, C. C., Mori, S., Sherman, D. L., Thakor, N. V., and Vitek, J. L. (2004). Electric field and stimulating influence generated by deep brain stimulation of the subthalamic nucleus. Clin. Neurophysiol. 115, 589–595. doi: 10.1016/j.clinph.2003.10.033

PubMed Abstract | CrossRef Full Text | Google Scholar

Niv, Y. (2009). Reinforcement learning in the brain. J. Math. Psychol. 53, 139–154. doi: 10.1016/j.jmp.2008.12.005

CrossRef Full Text | Google Scholar

Oyama, G., Shimo, Y., Natori, S., Nakajima, M., Ishii, H., Arai, H., et al. (2011). Acute effects of bilateral subthalamic stimulation on decision-making in Parkinson's disease. Parkinsonism Relat. Disord. 17, 189–193.

PubMed Abstract | Google Scholar

Plenz, D., and Kital, S. T. (1999). A basal ganglia pacemaker formed by the subthalamic nucleus and external globus pallidus. Nature 400, 677–682. doi: 10.1038/23281

PubMed Abstract | CrossRef Full Text | Google Scholar

Poletti, M., Cavedini, P., and Bonuccelli, U. (2011). Iowa gambling task in Parkinson's disease. J. Clin. Exp. Neuropsychol. 33, 395–409. doi: 10.1080/13803395.2010.524150

PubMed Abstract | CrossRef Full Text | Google Scholar

Reti, I. (2015). Brain Stimulation: Methodologies and Interventions. Hoboken, NJ: John Wiley & Sons.

Google Scholar

Rosa, M., Giannicola, G., Marceglia, S., Fumagalli, M., Barbieri, S., and Priori, A. (2012). “Emerging horizons in neuromodulation new Frontiers in brain and spine stimulation,” in International Review of Neurobiology, eds C. Hamani and E. Moro (San Diego, CA: Elsevier), 2–380.

Schultz, W. (1998). Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1–27.

PubMed Abstract | Google Scholar

Smeding, H. M. M., Goudriaan, A. E., Foncke, E. M. J., Schuurman, P. R., Speelman, J. D., and Schmand, B. (2007). Pathological gambling after bilateral subthalamic nucleus stimulation in Parkinson disease. J. Neurol. Neurosurg. Psychiatry 78, 517–519. doi: 10.1136/jnnp.2006.102061

PubMed Abstract | CrossRef Full Text | Google Scholar

Smith, Y., and Kieval, J. Z. (2000). Anatomy of the dopamine system in the basal ganglia. Trends Neurosci. 23, S28–S33. doi: 10.1016/S1471-1931(00)00023-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Sudhyadhom, A., Bova, F. J., Foote, K. D., Rosado, C. A., Kirsch-Darrow, L., and Okun, M. S. (2007). Limbic, associative, and motor territories within the targets for deep brain stimulation: potential clinical implications. Curr. Neurol. Neurosci. Rep. 7, 278–289. doi: 10.1007/s11910-007-0043-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Surmeier, D. J., Ding, J., Day, M., Wang, Z., and Shen, W. (2007). D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons. Trends Neurosci. 30, 228–235. doi: 10.1016/j.tins.2007.03.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Temel, Y., Blokland, A., Steinbusch, H. W., and Visser-Vandewalle, V. (2005). The functional role of the subthalamic nucleus in cognitive and limbic circuits. Prog. Neurobiol. 76, 393–413. doi: 10.1016/j.pneurobio.2005.09.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Tritsch, N. X., and Sabatini, B. L. (2012). Dopaminergic modulation of synaptic transmission in cortex and striatum. Neuron 76, 33–50. doi: 10.1016/j.neuron.2012.09.023

PubMed Abstract | CrossRef Full Text | Google Scholar

Vickers, D. (1970). Evidence for an accumulator model of psychophysical discrimination. Ergonomics 13, 37–58. doi: 10.1080/00140137008931117

PubMed Abstract | CrossRef Full Text | Google Scholar

Wilson, C. J., and Bevan, M. D. (2011). Intrinsic dynamics and synaptic inputs control the activity patterns of subthalamic nucleus neurons in health and in Parkinson's disease. Neuroscience 198, 54–68. doi: 10.1016/j.neuroscience.2011.06.049

PubMed Abstract | CrossRef Full Text | Google Scholar

Witt, K., Granert, O., Daniels, C., Volkmann, J., Falk, D., van Eimeren, T., et al. (2013). Relation of lead trajectory and electrode position to neuropsychological outcomes of subthalamic neurostimulation in Parkinson's disease: results from a randomized trial. Brain 136, 2109–2119. doi: 10.1093/brain/awt151

PubMed Abstract | CrossRef Full Text | Google Scholar

York, M. K., Wilde, E. A., Simpson, R., and Jankovic, J. (2009). Relationship between neuropsychological outcome and DBS surgical trajectory and electrode location. J. Neurol. Sci. 287, 159–171. doi: 10.1016/j.jns.2009.08.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Yousif, N., Purswani, N., Bayford, R., Nandi, D., Bain, P., and Liu, X. (2010). Evaluating the impact of the deep brain stimulation induced electric field on subthalamic neurons: a computational modelling study. J. Neurosci. Methods 188, 105–112. doi: 10.1016/j.jneumeth.2010.01.026

PubMed Abstract | CrossRef Full Text | Google Scholar

Zermatten, A., Van Der Linden, M., d'acremont, M., Jermann, F., and Bechara, A. (2005). Impulsivity and decision making. J. Nerv. Ment. Dis. 193, 647–650. doi: 10.1097/01.nmd.0000180777.41295.65

PubMed Abstract | CrossRef Full Text | Google Scholar