Fast Dynamical Coupling Enhances Frequency Adaptation of Oscillators for Robotic Locomotion Control

1. Introduction

Rhythmic processes are of central importance for many aspects of biological life (Winfree, 1967; Barkai and Leibler, 2000; Goldbeter et al., 2012). Examples include the cardiac rhythm, various circadian rhythms and, in particular, all forms of biological locomotion like walking, flying or swimming. The latter are controlled by specific neural circuits, so called central pattern generators (CPGs) (Hooper, 2001; Marder and Bucher, 2001). Theoretical models of CPGs range from detailed biophysical models (Hellgren and Grillner, 1992) to pure mathematical oscillators (Matsuoka, 1985). In general, CPGs can be described as nonlinear oscillators which have been used in numerous applications for different variants of robotic control problems (Nakamura et al., 2007; Ijspeert, 2008; Pinto et al., 2012; Nassour et al., 2014; Santos et al., 2017). For instance, compared to purely reflexive control schemes (Foth and Bässler, 1985; Cruse et al., 1995), oscillator-controlled robots enable more stable and robust locomotion (Kimura et al., 2001; Righetti and Ijspeert, 2008).

CPGs do not require any external input or feedback to produce basic rhythmic activity. However, they still require feedback signals to adapt and tune their produced activity, for instance its frequency. For the theoretical concept of nonlinear oscillators, a universal mechanism to adapt the intrinsic frequency of an oscillator according to the frequency of an external periodic signal, which is coupled to the oscillator, was formulated by Righetti et al. (2006). This frequency adaptation schema is applicable to many different types of oscillators. In contrast to the well-known phenomenon of entrainment, which is a purely reactive mechanism with only transient effect on the oscillatory system (Buchli et al., 2006), the frequency adaptation schema modifies the intrinsic frequency of the system permanently. Oscillators with this schema are commonly called adaptive frequency oscillators (AFOs). Several applications of AFOs have been proposed including adaptive control of compliant robots (Righetti et al., 2009), pendulum swing-up problems (Spong, 1995; Furuta, 2003), understanding, simulation and support of human locomotion (Ronsse et al., 2011a; Tropea et al., 2015; Santos et al., 2017), mimicking of fish swimming (Wang et al., 2013), frequency analysis of an input signal (Buchli et al., 2008), and construction of limit cycles of arbitrary shape (Righetti et al., 2009). However, all of these applications suffer from significantly long adaptation times.

For a given oscillatory system, the dynamics of a standard AFO is determined by only two parameters: the strength of the coupling of the external signal to the oscillator and the learning rate of the parameter determining the intrinsic frequency of the system. Here, we show that, when choosing these two parameters, one has to make a compromise between speed and precision of the resulting adaption dynamics. Furthermore, we demonstrate that the optimal parameters for a certain balance of speed and precision strongly depend on the initial intrinsic frequency of the oscillator and on the target frequency, i.e., the frequency of the external signal. As a result, situation-specific fine-tuning of the parameters is necessary.

In contrast, we propose an extension of the standard frequency adaptation mechanism which provides both fast as well as precise adaptation for a wide range of initial intrinsic and target frequencies without the need for parameter fine tuning. In the following, we call this mechanism “Adaptation through Fast Dynamical Coupling” (AFDC). It is based on dynamically adapting the coupling strength of the external signal. If the difference between the current intrinsic frequency and the target frequency is high, the coupling strength is increased in order to accelerate the adaptation. If the difference between the current intrinsic frequency and the target frequency becomes small, the coupling strength is reduced to increase the precision of the adaptation. This process is autonomous and can be integrated into the dynamical equations of the system. Neither the current intrinsic nor the target frequency need to be explicitly available as the mechanism solely relies on signal correlations. We compare the adaptation processes obtained by regular AFOs with those obtained with the new AFDC mechanism by means of quantitative measures of speed and precision of the adaptation. We find that the AFDC mechanism clearly outperforms standard AFOs within the tested frequency interval covering two orders of magnitudes.

2. Results

2.1. Standard Adaptive Frequency Oscillator

In very general terms, an oscillator is an autonomous dynamical system with at least one limit cycle attractor (Buchli et al., 2006). Naturally, every two-dimensional oscillatory system (x, y) can be expressed as a system of two equations $\dot{x}$(t) = g_x(x(t), y(t), θ) and $\dot{y}$(t) = g_y(x(t), y(t), θ) where the functions g_x and g_y define the dynamics of the system. We require that these two functions do not only depend on the state variables x and y but also explicitly on a variable θ which determines the intrinsic oscillation frequency f of the system. The function f(θ) may be of an arbitrary shape and in many cases is not explicitly known. We only assume it to be monotonic. The system can be transformed into an adaptive frequency oscillator (AFO) by coupling it to an external signal F(t):

$\begin{array}{l}\dot{x}(t)=g_x(x(t),y(t),\theta (t))+ϵF(t)\\ \dot{y}(t)=g_y(x(t),y(t),\theta (t)).& (1)\end{array}$

Here, ϵ denotes the coupling strength. Furthermore, additional dynamics of the θ-variable are introduced (Righetti et al., 2006):

$\begin{array}{ll}\dot{\theta }(t)=\pm \eta F(t)\frac{y(t)}{\sqrt{x{(t)}^2+y{(t)}^2}}.& (2)\end{array}$

with a learning rate η. The sign on the right-hand side depends on the direction of oscillation of the actual oscillatory system in the phase space. Note that in the original publication (Righetti et al., 2006), always η = ϵ is chosen as it emerges naturally when deriving the adaptation rule from analyzing the effect of the periodic external signal F on the phase velocity of the oscillator (Righetti et al., 2006). Apart from this, however, there is no a priori reason why this choice should provide optimal adaptation results. It has been shown that, using this rule, a wide range of oscillators can adapt their intrinsic frequencies to the frequency of basically any external periodic signal F(t). In this contribution, we consider the Hopf oscillator (Figure 1A), which possesses a harmonic limit cycle, and the Van der Pol oscillator (Van der Pol, 1920) (Figure 1B), which, depending on the choice of parameters, exhibits highly non-harmonic oscillations.

FIGURE 1

Figure 1. Adaptation of two standard adaptive frequency oscillators. The upper panels show the time course of the frequency determining parameter θ. The time during which the external signal is applied to the system is indicated by the yellow shaded area. The dashed horizontal lines indicate the values θ₀ and θ_ext corresponding to the initial intrinsic frequency f₀ and the target frequency f_ext of the external signal, respectively. The panels below show the oscillating state variables x and y and the external signal F at different short time windows during the adaptation process. In both cases, the initial intrinsic frequency of the oscillator is f₀ = 4.0 and the external signal is a sine wave with unit amplitude and frequency f_ext = 2.0. (A) Adaptive frequency Hopf oscillator with μ = 1.0 and ϵ = η = 1.0 (see Methods). The initial value of the parameter θ is given by θ₀ = 2πf₀ ≈ 25.1. Accordingly, the value corresponding to the frequency of the external signal is θ_ext = 2πf_ext ≈ 12.6. The external signal is applied for 100 ≤ t < 700. (B) Adaptive frequency Van der Pol oscillator with μ = 100.0 and ϵ = η = 0.7 (see Methods). The values of the parameter θ corresponding to f₀ and f_ext are θ₀ ≈ 34.8 and θ_ext ≈ 22.0 (see Methods). The external signal is applied for 100 ≤ t < 500.

For analyzing a given adaptation process, we start with an oscillator with an initial frequency variable θ₀ corresponding to an initial intrinsic frequency f₀ = f(θ₀). Here, the function f(θ) is not explicitly known but can be numerically approximated. We denote the target frequency, i.e., the frequency of the external signal, by f_ext. Furthermore, we define the target value θ_ext as the value of θ such that f_ext = f(θ_ext) for the given oscillator. The frequency variable θ is not modified by the adaptation rule (Equation 2) as long as the external signal F is zero (t < 100 in Figure 1A). After the onset of the external signal, θ is slowly adapted toward the target value θ_ext (100 < t < 700 in Figure 1A). The adaptation rate increases as θ gets closer to θ_ext. The final adaptation phase is typically characterized by a small θ-overshoot before it converges toward a quasi-constant state with only small periodic fluctuations (600 < t < 700 in Figure 1A). Now, when removing the external signal, i.e., setting F = 0, the oscillator maintains oscillations at the adapted frequency (t > 700 in Figure 1A). Note that it is not guaranteed that the finally reached value of θ corresponds exactly to θ_ext. In contrast, in some cases significant deviations can be observed (Figure 1B). As it turns out, reducing this deviation is only possible when accepting longer adaptation times.

2.1.1. Speed vs. Precision Trade-Off

In many applications, for instance in robotic systems, it is usually desired to have systems that are able to adapt to new situations or circumstances quickly. In contrast, AFOs with the usual choice of parameters require many periods of oscillations to complete a given adaptation process. The convergence time of the adaptation process, i.e., the time between the onset of the external signal and the quasi-convergence of the frequency parameter θ of the oscillator, can be adjusted by manipulating the coupling strength ϵ in Equation (1) or the learning rate η in Equation (2) (Figure 2). However, increasing ϵ or η does not only increase the speed of the frequency adaptation but also increases the general influence of the external signal on the oscillatory system. As a result, the dynamics of the parameter θ, once it has converged to a quasi-stable state, is affected as well (Figure 3). On the one hand, high learning rates η lead to increased fluctuations of the parameter θ in the finally reached state. On the other hand, higher values of ϵ result in a higher offset of the finally reached mean value $\stackrel{-}{\theta }$ from the value θ_ext. Therefore, shorter convergence times in the standard AFO systems go hand in hand with a loss of precision. Naturally, this trade-off complicates real-world applications of the mechanism.

FIGURE 2

Figure 2. Influence of the coupling strength ϵ and the learning rate η on the speed of adaptation of standard adaptive frequency oscillators. The yellow shaded area indicates the time during which the external signal is applied. In all cases, the initial intrinsic frequency of the oscillator is f₀ = 4.0 and the frequency of the external unit sine-wave signal is f_ext = 2.0. For the adaptive Hopf oscillator, we choose μ = 1.0. For the adaptive van-der-Pol oscillator, we choose μ = 100.0 (see Methods).

FIGURE 3

Figure 3. Quantitative measures to capture the quality of an adaptation process. Shown is the time course of the intrinsic frequency of an adaptive frequency oscillator during the adaptation to an external periodic signal with high coupling constant ϵ and high learning rate η. The yellow shaded area indicates the time during which the external signal is applied. The inset shows a close up of the data in the indicated area. We introduce three measures to quantify the quality of a given frequency adaption process. The convergence time Δ is the time interval between the onset of the external signal at time t₀ and the last deviation of the intrinsic frequency of the oscillator of more than 5% (orange horizontal lines) from the finally reached average intrinsic frequency $\stackrel{-}{f}$. The frequency offset δ measures the difference between the final average intrinsic frequency $\stackrel{-}{f}$ and the target frequency of the external signal f_ext. In order to also capture the periodic fluctuations of the intrinsic frequency from the average value $\stackrel{-}{f}$, we additionally introduce the final frequency fluctuation σ given by the standard deviation of the oscillations of the intrinsic frequency f in the finally reached state (area shaded in light red in the inset). The shown time course of the intrinsic frequency is taken from an adaptive frequency Hopf oscillator with μ = 1.0, ϵ = 5.0, η = 5.0, and f₀ = 2.0 adapting to an external unit sine-wave signal with frequency f_ext = 1.0.

2.1.2. Quantitative Adaptation Quality Measures

In order to quantitatively capture the trade-off between speed and precision, we introduce three measures characterizing the quality of a given adaptation process (Figure 3). As already discussed, in many applications fast adaptation is desired. This is captured by the convergence time Δ which measures the time interval between the onset of the external signal and the last deviation of the intrinsic frequency f of the system (determined by θ) of more than 5% (10% for the Van der Pol oscillator) from the finally reached mean value $\stackrel{-}{f}$. The precision of the adaptation, in turn, is reflected by two measures. First, the intrinsic frequency to which the system converges should be as close as possible to the frequency of the external signal. This is measured by the frequency offset δ which is given by the offset of the finally reached mean value of the intrinsic frequency from the frequency of the external signal. Second, the fluctuations of the intrinsic frequency around its mean value should be low as otherwise the value of the intrinsic frequency when switching off the external signal depends on the exact point of time of this event. The magnitude of these fluctuations is measured by σ which equals the standard deviation of the intrinsic frequency f in the converged state.

To allow interpretation of these measures independently from the chosen internal and external frequencies, we introduce relative measures scaled by the frequency f_ext or the cycle duration $f_{\mathrm{\text{ext}}}^{-1}$ of the external signal, respectively: $\tilde{\Delta }=\Delta /f_{\mathrm{\text{ext}}}^{-1}$, $\tilde{\delta }=\delta /f_{\mathrm{\text{ext}}}$ and $\tilde{\sigma }=\sigma /f_{\mathrm{\text{ext}}}$. In addition, we define a quality index Q combining these three relative measures into a single scalar value:

$\begin{array}{ll}Q=\text{max }\left(1-\frac{\tilde{\Delta }}{{\tilde{\Delta }}_{\text{max}}}-\frac{\left|\tilde{\delta }\right|}{{\tilde{\delta }}_{\text{max}}}-\frac{\tilde{\sigma }}{{\tilde{\sigma }}_{\text{max}}},0\right).& (3)\end{array}$

Here, ${\tilde{\Delta }}_{\mathrm{\text{max}}}$, ${\tilde{\delta }}_{\mathrm{\text{max}}}$, and ${\tilde{\sigma }}_{\mathrm{\text{max}}}$ are the maximum values of the respective measures which we allow for a reasonably good adaptation process. Accordingly, a Q value close to 1 corresponds to a very fast as well as very precise adaptation process. A value of Q = 0, in contrast, indicates that $\tilde{\Delta }>{\tilde{\Delta }}_{\mathrm{\text{max}}}$, $\tilde{\delta }>{\tilde{\delta }}_{\mathrm{\text{max}}}$, $\tilde{\sigma }>{\tilde{\sigma }}_{\mathrm{\text{max}}}$ or the weighted sum (Equation 3) of the individual measures is larger than 1. In the following, if not stated otherwise, we use ${\tilde{\Delta }}_{\mathrm{\text{max}}}=100$, ${\tilde{\delta }}_{\mathrm{\text{max}}}=0.05$ and ${\tilde{\sigma }}_{\mathrm{\text{max}}}=0.05$.

2.1.3. Finding Optimal Parameters

For an easy application of an adaptive oscillator in a given setup, no fine tuning of the system parameters for the specific application context should be necessary. It is therefore necessary to find a system which is able to adapt its intrinsic frequency to a wide range of external frequencies without the need for any parameter adaptation other than the one of the frequency determining parameter θ. It turns out, however, that already for the comparable simple case of the harmonic Hopf oscillator, the range of frequencies for which a given set of parameters allows fast as well as precise adaptations is very limited (Figure 4). Higher values of ϵ and η increase the intervals of initial intrinsic frequencies f₀ and external frequencies f_ext for which fast adaptation is achieved (left column in Figure 4). In contrast, small frequency offsets $\tilde{\delta }$ are achieved only for small values of the coupling strength ϵ (second column in Figure 4) and small values of the learning rate η enable small fluctuations as measured by σ (third column in Figure 4). The compilation of these observations is reflected by only small intervals of initial intrinsic and external frequencies for which the quality index Q attains non-zero values (right column in Figure 4).

FIGURE 4

Figure 4. Adaptation quality measures of the adaptive frequency Hopf oscillator in the (f₀, f_ext) frequency space for different values of the coupling strength ϵ and the learning rate η. For every given (ϵ, η)-parameter pair, from left to right, the relative convergence time $\tilde{\Delta }$, the relative final frequency offset $\tilde{\delta }$, the final relative frequency fluctuation $\tilde{\sigma }$, and the combined quality measure Q are shown in the plane spanned by the initial intrinsic frequency f₀ and the frequency of the external unit sine-wave signal f_ext. As the convergence time is defined as the time difference between the onset of the external signal and the last point of time of more than 5% deviation of the intrinsic frequency from the final average, for high values of $\tilde{\sigma }$, the convergence time cannot be reasonably determined, i.e., takes very high values. For the same reason, even on the diagonal f₀ = f_ext, high convergence times are measured for low values of f_ext.

Trying to find parameters that allow fast and precise adaptation for a range of initial intrinsic and external target frequencies spanning two orders of magnitudes reveals that actually no ϵ-η-combination allows for an average adaptation quality index 〈Q〉 higher than approximately 0.12 (Figure 5). We conclude that a standard AFO with a fixed set of parameters is not capable to provide fast as well as precise adaptation over a wide range of frequencies.

FIGURE 5

Figure 5. Average combined quality measure 〈Q〉 for different parameter values of the frequency adaptive Hopf oscillator. For every parameter pair of coupling strength ϵ and learning rate η, the average adaption quality measure 〈Q〉 over the logarithmically sampled space of initial intrinsic frequencies f₀ and frequencies of the external signal f_ext is shown (0.1 < f₀, f_ext < 10). The red circles indicate the four cases shown in Figure 4. In each case, the external signal is a sine-wave with unit amplitude.

2.2. Fast Dynamical Coupling Mechanism

As discussed, no fixed value pair for the coupling strength ϵ and the learning rate η suffices for fast and precise adaptation over a wider range of initial intrinsic and external target frequencies. In order to obtain a system without the requirement for application-specific fine-tuning, the down- or up-scaling of coupling strength and learning rate has to be accomplished in a self-organized manner. Here, we propose such a system. Instead of coupling the external signal F(t) directly to the oscillator, we now use a filtered signal P(t):

$\begin{array}{l}\dot{x}(t)=f_x(x(t),y(t),\theta (t))+P(t)\\ \dot{y}(t)=f_y(x(t),y(t),\theta (t)).& (4)\end{array}$

Accordingly, also the adaptation of θ is based on P(t):

$\begin{array}{ll}\dot{\theta }(t)=\pm \eta P(t)\frac{y(t)}{\sqrt{x{(t)}^2+y{(t)}^2}}.& (5)\end{array}$

P(t) is given by a weighted difference of the external signal F(t) and the oscillator variable x(t):

$\begin{array}{ll}P(t)=ϵ(t)F(t)-\beta (t)x(t)& (6)\end{array}$

with the adaptive coupling strengths ϵ(t) and β(t). Following the discussion of the quality measures introduced before, for an optimal adaptation process, the dynamics of ϵ(t) and β(t) has to fulfill two requirements: as long as the difference between the intrinsic frequency f and the target frequency f_ext of the external signal is high, P(t) should basically be an amplified version of F(t) in order to accelerate the adaptation process. In contrast, when f is already close to f_ext, P(t) is supposed to attain values close to zero such as to reduce the influence of the external signal to a minimum. Both of these requirements can be fulfilled by adapting β(t) and ϵ(t) according to a combination of correlation-based growth and a passive decay toward a low resting value. For β(t), we propose the following dynamics:

$\begin{array}{ll}\tau \dot{\beta }(t)={\beta }_0-\beta (t)+\kappa P(t)x(t)& (7)\end{array}$

with time constant τ and correlation learning rate κ. The value of β scales the subtraction of the system variable x from the external signal F(t) in Equation (6). The product of P and x (averaged over time) is large if the difference between the intrinsic frequency f and the external target frequency f_ext is low. At this stage, the influence of the external signal on the oscillator should be reduced, i.e., the amplitude of P should be decreased, as done by increasing β. The proposed dynamics for ϵ(t) are very similar:

$\begin{array}{ll}\tau \dot{ϵ}(t)={ϵ}_0-ϵ(t)+\kappa F(t)P(t).& (8)\end{array}$

The value of ϵ scales the influence of the external signal F on the filtered signal P (Equation 6). If the averaged product of F and P is large, this implies that the subtraction of x in Equation (6) cannot cancel the addition of F, i.e., the internal frequency of the oscillator is different from the target frequency of the external signal. Thus, an increase of ϵ is desired to increase the influence of the signal on the system and to herewith increase the adaptation speed. However, as for β(t) ≈ 0, the last term of Equation (8) can be approximated by κF(t)²ϵ(t), without adaptation of β(t), the value of ϵ(t) would not return to ϵ₀ as long as the external signal is present and therefore would not allow precise adaptation. Only the interplay of the dynamics of ϵ(t), which detects the onset of an external signal with a frequency different from the intrinsic frequency of the oscillator, and of β(t), which detects when the adaptation is nearly completed, allows fast as well as precise adaptation. In the following, we call the described mechanism “Adaptation through Fast Dynamical Coupling” (AFDC).

The process of frequency adaptation supported by the AFDC mechanism can be separated into several stages (Figure 6) as qualitatively described in the following: Before the onset of an external signal (F = 0), the average product of P and F is zero and the adaptive coupling constants β and ϵ converge toward their resting values ${\beta }_0/(1+\kappa \stackrel{-}{x^2})$ and ϵ₀. Here, $\stackrel{-}{x^2}$ is the mean over time of the squared signal x². As soon as the external signal is applied, the average product of P and F gets positive (Equation 6). As a result of this, ϵ starts to increase (Equation 8). A higher value of ϵ, in turn, increases the average product of P and F. This establishes a positive feedback loop that leads to a fast increase of the amplitude of P. The high amplitude of P results in a large influence of the external signal on the oscillator (Equation 4) as well as in a fast adaptation of the frequency determining variable θ toward the frequency of F (Equation 5). As a consequence of both of these effects, the oscillator follows the external frequency implying a positive correlation between P and x. This correlation leads to an increase of β (Equation 7). Higher values of β decrease the amplitude of P (Equation 6) and, as such, also the average product between P and x. This is a negative feedback loop. Note that a decrease of the amplitude of P also reduces the average product of P and F and therefore breaks the positive feedback loop between ϵ and the average product of P and F (Equation 8) yielding a decline of both β and ϵ to their respective resting values. At this point, switching off the external signal does not significantly change the system dynamics as the influence of the external signal has already been reduced to a minimum.

FIGURE 6

Figure 6. Adaptation of two oscillators with the AFDC mechanism. The upper-most panels show the time course of the frequency determining parameter θ. The time during which the external signal is applied to the system is indicated by the yellow shaded area. The dashed horizontal lines indicate the initial value θ₀ and the value θ_ext corresponding to the exact frequency of the external signal. The second panels from the top show the time course of the adaptive coupling strengths β and ϵ. The third panels from the top show the time course of the filtered external signal P. The panels on the bottom show the oscillating state variables x and y and the external signal F at different short time windows during the adaptation process. In both cases, the initial intrinsic frequency of the oscillator is f₀ = 4.0 and the external signal is a sine wave with unit amplitude and frequency f_ext = 2.0. (A) Hopf oscillator with AFDC mechanism with μ = 1.0, η = 0.5, κ = 5.0, τ = 2.0, β₀ = 0.0 and ϵ₀ = 0.01. The initial value of the parameter θ is given by θ₀ = 2πf₀ ≈ 25.1, the value corresponding to the frequency of the external signal is θ_ext = 2πf_ext ≈ 12.6. The external signal is applied for 5 ≤ t < 30. (B) Van der Pol oscillator with AFDC mechanism with μ = 100.0, η = 2.0, κ = 5.0, τ = 15.0, β₀ = 0.0 and ϵ₀ = 0.01. The values of the parameter θ corresponding to f₀ and f_ext are determined to be θ₀ ≈ 34.8 and θ_ext ≈ 22.0 (see Methods). The external signal is applied for 5 ≤ t < 150.

In summary, the described interplay of the dynamics of the two adaptive coupling constants β and ϵ scales up the magnitude of the external signal as long as adaptation of θ is needed and reduces it once the value corresponding to the frequency of the external signal is reached.

2.2.1. Adaptation Quality in Frequency Space

The dynamics of the AFDC mechanism is mainly dominated by three free parameters: The time scale τ of the adaptive coupling strengths, the correlation learning rate κ and the learning rate η of the frequency determining variable θ. While, in general, an oscillator equipped with an AFDC mechanism shows more tolerance with respect to large frequency ranges, certain parameter combinations allow a faster or more precise adaptation over a larger frequency range (Figure 7). Some combinations (for instance η = 1.0, κ = 100.0, τ = 1.0) result in a very good performance, as indicated by high values of Q, for the complete range of initial intrinsic frequencies f₀ and frequencies f_ext of the external signal analyzed here.

FIGURE 7

Figure 7. Adaptation quality measures for the Hopf oscillator with AFDC mechanism in the f₀-f_ext-frequency space for different parameter values. For each of the given parameter tuples (η, κ, τ), from left to right, the relative convergence time $\tilde{\Delta }$, the relative final frequency offset $\tilde{\delta }$, the final relative frequency fluctuation $\tilde{\sigma }$, and the combined quality measure Q for the given parameter values of η, κ, and τ are shown in the plane spanned by the initial intrinsic frequency f₀ and the frequency of the external unit sine-wave signal f_ext.

This is also reflected by the frequency space averaged quality 〈Q〉 (Figure 8). For a sufficiently high κ (κ ≳ 3), parameters ϵ and η can be found with an average quality value close to the theoretical maximum of 1 corresponding to very fast adaptation without significant frequency offset or frequency oscillations in the finally reached state (Figure 8). A comparison of the performance of the best found configuration of the regular adaptive Hopf oscillator with the performance of the best found configuration of the Hopf oscillator with AFDC mechanism shows that the AFDC mechanism outperforms the regular AFO mechanism in terms of all quality measures (Figure 9A). The same holds true for the comparison of the regular adaptive Van der Pol oscillator with the respective AFDC implementation (Figure 9B). In contrast to a regular AFO, the AFDC mechanism manages to provide fast and precise frequency adaptation over a wide frequency range with a fixed set of parameters.

FIGURE 8

Figure 8. Average combined quality measure 〈Q〉 for different values of κ in the ϵ-η-parameter space of the Hopf oscillator with AFDC mechanism. For every parameter triple of coupling strength ϵ, frequency learning rate η and correlation learning rate κ, the average adaption quality measure 〈Q〉 over the logarithmically sampled space of initial intrinsic frequencies f₀ and frequencies of the external signal f_ext is shown (0.1 < f₀, f_ext < 10). In each case, the external signal is a sine-wave with unit amplitude. The red circles indicate the four cases shown in Figure 7. Comparing these results to the ones obtained for the standard AFO in Figure 5, the AFDC mechanism provides significant higher quality values indicating versatility with respect to different initial intrinsic and external target frequencies.

FIGURE 9

Figure 9. Comparison of the frequency space averaged adaptation quality measures for the best found configurations of the regular adaptive oscillators and the respective oscillators with AFDC mechanism. Note that the averages of the relative convergence time $\tilde{\Delta }$, the final frequency offset $\left|\tilde{\delta }\right|$ and the relative final frequency fluctuation $\tilde{\sigma }$ include only values from (f₀, f_ext)-frequency pairs in which the combined quality measure Q has a nonzero value. The ratio of the number N_Q>0 of (f₀, f_ext)-pairs for which the quality Q has a nonzero value and the total number N_tot of frequency pairs is shown on the very right. All numbers are rounded. See methods for the used parameter values. (A) For the Hopf oscillator, all parameters are identical to the ones used in Figures 7, 8. (B) For the Van der Pol oscillator, we adapt the maximal allowed values of the quality measures. We use ${\tilde{\Delta }}_{\mathrm{\text{max}}}=200$, ${\tilde{\delta }}_{\mathrm{\text{max}}}=0.10$ and ${\tilde{\sigma }}_{\mathrm{\text{max}}}=0.05$. In addition, we calculate $\tilde{\delta }$ and $\tilde{\sigma }$ directly from the frequency determining variable θ and consider the last deviation of θ of more than 10% from the finally reached mean value $\stackrel{-}{\theta }$ to determine the adaptation time Δ. (*) Values shown as 0.00 are too small to be resolved in the figure. For the Hopf oscillator with AFDC mechanism, we find $\langle \widetilde{|}\delta |\rangle /{\tilde{\delta }}_{\mathrm{\text{max}}}\approx 5.3·10^{-7}$ and $\langle \tilde{\sigma }\rangle /{\tilde{\sigma }}_{\mathrm{\text{max}}}\approx 8.5·10^{-8}$. For the Van der Pol oscillator with AFDC mechanism, the average normalized final frequency fluctuation is $\langle \tilde{\sigma }\rangle /{\tilde{\sigma }}_{\mathrm{\text{max}}}\approx 2.5·10^{-3}$.

Note that the values of the additional parameters ϵ₀ and β₀ do not significantly influence the dynamics of the mechanism as long as they are chosen reasonably low.

2.2.2. Neural Implementation

The AFDC mechanism relies on dynamically adapting the coupling strengths ϵ and β. In terms of signal flow, ϵ can be interpreted as a feedforward coupling from the external signal to the filtered signal P. The value of β, in turn, determines the strength of feedback coupling from the oscillator back to P. A standard way to implement this kind of signal flow between different entities is in terms of artificial neural networks. Neural networks are composed of multiple comparably simple computational units, the neurons, which project signals to each other via so-called synapses. Every synapse is characterized by a scalar value, the synaptic weight, which determines the efficacy of the synaptic signal transmission.

There exist neuron models on many different levels of abstraction, ranging from simple binary units to complex biophysical plausible spiking models. Here, we restrict ourselves to a very basic model of point-like neurons described by time-discrete dynamics. It has been shown that already a fully connected network of only two of these very simple neurons suffices to autonomously produce oscillatory signals (Pasemann et al., 2003). In every time step, each neuron sums up the incoming outputs from other neurons as well as from itself weighted by the respective synaptic weights. This sum is transformed into the new neural output by means of a sigmoidal transfer function. The weight matrix of this two neuron network is given by a scaled rotational matrix for a rotation angle φ. The value of φ monotonically controls the frequency of the obtained oscillatory signal of this so-called SO(2)-oscillator.

As already shown earlier (Nachstedt et al., 2012), a neural SO(2)-oscillator with neurons H₀ and H₁ can be extended by an AFDC mechanism by introducing an additional neuron H₂ into the system (Figure 10A). Now, this neural implementation can be understood as a special implementation of the general AFDC mechanism. The additional neuron H₂ is used to calculate the filtered external signal P by receiving synapses from both the external input F and the output of neuron H₀. The latter takes the role of the variable x of the general oscillators discussed above. The synaptic weight w_2F of the synapse from the external signal F to the additional neuron H₂ implements the dynamics of the ϵ coupling. The weight w₂₀ of the synapse from the oscillator neuron H₀ to the neuron H₂ takes the role of β. Adapting the synaptic weights according to Equations (7) and (8) effectively introduces synaptic plasticity into the system (Abbott and Nelson, 2000). In contrast to earlier publications (Nachstedt et al., 2012), here, the weight w₀₂ of the synapse feeding the filtered signal P into the oscillator is simply kept constant.

FIGURE 10

Figure 10. Neural implementation of the AFDC mechanism. (A) The neurons H₀ and H₁ are fully connected by the synapses w₀₀, w₀₁, w₁₀, and w₁₁ and form a neural SO(2)-oscillator. The neuron H₂ calculates the signal P which is the weighted difference between the external signal F and the activity value of H₀. Accordingly, the weight w_2F corresponds to the coupling strength ϵ and the weight w₂₀ represents the variable β of the AFDC mechanism. The weight w₀₂ can either be fixed at a positive value or adapted with similar dynamics as w₂₀ and w_2F. (B) Example adaptation of the neural oscillator. It is initialized with an intrinsic frequency of f₀ = 0.04 corresponding to a value of φ₀ = 0.25 of the internal frequency determining variable. At time step t = 100, an external signal with a frequency of f_ext = 0.02 is applied until time step t = 1, 000 (yellow shaded area). For 1, 000 < t < 1, 900, the frequency of the external signal is changed to f_ext = 0.04 (green shaded area). For t ≥ 1, 900, there is no external signal. Shown from top to bottom are the activities o_i of the neurons H_i (i ∈ {1, 2, 3}), the external signal F, the synaptic weights w₂₀ and w_2F and the frequency determining variable φ of the SO(2)-oscillator.

The adaptation of the intrinsic oscillation frequency by modifying the parameter φ and hereby the synaptic weights of the neural SO(2)-oscillator is a long-lasting change. The discussed plasticity of the synaptic weights w₂₀ and w_2F, in contrast, has a transient character. The combination of these two different kinds of dynamics results in a fast and precise adaptive neural oscillator (Figure 10B) (Nachstedt et al., 2012). This shows that the AFDC mechanism can be easily integrated into existing neural control schemes, for instance, in robotic applications. In addition, the successful implementation of the AFDC mechanism in a time-discrete system shows that the concept can be generalized to this class of dynamical systems.

2.2.3. Closed-Loop Locomotion Control

In addition to the open-loop scenarios studied so far, the AFDC mechanism also allows to apply adaptive oscillators in closed-loop scenarios where fast adaptation toward a specific frequency is required. A classical problem of robotic locomotion control is the task to find the optimal frequency to drive the legs of a walking machine. For animals, it has been found that the frequency during locomotion is tightly related to the resonant frequency of the free swinging leg (Holt et al., 1990). This way, animals are able to maintain energy efficiency during locomotion (Ahlborn and Blake, 2002). Furthermore, it has been proposed that animals actively modify the resonant frequency of their legs in order to optimize for different walking speeds (Ahlborn and Blake, 2002).

Given that CPGs control locomotion, adaptation of CPGs toward a system's resonant frequency to optimize locomotion has been repeatedly investigated and modeled (Verdaasdonk et al., 2006, 2009). A simplistic model of this control problem is given by a mathematical pendulum which is driven by a torque signal according to the output of an oscillator (Nachstedt et al., 2012). The most energy-efficient control is achieved if the pendulum is driven at its resonant frequency determined by its physical length l and its mass m as well as the current amplitude of its oscillation. Here, a neural SO(2)-oscillator with AFDC mechanism is used to control the torque applied to the pendulum. The control loop is closed by feeding the current position of the pendulum as external signal back to the oscillator (Figure 11A).

FIGURE 11

Figure 11. Closed-loop pendulum control using a neural SO(2)-oscillator with AFDC mechanism. Energy-efficient control is realized if the pendulum is driven at its resonant frequency. (A) The output o₁ of neuron H₁ controls the torque M driving the pendulum with length l and mass m. The current angular displacement λ is converted into the external signal F which is fed back to the adaptive oscillator. The neural network is updated with a frequency of 25Hz. (B) Simulation of the system with varying pendulum length l. The initial length of the pendulum is l₀ = 0.2m. At t = 30 s, the length is changed to l₁ = 0.4m. At t = 50 s, the original length l₀ is restored. At t = 70 s, the feedback connection from the pendulum to the oscillator is cut to demonstrate that the oscillator has indeed learned the correct frequency to drive the pendulum. Shown are the current angular displacement λ of the pendulum, the outputs o₀, o₁, and o₂ of the three neurons, the synaptic weights w_2F and w₂₀ of the plastic synapses of the AFDC mechanism, and the intrinsic frequency of the oscillator and the resonant frequency of the undamped and undriven pendulum (target frequency for the oscillator). The resonant frequency of the pendulum does not only depend on the current physical properties of the pendulum but also on the current amplitude of its oscillations.

In this closed-loop system, the current frequency of the pendulum is completely determined by the current frequency of the driving neural oscillator. The observed oscillation frequencies of the pendulum and the neural oscillator are therefore always identical. Still, it is possible to adapt the intrinsic frequency of the neural oscillator to the target frequency given by the resonant frequency of the pendulum. The information about the difference between the intrinsic frequency of the oscillator and the resonant frequency of the pendulum is encoded in the phase relation between the internal oscillation and the feedback signal received as external signal by the oscillator. In particular, driving the pendulum at its resonant frequency is characterized by a phase shift of π/2 between the applied torque and the current angular position of the pendulum. In the neural SO(2)-oscillator, the same phase shift is found between the outputs of the neurons H₀ and H₁. Therefore, when using the output of H₁ to control the torque applied to the pendulum, at resonant frequency, the current angular position of the pendulum is exactly in phase with the output of neuron H₀. This corresponds to the converged state of the AFDC mechanism. If, in turn, the oscillation frequency of the neural SO(2)-oscillator is different from the resonant frequency of the pendulum, the output of H₀ and the angular position of the pendulum are not in phase. The respective phase difference encodes the information about the difference between the intrinsic frequency of the neural oscillator and the resonant frequency of the pendulum and allows the adaptation of the former into the direction of the latter.

In our simulation, we first let the neural SO(2)-oscillator with AFDC mechanism adapt its intrinsic frequency toward the pendulum's resonant frequency (0 s < t < 30 s in Figure 11B). We then simulate a change of the physical properties of the driven system by abruptly changing the length l of the pendulum. Accordingly, the neural oscillator readapts its intrinsic frequency to the new resonant frequency of the pendulum (30 s < t < 50 s in Figure 11B). Afterwards, we change the length l back to its original value. Finally, we cut the feedback connection from the pendulum to the oscillator (t > 70 s in Figure 11B) demonstrating that the oscillator has actually learned the proper frequency to drive the pendulum.

3. Discussion

Transferring key concepts of biological solutions for complex control problems to robotic applications has been proven to be a promising approach regarding the adaptivity, robustness, versatility and agility found in biological organisms (Pfeifer et al., 2007). One especially successful concept is the one of using oscillators, i.e., CPGs, to control complex locomotion. As such, the study of nonlinear oscillators, their entrainment and adaptation properties and possible applications in robotics has gained a lot of interest. The here presented AFDC mechanism overcomes the demonstrated trade-off between speed and precision inherent to regular AFOs as introduced by Righetti et al. (2006). As a result, the AFDC mechanism allows fast and precise adaptation to external signals for a wide range of frequencies with a fixed set of parameters.

Since the discovery of the AFO mechanism, various different mechanisms to improve or extend the adaptation capabilities have been proposed. Subtracting the output of an oscillator from the external signal, as also done in the AFDC mechanism, was used to decompose a signal into its Fourier components (Ronsse et al., 2011b) with the help of an array of AFOs. In order to more reliably find the basic frequency of the external signal, it was proposed to combine a single adaptive frequency oscillator with a Fourier decomposition (Petric et al., 2011). The detailed interaction between multiple AFOs has been studied in the context of networks of self-adaptive dynamical systems (Rodriguez and Hongler, 2014). As an alternative to adapting the system parameters in order to modify the frequency of an oscillator, switching between different oscillation frequencies of an oscillator operated in the chaotic regime by dynamically stabilizing different periods was demonstrated (Steingrube et al., 2010).

The main novelty of the here presented mechanism is the introduced dynamics of the adaptive coupling strengths between the external signal and the filtered signal as well as between the output of the oscillator and the filtered signal. This dynamics temporally increases the influence of the signal on the oscillator as long as it is necessary to achieve fast adaptation and decreases it once precision is needed toward the end of the adaptation process. Adaptive coupling strengths have been proposed earlier as a method to increase the synchronization in a network of phase oscillators with fixed intrinsic frequencies (Ren and Zhao, 2007). The interaction of the transient dynamics of the adaptive coupling strengths on the one hand and the permanent change of intrinsic frequency on the other hand resembles the interplay of long-term (Wood et al., 2011) and short-term (Zucker and Regehr, 2002) plasticity in biological organisms. The interplay of long-term and short-term plasticity in biological system has already been shown to be highly relevant for biological motor control, in particular for fast network reconfiguration (Nadim and Manor, 2000).

The AFDC mechanism increases the complexity of the oscillatory system by the addition of two dynamical equations. Their interplay is required to first scale up the influence of the external signal and later on reduce it again. In particular, this interplay is enabled by the weighted difference P of the external signal F and the oscillator variable x. The correlation of P and F determines the growth of the adaptive coupling constant ϵ which, in turn, increases the correlation of P and F. To counterbalance this self-enhancing dynamics, a second dynamic variable, i.e., β, is required. To make β increase, F and x have to be correlated which is the case once the oscillator has attained the externally applied frequency. This delay of the onsets of the growth processes of ϵ and β is crucial for the AFDC mechanism and cannot be realized by a single variable.

In this contribution, we focused on the Hopf oscillator and the Van der Pol oscillator for the detailed analyses of the regular AFO and the AFDC mechanism. It remains an interesting question for future research in how far the results obtained for these oscillators regarding the frequency-independent choice of parameters as well as regarding the quality measures of the adaptation process generalize to other types of nonlinear oscillators (Rayleigh, 1877; Duffing, 1918; Fitzhugh, 1961).

As the dynamics of the coupling strengths in the AFDC mechanism is solely correlation based, we showed that it is easy to implement the mechanism in neural networks. We demonstrated this by discussing the already earlier published neural time-discrete SO(2)-oscillator with AFDC mechanism (Nachstedt et al., 2012). This special realization of the AFDC mechanism has already been successfully applied in different robotic applications including self-organized control of a snake-like robot (Nachstedt et al., 2013), adaptive control of a robot leg with compliant tarsus (Canio et al., 2016b), and bipedal locomotion with robustness against global loss of sensory feedback (Canio et al., 2016a). In contrast to the neural implementation, the new general formulation of the AFDC mechanism makes it possible to apply the mechanism to all kinds of existing applications of regular AFOs (Buchli et al., 2005). Additionally, it allows the usage of adaptive oscillators in completely new scenarios where, up to now, regular AFOs could not provide sufficiently fast as well as precise adaptation.

4. Materials and Methods

4.1. Hopf Oscillator

The regular Hopf oscillator with the state variables x and y is given by the following system of dynamical equations:

$\begin{array}{l}\dot{x}(t)=\left(\mu -r{(t)}^2\right)x(t)-\theta y(t)\\ \dot{y}(t)=\left(\mu -r{(t)}^2\right)y(t)+\theta x(t)& (9)\end{array}$

with $r(t)=\sqrt{x{(t)}^2+y{(t)}^2}$. The variable μ > 0 determines the amplitude of the oscillations. Without an external signal (F(t) = 0), this system possesses an asymptotically stable and harmonic limit cycle with an angular frequency of exactly θ.

4.1.1. Adaptive Frequency Hopf Oscillator

The Hopf oscillator can be turned into an adaptive frequency oscillator by coupling an external signal F to the system and introducing the dynamics described by Equation (2) to the parameter θ. The complete system is given by

$\begin{array}{l}\dot{x}(t)=\left(\mu -r{(t)}^2\right)x(t)-\theta (t)y(t)+ϵF(t)\\ \dot{y}(t)=\left(\mu -r{(t)}^2\right)y(t)+\theta (t)x(t)\\ \dot{\theta }(t)=-\eta F(t)\frac{y(t)}{\sqrt{x{(t)}^2+y{(t)}^2}}.& (10)\end{array}$

4.1.2. Hopf Oscillator with Fast Dynamical Coupling

The Hopf oscillator equipped with the AFDC mechanism is given by the following system of differential equations:

$\begin{array}{l}\dot{x}(t)=\left(\mu -r{(t)}^2\right)x(t)-\theta (t)y(t)+P(t)\\ \dot{y}(t)=\left(\mu -r{(t)}^2\right)y(t)+\theta (t)x(t)\\ \tau \dot{\beta }(t)={\beta }_0-\beta (t)+\kappa P(t)x(t)\\ \tau \dot{ϵ}(t)={ϵ}_0-ϵ(t)+\kappa F(t)P(t)\\ \dot{\theta }(t)=-\eta P(t)\frac{y(t)}{\sqrt{x{(t)}^2+y{(t)}^2}}& (11)\end{array}$

with P(t) = ϵ(t)F(t) − β(t)x(t).

4.2. Van der Pol Oscillator

The Van der Pol oscillator with the state variables x and y is defined as follows:

$\begin{array}{l}\dot{x}(t)=y(t)\\ \dot{y}(t)=\mu \left(1-x{(t)}^2\right)y(t)-{\theta }^{2}x(t).& (12)\end{array}$

The parameter μ > 0 determines the “degree of nonlinearity” of the system. For μ = 0, the system is harmonic. The intrinsic frequency f depends in a nonlinear and non-trivial way on the parameter θ. We use a Fourier transform in conjunction with a sequence of nested intervals to determine the values of θ corresponding to a given frequency f.

4.2.1. Adaptive Frequency Van der Pol Oscillator

The adaptive frequency formulation of the Van der Pol Oscillator coupled to a time-dependent external signal F(t) requires a positive sign in Equation (2):

$\begin{array}{l}\dot{x}(t)=y(t)+ϵF(t)\\ \dot{y}(t)=\mu (1-x{(t)}^2)y(t)-\theta {\left(t\right)}^{2}x\hfill \\ \dot{\theta }(t)=+\eta F(t)\frac{y(t)}{\sqrt{x{(t)}^2+y{(t)}^2}}.\hfill & (13)\end{array}$

4.2.2. Van der Pol Oscillator with Fast Dynamical Coupling

Applying the AFDC mechanism to the Van der Pol oscillator is described by the following system of differential equations:

$\begin{array}{l}\dot{x}(t)=y(t)+P(t)\hfill \\ \dot{y}(t)=\mu (1-x{(t)}^2)y(t)-\theta {\left(t\right)}^{2}x\hfill \\ \tau \dot{\beta }(t)={\beta }_0-\beta (t)+\kappa P(t)x(t)\hfill \\ \tau \dot{ϵ}(t)={ϵ}_0-ϵ(t)+\kappa F(t)P(t)\hfill \\ \dot{\theta }(t)=+\eta P(t)\frac{y(t)}{\sqrt{x{(t)}^2+y{(t)}^2}}\hfill & (14)\end{array}$

with P(t) = ϵ(t)F(t) − β(t)x(t).

4.3. Neural SO(2)-Oscillator

We use standard additive time-discrete neurons H_i, i ∈ {0, …, N − 1}, where N is the number of neurons in the network. The activation a_i of neuron H_i at time t + 1 is given by the sum of incoming presynaptic neural firing rates o_j weighted by the synaptic weights w_ij at time t:

$\begin{array}{ll}{a}_i(t+1)={\displaystyle \sum _{j=0}^{N-1}{w}_{ij}}(t)o_j(t), i=0,\dots ,N-1.& (15)\end{array}$

The activation a_i of neuron H_i is transformed into its firing rate o_i by a sigmoidal transfer function:

$\begin{array}{ll}{o}_i(t)=\text{tanh }\left(a_i(t)\right).& (16)\end{array}$

The pure SO(2)-network consists of N = 2 fully connected neurons H₀ and H₁. The synaptic weight matrix is chosen according to

$\begin{array}{ll}\left(\begin{array}{cc}{w}_{00}(t)& w_{01}(t)\\ w_{10}(t)& w_{11}(t)\end{array}\right)=\alpha ·\left(\begin{array}{cc}\text{cos}\phi (t)& \text{sin}\phi (t)\\ -\text{sin}\phi (t)& \text{cos}\phi (t)\end{array}\right)& (17)\end{array}$

with 0 < φ(t) < π the frequency determining parameter. The factor α determines the amplitude as well as the nonlinearity of the oscillations. We use α = 1.01 to obtain very harmonic oscillations and an approximately linear relationship between φ and the intrinsic frequency of the oscillator.

4.3.1. SO(2)-Oscillator with Fast Dynamical Coupling

In order to equip the neural SO(2)-oscillator with the AFDC mechanism, an additional neuron H₂ is introduced. The external signal F(t) is fed into the neuron H₂ via a synapse w_2F. The neuron H₂ calculates the filtered version of the external signal and receives signals via the synapses w₂₀ (= β) and w_2F (= ϵ) governed by the following plasticity rules:

$\begin{array}{l}{w}_{20}(t+1)=w_{20}(t)+({\beta }_0-w_{20}(t)-\kappa o_2(t)o_0(t))/\tau \\ w_{2F}(t+1)=w_{2F}(t)+({ϵ}_0-w_{2F}(t)-\kappa F(t)o_2(t))/\tau .& (18)\end{array}$

In accordance with our earlier publication (Nachstedt et al., 2012), we simplify the frequency adaptation rule of the AFDC mechanism and reformulate it in terms of the signals arriving at neuron H₀:

$\begin{array}{ll}\phi (t+1)=\varphi (t)+\eta w_{02}(t)o_2(t)w_{01}(t)o_1(t).& (19)\end{array}$

For the example adaptation process (Figure 10B), we use α = 1.01, η = 1, κ = 100, τ = 100, β₀ = 0 and ϵ₀ = 0.01.

4.4. Mathematical Pendulum

The angular displacement λ of a mathematical pendulum with length l and mass m is described by the following differential equation:

$\begin{array}{ll}\ddot{\lambda }=-\frac{g}{l}\text{sin}\lambda -\frac{D}{m{l}^2}\dot{\lambda }+\frac{M}{m{l}^2}& (20)\end{array}$

with the gravitational acceleration g, the external torque M evoked on the system and the damping constant D. The resonant frequency f_res of the undamped (D = 0) and undriven (M = 0) mathematical pendulum is given by Ochs (2011):

$\begin{array}{ll}{f}_{\text{res}}=\frac{{\omega }_0}{4K(k)}& (21)\end{array}$

with

$\begin{array}{ll}k=\frac{{\dot{\lambda }}^2+4{\omega }_0^2{(\text{sin}\frac{\lambda }{2})}^2}{4{\omega }_0^2}& (22)\end{array}$

and ${\omega }_0=\sqrt{\frac{g}{l}}$. K(k) is the complete elliptic integral of the first kind. In Equation (22), the current values of the angular displacement λ and the angular velocity $\dot{\lambda }$ are used to obtain the current total energy of the system. For our simulations, we use g = 9.81 mm ⁻² and D = 0.005 kg m² s⁻¹.

4.5. Numerical Integration

The integrations of the different differential systems are carried out using the odeint method of the scipy python package (Jones et al., 2001). This methods relies on the LSODA algorithm (Brown and Hindmarsh, 1989) from the FORTRAN library odepack (Hindmarsh, 1983). The LSODA algorithm utilizes an adaptive step size.

4.6. Frequency and Parameter Scans

For the frequency scans performed for the adaptive Hopf oscillator (Figures 4, 5) and the adaptive Van der Pol oscillator as well as for the respective oscillators with AFDC mechanism (Figures 7, 8), we sample the frequency space in the range 0.1 ≤ f₀, f_ext ≤ 10.0. We consider 21 sample values uniformly spaced on a logarithmic axis of f₀ and f_ext and investigate the behavior of the oscillators for all possible 21² (f₀, f_ext)-pairs. For every frequency pair, in the case of the regular adaptive oscillators, we sample 21 parameter values again uniformly spaced on a logarithmic axes of each ϵ and η in the range 0.01 ≤ ϵ, η ≤ 100. Therefore, we investigate a total of 21⁴ (f₀, f_ext, ϵ, η)-configurations for each regular adaptive oscillator. In the case of the oscillators with AFDC mechanism, the parameters are investigated in the ranges 0.01 ≤ η, τ ≤ 100 and 1 ≤ κ ≤ 1, 000 again with 21 samples in every parameter dimension yielding a total of 21⁵ sampled (f₀, f_ext, η, κ, τ)-configurations each.

The best sampled parameter values of the frequency space averaged combined quality measure 〈Q〉 are ϵ ≈ 15.85 and η ≈ 15.85 for the regular adaptive Hopf oscillator and η ≈ 1.58, κ ≈ 398.11 and τ ≈ 3.98 for the Hopf oscillator with AFDC mechanism (Figure 9). For the regular adaptive Van der Pol oscillator, we find ϵ ≈ 0.0158 and η = 1.0 to perform best while η ≈ 0.158, κ = 100 and τ ≈ 1.585 yield the best result for the Van der Pol oscillator with AFDC mechanism.

Author Contributions

The AFDC mechanism was developed by TN and PM. TN and CT planned the presented analyses and experiments. Implementation and analysis of the data was done by TN. TN, CT, and PM wrote and reviewed the manuscript.

Funding

The research leading to these results has received funding from the Federal Ministry of Education and Research (BMBF) Germany to the Göttingen Bernstein Center for Computational Neuroscience under grant numbers 01GQ1005A [TN, PM] and 01GQ1005B [CT] and from the International Max Planck Research School for Physics of Biological and Complex Systems (IMPRS-PBCS) by stipends of the country of Lower Saxony with funds from the initiative Niedersächsisches Vorab and of the University of Göttingen [TN].

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank Florentin Wörgötter for fruitful discussions. We acknowledge support by the Open Access Publication Funds of the Göttingen University.

References

Abbott, L. F., and Nelson, S. B. (2000). Synaptic plasticity: taming the beast. Nat. Neurosci. 3(Suppl.), 1178–1183. doi: 10.1038/81453

PubMed Abstract | CrossRef Full Text | Google Scholar

Ahlborn, B. K., and Blake, R. W. (2002). Walking and running at resonance. Zoology (Jena) 105, 165–174. doi: 10.1078/0944-2006-00057

PubMed Abstract | CrossRef Full Text | Google Scholar

Barkai, N., and Leibler, S. (2000). Circadian clocks limited by noise. Nature 403, 267–268. doi: 10.1038/35002258

PubMed Abstract | CrossRef Full Text | Google Scholar

Brown, P. N., and Hindmarsh, A. C. (1989). Reduced storage matrix methods in stiff ODE systems. Appl. Math. Comput. 31, 40–91. doi: 10.1016/0096-3003(89)90110-0

CrossRef Full Text | Google Scholar

Buchli, J., Righetti, L., and Ijspeert, A. J. (2005). “A dynamical systems approach to learning: a frequency-adaptive hopper robot,” in Proceedings of the 8th European Conference on Advances in Artificial Life, eds M. S. Capcarrère, A. A. Freitas, P. J. Bentley, C. G. Johnson, and J. Timmis (Berlin; Heidelberg: Springer), 210–220.

Google Scholar

Buchli, J., Righetti, L., and Ijspeert, A. J. (2006). Engineering entrainment and adaptation in limit cycle systems: from biological inspiration to applications in robotics. Biol. Cybern. 95, 645–664. doi: 10.1007/s00422-006-0128-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Buchli, J., Righetti, L., and Ijspeert, A. J. (2008). Frequency analysis with coupled nonlinear oscillators. Physica D 237, 1705–1718. doi: 10.1016/j.physd.2008.01.014

CrossRef Full Text | Google Scholar

Canio, G. D., Stoyanov, S., Balmori, I. T., Larsen, J. C., and Manoonpong, P. (2016a). “Adaptive combinatorial neural control for robust locomotion of a biped robot,” in From Animals to Animats 14: Proceedings of the 11th International Conference on Simulation of Adaptive Behavior (Cham: Springer International Publishing), 317–328.

Google Scholar

Canio, G. D., Stoyanov, S., Larsen, J. C., Hallam, J., Kovalev, A., Kleinteich, T., et al. (2016b). A robot leg with compliant tarsus and its neural control for efficient and adaptive locomotion on complex terrains. Art. Life Robot. 21, 274–281. doi: 10.1007/s10015-016-0296-3

CrossRef Full Text | Google Scholar

Cruse, H., Bartling, C., Dreifert, M., Schmitz, J., Brunn, D. E., Dean, J., et al. (1995). Walking: a complex behavior controlled by simple networks. Adapt. Behav. 3, 385–418. doi: 10.1177/105971239500300403

CrossRef Full Text | Google Scholar

Duffing, G. (1918). Erzwungene Schwingungen bei Veränderlicher Eigenfrequenz, 1st Edn. Braunschweig: F. Vieweg u. Sohn.

Fitzhugh, R. (1961). Impulses and physiological states in theoretical models of nerve membrane. Biophys. J. 1, 445–466. doi: 10.1016/S0006-3495(61)86902-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Foth, E., and Bässler, U. (1985). Leg movements of stick insects walking with five legs on a treadwheel and with one leg on a motor-driven belt. I. general results and 1:1-coordination. Biol. Cybern. 51, 313–318. doi: 10.1007/BF00336918

PubMed Abstract | CrossRef Full Text | Google Scholar

Furuta, K. (2003). “Control of pendulum: From super mechano-system to human adaptive mechatronics,” in Proceedings of the IEEE Conference on Decision and Control (Piscataway, NJ: IEEE), 1498–1507.

Google Scholar

Goldbeter, A., Gérard, C., Gonze, D., Leloup, J. C., and Dupont, G. (2012). Systems biology of cellular rhythms. FEBS Lett. 586, 2955–2965. doi: 10.1016/j.febslet.2012.07.041

PubMed Abstract | CrossRef Full Text | Google Scholar

Hellgren, J., and Grillner, S. (1992). Computer simulation of the segmental neural network generating locomotion in lamprey by using populations of network interneurons. Biol. Cybern. 68, 1–13. doi: 10.1007/BF00203132

PubMed Abstract | CrossRef Full Text | Google Scholar

Hindmarsh, A. C. (1983). “ODEPACK, a systematized collection of ode solvers,” in IMACS Transactions on Scientific Computation: Scientific Computing, eds R. S. Stepleman, C. Carver, R. Peskin, W. F. Ames, and R. Vienevetsky (Amsterdam: North-Holland), 55–63.

Google Scholar

Holt, K. G., Hamill, J., and Andres, R. O. (1990). The force-driven harmonic oscillator as a model for human locomotion. Hum. Mov. Sci. 9, 55–68. doi: 10.1016/0167-9457(90)90035-C

CrossRef Full Text | Google Scholar

Hooper, S. L. (2001). “Central pattern generators,” in Encyclopedia of Life Sciences (New York, NY: Wiley), 1–12. Available online at: http://onlinelibrary.wiley.com/doi/10.1038/npg.els.0000032/abstract

Ijspeert, A. J. (2008). Central pattern generators for locomotion control in animals and robots: a review. Neural. Netw. 21, 642–653. doi: 10.1016/j.neunet.2008.03.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Jones, E., Oliphant, T., Peterson, P., and Al, E. (2001). Scipy: Open Source Scientific Tools for Python.

Kimura, H., Fukuoka, Y., and Konaga, K. (2001). Adaptive dynamic walking of a quadruped robot using a neural system model. Adv. Robot. 15, 859–878. doi: 10.1163/156855301317198179

CrossRef Full Text | Google Scholar

Marder, E., and Bucher, D. (2001). Central pattern generators and the control of rhythmic movements. Curr. Biol. 11, R986–R996. doi: 10.1016/S0960-9822(01)00581-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Matsuoka, K. (1985). Sustained oscillations generated by mutually inhibiting neurons with adaptation. Biol. Cybern. 52, 367–376. doi: 10.1007/BF00449593

PubMed Abstract | CrossRef Full Text | Google Scholar

Nachstedt, T., Wörgötter, F., and Manoonpong, P. (2012). “Adaptive neural oscillator with synaptic plasticity enabling fast resonance tuning,” in International Conference on Artificial Neural Networks, ICANN2012 (Berlin; Heidelberg: Springer), 451–458. doi: 10.1007/978-3-642-33269-2_57

CrossRef Full Text | Google Scholar

Nachstedt, T., Wörgötter, F., Manoonpong, P., Ariizumi, R., Ambe, Y., and Matsuno, F. (2013). “Adaptive neural oscillators with synaptic plasticity for locomotion control of a snake-like robot with screw-drive mechanism,” in Proceedings of the 2013 IEEE International Conference on Robotics and Automation (Piscataway, NJ: IEEE), 3389–3395.

Google Scholar

Nadim, F., and Manor, Y. (2000). The role of short-term synaptic dynamics in motor control. Curr. Opin. Neurobiol. 10, 683–690. doi: 10.1016/S0959-4388(00)00159-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Nakamura, Y., Mori, T., Sato, M. A., and Ishii, S. (2007). Reinforcement learning for a biped robot based on a CPG-actor-critic method. Neural Netw. 20, 723–735. doi: 10.1016/j.neunet.2007.01.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Nassour, J., Hénaff, P., Benouezdou, F., and Cheng, G. (2014). Multi-layered multi-pattern CPG for adaptive locomotion of humanoid robots. Biol. Cybern. 108, 291–303. doi: 10.1007/s00422-014-0592-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Ochs, K. (2011). A comprehensive analytical solution of the nonlinear pendulum. Eur. J. Phys. 32, 479–490. doi: 10.1088/0143-0807/32/2/019

CrossRef Full Text | Google Scholar

Pasemann, F., Hild, M., and Zahedi, K. (2003). “SO(2)-networks as neural oscillators,” in Computational Methods in Neural Modeling, ed J. Mira (Berlin; Heidelberg: Springer), 1042–1042.

Google Scholar

Petric, T., Gams, A., Ijspeert, A. J., and Zlajpah, L. (2011). On-line frequency adaptation and movement imitation for rhythmic robotic tasks. Int. J. Rob. Res. 30, 1775–1788. doi: 10.1177/0278364911421511

CrossRef Full Text | Google Scholar

Pfeifer, R., Lungarella, M., and Iida, F. (2007). Self-organization, embodiment, and biologically inspired robotics. Science 318, 1088–1093. doi: 10.1126/science.1145803

PubMed Abstract | CrossRef Full Text | Google Scholar

Pinto, C. M. A., Rocha, D., and Santos, C. P. (2012). Hexapod robots: new CPG model for generation of trajectories. SIAM J. Numer. Anal. 7, 15–26. doi: 10.1063/1.3636775

CrossRef Full Text | Google Scholar

Rayleigh, J. W. S. (1877). The Theory of Sound, 1st Edn. London: Macmillan and Co.

Ren, Q., and Zhao, J. (2007). Adaptive coupling and enhanced synchronization in coupled phase oscillators. Phys. Rev. E 76, 1–6. doi: 10.1103/PhysRevE.76.016207

PubMed Abstract | CrossRef Full Text | Google Scholar

Righetti, L., Buchli, J., and Ijspeert, A. J. (2006). Dynamic hebbian learning in adaptive frequency oscillators. Physica D 216, 269–281. doi: 10.1016/j.physd.2006.02.009

CrossRef Full Text | Google Scholar

Righetti, L., Buchli, J., and Ijspeert, A. J. (2009). Adaptive frequency oscillators and applications. Open Cybern. Syst. J. 3, 64–69. doi: 10.2174/1874110X00903010064

CrossRef Full Text | Google Scholar

Righetti, L., and Ijspeert, A. J. (2008). “Pattern generators with sensory feedback for the control of quadruped locomotion,” in Proceedings of the IEEE International Conference on Robotics and Automation (Piscataway, NJ: IEEE), 819–824.

Google Scholar

Rodriguez, J., and Hongler, M. O. (2014). Networks of self-adaptive dynamical systems. IMA. J. Appl. Math. 79, 201–240. doi: 10.1093/imamat/hxs057

CrossRef Full Text | Google Scholar

Ronsse, R., Lenzi, T., Vitiello, N., Koopman, B., Van Asseldonk, E., De Rossi, S. M. M., et al. (2011a). Oscillator-based assistance of cyclical movements: model-based and model-free approaches. Med. Biol. Eng. Comput. 49, 1173–1185. doi: 10.1007/s11517-011-0816-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Ronsse, R., van den Kieboom, J., and Ijspeert, A. J. (2011b). “Automatic resonance tuning and feedforward learning of biped walking using adaptive oscillators,” in Proceeding of the Conference in Multibody Dynamics 2011, ECCOMAS Thematic Conference (Louvain: Université catholique de Louvain), 4–7.

Google Scholar

Santos, C. P., Alves, N., and Moreno, J. C. (2017). Biped locomotion control through a biomimetic CPG-based controller. J. Intell. Robot. Syst. 85, 47–70. doi: 10.1007/s10846-016-0407-3

CrossRef Full Text | Google Scholar

Spong, M. W. (1995). Swing up control problem for the acrobot. IEEE Control Syst. Mag. N. Y. 15, 49–55. doi: 10.1109/37.341864

CrossRef Full Text | Google Scholar

Steingrube, S., Timme, M., Wörgötter, F., and Manoonpong, P. (2010). Self-organized adaptation of a simple neural circuit enables complex robot behaviour. Nat. Phys. 6, 224–230. doi: 10.1038/nphys1508

CrossRef Full Text | Google Scholar

Tropea, P., Vitiello, N., Martelli, D., Aprigliano, F., Micera, S., and Monaco, V. (2015). Detecting slipping-like perturbations by using adaptive oscillators. Ann. Biomed. Eng. 43, 416–426. doi: 10.1007/s10439-014-1175-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Van der Pol, B. (1920). A theory of the amplitude of free and forced triode vibrations. Radio Rev. 11, 701–710.

Verdaasdonk, B. W., Koopman, H. F., and Helm, F. C. (2006). Energy efficient and robust rhythmic limb movement by central pattern generators. Neural Netw. 19, 388–400. doi: 10.1016/j.neunet.2005.09.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Verdaasdonk, B. W., Koopman, H. F., and van der Helm, F. C. (2009). Energy efficient walking with central pattern generators: from passive dynamic walking to biologically inspired control. Biol. Cybern. 101, 49–61. doi: 10.1007/s00422-009-0316-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, T., Hu, Y., and Liang, J. (2013). Learning to swim: a dynamical systems approach to mimicking fish swimming with CPG. Robotica 31, 361–369. doi: 10.1017/S0263574712000343

CrossRef Full Text | Google Scholar

Winfree, A. T. (1967). Biological rhythms and the behavior of populations of coupled oscillators. J. Theor. Biol. 16, 15–42. doi: 10.1016/0022-5193(67)90051-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Wood, R., Baxter, P., and Belpaeme, T. (2011). A review of long-term memory in natural and synthetic systems. Adapt. Behav. 20, 81–103. doi: 10.1177/1059712311421219

CrossRef Full Text | Google Scholar

Zucker, R. S., and Regehr, W. G. (2002). Short-term synaptic plasticity. Annu. Rev. Physiol. 64, 355–405. doi: 10.1146/annurev.physiol.64.092501.114547

PubMed Abstract | CrossRef Full Text | Google Scholar