We present here an example of a slow-fast ordinary differential equation perturbed by a fast external periodic input. We have chosen this example since it readily illustrates many ideas that will be developed in the following sections. In particular, this example shows how the ratio between the time-scale separation of the system and the time-scale of the input appears as a new crucial parameter.

Example 2.1 Consider the following linear time-inhomogeneous dynamical system with ϵ 1 , ϵ 2 > 0 two parameters:

d v ϵ d t = 1 ϵ 1 ( − v ϵ + sin ( t ϵ 2 ) ) , d w ϵ d t = − w ϵ + v ϵ 2 .

This system is particularly handy since one can solve analytically the first ordinary differential equation, that is,

v ( t ) = 1 1 + μ 2 ( sin ( t ϵ 2 ) − μ cos ( t ϵ 2 ) ) + v 0 e − t ϵ 1 ,

where we have introduced the time-scales ratio

μ : = ϵ 1 ϵ 2 .

In this system, one can distinguish various asymptotic regimes when ϵ 1 and ϵ 2 are small according to the asymptotic value of μ:

• Regime 1: Slow input μ = 0 :

First, if ϵ1→0 and ϵ2 is fixed, then v ( t ) is close to sin ( t ϵ 2 ) , and from geometric singular perturbation theory 2122 one can approximate the slow variable w ϵ by the solution of

d w d t = − w + ( sin ( t ϵ 2 ) ) 2 .

Now taking the limit ϵ2→0 and applying the classical averaging principle 4 for periodically driven differential equations, one can approximate wϵ by the solution of

d w d t = − w + 1 2 ,

since 1 2 π ∫ 0 2 π sin ( s ) 2 d s = 1 2 .

• Regime 2: Fast input μ = ∞ :

If ϵ2→0 and ϵ1 is fixed, then the classical averaging principle implies that v ϵ is close to the solution of

d v d t = − v ϵ 1 ,

so that wϵ can be approximated by

d w d t = − w + ( v 0 e − t / ϵ 1 ) 2 ,

and when ϵ1→0, one does not recover the same asymptotic behavior as in Regime 1.

• Regime 3: Time-scales matching 0 < μ < ∞ :

Now consider the intermediate case where ϵ1 is asymptotically proportional to ϵ2. In this case, vϵ can be approximated on the fast time-scale t / ϵ 1 by the periodic solution v ¯ μ ( t ) = 1 1 + μ 2 ( sin ( μ t ) − μ cos ( μ t ) ) of d v d t = − v + sin ( μ t ) . As a consequence, wϵ will be close to the solution of

d w d t = − w + 1 2 ( 1 + μ 2 ) ,

since 1 2 π ∫ 0 2 π v ¯ μ ( t / μ ) 2 d t = 1 2 ( 1 + μ 2 ) .

Thus, we have seen in this example that

1. the two limits ϵ1→0 and ϵ2→0 do not commute,

2. the ratio μ between the internal time-scale separation ϵ1 and the input time-scale ϵ2 is a key parameter in the study of slow-fast systems subject to a time-dependent perturbation.

2.2 Stochastic averaging principle

Time-scales separation is a key property to investigate the dynamical behavior of non-linear multiscale systems, with techniques ranging from averaging principles to geometric singular perturbation theory. This property appears to be also crucial to understanding the impact of noise. Instead of carrying a small noise analysis, a multiscale approach based on the stochastic averaging principle 2 can be a powerful tool to unravel subtle interplays between noise properties and non-linearities. More precisely, consider a system of SDEs in R p + q :

d v t ϵ = 1 ϵ F ( v t ϵ , w t ϵ ) d t + 1 ϵ Σ ( v t ϵ , w t ϵ ) ⋅ d B ( t ) , d w t ϵ = G ( v t ϵ , w t ϵ ) d t ,

with initial conditions v ϵ ( 0 ) = v 0 , w ϵ ( 0 ) = w 0 , and where w ϵ ∈ R q is called the slow variable, v ϵ ∈ R p is the fast variable, with F, G, Σ smooth functions ensuring the existence and uniqueness for the solution ( v ϵ , w ϵ ) , and B ( t ) a p-dimensional standard Brownian motion, defined on a filtered probability space ( Ω , F , P ) . Time-scale separation in encoded in the small parameter ϵ, which denotes in this section a single positive real number.

In order to approximate the behavior of ( v ϵ , w ϵ ) for small ϵ, the idea is to average out the equation for the slow variable with respect to the stationary distribution of the fast one. More precisely, one first assumes that for each w ∈ R q fixed, the frozen fast SDE,

d v t = F ( v t , w ) d t + Σ ( v t , w ) ⋅ d B ( t ) ,

admits a unique invariant measure, denoted ρ w ( d v ) . Then, one defines the averaged drift vector field G¯

G ¯ ( w ) : = ∫ R m G ( v , w ) ρ w ( d v )

and w the solution of d w d t = G ¯ ( w ) with the initial condition w ( 0 ) = y 0 . Under some dissipativity assumptions, the stochastic averaging principle 2 states:

Theorem 2.1 For any δ > 0 and T > 0 ,

lim ϵ → 0 P [ sup t ∈ [ 0 , T ] ∥ w t ϵ − w t ∥ 2 > δ ] = 0 .

As a consequence, analyzing the behavior of the deterministic solution w can help to understand useful features of the stochastic process (vϵ,wϵ).

Example 2.2 In this example we consider a similar system as in Example 2.1, but with a noise term instead of the periodic perturbation. Namely, we consider (vϵ,wϵ) the solution of the system of SDEs,

d v ϵ = − 1 ϵ v ϵ d t + σ ϵ d B ( t ) , d w ϵ = ( − w ϵ + ( v ϵ ) 2 ) d t ,

with ϵ > 0 a small parameter and σ > 0 a positive constant. From Theorem 2.1, the stochastic slow variable w ϵ can be approximated in the sense of (3) by the deterministic solution w of

d w d t = ∫ v ∈ R ( − w + v 2 ) ρ ( d v ) ,

where ρ ( d v ) is the stationary measure of the linear diffusion process,

d v = − v d t + σ d B ( t ) ,

that is,

ρ ( d v ) = 1 σ π e − v 2 σ 2 .

Consequently, wϵ can be approximated in the limit ϵ → 0 by the solution of

d w d t = − w + σ 2 2 .

Applying (3) leads to the following result: for any T>0 and δ>0,

lim ϵ → 0 P [ sup t ∈ [ 0 , T ] | w t ϵ − ( y 0 − σ 2 2 ) e − t + σ 2 2 | 2 > δ ] = 0 .

Interestingly, the asymptotic behavior of wϵ for small ϵ is characterized by a deterministic trajectory that depends on the strength σ of the noise applied to the system. Thus, the stochastic averaging principle appears particularly interesting when unraveling the impact of noise strength on slow-fast systems.

Many other results have been developed since, extending the set-up to the case where the slow variable has a diffusion component or to infinite-dimensional settings for instance, and also refining the convergence study, providing homogenization results concerning the limit of ϵ − 1 / 2 ( w ϵ − w ) or establishing large deviation principles (see 23 for a recent monograph). However, fewer results are available in the case of non-homogeneous SDEs, that is, when the system is perturbed by an external time-dependent signal. This setting is of particular interest in the framework of stochastic learning models, and we present the main relevant mathematical results in the following section.

Combining ideas of periodic and stochastic averaging introduced previously, we present here theoretical results concerning multiscale SDEs driven by an external time-periodic input. Consider ( v ϵ , w ϵ ) the solution of

d v ϵ = 1 ϵ 1 [ F ( v ϵ , w ϵ , t ϵ 2 ) ] d t + 1 ϵ 1 Σ ( v ϵ , w ϵ ) ⋅ d B ( t ) , d w ϵ = G ( v ϵ , w ϵ ) d t ,

with t → F ( v , w , t ) ∈ R p a τ-periodic function and ϵ = ( ϵ 1 , ϵ 2 ) ∈ R + 2 . Parameter ϵ1 represents the internal time-scale separation and ϵ2 the input time-scale. We consider the case where both ϵ1 and ϵ2 are small, that is, a strong time-scale separation between the fast variable vϵ∈Rp and the slow one wϵ∈Rq, and a fast periodic modulation of the fast drift F ( v , w , ⋅ ) .

We further denote z = ( v , w ) .

Definition 2.1 We define the asymptotic time-scale ratio

μ : = lim | ϵ | → 0 ϵ 1 ϵ 2 .

Accordingly, we denote lim | ϵ | → 0 μ the distinguished limit when ϵ1→0, ϵ2→0 with ϵ 1 / ϵ 2 → μ .

The following assumption is made to ensure existence and uniqueness of a strong solution to system (4). In the following, 〈 z 1 , z 2 〉 will denote the usual scalar product for vectors.

Assumption 2.1 Existence and uniqueness of a strong solution

(i) The functions F, G, and Σ are locally Lipschitz continuous in the space variable z. More precisely, for any R > 0 , there exists a constant α R such that

∥ F ( z ) − F ( z ′ ) ∥ ≤ α R ∥ z − z ′ ∥ for any  z , z ′ ∈ R p + q  with  ∥ z ∥ ≤ R  and  ∥ z ′ ∥ ≤ R .

(ii) There exists a constant R>0 such that

sup ∥ z ∥ > R , t > 0 〈 ( F ( z , t ) , G ( z ) ) , z 〉 ∥ z ∥ 2 < 0 .

To control the asymptotic behavior of the fast variable, one further assumes the following.

Assumption 2.2 Asymptotic behavior of the fast process:

(i) The diffusion matrix Σ is bounded

∃ M Σ > 0 s.t.  ∀ z , ∥ Σ ( z ) ∥ < M Σ

and uniformly non-degenerate

∃ η 0 > 0 s.t.  ∀ v , z 〈 Σ ( z ) ⋅ Σ ( z ) ′ v , v 〉 ≥ η 0 ∥ v ∥ 2 .

(ii) There exists r 0 < 0 such that for all t ≥ 0 and for all z , x ∈ R p + q ,

〈 ∇ z F ( z , t ) ⋅ x , x 〉 ≤ r 0 ∥ x ∥ 2 .

According to the value of μ ∈ { 0 , R + ∗ , ∞ } , the stochastic averaging principle is based on a description of the asymptotic behavior of various rescaled fast frozen processes. More precisely, under Assumptions 2.1 and 2.2, one can deduce that:

• For any fixed w 0 ∈ R q and t 0 > 0 fixed, the law of the rescaled time-homogeneous frozen process,

d v = F ( v , w 0 , t 0 ) d t + Σ ( v , w 0 ) d B ( t ) ,

converges exponentially fast to a unique invariant probability measure denoted by ρ w 0 , t 0 ( d v ) .

• For any fixed w0∈Rq, there exists a τ μ -periodic evolution system of measures ν μ w 0 ( t , d v ) , different from ρ w 0 , t ( d v ) above, such that the law of the rescaled time-inhomogeneous frozen process,

d v = F ( v , w 0 , μ t ) d t + Σ ( v , w 0 ) d B ( t ) ,

converges exponentially fast towards ν μ w 0 ( t , ⋅ ) , uniformly with respect to w0 (cf. the Appendix Theorem A.1).

• For any fixed w0∈Rq, the law of the rescaled time-homogeneous frozen process,

d v = F ¯ ( v , w 0 ) d t + Σ ( v , w 0 ) d B ( t ) ,

where F ¯ ( v , w 0 ) : = τ − 1 ∫ 0 τ F ( v , w 0 , t ) d t , converges exponentially fast towards a unique invariant probability measure denoted by ρ ¯ w 0 ( d v ) .

According to the value of μ, we introduce a vector field G ¯ μ which will play a role similar to G¯ introduced in equation (2).

Definition 2.2 We define G ¯ μ : R q → R q as follows. In the time-scale matching case, that is, when 0 < μ < ∞ , then

G ¯ μ ( w ) : = ( τ μ ) − 1 ∫ 0 τ μ ∫ v ∈ R p G ( v , w ) ν μ w ( t , d v ) d t .

Notation We may denote the periodic system of measures ν μ w ( t , d v ) associated with (6) by ν μ w [ F , Σ ] ( t , d v ) to emphasize its relationship with F and Σ. Accordingly, we may denote G ¯ μ ( w ) by G ¯ μ [ F , Σ ] ( w ) .

We are now able to present our main mathematical result. Extending Theorem 2.1, the following theorem describes the asymptotic behavior of the slow variable w ϵ when ϵ→0 with ϵ1/ϵ2→μ. We refer to 6 for more details about the full mathematical proof of this result.

Theorem 2.2 Let μ∈(0,∞). If w is the solution of

d w d t = G ¯ μ ( w ) with  w ( 0 ) = w ϵ ( 0 ) ,

then the following convergence result holds, for all T>0 and δ>0:

lim | ϵ | → 0 μ P [ sup t ∈ [ 0 , T ] | w t ϵ − w t | 2 > δ ] = 0 .

Remark 2.1

1. The extremal cases μ = 0 and μ=∞ are not covered in full rigor by Theorem 2.2. However, the study of the sequential limits ϵ1→0 followed by ϵ 2 → 0 or ϵ2→0 followed by ϵ 1 → 0 can be deduced from an appropriate combination of classical periodic and stochastic averaging theorems:

• Slow input: If one considers the case where the limit ϵ1→0 is taken first, so that from Theorem 2.1 with fast variable v ϵ and slow variables w ϵ and t (with the trivial equation t ˙ = 1 ), wϵ is close in probability on finite time-intervals to the solution of the following inhomogeneous ordinary differential equation:

d w ˜ d t = ∫ v ∈ R p G ( v , w ˜ ) ρ w ˜ , t / ϵ 2 ( d v ) : = G ˜ ( w ˜ , t / ϵ 2 ) .

Then taking the limit ϵ2→0, one can apply the deterministic averaging principle to the fast periodic vector field G ˜ ( w , t / ϵ 2 ) , so that w ˜ converges when ϵ2→0 to the solution of

d w d t = τ − 1 ∫ 0 τ G ˜ ( v , w ) d t = G ¯ 0 ( w ) ,

where

G ¯ 0 ( w ) : = τ − 1 ∫ 0 τ ∫ v ∈ R p G ( v , w ) ρ w , t ( d v ) d t .

• Fast input: If the limit ϵ 2 → 0 is taken first, one first has to perform a classical averaging of the periodic drift F ( v , w , t / ϵ 2 ) leading to the homogeneous system of SDEs (4), but with F ¯ ( v , w ) instead of F(v,w,t/ϵ2). Then, an application of Theorem 2.1 on this system gives an averaged vector field

G ¯ ∞ ( w ) : = ∫ v ∈ R p G ( v , w ) ρ ¯ w ( d v ) .

2. To study the extremal cases μ=0 and μ=∞ in full generality, one would need to consider all the possible relationships between ϵ 1 and ϵ 2 , not only the linear one as in the present article, but also of the type ϵ 1 = ϵ 2 α for example. In this case, we believe that the regime α < 1 converges to the same limit as taking the limit ϵ2 first and the regime α > 1 corresponds to taking the limit ϵ1 first. The intermediate regime α = 1 seems to be the only one for which the limit cannot be obtained by combining classical averaging principles. Therefore, the present article is focused on this case, in which the averaged system depends explicitly on the scaling parameter μ. Moreover, in terms of applications, this parameter can have a relatively easy interpretation in terms of the ratio of time-scales between intrinsic neuronal activity and typical stimulus time-scales in a given situation. Although the zeroth order limit (i.e., the averaged system) seems to depend only on the position of α with respect to 1, it seems reasonable to expect that the fluctuations around the limit would depend on the precise value of α. This is a difficult question which may deserve further analysis.

The case 0 < μ < ∞ is already very rich in the sense that it combines simultaneously both the periodic and stochastic averaging principles in a new way that cannot be recovered by sequential applications of those principles. A particular role is played by the frozen periodically-forced SDE (6). The equivalent of the quasi-stationary measure ρ w of Theorem 2.1 is given by the asymptotically periodic behavior of equation (6), represented by the periodic family of measures νμw(t,dv).

3. By a rescaling of the frozen process (6), one deduces the following scaling relationships:

ν μ w [ F , Σ ] ( t , d v ) = ν 1 w [ F μ , Σ μ ] ( μ t , d v )

and

G ¯ μ [ F , Σ ] ( w ) = G ¯ 1 [ F μ , Σ μ ] ( w ) .

Therefore, if one knows, in the case μ = 1 , the averaged vector field associated with the fast process generated by a drift F and a diffusion coefficient σ, denoted G ¯ 1 [ F , Σ ] , it is possible to deduce G¯μ in the general case μ ∈ ( 0 , ∞ ) with a change F → μ F and Σ → μ Σ .

4. It seems reasonable to expect that the above result is still valid when considering ergodic, but not necessarily periodic, time dependency of the function F ( v , w , ⋅ ) . In equation (7), instead of integrating νμw(t,dv) over one period, one should integrate it with respect to an ergodic stationary measure. However, this extension requires non-trivial technical improvements of 5 which are beyond the scope of this paper.

2.3.1 Case of a fast linear SDE with periodic input

We present here an elementary case where one can compute explicitly the quasi-stationary time-periodic family of measures ν μ w ( t , x ) , when the equation for the fast variable is linear. Namely, we consider v ∈ R p the solution of

d v ( t ) = ( − A ⋅ v ( t ) + u ( μ t ) ) d t + Σ ⋅ d B ( t ) ,

with initial condition v ( 0 ) = v 0 ∈ R p , and where A ∈ R p × p is a matrix whose eigenvalues have positive real parts and u ( ⋅ ) is a τ-periodic function.

We are interested in the large time behavior of the law of v ( t ) , which is a time-inhomogeneous Ornstein-Uhlenbeck process. From 5 we know that its law converges to a τ-periodic family of probability measures ν ( t , d v ) . Due to the linearity in the previous equation, ν(t,dv) is Gaussian with a time-dependent mean and a constant covariance matrix

ν ( t , d v ) = N v ¯ ( t ) , Q ( d v ) ,

where v ¯ is the τ μ -periodic attractor of d v ¯ d t = − A ⋅ v ¯ ( t ) + u ( μ t ) , i.e.,

v ¯ ( t ) = ∫ − ∞ t e − A ( t − s ) u ( μ s ) d s ,

and Q is the unique solution of the Lyapunov equation

A ⋅ Q + Q ⋅ A ′ + Σ ⋅ Σ ′ = 0 .

Indeed, if one denotes c ( t ) = v ( t ) − v ¯ ( t ) , then c ( t ) is a solution of the classical homogeneous Ornstein-Uhlenbeck equation

d c ( t ) = − A c ( t ) d t + Σ d B ( t ) ,

whose stationary distribution is known to be a centered Gaussian measure with the covariance matrix Q solution of (9); see Chapter 3.2 of 24. Notice that if A is self-adjoint with respect to ( Σ ⋅ Σ ′ ) − 1 (i.e., A ⋅ ( Σ ⋅ Σ ′ ) = ( Σ ⋅ Σ ′ ) ⋅ A ′ ), then the solution is Q = A − 1 ⋅ ( Σ ⋅ Σ ′ ) 2 = ( Σ ⋅ Σ ′ ) ⋅ A ′ − 1 2 , which will be used in Appendix B.2.

Hence, in the linear case, the averaged vector field of equation (7) becomes

G ¯ μ ( w ) : = ( τ μ ) − 1 ∫ 0 τ μ ∫ v ∈ R p G ( v ¯ ( t ) + v , w ) N 0 , Q ( d v ) d t ,

where N x , Q is the probability density function of the Gaussian law with mean x ∈ R q and covariance Q ∈ R p × p .

Therefore, due to the linearity of the fast SDE, the periodic system of measure ν is just a constant Gaussian distribution shifted by a periodic function of time v ( t ) . In case G is quadratic in v, this remark implies that one can perform independently the integral over time and over R p in formula (10) (noting that the crossed term has a zero average). In this case, contributions from the periodic input and from noise appear in the averaged vector field in an additive way.

Example 2.3 In this last example, we consider a combination between Example 2.1 and Example 2.2, namely we consider the following system of periodically forced SDEs:

d v ϵ = 1 ϵ 1 [ − v ϵ + sin ( t ϵ 2 ) ] d t + σ ϵ 1 d B ( t ) , d w ϵ = ( − w ϵ + ( v ϵ ) 2 ) d t .

As in Example 2.1 and as shown above, the behavior of this system when both ϵ1 and ϵ2 are small depends on the parameter μ defined in (5). More precisely, we have the following three regimes:

• Regime 1: slow input:

G ¯ 0 ( w ) = − w + σ 2 2 + 1 2 .

• Regime 2: fast input:

G ¯ ∞ ( w ) = − w + σ 2 2 .

• Regime 3: time-scale matching:

G ¯ μ ( w ) = − w + σ 2 2 + 1 2 ( 1 + μ 2 ) .

2.4 Truncation and asymptotic well-posedness

In some cases, Assumptions 2.1-2.2 may not be satisfied on the entire phase space R p × R q , but only on a subset. Such situations will appear in Section 3 when considering learning models. We introduce here a more refined set of assumptions ensuring that Theorem 2.2 still applies.

Let us start with an example, namely the following bi-dimensional system with white noise input:

{ d v ϵ = 1 ϵ ( − l v ϵ + w ϵ v ϵ ) d t + σ ϵ d B ( t ) , d w ϵ = ( − κ w ϵ + ( v ϵ ) 2 ) d t ,

with ϵ>0, σ>0, l > 0 , μ > 0 .

For the fast drift − ( l − w ) v to be non-explosive, it is necessary to have w < l − α with α > 0 for all time. The concern about this system comes from the fact that the slow variable w may reach l due to the fluctuations captured in the term v 2 , for instance, if κ is not large enough. Such a system may have exponentially growing trajectories. However, we claim that for small enough ϵ, wϵ will remain close to its averaged limit w for a very long time, and if this limit remains below l − α , then wϵ can be considered as well-posed in the asymptotic limit ϵ→0. To make this argument more rigorous, we suggest the following definition.

Definition 2.3 A stochastic differential equation with a given initial condition is asymptotically well posed in probability if for the given initial condition,

1. a unique solution exists until a random time τ ϵ

2. for all T>0,

lim ϵ → 0 P [ τ ϵ ≥ T ] = 1 .

We give in the following proposition sufficient conditions for system (4) to be asymptotically well posed in probability and to satisfy conclusions of Theorem 2.2.

Let us introduce the following set of additional assumptions.

Assumption 2.3 Moment conditions:

(i) There exists p > 2 such that

for any  T > 0 , sup ϵ E [ sup 0 ≤ t ≤ T ∥ v t ϵ ∥ p + ∥ w t ϵ ∥ p ] < ∞ .

(ii) For any T>0 and any bounded subset K of R q ,

sup ϵ 1 > 0 , ϵ 2 > 0 , w ∈ K E [ sup 0 ≤ t ≤ T ∥ G ( v t ϵ , w ) ∥ 2 ] < ∞ .

Remark 2.2 This last set of assumptions will be satisfied in all the applications of Section 3 since we consider linear models with additive noise for the equation of v, implying this variable to be Gaussian and the function G only involves quadratic moments of v; therefore, the moment conditions (i) and (ii) will be satisfied without any difficulty. Moreover, if one considers non-linear models for the variable v, then the Gaussian property may be lost; however, adding sigmoidal non-linearity has, in general, the effect of bounding the dynamics, thus making these moment assumptions reasonable to check in most models of interest.

Property 2.3 If there exists a subset ℰ of Rq such that

1. The functions F, G, Σ satisfy Assumptions 2.1-2.3 restricted on R p × E .

2. ℰ is invariant under the flow of G¯μ, as defined in (7).

Then for any initial condition w 0 ∈ E , system (4) is asymptotically well posed in probability and wϵ satisfies the conclusion of Theorem 2.2.

Proof See Appendix A.2. □

Here, we show that it applies to system (11). First, with E α = { w ∈ R , w < l − α } , for some α ∈ ] 0 , l [ , it is possible to show that Assumptions 2.1-2.2 are satisfied on R p × E α . Then, as a special case of (10), we obtain the following averaged system:

d w d t = − κ w + σ 2 2 ( l − w ) : = G ¯ ( w ) .

It remains to check that the solution of this system satisfies

∃ α > 0 , such that  w ( 0 ) < l − α ⇒ ∀ t > 0 , w ( t ) < l − α ,

that is, the subset E α is invariant under the flow of G¯.

This property is satisfied as soon as

η : = 2 σ 2 κ l 2 < 1 .

Indeed, one can show that G ¯ ( w ) = 0 admits two solutions iff η < 1 ,

w ± = l 2 ( 1 ± 1 − η ) ∈ ( 0 , l ) ,

and that w − is stable whereas w + is unstable. Thus, if w ( 0 ) < l − α with α = l − w + > 0 , then w ( t ) < l − α for all t > 0 . In fact, the invariance property is true for all α ∈ ] l − w − , l − w + [ .

We now introduce a large class of stochastic neuronal networks with learning models. They are defined as coupled systems describing the simultaneous evolution of the activity of n ∈ N neurons and the connectivity between them. We define v∈Rn, the activity field of the network, and W ∈ R n × n , the connectivity matrix.

Each neuron variable v i is assumed to follow the SDE

d v i = ( f i ( v i ) + u i ) d t + Σ ⋅ d B i ( t ) ,

where the function f i characterizes the intrinsic non-linear dynamical behavior of neuron i and u i is the input received by neuron i. The stochastic term Σ ⋅ d B i ( t ) is added to account for internal sources of noise. In terms of notations, ( B ( t ) ) t ≥ 0 is a standard n-dimensional Brownian motion, Σ is an n × n matrix, possibly function of v or other variables, and Σ⋅dBi(t) denotes the ith component of the vector Σ ⋅ d B ( t ) .

The input ui to neuron i has mainly two components: the external input u i ext and the input coming from other neurons in the network u i syn . The latter is a priori a complex combination of post-synaptic potentials coming from many other neurons. The coefficient W i j of the connectivity matrix accounts for the strength of a synapse j → i . Note that neurons can be connected to themselves, i.e., W i i is not necessarily null. Thus, we can write

u i syn : = S ( ∑ j = 1 n W i j H ( v i , v j ) ) ,

where S : R → R and ℋ is a function taking the history of vi and v j and returning a real for each time t (to take convolutions into account). In practical cases, they are often taken to be sigmoidal functions. We abusively redefine S and ℋ as vector valued operators corresponding to the element-wise application of their real counterparts. We also define the function F : R n → R n such that F ( v ) i = f i ( v i ) . Together with a slow generic learning rule, this leads to defining a stochastic learning model as the following system of SDEs.

Definition 3.1 { d v ϵ = 1 ϵ [ F ( v ϵ ) + S ( W ϵ ⋅ H ( v ϵ ) ) + u ext ( t ) ] d t + 1 ϵ Σ ( v ϵ , W ϵ ) ⋅ d B ( t ) , d W ϵ = G ( W ϵ , v ϵ ) d t .

Before applying the general theory of Section 2, let us make several comments about this generic model of neural network with learning. This model is a non-autonomous, stochastic, non-linear slow-fast system.

In order to apply Theorem 2.2, one needs Assumptions 2.1, 2.2, and 2.3 to be satisfied, restricting the space of possible functions S, ℋ, ℱ, Σ, and G. In particular, Assumption 2.2(ii) seems rather restrictive since it excludes systems with multiple equilibria and suggests that the general theory is more suited to deal with rate-based networks. However, one should keep in mind that these assumptions are only sufficient, and that the double averaging principle may work as well in systems which do not satisfy readily those assumptions.

As we will show in Section 3.3, a particular form of history-dependence can be taken into account, to a certain extent. Indeed, for instance, if the function ℱ is actually a functional of the past trajectory of variable v ϵ which can be expressed as the solution of an additional SDE, then it may be possible to include a certain form of history-dependence. However, purely time-delayed systems do not enter the scope of this theory, although it might be possible to derive an analogous averaging method in this framework.

The noise term can be purely additive or set by a particular function Σ ( v , W ) as long as it satisfies Assumption 2.2(i), meaning that it must be uniformly non-degenerate.

In the following subsection, we apply the averaging theory to various combinations of neuronal network models, embodied by choices of functions S, ℋ, ℱ, Σ, and various learning rules, embodied by a choice of the function G. We will also analyze the obtained averaged system, describing the slow dynamics of the connectivity matrix in the limit of perfect time-scale separation and, in particular, study the convergence of this averaged system to an equilibrium point.

One of the simplest, yet non-trivial, stochastic learning models is obtained when considering

• A linear model for neuronal activity, namely f i ( v i ) = − l v i with l a positive constant.

• A linear model for the synaptic transmission, namely S ( v i ) = v i and H ( v i , v j ) = v j .

• A constant diffusion matrix Σ (additive noise) proportional to the identity Σ = σ I d (spatially uncorrelated noise).

• A Hebbian learning rule with linear decay, namely G i j ( W , v ) = − κ W i j + v i v j . Actually, it corresponds to the tensor product: { v ⊗ v } i j = v i v j .

This model can be written as follows:

{ d v ϵ = 1 ϵ 1 ( − L ⋅ v ϵ + W ϵ ⋅ v ϵ + u ( t ϵ 2 ) ) d t + σ ϵ 1 d B ( t ) , d W ϵ d t = G ( v ϵ , W ϵ ) = − κ W ϵ + v ϵ ⊗ v ϵ ,

where neurons are assumed to have the same decay constant: L = l I d ; u is a periodic continuous input (it replaces u ext in the previous section); σ , ϵ 1 , ϵ 2 , κ ∈ R + with ϵ 1 , ϵ 2 ≪ 1 and B(t) is n-dimensional Brownian noise.

The first question that arises is about the well-posedness of the system: What is the definition interval of the solutions of system (12)? Do they explode in finite time? At first sight, it seems there may be a runaway of the solution if the largest real part among the eigenvalues of W grows bigger than l. In fact, it turns out this scenario can be avoided if the following assumption linking the parameters of the system is satisfied.

Assumption 3.1 There exists p ∈ ] 0 , 1 [ such that

( σ 2 l 2 p ( 1 − p ) + u m 2 p ( 1 − p ) 2 ) < κ l 3 ,

where u m = sup t ∈ R + ∥ u ( t ) ∥ 2 .

It corresponds to making sure the external (i.e., u m ) or internal (i.e., σ) excitations are not too large compared to the decay mechanism (represented by κ and l). Note that if p∈]0,1[, um and d are fixed, it is sufficient to increase κ or l for this assumption to be satisfied.

Under this assumption, the space

E p = { W ∈ R n × n : W  is symmetric , W ≥ 0  and  W < p L }

is invariant by the flow of the averaged system G¯, where W ≥ 0 means W is semi-definite positive and W < p L means p L − W is definite positive. Therefore, the averaged system is defined and bounded on R + . The slow/fast system being asymptotically close to the averaged system, it is therefore asymptotically well-defined in probability. This is summarized in the following theorem.

Theorem 3.1 If Assumption 3.1 is verified for p ∈ ] 0 , 1 [ , then system (12) is asymptotically well posed in probability and the connectivity matrix W ϵ , the solution of system (12), converges to W in the sense that for all δ , T > 0 ,

lim ϵ → 0 μ P [ sup t ∈ [ 0 , T ] | W t ϵ − W t | 2 > δ ] = 0 ,

where W is the deterministic solution of

d W i j d t = G ¯ ( W ) i j = − κ W i j ︸ decay + μ τ ∫ 0 τ μ v ¯ i ( s ) v ¯ j ( s ) d s ︸ correlation + σ 2 2 ( L − W ) i j − 1 ︸ noise ,

where v ¯ ( t ) is the τμ-periodic attractor of d v ¯ d t = ( W − L ) ⋅ v ¯ + u ( μ t ) , where W ∈ R n × n is supposed to be fixed.

Proof See Theorem B.1 in Appendix B.2. □

In the following, we focus on the averaged system described by (13). Its right-hand side is made of three terms: a linear and homogeneous decay, a correlation term, and a noise term. The last two terms are made explicit in the following.

In this section, we study an improvement of the learning model by adding a certain form of history dependence in the system and explain the way it changes the results of the previous section. Given that Theorem 2.2 only applies to an instantaneous process, we will only be able to treat the history-dependent systems which can be reformulated as instantaneous processes. Actually, this class of systems contains models which are biologically more relevant than the previous model and which will exhibit interesting additional functional behaviors. In particular, this covers the following features:

• Trace learning.

It is likely that a biological learning rule will integrate the activity over a short time. As Földiàk suggested in 27, it makes sense to consider the learning equation as being

d W ϵ d t = − κ W ϵ + ( v ϵ ∗ g 1 ) ⊗ ( v ϵ ∗ g 1 ) ,

where ∗ is the convolution and g 1 : t ∈ R ↦ β 1 e − β 1 t H ( t ) . Rolls and Deco numerically show 15 that the temporal convolution, leading to a spatio-temporal learning, makes it possible to perform invariant object recognition. Besides, trace learning appears to be the symmetric part of the biological STDP rule that we detail in Section 3.4.

• Damped oscillatory neurons.

Many neurons have an oscillatory behavior. Although we cannot take this into account in a linear model, we can model a neuron by a damped oscillator, which also introduces a new important time-scale in the system. Adding adaptation to neuronal dynamics is an elementary way to implement this idea. This corresponds to modeling a single neuron without inputs by the equivalent formulations

{ d v ϵ d t = − l z ϵ , d z ϵ d t = β 2 ( v ϵ − z ϵ ) ⇔ { d v ϵ d t = − l v ϵ ∗ g 2 , where  g 2 ( t ) = β 2 e − β 2 t H ( t ) .

• Dynamic synapses.

The electro-chemical process of synaptic communication is very complicated and non-linear. Yet, one of the features of synaptic communication we can take into account in a linear model is the shape of the post-synaptic potentials. In this section, we consider that each synapse is a linear filter whose finite impulse response (i.e., the post-synaptic potential) has the shape g 3 ( t ) = β 3 e − β 3 t H ( t ) . This is a common assumption which, for instance, is at the basis of traditional rate based models; see Chapter 11 of 7.

For mathematical tractability, we assume in the following that β = β 1 = β 2 = β 3 ∈ R + such that g β = g 1 = g 2 = g 3 , i.e., the time-scales of the neurons, those of the synapses and those of the learning windows are the same. Actually, there is a large variety of temporal scales of neurons, synapses, and learning windows, which makes this assumption not absurd. Besides, in many brain areas, examples of these time constants are in the same range (≃10 ms). Yet, investigating the impact of breaking this assumption would be necessary to model better biological networks. This leads to the following system:

{ d v ϵ = 1 ϵ 1 ( ( W ϵ − L ) ⋅ v ϵ ∗ g β + u ( t ϵ 2 ) ) d t + σ ϵ 1 d B ( t ) , d W ϵ d t = − κ W ϵ + ( v ϵ ∗ g β ) ⊗ ( v ϵ ∗ g β ) ,

where the notations are the same as in Section 3.2. The behavior of a single neuron will be oscillatory damped if Δ = 1 − 4 l β is a pure imaginary number, i.e., 4 l > β . This is the regime on which we focus. Actually, the Hebbian linear case of Section 3.2 corresponds to β = + ∞ in this delayed system.

To comply with the hypotheses of Theorem 2.2 (i.e., no dependence of the history of the process), we can add a variable z to the system which takes care of integrating the variable v over an exponential window. It leads to the equivalent system (in the limit σ z → 0 )

{ d ( v ϵ z ϵ ) = 1 ϵ 1 [ ( 0 W − L β − β ) ( v ϵ z ϵ ) + ( u ( t ϵ 2 ) 0 ) ] d t + ( σ ϵ 1 d B ( t ) σ z ϵ 1 d B ( t ) ) , d W ϵ d t = − κ W ϵ + z ϵ ⊗ z ϵ .