Neuro-Psychological Interpretation of Mathematical Results Reported in Case of Continuous-Time Hopfield Neural Networks Volume 60- Issue 3

Andreea V Cojocaru¹ and Stefan Balint²*

• ¹Independent researcher, Romania
• ²Department of Computer Science, West University of Timisoara, Romania

Received: January 17, 2025; Published: January 31, 2025

*Corresponding author: Stefan Balint, Department of Computer Science, West University of Timisoara, Blvd. V. Parvan 4, 300223 Timisoara, Romania

DOI: 10.26717/BJSTR.2025.60.009450

Abstract PDF

In this paper, for the mathematical description of electrical phenomena (voltage state) appearing in nervous system, continuous-time Hopfield neural network is used. The equilibriums of the continuous-time Hopfield neural network are interpreted as equilibriums of the nervous system. An equilibrium for which the steady state is locally exponentially stable is interpreted as robust equilibrium of the nervous system. That is because a small perturbation of steady voltage, the network recover the steady voltage. A path of equilibrium states which steady states are locally exponentially stable is interpreted as a path of robust equilibriums of the nervous system. This is a way to follow in healthcare for conduct the nervous system from a pathologic robust equilibrium into in a non-pathologic robust equilibrium. For illustration, path of robust equilibriums to follow in nervous control is computed.

Keywords: Continuous-Time Hopfield Neural Network; Nervous System; Robust Equilibrium; Fragile Equilibrium, Repulsive Equilibrium, Nervous System Control

MSC: 37B25; 62M45; 65P20; 92B20

Continuous-time Hopfield neural networks claims to be mathematical descriptions of electrical phenomena appearing in nervous system. If such neural network is used to describe associative memories, several locally exponentially stable equilibriums are desired for one external electrical input, as they store information and constitute distributed and parallel neural memory networks. In this case, the purpose of the mathematical model is the description of the locally exponentially stable steady states (existence, number, regions of attraction, bifurcation) so as to ensure the recall capability. Mathematical results on the existence, number, regions of attraction, estimation of the local convergence rate, and bifurcation in the case of continuous- time Hopfield neural networks are given in Balint, et al. [1]. Our aim here is to present neuro-psychological interpretations of some reported mathematical results presented in Balint, et al. [1] and in the papers referred herein. Namely we want to answer the following questions: what represent an equilibrium of the neural network for the nervous system? which kind of equilibriums exists? when a nervous system has one or several equilibrium for the same external electrical input? what means for a nervous system the transfer of a pathologic equilibrium into a non-pathologic equilibrium? what is the importance of the local exponential stability of a steady state in case of transfer?

what represent for a nervous system a path of locally exponentially steady states? The answer to these questions can be important for the computation of path of equilibriums having locally exponentially stable steady states for use them in healthcare.

According to Balint, et al. [1] formula (5.1) a continuous-time Hopfield-type neural network, describing the voltage evolution in a network of biological neurons, is a system of nonlinear differential equations of the form

where: a_i > 0, li are constants: a_i related to the neuron i membrane capacity and l_i related to the external electrical input, is a constant matrix referred to as the interconnection matrix, g : j R→ R , j=1...n represent the neuron input-output activations. The activation functions are bounded and without restraining generality, we may suppose that for any s∈R , j=1...n . If it not mentioned otherwise, it is assumed that g_j (0) = 0 , for j=1...n . The activation functions are increasing and have bounded derivatives. More precisely, there exist k_j > 0 such that for any s∈R, , j=1...n . Frequently the activation functions are assumed to verify and .

The system (2.1) can be written in matrix form:

By definition an equilibrium of (2.3) is a solution of the equation:

In other words an equilibrium E is a couple ( X , I ) from Rⁿ × Rⁿ which verifies (2.4). An equilibrium E=( X , I ) (if exist) then I is called external electrical input and X is called steady state .The name steady state is justified by the fact that if E⁰ = (X⁰ , I⁰ ) is an equilibrium then for I equal to I⁰ the solution of (2.3) which verifies the initial condition X(t₀)=X⁰is constant equal to X⁰ . According to Balint, et al. [1]. for any given voltage steady state X taking the external electrical input given by the formula :

an equilibrium E=( X , I ) is obtained.

On the other hand theoretically it can happen that for a given input I⁰ there is no, there exist one, or there exits several different voltage states X^j , J =1...m such that E^j = (X^j , I⁰ ) for J =1...m are equilibriums for (2.3).

A natural neuro-psychological interpretation of the concept of equilibrium E⁰ = ( X⁰ , I⁰ ) of the neural network is that it represent an equilibrium of the nervous system. Hence, come the idea that in order to change a (none desired) pathological equilibrium E⁰ = ( X⁰ , I⁰ ) of the nervous system, a new external electrical input I¹ has to be applied. If the voltage component of the new non pathologic equilibrium is X¹ then it is natural to think that the new external electrical input I¹ ,which has to be applied, has to be taken according to the formula (2.5) i.e. I¹ = −A× X¹ −T ×G(X¹ ) hoping that after the change I⁰ → I¹ of the external electrical input , the voltage X⁰ of the nervous system evolve to the voltage X¹ of the nervous system. Mathematically this neuro-psychological though is correct if after the moment t₁ when the change I⁰ → I¹ takes place the voltage state of the neural network, described by the solution of the initial value problem

tends to the voltage state X¹ .

This kind of reasoning make sense if I¹ ≠ I⁰ . That is because if I¹ = I⁰ then there is no change in input and the voltage state of the neural network will rest in the state X⁰ i.e. the voltage state evolution of the neural network is described by (3.1) is constant equal to X 0 . Moreover, even if I¹ = I⁰ and the reasoning make sense, it can happen that for the new electrical input I¹ ,beside the desired voltage state X¹ , there exist a second voltage state X^², and applying the electrical input I¹ beside the non-pathological equilibrium E¹ = (X¹, I¹ ) a second equilibrium E2 = (X2 , I¹ ) appear. It can happen that the equilibrium (X2 , I¹ ) is pathologic too. Therefore the problem is to find supplementary condition assuring that the solution of the initial value problem (3.1) tends to X¹ as it was planned.

In Balint, et al. [1] provide computational simulation of the above-described phenomena.

The neural network (5.2) considered in Balint, et al. [1] is defined by the system of differential equations:

For obtain the prior given steady voltage state X=(1,2)^T according to (2.5) the external input which has to be applied is I=( 0.514616132,0.803433824)^T − . In order to see the voltage evolution of the neural network the following initial value problem has to be solved:

The solution of the initial value problem (3.3) is represented on the Figures 1 & 2. These figures show that maintaining the external input value in the first five seconds the voltage of the neural network is constant equal to the initial value and after that oscillate around the initial value with an amplitude less than 10-6. This means that practically there is no change in the voltage of the neural network. According to the neuro-psychological interpretation, this type of the neural network voltage behavior indicates that EE = ( X , I ) is an equilibrium of the corresponding nervous system. If the steady state E = ( X , I ) is pathologic then a neurological or psychological intervention is needed. The change of the external electrical input represents a type of intervention. Assume that the medical decision is to transform the pathologic equilibrium E = ( X , I ) with X=(1,2)^T and I=( 0.514616132,0.803433824)^T − into the new non-pathologic equilibrium E¹ = ( X⁰ , I¹ ) with X⁰ (0,0)^T = and I¹ (0,0)^T = computed using formula (2.5). Before describing the effect of the external input change I→I¹ remark that for the new external input I¹ = (0,0)^T the system (3.2) possesses several equilibriums: E¹ ((0,0)^T ,(0,0)^T ) , E² = ((l n 4,l n 4)^T, (0,0)^T ) , E³ =((−l n 4,−l n 4)^T, (0,0)^T ). In case of the above equilibriums the steady states can be obtained solving the system of nonlinear algebraic equations:

Figure 1

Figure 2

The effect of the external input change I → I¹ can be seen solving the following initial value problem:

The solution of the initial value problem (3.5) is represented on the figures (Figure 3 & 4).

Figure 3

Figure 4

These figures show that the effect of the change I → I¹ is the transfer of the steady state (1,2)^T into the steady state (l n 4,l n 4),^T and not into the steady state (0,0)^T as were expected.

The mathematical explanation is: the steady state (l n 4,l n 4),^T is locally exponentially stable and the steady state (1,2)^T belongs to the region of attraction of the steady state (l n 4,l n 4),^T . In the same time, the steady state 〖(0,0)〗^^T is unstable and repulsive. The unstable character of steady state (0,0)^T means that for any small perturbation of initial condition, the solution of the perturbed initial value problem

do not recover the steady state (0,0)^T .

The unstable character of the steady state voltage in case of equilibrium E¹ is illustrated on the next figures (Figures 5 & 6).

Figure 5

Figure 6

The above figures illustrate also the repulsive character of the equilibrium E¹ ((0,0)^T ,(0,0)^T ) That is because the solution of the initial value problem (3.6) represent also the evolution of the steady state ,X^εδ = (ε, δ)^T of an equilibrium E^ε,δ = (X^ε,δ , I^ε,δ ) for , I^εδ → (0,0)^T .The external electrical input Iε ,δ appearing here is obtained from the steady state Xε ,δ using formula (2.5). Figures shows that the transfer , E^εδ → E¹ = (0,0)^Tis not possible because the components of the steady state Xε ,δ move away from the steady state X¹ = (0,0)^T According to the neuro-psychological interpretation of equilibrium, is important to keep in mind that in a nervous system there are three types of equilibriums: -Equilibriums of first type for which after a small perturbation of the steady state the nervous system return to the equilibrium. This is the situation if the steady state of the corresponding neural network is locally exponentially stable. (as is the equilibrium E² = E² = ((l n 4,l n 4),^T , I¹)) .Due to this property the nervous system return to the equilibrium automatically ,without any external input. We will say that this equilibrium of the nervous system is robust.

-Equilibriums of second type for which after a small perturbation of the steady state the nervous system do not return to the equilibrium. This is the situation if the steady state of the corresponding neural network is unstable. (as is the equilibrium E = ( X , I ) with I =(1,2)^T and I = (-0.514616132,0.803433824)^T . Due to the property that the nervous system do not return to the equilibrium automatically, without applying an external input, we will say that this type of equilibrium of the nervous system is fragile. --Equilibriums of third type having the property that there is no equilibrium, which can be transferred in such type of equilibrium. Due to this property, we will say that this type of equilibrium of the nervous system is repulsive.

A correct neuro psychological interpretation and understanding of the possible equilibriums of the neural network permit to neurologist and psychologist to choose appropriate tool in a specific case. On this basis people, working in neural and mental healthcare, can choose appropriate tool for transfer the pathologic equilibrium of a patient into a non-pathologic equilibrium. The choice of the appropriate tool means : start from a fragile or robust pathologic equilibrium E⁰ = ( X⁰ , I⁰ ) , fix a new non-pathologic steady state X1 and compute for X¹ , the corresponding new external electrical input I¹ (using formula (2.5)), build up the new non-pathologic equilibrium E¹ = (X¹, I¹ ) .After that, several computations has to be made in order to be able to say that step by step the transfer E⁰ = (X⁰, I⁰ )→ E¹ = (X¹, I¹ )is possible.

1. Verify if the equilibrium E0 is fragile or robust. A way is to solve and represent the solutions of the initial value problems

2. Verify if the equilibrium E¹ is fragile or robust. A way is to solve and represent the solutions of the initial value problems

where 1

X₁ are small perturbations of X¹ .

3. Verify if the region of attraction of the steady state X¹ contains the steady state X⁰ . A way is to solve and represent the solution of the initial value problem:

4. Verify if the region of attraction of the steady state X⁰ contains the steady state X¹ . A way is to solve and represent the solution of the initial value problem:

In order to see how this work in practice, consider the neural network (3.2) and the equilibrium . Fix the new steady state and using (2.5) compute the corresponding new external electrical input I¹ finding . So the new equilibrium is E¹ = (X¹, I¹ ).

-For test the fragility or robustness of E⁰ = (X⁰, I⁰ ) , solve and represent the initial value problem (4.1). Taking for example X₁⁰ = 1.11,1.11 . The solution of the initial value problem (4.1) is presented on the next figures (Figures 7 & 8):

Figure 7

Figure 8

These figures suggest that the equilibrium E⁰ is robust.

-For test the fragility or robustness of E¹ = (X¹, I¹ ), solve and represent the initial value problem (4.2).

Taking for example the solution of the initial value problem (4.2) is presented on the next figures (Figures 9 & 10).

Figure 9

Figure 10

Figure 11

These figures suggest that the equilibrium E¹ is robust.

-For test if the change of the external electrical input I⁰ → I¹ lead to the transfer E⁰ → E¹ the initial value problem (4.3) has to be solved.

The solution of the initial value problem (4.3) is represented on the next figures (Figures 11 & 12).

Figure 12

These figures suggest that the change of the external electrical input I⁰ → I¹ lead to the transfer E⁰ → E¹ .

For test if the change of the external electrical input I¹ → I ⁰ lead to the transfer E¹ → E⁰ the initial value problem (4.4) has to be solved.

The solution of the initial value problem (4.4) is represented on the next figures (Figures 13 & 14).

Figure 13

Figure 14

These figures suggest that the change of the external electrical input I<sup>1 → I<sup>0 lead to the transfer E¹ → E⁰ .

Remember that the starting motivation of the above presented computations was that the equilibrium E⁰ is pathologic and the equilibrium E¹ is non-pathologic and an external medical intervention is necessary. We underline that in general the situation is much more complex concerning the configuration of the equilibriums, the type of equilibrium (robust, fragile, and repulsive) and the transfer of an equilibrium into another equilibrium. In the following, we present results from Balint, et al. [1], which can offer an overview about the complexity. Illustrative computational examples are given. In Balint, et al. [1] states: If Δ is a rectangle in Rⁿ ,(i.e. for i=1…n there exist , such that and det(A+T ×DG(X )) ≠ 0 for any X ∈Δ ,then the function I_Δ ,the restriction of the external electrical input function I(X) = −A× X −T ×G(X ) , is injective. This theorem reveal that in a prior given rectangle Δ (included in Rn ) if det(A+T ×DG(X )) ≠ 0 for any X ∈Δ , then for any X⁰ ∈Δ the input I (X⁰) = −A× X⁰ −T ×G(X⁰ ) is unique. Therefore, the equilibrium E⁰ = (X⁰ , I⁰ ) of the nervous system is unique. This situation is completely different from that described in Example (5.2) pg.184 [1] where in case of the rectangle Δ = (−1.5,1.5)×(−1.5,1.5) for the input I 0 = (0,0)T three different equilibriums and exists each of them having the steady state in Δ . A numerical illustration of the situation described by Theorem 5.3. pg. 175[1]. can be found in Example 5.1. pg. 183 [1]. In this example the following one- dimensional neural continuous-time Hopfield neural network is considered:

where a > 0,T and I are constants.

For any state x∈Δ = (−∞,∞) the external input I (x) for which x is a steady state of (4.5) is

In Balint and..2008 theorem 5.4. pg. 176 states: For any prior given external electrical input X ∈Rn nthe following, statements hold: -There exists at least one steady state X ∈Rⁿ in the rectangle where for i=1…n, such that F(X , I ) = 0 . -Every steady state X, solution of the equation F(X, I ) = 0 ,belongs to the rectangle Δ defined above. -If in addition det(A+T ×DG(X)) ≠ 0 for any X ∈Δ , then the equation F(X , I ) = 0 has a unique solution X ∈Δ .

This theorem clarify several things: -First, the theorem assure that applying an arbitrary prior given external electrical input I⁰ ∈Rⁿ to the nervous system there exist at least one steady state X⁰ such that E⁰ = (X⁰ , I⁰ ) is an equilibrium of the nervous system. The steady state X⁰ of the equilibrium E⁰ = (X⁰ , I⁰ ) is located in the rectangle Δ specified above. This is in fact mainly a localization of the equilibrium steady state. For find effective, the steady state X⁰ , the nonlinear algebraic equation F(X , I⁰) = 0 has to be solved in Δ .In case of the nervous system described by the one- dimensional neural network (4.5) for T = a =1 and I =1 the rectangle Δ = [−2, 2] .For the nonlinear algebraic equation −x + tanh x +1 = 0 a solution has to be searched in the rectangle Δ = [−2, 2].By solving this equation the solution x0 =1.961179751 is found. -Second, the theorem assure that applying an arbitrary prior given external electrical input I⁰ ∈Rn to the nervous system, every steady state X^0which appear due to that is in the rectangle Δ . -Third if det(A+T ×DG(X )) ≠ 0 for any X ∈Δ , (specified above) then the obtained steady state X⁰ is unique. This means that the obtained equilibrium E⁰ = (X⁰ , I⁰ ) is unique. In case of the nervous system described by the one- dimensional neural network (4.5) for T =1,a = 2, I =1 and the rectangle Δ = [−1,1] , I1(x) > 0 for any x∈Δ .Therefore the nonlinear algebraic equation −2× x + tanh x +1 = 0 has a unique solution in Δ = [−1,1] . By solving this equaton, the solution X⁰ = 0.8439469994 is found. The supplementary information is that X⁰ is unique and the equilibrium E⁰ = (X⁰ , I⁰ ) is unique. In Balint, et al. [1] states that if the neuron input-output activation functions verifies the general conditions described in section2 and for an external input I the following inequalities hold

The mathematical condition (4.6) concerns the magnitude of the external input (left hand side) and the coefficients of the neural network (right hand side). If an input I⁰ , which verifies (4.6), is applied to the nervous system then due to that, in the nervous system, 2ⁿ equilibriumsEε ,I⁰ = (X^ε,I0 , I⁰ )appear .Each steady state X^ε,I0 is unique and located in a rectangle ε Δ . This is an extremely complex configuration of steady states, which appear after applying an external electrical input I⁰ . A modified variant of the above theorem, is Theorem (5.7) pg.178 Balint, et al. [1]. According to Theorem 5.7. pg. 178. Balint and..2008, the next statement hold. If there exits α ∈(0,1) such that the functions g_i , i =1,2,...n satisfy:

ii). Every rectangle , is invariant to the voltage dynamic of the network. This theorem reveal that if the neuron input-output activations verify (4.7) and one input I⁰ ,which verifies (4.8), is applied to the nervous system then due to that in the nervous system

Each steady state Xε ,I⁰ is unique and located in a rectangle . This configuration of steady states, is similar which appear in theorem 5.6. What is new is the invariance of to the voltage dynamics. According to Theorem 5.10. pg. 179. Balint, et al. [1] if the conditions of Theorem 5.6. pg. 177 are satisfied then the steady state X^ε,I corresponding to I and belonging to is locally exponentially stable, and its region of attraction includes . A numerical illustration of the phenomena described in Theorem 5.7. pg. 178. Balint, et al. [1] is given in Example (5.3) pg.186 Balint, et al. [1]. In this example the following Hopfield neural network is considered:

there exists a unique steady state Xε ,I in every rectangle ,it is locally exponentially stable and its region of attraction includes . For it follows that for any input I such that in every rectangle there exists a unique steady state X^ε,I which is locally exponentially stable and whose region of attraction includes .The four rectangles are:

Let be { 300, 1, 2} .In the next figure the gray rectangles represent the four sets Sε The four spirals in Figure 15. represents locally exponentially stable steady states corresponding to the inputs I^u = (20×u ×cosu, 20×u ×sin u) with u∈[0,2π ] .Each spiral is a path of robust equilibriums of nervous system. In other words, each spiral is a possible way to follow in healthcare in order to transfer pathologic equilibriums in non-pathologic equilibrium. (Figure 15). Path of robust equilibriums. In the following a numerical illustration is given. For

Figure 15

For seeing that the steady state can be transferred into the steady state by making the external input change I the following initial value problem has to be solved:

The solution of (4.13) is represented in (Figures 7 & 8). (Figures 16 & 17) show that the external input change lead the steady state into the steady state 3. This example intends to illustrates the existence of several robust equilibrium paths and the equilibrium steady states transfer along the equilibrium pats. The Artificial Intelligence strategy for healthcare has to be the buildup path of locally exponentially stable steady states along which by small successive changes, the neural network voltage can be conducted, through the regions of attraction of intermediary locally exponentially stable steady states to the final non pathologic steady state.

Figure 16

Figure 17

The interpretation of mathematical results reported in Balint, et al. [1] reveal that the configuration of the possible equilibriums in nervous system is very complex. There exists different kind of equilibriums: fragile, robust and repulsive. In equilibrium, the voltage state of the neural network is constant and does not change if the external electrical input value is maintained constant. For this reason, if an equilibrium is pathologic then a neurological or psychological intervention is needed. The analysis presented in the paper reveal a way to follow in the practice for the treatment of a pathologic equilibrium. That is to connect the pathologic equilibrium with a non-pathologic equilibrium computing a path of robust equilibriums and make transfer gradually following the path. A treatment procedure usually follows such a path.

1. St Balint, L Braescu, E Kaslik (2008) Regions of attraction and applications to control theory. In: S. Sivasundaram (Edt.)., Cambridge Scientific Publishers Ltd.