Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

The Optimal Adaptive-Based Neurofuzzy Control of the 3-DOF Musculoskeletal System of Human Arm in a 2D Plane

The Optimal Adaptive-Based Neurofuzzy Control of the 3-DOF Musculoskeletal System of Human Arm in... Hindawi Applied Bionics and Biomechanics Volume 2021, Article ID 5514693, 10 pages https://doi.org/10.1155/2021/5514693 Research Article The Optimal Adaptive-Based Neurofuzzy Control of the 3-DOF Musculoskeletal System of Human Arm in a 2D Plane Amin Valizadeh and Ali Akbar Akbari Department of Mechanical Engineering, Ferdowsi University of Mashhad, Iran Correspondence should be addressed to Ali Akbar Akbari; akbari@um.ac.ir Received 7 February 2021; Revised 4 March 2021; Accepted 15 March 2021; Published 7 April 2021 Academic Editor: Fahd Abd Algalil Copyright © 2021 Amin Valizadeh and Ali Akbar Akbari. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Each individual performs different daily activities such as reaching and lifting with his hand that shows the important role of robots designed to estimate the position of the objects or the muscle forces. Understanding the body’s musculoskeletal system’s learning control mechanism can lead us to develop a robust control technique that can be applied to rehabilitation robotics. The musculoskeletal model of the human arm used in this study is a 3-link robot coupled with 6 muscles which a neurofuzzy controller of TSK type along multicritic agents is used for training and learning fuzzy rules. The adaptive critic agents based on reinforcement learning oversees the controller’s parameters and avoids overtraining. The simulation results show that in both states of with/without optimization, the controller can well track the desired trajectory smoothly and with acceptable accuracy. The magnitude of forces in the optimized model is significantly lower, implying the controller’s correct operation. Also, links take the same trajectory with a lower overall displacement than that of the nonoptimized mode, which is consistent with the hand’s natural motion, seeking the most optimum trajectory. 1. Introduction dynamic model is applied to generate joint torques in this robot [13]. The motion or force predetermined and designed by powerful controllers is used in rehabilitation applications. In many countries, population aging leads to a decrease in productivity of useful work, and this will cause serious prob- The training is an important factor for controlling the arm to lems. Many robots are designed and employed for self- achieve a static goal, and the body’s musculoskeletal system rehabilitation of elderly, disabled, damaged people in daily gradually gains this capability through interaction with the activities [1–11]. The hand is one part of the body that is fre- surrounding environment. For example, a soccer player per- forms a series of random activities to deliver the ball to the quently involved and employed in most individuals’ daily activities. Each individual performs different daily activities gate, but the more professional he becomes in this field, the such as reaching and lifting with his hand that shows the faster and more efficient he hits the ball [14, 15]. This is important role of robots designed in this field to estimate achieved by the gradual training of the muscles’ kinematic, the position of the forces exerted by the hand. There is a and the related information can be saved and used in the growing trend worldwide for the application of handling future [16]. Therefore, understanding the training mecha- machines, inspired by human arms, in all industrial sectors, nism of the musculoskeletal system of the body can lead us to carry materials from one destination to the other under to employ a powerful controller for body rehabilitation limited operating conditions. Advances in manipulators are robotics. Many researchers used the training controls, which manifested both in their high technical level and growing will gradually train the arm controller [17–21]. Golkhou et al. economy and safety [12]. In the robotic human arm, two [22] employed an improved Actor-Critic algorithm for the links are usually used as the arm and forearm segments with controller of a single-link musculoskeletal arm with two two-degree-of-freedom (DOF), and at least four muscle ele- extensor and flexor muscles during vibrational motion. A ments are used for moving it in the 2D space. The inverse CMAC controller was applied to the Critic section to 2 Applied Bionics and Biomechanics estimate the optimal activities and update the Actor section’s contractile muscle force, α expresses the activation level of coefficients. Zacharie et al. [23] applied an advanced logic- controlled muscle, and l is the contractile muscle velocity. b based neural network to a robotic hand. The logical function , c is also the muscle damping coefficients and stiffness, was determined based on the endpoint of the arm’s arbitrary respectively. Considering the number of six muscles, the trajectory in space to compute the possible conditions of the matrix form of Eq. (1) is neuron’s activity to respond to the desired field. Bouganis and Shanahan [24] presented a neural network that could automatically learn to control a robotic hand with 5 degrees _ f = f f f f f f =AU − Bl − CΔl 1 2 3 4 5 6 of freedom and the motor’s initial time conditions. Kambara 6×6 et al. [25] proposed a control model for motion training A = diagðÞ ω , ω , ⋯, ω ϵ R 1 2 6 based on the inverse static model, direct dynamic model, 6×1 U =1ðÞ ,1, ⋯,1 ϵ R and feedback control combined with Actor-Critic. Their ð2Þ model supported the trajectory prediction of a 2-DOF arm 6×6 B = diag b , b , ⋯, b ϵ R : ðÞ 1 2 6 with six artificial muscles. Thomas et al. [26] applied an 6×6 improved learning controller based on a proportional deriv- C = diag c , c , ⋯, c ϵ R ðÞ 1 2 6 ative control technique (PD) to control a robotic hand with T 6×1 Δl = diag Δl ,Δl , ⋯,Δl ϵ R ðÞ 1 2 6 four muscles for conducting the Reaching activity. Dong et al. [27, 28] implemented an adaptive sliding mode control strategy on a 2-DOF robotic hand with biarticular muscles so The following equation expresses the relation between that the dynamic parameters were updated, which caused the the position vector of the end effector of the arm and joint input disturbances and stimulations of the system to be con- angles: sidered. Zadravec et al. [29] implemented an optimal con- troller, whose cost function was to minimize the joint "# torques, on a 2-DOF robotic hand. In this study, the authors L cosðÞ θ + L cosðÞ θ + θ + L cosðÞ θ + θ + θ 1 1 2 1 2 3 1 2 3 could predict optimum trajectories along with the functional X = , L sinðÞ θ + L sinðÞ θ + θ + L sinðÞ θ + θ + θ constraints of the muscles. 1 1 2 1 2 3 1 2 3 This model requires accurate dynamic parameters; how- ð3Þ ever, accurately determine these parameters for different peo- ple is impractical. According to the literature review above, adaptability and optimality are the basic characteristics of where L , L , and L represent the first, second, and third 1 2 3 the human brain, and the lack of a powerful controller that links, respectively. θ , θ , and θ are also the relevant link’s 1 2 3 can implement the control strategy of the brain to some angle to the x-axis, the second and third link. The velocity extent is very noticeable. In the present study, first, the equa- at the end effector of the arm, which is dependent on angular tions governing the 3-link human arm’s motion and the velocities, are expressed as follows: related dynamic equations are expressed in Section 2. An adaptive neurofuzzy controller is presented in the next sec- tion. The results obtained from the simulation of controllers X = Jθ "# with/without optimization are presented in Section 4. Finally, J J J 11 12 13 the concluded remarks of this study are described in Section J = J J J 21 22 23 J = −L sin θ − L sin θ + θ − L sin θ + θ + θ ; ðÞ ðÞ ðÞ 11 1 1 2 1 2 3 1 2 3 2. The 3-DOF Human Arm Musculoskeletal Model J = −L sin θ + θ − L sin θ + θ + θ ; ðÞ ðÞ 12 2 1 2 3 1 2 3 J = −L sin θ + θ − L sin θ + θ + θ ; ðÞ ðÞ The multibody planar model of the human arm with 3-DOF 13 2 1 2 3 1 2 3 is presented in Figure 1, in which the upper arm, forearms, J = L cos θ − L cos θ + θ + L cos θ + θ + θ ; ðÞ ðÞ ðÞ 21 1 1 2 1 2 3 1 2 3 and hand are considered three rigid links. This model con- siders the planar motion around three revolute joints at the J = L cos θ + θ + L cos θ + θ + θ ; ðÞ ðÞ 22 2 1 2 3 1 2 3 shoulder, elbow, and wrist and neglects the gravitational J = L cos θ + θ + θ ðÞ 23 3 1 2 3 effects. As shown in Figure 1, this model consists of six mus- cles that can only apply tensile forces so that each joint ð4Þ rotates by some of these related muscles. Muscles are assumed to be without weight and designed based on the Hill 2×3 J ∈ R is the Jacobian matrix that shows the relation model, which are directly connected to links as [30]: between the arm’s end effector’s linear velocities and angular velocities. The length vectors of the muscles are defined as f = ω  − ω bl − ω cΔl, ω  = α:f ð1Þ i 0 ð5Þ l =½ l l l l l l , f denotes the output force of ith muscle, f is the maximum 1 2 3 4 5 6 i 0 Wrist joint Applied Bionics and Biomechanics 3 (x, y) r L 6 2 Lg Lg 2 Lg 1 L r r 1 2 Shoulder joint Figure 1: Schematic view of the 3-DOF musculoskeletal model. and time derivative of the above equation to time: _ _ l = Wθ: ð7Þ 2  3 qffiffiffiffiffiffiffiffiffiffiffiffi 1 2 2 r π − θ − acos + s − r 1 1 1 1 6 7 6×3 W ∈ R is the Jacobian matrix, which relates the mus- 6 7 6 7 qffiffiffiffiffiffiffiffiffiffiffiffi _ 6 7 cles’ contractile rate to the joints’ angular velocity, and l = π r 2 3 2 2 6 7 r + θ − acos + s − r l T 2 1 2 2 6 7 _ _ _ _ _ _ 2 s represents the stretch rate of mus- l l l l l l 6 7 1 2 3 4 5 6 6 7 6 7 6 7 qffiffiffiffiffiffiffiffiffiffiffiffi l cles. By applying the principle of virtual work, the work done 6 7 6 7 6 3 7 2 2 6 7 r π − θ − acos + s − r by muscle torque is defined as follows: 6 7 3 2 3 3 6 7 l s 6 7 3 3 6 7 6 7 = , ð6Þ 6 7 qffiffiffiffiffiffiffiffiffiffiffiffi T 6 7 6 7 τ = −W f , ð8Þ l π r 6 7 m 4 2 2 6 7 r + θ − acos + s − r 6 7 4 2 4 4 6 7 6 2 s 7 6 7 4 6 7 T 4 5 where f = is the vector rep- ½ f f f f f f 6 qffiffiffiffiffiffiffiffiffiffiffiffi7 m 1 2 3 4 5 6 π r 6 7 5 T 2 2 l 6 7 r + θ − acos + s − r resents the tensile forces of muscles and τ =½ τ τ τ is 5 3 5 5 1 2 3 6 7 2 s 6 7 the joint torque vector. As depicted in Table 1, by putting 6  7 qffiffiffiffiffiffiffiffiffiffiffiffi 4 5 muscle parameters in Eq. (6), W is defined as follows: π r 6 2 2 r + θ − acos + s − r 6 3 6 6 2 s 6 2 3 −r r 0 000 1 2 6 7 6 7 ð9Þ W = : 00 −r r 00 3 4 4 5 where r and s represent the torque surfaces, as shown in 1−6 1−6 Figure 1. The following equation is obtained by taking the 000 0 −r r 5 6 Elbow joint 4 Applied Bionics and Biomechanics such as mass and inertia made them use this controller Table 1: Numerical values related to the muscle’s geometrical shape. because it is independent of the model parameters. More- over, this controller’s generated inputs are optimum, which Muscle Value (m) is significant in the musculoskeletal system due to the biolog- l r =0:055 s =0:080 1 1 1 ical limitations of human muscle limitations. This controller is implemented for the existing 3-DOF model in this study l r =0:055 s =0:080 2 2 2 because of the advantages mentioned above. The model’s l r =0:030 s =0:120 3 3 3 endpoint has to be directed on the arbitrary trajectory for l r =0:030 s =0:120 4 4 4 all initial values in the X and Y direction by multiple muscle l r =0:035 s =0:220 contractile forces. Hence, a multiple-input and multiple- 5 5 5 output system (MIMO) consisting of muscle inputs and end- l r =0:040 s =0:250 6 6 6 point outputs should be considered. 3.1. Neurofuzzy Network. Fuzzy systems consist a fuzzifica- Using Lagrange’s equations [29] tion unit, a defuzzification unit, a fuzzy rule base, and an inference engine. The fuzzy system can be regarded as per- € _ _ _ H θ θ + H θ + C θ, θ θ = τ: ð10Þ forming a real and nonlinear mapping from an input vector ðÞ ðÞ n m x ∈ R to an output vector y = f ðxÞ ∈ R , where m and n are the dimensions of the input and output vectors, respectively. H is a symmetric matrix representing the mass momen- The bitwise interfaces of the real and fuzzy worlds are fuzzi- tum, and C is a skew-symmetric matrix of Coriolis, centrifu- fier and defuzzifier, respectively. The earlier addresses real gal, and friction torques. By substituting Eq. (8) into the inputs to the associated fuzzy sets, and the latter serves to above equation, the dynamic equations of the musculoskele- address the fuzzy sets of output variables to the associated tal system are obtained as real outputs in the reverse direction. Two types of fuzzy systems, called Takagi-Sugeno-Kang (TSK) and systems with fuzzifiers and defuzzifiers (Mam- € _ _ H θ θ + H θ + C θ, θ θ = −W f : ð11Þ ðÞ ðÞ dani), are more common in the literature, and the TSK type is used in this study for adaptive neurofuzzy control frame- work. The multi-input single-output (MISO) neurofuzzy sys- 3. Controller Design tem—including N rules—is defined as follows: Rule : if (u is A ) and if (u is A ) and … and The controller design’s main purpose is to use appropriate i 1 i1 2 i2 if (u is A ) n im motion commands for each muscle in the process of interact- then if y = G ðu , u , ⋯, u Þ ing with the environment and learning the kinematics of the i 1 2 m where i is the rule number, u are the inputs with m arm in the movement toward a fixed target. Neurofuzzy sys- number, A indicates the fuzzy set for inputs, and G which tems are a combination of neural networks with fuzzy logic im i is the linear relation of inputs evaluated as a crisp function as systems and utilized to simplify problems and apply the sub- jective, complex rules and concepts. To mimic the human G = ω + ω u +⋯+ω u : ð12Þ brain’s function in these systems, which consists of a set of i i0 i1 1 im m artificial neurons, an artificial neural network is used with Consequently, the TSK neurofuzzy output can be fuzzy logic rules. Ghanooni et al. [31] found that the adaptive expressed as multicritic neurofuzzy control framework can help identify the unknown systems and suggested that the computational M M P load required for this controller’s parameters compatibility ∑ μ G i=1 i i y = = 〠 μ G , μ u , u , ⋯, u = μ u : ðÞ i i i 1 2 m ij j is lower than the conventional neurofuzzy controllers, and ∑ μ i=1 i=1 j=1 this is one of the advantages of this controller in real-time applications. They also claimed that their controller would ð13Þ benefit from the reinforcement learning compared to super- visory learning in the online evaluation of the output, which In Eq. (12), M is the number of rules, and μ is the mem- led to the capability of controlling any uncertainty in the bership function for the ith rule. system. The inputs of the adaptive critic-based neurofuzzy con- A new structure of adaptive neurofuzzy control frame- troller applied to the endpoint of the human arm model in work composed of several inputs and outputs based on rein- this study are e , e_ , e , and e_ as x x y y forcement learning was investigated by Balaghi et al. [32]. ( ( Their study aimed to control the motion trajectory by opti- e = x − x e_ = x_ − x_ x d x d mizing a 2-DOF model of the human arm’s contractile mus- , , ð14Þ _ _ _ e = y − y e = y − y cle forces. The “critic estimates the system’s achievement,” y d y d and the “actor” updates the controller parameters by generat- ing the associated signal. They argued that the difficulty of where ðx , y Þ and ðx, yÞ are the desired and real output of determining the precise arm’s biological specification values the system in the 2-D workspace, respectively. ðx_ , y_ Þ and d Applied Bionics and Biomechanics 5 NZ P 0.8 0.6 0.4 0.2 –em 0 em Input value Figure 2: The input membership function of the TSK fuzzy system. st nd 1 critic agent 2 critic agent e 𝜔 x 1 Learning 𝜔 2 x x mechanism d � e 𝜔 y 6 y � Human Control Neuro-fuzzy controller Trajectory + Input Output arm model Output Feed back from workspace Figure 3: Controller block diagram and system critic rules. Table 2: Numerical values of the model. _ _ ðx, yÞ are also the desired and real velocity of the arm’s end- point in the task space. As a consequence, the vector form of Length Mass Inertial moment CoM position the TSK neurofuzzy output for M rules is calculated as (m) (kg) (kg·m ) (m) st ω  = μωX 0.31 1.93 0.0141 0.165 link 2 32 3 ω ω ω ω ω 1 nd 1,0 1,1 1,2 1,3 1,4 6 76 7 0.27 1.32 0.0120 0.135 6 76 7 link ::: e 6 76 7 6 76 7 nd 6 76 7 3 6 76 7 0.15 0.35 0.0010 0.075 = e_ : ½ μ μ ⋯ μ ω ω ω ω ω 1 2 M j,0 j,1 j,2 j,3 j,4 6 76 7 link 6 76 7 6 76 7 ::: e 6 76 7 4 54 5 performs decision-making with Max-Product law. Therefore, e_ ω ω ω ω ω M,0 M,1 M,2 M,3 M,4 y there are 81 rules for each controller of the TSK system. ð15Þ 3.2. Adaptive Critic. The critic agent is the main part of any learning system. Each critic agent examines a system’s state The fuzzy system in an adaptive neural network is a stan- dard TSK system, which leads to the formation of a four- by evaluating its output and generates a critic signal called r layered network. In the first layer, all inputs are directed into . The signal r is a real number in the range of [-1, 1] and is the [-1, 1] scope of the membership function. Based on implemented by the learning process to train and adjust the Figure 2, three membership functions were determined for TSK fuzzy system’s parameters to minimize the signal to reach zero value indicates that the system does not require each input and labeled using N, Z, and P, representing the negative, zero, and positive expression, respectively. Also, more training. In multicritic systems, the evaluation of a sys- the fuzzification and defuzzification process is performing tem’s performance is carried out by each agent separately. in the second and fourth layers, respectively. The third layer Accordingly, all critic signals should become zero, which Membership value 6 Applied Bionics and Biomechanics 0.66 0.64 0.62 0.6 0.58 0.56 0.54 –0.3 –0.25 –0.2 –0.15 –0.1 –0.05 X (m) Desired path No muscle optimization With muscle optimization Figure 4: The moving trajectory of the model with/without optimizing the muscular forces. indicates the critic is satisfied by the system’s performance. 3.3. Learning System. As previously described, the primary Here, two cost functions are studied to gratified the critics purpose of the learning mechanism is to minimize the error by minimizing as [33] function’scriticeffects and satisfy all critic’s criteria. In a learn- ing system, updating neurofuzzy control parameters by critical signals is called emotional training. Therefore, emotional 1 1 2 2 E = E + E = k h e + h e_ + k abs α , ð16Þ ðÞ ðÞ ðÞ e f e 1 2 f training aims to minimize the cost function E in Eq. (17). By 2 2 using the Newton gradient descent method, the variation in critic weights should conform to the following rule: where e and e_ are position and velocity tracing error of the arm’s endpoint, k and k are the critics’ weight, which e f ∂E indicates the component preferences in the cost function, Δω = −η , ð18Þ h and h scale variables bring the items in [-1, 1], and α as ∂ω 1 2 mentioned before is the activation level of controlled muscle. In Eq. (16), the second term of the right-hand side is the TSK where η is the learning rate of the corresponding neurofuzzy system’s optimization, which minimizes the muscles’ tensile controller and ω is the adjustable parameter of the controller. forces. The reform of the above equation for the number of Substituting Eq. (17) and Eq. (18) in the above equation and m muscles and s system’s output is represented as using chain rule yields in s 2 m 1 e + d e 1 j ∂ E + E i i i e f E = 〠 k + 〠 k , ð17Þ i j Δω = −η 2 e + d e_ 2 f ðÞ i i i j,max ∂ω i=1 max j=1 3 3 ∂E ∂e ∂x ∂θ ∂τ ∂f e x i i m = −η 〠 〠 where f is the contractile force of jth muscle, f is the max j j,max ∂e ∂x ∂θ ∂τ ∂f ∂ω x j i m j=1 i=1 m amount of f , k and k are the critic weights, and d is an arbi- ð19Þ j i j i 3 3 ∂e ∂E ∂y ∂θ ∂τ ∂f trary positive number. As stated, the aim of controlling the e i i m − η 〠 〠 ∂e ∂y ∂θ ∂τ ∂f ∂ω musculoskeletal system is that the arm’s endpoint reaches the y j i m j=1 i=1 m desired position simultaneously with minimizing the contractile ∂E ∂f muscle forces; thus, in Eq. (17), s =2 and m =6. The block dia- − η ,ðÞ m =1,2, ⋯,6 : ∂f ∂ω gram and critic rules of the controller are shown in Figure 3. m Y (m) Applied Bionics and Biomechanics 7 Muscle 1 Muscle 2 10 20 5 10 0 0 01 0.5 1.5 2 2.5 3 3.5 0 0.5 1 1.5 2 2.5 3 3.5 t (sec) t (sec) Muscle 3 Muscle 4 10 5 0 0 01 0.5 1.5 2 2.5 3 3.5 01 0.5 1.5 2 2.5 3 3.5 t (sec) t (sec) Muscle 5 Muscle 6 20 10 10 5 0 0 01 0.5 1.5 2 2.5 3 3.5 01 0.5 1.5 2 2.5 3 3.5 t (sec) t (sec) No muscle optimization With muscle optimization Figure 5: The input forces of each muscle during the motion. m is the number of inputs to the model and the term, and ½−100,100 are assigned to matrix ω in Eq. (15), for six mus- ∂τ /∂f is the Jacobian matrix in Eq. (7). According to the cles. These coefficients are updated by Eq. (19) in each step to minimize the cost function value. The minimization is origi- method in Ref. [33], which proposes a matrix by implement- ing a neural network, the Jacobian term ð∂θ /∂τ Þ is obtained nally conducted by minimizing the error values of e and e_ i i that finally resulted in the appropriate system’s performance. as The parameters related to the controller are selected as ∂θ i −1 ´ 8 J = ≈ H θ , ð20Þ ðÞ −4 3×3 d =1, k =10ðÞ for i =1,2 ∂τ > i i i > −5 k = ones1:6×10 ðÞ for j =6 : ð22Þ where HðθÞ is the mass momentum in the dynamic equation of the system. Also, by taking Eq. (1), into account, the term η =10 ∂f /∂ω is calculated as m m To indicate the controller’s performance without consid- ∂f ∂f ∂ω ering the effect of muscle optimization, the process is per- m m = =1 − bl − cΔl μX: ð21Þ −5 formed also with k = zeros1:6×10 ðfor j =6Þ. The model ∂ω ∂ω  ∂ω m m parameters and the values associated with the joint types of muscles are listed in Table 2 and Table 1. Eq. (19) updates the coefficients of the TSK controller as For evaluating the controller’s performance, a semicircu- the critic rule. lar trajectory is applied by the following equation in the workspace: 4. Results In this section, the 3-DOF model, along with the neurofuzzy x =0:2+0:1 cos ðÞ t critic-based controller allocated individually to each of the , x , y = −0:1,0:55 : ð23Þ ðÞðÞ 0 0 y =0:55 + 0:1 sin ðÞ t muscles, is simulated numerically. First, the limits of mem- bership functions e and e_ are determined for the TSK system. Simulation of the model by arbitrary shows that the values of The total simulation time is assumed to be T = π ðsÞ, and _ _ e = e =0:2, as well as e = e =0:4, can be acceptable. In the during the aforementioned period, the model is expected to x y x y next step, the initial values selected randomly in the range of fully go through the trajectory. To show the controller’s NN N N N N 8 Applied Bionics and Biomechanics 2.5 1.5 0.5 –0.5 –1 –1.5 –2 0 0.5 1 1.5 2 2.5 3 3.5 t (sec) 𝜃1 with opt. 𝜃1 no opt. 𝜃2 with opt. 𝜃2 no opt. 𝜃3 with opt. 𝜃3 no opt. Figure 6: The displacement of each joint during motion. robustness against the system uncertainties, a 10% diversion 5. Conclusion is considered for the values of the mass and inertia in the model. Figure 4 displays the arm model’s motion trajectory The given trajectory was followed properly by controllers in both cases with and without considering muscles’ optimi- with/without muscle optimization. However, the tracking zation. As it is depicted, both models can follow the desired error was slightly lower in the absence of optimization, trajectory with acceptable accuracy. It should be noted that caused by the controller’s focus to track the desired trajectory the following error is a little less for nonoptimized mode. without minimizing the muscle forces. In conjunction with This is because the focus, in this case, is only on reducing the optimal controller, the muscle forces were much lower the trajectory error, and the model is not seeking to optimize than those of the nonoptimal controller, suggesting a signifi- the muscle forces. cant role of muscle optimization in improving the control- Figure 5 shows the magnitude of forces applied to each ler’s performance. The maximum values of muscle forces muscle during the motion. The muscle forces’ values are sig- were also in the desired range and well-controlled. This lim- nificantly lower in the optimized mode, showing the control- ited force is one of the main features of the optimal control ler’s correct performance. The maximum values of the forces strategy applied to the model. In the case of optimized mus- are also in the intended range and controlled properly. These cles, the joints displacement was lower, i.e., links go through limited values in muscle forces are one of the main features the trajectory with a lower overall displacement compared to that resulted from applying optimal control to the model. nonoptimized muscles case, and this shows the good agree- Finally, Figure 6 illustrates how each joint displaces dur- ment of results with the natural motion of the hand, which ing motion. The proposed values imply that the two cases is always sought the optimal trajectory of motion. We intend have select completely different configurations to go through to enable the movement of the arm exactly along complex the trajectory. In the case of optimized muscles, the displace- trajectories as well as the compensation of dominant external ment of muscles is lower, i.e., links got through the trajectory disturbances [34, 35]. Moreover, future research will mainly with a lower overall displacement than the optimized mus- aim to experimentally analyze the results obtained. The feasi- cles. The obtained results are in good agreement with the bility of the proposed neurofuzzy control system is proposed hand’s natural motion, which is always sought the optimal for future researches. The proposed neurofuzzy controller trajectory of motion. This figure shows the advantage of the should contain essential features such as adaptivity and mus- muscle optimization method. cle force optimization. Moreover, other methods such as (rad) Applied Bionics and Biomechanics 9 machine learning and artificial intelligence can be applied to [13] H. Kazerooni and R. Steger, “The Berkeley lower extremity exoskeleton,” Journal of Dynamic Systems, Measurement, and reach optimum results. Control, vol. 128, no. 1, pp. 14–25, 2006. [14] J. Konczak and J. Dichgans, “The development toward stereo- Data Availability typic arm kinematics during reaching in the first 3 years of life,” Experimental Brain Research, vol. 117, no. 2, pp. 346– The data is extracted from the paper entitled “On Control of 354, 1997. Reaching Movements for Musculo-Skeletal Redundant Arm [15] K. Tahara, S. Arimoto, M. Sekimoto, and Z. W. Luo, “On con- Model”. trol of reaching movements for musculo-skeletal redundant arm model,” Applied Bionics and Biomechanics, vol. 6, no. 1, Conflicts of Interest pp. 11–26, 2009. [16] A. R. Luft and M. M. Buitrago, “Stages of motor skill learning,” The authors declare that they have no conflicts of interest in Molecular Neurobiology, vol. 32, no. 3, pp. 205–216, 2005. relation to this research. [17] Y. Uno, M. Kawato, and R. Suzuki, “Formation and control of optimal trajectory in human multijoint arm movement,” Bio- logical Cybernetics, vol. 61, no. 2, pp. 89–101, 1989. Acknowledgments [18] P. Haggard, G. Leschziner, R. C. Miall, and J. F. Stein, “Local The authors declare that they have no financial fund from learning of inverse kinematics in human reaching movement,” any resource in relation to this research and its publication. Human Movement Science, vol. 16, no. 1, pp. 133–147, 1997. [19] M. G. Pandy, F. C. Anderson, and D. G. Hull, “A parameter optimization approach for the optimal control of large-scale References musculoskeletal systems,” Journal of Biomechanical Engineer- ing, vol. 114, no. 4, pp. 450–460, 1992. [1] E. Guizzo and H. Goldstein, “The rise of the body bots [robotic [20] E. Todorov and M. I. Jordan, “Optimal feedback control as a exoskeletons],” IEEE Spectrum, vol. 42, no. 10, pp. 50–56, 2005. theory of motor coordination,” Nature Neuroscience, vol. 5, no. 11, pp. 1226–1235, 2002. [2] B. Dellon and Y. Matsuoka, “Prosthetics, exoskeletons, and rehabilitation [grand challenges of robotics],” IEEE robotics [21] H. Miyamoto, E. Nakano, D. M. Wolpert, and M. Kawato, & automation magazine, vol. 14, no. 1, pp. 30–34, 2007. “TOPS (task optimization in the presence of signal- dependent noise) model,” Systems and Computers in Japan, [3] M. A. Oskoei and H. Hu, “Myoelectric control systems–A sur- vol. 35, no. 11, pp. 48–58, 2004. vey,” Biomedical Signal Processing and Control, vol. 2, no. 4, pp. 275–294, 2007. [22] V. GOLKHOU, C. LUCAS, and M. PARNIANPOUR, “Appli- [4] J. Rosen, M. Brand, M. B. Fuchs, and M. Arcan, “A myosignal- cation of actor-critic reinforcement learning method for con- rol of a sagittal arm during oscillatory movement,” based powered exoskeleton system,” IEEE Transactions on sys- tems, Man, and Cybernetics-part A: Systems and humans, Biomedical Engineering: Applications, Basis and Communica- tions, vol. 16, no. 6, pp. 305–312, 2004. vol. 31, no. 3, pp. 210–222, 2001. [5] K. Kiguchi, T. Tanaka, and T. Fukuda, “Neuro-fuzzy control of [23] M. Zacharie, “Advanced logistic belief neural network algo- a robotic exoskeleton with EMG signals,” IEEE Transactions rithm for robot arm control,” Journal of Computer Science, on Fuzzy Systems, vol. 12, no. 4, pp. 481–490, 2004. vol. 8, no. 6, p. 965, 2012. [6] K. Kiguchi, M. H. Rahman, M. Sasaki, and K. Teramoto, [24] A. Bouganis and M. Shanahan, “Training a spiking neural net- “Development of a 3DOF mobile exoskeleton robot for human work to control a 4-dof robotic arm based on spike timing- upper-limb motion assist,” Robotics and Autonomous Systems, dependent plasticity,” in The 2010 International Joint Confer- vol. 56, no. 8, pp. 678–691, 2008. ence on Neural Networks (IJCNN), pp. 1–8, IEEE, Barcelona, Spain, 2010, July. [7] D. Sasaki, “Development of pneumatic soft Robot hand for human friendly robot,” Journal of Robotics and Mechatronics, [25] H. Kambara, K. Kim, D. Shin, M. Sato, and Y. Koike, “Motor vol. 15, no. 2, pp. 164–171, 2003. control-learning model for reaching movements,” in The [8] J. L. Pons, E. Rocon, A. F. Ruiz, and J. C. Moreno, “Upper-limb 2006 IEEE International Joint Conference on Neural Network Proceedings, pp. 555–562, IEEE, Vancouver, BC, Canada, robotic rehabilitation exoskeleton: tremor suppression,” Reha- bilitation Robotics, vol. 1, pp. 453–470, 2007. 2006, July. [9] N. G. Tsagarakis and D. G. Caldwell, “Development and con- [26] K. M. Jagodnik, P. S. Thomas, A. J. van den Bogert, M. S. Bra- trol of a ‘soft-actuated’exoskeleton for use in physiotherapy nicky, and R. F. Kirsch, “Training an actor-critic reinforce- and training,” Autonomous Robots, vol. 15, no. 1, pp. 21–33, ment learning controller for arm movement using human- 2003. generated rewards,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 10, pp. 1892– [10] K. Kong and D. Jeon, “Design and control of an exoskeleton 1905, 2017. for the elderly and patients,” IEEE/ASME Transactions on Mechatronics, vol. 11, no. 4, pp. 428–432, 2006. [27] H. Dong and N. Mavridis, “Adaptive biarticular muscle force control for humanoid robot arms,” in 2012 12th IEEE-RAS [11] L. Lucas, M. DiCicco, and Y. Matsuoka, “An EMG-controlled hand exoskeleton for natural pinching,” Journal of Robotics International Conference on Humanoid Robots (Humanoids 2012), pp. 284–290, IEEE, Osaka, Japan, 2012, November. and Mechatronics, vol. 16, no. 5, pp. 482–488, 2004. [12] M. Blatnický, J. Dižo, M. Sága, J. Gerlici, and E. Kuba, “Design [28] H. Dong, N. Figueroa, and A. El Saddik, “Muscle force control of a mechanical part of an automated platform for oblique of a kinematically redundant bionic arm with real-time manipulation,” Applied Sciences, vol. 10, no. 23, p. 8467, 2020. parameter update,” in 2013 IEEE International Conference on 10 Applied Bionics and Biomechanics Systems, Man, and Cybernetics, pp. 1640–1647, IEEE, Man- chester, UK, 2013, October. [29] M. Zadravec and Z. Matjačić, “Planar arm movement trajec- tory formation: an optimization based simulation study,” Bio- cybernetics and Biomedical Engineering, vol. 33, no. 2, pp. 106– 117, 2013. [30] K. Tahara, Z. W. Luo, S. Arimoto, and H. Kino, “Task-space feedback control for a two-link arm driven by six muscles with variable damping and elastic properties,” in Proceedings of the 2005 IEEE International Conference on Robotics and Automa- tion, pp. 223–228, Barcelona, Spain, 2005. [31] P. Ghanooni, A. M. Yazdani, M. Amin, S. Mahmoud Zadeh, M. Ahmadi Movahed, and M. Fathi, “Robust precise trajectory tracking of hybrid stepper motor using adaptive critic-based neuro-fuzzy controller,” Computers and Electrical Engineering, vol. 81, article 106535, 2020. [32] R. Vatankhah, M. Broushaki, and A. Alasty, “Adaptive optimal multi-critic based neuro-fuzzy control of MIMO human mus- culoskeletal arm model,” Neurocomputing, vol. 173, pp. 1529– 1537, 2016. [33] B. Nasseroleslami, G. Vossoughi, M. Boroushaki, and M. Parnianpour, “Simulation of movement in three- dimensional musculoskeletal human lumbar spine using directional encoding-based neurocontrollers,” Journal of Bio- mechanical Engineering, vol. 136, no. 9, p. 091010, 2014. [34] P. Bozek, A. Lozkin, and A. Gorbushin, “Geometrical method for increasing precision of machine building parts,” Procedia Engineering, vol. 149, pp. 576–580, 2016. [35] A. Kilin, P. Bozek, Y. Karavaev, A. Klekovkin, and V. Shestakov, “Experimental investigations of a highly maneu- verable mobile omniwheel robot,” International Journal of Advanced Robotic Systems, vol. 14, pp. 1–9, 2017. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Applied Bionics and Biomechanics Hindawi Publishing Corporation

The Optimal Adaptive-Based Neurofuzzy Control of the 3-DOF Musculoskeletal System of Human Arm in a 2D Plane

Loading next page...
 
/lp/hindawi-publishing-corporation/the-optimal-adaptive-based-neurofuzzy-control-of-the-3-dof-SOAgZzULFI
Publisher
Hindawi Publishing Corporation
Copyright
Copyright © 2021 Amin Valizadeh and Ali Akbar Akbari. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
ISSN
1176-2322
eISSN
1754-2103
DOI
10.1155/2021/5514693
Publisher site
See Article on Publisher Site

Abstract

Hindawi Applied Bionics and Biomechanics Volume 2021, Article ID 5514693, 10 pages https://doi.org/10.1155/2021/5514693 Research Article The Optimal Adaptive-Based Neurofuzzy Control of the 3-DOF Musculoskeletal System of Human Arm in a 2D Plane Amin Valizadeh and Ali Akbar Akbari Department of Mechanical Engineering, Ferdowsi University of Mashhad, Iran Correspondence should be addressed to Ali Akbar Akbari; akbari@um.ac.ir Received 7 February 2021; Revised 4 March 2021; Accepted 15 March 2021; Published 7 April 2021 Academic Editor: Fahd Abd Algalil Copyright © 2021 Amin Valizadeh and Ali Akbar Akbari. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Each individual performs different daily activities such as reaching and lifting with his hand that shows the important role of robots designed to estimate the position of the objects or the muscle forces. Understanding the body’s musculoskeletal system’s learning control mechanism can lead us to develop a robust control technique that can be applied to rehabilitation robotics. The musculoskeletal model of the human arm used in this study is a 3-link robot coupled with 6 muscles which a neurofuzzy controller of TSK type along multicritic agents is used for training and learning fuzzy rules. The adaptive critic agents based on reinforcement learning oversees the controller’s parameters and avoids overtraining. The simulation results show that in both states of with/without optimization, the controller can well track the desired trajectory smoothly and with acceptable accuracy. The magnitude of forces in the optimized model is significantly lower, implying the controller’s correct operation. Also, links take the same trajectory with a lower overall displacement than that of the nonoptimized mode, which is consistent with the hand’s natural motion, seeking the most optimum trajectory. 1. Introduction dynamic model is applied to generate joint torques in this robot [13]. The motion or force predetermined and designed by powerful controllers is used in rehabilitation applications. In many countries, population aging leads to a decrease in productivity of useful work, and this will cause serious prob- The training is an important factor for controlling the arm to lems. Many robots are designed and employed for self- achieve a static goal, and the body’s musculoskeletal system rehabilitation of elderly, disabled, damaged people in daily gradually gains this capability through interaction with the activities [1–11]. The hand is one part of the body that is fre- surrounding environment. For example, a soccer player per- forms a series of random activities to deliver the ball to the quently involved and employed in most individuals’ daily activities. Each individual performs different daily activities gate, but the more professional he becomes in this field, the such as reaching and lifting with his hand that shows the faster and more efficient he hits the ball [14, 15]. This is important role of robots designed in this field to estimate achieved by the gradual training of the muscles’ kinematic, the position of the forces exerted by the hand. There is a and the related information can be saved and used in the growing trend worldwide for the application of handling future [16]. Therefore, understanding the training mecha- machines, inspired by human arms, in all industrial sectors, nism of the musculoskeletal system of the body can lead us to carry materials from one destination to the other under to employ a powerful controller for body rehabilitation limited operating conditions. Advances in manipulators are robotics. Many researchers used the training controls, which manifested both in their high technical level and growing will gradually train the arm controller [17–21]. Golkhou et al. economy and safety [12]. In the robotic human arm, two [22] employed an improved Actor-Critic algorithm for the links are usually used as the arm and forearm segments with controller of a single-link musculoskeletal arm with two two-degree-of-freedom (DOF), and at least four muscle ele- extensor and flexor muscles during vibrational motion. A ments are used for moving it in the 2D space. The inverse CMAC controller was applied to the Critic section to 2 Applied Bionics and Biomechanics estimate the optimal activities and update the Actor section’s contractile muscle force, α expresses the activation level of coefficients. Zacharie et al. [23] applied an advanced logic- controlled muscle, and l is the contractile muscle velocity. b based neural network to a robotic hand. The logical function , c is also the muscle damping coefficients and stiffness, was determined based on the endpoint of the arm’s arbitrary respectively. Considering the number of six muscles, the trajectory in space to compute the possible conditions of the matrix form of Eq. (1) is neuron’s activity to respond to the desired field. Bouganis and Shanahan [24] presented a neural network that could automatically learn to control a robotic hand with 5 degrees _ f = f f f f f f =AU − Bl − CΔl 1 2 3 4 5 6 of freedom and the motor’s initial time conditions. Kambara 6×6 et al. [25] proposed a control model for motion training A = diagðÞ ω , ω , ⋯, ω ϵ R 1 2 6 based on the inverse static model, direct dynamic model, 6×1 U =1ðÞ ,1, ⋯,1 ϵ R and feedback control combined with Actor-Critic. Their ð2Þ model supported the trajectory prediction of a 2-DOF arm 6×6 B = diag b , b , ⋯, b ϵ R : ðÞ 1 2 6 with six artificial muscles. Thomas et al. [26] applied an 6×6 improved learning controller based on a proportional deriv- C = diag c , c , ⋯, c ϵ R ðÞ 1 2 6 ative control technique (PD) to control a robotic hand with T 6×1 Δl = diag Δl ,Δl , ⋯,Δl ϵ R ðÞ 1 2 6 four muscles for conducting the Reaching activity. Dong et al. [27, 28] implemented an adaptive sliding mode control strategy on a 2-DOF robotic hand with biarticular muscles so The following equation expresses the relation between that the dynamic parameters were updated, which caused the the position vector of the end effector of the arm and joint input disturbances and stimulations of the system to be con- angles: sidered. Zadravec et al. [29] implemented an optimal con- troller, whose cost function was to minimize the joint "# torques, on a 2-DOF robotic hand. In this study, the authors L cosðÞ θ + L cosðÞ θ + θ + L cosðÞ θ + θ + θ 1 1 2 1 2 3 1 2 3 could predict optimum trajectories along with the functional X = , L sinðÞ θ + L sinðÞ θ + θ + L sinðÞ θ + θ + θ constraints of the muscles. 1 1 2 1 2 3 1 2 3 This model requires accurate dynamic parameters; how- ð3Þ ever, accurately determine these parameters for different peo- ple is impractical. According to the literature review above, adaptability and optimality are the basic characteristics of where L , L , and L represent the first, second, and third 1 2 3 the human brain, and the lack of a powerful controller that links, respectively. θ , θ , and θ are also the relevant link’s 1 2 3 can implement the control strategy of the brain to some angle to the x-axis, the second and third link. The velocity extent is very noticeable. In the present study, first, the equa- at the end effector of the arm, which is dependent on angular tions governing the 3-link human arm’s motion and the velocities, are expressed as follows: related dynamic equations are expressed in Section 2. An adaptive neurofuzzy controller is presented in the next sec- tion. The results obtained from the simulation of controllers X = Jθ "# with/without optimization are presented in Section 4. Finally, J J J 11 12 13 the concluded remarks of this study are described in Section J = J J J 21 22 23 J = −L sin θ − L sin θ + θ − L sin θ + θ + θ ; ðÞ ðÞ ðÞ 11 1 1 2 1 2 3 1 2 3 2. The 3-DOF Human Arm Musculoskeletal Model J = −L sin θ + θ − L sin θ + θ + θ ; ðÞ ðÞ 12 2 1 2 3 1 2 3 J = −L sin θ + θ − L sin θ + θ + θ ; ðÞ ðÞ The multibody planar model of the human arm with 3-DOF 13 2 1 2 3 1 2 3 is presented in Figure 1, in which the upper arm, forearms, J = L cos θ − L cos θ + θ + L cos θ + θ + θ ; ðÞ ðÞ ðÞ 21 1 1 2 1 2 3 1 2 3 and hand are considered three rigid links. This model con- siders the planar motion around three revolute joints at the J = L cos θ + θ + L cos θ + θ + θ ; ðÞ ðÞ 22 2 1 2 3 1 2 3 shoulder, elbow, and wrist and neglects the gravitational J = L cos θ + θ + θ ðÞ 23 3 1 2 3 effects. As shown in Figure 1, this model consists of six mus- cles that can only apply tensile forces so that each joint ð4Þ rotates by some of these related muscles. Muscles are assumed to be without weight and designed based on the Hill 2×3 J ∈ R is the Jacobian matrix that shows the relation model, which are directly connected to links as [30]: between the arm’s end effector’s linear velocities and angular velocities. The length vectors of the muscles are defined as f = ω  − ω bl − ω cΔl, ω  = α:f ð1Þ i 0 ð5Þ l =½ l l l l l l , f denotes the output force of ith muscle, f is the maximum 1 2 3 4 5 6 i 0 Wrist joint Applied Bionics and Biomechanics 3 (x, y) r L 6 2 Lg Lg 2 Lg 1 L r r 1 2 Shoulder joint Figure 1: Schematic view of the 3-DOF musculoskeletal model. and time derivative of the above equation to time: _ _ l = Wθ: ð7Þ 2  3 qffiffiffiffiffiffiffiffiffiffiffiffi 1 2 2 r π − θ − acos + s − r 1 1 1 1 6 7 6×3 W ∈ R is the Jacobian matrix, which relates the mus- 6 7 6 7 qffiffiffiffiffiffiffiffiffiffiffiffi _ 6 7 cles’ contractile rate to the joints’ angular velocity, and l = π r 2 3 2 2 6 7 r + θ − acos + s − r l T 2 1 2 2 6 7 _ _ _ _ _ _ 2 s represents the stretch rate of mus- l l l l l l 6 7 1 2 3 4 5 6 6 7 6 7 6 7 qffiffiffiffiffiffiffiffiffiffiffiffi l cles. By applying the principle of virtual work, the work done 6 7 6 7 6 3 7 2 2 6 7 r π − θ − acos + s − r by muscle torque is defined as follows: 6 7 3 2 3 3 6 7 l s 6 7 3 3 6 7 6 7 = , ð6Þ 6 7 qffiffiffiffiffiffiffiffiffiffiffiffi T 6 7 6 7 τ = −W f , ð8Þ l π r 6 7 m 4 2 2 6 7 r + θ − acos + s − r 6 7 4 2 4 4 6 7 6 2 s 7 6 7 4 6 7 T 4 5 where f = is the vector rep- ½ f f f f f f 6 qffiffiffiffiffiffiffiffiffiffiffiffi7 m 1 2 3 4 5 6 π r 6 7 5 T 2 2 l 6 7 r + θ − acos + s − r resents the tensile forces of muscles and τ =½ τ τ τ is 5 3 5 5 1 2 3 6 7 2 s 6 7 the joint torque vector. As depicted in Table 1, by putting 6  7 qffiffiffiffiffiffiffiffiffiffiffiffi 4 5 muscle parameters in Eq. (6), W is defined as follows: π r 6 2 2 r + θ − acos + s − r 6 3 6 6 2 s 6 2 3 −r r 0 000 1 2 6 7 6 7 ð9Þ W = : 00 −r r 00 3 4 4 5 where r and s represent the torque surfaces, as shown in 1−6 1−6 Figure 1. The following equation is obtained by taking the 000 0 −r r 5 6 Elbow joint 4 Applied Bionics and Biomechanics such as mass and inertia made them use this controller Table 1: Numerical values related to the muscle’s geometrical shape. because it is independent of the model parameters. More- over, this controller’s generated inputs are optimum, which Muscle Value (m) is significant in the musculoskeletal system due to the biolog- l r =0:055 s =0:080 1 1 1 ical limitations of human muscle limitations. This controller is implemented for the existing 3-DOF model in this study l r =0:055 s =0:080 2 2 2 because of the advantages mentioned above. The model’s l r =0:030 s =0:120 3 3 3 endpoint has to be directed on the arbitrary trajectory for l r =0:030 s =0:120 4 4 4 all initial values in the X and Y direction by multiple muscle l r =0:035 s =0:220 contractile forces. Hence, a multiple-input and multiple- 5 5 5 output system (MIMO) consisting of muscle inputs and end- l r =0:040 s =0:250 6 6 6 point outputs should be considered. 3.1. Neurofuzzy Network. Fuzzy systems consist a fuzzifica- Using Lagrange’s equations [29] tion unit, a defuzzification unit, a fuzzy rule base, and an inference engine. The fuzzy system can be regarded as per- € _ _ _ H θ θ + H θ + C θ, θ θ = τ: ð10Þ forming a real and nonlinear mapping from an input vector ðÞ ðÞ n m x ∈ R to an output vector y = f ðxÞ ∈ R , where m and n are the dimensions of the input and output vectors, respectively. H is a symmetric matrix representing the mass momen- The bitwise interfaces of the real and fuzzy worlds are fuzzi- tum, and C is a skew-symmetric matrix of Coriolis, centrifu- fier and defuzzifier, respectively. The earlier addresses real gal, and friction torques. By substituting Eq. (8) into the inputs to the associated fuzzy sets, and the latter serves to above equation, the dynamic equations of the musculoskele- address the fuzzy sets of output variables to the associated tal system are obtained as real outputs in the reverse direction. Two types of fuzzy systems, called Takagi-Sugeno-Kang (TSK) and systems with fuzzifiers and defuzzifiers (Mam- € _ _ H θ θ + H θ + C θ, θ θ = −W f : ð11Þ ðÞ ðÞ dani), are more common in the literature, and the TSK type is used in this study for adaptive neurofuzzy control frame- work. The multi-input single-output (MISO) neurofuzzy sys- 3. Controller Design tem—including N rules—is defined as follows: Rule : if (u is A ) and if (u is A ) and … and The controller design’s main purpose is to use appropriate i 1 i1 2 i2 if (u is A ) n im motion commands for each muscle in the process of interact- then if y = G ðu , u , ⋯, u Þ ing with the environment and learning the kinematics of the i 1 2 m where i is the rule number, u are the inputs with m arm in the movement toward a fixed target. Neurofuzzy sys- number, A indicates the fuzzy set for inputs, and G which tems are a combination of neural networks with fuzzy logic im i is the linear relation of inputs evaluated as a crisp function as systems and utilized to simplify problems and apply the sub- jective, complex rules and concepts. To mimic the human G = ω + ω u +⋯+ω u : ð12Þ brain’s function in these systems, which consists of a set of i i0 i1 1 im m artificial neurons, an artificial neural network is used with Consequently, the TSK neurofuzzy output can be fuzzy logic rules. Ghanooni et al. [31] found that the adaptive expressed as multicritic neurofuzzy control framework can help identify the unknown systems and suggested that the computational M M P load required for this controller’s parameters compatibility ∑ μ G i=1 i i y = = 〠 μ G , μ u , u , ⋯, u = μ u : ðÞ i i i 1 2 m ij j is lower than the conventional neurofuzzy controllers, and ∑ μ i=1 i=1 j=1 this is one of the advantages of this controller in real-time applications. They also claimed that their controller would ð13Þ benefit from the reinforcement learning compared to super- visory learning in the online evaluation of the output, which In Eq. (12), M is the number of rules, and μ is the mem- led to the capability of controlling any uncertainty in the bership function for the ith rule. system. The inputs of the adaptive critic-based neurofuzzy con- A new structure of adaptive neurofuzzy control frame- troller applied to the endpoint of the human arm model in work composed of several inputs and outputs based on rein- this study are e , e_ , e , and e_ as x x y y forcement learning was investigated by Balaghi et al. [32]. ( ( Their study aimed to control the motion trajectory by opti- e = x − x e_ = x_ − x_ x d x d mizing a 2-DOF model of the human arm’s contractile mus- , , ð14Þ _ _ _ e = y − y e = y − y cle forces. The “critic estimates the system’s achievement,” y d y d and the “actor” updates the controller parameters by generat- ing the associated signal. They argued that the difficulty of where ðx , y Þ and ðx, yÞ are the desired and real output of determining the precise arm’s biological specification values the system in the 2-D workspace, respectively. ðx_ , y_ Þ and d Applied Bionics and Biomechanics 5 NZ P 0.8 0.6 0.4 0.2 –em 0 em Input value Figure 2: The input membership function of the TSK fuzzy system. st nd 1 critic agent 2 critic agent e 𝜔 x 1 Learning 𝜔 2 x x mechanism d � e 𝜔 y 6 y � Human Control Neuro-fuzzy controller Trajectory + Input Output arm model Output Feed back from workspace Figure 3: Controller block diagram and system critic rules. Table 2: Numerical values of the model. _ _ ðx, yÞ are also the desired and real velocity of the arm’s end- point in the task space. As a consequence, the vector form of Length Mass Inertial moment CoM position the TSK neurofuzzy output for M rules is calculated as (m) (kg) (kg·m ) (m) st ω  = μωX 0.31 1.93 0.0141 0.165 link 2 32 3 ω ω ω ω ω 1 nd 1,0 1,1 1,2 1,3 1,4 6 76 7 0.27 1.32 0.0120 0.135 6 76 7 link ::: e 6 76 7 6 76 7 nd 6 76 7 3 6 76 7 0.15 0.35 0.0010 0.075 = e_ : ½ μ μ ⋯ μ ω ω ω ω ω 1 2 M j,0 j,1 j,2 j,3 j,4 6 76 7 link 6 76 7 6 76 7 ::: e 6 76 7 4 54 5 performs decision-making with Max-Product law. Therefore, e_ ω ω ω ω ω M,0 M,1 M,2 M,3 M,4 y there are 81 rules for each controller of the TSK system. ð15Þ 3.2. Adaptive Critic. The critic agent is the main part of any learning system. Each critic agent examines a system’s state The fuzzy system in an adaptive neural network is a stan- dard TSK system, which leads to the formation of a four- by evaluating its output and generates a critic signal called r layered network. In the first layer, all inputs are directed into . The signal r is a real number in the range of [-1, 1] and is the [-1, 1] scope of the membership function. Based on implemented by the learning process to train and adjust the Figure 2, three membership functions were determined for TSK fuzzy system’s parameters to minimize the signal to reach zero value indicates that the system does not require each input and labeled using N, Z, and P, representing the negative, zero, and positive expression, respectively. Also, more training. In multicritic systems, the evaluation of a sys- the fuzzification and defuzzification process is performing tem’s performance is carried out by each agent separately. in the second and fourth layers, respectively. The third layer Accordingly, all critic signals should become zero, which Membership value 6 Applied Bionics and Biomechanics 0.66 0.64 0.62 0.6 0.58 0.56 0.54 –0.3 –0.25 –0.2 –0.15 –0.1 –0.05 X (m) Desired path No muscle optimization With muscle optimization Figure 4: The moving trajectory of the model with/without optimizing the muscular forces. indicates the critic is satisfied by the system’s performance. 3.3. Learning System. As previously described, the primary Here, two cost functions are studied to gratified the critics purpose of the learning mechanism is to minimize the error by minimizing as [33] function’scriticeffects and satisfy all critic’s criteria. In a learn- ing system, updating neurofuzzy control parameters by critical signals is called emotional training. Therefore, emotional 1 1 2 2 E = E + E = k h e + h e_ + k abs α , ð16Þ ðÞ ðÞ ðÞ e f e 1 2 f training aims to minimize the cost function E in Eq. (17). By 2 2 using the Newton gradient descent method, the variation in critic weights should conform to the following rule: where e and e_ are position and velocity tracing error of the arm’s endpoint, k and k are the critics’ weight, which e f ∂E indicates the component preferences in the cost function, Δω = −η , ð18Þ h and h scale variables bring the items in [-1, 1], and α as ∂ω 1 2 mentioned before is the activation level of controlled muscle. In Eq. (16), the second term of the right-hand side is the TSK where η is the learning rate of the corresponding neurofuzzy system’s optimization, which minimizes the muscles’ tensile controller and ω is the adjustable parameter of the controller. forces. The reform of the above equation for the number of Substituting Eq. (17) and Eq. (18) in the above equation and m muscles and s system’s output is represented as using chain rule yields in s 2 m 1 e + d e 1 j ∂ E + E i i i e f E = 〠 k + 〠 k , ð17Þ i j Δω = −η 2 e + d e_ 2 f ðÞ i i i j,max ∂ω i=1 max j=1 3 3 ∂E ∂e ∂x ∂θ ∂τ ∂f e x i i m = −η 〠 〠 where f is the contractile force of jth muscle, f is the max j j,max ∂e ∂x ∂θ ∂τ ∂f ∂ω x j i m j=1 i=1 m amount of f , k and k are the critic weights, and d is an arbi- ð19Þ j i j i 3 3 ∂e ∂E ∂y ∂θ ∂τ ∂f trary positive number. As stated, the aim of controlling the e i i m − η 〠 〠 ∂e ∂y ∂θ ∂τ ∂f ∂ω musculoskeletal system is that the arm’s endpoint reaches the y j i m j=1 i=1 m desired position simultaneously with minimizing the contractile ∂E ∂f muscle forces; thus, in Eq. (17), s =2 and m =6. The block dia- − η ,ðÞ m =1,2, ⋯,6 : ∂f ∂ω gram and critic rules of the controller are shown in Figure 3. m Y (m) Applied Bionics and Biomechanics 7 Muscle 1 Muscle 2 10 20 5 10 0 0 01 0.5 1.5 2 2.5 3 3.5 0 0.5 1 1.5 2 2.5 3 3.5 t (sec) t (sec) Muscle 3 Muscle 4 10 5 0 0 01 0.5 1.5 2 2.5 3 3.5 01 0.5 1.5 2 2.5 3 3.5 t (sec) t (sec) Muscle 5 Muscle 6 20 10 10 5 0 0 01 0.5 1.5 2 2.5 3 3.5 01 0.5 1.5 2 2.5 3 3.5 t (sec) t (sec) No muscle optimization With muscle optimization Figure 5: The input forces of each muscle during the motion. m is the number of inputs to the model and the term, and ½−100,100 are assigned to matrix ω in Eq. (15), for six mus- ∂τ /∂f is the Jacobian matrix in Eq. (7). According to the cles. These coefficients are updated by Eq. (19) in each step to minimize the cost function value. The minimization is origi- method in Ref. [33], which proposes a matrix by implement- ing a neural network, the Jacobian term ð∂θ /∂τ Þ is obtained nally conducted by minimizing the error values of e and e_ i i that finally resulted in the appropriate system’s performance. as The parameters related to the controller are selected as ∂θ i −1 ´ 8 J = ≈ H θ , ð20Þ ðÞ −4 3×3 d =1, k =10ðÞ for i =1,2 ∂τ > i i i > −5 k = ones1:6×10 ðÞ for j =6 : ð22Þ where HðθÞ is the mass momentum in the dynamic equation of the system. Also, by taking Eq. (1), into account, the term η =10 ∂f /∂ω is calculated as m m To indicate the controller’s performance without consid- ∂f ∂f ∂ω ering the effect of muscle optimization, the process is per- m m = =1 − bl − cΔl μX: ð21Þ −5 formed also with k = zeros1:6×10 ðfor j =6Þ. The model ∂ω ∂ω  ∂ω m m parameters and the values associated with the joint types of muscles are listed in Table 2 and Table 1. Eq. (19) updates the coefficients of the TSK controller as For evaluating the controller’s performance, a semicircu- the critic rule. lar trajectory is applied by the following equation in the workspace: 4. Results In this section, the 3-DOF model, along with the neurofuzzy x =0:2+0:1 cos ðÞ t critic-based controller allocated individually to each of the , x , y = −0:1,0:55 : ð23Þ ðÞðÞ 0 0 y =0:55 + 0:1 sin ðÞ t muscles, is simulated numerically. First, the limits of mem- bership functions e and e_ are determined for the TSK system. Simulation of the model by arbitrary shows that the values of The total simulation time is assumed to be T = π ðsÞ, and _ _ e = e =0:2, as well as e = e =0:4, can be acceptable. In the during the aforementioned period, the model is expected to x y x y next step, the initial values selected randomly in the range of fully go through the trajectory. To show the controller’s NN N N N N 8 Applied Bionics and Biomechanics 2.5 1.5 0.5 –0.5 –1 –1.5 –2 0 0.5 1 1.5 2 2.5 3 3.5 t (sec) 𝜃1 with opt. 𝜃1 no opt. 𝜃2 with opt. 𝜃2 no opt. 𝜃3 with opt. 𝜃3 no opt. Figure 6: The displacement of each joint during motion. robustness against the system uncertainties, a 10% diversion 5. Conclusion is considered for the values of the mass and inertia in the model. Figure 4 displays the arm model’s motion trajectory The given trajectory was followed properly by controllers in both cases with and without considering muscles’ optimi- with/without muscle optimization. However, the tracking zation. As it is depicted, both models can follow the desired error was slightly lower in the absence of optimization, trajectory with acceptable accuracy. It should be noted that caused by the controller’s focus to track the desired trajectory the following error is a little less for nonoptimized mode. without minimizing the muscle forces. In conjunction with This is because the focus, in this case, is only on reducing the optimal controller, the muscle forces were much lower the trajectory error, and the model is not seeking to optimize than those of the nonoptimal controller, suggesting a signifi- the muscle forces. cant role of muscle optimization in improving the control- Figure 5 shows the magnitude of forces applied to each ler’s performance. The maximum values of muscle forces muscle during the motion. The muscle forces’ values are sig- were also in the desired range and well-controlled. This lim- nificantly lower in the optimized mode, showing the control- ited force is one of the main features of the optimal control ler’s correct performance. The maximum values of the forces strategy applied to the model. In the case of optimized mus- are also in the intended range and controlled properly. These cles, the joints displacement was lower, i.e., links go through limited values in muscle forces are one of the main features the trajectory with a lower overall displacement compared to that resulted from applying optimal control to the model. nonoptimized muscles case, and this shows the good agree- Finally, Figure 6 illustrates how each joint displaces dur- ment of results with the natural motion of the hand, which ing motion. The proposed values imply that the two cases is always sought the optimal trajectory of motion. We intend have select completely different configurations to go through to enable the movement of the arm exactly along complex the trajectory. In the case of optimized muscles, the displace- trajectories as well as the compensation of dominant external ment of muscles is lower, i.e., links got through the trajectory disturbances [34, 35]. Moreover, future research will mainly with a lower overall displacement than the optimized mus- aim to experimentally analyze the results obtained. The feasi- cles. The obtained results are in good agreement with the bility of the proposed neurofuzzy control system is proposed hand’s natural motion, which is always sought the optimal for future researches. The proposed neurofuzzy controller trajectory of motion. This figure shows the advantage of the should contain essential features such as adaptivity and mus- muscle optimization method. cle force optimization. Moreover, other methods such as (rad) Applied Bionics and Biomechanics 9 machine learning and artificial intelligence can be applied to [13] H. Kazerooni and R. Steger, “The Berkeley lower extremity exoskeleton,” Journal of Dynamic Systems, Measurement, and reach optimum results. Control, vol. 128, no. 1, pp. 14–25, 2006. [14] J. Konczak and J. Dichgans, “The development toward stereo- Data Availability typic arm kinematics during reaching in the first 3 years of life,” Experimental Brain Research, vol. 117, no. 2, pp. 346– The data is extracted from the paper entitled “On Control of 354, 1997. Reaching Movements for Musculo-Skeletal Redundant Arm [15] K. Tahara, S. Arimoto, M. Sekimoto, and Z. W. Luo, “On con- Model”. trol of reaching movements for musculo-skeletal redundant arm model,” Applied Bionics and Biomechanics, vol. 6, no. 1, Conflicts of Interest pp. 11–26, 2009. [16] A. R. Luft and M. M. Buitrago, “Stages of motor skill learning,” The authors declare that they have no conflicts of interest in Molecular Neurobiology, vol. 32, no. 3, pp. 205–216, 2005. relation to this research. [17] Y. Uno, M. Kawato, and R. Suzuki, “Formation and control of optimal trajectory in human multijoint arm movement,” Bio- logical Cybernetics, vol. 61, no. 2, pp. 89–101, 1989. Acknowledgments [18] P. Haggard, G. Leschziner, R. C. Miall, and J. F. Stein, “Local The authors declare that they have no financial fund from learning of inverse kinematics in human reaching movement,” any resource in relation to this research and its publication. Human Movement Science, vol. 16, no. 1, pp. 133–147, 1997. [19] M. G. Pandy, F. C. Anderson, and D. G. Hull, “A parameter optimization approach for the optimal control of large-scale References musculoskeletal systems,” Journal of Biomechanical Engineer- ing, vol. 114, no. 4, pp. 450–460, 1992. [1] E. Guizzo and H. Goldstein, “The rise of the body bots [robotic [20] E. Todorov and M. I. Jordan, “Optimal feedback control as a exoskeletons],” IEEE Spectrum, vol. 42, no. 10, pp. 50–56, 2005. theory of motor coordination,” Nature Neuroscience, vol. 5, no. 11, pp. 1226–1235, 2002. [2] B. Dellon and Y. Matsuoka, “Prosthetics, exoskeletons, and rehabilitation [grand challenges of robotics],” IEEE robotics [21] H. Miyamoto, E. Nakano, D. M. Wolpert, and M. Kawato, & automation magazine, vol. 14, no. 1, pp. 30–34, 2007. “TOPS (task optimization in the presence of signal- dependent noise) model,” Systems and Computers in Japan, [3] M. A. Oskoei and H. Hu, “Myoelectric control systems–A sur- vol. 35, no. 11, pp. 48–58, 2004. vey,” Biomedical Signal Processing and Control, vol. 2, no. 4, pp. 275–294, 2007. [22] V. GOLKHOU, C. LUCAS, and M. PARNIANPOUR, “Appli- [4] J. Rosen, M. Brand, M. B. Fuchs, and M. Arcan, “A myosignal- cation of actor-critic reinforcement learning method for con- rol of a sagittal arm during oscillatory movement,” based powered exoskeleton system,” IEEE Transactions on sys- tems, Man, and Cybernetics-part A: Systems and humans, Biomedical Engineering: Applications, Basis and Communica- tions, vol. 16, no. 6, pp. 305–312, 2004. vol. 31, no. 3, pp. 210–222, 2001. [5] K. Kiguchi, T. Tanaka, and T. Fukuda, “Neuro-fuzzy control of [23] M. Zacharie, “Advanced logistic belief neural network algo- a robotic exoskeleton with EMG signals,” IEEE Transactions rithm for robot arm control,” Journal of Computer Science, on Fuzzy Systems, vol. 12, no. 4, pp. 481–490, 2004. vol. 8, no. 6, p. 965, 2012. [6] K. Kiguchi, M. H. Rahman, M. Sasaki, and K. Teramoto, [24] A. Bouganis and M. Shanahan, “Training a spiking neural net- “Development of a 3DOF mobile exoskeleton robot for human work to control a 4-dof robotic arm based on spike timing- upper-limb motion assist,” Robotics and Autonomous Systems, dependent plasticity,” in The 2010 International Joint Confer- vol. 56, no. 8, pp. 678–691, 2008. ence on Neural Networks (IJCNN), pp. 1–8, IEEE, Barcelona, Spain, 2010, July. [7] D. Sasaki, “Development of pneumatic soft Robot hand for human friendly robot,” Journal of Robotics and Mechatronics, [25] H. Kambara, K. Kim, D. Shin, M. Sato, and Y. Koike, “Motor vol. 15, no. 2, pp. 164–171, 2003. control-learning model for reaching movements,” in The [8] J. L. Pons, E. Rocon, A. F. Ruiz, and J. C. Moreno, “Upper-limb 2006 IEEE International Joint Conference on Neural Network Proceedings, pp. 555–562, IEEE, Vancouver, BC, Canada, robotic rehabilitation exoskeleton: tremor suppression,” Reha- bilitation Robotics, vol. 1, pp. 453–470, 2007. 2006, July. [9] N. G. Tsagarakis and D. G. Caldwell, “Development and con- [26] K. M. Jagodnik, P. S. Thomas, A. J. van den Bogert, M. S. Bra- trol of a ‘soft-actuated’exoskeleton for use in physiotherapy nicky, and R. F. Kirsch, “Training an actor-critic reinforce- and training,” Autonomous Robots, vol. 15, no. 1, pp. 21–33, ment learning controller for arm movement using human- 2003. generated rewards,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 10, pp. 1892– [10] K. Kong and D. Jeon, “Design and control of an exoskeleton 1905, 2017. for the elderly and patients,” IEEE/ASME Transactions on Mechatronics, vol. 11, no. 4, pp. 428–432, 2006. [27] H. Dong and N. Mavridis, “Adaptive biarticular muscle force control for humanoid robot arms,” in 2012 12th IEEE-RAS [11] L. Lucas, M. DiCicco, and Y. Matsuoka, “An EMG-controlled hand exoskeleton for natural pinching,” Journal of Robotics International Conference on Humanoid Robots (Humanoids 2012), pp. 284–290, IEEE, Osaka, Japan, 2012, November. and Mechatronics, vol. 16, no. 5, pp. 482–488, 2004. [12] M. Blatnický, J. Dižo, M. Sága, J. Gerlici, and E. Kuba, “Design [28] H. Dong, N. Figueroa, and A. El Saddik, “Muscle force control of a mechanical part of an automated platform for oblique of a kinematically redundant bionic arm with real-time manipulation,” Applied Sciences, vol. 10, no. 23, p. 8467, 2020. parameter update,” in 2013 IEEE International Conference on 10 Applied Bionics and Biomechanics Systems, Man, and Cybernetics, pp. 1640–1647, IEEE, Man- chester, UK, 2013, October. [29] M. Zadravec and Z. Matjačić, “Planar arm movement trajec- tory formation: an optimization based simulation study,” Bio- cybernetics and Biomedical Engineering, vol. 33, no. 2, pp. 106– 117, 2013. [30] K. Tahara, Z. W. Luo, S. Arimoto, and H. Kino, “Task-space feedback control for a two-link arm driven by six muscles with variable damping and elastic properties,” in Proceedings of the 2005 IEEE International Conference on Robotics and Automa- tion, pp. 223–228, Barcelona, Spain, 2005. [31] P. Ghanooni, A. M. Yazdani, M. Amin, S. Mahmoud Zadeh, M. Ahmadi Movahed, and M. Fathi, “Robust precise trajectory tracking of hybrid stepper motor using adaptive critic-based neuro-fuzzy controller,” Computers and Electrical Engineering, vol. 81, article 106535, 2020. [32] R. Vatankhah, M. Broushaki, and A. Alasty, “Adaptive optimal multi-critic based neuro-fuzzy control of MIMO human mus- culoskeletal arm model,” Neurocomputing, vol. 173, pp. 1529– 1537, 2016. [33] B. Nasseroleslami, G. Vossoughi, M. Boroushaki, and M. Parnianpour, “Simulation of movement in three- dimensional musculoskeletal human lumbar spine using directional encoding-based neurocontrollers,” Journal of Bio- mechanical Engineering, vol. 136, no. 9, p. 091010, 2014. [34] P. Bozek, A. Lozkin, and A. Gorbushin, “Geometrical method for increasing precision of machine building parts,” Procedia Engineering, vol. 149, pp. 576–580, 2016. [35] A. Kilin, P. Bozek, Y. Karavaev, A. Klekovkin, and V. Shestakov, “Experimental investigations of a highly maneu- verable mobile omniwheel robot,” International Journal of Advanced Robotic Systems, vol. 14, pp. 1–9, 2017.

Journal

Applied Bionics and BiomechanicsHindawi Publishing Corporation

Published: Apr 7, 2021

References