Periodic optimal control of nonlinear constrained systems using economic model predictive control

Johannes Köhler; Matthias A. Müller; Frank Allgöwer

doi:10.1016/j.jprocont.2020.06.004

Periodic optimal control of nonlinear constrained systems using economic model predictive control

Köhler, Johannes;Müller, Matthias A.;Allgöwer, Frank 2020-05-11 00:00:00 Periodic optimal control of nonlinear constrained systems using economic model predictive control 1 2 1 Johannes Ko¨hler , Matthias A. Mu¨ller , Frank Allgo¨wer Abstract—In this paper, we consider the problem of periodic and HVAC [11], [12] due to the inherent periodicity (day- optimal control of nonlinear systems subject to online changing night cycle) in changing price signals (supply and demand) and periodically time-varying economic performance measures or dynamics (outside temperature). Even in the time-invariant using model predictive control (MPC). The proposed economic problem of maximizing the production in (nonlinear) contin- MPC scheme uses an online optimized artiﬁcial periodic orbit uous stirred-tank reactors (CSTR), periodic operation can be to ensure recursive feasibility and constraint satisfaction despite unpredictable changes in the economic performance index. We economically beneﬁcial, compare [13], [14]. Periodic oper- demonstrate that the direct extension of existing methods to ations also naturally arise in periodic/cyclic scheduling [15] periodic orbits does not necessarily yield the desirable closed- and power generation using kites [16]. In addition to the loop economic performance. Instead, we carefully revise the challenges related to dynamic/periodic operation, the external constraints on the artiﬁcial trajectory, which ensures that the operating conditions may change unpredictably and the sys- closed-loop average performance is no worse than a locally optimal periodic orbit. In the special case that the prediction tem is expected to reliably and economically operate despite horizon is set to zero, the proposed scheme is a modiﬁed version these changes. In this paper, we present an economic MPC of recent publications using periodicity constraints, with the framework that provides economic performance guarantees for important difference that the resulting closed loop has more periodic operation of nonlinear systems. degrees of freedom which are vital to ensure convergence to an Related Work: In [17], [18], [15] performance guarantees optimal periodic orbit. In addition, we detail a tailored ofﬂine computation of suitable terminal ingredients, which are both are obtained by using terminal constraints for a-priori known theoretically and practically beneﬁcial for closed-loop perfor- optimal periodic orbits. Online changes in the optimal system mance improvement. Finally, we demonstrate the practicality operation cannot be incorporated. and performance improvements of the proposed approach on In [7] an economic MPC scheme without terminal ingredi- benchmark examples. ents is studied, which has recently been extended to periodic Index Terms—Nonlinear model predictive control, economic operation [19] and time-varying problems [20]. The resulting MPC, dynamic real time optimization, periodic optimal control, closed-loop performance guarantees are, however, only valid changing economic criteria if a potentially large prediction horizon is used. In [21], [9] an economic MPC scheme based on a peri- I. INTRODUCTION odicity constraint is suggested, which minimizes the cost of a periodic orbit starting at the current measured state. The Model Predictive Control (MPC) is a well established con- convergence to the optimal periodic orbit can be theoretically trol method that can cope with nonlinear dynamics, hard state studied based on convergence results for coordinate descent and input constraints, and the inclusion of general performance methods [21, Lemma 3, Thm. 4] or online examined based criteria [1]. Economic MPC [2], [3], [4], [5] is a variant on dual variables [9, Thm. 1]. Crucially, even in the linear of MPC that directly aims at improving a user speciﬁed convex case, the closed loop does not necessarily converge to economic performance index instead of stabilizing some given the optimal periodic orbit [9, Example 6]. setpoint or trajectory. In case the system is optimally operated A promising approach to deal with online changing con- at a given setpoint, there exist well established methods to ditions is to simultaneously optimize an artiﬁcial reference, design economic MPC schemes with closed-loop performance which is a well established method in setpoint tracking guarantees [2], [6], [7], [5]. MPC [22] and has recently been extended to periodic tracking Motivation: Many practical problems require a paradigm and nonlinear systems [23], [24], [25]. This idea of using that goes beyond steady-state operation and embraces dynamic an artiﬁcial setpoint in combination with an external update operation and online changing conditions. Periodic operation scheme are used in [26], [27], [28] to design economic MPC is, e.g., economically beneﬁcial (and necessary) in water dis- schemes with performance guarantees relative to steady-state tribution networks [8], [9], electrical networks [10], buildings operation, compare also [29]. Johannes Ko¨hler and Frank Allgo¨wer are with the Institute for Systems In [8] for linear systems an artiﬁcial reference is used to Theory and Automatic Control, University of Stuttgart, 70550 Stuttgart, compute the economic optimal periodic orbit and a tracking Germany. (email:{johannes.koehler, frank.allgower}@ist.uni-stuttgart.de). 2 stage cost is used to ensure stability, compare also [30]. Matthias A. Mu¨ller is with the Institute of Automatic Control, Leibniz University Hannover, 30167 Hannover, Germany. (email:mueller@irt.uni- Recently, in [31] a nonlinear version has been proposed, hannover.de). which also allows to optimize over the period length us- This work was supported by the German Research Foundation (DFG) ing a continuous-time formulation. Furthermore, instead of a under Grants GRK 2198/1 - 277536708, AL 316/12-2, and MU 3929/1-2 - 279734922. standard tracking stage cost a regularization with respect to ©2020 Elsevier Ltd. All rights reserved. arXiv:2005.05245v2 [eess.SY] 20 Oct 2020 2 the non-periodicity in the input and economic cost is used, II. PERIODIC ECONOMIC MODEL PREDICTIVE CONTROL which also guarantees convergence to the optimal periodic In Section II-A we introduce the problem setup. Some orbit under appropriate conditions (controllability, no local existing economic MPC methods for this problem setup are minima,. . . ). However, the usage of a tracking cost/periodic brieﬂy discussed in Section II-B. We demonstrate the potential regularization can reduce the transient economic performance, difﬁculties in periodic problems (compared to steady-state) compare, .e.g. [32]. with a simple system in Section II-C. Contribution: We present an economic MPC scheme that ensures recursive feasibility, constraint satisfaction and perfor- A. Problem setup mance guarantees for nonlinear systems despite unpredictable We consider periodic problems with a ﬁxed known period changes in the economic performance index. Recursive feasi- length T ∈ N. For many systems (HVAC, water distribution bility is achieved by including an artiﬁcial periodic reference networks) this periodicity is inherent to the problem setup (in trajectory in the online optimization. We use a self-tuning the dynamics and/or cost function). In time-invariant problems weight for the cost of the artiﬁcial reference trajectory in order (chemical reactor) this period length T is a user speciﬁed to obtain suitable bounds on the closed-loop performance, as decision variable that inﬂuences the possible performance an extension to [26], [27], where this idea was introduced improvement (compared to the steady-state operation). Both with artiﬁcial setpoints. We demonstrate by means of a simple cases are illustrated in the numerical examples in Section V. motivating example (similar to [19]), that a direct extension We consider nonlinear periodically time-varying discrete- of [26], [27] to the periodic case does not necessarily yield time systems the desired closed-loop performance. Instead, we use a novel continuity condition and reformulate the constraints on the x(t + 1) = f(x(t), u(t), t) (1) optimal periodic orbit to ensure that: n m with the state x ∈ R , control input u ∈ R , and time step a) the average performance of the artiﬁcial reference con- t ∈ N. We assume that the dynamics are periodic with the verges to that of a locally optimal periodic trajectory, (known) period length T , i.e., f(x, u, t) = f(x, u, t + T ). b) the closed-loop average performance is no worse than that We impose point-wise in time constraints on the state and of the (limiting) artiﬁcial reference trajectory. input (x(t), u(t)) ∈ Z(t), with compact periodically time- n+m varying sets Z(t) ⊂ R , i.e., Z(t) = Z(t+T ). We consider In the special case of linear systems with convex cost and reference constraint sets r(t) = (x (t), u (t)) ∈ Z (t), that r r r convex constraints, the closed-loop average performance is no satisfy Z (t) = Z (t+T ) and Z (t) ⊆ int(Z(t)) for all t ≥ 0. r r r worse than the (globally) optimal periodic orbit. Furthermore, We deﬁne the set of feasible T -periodic reference trajectories if we consider a prediction horizon of N = 0, we obtain (n+m)T r = (x , u ) ∈ R as T r r T T a modiﬁed version of recent publications using periodicity constraints [21], [9], with the same number of optimization Z (t) ={r = (x , u )| T T r r T T variables and guaranteed convergence to a (local) optimum, x (0) = f(x (T − 1), u (T − 1), t + T − 1), r r r T T T which in [21], [9] can only be ensured under signiﬁcantly x (k + 1) = f(x (k), u (k), t + k), k = 0, . . . , T − 2, r r r T T T more restrictive conditions. r (k) ∈ Z (t + k), k = 0, . . . , T − 1}. T r Some of the improved performance properties require the usage of suitable terminal ingredients (economic terminal cost (n+m)T×(n+m)T We deﬁne a periodic shift matrix R ∈ R , and terminal set). To this end, we provide a novel design pro- which satisﬁes r ∈ Z (t) ⇒ R r ∈ Z (t + 1) and T T T T T cedure that is applicable to dynamic operation and economic T−1 Π R = R = I. Whenever clear from the context, j=0 T costs, as an extension of the approach in [6]. we denote the ﬁrst element of the periodic reference r by We demonstrate the practicality and performance of the (x , u ) = r = r (0). r r T proposed framework using a periodic time-varying HVAC The economic performance measure is given by a general system [33] and a time-invariant nonlinear CSTR [13]. (non-convex) periodically time-varying function Outline: Section II introduces the problem setup and ℓ(x, u, t, y) = ℓ(x, u, t + T, y), demonstrates that the periodic case requires additional care which can depend on external parameters y ∈ Y, with Y with a simple system. The proposed economic MPC frame- compact. At each time step t, the parameters y(t) ∈ Y are work with theoretical analysis is presented in Section III. assumed to be available as an external (user deﬁned) input. Additional details and variations are discussed in Section IV. These parameters might incorporate online changing prices Section V illustrates the results with numerical examples. Sec- and/or general changes in the desired production/operation. tion VI concludes the paper. Appendix A contains additional For simplicity, we assume that ℓ and f are continuous, which details regarding the CSTR example. (in combination with compact constraints) implies that ℓ is n×n Notation: The identity matrix is I ∈ R . The interior of bounded. a set X ⊂ R is int(X ). A ball with radius ǫ around a point n n x ∈ R is B (x) = {y ∈ R | kx−yk ≤ ǫ}. By K we denote Remark 1. We consider the setting with a constant parameter ǫ ∞ the class of functions α : R → R , which are continuous, y(t) in the predictions to simplify the notation. However, the ≥0 ≥0 strictly increasing, unbounded and satisfy α(0) = 0. presented guarantees hold equally if we consider a predicted 3 (periodic) sequence of parameters y(·|t). In case some of the for economic performance improvement and may hence result constraints in Z, Z are relaxed to soft constraints using in severe suboptimality, compare [32] and [5, Sec. 3.4]. penalty terms in the stage cost ℓ, the external parameters y Thus, to ensure theoretical properties (recursive feasibil- can also model online unpredictably changing constraints sets. ity, performance bounds) and allow for online performance improvement, we will use an artiﬁcial periodic reference An optimal T -periodic orbit at time t is the solution to the trajectory r ∈ Z and combine it with a purely economic T T following optimization problem formulation (without using any tracking costs), as an extension T−1 to [26], [27], [28]. In the numerical example in Section V-B, min J (r (·|t), t, y(t)) := ℓ(r (j|t), t + j, y(t)) T T T we demonstrate the advantages of such a formulation com- r (·|t)∈Z (t) T T j=0 pared to some of the existing methods. (2) and is denoted by r (·|t). If the external parameters remain C. Pitfalls - Generalized periodic constraints constant, i.e. y(t + 1) = y(t), then an optimal periodic trajec- In the following, we show that a direct (naive) extension of tory at the next time step is given by r (·|t+1) = R r (·|t), T T T existing generalized terminal setpoint constraints in [26], [27], i.e., r (1|t) = r (0|t + 1). T T [28] to periodic reference trajectories r , does not necessarily Given some initial state x(0), the closed-loop average imply the desirable economic performance guarantees and thus economic cost is deﬁned as requires further modiﬁcations (which will be introduced in K−1 Section III-A). J (x(0)) = lim sup ℓ(x(k), u(k), k, y(k)). (3) cl Consider a scalar time-invariant system with f(x, u) = u, K→∞ k=0 x ∈ {0, 1, 2} which is depicted as a graph in Figure 1 with some arbitrary small positive constant ǫ, similar to [19, The control goal is to minimize this closed-loop average Example 4]. The optimal periodic orbit is x = (1, 2) with cost (3) and achieve constraint satisfaction (x(t), u(t)) ∈ Z(t), r cost J (r ) = ǫ and T = 2. for all t ≥ 0. T T u=1 ℓ(x,u)=−1 B. Existing methods u=0 One way to approach this problem, would be to solve (2) u=1 0 1 2 ℓ(x,u)=1 ℓ(x,u)=0 ofﬂine to obtain r and then use terminal constraints, similar u=2 to [17], [18], [15]. If the parameters y stay constant, this ℓ(x,u)=1+ǫ strategy ensures the following performance bound J (x(0)) ≤ J (r )/T. (4) cl T T Fig. 1. Academic counter example - Illustration of feasible transitions. However, if the parameters y change online the performance The following economic MPC scheme with an artiﬁcial deteriorates. Directly recomputing the optimal periodic orbit periodic trajectory r can be viewed as a generalization of r and adjusting the terminal constraints based on online the steady-state methods in [26], [27], [28]: changes in the parameters y can cause feasibility issues. If N−1 there exists only a ﬁnite set of possible parameter values min ℓ(x(k|t), u(k|t)) + J (r (·|t)) (5a) T T y ∈ Y, feasible transition trajectories can be computed ofﬂine u(·|t),r (·|t) k=0 to avoid such issues, compare [34]. s.t. x(k + 1|t) = f(x(k|t), u(k|t)), x(0|t) = x(t), (5b) The issue of recomputing the optimal orbit r under (x(k|t), u(k|t)) ∈ Z, k = 0, . . . , N − 1, (5c) changing parameters y can be avoided by using economic MPC schemes without any terminal constraints [7], [19], [20] r (·|t) ∈ Z , x(N|t) = x (0|t). (5d) T T r which implicitly ﬁnd the optimal mode of operation (using This optimization problem computes an open-loop trajectory turnpike arguments). The corresponding theoretical properties x (·|t) starting at x(t) that ends on some periodic trajectory may require a-priori assumptions on the optimal mode of r ∈ Z . In closed-loop operation, the optimization prob- T T operation (dissipativity, turnpike, optimal period length T ), lem (5) would be solved in each time step t and the ﬁrst part which can be difﬁcult to verify for practical systems. In of the optimized input trajectory applied to the system, i.e., addition, a potentially very large prediction horizon N may ∗ ∗ u(t) = u (0|t), x(t + 1) = x (1|t). Although the steady-state be required to ensure these properties. schemes [26], [27], [28] often have additional modiﬁcations A reliable method to deal with these issues is to simul- (terminal cost, self-tuning weights, additional constraints on taneously optimize an artiﬁcial periodic reference r ∈ Z T T r ), the problem we discuss in the following remains the same. in the MPC problem and use a tracking formulation with Consider the initial condition x(0) = 0 and a prediction terminal constraints to stabilize this artiﬁcial reference tra- horizon of N = 2. The artiﬁcial reference is the optimal jectory, compare [8] and [22], [23], [24], [25]. This direct periodic orbit r (·|t) ∈ {1, 2}. The only feasible trajectories stabilization, however, does not take into account the potential that satisfy x(N|t) = x(2|t) ∈ {1, 2} are u(·|t) = (0, 1) and u(·|t) = (1, 2), and the corresponding open-loop cost is 1 + 0 In some cases, a multi-step MPC scheme based on the optimal period length T needs to be implemented, compare [19]. and 0 + 1 + ǫ, respectively. Thus, the optimal solution to (5) 4 satisﬁes x (1|t) = 0 = x(t). Correspondingly, the closed-loop Optimization problem: The corresponding optimization system based on (5) stays at x(t) = 0 and encounters the problem at each time step t is given by economic cost ℓ(x(t), u(t)) = 1, ∀t ≥ 0 and does not achieve N−1 the same performance as the artiﬁcial periodic reference r . min ℓ(x(k|t), u(k|t), t + k, y(t)) (6a) u(·|t),r (·|t) This issue can persist, even if we choose an arbitrarily large k=0 (even) prediction horizon N. In particular, with (5), we can + V (x(N|t), r (·|t), t + N, y(t)) (6b) f T only ensure + β(t) · J (r (·|t), t + N, y(t)) (6c) T T J (x(0)) ≤ max ℓ(r (k)) = 1 + ǫ. cl T s.t. x(k + 1|t) = f(x(k|t), u(k|t), t + k), x(0|t) = x(t), (6d) This is in contrast to existing results for the steady-state case (x(k|t), u(k|t)) ∈ Z(t + k), k = 0, . . . , N − 1, (6e) (T = 1) [26], [27], [28], which can ensure the superior bound (4). The same problem appears in economic MPC x(N|t) ∈ X (r (·|t), t + N), (6f) f T schemes without terminal constraints, compare [19, Examples r (·|t) ∈ Z (t + N), (6g) T T 4 and 18]. One way to alleviate this problem is to apply the T−1 ﬁrst T components of the open-loop input sequence u (·|t) Δκ(t) = [ℓ(r (j|t), t + N + j, y(t)) − κ (t)] , (6h) T j (multi-step MPC) [19], [35], which transforms the problem j=0 to a higher dimensional steady-state problem (T -step sys- ℓ(r (j|t), t + N + j, y(t)) ≤ κ (t) − c Δκ(t), (6i) T j κ tem [36], [19], [35]). Since we wish to consider problems with j = 0, . . . , T − 1, possibly large period lengths T , this solution seems, however, inadequate. If we would use an economic MPC scheme based with some positive constant c . The solution to (6) are optimal ∗ ∗ on periodicity constraints [21], [9], the closed-loop system state and input trajectories x (·|t), u (·|t) and an artiﬁcial would also stay at x(t) = 0 for all t ≥ 0, since there exists periodic reference trajectory r (·|t). The input trajectory only one feasible periodic orbit starting at x(0) = 0. The minimizes the predicted economic cost (6a) with a terminal theoretical results in [21] do not apply, since the one-step cost (6b), to be speciﬁed later (Ass. 1). The economic cost controllability condition [21, Lemma 4] is not satisﬁed. of the artiﬁcial periodic reference trajectory r is weighted with a self-tuning (time-varying) weight β(t) (6c), similar To summarize, as also discussed in [15], the existing ap- to [26], [27]. The resulting state and input trajectory satisfy the proaches with online optimized periodic reference trajectory dynamics (6d) and the posed state and input constraints (6e). In r do not come with any closed-loop performance guarantees addition, the terminal state of the predicted state sequence is in similar to (4). a terminal set (see Ass. 1 below) around the artiﬁcial reference trajectory (6f). The artiﬁcial reference is a feasible periodic orbit (6g). Conditions (6h)–(6i) pose additional constraints on the improvement of the economic cost of the artiﬁcial reference r compared to κ , similar to [26], [27], [28]. In T j III. PROPOSED PERIODIC ECONOMIC MPC FRAMEWORK particular, if Δκ is negative (the cost J of the reference improves), then ℓ(r (j|t), t + N + j, y(t)) can be larger than κ (t). Hence, the constraint (6i) is less restrictive than This section contains the main result of the paper. The ℓ(r (j|t), t + N + j, y(t)) ≤ κ (t). The memory states κ in T j j proposed scheme is detailed in Section III-A. Performance combination with the self-tuning weight β and the constant guarantees relative to the limiting artiﬁcial reference trajectory c (Ass. 5) are crucial to establish the desired performance are derived in Section III-B. In Section III-C, improved a guarantees and are discussed in more detail in the follow- priori performance bounds are derived based on the self- ing theoretical analysis. For notational simplicity, we deﬁne tuning weight. The theoretical properties are summarized in r (T|t) := r (0|t). T T Theorem 1. Closed-loop operation: The closed-loop system with (6) is given by x(t + 1) =f(x(t), u (0|t), t), (7a) β(t + 1) =B(β(·), κ(·), x(·)), (7b) A. Proposed scheme κ (t + 1) =ℓ(r (j + 1|t), t + N + 1 + j, y(t + 1)), (7c) j = 0, . . . , T − 1. In the following, we detail the proposed scheme and discuss the relation to other existing methods. The main idea is to At each time t, we measure a state x(t) and an external directly minimize the predicted economic stage cost ℓ with parameter y(t). As in a standard MPC framework, we apply some terminal cost V and terminal set X around the artiﬁcial the ﬁrst part of the optimized input sequence (7a). The cost f f reference trajectory r , and use an updating scheme to ensure of the last optimal artiﬁcial reference trajectory r is saved that the optimized artiﬁcial reference trajectory r converges in the memory states κ (7c). The tuning weight β(t) can T j to the best possible periodic orbit r . be determined by some general (causal) update rule B [26], T 5 [27], or simply chosen by a user as a time-varying or constant guarantees with a terminal equality constraint. signal (c.f. [26, Update rule 1] and Sec. IV-D). The constant The tracking MPC scheme [8] can be viewed as a modiﬁed c > 0 and the terminal ingredients V ,X need to be κ f f version, which uses a tracking stage cost ℓ in the optimization designed ofﬂine, which will be detailed later. Hence, compared problem (6) and hence does not require the additional tuning to a standard MPC closed loop, we have T additional scalar variable β or memory state κ . memory states κ (7c) and one scalar tuning-variable β (7b). j The standard economic MPC formulations for periodic Initialization: We assume that the initial state x(0) is such orbits [17], [18] and steady-states [2], [6] are contained as that there exists a feasible trajectory to some feasible periodic a special case, if we ﬁx the artiﬁcial reference trajectory orbit r . The memory states κ (0) can be initialized arbitrarily T j r = r . T T large, such that the constraint (6i) is inactive at t = 0. Correspondingly, the optimization problem (6) is feasible at B. Relative performance guarantees t = 0. The tuning variable β can be initialized with any positive scalar, most naturally β(0) = 1. In Proposition 1, we show that the proposed formulation is Existing schemes = special cases: In the following, we recursively feasible. For constant parameters y, Proposition 2 discuss in detail how various existing methods for economic shows that the average closed-loop performance is no worse MPC are contained in this formulation as special cases. than the performance of the limiting artiﬁcial references. The proposed formulation can best be viewed as an exten- Terminal ingredients: The following assumption captures sion to [26], [27], [28], which considers an artiﬁcial refer- the (standard) sufﬁcient conditions for the terminal ingredients. ence setpoint (T = 1). In particular, if we assume a time- invariant problem setup and choose T = 1, we get the Assumption 1. There exists a terminal set X (r , t), a f T optimization problem and closed-loop operation in [26], [27], (bounded) terminal cost V (x, r , t, y) and terminal controller f T [28]. For c ≥ 0, the constraints (6h)–(6i) are equivalent to k (x, r , t), such that at any time t ∈ N, for any parameters κ f T ℓ(r (t)) ≤ κ(t) = ℓ(r (t − 1)) which is used in [26], [27], y ∈ Y, reference r ∈ Z (t) and any x ∈ X (r , t), the T T T f T [28] to ensure that the cost of the artiﬁcial reference r is following conditions hold improving. Although one can directly see that [26], [27], [28] + + x ∈X (r , t + 1), (8a) is a special case of the posed formulation, it is not obvious (x, u) ∈Z(t), (8b) from the onset that the extension of [26], [27], [28] to periodic + + problems should be given by the optimization problem (6). V (x , r , t + 1, y) − V (x, r , t, y) (8c) f f T A more intuitive extension might be to use the constraint ≤ − ℓ(x, u, t, y) + ℓ(r (0), t, y), J (r (·|t)) ≤ κ(t) = J (r (·|t − 1)) (as an alternative T T T with x = f(x, u, t), r = R r ∈ Z (t + 1), u = to (6h)–(6i)). The possibly suboptimal performance of such T T T k (x, r , t). an approach has, however, been illustrated in Section II-C. f T In Section IV-B we show that we can guarantee the same This assumption can always be satisﬁed by using a ter- propertiesIn Section IV-B we show that we can guarantee minal equality constraint X (r , t) = {x }, k = u , with f T r f r the same properties with this more intuitive extension, if (x , u ) = r (0). However, for the improved performance r r T we instead suitably reformulate the cost function. Another guarantees discussed in Section III-C we will require stronger possible formulation for periodic orbits would be the constraint conditions for the terminal ingredients (Ass. 3, Sec. IV-A), ℓ(r (j|t)) ≤ κ (t) without the additional term c in (6i). This T j κ compare [27]. modiﬁcation is sufﬁcient to avoid the pitfall in Section II-C, if Recursive feasibility: The following proposition shows that the artiﬁcial reference is initialized as an optimal periodic orbit feasibility of the proposed scheme is independent of the r . However, this more restrictive constraint can potentially exogenous parameters y. prevent the artiﬁcial reference trajectory r to converge to the optimal periodic orbit r , compare also Ass. 5 and the T Proposition 1. Let Assumption 1 hold and assume that (6) numerical example in Section V. is feasible at t = 0. Then the optimization problem (6) is If we consider a prediction horizon of N = 0 and a terminal recursively feasible for the resulting closed-loop system (7). equality constraint X (r , t) = {x }, then the proposed f T r Proof. This result is a straightforward extension of existing formula yields a modiﬁed version of the MPC scheme using results for MPC with artiﬁcial reference trajectories [22], [23], periodicity constraints [21], [9]. The only difference would [24], [25], [26], [27], [28], [29], [21], [9], [8]. Given the be the additional performance constraints on the periodic feasible reference r (·|t) ∈ Z (t + N) at time t, the shifted orbit (6h)–(6i), which may not be necessary in many cases, reference r (·|t+1) = R r (·|t) satisﬁes (6g). This reference T T compare Section IV-B. Crucially, if we choose a suitable satisﬁes the constraints (6i) with equality, since Δκ(t+1) = 0. terminal cost with a non-empty terminal set (Ass. 3), then we A corresponding candidate input sequence is given by can establish closed-loop performance guarantees (Thm. 1), which are in general not valid for MPC schemes using peri- u (k + 1|t) k ≤ N − 2 u(k|t + 1) = . odicity constraints [21], [9]. In particular, the terminal ingre- ∗ ∗ k (x (N|t), r (·|t), t + N) k = N − 1 dients (Ass. 3) relax the one-step controllability condition [21, Lemma 4] to an incremental stabilizability condition (Ass. 3). The resulting state and input sequences satisfy the con- In Lemma 4, we discuss how to retain these performance straints (6e) and the terminal constraint (6f) due to Ass. 1. 6 Self-tuning weight: Deﬁne the change in the weight β as with κ := κ . Using (13) recursively implies T 0 γ(t) = β(t+1)−β(t). The following assumption characterizes some of the properties the update scheme B (7b) should have, κ (t + k + 1) ≤κ (t) − c Δκ(t + j) (14) T−1 k κ such that performance guarantees hold despite online changing j=0 values of β, compare [26] for a more nuanced discussion for k = 0, . . . , T − 1. Using the deﬁnition of κ in (9), we can and an alternative condition on B resulting in slightly weaker bound the T -step sum as performance guarantees.   T−1 T−1 k X (14) X X Assumption 2. [26, Ass. 1] The sequence β(·) satisﬁes   κ (t + 1 + k) ≤ κ (t) − c Δκ(t + j) T−1 k κ lim sup γ(t) ≤ 0 and γ(t) ≤ c , β(t) ≥ 0 for all t ≥ 0 t→∞ k=0 k=0 j=0 with a constant c . T−1 T−1 X X ≤ κ (t) − c T Δκ(t + k) k κ Deﬁne the cost of the artiﬁcial reference as k=0 k=0 T−1 T−1 κ(t) = κ (t) = J (r (·|t − 1), t + N − 1, y(t)). (9) (9) j T =κ(t) − c T Δκ(t + k) = κ(t) + c T (κ(t) − κ(t + T )). κ κ j=0 k=0 Suppose that the parameters y(t) remain constant, then the Thus, the closed-loop transient cost over one period T satisﬁes conditions (6i),(7c) with c ≥ 0 ensure that Δκ(t) = κ(t + t+T−1 1) − κ(t) ≤ 0 and thus κ is non-increasing. Boundedness of W (t + T ) − W (t) + ℓ(x(k), u(k), k, y(k)) (15) ℓ implies boundedness of κ(t). Thus κ(t) converges to some k=t limit κ . T−1 Average performance: The following proposition estab- ≤κ(t) + c T (κ(t) − κ(t + T )) + γ(t + k)κ(t + 1 + k). lishes that the closed-loop performance is no worse than κ k=0 (the performance of the limiting artiﬁcial trajectories r ), as Abbreviate ℓ(t) = ℓ(x(t), u(t), t, y(t)) and deﬁne κ (t) = an extension to [26, Thm. 1]. κ(t) − κ . Then (15) evaluated over a time interval K · T starting at t = 0 can be rewritten as Proposition 2. Let Assumptions 1 and 2 hold and assume that (6) is feasible at t = 0 and y(t) is constant, then W (K · T ) − W (0) (16) the resulting closed-loop system (7) satisﬁes the following K−1 performance bound ≤Kκ + c T (κ(0) − κ(T K)) + κ (k · T ) ∞ κ ǫ TK−1 ℓ(x(t), u(t), t, y(t)) k=0 t=0 lim sup ≤ κ /T. (10) KT−1 T K K→∞ + [γ(t)κ + γ(t)κ (t + 1) − ℓ(t)]. ∞ ǫ Proof. Deﬁne the value function t=0 The remainder of the proof is analogous to [26, Thm. 1]. W (t) =W (x(t), y(t), β(t), κ (t)) (11) Boundedness of ℓ, V and β(t) ≥ 0 ensures that W (T K) N−1 ∗ ∗ is lower bounded and thus = ℓ(x (k|t), u (k|t), t + k, y(t)) k=0 W (T K) − W (0) 0 ≤ lim inf . ∗ ∗ + V (x (N|t), r (·|t), t + N, y(t)) K→∞ K + β(t)J (r (·|t), t + N, y(t)). Taking averages on both sides of (16) yields W (T K) − W (0) Proposition 1 provides a feasible candidate solution u(·|t+1), 0 ≤ lim inf K→∞ K r (·|t+1) to the optimization problem (6) at time t+1. Hence, K−1 we can use the cost of the candidate solution to upper bound X c T ≤κ + lim sup κ (k · T ) + lim (κ(0) − κ(T K)) ∞ ǫ the value function W (t + 1), which in combination with y(t) K K→∞ K K→∞ k=0 constant and the terminal cost (Ass. 1) yields " # KT−1 W (t + 1) − W (t) + ℓ(x(t), u(t), t, y(t)) (12) + lim sup γ(t)κ + γ(t)κ (t + 1) ∞ ǫ K→∞ t=0 ≤ℓ(r (0|t), t + N, y(t)) + γ(t)κ(t + 1) KT−1 (7c) = κ (t + 1) + γ(t)κ(t + 1), T−1 − lim sup ℓ(t) K→∞ t=0 compare [26, Thm. 1][27, Thm. 1] for details. The deﬁni- TK−1 tion of κ in (7c), constant parameters y(t) and the con- ≤κ − lim sup ℓ(t), straints (6h),(6i) ensure K→∞ t=0 (7c) and thus (10). The ﬁrst inequality follows from (16), by using κ (t + 1) = ℓ(r (j + 1|t), t + N + j + 1, y(t)) (6i) lim inf a − b ≤ lim inf −b + lim sup a = lim sup a − lim sup b . n n n n n n ≤κ (t) − c Δκ(t), j = 0, . . . , T − 1, (13) j+1 κ n n n n n 7 The second inequality follows from γ(t) ≤ c , κ (t) ∈ [0,∞), where the last step follows using the case distinction c ≤ α/2 γ ǫ and c ≥ α and the fact that α(r ) ≥ α. Given kr −r˜ k ≤ ǫ, T T lim κ (t) = 0, lim sup γ(t) ≤ 0, lim Δκ(t) = 0. we have t→∞ t→∞ t→∞ (17c) + + + V (x , r˜ , t + 1) ≤ V (x , r , t + 1) + α (ǫ) δ T δ 4 (19) ≤ α(r ) − Δα + α (ǫ) T 4 (18) C. Improved a priori performance bounds ≤ α(r˜ ) + α (ǫ) + α (ǫ) − Δα = α(r˜ ), T 5 4 T −1 In the following, we provide sufﬁcient conditions to ensure with ǫ := (α + α ) (Δα). 4 5 that the average cost of the artiﬁcial periodic orbit converges This lemma is an extension to [27, Lemma 1] and shows that to a local minimum. the reference r can be incrementally changed in closed-loop Terminal ingredients: The following assumption is a operation without losing recursive feasibility. Similar results stronger version of Assumption 1, which is used to derive are used in nonlinear tracking MPC schemes [24], [25]. the improved performance guarantees. Self-tuning weight: Given a state x at time t, the set of periodic reference trajectories r with a terminal set X that Assumption 3. Consider the terminal set X , terminal cost T f can be reached within the prediction horizon N is deﬁned as V and controller k from Assumption 1. There exists an f f incremental Lyapunov function V (x, r , t), such that for δ T mN R (x, t) = {r ∈ Z (t + N)| ∃u ∈ R s.t. x(t) = x, N T T any time t ∈ N, any reference r , r˜ ∈ Z (t) and any T T T x(k + 1) = f(x(k), u(k), k), (x(k), u(k)) ∈ Z(k), x ∈ X (r , t), the following inequalities hold f T k = t, . . . , t + N − 1, x(N + t) ∈ X (r , t + N)}. f T + + V (x , r , t + 1) − V (x, r , t) ≤ − α (kx − x k), (17a) δ δ T 1 r Similarly, we deﬁne the set of reference trajectories that α (kx − x k) ≤ V (x, r , t) ≤α (kx − x k), (17b) 2 r δ T 3 r additionally satisfy the constraints (6h)–(6i) |V (x, r , t) − V (x, r˜ , t)| ≤α (kr − r˜ k), (17c) δ T δ T 4 T T R (x, t, y, κ ) = {r ∈ R (x, t)| s.t. r satisﬁes (6h)–(6i)}. N j T N T + + with x = f(x, u, t), u = k (x, r , t), r = R r ∈ Z (t+ f T T T T 1), (x , u ) = r (0) and functions α , α , α , α ∈ K . Given a point x ∈ R at time t with parameters y and κ , the r r T 1 2 3 4 ∞ j Furthermore, the terminal set is given by X (r , t) = {x ∈ cost of the best reachable periodic orbit is given as f T R | V (x, r , t) ≤ α(r )} and the terminal set size α(r ) δ T T T J (x, t, y, κ ) = min J (r , t + N, y). (20) T,min j T T satisﬁes r ∈R (x,t,y,κ ) T N j |α(r ) − α(r˜ )| ≤ α (kr − r˜ k), α(r ) = α(r ) ∈ [α, α] Assumption 4. The update rule B is such that for any y(t) = T T 5 T T T (18) y for all t ≥ 0 and for all sequences x(·), κ(·), it holds that with constants α, α > 0 and a function α ∈ K . 5 ∞ κ − lim inf J (x(t), t, y(t), κ (t)) > 0 ∞ T,min j t→∞ The ofﬂine design of such terminal ingredients is discussed ⇒ lim inf β(t) = ∞. t→∞ in detail in Section IV-A. The conditions (17a)–(17c) ensure The main idea is that in closed-loop operation the self- that the terminal set has a non-empty interior and that the tuning weight β increases if necessary and thus ensures that the terminal controller k stabilizes the reference r with a f T artiﬁcial trajectory converges to the optimal mode of operation, continuous incremental Lyapunov function V . compare [26], [27]. A detailed discussion on update schemes Lemma 1. Let Assumptions 1 and 3 hold. There exists a B satisfying Assumptions 2 and 4 is given in [26]. constant ǫ > 0, such that at each time t ∈ N, for any Periodic economic continuity: As discussed in Section II-C r ∈ Z (t) and any x ∈ X (r , t), it holds that T T f T and Section III-A, the constraints (6h)–(6i) are crucial for the desired properties. However, the constraint (6i) limits how x = f(x, k (x, r , t), t) ∈ X (r˜ , t + 1), f T f T the shape of the artiﬁcial reference trajectory may change. for all r˜ ∈ Z (t + 1) ∩ B (R r ). In particular, for c = 0 this constraint ensures that the T T ǫ T T κ reference can only be updated if the economic cost on all Proof. First, note that Assumption 3 ensures that the positive points of the reference trajectory does not increase. For c invariance condition (8a) is strictly satisﬁed arbitrarily large, the constraint (6i) becomes inactive, if the (17a) overall cost of the artiﬁcial trajectory decreases (Δκ < 0). + + V (x , r , t + 1) ≤ V (x, r , t) − α (kx − x k) δ δ T 1 r However, both for numerical and technical reasons we consider (17b) −1 the smooth constraint (6i) with a ﬁnite value c . Thus, we ≤ V (x, r , t) − α (α (V (x, r , t))) δ T 1 δ T require the following technical continuity assumption on the −1 ≤ sup c − α (α (c)) periodic economic optimization problem (2). c∈[0,α(rT )] (18) Assumption 5. There exists a positive constant c , such that −1 κ ≤ α(r ) − min{α (α (α/2)), α/2}, (19) T 1 at any time step t ∈ N, for any parameters y ∈ Y, for {z } =:Δα>0 any periodic trajectory r ∈ Z (t), which is not a local T T 8 minimum of (2) and any ǫ > 0, there exists a change Δr Assume further that y(t) is constant and the update rule B with kΔr k ≤ ǫ, r + Δr ∈ Z (t), J (r + Δr , t, y) < satisﬁes Assumptions 2 and 4, then κ is a local minimum T T T T T T T ∞ J (r , t, y), such that the following bound holds of (2) and the following performance bound holds T T TK−1 ℓ(r (j) + Δr (j), t + j, y) − ℓ(r (j), t + j, y) T T T ℓ(x(t), u(t), t, y(t)) t=0 c ≥ max . κ lim sup ≤ κ . j J (r , t, y) − J (r + Δr , t, y) T T T T T K→∞ (21) Proof. The results follow from Propositions 1–3. This assumption ensures that it is possible to incrementally Corollary 1. Let Assumptions 1, 3 and 5 hold. Assume change the overall cost J , with incremental changes in the that (6) is feasible at t = 0 and y(t) is constant. If the update reference r and the local cost ℓ. If we expand the fraction rule B is chosen as update scheme 2 or 6 in [26], then the by Δr and take the limit Δr → 0, we can see that this T T closed-loop average economic performance is no worse than condition is similar to a continuity assumption on the fraction the performance at a locally optimal periodic orbit (2). of the gradients of ℓ and J . Additional details regarding this condition are discussed in Sections IV-B and IV-C. The Proof. This results follows directly from Theorem 1. It sufﬁces following lemma shows that this continuity condition (Ass. 5) to note, that the update schemes 2 and 6 satisfy Assumptions 2 in combination with the incremental stabilizability property and 4, compare [26, Lemmas 1 and 4]. (Ass. 3) allows for the convergence to local minima. Remark 2. For simplicity, we have presented the proposed Lemma 2. Let Assumptions 1, 3 and 5 hold and assume that framework in a discrete-time setting. However, the approach y(t) is constant. Suppose that the optimization problem (6) is can be directly applied to continuous-time problems by deﬁn- feasible at time t with some reference trajectory r (·|t), which ing the discrete-time stage cost ℓ and dynamics f implicitly is not a local minimum to (2). Then there exists a reference as the integration of some continuous-time dynamics f and r˜ which is a feasible candidate solution to (6) at t+1, which T the average continuous-time cost ℓ over some sampling satisﬁes period h. One advantage of considering a continuous-time formulation is that the design of terminal ingredients satisfying J (r˜ , t + N + 1, y(t)) < J (R r (·|t), t + N + 1, y(t)). T T T T Assumption 3 (compare Section IV-A) simpliﬁes, compare (22) e.g. [37, App. C]. Furthermore, in a continuous-time setting it Proof. Given that R r (·|t) ∈ Z (t + N + 1) is not a local T T is possible to use a variable sampling time h ∈ [h , h ], min max minimum, Assumption 5 ensures that there exists a reference by considering the decision variable u = (u , h), where u c c r˜ = R r (·|t) + Δr ∈ Z (t + N + 1), that improves T T T T denotes the (typically piece-wise constant) control input. As the reference cost J (22) and satisﬁes kΔr k ≤ ǫ and T T a result, in a time-invariant setting the ﬁxed constant T does (21). Satisfaction of the posed constraints (6h)–(6i) follows not directly impose a time length on the set of periodic orbits from (21), by noting that Δκ(t) = J (r˜ , t + N + 1, y(t)) − T T Z , but only a ﬁnite parametrization. The constants h , T min J (R r (·|t), t+N +1, y(t)). With ǫ according to Lemma 1, T T h need to be chosen, such that the (typically explicit) T max the candidate input u(·|t + 1) from Proposition 2 satisﬁes the discretization scheme is stable and the MPC can react fast terminal set constraint (6f) with the incrementally changed enough. The advantages of such a formulation will also be reference r˜ and is thus a feasible solution to (6). explored in a numerical example in Section V-B. We point out that the beneﬁts of using such a variable continuous-time A priori performance bounds: The following proposition period length have also been recently investigated in [31] establishes a priori performance bounds on the artiﬁcial refer- using a direct multiple shooting method. ence trajectory as an extension to [27, Thm 2/3, Corollary 1]. Design parameters: Overall, the proposed framework pro- Proposition 3. Let Assumptions 1, 3 and 5 hold and assume vides desired performance guarantees, if the constant c , the that y(t) is constant. Assume that the optimization problem (6) self-tuning weight β(t) and the terminal ingredients V ,X f f is feasible at t = 0. If the update rule B satisﬁes Assumption 4, are chosen properly (Ass. 1–5). In numerical experiments, we then κ is a local minimum of (2). found that the closed loop is insensitive to changes in the Proof. Using a proof of contradiction one can show κ = constant c , even by orders of magnitude, as long as c is ∞ κ κ lim J (x(t), t, y(t), κ (t)), compare [27, Thm. 2], sufﬁciently large (e.g. c = 100 in App. A). In Section IV-B t→∞ T,min j κ [26, Thm. 2]. Furthermore, suppose that there exists a limiting we also show how the problem can be reformulated to get rid reference r , which is not a local minimizer of (2). Lemma 2 of the constant c and the continuity condition in Assump- T κ ensures that there exists a feasible reference r˜ , with an im- tion 5. A large self-tuning weight β(t) can deteriorate the proved cost, which implies J < κ and thus contradicts transient performance, but is useful to ensure convergence of T,min ∞ the assumption. the artiﬁcial reference to a local minimum. In Section IV-D, we show that similar performance bounds hold when choosing The following theorem summarizes the theoretical proper- a constant weight β. For the special case of T = 1 (artiﬁcial ties of the proposed MPC scheme. setpoint), more details on the effect of β on the closed loop can Theorem 1. Let Assumptions 1, 3 and 5 hold and assume be found in [26], [27] and [28]. Different design procedures that (6) is feasible at t = 0. Then the optimization problem (6) for the terminal ingredients will be discussed in detail in is recursively feasible for the resulting closed-loop system (7). Section IV-A. 9 IV. PROPOSED FRAMEWORK - DETAILS AND VARIATIONS for any t ∈ N, r ∈ Z (t), y ∈ Y and any x ∈ R with T T V (x, r , t) ≤ α : δ T 1 In the following, we discuss details and variations of the S(r, t) ℓ (r, t, y), (23) ξξ proposed framework. In Section IV-A we discuss in detail ℓ (x, r , t, y) ≥ℓ(x, r , t, y) + kx − x k , (24) q T T r how to design terminal ingredients that satisfy Assumption 3. In Section IV-B we discuss how to modify the cost function, with such that the continuity condition in Assumption 5 can be dropped. In Section IV-C we consider the special case of ℓ (x, r , t, y) :=ℓ (x , r , t, y) · (x − x ) + kx − x k , q T x r T r r Q (r ,t) convex periodic optimal control problems. In Section IV-D, we (25) discuss the theoretical properties without self-tuning weights β(t), similar to [28]. I I n n Q (r , t) := S(r, t) k (x , r , t) k (x , r , t) f,x r T f,x r T + 2ǫI + ℓ k (x , r , t), (26) n uj f,j,xx r T A. Terminal cost for economic dynamic operation j=1 In the following, we detail how to design terminal ingre- where (x , u ) = r = r (0), ℓ denotes the Hessian of ℓ w.r.t. r r T ξξ dients that satisfy Assumption 3. First, in Section IV-A1 we ξ = (x, u), k the Jacobian of k w.r.t. x, k the Hessian f,x f f,j,xx show how a suitable tailored economic terminal cost V can of the j-th component of k w.r.t. x and ℓ the Jacobian of f u be designed using local linear and quadratic approximation ℓ w.r.t. the j-th component of u. of the dynamics f and the economic cost ℓ, as an extension Proof. We point out that the derivative of ℓ w.r.t. x is the total and combination of the methods in [6], [27], [37]. Then, in derivative of ℓ w.r.t. x, for u = k . Hence, the Jacobian and Section IV-A2 we show how a simple (and hence conservative) Hessian of ℓ are given by positive deﬁnite terminal cost V can be computed based on any existing incremental Lyapunov function V , similar to ℓ =ℓ , x ξ the design in [38], [18]. Finally, in Section IV-A3, we show f,x that the theoretical properties can also be guaranteed with m ⊤ ⊤ a simple terminal equality constraint (TEC), if a multi-step I k I k ℓ = ℓ + ℓ k , xx n ξξ n u f,j,xx f,x f,x j implementation is considered. j=1 1) Reference generic ofﬂine computations: In the follow- 1×(n+m) (n+m)×(n+m) where ℓ ∈ R , ℓ ∈ R denote the ξ ξξ ing, we detail a procedure to compute a suitable terminal cost Jacobian and Hessian of ℓ w.r.t. ξ = (x, u). Twice continuous V (Ass. 3) based on the linearization of the dynamics and a differentiability of ℓ and compactness imply that there exists quadratic approximations of the stage cost ℓ (using the Hessian a ﬁnite constant and gradient). The following derivation is an extension of the c = sup λ (ℓ (r, t, y)). approach in [6] to dynamic/periodic trajectories. Furthermore, max ξξ t∈N,r∈Zr(t),y∈Y we extend the approach to online optimized/changing refer- ence trajectories by extending the reference generic ofﬂine Thus the matrix S = (max{c, 0})I is positive semi- n+m computation from [37] to an economic stage cost ℓ. In ad- deﬁnite and satisﬁes S ℓ . The construction in (26), the ξξ dition, this online computation involves an online computed deﬁnition of the Hessian ℓ and S ℓ directly imply xx ξξ adjoint for periodic trajectories, similar to the local gradient Q (r , t) ℓ (x , r , t) + 2ǫI . Similar to [6, Lemma 22], T xx r T n correction employed in [39]. there exists a small enough constant α > 0 (uniform in r , t, y), such that Q (r , t) ℓ (x, r , t, y) + ǫI , ∀t ∈ N, T T xx T n Linear-quadratic local auxiliary stage cost: The following y ∈ Y, r ∈ Z (t), V (x, r , t) ≤ α . Abbreviate Δx = T T δ T 1 Lemma extends the results in [6, Lemma 22-23] to compute x − x , which implies ℓ = kΔxk + ℓ Δx. Convexity of r q x an auxiliary stage cost ℓ , that locally upper bounds the stage Q the sublevel sets of V implies that V (x + sΔx, r , t) ≤ α δ δ r T 1 cost ℓ, which will later be used to derive sufﬁcient conditions for all s ∈ [0, 1] and any V (x + Δx, r , t) ≤ α . Hence, δ r T 1 for inequality (8c). we can use the mean value theorem for vector functions [1, Lemma 3. Suppose there exists some V , k satisfying δ f Prop. A.11 (b)], similar to [6, Lemma 23], to obtain the conditions in Assumption 3. Suppose further that the ℓ (x, r , t, y) − ℓ(x, r , t, y) (27) q T T sublevel sets of V are convex in x, the controller k is δ f twice continuously differentiable in x, continuous in r , and ⊤ ∗ = (1 − s)Δx (Q (r , t) − ℓ (x + s · Δx, r , t, y))Δxds T xx r T satisﬁes k (x , r , t) = u . Suppose that the stage cost ℓ f r T r and the dynamics f are locally Lipschitz continuous and 2 2 ≥ (1 − s)ǫkΔxk ds = ǫ/2kΔxk . twice continuously differentiable w.r.t (x, u). Then the function ℓ(x, r , t, y) = ℓ(x, k (x, r , t), t, y) − ℓ(r, t, y) is twice con- T f T tinuously differentiable with respect to x. For any ǫ > 0, there exists a constant α > 0 and a positive semi-deﬁnite matrix n+m×n+m S(r, t) ∈ R , such that the following conditions hold Basically, ℓ is a local linear-quadratic over approximation q 10 of the stage cost ℓ. Hence, we will formulate a sufﬁcient approximation at x = x yields condition for (8c) using the auxiliary stage cost ℓ . We point Δx =f(x, k (x, r , t), t) − f(x , u , t) f T r r out that Lemma 3 does not impose any deﬁniteness conditions =A (r , t)Δx + Φ (Δx), cl T r ,t on the Hessian of the stage cost ℓ, but instead upper bounds T the Hessian using the positive semi-deﬁnite matrix S. with the remainder term Φ . Twice continuous differentia- r ,t bility of f and compact constraints imply that the remainder Sufﬁcient conditions based on the linearization: We denote term is uniformly Lipschitz continuous in the terminal set, the Jacobian of f evaluated around an arbitrary point r ∈ i.e., kΦ (Δx)k ≤ L kΔk for all t ≥ 0, with a constant Z (t) at some time t ∈ N by r r ,t Φ,α T 1 L arbitrary small for α arbitrary small. Using this bound Φ,α 1 ∂f ∂f in combination with condition (30) implies that there exists A(r, t) := , B(r, t) := . (28) ∂x ∂u (x,u)=r (x,u)=r a sufﬁciently small constant α > 0, such that the nonlinear system (locally) satisﬁes Given some periodic trajectory r (·|t) ∈ Z (t), we denote T T the Jacobian w.r.t. x of the system f in closed loop with the + 2 2 2 kΔx k − kΔxk ≤ −kΔxk ∗ (33) P (R r ,t+1) P (r ,t) Q (r ,t) T T T T terminal control law k by for all x ∈ X (r , t), compare [37, Lemma 1] for details. f T A (r (·|t), t) := A(r (0|t), t) + B(r (0|t), t)k (r , t). cl T T T f,x T Given that T is ﬁnite, ℓ uniformly bounded and condition (32) holds, the vector p admits a uniform bound. Using the deﬁni- In the following, we introduce a corresponding adjoint periodic tion of p, we get trajectory p(k|t), which can be computed online based on the following set of n · T linear (in p) equality constraints ⊤ + p (1)Δx (34) ⊤ j A (R r (·|t), t + j)p(j + 1|t) (29) ≤p (1)A (r , t)Δx + kp(1)kkΦ (Δx)k cl T cl T r ,t (29) =p(j|t) − ℓ (x (j|t),R r (·|t), t + j, y), j = 0, . . . , T − 1, r T = p (0)Δx − ℓ (x , r , t, y)Δx + kp(1)kkΦ (Δx)k x T T x r T r ,t ⊤ 2 ≤p (0)Δx − ℓ (x , r , t, y)Δx + ǫ/2kΔxk , with p(N|t) = p(0|t). In the setpoint case (T = 1), this x r T reduces to p (A − I) = −ℓ , similar to [6], [27]. Similar cl where the last inequality holds for a sufﬁciently small constant to the adjoints used in [39], this vector p corrects the effect α > 0, given the uniform bound on kpk and the properties of of ℓ , the gradient of the stage cost. the remainder term Φ . By combining (33) with (34) and r ,t The following proposition shows that such an online com- using the auxiliary stage cost from Lemma 3, the terminal puted adjoint vector p in combination with an ofﬂine computed cost (31) satisﬁes matrix valued function P provides a suitable terminal cost for V (x ,R r , t + 1, y) − V (x, r , t, y) f T T f T dynamic operation with economic cost. 2 2 ≤ − kΔxk − ℓ (x , r , t, y)Δx + kΔxk ∗ x r T Q (r ,t) Proposition 4. Suppose the conditions in Lemma 3 hold. (24) Assume further that there exists a positive deﬁnite T -periodic ≤ − ℓ(x, r , t, y), matrix P (r , t), continuous in r, such that for any t ∈ N, and hence condition (8c). r ∈ Z (t), the following matrix inequality is satisﬁed T T A (r , t)P (R r, t + 1)A (r , t) − P (r , t) (30) T T cl T T cl Corollary 2. Suppose that the stage cost ℓ and the dynamics − Q (r , t) − ǫ˜I , f are locally Lipschitz continuous and twice continuously T n differentiable w.r.t (x, u). Assume further that there exists a with ˜ǫ > 0. Then for any periodic reference r ∈ Z (t), T T positive deﬁnite matrix P (r , t) and a matrix k (r , t), both T f,x T t ∈ N, the conditions (29) have a unique solution p(·|t). In continuous in r and T-periodic in t, such that for any t ∈ N, addition, there exists a constant α , such that the terminal r ∈ Z (t), the matrix inequality (30) is satisﬁed with some T T cost ǫ˜ > 0. Then there exists a function α(r ), such that the termi- 2 ⊤ nal controller k (x, r , t) = u + k (x − x ), the terminal V (x, r , t, y) := kx − xk + p (0|t)(x − x ), (31) f T r f,x r f T r r P (r ,t) n 2 set X (r , t) = {x ∈ R | kx − x k ≤ α(r )} and the f T r T P (r ,t) with p according to (29) and x = x (0), satisﬁes condi- r r terminal cost V according to (31) satisfy Assumptions 1 and tion (8c) with X (r , t) := {x ∈ R | V (x, r , t) ≤ α }. f T δ T 1 Proof. Part I. Condition (30) ensures that the linearized Proof. Given that P and Q + ǫ˜I are positive deﬁnite, the (time-varying) dynamics along the periodic trajectory r are T conditions (17a)–(17b) in Assumption 3 are satisﬁed with the (uniformly) exponentially stable, which implies incremental Lyapunov function V (x, r , t) = kx−x k δ T r P (r ,t) j and quadratic functions α , α , α ∈ K . Convexity of T−1 ⊤ 1 2 3 ∞ det(I − Π A (R r , t + j)) > 0. (32) n T cl j=0 T the terminal set X w.r.t. x (compare conditions Lemma 3) Thus, the constraints (29) have a unique solution p for any follows from V quadratic in x. Condition (17c) follows from r ∈ Z (t), compare also the reformulation of (29) in V quadratic and the assumed continuity of P w.r.t. r . T T δ T Remark 4. Conditions (8a), (8c), (17) hold for any α ≤ α , using Prop. 4. Part II. Denote Δx = x − x . The ﬁrst order Taylor Furthermore, given that Z (t) ∈ int(Z(t)) and k bounded, r r f,x 11 Algorithm 1 Ofﬂine computation there exists a small enough constant α > 0, such that 1: Compute Jacobian, Hessian A, B, ℓ , ℓ . conditions (8b), (18) hold for any α ≤ α . Hence, choosing 2 ξ ξξ 2: Determine matrix S 0, such that S ℓ (23). the constant terminal set size α(r ) = min{α , α } satisﬁes T 1 2 ξξ 3: Compute matrix P (and possibly k ) such that (30) holds all the conditions (condition (18) is trivially satisﬁed). f using LMIs, compare [37, App. D]. 4: Compute maximal terminal set size α using [37, Alg. 1]. 5: Derive α(r) ∈ (0, α ] for constraint satisfaction: a) Compute constant α > 0 using [37, Equation (24)] With this result, we can directly specify a procedure to com- and set α = min{α , α }. 1 2 pute suitable terminal ingredients. First, a symbolic expression b) Compute α(r ) online with a scalar optimization for the Jacobian A, B, ℓ and the Hessian ℓ are computed. variable, compare [25, Sec. 3.3]. ξ ξξ Then a positive semi-deﬁnite matrix S is computed, which satisﬁes (23). This can either be achieved with a constant The proposed procedure is a combination and extension of matrix S (c.f. proof Lemma 3) or by computing a suitably the reference generic ofﬂine computations in [37], the termi- parametrized matrix S using linear matrix inequalities (LMIs). nal ingredients for economic MPC [6], [27] and the online computation of p using (29). Regarding the online operation, Given S, we have to compute a parametrized matrix P , we simply include the constraints (29) to compute p(·|t) and such that condition (30) holds. Suppose we want to compute possibly constraints to compute α(r ) online (compare [25, a feedback of the form k = u + k Δx with some f r f,x Sec. 3.3.] for details) in the MPC optimization problem (6). parametrized feedback gain k (k = 0). In this case, f,x f,xx This procedure signiﬁcantly simpliﬁes in the special case of condition (30) with Q according to (26) is equivalent to [37, linear systems with linear/quadratic stage costs ℓ, which is Inequality (36)] with the following (output) tracking stage cost discussed in Section IV-C. Furthermore, in the special case of 2 2 ℓ = k(C + Dk )Δxk + (2ǫ + ǫ˜)kΔxk (35) f,x S artiﬁcial setpoints (T = 1), we recover the schemes in [26], I 0 [27], [28] and Algorithm 1 provides a corresponding procedure n n (n+m)×n (n+m)×m C = ∈ R , D = ∈ R . to derive suitable terminal ingredients. 0 I m m Hence, we can use the result in [37, Lemma 6, Prop. 6] to Remark 3. The matrices S, Q and P can also be compute suitable matrices k and P . In particular, in [37] parametrized by y, to yield a terminal cost V that depends f,x f the matrices P, k , A, B, S are parametrized based on a on the price signal y. However, the incremental Lyapunov f,x quasi-LPV (linear-parameter-varying system) approach and the function V used for the terminal set X (Ass. 3) may not δ f conditions are transformed into LMIs, that can be efﬁciently depend on y to ensure recursive feasibility independent of computed ofﬂine. online changes in the price signal y. Thus, the choice of X in Corollary 2 is only valid for P independent of y. Given that the vector p needs to satisfy condition (29) with equality, a similar parametrized ofﬂine computation for p Remark 4. As already discussed, the vector p needs to be seems intractable (with the exception of linear systems, com- computed online using (29), which adds nT optimization pare Section IV-C). Hence, we simply add the constraint (29) variables and n·T equality constraints (linear in p) to the op- to the MPC optimization problem (6) and compute p(·|t) timization problem (6). Abbreviate A(j|t) = A(r (j|t), t+j), online. B(j|t) = B(r (j|t), t + j), ℓ (j|t) = ℓ (r (j|t), t + j, y(t)) T ξ ξ T and suppose the feedback k is parametrized in the form f,x Finally, regarding the terminal set size α, we ﬁrst compute −1 k (j|t) = Y (j|t)X(j|t) with matrices X, Y (as is the f,x the constant α > 0, such that (8c) holds for all V ≤ α , 1 δ 1 case with the LMIs considered in [37]). Then, multiplying e.g. using Algorithm 1 from [37]. There are two options to the constraints (29) by X from the left yields the following compute a terminal set size α that also ensures constraint equivalent constraint satisfaction (8b). The deﬁnition of Z ⊆ int(Z) can be used to compute a constant α ∈ (0, α ], similar to the optimization (A(j|t)X(j|t) + B(j|t)Y (j|t)) p(j + 1|t) (36) problem (24) in [37]. However, such a constant α depends on ⊤ ⊤ =X(j|t)p(j|t) − X(j|t) Y (j|t) ℓ (j|t), the choice of Z and thus can yield arbitrary small values α (and thus slow convergence of r , compare Lemma 1), or re- T where we use k X = Y and the formula for ℓ from the f,x x quires restrictive constraints on the set of periodic trajectories proof of Lemma 3. The resulting constraint can be imple- Z . This problem can be alleviated by computing a reference T mented directly in terms of X, Y . The constraints (29) can trajectory dependent terminal set size α(r ) ∈ [α, α] online, T also be compressed into one n dimensional equality constraint which can be done by using an additional scalar optimization with only the n dimensional optimization variable p(0|t). In variable α in (6), compare [25, Sec. 3.3] for details. particular, denote ℓ (j|t) = ℓ (x (j|t), r (·|t), t + j, y(t)) x x r T k−1 j and A (k|t) = Π A (R r (·|t), t + j). Then p(0|t) cl cl T j=0 T The overall design procedure is summarized in Algorithm 1. satisfying (29) can be equivalently computed using T−1 ⊤ ⊤ ⊤ (I − A (T|t))p(0|t) = A (j|t)ℓ (j|t). (37) cl cl x 2 j=0 Due to the prediction horizon N the time index t changes to t+N in (29). 12 Furthermore, if the period length T is very large, an approx- (8b) and Assumption 3. Condition (8c) follows with imate solution can be obtained by assuming A (j|t) ≈ 0 cl + + V (x , r , t + 1) − V (x, r , t) f f T for j ≥ T with some T < T , which results in p(0|t) ≈ c c ⊤ ⊤ T −1 (40) k k + + k A (j|t)ℓ (j|t). The fact that we need to take the full cl x j=0 = (V (x , r , t + 1) − V (x, r , t)) δ δ k T c (1 − ρ ) trajectory r into account to compute the correct gradient k=1 correction p indicates that the computation of a terminal cost ν (38a)X k k for nonperiodic time-varying trajectories may be non-trivial. ≤ (ρ − 1)V (x, r , t) δ T k k c (1 − ρ ) k=1 ν ν X X (38b) V (x, r , t) 2) Given incremental Lyapunov function V : In the follow- δ δ T = − a ≤ − a kx − x k k k r ing, we consider the case, where some incremental Lyapunov k=1 l=1 function V with a corresponding feedback k is available δ f (39) and focus on computing a simple (positive deﬁnite) terminal ≤ ℓ(r, t, y) − ℓ(x, k (x, r , t), t, y). f T cost V . For simplicity, we consider exponential stability, as a special case of Ass. 3. This terminal cost is easy to compute, but can also be Assumption 6. There exists an incremental Lyapunov function quite conservative. In particular, an interesting feature of this V (x, t, r ), a controller k (x, r , t), a terminal set size δ T f T approach is that the terminal cost V is a sum of incremental α(r ) and constants c , c > 0, ρ ∈ (0, 1), such that at T l u Lyapunov functions and as such also an incremental Lyapunov any time t ∈ N, for any reference r ∈ Z (t) and any T T function, which is positive deﬁnite w.r.t. to the reference x ∈ X (r , t) := {x ∈ R | V (x, r , t) ≤ α(r )}, the f T δ T T trajectory r . Correspondingly, the terminal cost incentives following conditions hold regulation towards the reference trajectory r . In general the + + reference r will have a suboptimal performance, which is V (x , r , t + 1) ≤ρV (x, r , t), (38a) δ δ T why a purely economic (not positive deﬁnite) terminal cost as c kx − x k ≤ V (x, r , t) ≤c kx − x k, (38b) l r δ T u r in Proposition 4 may be advantageous. (x, u) ∈Z(t), (38c) Remark 5. Note, that if a matrix P is computed ofﬂine + + with x = f(x, u, t), r = R r ∈ Z (t + 1), u = T T T that satisﬁes condition (30), the results in Corollary 2 and k (x, r , t). In addition, Inequalities (17c) and (18) from f T Prop. 5 can be combined to show that the terminal cost V = Assumption 3 hold with some α , α ∈ K . 4 5 ∞ kΔxk +ckΔxk locally satisﬁes condition (8c), by choosing c > 0 suitably. In particular, consider V = kΔxk satisfying δ P Assumption 6 and the following local linear quadratic bound Such an incremental Lyapunov function can for example be 2 ∗ ℓ ≤ a kΔxk +kΔxk ∗, with Q according to Lemma 3 and 1 P computed using quasi linear parameter varying methods [37], Q a suitable constant a > 0 using continuous differentiability control contraction metrics [40], back stepping [41] or feed- of ℓ. Then choosing c := a /(1 − ρ) ensures satisfaction of back linearization. In addition, we assume that the stage cost (8c) using can be locally bounded by some polynomial, similar to [38, Ass. 5], [18, Ass. 26]. (33),(38a) + 2 V − V ≤ −kΔxk ∗ − c(1 − ρ)kΔxk ≤ −ℓ. f P Assumption 7. Consider the incremental Lyapunov function 3) Terminal equality constraints: In the following, we V , the feedback k and the terminal set X from Assump- δ f f discuss how to replace the general terminal set (Assumption 3) tion 6. There exists constants a ≥ 0, k = 1, . . . , ν, such that k with a simple terminal equality constraint (TEC, X (r , t) = f T the stage cost ℓ satisﬁes x ). In principle, the conditions (17a)–(17c) are quite general ν and not restrictive, but the explicit knowledge of V (which ℓ(x, k (x, r , t), t, y) − ℓ(r, t, y) ≤ a kx − x k , (39) characterizes the terminal set X ) can pose challenges. f T k r k=1 The following analysis is similar to [28] and [24], which also considered TEC in the steady-state case. To this end, we for all t ∈ N, y ∈ Y, r ∈ Z (t), x ∈ X (r , t). T T f T consider the following ﬁnite-time local incremental controlla- bility condition. The following proposition follows the arguments from [38, Prop. 2], compare also [18, Prop. 27]. Assumption 8. There exist constants ν ∈ N, ǫ > 0, such that at any time t ∈ N, for any references r , r˜ ∈ Z (t) T T T with kr − r˜ k ≤ ǫ, there exists a state and input sequence T T Proposition 5. Let Assumptions 6–7 hold. Then the following ν×(n+m) (x, u) ∈ R , such that terminal cost V satisﬁes Assumptions 1 and 3 x(0) =x (0), x(k + 1) = f(x(k), u(k), t + k), V (x, r , t) := V (x, r , t). (40) f T T k x(ν) =x ˜ (ν), (x(k), u(k)) ∈ Z(t + k), k = 0, . . . , ν − 1. c (1 − ρ ) T k=1 This condition is for example satisﬁed with ν ≤ n if the lin- Proof. The terminal set X directly satisﬁes conditions (8a), earization along any feasible periodic trajectory is (uniformly) f 13 controllable [1, Ass. 2.37], [8, Ass. 2], compare also [28, constraints. However, instead of a ν-step MPC implementa- Ass. 7] [19, Ass. 10]. Typically, an additional continuity tion (41), in [28, Algorithm 3] it was suggested to augment bound on ℓ is used (compare for example [24, Ass. 4]), the MPC with an algorithm that decides at each time t if which is, however, not necessary in the considered setup with the candidate solution or the standard MPC feedback (7a) is the bounded stage cost ℓ and the self-tuning weight β. The applied. In particular, if the cost of the artiﬁcial reference following result is an adaptation of Lemmas 1–2 to terminal r does not improve by a minimal amount ǫ˜, the candidate equality constraints (TEC) using Assumption 8 and a multi- solution is applied. Given Assumption 8, after at most ν steps, step implementation. it is possible to incrementally move the reference trajectory and thus improve the cost. Thus, by augmenting the MPC with Lemma 4. Let Assumptions 5 and 8 hold and assume that y(t) such an algorithm, it may not be necessary to apply the ﬁrst is constant. Consider the terminal ingredients X (r , t) = f T ν steps of the computed input trajectory, which can speed up {x }, V (x, r , t, y) = 0 and a prediction horizon N ≥ ν. r f T convergence. Suppose that the optimization problem (6) is feasible at time t with some reference trajectory r (·|t), which is not a local B. Modiﬁed reference cost minimum to (2). Then under the ν-step closed-loop system The proposed formulation (6) uses standard conditions for ∗ ∗ u(t + k) = u (k|t), x(t + k + 1) = x (k + 1|t), (41) the terminal ingredients (Ass. 1) and contains many economic MPC formulations as special cases, compare [26], [27], [28], k = 0, . . . , ν − 1, [21], [9], [17], [18], [2], [6]. However, the formulation also there exists a reference r˜ which is a feasible candidate requires the additional constraints (6h)–(6i), based on the con- solution to (6) at t + ν and satisﬁes tinuity condition (Ass. 5). In the following, we brieﬂy discuss ν ∗ an alternative solution to this problem, based on a modiﬁed J (r˜ , t + N + ν, y(t)) < J (R r (·|t), t + N + ν, y(t)). T T T T T cost for the artiﬁcial reference trajectory. The following result ν ∗ Proof. Given that R r (·|t) ∈ Z (t + N + ν) is not a local T T is based on [42, Prop. 1], which in turn is motivated by the minimum, Assumption 5 ensures that there exists a reference analysis of non-monotonic Lyapunov functions [43]. ν ∗ r˜ = R r (·|t) + Δr ∈ Z (t + N + ν), that satisﬁes T T T T T Lemma 5. Consider the terminal ingredients from Assump- the posed constraints (6h)–(6i), improves the reference cost ν ∗ tion 1. For any t ∈ N, y ∈ Y, r ∈ Z (t), x ∈ X (r , t), the T T f T J and satisﬁes kr˜ − R r (·|t)k ≤ ǫ. Due to the multi- T T T T ∗ modiﬁed terminal cost step implementation (41), the sequence x(k|t + ν) = x (k + ν|t), u(k|t + ν) = u (k + ν|t) satisﬁes the dynamics (6d), the V (x, r , t, y) :=V (x, r , t, y) f T f T constraints (6e) and ends in the reference (6f), i.e., T−2 T − 1 − k ∗ ∗ + ℓ(r (k), t + k, y) x(N − ν|t + ν) = x (N|t) = x (0|t). k=0 Correspondingly, for any r˜ ∈ Z (t + N + ν) with T T satisﬁes ν ∗ kR r (·|t) − r˜ k ≤ ǫ, we can append the state and input T T ˜ ˜ sequence x(·|t+ν), u(·|t+ν) with the candidate solution from V (x ,R r , t + 1, y) − V (x, r , t, y) f T T f T Assumption 8, which satisﬁes the constraints (6e)–(6g). ≤ − ℓ(x, u, t, y) + J (r , t, y)/T, T T Compared to the results in Lemmas 1–2 based on terminal + with x = f(x, u, t), u = k (x, r , t). f T sets, the resulting properties with terminal equality constraints Proof. Abbreviate ℓ(k) = ℓ(r (k), t + k, y), V = T f are only valid if we apply the ﬁrst ν parts of the computed ˜ ˜ ˜ V (x, r , t, y), V = V (x ,R r , t + 1, y). The modiﬁed f T f T T input sequence. For comparison, in tracking MPC with positive terminal cost satisﬁes deﬁnite stage cost ℓ, such a multi-step implementation is not needed, since the closed-loop system eventually converges + ˜ ˜ T (V − V ) to an ǫ neighbourhood of the reference trajectory r , com- T−2 pare [24, Thm. 2], [8, Thm. 3] [23, Thm. 2]. =T (V − V ) + (T − 1 − k)(ℓ(k + 1) − ℓ(k)) The main beneﬁt of the terminal equality constraint im- k=0 plementation is the simple design. Although we need to use T−2 (8c) a multi-step implementation with ν steps, we would like ≤ − T ℓ(x, u, t, y) + (T + 1 − T )ℓ(0) + ℓ(T − 1) + ℓ(k) to point out that ν is independent of T , and hence this k=1 method does not suffer from the same limitations as the = − T ℓ(x, u, t, y) + J (r , t, y). T T approaches based on T -step systems, such as [19], [35]. On the other hand, an implementation with a suitable terminal cost (Ass. 3) can use any prediction horizon N, requires no multi- We point out, that the modiﬁcation of the cost in Lemma 5 is step implementation and typically yields better closed-loop applicable both to terminal equality constraints (Lemma 4) and performance and (inherent) robustness properties, compare the terminal cost/sets (Ass. 3). The following proposition shows numerical example in Section V-B. that this modiﬁed terminal cost can ensure the same theoretical Remark 6. In [28] a similar economic MPC scheme for properties (Prop. 2) as the proposed scheme (6), without using setpoints (T = 1) has been considered with terminal equality the continuity condition (Ass. 5). 14 Proposition 6. Let Assumption 1 hold and assume that y(t) is in Lemma 3 is independent of r and we can consider a time- constant. Consider the MPC formulation (6) with V replaced varying matrix P (t) to satisfy condition (30) in Proposition 4. by V (Lemma 5) and the constraints (6h)–(6i) replaced by Matrices P (t), K(t) satisfying condition (30) can be computed by solving T coupled LMIs similar to [44]. Alternatively, the T−1 computation of K(t), P (t) can be achieved using the discrete- J (r (·|t), t + N, y(t)) ≤ κ(t) := κ (t). (42) T T j time LQR for a suitably deﬁned T -step system with x˜ ∈ R j=0 Tm and u ˜ ∈ R . The resulting closed-loop system satisﬁes the performance Using the reformulation of the constraints (29) discussed bound (10), if the update rule B satisﬁes Assumption 2. in Remark 4, the possibly nonlinear constraints (29) can be dropped by adding the explicit nonlinear term for p(0|t) in Proof. Similar to Prop. 2 the candidate solution from Prop. 1 the cost function. For the terminal set X , we can either use with the modiﬁed terminal cost implies an ellipsoidal set X (r , t) = {x| kx − x k ≤ α} or a f T r P (t) W (t + 1) − W (t) + ℓ(x(t), u(t), t, y(t)) polytopic (periodically time-varying) invariant set X . Thus, ≤J (r (·|t), t + N, y(t))/T + γ(t)κ(t + 1) T ℓ convex implies that the constraints in (6) are convex. (9) In case ℓ quadratic, Lemma 3 and Prop. 4 contain no =κ(t + 1)/T + γ(t)κ(t + 1). nonlinear terms that need to be locally over-approximated Correspondingly, the T -step bound (15) holds with c = 0, and hence we can set ǫ = ǫ˜ = 0 and α arbitrary large. since κ(t + 1) ≤ κ(t). The remainder of the proof follows Furthermore, for ℓ quadratic, the problem (6) is a quadrati- from the arguments in Prop. 2, similar to [26, Thm. 1][27, cally constrained quadratic program (QCQP). If we drop the Thm. 1]. constraints (6i) (compare Section IV-B) and use a polytopic terminal set X , the optimization problem (6) is a (linearly The properties in Prop. 3 and Theorem 1 hold equally with constrained) quadratic program (QP). the modiﬁed terminal cost V and a terminal set (Ass. 3), with In the special case that ℓ is linear, i.e., ℓ = x q(t, y), the the simpler constraint (42) (without requiring Assumption 5). vector p can be explicitly computed (independent of the online The main advantage of using this modiﬁed terminal cost V optimized reference r ) for a given price y. Furthermore, is that the technical continuity condition Assumption 5 is since S = 0, the terminal cost is linear (P = 0). Thus, if not required and the number of constraints in (6) is smaller, a polyhedral terminal set is chosen, the proposed scheme with while the theoretical properties are the same. However, this a linear cost ℓ only requires the solution to a linear program modiﬁed terminal cost yields an objective function, which to (LP), which can be done efﬁciently. Furthermore, with a the best knowledge of the authors differs from any existing polytopic incremental Lyapunov function V , the terminal cost MPC formulation (for T > 1). Correspondingly, it is unclear in Proposition 5 can be formulated in the MPC problem (6) what the practical effect on the closed-loop performance is, using linear constraints, resulting in an LP (or QP in case of which will be studied in the numerical example in Section V-B. ℓ quadratic). For the example considered in Section II-C with a terminal For this special case with a prediction horizon of N = 0, the equality constraint and T = 2, the modiﬁed terminal cost is proposed MPC scheme is almost equivalent to the periodicity V = ℓ(r). With this modiﬁed cost the closed loop also does constraint MPC proposed in [21], [9]. The main difference the right thing, i.e., converges to the T -periodic orbit {1, 2}. is that we use a linear terminal cost and the polyhedral constraint, instead of a terminal equality constraint (and the constraints (6i) which can typically be neglected). This small C. Convex problem difference in the design allows us to derive the desired In the following, we discuss the special case, when the peri- performance guarantees, while the periodicity constraint MPC odic optimal problem (2) is convex. Suppose that the dynamics can lead to suboptimal performance, compare [9, Example 6]. f are afﬁne, i.e. f(x, u, t) = A(t)x + B(t)u + c(t), and the We can get the same theoretical properties with a terminal constraint sets Z are polytopes, which implies that Z is a r T equality constraint (TEC) and N ≥ ν, if we use Lemma 4. convex polytope. For ℓ convex, this implies that the periodic In case that some of the input variables u are also subject optimal problem (2) is convex and Theorem 1/Corollary 1 to integer constraints (c.f. for example periodic scheduling guarantee that the closed-loop performance is no worse than problems with discrete decisions [15] and the HVAC numerical operation at an optimal T -periodic orbit. example in Section V-A), the problem can be formulated as a In Section IV-C1 we discuss how the construction of the mixed-integer linear program (MILP). terminal ingredients (Sec. IV-A) and the online optimization 2) Continuity condition - Assumption 5: In the following, simpliﬁes in the convex case. In Section IV-C2, we discuss we discuss sufﬁcient conditions for Assumption 5. Suppose the continuity condition (Ass. 5) for periodic optimal control. that ℓ is continuously differentiable and Z is a convex 1) Ofﬂine design and online optimization: In the following, we discuss how the design procedure (Sec. IV-A) and the 3 The optional consideration of an online optimized terminal set size α(r ) online optimization simpliﬁes for the considered special case. can be expressed using linear constraints, for both cases. In addition to the possibly ellipsoidal terminal set, the constraints (6i) Since we have a linear (time-varying) system, we consider a are quadratic, leading to a non negligible increase in the online computation. linear time-varying feedback k = K(t) and a time-varying f,x Given that p(0|t) in (37) is linear in r , the terminal cost V is quadratic in T f matrix S(t) satisfying condition (23). Thus, the matrix Q (t) the decision variables. 15 polytope. Given a reference r , the direction of feasible satisﬁes W (t) ≤ W , which is equivalent to changes Δr , which imply a decrease in J , i.e. {Δr ∇J ≤ T T T β(J (r (·|t), t + N, y) − J (t)) T T,min 0, r + Δr ∈ Z }, is a polytope. The condition in T T T N−1 Assumption 5 reduces to the existence of a direction Δr in ∗ ∗ ≤ ℓ(x ˜(k), u ˜(k), t + k, y) − ℓ(x (k|t), u (k|t), t + k, y) this set, such that the directional derivative of ℓ is uniformly k=0 bounded relative to the directional derivative of J . + V (x ˜(N), r (t), t + N, y) f min Although this condition is reasonable in the context of ∗ ∗ − V (x (N|t), r (·|t), t + N, y) optimal periodic control, it is not necessarily satisﬁed for ≤η, any convex problem. In particular, it is not valid if the (directional) derivative of J vanishes, but the gradient of T with some constant η > 0. The last inequality follows from ℓ is (uniformly) lower bounded. This is for example the boundedness of ℓ, V and N ﬁnite. This inequality directly case, if the economic cost J is quadratic and the inequality T implies constraints are not active at the optimal periodic orbit, yielding κ(t + 1) = J (r (·|t), t + N, y) ≤ J (t) + ǫ, T T,min a vanishing gradient of J . If this problem occurs, we need to use the reformulation in Section IV-B to guarantee optimal for β ≥ β(ǫ) := η/ǫ. performance without Assumption 5. Thus, for a large enough weight β, the cost of the artiﬁcial Suppose that there exists a constant ǫ > 0, such that for periodic orbit r is arbitrarily (ǫ) close to the cost of the any r ∈ Z which is not a local minimum, there exists T T optimal reachable periodic orbit (20). Combining this result a feasible direction Δr with ∇J Δr ≤ −ǫkΔr k. Then T T T with Lemma 2 and the stronger terminal ingredients (Ass. 3), Assumption 5 is satisﬁed with some ﬁnite c , if ℓ is Lipschitz J is a local minimizer to (2). Correspondingly, it is pos- T,min continuous. Note that such a directional derivative always sible to derive performance bounds similar to Theorem 1 with exists if, e.g., ℓ is linear. an additional suboptimality term ǫ, compare [28, Thm. 2]. V. NUMERICAL EXAMPLES The following examples demonstrate the applicability of the proposed framework to dynamic operation and online chang- ing conditions. We ﬁrst consider a simple HVAC systems [33], where dynamics, cost and constraints are periodically time- D. Constant parameter β varying and discrete inputs are considered. Then we consider the classical problem of increasing the yield of continuous stirred-tank reactors (CSTR) with dynamic operation, com- In the following, we brieﬂy discuss how the performance pare [13]. In this example, we compare the performance of bounds in Proposition 3 can be approximately guaranteed the proposed approach with periodicity constraint MPC [21], with a constant weight β. In particular, in [28] a competing [9], tracking MPC formulations [8], [25] and MPC without approach to [26], [27] has been considered with a constant terminal constraints [7], [19], [20]. In addition, we study the weight β. Instead of changing the weight β online to achieve effect of the various degrees of freedoms in the formulation (locally) optimal performance, a ﬁxed weight β is considered (terminal ingredients (Sec. IV-A), alternative cost formulations and a suboptimality bound on the performance is established. (Sec. IV-B), continuous-time formulations (Remark 2)) on The following proposition shows that the same result applies closed-loop performance. here, as an alternative to Proposition 3, similar to [28, Prop. 2]. A. Building cooling Proposition 7. Let Assumptions 1 and 5 hold with a bounded terminal cost V and assume that y(t) and β(t) are constant. In the following, we show the applicability of the proposed Assume that the optimization problem (6) is feasible at time t. framework to periodic optimal control problems subject to There exists a function β, such that for any ǫ > 0, β ≥ β(ǫ) online changing performance measures. We consider a simple implies κ(t + 1) ≤ J (x(t), t, y, κ (t)) + ǫ. T,min j building temperature evolution example from [33, Sec. IV.A] governed by Proof. Denote the minimizer and minimum of (20) at time t by r (t) and J (t), respectively. By deﬁnition, there m T (t) = −k(T (t) − T (t)) + q (t) − q(t), min T,min amb amb dt exists a feasible input sequence u ˜ with corresponding state with air temperature T , cooling rate q, ambient temperature sequence x ˜ such that r (t) satisﬁes the constraints in (6). min T , rate of direct heat by the ambient q and model amb amb Due to optimality, the cost constants m, k > 0. The cooling rate q is generated using N−1 N = 2 chillers and is subject to the following (time- chiller W = ℓ(x ˜(k), u˜(k), t + k, y) invariant) disjoint constraint set k=0 + V (x ˜(N), r (t), t + N, y) + βJ (t), q ∈ U = {0} × [0.75, 1] × [1.5, 2], f min T,min 16 1.5 which is implemented using an additional discrete decision variable v ∈ {0, 1, 2}, corresponding to the number of active chillers. The state is subject to time-varying comfort bounds 1 centred around T = 0: 0.5 T (t) ≤ T ≤ T (t). min max The corresponding discrete-time system is given by x(t + 1) = Ax(t) + Bu(t) + e(t), (x(t), u(t)) ∈ Z(t), -0.5 with x = T , u = q and periodically time-varying e(t) based on q , T . The economic cost function is to minimize the amb amb -1 electricity cost given by ℓ(x, u, t, y) = ρ(t, y) · u, with the 50 100 150 price proﬁle ρ(t, y). In [33], for ﬁxed periodic price signals ρ(t), it was shown that periodic economic formulations [17], [18], [15] outper- form tracking formulations [8], [25]. We consider the more challenging problem, where the price proﬁle ρ changes each 1.5 day. Furthermore, we assume that only the price proﬁle for the current day is available as a forecast, which is modelled using the external variable y that changes every 24 hours. The considered price proﬁle ρ is taken from the real data considered in [12] over the span of one week. 0.5 We implemented the proposed approach using N = 2, β = 10 with the modiﬁed cost from Prop. 6 and a terminal equality constraint (TEC), which also satisﬁes the properties in Lemma 1 for the considered scalar stable system. The resulting 0 50 100 150 MPC optimization problem (6) is a small scale mixed-integer linear program (MILP), which was solved to optimality using intlinprog from MATLAB. The resulting closed loop can be Fig. 2. Transient performance under online changing price signals ρ for seen in Figure 2. The closed loop yields a periodic like HVAC. Top: Closed-loop temperature x = T of the proposed approach (blue, solid) and the optimal operation (red, dotted), with time-varying constraints operation for each day, with small changes between each day T (black, dashed). Bottom: Closed-loop applied cooling rate u = q min / max based on the different price proﬁle ρ. The adjustment of the for the proposed approach (blue, solid) and optimal operation (red, dotted); closed-loop response based on the price ρ can be directly seen and the price signal ρ (green, dotted). with the applied input u, which is always at a maximum when the electricity price ρ is low. We also compared the proposed framework to the optimal operation in Figure 2, assuming full System model: We consider a continuous-time model of a knowledge of the future price proﬁle ρ(t) for all coming days. continuous stirred tank reactor (CSTR)   The proposed framework results in state and input trajectories   −1 −0.55 4 2 1 − x − 10 x exp − 400x exp 1 1 x˙ 1 very similar to the optimal operation, resulting in a minimal x x 1 3 3       −1 x˙ = 4 2 , increase of 0.1% in the overall electricity price. 10 x exp − x   x˙ u − x where u ∈ R is related to the heat ﬂux and x = ⊤ 3 (x , x , x ) ∈ R correspond to the concentration of the re- 1 2 3 B. Continuous stirred-tank reactor action, the desired product and the temperature, compare [13], [45], [5, Sec. 3.4]. The constraints are In the following time-invariant example, we ﬁrst demon- strate average performance improvement of the proposed Z =[0.05, 0.4] × [0.05, 0.2] × [0.059, 0.439], framework compared to ﬁxed periodic operation or steady- Z =[0.03, 1] × [0.049, 0.449], state operation. Then we show reliable economic performance and we consider the economic stage cost ℓ(x, u, y) = −x + under online changing dynamic operation due to changing cost 2 y · (u − u ) with u = 0.1491 and y ∈ Y = [0, 1]. If the functions. s s external parameter is y = 0, the stage cost ℓ tries to maximize the production of the desired product x . The online tunable The proposed framework can also consider peak-demand prices using part in the cost function is a regularization of the input u the formulation in [33] or unpredictably changing constraint sets (reﬂecting relative to the optimal steady-state input u . For y = 1 the comfort levels set by a user) using soft constraints (c.f. Remark 1). system is optimally operated at a steady-state (x , u ), while s s To allow for a consistent comparison, the same initial and ﬁnal state is considered. for y = 0 dynamic operation signiﬁcantly outperforms steady- 17 state operation, compare [5, Sec. 3.4]. Hence, treating y as an external variable allows a user to smoothly transition between steady-state and dynamic operation. The discrete-time model is deﬁned with a fourth order Runge-Kutta discretization and a sampling time of h = 0.05. For the following simulations, the initial condition is always chosen as the optimal steady-state. Average performance improvement: We ﬁrst consider the problem of maximizing the concentration x (y = 0) to show average performance improvements. In the absence of transient changes (y constant), the average performance of periodic- ity constraint MPC [21], [9] and tracking formulations [8], [25] are equivalent (assuming that convergence is achieved). Similarly, the proposed framework yields the same asymptotic 0 10 20 30 40 50 performance as [17], [18], [15] assuming that r converges. We implemented the proposed approach with T ∈ {1, 10, 20} and horizons N ∈ [1, 50] and tested dif- Fig. 3. Average performance improvement due to dynamic operation ferent proposed designs regarding the terminal ingredients relative to the optimal steady-state x - CSTR. Periodic operation (dotted) (V ,X ,Corollary 2, Remark 5, Lemma 4) and the cost f f vs. proposed economic MPC scheme (solid with circles) for T = 1 (blue), function (V , Lemma 5, Prop. 6). The detailed numerical T = 10 (green) and T = 20 (red). results for all the considered implementations can be found in Appendix A. In the following, we only consider the approach utilizing the positive deﬁnite terminal cost V from Remark 5 (based on Alg. 1) in combination with the modiﬁed cost V from Lemma 5, which seems most suitable for practi- 0.5 cal applications (in terms of computational complexity and performance). Figure 3 exemplarily shows the performance of this approach with T ∈ {1, 10, 20} for increasing N in comparison to the average cost at the optimal periodic orbit of length T = {10, 20} and the optimal steady-state (T = 1). We note that, neglecting small initial deviations , -0.5 the proposed EMPC outperforms optimal periodic operation with the same period length T , even though a constant value β is used (Ass. 4 does not hold). We can see in general that -1 the performance increases (for both purely periodic operation 0 100 200 300 400 500 and the proposed approach) if we increase T and N. This implies that the proposed framework utilizing periodic orbits (T > 1) and additional predictions (N ≥ 1) with a purely Fig. 4. Dynamic operation under online changing conditions: deviation in economic formulation can outperform periodicity constrained production x −x (blue, solid), deviation in heat ﬂux u−u (red, dashed), 2 2,s s formulations [21], [9] (N = 0), steady-state formulations [26], price signal y (green, dotted). All signals are normalized to |x| ≤ 1. [27], [28], [29] (T = 1) and periodic tracking formulations [8], [23], [24], [25] (ℓ positive deﬁnite), even in case of ﬁxed optimal operation (y constant). (e.g. t ∈ [185, 246], t ∈ [400, 470]) to the new optimal Transient performance under online changing conditions: mode of operation, i.e., the steady-state x . In this scenario, In the following, we study the performance of the proposed the MPC scheme on average still increases production by scheme under online changing conditions, i.e., y unpre- 2.8% compared to steady-state operation, while a 5% increase dictably time-varying. The resulting closed loop for the pro- was achieved for y ≡ 0 (c.f. Fig. 3). For comparison, the posed MPC scheme with N = 10, T = 20 can be seen performance of the proposed approach, the tracking MPC [8], in Figure 4. As y → 0 (e.g. t = 15 or t = 246), the [25] with T = 20, N ∈ [0, 50] and the periodicity constraint system operates dynamically to increase production x and MPC [21], [9] with T ∈ [0, 90] can be seen in Figure 5. once the weight y on input deviations increases the system First, note that the number of decision variables in a condensed quickly minimizes the control effort and smoothly converges formulation are n + m · (T + N) for the proposed approach and the tracking MPC [8], [25], and m · T for the periodicity In [45], [5] a sampling time of h = 0.1 is used. However, with the constraint MPC [21], [9] (N = 0). Thus, the x-axis (N + T ) considered fourth order explicit Runge-Kutta discretization, a sampling time in Figure 5 is a measure for the computational complexity. of h = 0.1 does not preserve stability of the continuous-time system. In addition, we consider x ≥ 0.03 instead of x ≥ 0, to avoid discretization i i First, note that we can further improve the performance of errors for x ≈ 0. the proposed approach by increasing N. Similarly, the perfor- The average performance is computed in the interval t ∈ [1000, 2000] mance of the periodicity constraint MPC [21], [9] improves starting with initial condition x = x . Thus, for very short horizons N, κ has not yet converged. for a larger period length T , but at a smaller pace. Thus, given 18 the same number of decision variables, the proposed approach or modiﬁed versions thereof [28], [21], [9], [18], [34] are can achieve a better performance. If we consider the tracking contained as special cases. MPC, for small values of N, the performance is similar to The practical application of the proposed framework to large the proposed economic formulation. However, in contrast to scale HVAC systems is part of future work. A theoretical anal- the proposed formulation, the economic performance of the ysis of (bounded) online changing constraint sets/dynamics tracking MPC does not improve signiﬁcantly with a large with robust performance guarantees is an open problem. horizon N. Additional numerical results can be found in Appendix A. REFERENCES To summarize, in the considered example we have shown [1] J. B. Rawlings, D. Q. Mayne, and M. Diehl, Model Predictive Control: the applicability of the proposed approach to nonlinear eco- Theory, Computation, and Design. Nob Hill Publishing, 2017. [2] D. Angeli, R. Amrit, and J. B. Rawlings, “On average performance nomic optimal control problems. In particular, the proposed and stability of economic model predictive control,” IEEE Trans Autom approach: (i) improves performance compared to (ﬁxed) Control, vol. 57, pp. 1615–1626, 2012. [3] M. Ellis, J. Liu, and P. D. Christoﬁdes, Economic Model Predictive steady-state or periodic operation, (ii) reliably operates under Control: Theory, Formulations and Chemical Process Applications. online changing conditions, (iii) in general achieves better Springer, 2016. performance than periodicity constraint formulations [21], [9] [4] M. A. Mu¨ller and F. Allgo¨wer, “Economic and distributed model predictive control: Recent developments in optimization-based control,” or tracking formulations [8], [25]. SICE J. of Control, Measurement, and System Integration, vol. 10, pp. 39–52, 2017. [5] T. Faulwasser, L. Gru¨ne, and M. A. Mu¨ller, “Economic nonlinear model predictive control,” Foundations and Trends® in Systems and Control, vol. 5, pp. 1–98, 2018. -1 [6] R. Amrit, J. B. Rawlings, and D. Angeli, “Economic optimization using model predictive control with a terminal cost,” Annual Reviews in -2 Control, vol. 35, pp. 178–186, 2011. [7] L. Gru¨ne, “Economic receding horizon control without terminal con- -3 straints,” Automatica, vol. 49, pp. 725–734, 2013. [8] D. Limon, M. Pereira, D. M. De La Pen˜a, T. Alamo, and J. M. Grosso, -4 “Single-layer economic model predictive control for periodic operation,” J. Proc. Contr., vol. 24, pp. 1207–1224, 2014. -5 [9] Y. Wang, J. R. Salvador, D. M. de la Pen˜a, V. Puig, and G. Cembrano, “Economic model predictive control based on a periodicity constraint,” J. Proc. Contr., vol. 68, pp. 226–239, 2018. -6 [10] M. Pereira, D. Limon, D. M. de la Pen˜a, L. Valverde, and T. Alamo, “Periodic economic control of a nonisolated microgrid,” IEEE Trans Ind -7 Electron, vol. 62, no. 8, pp. 5247–5255, 2015. [11] G. Serale, M. Fiorentini, A. Capozzoli, D. Bernardini, and A. Bemporad, 20 40 60 80 “Model predictive control (MPC) for enhancing building and HVAC system energy efﬁciency: Problem formulation, applications and oppor- tunities,” Energies, vol. 11, no. 3, p. 631, 2018. 2 [12] J. B. Rawlings, N. R. Patel, M. J. Risbeck, C. T. Maravelias, M. J. Fig. 5. Transient performance ℓ(x, u, y) = −x + y · (u − u ) in dynamic 2 s Wenzel, and R. D. Turney, “Economic MPC and real-time decision operation under online changing conditions for CSTR relative to steady-state making with application to large-scale hvac energy systems,” Computers operation ℓ = ℓ(x , u ): Proposed approach (blue, solid), tracking MPC [8], s s s & Chemical Engineering, vol. 114, pp. 89–98, 2018. [25] (red, dashed) with T = 20 and N ∈ [0, 50] and periodicity constraint [13] J. Bailey, F. Horn, and R. Lin, “Cyclic operation of reaction systems: MPC [21], [9] (green, dotted) with T ∈ [0, 90]. Effects of heat and mass transfer resistance,” AIChE Journal, vol. 17, pp. 818–825, 1971. [14] H. Budman and P. L. Silveston, “Control of periodically operated reactors,” in Periodic Operation of Chemical Reactors. Elsevier, 2013, VI. CONCLUSION pp. 543–567. [15] M. J. Risbeck, C. T. Maravelias, and J. B. Rawlings, “Uniﬁcation of We have presented an economic MPC framework that is closed-loop scheduling and control: State-space formulations, terminal applicable to nonlinear (periodically) time-varying problems constraints, and nominal theoretical properties,” Computers & Chemical Engineering, vol. 129, p. 106496, 2019. and online changing operation conditions. We have shown [16] M. Diehl, L. Magni, and G. De Nicolao, “Efﬁcient NMPC of unstable recursive feasibility, constraint satisfaction and derived per- periodic systems using approximate inﬁnite horizon closed loop costing,” formance guarantees relative to periodic (locally) optimal Annual Reviews in Control, vol. 28, no. 1, pp. 37–45, 2004. [17] M. Zanon, L. Gru¨ne, and M. Diehl, “Periodic optimal control, dissipa- operation. Interestingly, the problem of economic periodic tivity and MPC,” IEEE Trans. Autom. Control, vol. 62, pp. 2943–2949, operation requires additional techniques compared to opti- mal steady-state operation, which was also the case in [19]. [18] A. Alessandretti, A. P. Aguiar, and C. N. Jones, “On convergence and performance certiﬁcation of a continuous-time economic model predic- In particular, we used a novel continuity condition on the tive control scheme with time-varying performance index,” Automatica, economic cost of periodic orbits, to reformulate the update vol. 68, pp. 305–313, 2016. scheme and constraints on the artiﬁcial periodic reference [19] M. A. Mu¨ller and L. Gru¨ne, “Economic model predictive control without terminal constraints for optimal periodic behavior,” Automatica, vol. 70, trajectory. In addition, we proposed novel ofﬂine computations pp. 128–139, 2016. to obtain suitable terminal ingredients. We have demonstrated [20] L. Gru¨ne and S. Pirkelmann, “Economic model predictive control for the applicability and practicality of the proposed framework time-varying system: Performance and stability results,” Optimal Control Applications and Methods, 2018. for economic periodic control with an HVAC and a CSTR [21] B. Houska and M. A. Mu¨ller, “Cost-to-travel functions: A new per- example. The strength of the framework comes from the fact spective on optimal and model predictive control,” Systems & Control that many existing schemes [26], [27], [17], [6], [38], [2] Letters, vol. 106, pp. 79–86, 2017. 19 [22] D. Limon, I. Alvarado, T. Alamo, and E. F. Camacho, “MPC for [47] J. Andersson, J. Akesson, and M. Diehl, “Casadi: A symbolic package tracking piecewise constant references for constrained linear systems,” for automatic differentiation and optimal control,” in Recent advances Automatica, vol. 44, pp. 2382–2387, 2008. in algorithmic differentiation. Springer, 2012, pp. 297–307. [48] P. O. Scokaert, D. Q. Mayne, and J. B. Rawlings, “Suboptimal model [23] D. Limon, M. Pereira, D. M. de la Pen˜a, T. Alamo, C. N. Jones, and predictive control (feasibility implies stability),” IEEE Trans. Autom. M. N. Zeilinger, “MPC for tracking periodic references,” IEEE Trans. Control, vol. 44, pp. 648–654, 1999. Autom. Control, vol. 61, pp. 1123–1128, 2016. [24] D. Limon, A. Ferramosca, I. Alvarado, and T. Alamo, “Nonlinear MPC for tracking piece-wise constant reference signals,” IEEE Trans. Autom. Control, vol. 63, pp. 3735–3750, 2018. [25] J. Ko¨hler, M. A. Mu¨ller, and F. Allgo¨wer, “A nonlinear tracking model predictive control scheme for unreachable dynamic target signals,” Automatica, vol. 118, p. 109030, 2020. [26] M. A. Mu¨ller, D. Angeli, and F. Allgo¨wer, “Economic model predictive control with self-tuning terminal cost,” European Journal of Control, vol. 19, pp. 408–416, 2013. [27] ——, “On the performance of economic model predictive control with self-tuning terminal cost,” J. Proc. Contr., vol. 24, pp. 1179–1186, 2014. [28] L. Fagiano and A. R. Teel, “Generalized terminal state constraint for model predictive control,” Automatica, vol. 49, pp. 2622–2631, 2013. [29] A. Ferramosca, D. Limon, and E. F. Camacho, “Economic MPC for a changing economic criterion for linear systems,” IEEE Trans. Autom. Control, vol. 59, pp. 2657–2667, 2014. [30] T. J. Broomhead, C. Manzie, R. C. Shekhar, and P. Hield, “Robust periodic economic MPC for linear systems,” Automatica, vol. 60, pp. 30–37, 2015. [31] J. Gutekunst, H. G. Bock, and A. Potschka, “Economic NMPC for averaged inﬁnite horizon problems with periodic approximations,” Au- tomatica, vol. 117, p. 109001, 2020. [32] J. B. Rawlings, D. Bonne´, J. B. Jorgensen, A. N. Venkat, and S. B. Jorgensen, “Unreachable setpoints in model predictive control,” IEEE Trans. Autom. Control, vol. 53, pp. 2209–2215, 2008. [33] M. J. Risbeck and J. B. Rawlings, “Economic model predictive control for time-varying cost and peak demand charge optimization,” IEEE Transactions on Automatic Control, 2019. [34] D. Angeli, A. Casavola, and F. Tedesco, “Theoretical advances on economic model predictive control with time-varying costs,” Annual Reviews in Control, vol. 41, pp. 218–224, 2016. [35] K. P. Wabersich, F. A. Bayer, M. A. Mu¨ller, and F. Allgo¨wer, “Economic model predictive control for robust periodic operation with guaranteed closed-loop performance,” in Proc. European Control Conf. (ECC), 2018, pp. 507–513. [36] M. A. Mu¨ller, L. Gru¨ne, and F. Allgo¨wer, “On the role of dissipativity in economic model predictive control,” in Proc. 5th IFAC Conf. Nonlinear Model Predictive Control, 2015, pp. 110–116. [37] J. Ko¨hler, M. A. Mu¨ller, and F. Allgo¨wer, “A nonlinear model predictive control framework using reference generic terminal ingredients,” IEEE Trans. Autom. Control, vol. 65, no. 8, pp. 3576–3583, 2020, extended version online: arXiv:1909.12765. [38] A. Alessandretti, A. P. Aguiar, and C. N. Jones, “On the design of discrete-time economic model predictive controllers,” in Proc. 55th IEEE Conf. Decision and Control (CDC). IEEE, 2016, pp. 3196–3201. [39] M. Zanon and T. Faulwasser, “Economic MPC without terminal con- straints: Gradient-correcting end penalties enforce asymptotic stability,” J. Proc. Contr., vol. 63, pp. 1–14, 2018. [40] I. R. Manchester and J.-J. E. Slotine, “Control contraction metrics: Convex and intrinsic criteria for nonlinear feedback design,” IEEE Trans. Autom. Control, vol. 62, pp. 3046–3053, 2017. [41] M. Zamani, N. van de Wouw, and R. Majumdar, “Backstepping con- troller synthesis and characterizations of incremental stability,” Systems & Control Letters, vol. 62, no. 10, pp. 949–962, 2013. [42] J. Ko¨hler, M. A. Mu¨ller, and F. Allgo¨wer, “On periodic dissipativity notions in economic model predictive control,” IEEE Control Systems Letters, vol. 2, pp. 501–506, 2018. [43] A. A. Ahmadi and P. A. Parrilo, “Non-monotonic Lyapunov functions for stability of discrete time nonlinear and switched systems,” in Proc. 47th IEEE Conf. Decision and Control (CDC), 2008, pp. 614–621. [44] E. Aydiner, M. A. Mu¨ller, and F. Allgo¨wer, “Periodic reference tracking for nonlinear systems via model predictive control,” in Proc. European Control Conf. (ECC), 2016, pp. 2602–2607. [45] M. A. Mu¨ller, D. Angeli, F. Allgo¨wer, R. Amrit, and J. B. Rawlings, “Convergence in economic model predictive control with average con- straints,” Automatica, vol. 50, pp. 3100–3111, 2014. [46] J. F. Sturm, “Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones,” Optimization methods and software, vol. 11, pp. 625–653, 1999. 20 APPENDIX ﬁrst 1000 steps are discarded to better reﬂect the asymptotic average performance. A. Continuous stirred-tank reactor - details First, Figure 6 shows that we can in general achieve better In the following, we provide additional details regarding performance by using a dynamic operation. We can see that the implementation of the CSTR example in Section V-B increasing the length T of the optimal T -periodic orbit (2) and investigate the economic performance of the different can increase the average production by approximately 8%. We considered implementations in more detail. would like to point out, that the “optimal” T-periodic orbits Implementation details: In the implementation, we consider where computed using CasADi and may thus also represent c = 100 β = 10. The ofﬂine computation is done with an local minima. In addition to the periodic orbit, we also consid- Intel Core i7 using the semideﬁnite programming (SDP) solver ered and implemented an economic MPC scheme without any SeDuMi-1.3 [46] and the online optimization is done with terminal ingredients or artiﬁcial periodic trajectories [7], [19], CasADi [47]. The iterations in CasADi were stopped after [20]. We would ﬁrst like to point out again, that in general 1000 iterations, although typically the standard tolerance was for this setup (nonlinear system, unknown mode of optimal satisﬁed. Since the optimization problem (6) is not solved to dynamic operation), guaranteeing a priori desired performance optimality but a ﬁxed number of iterations are done online, bounds with such an implementation is difﬁcult. Neverthe- the resulting reference r is not necessarily a feasible periodic less, this implementation supports two important arguments orbit. Hence, the primal infeasibility of the reference r should of the proposed approach. First, we can see that for short be taken into account when deﬁning κ in (7c) (e.g. in terms horizons N (e.g. N ≤ 20 here) the performance of such of a penalty factor), to avoid persistent feasibility issues. Thus, a simple implementation without suitable modiﬁcations can we replaced (7c) by be worse than steady-state performance and thus worse than ∗ 3 implementing a simple tracking/stabilizing MPC. Second, we κ (t + 1) =ℓ(r (j + 1|t), t + N + 1 + j, y(t + 1)) + 10 ǫ, can see for large horizons N, e.g. N ≥ 50, that it is possible to where ǫ is the largest constraint violation (measured in terms of obtain signiﬁcantly better performance than periodic operation the inﬁnity norm) in the periodicity constraint (6g). In addition, with a ﬁxed period length T , which can be obtained with we replaced the resulting optimized trajectory by the feasible periodic tracking MPC schemes like [8], [23], [25]. We also −4 candidate solution in case it has a worse cost (up to 10 ) or included a scheme with multi-step implementation, since there −3 does not satisfy the constraints (up to 10 ), which ensures exist theoretical reasons to expect better performance if the that the performance guarantees in Proposition 2 remain valid optimal period length is used [19]. However, as it is not clear independent of numerical issues . if the system is optimally periodically operated and/or with Ofﬂine computations: For the design of the terminal cost which period length, we simply implemented a small value V and the terminal set X , we proceed along the lines of f f ν = 3, which, however, did not show any performance beneﬁts Algorithm 1. We consider S = diag(0, I ) ℓ and 2ǫ+ǫ˜ = m rr (except for the reduced computational demand). 0.1 in (35). The matrices P (r), K(r) are computed using the LMIs from [37, Lemma 2] and gridding (x , x , x , u, u ) 1 2 3 using 20 · 2 = 16.000 points, which takes 37 minutes. The terminal set is X = {x| kx − x k ≤ α}, with α = f r P (r) −8 −8 1.2 · 10 . The veriﬁcation of α = α = 1.2 · 10 along 1 2 the lines of [37, Alg. 1] took approximately 45 minutes. For this example, the convex formulation [37, Prop. 1] can only be used if the sampling time h is reduced and the resulting terminal cost tends to be very conservative. For details on the different formulations, compare also [37, Example 1]. In addition, we computed a constant c ≈ 21 ofﬂine, such that -5 the simpler terminal cost proposed in Remark 5 also satisﬁes 0 50 100 150 Assumptions 1 and 3. Fig. 6. Average performance improvement due to dynamic operation relative Average performance improvement to optimal steady-state x - CSTR. Periodic orbit (blue, solid, periodic) with We ﬁrst consider the general economic performance with period length T , unconstrained economic MPC (red, dashed, UCON) with horizon N and unconstrained economic MPC with multi-step implementation the ﬁxed economic stage cost ℓ(x, u) = −x , using y = using ν = 3 (green, dotted, UCON - ν = 3). 0 and compare the performance relative to the perfor- mance at the optimal steady-state x = (x , x , x ) = s 1,s 2,s 3,s We implemented the proposed approach (Problem (6)) using (0.0832; 0.0846; 0.1491). The performance is speciﬁed as im- N ∈ [0, 50], T ∈ {1, 10, 20}, β = 10 and c = 100. provement (in %) of the average concentration x and is For the terminal ingredients; we implemented the design computed over the time-interval t ∈ [1000, 2000], where the in Corollary 2 (QINF), the positive deﬁnite terminal cost V While ideally any solver should guarantee that the resulting solution is no worse than any provided feasible initial solution [48], this was not always We did not formally verify satisfaction of Assumption 5 with c = 100. the case in the considered implementation with IPOPT, probably due to the Instead ,we simply checked numerically that the proposed approach was able difﬁculty of initializing the dual variables. to converge to (local) optimal periodic orbits r with the chosen value. T 21 from Remark 5 (QINF-pdf), and a terminal equality constraint (TEC) with ν = 3 (Lemma 4). The corresponding results can be seen in Figure 7. For the economic terminal cost V from Corollary 2, the constraints (29) to compute p(·|t) are implemented using (36) from Remark 4. We also tested the alternative explicit formulation (37), however, except for the trivial case T = 1, this lead to numerical difﬁculties and a signiﬁcant increase in online iterations without any major beneﬁt. We also implemented the terminal equality constraint (TEC) MPC using a one-step implementation (ν = 1), which resulted in an improved performance for T, N large. If we use an artiﬁcial setpoint (T = 1, top ﬁgure), we 0 10 20 30 40 50 can a) clearly see that the performance further improves if we increase the prediction horizon N; b) we can see that a terminal cost/set (QINF) improves the performance relative to a terminal equality constraint (TEC). For example, the performance with N = 50, T = 1 with TEC is worse than the performance with a terminal cost using N = 40, T = 1, thus showing the potential for performance improve- ment/computational saving using suitable terminal ingredients. On the other hand, the difference between the positive deﬁnite terminal cost (QINF-pdf) and the economic terminal cost (QINF) is often small and thus the simple design in Remark 5 may be most favourable for practical implementations. For T = 10 (middle ﬁgure) and T = 20 (bottom ﬁgure) we can see 0 10 20 30 40 50 similar performance differences for larger horizons N: We can achieve better performance with a large horizon N or larger period length T , and we can achieve better performance if we use a terminal cost (QINF, QINF-pdf) instead of a terminal equality constraint (TEC). We also implemented the different terminal ingredients with the modiﬁed cost V from Lemma 5, compare Figure 8. If we compare the results in Figure 7 and Figure 8, the performance is essentially similar. Thus, the modiﬁed formula V (Lemma 5) does not seem to have any particularly negative impact on the performance, while reducing the computational demand by replacing the constraints (6h)–(6i) with (42). 0 10 20 30 40 50 We also implemented the continuous-time formulation dis- cussed in Remark 2 with a variable sampling time h ∈ −2 −2 [h , h ] = [10 , 5 · 10 ]. The resulting performance min max Fig. 7. Average performance improvement relative to optimal steady-state x for T = 1 (top), T = 10 (middle) and T = 20 (bottom). Approach for the optimal T -periodic orbit (2), and an economic MPC s with economic terminal cost (red, solid, QINF); positive deﬁnite terminal cost scheme without any terminal ingredients (UNCON) can be (blue, dashed, QINF-pdf); terminal equality constraint (black, dotted, TEC) seen in Figure 9. For comparison, we also implemented the with prediction horizon N ∈ [1, 50]. Performance at the optimal T -periodic orbit is show in green, dashed for comparison. proposed approach with a terminal equality constraint (TEC, Lemma 4), ν = 3 and T = 20. We can see that the variable sampling time h signiﬁcantly improves the overall perfor- We also tested the naive implementation discussed in Sec- mance compared to the ﬁxed sampling time implementation tion II-C. Although this approach is in general not satisfactory shown in Figure 6. Obtaining improved terminal ingredients (as shown in Sec. II-C), in the considered example this similar to Corollary 2 for this scenario, however, requires 11 approach often performs similar to the proposed formulation. further research . However, for small horizons N and a small weighting β, ne- glecting the constraints (6h)–(6i) can yield worse performance As the continuous-time analogue to (30) in Prop. 4, we can design 2 2 than steady-state operation, similar to the implementation a quadratic term satisfying kx − x k ≤ −kx − x k r r dt P(r ,t) Q (r ,t) T T using [37, Lemma 4/Prop. 5], which only requires 10s and 53s, respectively without terminal ingredients (UCON) in Figure 6. (compared to 37 min for the discrete-time formulation). However, a suitable Summary: Improved performance: In this scenario, we terminal cost V also requires a simple means to compute a corresponding have only considered the case of ﬁxed known stage cost ℓ gradient correction term p in continuous-time. In addition, due to the average dV (y constant). In this case, the proposed framework reduces to cost function proposed in Remark 2, we need ≤ −ℓ /h, which might dt only hold by scaling the terminal cost conservatively using . the periodic economic MPC approaches in [17], [18], [15], min 22 0 50 100 150 200 0 10 20 30 40 50 Fig. 9. Average performance improvement relative to optimal steady-state x with ﬂexible sampling time h. Periodic orbit (blue, solid, periodic cont.) with period length T , unconstrained economic MPC (red, dashed, UCON cont.) with horizon N and terminal equality constraint using ν = 3 and T = 20 (green, dotted, TEC T = 20 cont.) (y constant). The second scenario investigates this difference in performance for the transient case of changing operation conditions (y not constant). Different formulations: A second key ﬁnding in this scenario is the comparison of the different design choices 0 10 20 30 40 50 possible in the proposed framework, in particular the termi- nal ingredients (V ,X , Ass. 1, 3 Corollary 2, Remark 5, f f Lemma 4) and the modiﬁed cost function (V , Lemma 5, Prop. 6). First, even though the cost formulation V proposed in Lemma 5 may be unconventional, we have not found any dis- advantages in the performance comparison. At the same time, this formulation enjoys stronger theoretical properties (does not requires Ass. 5) and is easier to implement (T nonlinear constraints (6h)–(6i) are replaced by one constraint (42)). Regarding the terminal cost V , we can see that a properly designed terminal cost V (Corollary 2, Remark 5) allows 0 us to achieve a better performance with a shorter horizon 0 10 20 30 40 50 N compared to a simple terminal equality constraint (TEC, Lemma 4). However, the difference in performance between the terminal cost V in Corollary 2 and Remark 5 seems rather Fig. 8. Average performance improvement relative to optimal steady-state x s f for T = 1 (top), T = 10 (middle) and T = 20 (bottom) using the modiﬁed small, while the design in Corollary 2 requires T ·n additional cost V . Approach with economic terminal cost (red, solid, QINF); positive nonlinear constraints (36), which does not seem appropriate deﬁnite terminal cost (blue, dashed, QINF-pdf); terminal equality constraint for T >> 1 given the small performance improvement. (black, dotted, TEC) with prediction horizon N ∈ [1, 50]. Performance at the optimal T -periodic orbit is show in green, dashed for comparison. Given this comparison, the most suitable formulation seems to be the positive deﬁnite terminal cost V from Remark 5 combined with the modiﬁed cost V in Lemma 5, although with a ﬁxed periodic trajectory r (assuming convergence the simple terminal equality constraint may also be appropriate of r and considering the special case of periodic operation if the design in Alg. 1 is not applicable. In addition, further in [18]). We have seen that the proposed fully economic investigations into continuous-time formulations for dynamic formulation yields a better performance than operating the operations along the lines of Remark 2 may yield signiﬁcant system at the optimal steady-state (T = 1) or optimal periodic performance improvements. orbit (T > 1). In particular, this implies that the proposed framework utilizing periodic orbits (T > 1) and additional Transient performance under online changing conditions predictions (N ≥ 1), both with the economic stage cost ℓ, outperforms periodicity constrained formulations [21], [9] In the following, we provide additional details regarding (N = 0), steady-state formulations [26], [27], [28], [29] the transient performance comparison. The tracking MPC uses (T = 1) and periodic tracking formulations [8], [23], [24], [25] the quadratic part of the economic terminal cost from Prop. 4 (ℓ positive deﬁnite), even in case of ﬁxed optimal operation V (x, r) = kx− x k (similar to [37]) and the quadratic f,tr r P (r) 23 2 2 tracking cost ℓ (x, u, r) = kx − x k + ku − u k with tr r r Q R Q = 0.05, R = 1. First, we consider T = 20 and N = 10 as in Figure 4. In Figure 10, we see the closed-loop cost ℓ in the transition periods t ∈ [182, 220] and t ∈ [398, 440] (y → 1), for the proposed approach, the periodicity-constraint MPC [21], [9] and the tracking MPC [8], [25]. We can see that all approaches quickly converge to the optimal steady-state. Crucially, in the transient period, while the system converges to the optimal steady-state, the proposed approach and the tracking MPC [8], [25] seem to be able to utilize the transition period to further minimize the stage cost ℓ, while the periodicity constraint MPC [21], [9] does not seem to have the necessary degrees of freedom. The periodicity constraint MPC cannot really take advantage of the transient changes yielding faster convergence but overall small improvement (only 0.9%) compared to steady-state. For comparison, the proposed approach and the tracking MPC [8], [25] can improve overall performance by 3.4% and 2.9%, respectively. Thus, the proposed approach achieves better performance than the periodicity constraint MPC [21], [9] and the tracking MPC [8], [25], even in case of online changing operation conditions (y ) by utilizing a purely economic formulation with additional degrees of freedom. -0.084 -0.086 -0.088 -0.09 -0.092 -0.094 -0.096 190 200 210 220 -0.055 -0.06 -0.065 -0.07 -0.075 -0.08 -0.085 -0.09 400 410 420 430 440 Fig. 10. Transient performance in dynamic operation under online changing conditions for CSTR: Proposed approach (blue, solid), tracking MPC [8], [25] (red, dashed) with T = 20 and N = 10 and periodicity constraint MPC [21], [9] (green, dotted) with T = 20 at t ∈ [182, 220] (top) and t ∈ [398, 440] (bottom). http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Mathematics arXiv (Cornell University) http://www.deepdyve.com/lp/arxiv-cornell-university/periodic-optimal-control-of-nonlinear-constrained-systems-using-n4pQxRomI2

Loading next page...

References (48)

J. Rawlings, Nishith Patel, M. Risbeck, C. Maravelias, M. Wenzel, R. Turney (2017)
Economic MPC and real-time decision making with application to large-scale HVAC energy systems
Comput. Chem. Eng., 114
A. Alessandretti, Antonio Aguiar, C. Jones (2016)
On the design of discrete-time Economic Model Predictive Controllers
2016 IEEE 55th Conference on Decision and Control (CDC)
M. Müller, D. Angeli, F. Allgöwer, R. Amrit, J. Rawlings (2014)
Convergence in economic model predictive control with average constraints
Autom., 50
Lars Grüne (2013)
Economic receding horizon control without terminal constraints
Autom., 49
M. Müller, L. Grüne, F. Allgöwer (2015)
On the role of dissipativity in economic model predictive control
IFAC-PapersOnLine, 48
Johannes Kohler, Matthias Muller, F. Allgower (2019)
A nonlinear tracking model predictive control scheme for dynamic target signals
Automatica
Mario Zanon, T. Faulwasser (2018)
Economic MPC without terminal constraints: Gradient-correcting end penalties enforce asymptotic stability
Journal of Process Control, 63
Johannes Kohler, Matthias Muller, F. Allgower (2019)
A nonlinear model predictive control framework using reference generic terminal ingredients -- extended version
J. Bailey, F. Horn, R. Lin (1971)
Cyclic operation of reaction systems: Effects of heat and mass transfer resistance
Aiche Journal, 17
Gianluca Serale, M. Fiorentini, Alfonso Capozzoli, Daniele Bernardini, A. Bemporad (2018)
Model Predictive Control (MPC) for Enhancing Building and HVAC System Energy Efficiency: Problem Formulation, Applications and Opportunities
Energies, 11
M. Müller, D. Angeli, F. Allgöwer (2013)
Economic model predictive control with self-tuning terminal cost
Eur. J. Control, 19
M. Müller, D. Angeli, F. Allgöwer (2014)
On the performance of economic model predictive control with self-tuning terminal cost
Journal of Process Control, 24
L. Fagiano, A. Teel (2012)
Generalized terminal state constraint for model predictive control
Autom., 49
D. Limón, A. Ferramosca, I. Alvarado, T. Alamo (2018)
Nonlinear MPC for Tracking Piece-Wise Constant Reference Signals
IEEE Transactions on Automatic Control, 63
H. Budman, P. Silveston (2008)
Control of periodically operated reactors
Chemical Engineering Science, 63
Johannes Köhler, M. Müller, F. Allgöwer (2018)
On Periodic Dissipativity Notions in Economic Model Predictive Control
IEEE Control Systems Letters, 2
M. Müller, F. Allgöwer (2017)
Economic and Distributed Model Predictive Control: Recent Developments in Optimization-Based Control
SICE journal of control, measurement, and system integration, 10
M. Risbeck, C. Maravelias, J. Rawlings (2019)
Unification of closed-loop scheduling and control: State-space formulations, terminal constraints, and nominal theoretical properties
Comput. Chem. Eng., 129
K. Wabersich, Florian Bayer, M. Müller, F. Allgöwer (2018)
Economic Model Predictive Control for Robust Periodic Operation with Guaranteed Closed-Loop Performance
2018 European Control Conference (ECC)
Johannes Köhler, M. Müller, F. Allgöwer (2020)
A nonlinear tracking model predictive control scheme for dynamic target signals
Autom., 118
T. Faulwasser, L. Grüne, M. Müller (2018)
Economic Nonlinear Model Predictive Control
Found. Trends Syst. Control., 5
Khadir Mohamed (2014)
Model Predictive Control: Theory and Design
M. Müller, L. Grüne (2015)
Economic model predictive control without terminal constraints for optimal periodic behavior
Autom., 70
Majid Zamani, N. Wouw, R. Majumdar (2012)
Backstepping controller synthesis and characterizations of incremental stability
Syst. Control. Lett., 62
B. Houska, M. Müller (2017)
Cost-to-travel functions: A new perspective on optimal and model predictive control
Syst. Control. Lett., 106
A. Ferramosca, D. Limón, E. Camacho (2014)
Economic MPC for a Changing Economic Criterion for Linear Systems
IEEE Transactions on Automatic Control, 59
L. Grüne, S. Pirkelmann (2019)
Economic model predictive control for time‐varying system: Performance and stability results
Optimal Control Applications and Methods, 41
Ye Wang, J. Salvador, D. Peña, V. Puig, G. Cembraño (2018)
Economic model predictive control based on a periodicity constraint
Journal of Process Control
Emre Aydiner, M. Müller, F. Allgöwer (2016)
Periodic reference tracking for nonlinear systems via model predictive control
2016 European Control Conference (ECC)
Mario Zanon, L. Grüne, M. Diehl (2017)
Periodic Optimal Control, Dissipativity and MPC
IEEE Transactions on Automatic Control, 62
D. Limón, M. Pereira, D. Peña, T. Alamo, C. Jones, M. Zeilinger (2016)
MPC for Tracking Periodic References
IEEE Transactions on Automatic Control, 61
J. Rawlings, Dennis Bonné, J. Jørgensen, Aswin Venkat, S. Jørgensen (2008)
Unreachable Setpoints in Model Predictive Control
IEEE Transactions on Automatic Control, 53
P. Scokaert, D. Mayne, J. Rawlings (1999)
Suboptimal model predictive control (feasibility implies stability)
IEEE Trans. Autom. Control., 44
M. Risbeck, J. Rawlings (2020)
Economic Model Predictive Control for Time-Varying Cost and Peak Demand Charge Optimization
IEEE Transactions on Automatic Control, 65
D. Angeli, R. Amrit, J. Rawlings (2012)
On Average Performance and Stability of Economic Model Predictive Control
IEEE Transactions on Automatic Control, 57
Amir Ahmadi, P. Parrilo (2008)
Non-monotonic Lyapunov functions for stability of discrete time nonlinear and switched systems
2008 47th IEEE Conference on Decision and Control
D. Limón, I. Alvarado, T. Alamo, E. Camacho (2008)
MPC for tracking piecewise constant references for constrained linear systems
Autom., 44
I. Manchester, J. Slotine (2015)
Control Contraction Metrics: Convex and Intrinsic Criteria for Nonlinear Feedback Design
IEEE Transactions on Automatic Control, 62
M. Diehl, L. Magni, G. Nicolao (2004)
Efficient NMPC of unstable periodic systems using approximate infinite horizon closed loop costing
Annu. Rev. Control., 28
M. Pereira, D. Limón, D. Peña, Luis Valverde, T. Alamo (2015)
Periodic Economic Control of a Nonisolated Microgrid
IEEE Transactions on Industrial Electronics, 62
Joel Andersson, J. Åkesson, M. Diehl (2012)
CasADi -- A symbolic package for automatic differentiation and optimal control
J. Sturm, Gerardo Guerra (1999)
A Matlab toolbox for optimization over symmetric cones
A. Alessandretti, Antonio Aguiar, C. Jones (2016)
On convergence and performance certification of a continuous-time economic model predictive control scheme with time-varying performance index
Autom., 68
R. Amrit, J. Rawlings, D. Angeli (2011)
Economic optimization using model predictive control with a terminal cost
Annu. Rev. Control., 35
D. Limón, M. Pereira, D. Peña, T. Alamo, J. Grosso (2014)
Single-layer economic model predictive control for periodic operation
Journal of Process Control, 24
Timothy Broomhead, C. Manzie, R. Shekhar, P. Hield (2015)
Robust periodic economic MPC for linear systems
Autom., 60
Jürgen Gutekunst, H. Bock, A. Potschka (2020)
Economic NMPC for averaged infinite horizon problems with periodic approximations
Autom., 117
D. Angeli, A. Casavola, Francesco Tedesco (2016)
Theoretical advances on Economic Model Predictive Control with time-varying costs
Annu. Rev. Control., 41

ISSN: 0959-1524
eISSN: ARCH-3343
DOI: 10.1016/j.jprocont.2020.06.004
Publisher site: See Article on Publisher Site

Abstract

Periodic optimal control of nonlinear constrained systems using economic model predictive control 1 2 1 Johannes Ko¨hler , Matthias A. Mu¨ller , Frank Allgo¨wer Abstract—In this paper, we consider the problem of periodic and HVAC [11], [12] due to the inherent periodicity (day- optimal control of nonlinear systems subject to online changing night cycle) in changing price signals (supply and demand) and periodically time-varying economic performance measures or dynamics (outside temperature). Even in the time-invariant using model predictive control (MPC). The proposed economic problem of maximizing the production in (nonlinear) contin- MPC scheme uses an online optimized artiﬁcial periodic orbit uous stirred-tank reactors (CSTR), periodic operation can be to ensure recursive feasibility and constraint satisfaction despite unpredictable changes in the economic performance index. We economically beneﬁcial, compare [13], [14]. Periodic oper- demonstrate that the direct extension of existing methods to ations also naturally arise in periodic/cyclic scheduling [15] periodic orbits does not necessarily yield the desirable closed- and power generation using kites [16]. In addition to the loop economic performance. Instead, we carefully revise the challenges related to dynamic/periodic operation, the external constraints on the artiﬁcial trajectory, which ensures that the operating conditions may change unpredictably and the sys- closed-loop average performance is no worse than a locally optimal periodic orbit. In the special case that the prediction tem is expected to reliably and economically operate despite horizon is set to zero, the proposed scheme is a modiﬁed version these changes. In this paper, we present an economic MPC of recent publications using periodicity constraints, with the framework that provides economic performance guarantees for important difference that the resulting closed loop has more periodic operation of nonlinear systems. degrees of freedom which are vital to ensure convergence to an Related Work: In [17], [18], [15] performance guarantees optimal periodic orbit. In addition, we detail a tailored ofﬂine computation of suitable terminal ingredients, which are both are obtained by using terminal constraints for a-priori known theoretically and practically beneﬁcial for closed-loop perfor- optimal periodic orbits. Online changes in the optimal system mance improvement. Finally, we demonstrate the practicality operation cannot be incorporated. and performance improvements of the proposed approach on In [7] an economic MPC scheme without terminal ingredi- benchmark examples. ents is studied, which has recently been extended to periodic Index Terms—Nonlinear model predictive control, economic operation [19] and time-varying problems [20]. The resulting MPC, dynamic real time optimization, periodic optimal control, closed-loop performance guarantees are, however, only valid changing economic criteria if a potentially large prediction horizon is used. In [21], [9] an economic MPC scheme based on a peri- I. INTRODUCTION odicity constraint is suggested, which minimizes the cost of a periodic orbit starting at the current measured state. The Model Predictive Control (MPC) is a well established con- convergence to the optimal periodic orbit can be theoretically trol method that can cope with nonlinear dynamics, hard state studied based on convergence results for coordinate descent and input constraints, and the inclusion of general performance methods [21, Lemma 3, Thm. 4] or online examined based criteria [1]. Economic MPC [2], [3], [4], [5] is a variant on dual variables [9, Thm. 1]. Crucially, even in the linear of MPC that directly aims at improving a user speciﬁed convex case, the closed loop does not necessarily converge to economic performance index instead of stabilizing some given the optimal periodic orbit [9, Example 6]. setpoint or trajectory. In case the system is optimally operated A promising approach to deal with online changing con- at a given setpoint, there exist well established methods to ditions is to simultaneously optimize an artiﬁcial reference, design economic MPC schemes with closed-loop performance which is a well established method in setpoint tracking guarantees [2], [6], [7], [5]. MPC [22] and has recently been extended to periodic tracking Motivation: Many practical problems require a paradigm and nonlinear systems [23], [24], [25]. This idea of using that goes beyond steady-state operation and embraces dynamic an artiﬁcial setpoint in combination with an external update operation and online changing conditions. Periodic operation scheme are used in [26], [27], [28] to design economic MPC is, e.g., economically beneﬁcial (and necessary) in water dis- schemes with performance guarantees relative to steady-state tribution networks [8], [9], electrical networks [10], buildings operation, compare also [29]. Johannes Ko¨hler and Frank Allgo¨wer are with the Institute for Systems In [8] for linear systems an artiﬁcial reference is used to Theory and Automatic Control, University of Stuttgart, 70550 Stuttgart, compute the economic optimal periodic orbit and a tracking Germany. (email:{johannes.koehler, frank.allgower}@ist.uni-stuttgart.de). 2 stage cost is used to ensure stability, compare also [30]. Matthias A. Mu¨ller is with the Institute of Automatic Control, Leibniz University Hannover, 30167 Hannover, Germany. (email:mueller@irt.uni- Recently, in [31] a nonlinear version has been proposed, hannover.de). which also allows to optimize over the period length us- This work was supported by the German Research Foundation (DFG) ing a continuous-time formulation. Furthermore, instead of a under Grants GRK 2198/1 - 277536708, AL 316/12-2, and MU 3929/1-2 - 279734922. standard tracking stage cost a regularization with respect to ©2020 Elsevier Ltd. All rights reserved. arXiv:2005.05245v2 [eess.SY] 20 Oct 2020 2 the non-periodicity in the input and economic cost is used, II. PERIODIC ECONOMIC MODEL PREDICTIVE CONTROL which also guarantees convergence to the optimal periodic In Section II-A we introduce the problem setup. Some orbit under appropriate conditions (controllability, no local existing economic MPC methods for this problem setup are minima,. . . ). However, the usage of a tracking cost/periodic brieﬂy discussed in Section II-B. We demonstrate the potential regularization can reduce the transient economic performance, difﬁculties in periodic problems (compared to steady-state) compare, .e.g. [32]. with a simple system in Section II-C. Contribution: We present an economic MPC scheme that ensures recursive feasibility, constraint satisfaction and perfor- A. Problem setup mance guarantees for nonlinear systems despite unpredictable We consider periodic problems with a ﬁxed known period changes in the economic performance index. Recursive feasi- length T ∈ N. For many systems (HVAC, water distribution bility is achieved by including an artiﬁcial periodic reference networks) this periodicity is inherent to the problem setup (in trajectory in the online optimization. We use a self-tuning the dynamics and/or cost function). In time-invariant problems weight for the cost of the artiﬁcial reference trajectory in order (chemical reactor) this period length T is a user speciﬁed to obtain suitable bounds on the closed-loop performance, as decision variable that inﬂuences the possible performance an extension to [26], [27], where this idea was introduced improvement (compared to the steady-state operation). Both with artiﬁcial setpoints. We demonstrate by means of a simple cases are illustrated in the numerical examples in Section V. motivating example (similar to [19]), that a direct extension We consider nonlinear periodically time-varying discrete- of [26], [27] to the periodic case does not necessarily yield time systems the desired closed-loop performance. Instead, we use a novel continuity condition and reformulate the constraints on the x(t + 1) = f(x(t), u(t), t) (1) optimal periodic orbit to ensure that: n m with the state x ∈ R , control input u ∈ R , and time step a) the average performance of the artiﬁcial reference con- t ∈ N. We assume that the dynamics are periodic with the verges to that of a locally optimal periodic trajectory, (known) period length T , i.e., f(x, u, t) = f(x, u, t + T ). b) the closed-loop average performance is no worse than that We impose point-wise in time constraints on the state and of the (limiting) artiﬁcial reference trajectory. input (x(t), u(t)) ∈ Z(t), with compact periodically time- n+m varying sets Z(t) ⊂ R , i.e., Z(t) = Z(t+T ). We consider In the special case of linear systems with convex cost and reference constraint sets r(t) = (x (t), u (t)) ∈ Z (t), that r r r convex constraints, the closed-loop average performance is no satisfy Z (t) = Z (t+T ) and Z (t) ⊆ int(Z(t)) for all t ≥ 0. r r r worse than the (globally) optimal periodic orbit. Furthermore, We deﬁne the set of feasible T -periodic reference trajectories if we consider a prediction horizon of N = 0, we obtain (n+m)T r = (x , u ) ∈ R as T r r T T a modiﬁed version of recent publications using periodicity constraints [21], [9], with the same number of optimization Z (t) ={r = (x , u )| T T r r T T variables and guaranteed convergence to a (local) optimum, x (0) = f(x (T − 1), u (T − 1), t + T − 1), r r r T T T which in [21], [9] can only be ensured under signiﬁcantly x (k + 1) = f(x (k), u (k), t + k), k = 0, . . . , T − 2, r r r T T T more restrictive conditions. r (k) ∈ Z (t + k), k = 0, . . . , T − 1}. T r Some of the improved performance properties require the usage of suitable terminal ingredients (economic terminal cost (n+m)T×(n+m)T We deﬁne a periodic shift matrix R ∈ R , and terminal set). To this end, we provide a novel design pro- which satisﬁes r ∈ Z (t) ⇒ R r ∈ Z (t + 1) and T T T T T cedure that is applicable to dynamic operation and economic T−1 Π R = R = I. Whenever clear from the context, j=0 T costs, as an extension of the approach in [6]. we denote the ﬁrst element of the periodic reference r by We demonstrate the practicality and performance of the (x , u ) = r = r (0). r r T proposed framework using a periodic time-varying HVAC The economic performance measure is given by a general system [33] and a time-invariant nonlinear CSTR [13]. (non-convex) periodically time-varying function Outline: Section II introduces the problem setup and ℓ(x, u, t, y) = ℓ(x, u, t + T, y), demonstrates that the periodic case requires additional care which can depend on external parameters y ∈ Y, with Y with a simple system. The proposed economic MPC frame- compact. At each time step t, the parameters y(t) ∈ Y are work with theoretical analysis is presented in Section III. assumed to be available as an external (user deﬁned) input. Additional details and variations are discussed in Section IV. These parameters might incorporate online changing prices Section V illustrates the results with numerical examples. Sec- and/or general changes in the desired production/operation. tion VI concludes the paper. Appendix A contains additional For simplicity, we assume that ℓ and f are continuous, which details regarding the CSTR example. (in combination with compact constraints) implies that ℓ is n×n Notation: The identity matrix is I ∈ R . The interior of bounded. a set X ⊂ R is int(X ). A ball with radius ǫ around a point n n x ∈ R is B (x) = {y ∈ R | kx−yk ≤ ǫ}. By K we denote Remark 1. We consider the setting with a constant parameter ǫ ∞ the class of functions α : R → R , which are continuous, y(t) in the predictions to simplify the notation. However, the ≥0 ≥0 strictly increasing, unbounded and satisfy α(0) = 0. presented guarantees hold equally if we consider a predicted 3 (periodic) sequence of parameters y(·|t). In case some of the for economic performance improvement and may hence result constraints in Z, Z are relaxed to soft constraints using in severe suboptimality, compare [32] and [5, Sec. 3.4]. penalty terms in the stage cost ℓ, the external parameters y Thus, to ensure theoretical properties (recursive feasibil- can also model online unpredictably changing constraints sets. ity, performance bounds) and allow for online performance improvement, we will use an artiﬁcial periodic reference An optimal T -periodic orbit at time t is the solution to the trajectory r ∈ Z and combine it with a purely economic T T following optimization problem formulation (without using any tracking costs), as an extension T−1 to [26], [27], [28]. In the numerical example in Section V-B, min J (r (·|t), t, y(t)) := ℓ(r (j|t), t + j, y(t)) T T T we demonstrate the advantages of such a formulation com- r (·|t)∈Z (t) T T j=0 pared to some of the existing methods. (2) and is denoted by r (·|t). If the external parameters remain C. Pitfalls - Generalized periodic constraints constant, i.e. y(t + 1) = y(t), then an optimal periodic trajec- In the following, we show that a direct (naive) extension of tory at the next time step is given by r (·|t+1) = R r (·|t), T T T existing generalized terminal setpoint constraints in [26], [27], i.e., r (1|t) = r (0|t + 1). T T [28] to periodic reference trajectories r , does not necessarily Given some initial state x(0), the closed-loop average imply the desirable economic performance guarantees and thus economic cost is deﬁned as requires further modiﬁcations (which will be introduced in K−1 Section III-A). J (x(0)) = lim sup ℓ(x(k), u(k), k, y(k)). (3) cl Consider a scalar time-invariant system with f(x, u) = u, K→∞ k=0 x ∈ {0, 1, 2} which is depicted as a graph in Figure 1 with some arbitrary small positive constant ǫ, similar to [19, The control goal is to minimize this closed-loop average Example 4]. The optimal periodic orbit is x = (1, 2) with cost (3) and achieve constraint satisfaction (x(t), u(t)) ∈ Z(t), r cost J (r ) = ǫ and T = 2. for all t ≥ 0. T T u=1 ℓ(x,u)=−1 B. Existing methods u=0 One way to approach this problem, would be to solve (2) u=1 0 1 2 ℓ(x,u)=1 ℓ(x,u)=0 ofﬂine to obtain r and then use terminal constraints, similar u=2 to [17], [18], [15]. If the parameters y stay constant, this ℓ(x,u)=1+ǫ strategy ensures the following performance bound J (x(0)) ≤ J (r )/T. (4) cl T T Fig. 1. Academic counter example - Illustration of feasible transitions. However, if the parameters y change online the performance The following economic MPC scheme with an artiﬁcial deteriorates. Directly recomputing the optimal periodic orbit periodic trajectory r can be viewed as a generalization of r and adjusting the terminal constraints based on online the steady-state methods in [26], [27], [28]: changes in the parameters y can cause feasibility issues. If N−1 there exists only a ﬁnite set of possible parameter values min ℓ(x(k|t), u(k|t)) + J (r (·|t)) (5a) T T y ∈ Y, feasible transition trajectories can be computed ofﬂine u(·|t),r (·|t) k=0 to avoid such issues, compare [34]. s.t. x(k + 1|t) = f(x(k|t), u(k|t)), x(0|t) = x(t), (5b) The issue of recomputing the optimal orbit r under (x(k|t), u(k|t)) ∈ Z, k = 0, . . . , N − 1, (5c) changing parameters y can be avoided by using economic MPC schemes without any terminal constraints [7], [19], [20] r (·|t) ∈ Z , x(N|t) = x (0|t). (5d) T T r which implicitly ﬁnd the optimal mode of operation (using This optimization problem computes an open-loop trajectory turnpike arguments). The corresponding theoretical properties x (·|t) starting at x(t) that ends on some periodic trajectory may require a-priori assumptions on the optimal mode of r ∈ Z . In closed-loop operation, the optimization prob- T T operation (dissipativity, turnpike, optimal period length T ), lem (5) would be solved in each time step t and the ﬁrst part which can be difﬁcult to verify for practical systems. In of the optimized input trajectory applied to the system, i.e., addition, a potentially very large prediction horizon N may ∗ ∗ u(t) = u (0|t), x(t + 1) = x (1|t). Although the steady-state be required to ensure these properties. schemes [26], [27], [28] often have additional modiﬁcations A reliable method to deal with these issues is to simul- (terminal cost, self-tuning weights, additional constraints on taneously optimize an artiﬁcial periodic reference r ∈ Z T T r ), the problem we discuss in the following remains the same. in the MPC problem and use a tracking formulation with Consider the initial condition x(0) = 0 and a prediction terminal constraints to stabilize this artiﬁcial reference tra- horizon of N = 2. The artiﬁcial reference is the optimal jectory, compare [8] and [22], [23], [24], [25]. This direct periodic orbit r (·|t) ∈ {1, 2}. The only feasible trajectories stabilization, however, does not take into account the potential that satisfy x(N|t) = x(2|t) ∈ {1, 2} are u(·|t) = (0, 1) and u(·|t) = (1, 2), and the corresponding open-loop cost is 1 + 0 In some cases, a multi-step MPC scheme based on the optimal period length T needs to be implemented, compare [19]. and 0 + 1 + ǫ, respectively. Thus, the optimal solution to (5) 4 satisﬁes x (1|t) = 0 = x(t). Correspondingly, the closed-loop Optimization problem: The corresponding optimization system based on (5) stays at x(t) = 0 and encounters the problem at each time step t is given by economic cost ℓ(x(t), u(t)) = 1, ∀t ≥ 0 and does not achieve N−1 the same performance as the artiﬁcial periodic reference r . min ℓ(x(k|t), u(k|t), t + k, y(t)) (6a) u(·|t),r (·|t) This issue can persist, even if we choose an arbitrarily large k=0 (even) prediction horizon N. In particular, with (5), we can + V (x(N|t), r (·|t), t + N, y(t)) (6b) f T only ensure + β(t) · J (r (·|t), t + N, y(t)) (6c) T T J (x(0)) ≤ max ℓ(r (k)) = 1 + ǫ. cl T s.t. x(k + 1|t) = f(x(k|t), u(k|t), t + k), x(0|t) = x(t), (6d) This is in contrast to existing results for the steady-state case (x(k|t), u(k|t)) ∈ Z(t + k), k = 0, . . . , N − 1, (6e) (T = 1) [26], [27], [28], which can ensure the superior bound (4). The same problem appears in economic MPC x(N|t) ∈ X (r (·|t), t + N), (6f) f T schemes without terminal constraints, compare [19, Examples r (·|t) ∈ Z (t + N), (6g) T T 4 and 18]. One way to alleviate this problem is to apply the T−1 ﬁrst T components of the open-loop input sequence u (·|t) Δκ(t) = [ℓ(r (j|t), t + N + j, y(t)) − κ (t)] , (6h) T j (multi-step MPC) [19], [35], which transforms the problem j=0 to a higher dimensional steady-state problem (T -step sys- ℓ(r (j|t), t + N + j, y(t)) ≤ κ (t) − c Δκ(t), (6i) T j κ tem [36], [19], [35]). Since we wish to consider problems with j = 0, . . . , T − 1, possibly large period lengths T , this solution seems, however, inadequate. If we would use an economic MPC scheme based with some positive constant c . The solution to (6) are optimal ∗ ∗ on periodicity constraints [21], [9], the closed-loop system state and input trajectories x (·|t), u (·|t) and an artiﬁcial would also stay at x(t) = 0 for all t ≥ 0, since there exists periodic reference trajectory r (·|t). The input trajectory only one feasible periodic orbit starting at x(0) = 0. The minimizes the predicted economic cost (6a) with a terminal theoretical results in [21] do not apply, since the one-step cost (6b), to be speciﬁed later (Ass. 1). The economic cost controllability condition [21, Lemma 4] is not satisﬁed. of the artiﬁcial periodic reference trajectory r is weighted with a self-tuning (time-varying) weight β(t) (6c), similar To summarize, as also discussed in [15], the existing ap- to [26], [27]. The resulting state and input trajectory satisfy the proaches with online optimized periodic reference trajectory dynamics (6d) and the posed state and input constraints (6e). In r do not come with any closed-loop performance guarantees addition, the terminal state of the predicted state sequence is in similar to (4). a terminal set (see Ass. 1 below) around the artiﬁcial reference trajectory (6f). The artiﬁcial reference is a feasible periodic orbit (6g). Conditions (6h)–(6i) pose additional constraints on the improvement of the economic cost of the artiﬁcial reference r compared to κ , similar to [26], [27], [28]. In T j III. PROPOSED PERIODIC ECONOMIC MPC FRAMEWORK particular, if Δκ is negative (the cost J of the reference improves), then ℓ(r (j|t), t + N + j, y(t)) can be larger than κ (t). Hence, the constraint (6i) is less restrictive than This section contains the main result of the paper. The ℓ(r (j|t), t + N + j, y(t)) ≤ κ (t). The memory states κ in T j j proposed scheme is detailed in Section III-A. Performance combination with the self-tuning weight β and the constant guarantees relative to the limiting artiﬁcial reference trajectory c (Ass. 5) are crucial to establish the desired performance are derived in Section III-B. In Section III-C, improved a guarantees and are discussed in more detail in the follow- priori performance bounds are derived based on the self- ing theoretical analysis. For notational simplicity, we deﬁne tuning weight. The theoretical properties are summarized in r (T|t) := r (0|t). T T Theorem 1. Closed-loop operation: The closed-loop system with (6) is given by x(t + 1) =f(x(t), u (0|t), t), (7a) β(t + 1) =B(β(·), κ(·), x(·)), (7b) A. Proposed scheme κ (t + 1) =ℓ(r (j + 1|t), t + N + 1 + j, y(t + 1)), (7c) j = 0, . . . , T − 1. In the following, we detail the proposed scheme and discuss the relation to other existing methods. The main idea is to At each time t, we measure a state x(t) and an external directly minimize the predicted economic stage cost ℓ with parameter y(t). As in a standard MPC framework, we apply some terminal cost V and terminal set X around the artiﬁcial the ﬁrst part of the optimized input sequence (7a). The cost f f reference trajectory r , and use an updating scheme to ensure of the last optimal artiﬁcial reference trajectory r is saved that the optimized artiﬁcial reference trajectory r converges in the memory states κ (7c). The tuning weight β(t) can T j to the best possible periodic orbit r . be determined by some general (causal) update rule B [26], T 5 [27], or simply chosen by a user as a time-varying or constant guarantees with a terminal equality constraint. signal (c.f. [26, Update rule 1] and Sec. IV-D). The constant The tracking MPC scheme [8] can be viewed as a modiﬁed c > 0 and the terminal ingredients V ,X need to be κ f f version, which uses a tracking stage cost ℓ in the optimization designed ofﬂine, which will be detailed later. Hence, compared problem (6) and hence does not require the additional tuning to a standard MPC closed loop, we have T additional scalar variable β or memory state κ . memory states κ (7c) and one scalar tuning-variable β (7b). j The standard economic MPC formulations for periodic Initialization: We assume that the initial state x(0) is such orbits [17], [18] and steady-states [2], [6] are contained as that there exists a feasible trajectory to some feasible periodic a special case, if we ﬁx the artiﬁcial reference trajectory orbit r . The memory states κ (0) can be initialized arbitrarily T j r = r . T T large, such that the constraint (6i) is inactive at t = 0. Correspondingly, the optimization problem (6) is feasible at B. Relative performance guarantees t = 0. The tuning variable β can be initialized with any positive scalar, most naturally β(0) = 1. In Proposition 1, we show that the proposed formulation is Existing schemes = special cases: In the following, we recursively feasible. For constant parameters y, Proposition 2 discuss in detail how various existing methods for economic shows that the average closed-loop performance is no worse MPC are contained in this formulation as special cases. than the performance of the limiting artiﬁcial references. The proposed formulation can best be viewed as an exten- Terminal ingredients: The following assumption captures sion to [26], [27], [28], which considers an artiﬁcial refer- the (standard) sufﬁcient conditions for the terminal ingredients. ence setpoint (T = 1). In particular, if we assume a time- invariant problem setup and choose T = 1, we get the Assumption 1. There exists a terminal set X (r , t), a f T optimization problem and closed-loop operation in [26], [27], (bounded) terminal cost V (x, r , t, y) and terminal controller f T [28]. For c ≥ 0, the constraints (6h)–(6i) are equivalent to k (x, r , t), such that at any time t ∈ N, for any parameters κ f T ℓ(r (t)) ≤ κ(t) = ℓ(r (t − 1)) which is used in [26], [27], y ∈ Y, reference r ∈ Z (t) and any x ∈ X (r , t), the T T T f T [28] to ensure that the cost of the artiﬁcial reference r is following conditions hold improving. Although one can directly see that [26], [27], [28] + + x ∈X (r , t + 1), (8a) is a special case of the posed formulation, it is not obvious (x, u) ∈Z(t), (8b) from the onset that the extension of [26], [27], [28] to periodic + + problems should be given by the optimization problem (6). V (x , r , t + 1, y) − V (x, r , t, y) (8c) f f T A more intuitive extension might be to use the constraint ≤ − ℓ(x, u, t, y) + ℓ(r (0), t, y), J (r (·|t)) ≤ κ(t) = J (r (·|t − 1)) (as an alternative T T T with x = f(x, u, t), r = R r ∈ Z (t + 1), u = to (6h)–(6i)). The possibly suboptimal performance of such T T T k (x, r , t). an approach has, however, been illustrated in Section II-C. f T In Section IV-B we show that we can guarantee the same This assumption can always be satisﬁed by using a ter- propertiesIn Section IV-B we show that we can guarantee minal equality constraint X (r , t) = {x }, k = u , with f T r f r the same properties with this more intuitive extension, if (x , u ) = r (0). However, for the improved performance r r T we instead suitably reformulate the cost function. Another guarantees discussed in Section III-C we will require stronger possible formulation for periodic orbits would be the constraint conditions for the terminal ingredients (Ass. 3, Sec. IV-A), ℓ(r (j|t)) ≤ κ (t) without the additional term c in (6i). This T j κ compare [27]. modiﬁcation is sufﬁcient to avoid the pitfall in Section II-C, if Recursive feasibility: The following proposition shows that the artiﬁcial reference is initialized as an optimal periodic orbit feasibility of the proposed scheme is independent of the r . However, this more restrictive constraint can potentially exogenous parameters y. prevent the artiﬁcial reference trajectory r to converge to the optimal periodic orbit r , compare also Ass. 5 and the T Proposition 1. Let Assumption 1 hold and assume that (6) numerical example in Section V. is feasible at t = 0. Then the optimization problem (6) is If we consider a prediction horizon of N = 0 and a terminal recursively feasible for the resulting closed-loop system (7). equality constraint X (r , t) = {x }, then the proposed f T r Proof. This result is a straightforward extension of existing formula yields a modiﬁed version of the MPC scheme using results for MPC with artiﬁcial reference trajectories [22], [23], periodicity constraints [21], [9]. The only difference would [24], [25], [26], [27], [28], [29], [21], [9], [8]. Given the be the additional performance constraints on the periodic feasible reference r (·|t) ∈ Z (t + N) at time t, the shifted orbit (6h)–(6i), which may not be necessary in many cases, reference r (·|t+1) = R r (·|t) satisﬁes (6g). This reference T T compare Section IV-B. Crucially, if we choose a suitable satisﬁes the constraints (6i) with equality, since Δκ(t+1) = 0. terminal cost with a non-empty terminal set (Ass. 3), then we A corresponding candidate input sequence is given by can establish closed-loop performance guarantees (Thm. 1), which are in general not valid for MPC schemes using peri- u (k + 1|t) k ≤ N − 2 u(k|t + 1) = . odicity constraints [21], [9]. In particular, the terminal ingre- ∗ ∗ k (x (N|t), r (·|t), t + N) k = N − 1 dients (Ass. 3) relax the one-step controllability condition [21, Lemma 4] to an incremental stabilizability condition (Ass. 3). The resulting state and input sequences satisfy the con- In Lemma 4, we discuss how to retain these performance straints (6e) and the terminal constraint (6f) due to Ass. 1. 6 Self-tuning weight: Deﬁne the change in the weight β as with κ := κ . Using (13) recursively implies T 0 γ(t) = β(t+1)−β(t). The following assumption characterizes some of the properties the update scheme B (7b) should have, κ (t + k + 1) ≤κ (t) − c Δκ(t + j) (14) T−1 k κ such that performance guarantees hold despite online changing j=0 values of β, compare [26] for a more nuanced discussion for k = 0, . . . , T − 1. Using the deﬁnition of κ in (9), we can and an alternative condition on B resulting in slightly weaker bound the T -step sum as performance guarantees.   T−1 T−1 k X (14) X X Assumption 2. [26, Ass. 1] The sequence β(·) satisﬁes   κ (t + 1 + k) ≤ κ (t) − c Δκ(t + j) T−1 k κ lim sup γ(t) ≤ 0 and γ(t) ≤ c , β(t) ≥ 0 for all t ≥ 0 t→∞ k=0 k=0 j=0 with a constant c . T−1 T−1 X X ≤ κ (t) − c T Δκ(t + k) k κ Deﬁne the cost of the artiﬁcial reference as k=0 k=0 T−1 T−1 κ(t) = κ (t) = J (r (·|t − 1), t + N − 1, y(t)). (9) (9) j T =κ(t) − c T Δκ(t + k) = κ(t) + c T (κ(t) − κ(t + T )). κ κ j=0 k=0 Suppose that the parameters y(t) remain constant, then the Thus, the closed-loop transient cost over one period T satisﬁes conditions (6i),(7c) with c ≥ 0 ensure that Δκ(t) = κ(t + t+T−1 1) − κ(t) ≤ 0 and thus κ is non-increasing. Boundedness of W (t + T ) − W (t) + ℓ(x(k), u(k), k, y(k)) (15) ℓ implies boundedness of κ(t). Thus κ(t) converges to some k=t limit κ . T−1 Average performance: The following proposition estab- ≤κ(t) + c T (κ(t) − κ(t + T )) + γ(t + k)κ(t + 1 + k). lishes that the closed-loop performance is no worse than κ k=0 (the performance of the limiting artiﬁcial trajectories r ), as Abbreviate ℓ(t) = ℓ(x(t), u(t), t, y(t)) and deﬁne κ (t) = an extension to [26, Thm. 1]. κ(t) − κ . Then (15) evaluated over a time interval K · T starting at t = 0 can be rewritten as Proposition 2. Let Assumptions 1 and 2 hold and assume that (6) is feasible at t = 0 and y(t) is constant, then W (K · T ) − W (0) (16) the resulting closed-loop system (7) satisﬁes the following K−1 performance bound ≤Kκ + c T (κ(0) − κ(T K)) + κ (k · T ) ∞ κ ǫ TK−1 ℓ(x(t), u(t), t, y(t)) k=0 t=0 lim sup ≤ κ /T. (10) KT−1 T K K→∞ + [γ(t)κ + γ(t)κ (t + 1) − ℓ(t)]. ∞ ǫ Proof. Deﬁne the value function t=0 The remainder of the proof is analogous to [26, Thm. 1]. W (t) =W (x(t), y(t), β(t), κ (t)) (11) Boundedness of ℓ, V and β(t) ≥ 0 ensures that W (T K) N−1 ∗ ∗ is lower bounded and thus = ℓ(x (k|t), u (k|t), t + k, y(t)) k=0 W (T K) − W (0) 0 ≤ lim inf . ∗ ∗ + V (x (N|t), r (·|t), t + N, y(t)) K→∞ K + β(t)J (r (·|t), t + N, y(t)). Taking averages on both sides of (16) yields W (T K) − W (0) Proposition 1 provides a feasible candidate solution u(·|t+1), 0 ≤ lim inf K→∞ K r (·|t+1) to the optimization problem (6) at time t+1. Hence, K−1 we can use the cost of the candidate solution to upper bound X c T ≤κ + lim sup κ (k · T ) + lim (κ(0) − κ(T K)) ∞ ǫ the value function W (t + 1), which in combination with y(t) K K→∞ K K→∞ k=0 constant and the terminal cost (Ass. 1) yields " # KT−1 W (t + 1) − W (t) + ℓ(x(t), u(t), t, y(t)) (12) + lim sup γ(t)κ + γ(t)κ (t + 1) ∞ ǫ K→∞ t=0 ≤ℓ(r (0|t), t + N, y(t)) + γ(t)κ(t + 1) KT−1 (7c) = κ (t + 1) + γ(t)κ(t + 1), T−1 − lim sup ℓ(t) K→∞ t=0 compare [26, Thm. 1][27, Thm. 1] for details. The deﬁni- TK−1 tion of κ in (7c), constant parameters y(t) and the con- ≤κ − lim sup ℓ(t), straints (6h),(6i) ensure K→∞ t=0 (7c) and thus (10). The ﬁrst inequality follows from (16), by using κ (t + 1) = ℓ(r (j + 1|t), t + N + j + 1, y(t)) (6i) lim inf a − b ≤ lim inf −b + lim sup a = lim sup a − lim sup b . n n n n n n ≤κ (t) − c Δκ(t), j = 0, . . . , T − 1, (13) j+1 κ n n n n n 7 The second inequality follows from γ(t) ≤ c , κ (t) ∈ [0,∞), where the last step follows using the case distinction c ≤ α/2 γ ǫ and c ≥ α and the fact that α(r ) ≥ α. Given kr −r˜ k ≤ ǫ, T T lim κ (t) = 0, lim sup γ(t) ≤ 0, lim Δκ(t) = 0. we have t→∞ t→∞ t→∞ (17c) + + + V (x , r˜ , t + 1) ≤ V (x , r , t + 1) + α (ǫ) δ T δ 4 (19) ≤ α(r ) − Δα + α (ǫ) T 4 (18) C. Improved a priori performance bounds ≤ α(r˜ ) + α (ǫ) + α (ǫ) − Δα = α(r˜ ), T 5 4 T −1 In the following, we provide sufﬁcient conditions to ensure with ǫ := (α + α ) (Δα). 4 5 that the average cost of the artiﬁcial periodic orbit converges This lemma is an extension to [27, Lemma 1] and shows that to a local minimum. the reference r can be incrementally changed in closed-loop Terminal ingredients: The following assumption is a operation without losing recursive feasibility. Similar results stronger version of Assumption 1, which is used to derive are used in nonlinear tracking MPC schemes [24], [25]. the improved performance guarantees. Self-tuning weight: Given a state x at time t, the set of periodic reference trajectories r with a terminal set X that Assumption 3. Consider the terminal set X , terminal cost T f can be reached within the prediction horizon N is deﬁned as V and controller k from Assumption 1. There exists an f f incremental Lyapunov function V (x, r , t), such that for δ T mN R (x, t) = {r ∈ Z (t + N)| ∃u ∈ R s.t. x(t) = x, N T T any time t ∈ N, any reference r , r˜ ∈ Z (t) and any T T T x(k + 1) = f(x(k), u(k), k), (x(k), u(k)) ∈ Z(k), x ∈ X (r , t), the following inequalities hold f T k = t, . . . , t + N − 1, x(N + t) ∈ X (r , t + N)}. f T + + V (x , r , t + 1) − V (x, r , t) ≤ − α (kx − x k), (17a) δ δ T 1 r Similarly, we deﬁne the set of reference trajectories that α (kx − x k) ≤ V (x, r , t) ≤α (kx − x k), (17b) 2 r δ T 3 r additionally satisfy the constraints (6h)–(6i) |V (x, r , t) − V (x, r˜ , t)| ≤α (kr − r˜ k), (17c) δ T δ T 4 T T R (x, t, y, κ ) = {r ∈ R (x, t)| s.t. r satisﬁes (6h)–(6i)}. N j T N T + + with x = f(x, u, t), u = k (x, r , t), r = R r ∈ Z (t+ f T T T T 1), (x , u ) = r (0) and functions α , α , α , α ∈ K . Given a point x ∈ R at time t with parameters y and κ , the r r T 1 2 3 4 ∞ j Furthermore, the terminal set is given by X (r , t) = {x ∈ cost of the best reachable periodic orbit is given as f T R | V (x, r , t) ≤ α(r )} and the terminal set size α(r ) δ T T T J (x, t, y, κ ) = min J (r , t + N, y). (20) T,min j T T satisﬁes r ∈R (x,t,y,κ ) T N j |α(r ) − α(r˜ )| ≤ α (kr − r˜ k), α(r ) = α(r ) ∈ [α, α] Assumption 4. The update rule B is such that for any y(t) = T T 5 T T T (18) y for all t ≥ 0 and for all sequences x(·), κ(·), it holds that with constants α, α > 0 and a function α ∈ K . 5 ∞ κ − lim inf J (x(t), t, y(t), κ (t)) > 0 ∞ T,min j t→∞ The ofﬂine design of such terminal ingredients is discussed ⇒ lim inf β(t) = ∞. t→∞ in detail in Section IV-A. The conditions (17a)–(17c) ensure The main idea is that in closed-loop operation the self- that the terminal set has a non-empty interior and that the tuning weight β increases if necessary and thus ensures that the terminal controller k stabilizes the reference r with a f T artiﬁcial trajectory converges to the optimal mode of operation, continuous incremental Lyapunov function V . compare [26], [27]. A detailed discussion on update schemes Lemma 1. Let Assumptions 1 and 3 hold. There exists a B satisfying Assumptions 2 and 4 is given in [26]. constant ǫ > 0, such that at each time t ∈ N, for any Periodic economic continuity: As discussed in Section II-C r ∈ Z (t) and any x ∈ X (r , t), it holds that T T f T and Section III-A, the constraints (6h)–(6i) are crucial for the desired properties. However, the constraint (6i) limits how x = f(x, k (x, r , t), t) ∈ X (r˜ , t + 1), f T f T the shape of the artiﬁcial reference trajectory may change. for all r˜ ∈ Z (t + 1) ∩ B (R r ). In particular, for c = 0 this constraint ensures that the T T ǫ T T κ reference can only be updated if the economic cost on all Proof. First, note that Assumption 3 ensures that the positive points of the reference trajectory does not increase. For c invariance condition (8a) is strictly satisﬁed arbitrarily large, the constraint (6i) becomes inactive, if the (17a) overall cost of the artiﬁcial trajectory decreases (Δκ < 0). + + V (x , r , t + 1) ≤ V (x, r , t) − α (kx − x k) δ δ T 1 r However, both for numerical and technical reasons we consider (17b) −1 the smooth constraint (6i) with a ﬁnite value c . Thus, we ≤ V (x, r , t) − α (α (V (x, r , t))) δ T 1 δ T require the following technical continuity assumption on the −1 ≤ sup c − α (α (c)) periodic economic optimization problem (2). c∈[0,α(rT )] (18) Assumption 5. There exists a positive constant c , such that −1 κ ≤ α(r ) − min{α (α (α/2)), α/2}, (19) T 1 at any time step t ∈ N, for any parameters y ∈ Y, for {z } =:Δα>0 any periodic trajectory r ∈ Z (t), which is not a local T T 8 minimum of (2) and any ǫ > 0, there exists a change Δr Assume further that y(t) is constant and the update rule B with kΔr k ≤ ǫ, r + Δr ∈ Z (t), J (r + Δr , t, y) < satisﬁes Assumptions 2 and 4, then κ is a local minimum T T T T T T T ∞ J (r , t, y), such that the following bound holds of (2) and the following performance bound holds T T TK−1 ℓ(r (j) + Δr (j), t + j, y) − ℓ(r (j), t + j, y) T T T ℓ(x(t), u(t), t, y(t)) t=0 c ≥ max . κ lim sup ≤ κ . j J (r , t, y) − J (r + Δr , t, y) T T T T T K→∞ (21) Proof. The results follow from Propositions 1–3. This assumption ensures that it is possible to incrementally Corollary 1. Let Assumptions 1, 3 and 5 hold. Assume change the overall cost J , with incremental changes in the that (6) is feasible at t = 0 and y(t) is constant. If the update reference r and the local cost ℓ. If we expand the fraction rule B is chosen as update scheme 2 or 6 in [26], then the by Δr and take the limit Δr → 0, we can see that this T T closed-loop average economic performance is no worse than condition is similar to a continuity assumption on the fraction the performance at a locally optimal periodic orbit (2). of the gradients of ℓ and J . Additional details regarding this condition are discussed in Sections IV-B and IV-C. The Proof. This results follows directly from Theorem 1. It sufﬁces following lemma shows that this continuity condition (Ass. 5) to note, that the update schemes 2 and 6 satisfy Assumptions 2 in combination with the incremental stabilizability property and 4, compare [26, Lemmas 1 and 4]. (Ass. 3) allows for the convergence to local minima. Remark 2. For simplicity, we have presented the proposed Lemma 2. Let Assumptions 1, 3 and 5 hold and assume that framework in a discrete-time setting. However, the approach y(t) is constant. Suppose that the optimization problem (6) is can be directly applied to continuous-time problems by deﬁn- feasible at time t with some reference trajectory r (·|t), which ing the discrete-time stage cost ℓ and dynamics f implicitly is not a local minimum to (2). Then there exists a reference as the integration of some continuous-time dynamics f and r˜ which is a feasible candidate solution to (6) at t+1, which T the average continuous-time cost ℓ over some sampling satisﬁes period h. One advantage of considering a continuous-time formulation is that the design of terminal ingredients satisfying J (r˜ , t + N + 1, y(t)) < J (R r (·|t), t + N + 1, y(t)). T T T T Assumption 3 (compare Section IV-A) simpliﬁes, compare (22) e.g. [37, App. C]. Furthermore, in a continuous-time setting it Proof. Given that R r (·|t) ∈ Z (t + N + 1) is not a local T T is possible to use a variable sampling time h ∈ [h , h ], min max minimum, Assumption 5 ensures that there exists a reference by considering the decision variable u = (u , h), where u c c r˜ = R r (·|t) + Δr ∈ Z (t + N + 1), that improves T T T T denotes the (typically piece-wise constant) control input. As the reference cost J (22) and satisﬁes kΔr k ≤ ǫ and T T a result, in a time-invariant setting the ﬁxed constant T does (21). Satisfaction of the posed constraints (6h)–(6i) follows not directly impose a time length on the set of periodic orbits from (21), by noting that Δκ(t) = J (r˜ , t + N + 1, y(t)) − T T Z , but only a ﬁnite parametrization. The constants h , T min J (R r (·|t), t+N +1, y(t)). With ǫ according to Lemma 1, T T h need to be chosen, such that the (typically explicit) T max the candidate input u(·|t + 1) from Proposition 2 satisﬁes the discretization scheme is stable and the MPC can react fast terminal set constraint (6f) with the incrementally changed enough. The advantages of such a formulation will also be reference r˜ and is thus a feasible solution to (6). explored in a numerical example in Section V-B. We point out that the beneﬁts of using such a variable continuous-time A priori performance bounds: The following proposition period length have also been recently investigated in [31] establishes a priori performance bounds on the artiﬁcial refer- using a direct multiple shooting method. ence trajectory as an extension to [27, Thm 2/3, Corollary 1]. Design parameters: Overall, the proposed framework pro- Proposition 3. Let Assumptions 1, 3 and 5 hold and assume vides desired performance guarantees, if the constant c , the that y(t) is constant. Assume that the optimization problem (6) self-tuning weight β(t) and the terminal ingredients V ,X f f is feasible at t = 0. If the update rule B satisﬁes Assumption 4, are chosen properly (Ass. 1–5). In numerical experiments, we then κ is a local minimum of (2). found that the closed loop is insensitive to changes in the Proof. Using a proof of contradiction one can show κ = constant c , even by orders of magnitude, as long as c is ∞ κ κ lim J (x(t), t, y(t), κ (t)), compare [27, Thm. 2], sufﬁciently large (e.g. c = 100 in App. A). In Section IV-B t→∞ T,min j κ [26, Thm. 2]. Furthermore, suppose that there exists a limiting we also show how the problem can be reformulated to get rid reference r , which is not a local minimizer of (2). Lemma 2 of the constant c and the continuity condition in Assump- T κ ensures that there exists a feasible reference r˜ , with an im- tion 5. A large self-tuning weight β(t) can deteriorate the proved cost, which implies J < κ and thus contradicts transient performance, but is useful to ensure convergence of T,min ∞ the assumption. the artiﬁcial reference to a local minimum. In Section IV-D, we show that similar performance bounds hold when choosing The following theorem summarizes the theoretical proper- a constant weight β. For the special case of T = 1 (artiﬁcial ties of the proposed MPC scheme. setpoint), more details on the effect of β on the closed loop can Theorem 1. Let Assumptions 1, 3 and 5 hold and assume be found in [26], [27] and [28]. Different design procedures that (6) is feasible at t = 0. Then the optimization problem (6) for the terminal ingredients will be discussed in detail in is recursively feasible for the resulting closed-loop system (7). Section IV-A. 9 IV. PROPOSED FRAMEWORK - DETAILS AND VARIATIONS for any t ∈ N, r ∈ Z (t), y ∈ Y and any x ∈ R with T T V (x, r , t) ≤ α : δ T 1 In the following, we discuss details and variations of the S(r, t) ℓ (r, t, y), (23) ξξ proposed framework. In Section IV-A we discuss in detail ℓ (x, r , t, y) ≥ℓ(x, r , t, y) + kx − x k , (24) q T T r how to design terminal ingredients that satisfy Assumption 3. In Section IV-B we discuss how to modify the cost function, with such that the continuity condition in Assumption 5 can be dropped. In Section IV-C we consider the special case of ℓ (x, r , t, y) :=ℓ (x , r , t, y) · (x − x ) + kx − x k , q T x r T r r Q (r ,t) convex periodic optimal control problems. In Section IV-D, we (25) discuss the theoretical properties without self-tuning weights β(t), similar to [28]. I I n n Q (r , t) := S(r, t) k (x , r , t) k (x , r , t) f,x r T f,x r T + 2ǫI + ℓ k (x , r , t), (26) n uj f,j,xx r T A. Terminal cost for economic dynamic operation j=1 In the following, we detail how to design terminal ingre- where (x , u ) = r = r (0), ℓ denotes the Hessian of ℓ w.r.t. r r T ξξ dients that satisfy Assumption 3. First, in Section IV-A1 we ξ = (x, u), k the Jacobian of k w.r.t. x, k the Hessian f,x f f,j,xx show how a suitable tailored economic terminal cost V can of the j-th component of k w.r.t. x and ℓ the Jacobian of f u be designed using local linear and quadratic approximation ℓ w.r.t. the j-th component of u. of the dynamics f and the economic cost ℓ, as an extension Proof. We point out that the derivative of ℓ w.r.t. x is the total and combination of the methods in [6], [27], [37]. Then, in derivative of ℓ w.r.t. x, for u = k . Hence, the Jacobian and Section IV-A2 we show how a simple (and hence conservative) Hessian of ℓ are given by positive deﬁnite terminal cost V can be computed based on any existing incremental Lyapunov function V , similar to ℓ =ℓ , x ξ the design in [38], [18]. Finally, in Section IV-A3, we show f,x that the theoretical properties can also be guaranteed with m ⊤ ⊤ a simple terminal equality constraint (TEC), if a multi-step I k I k ℓ = ℓ + ℓ k , xx n ξξ n u f,j,xx f,x f,x j implementation is considered. j=1 1) Reference generic ofﬂine computations: In the follow- 1×(n+m) (n+m)×(n+m) where ℓ ∈ R , ℓ ∈ R denote the ξ ξξ ing, we detail a procedure to compute a suitable terminal cost Jacobian and Hessian of ℓ w.r.t. ξ = (x, u). Twice continuous V (Ass. 3) based on the linearization of the dynamics and a differentiability of ℓ and compactness imply that there exists quadratic approximations of the stage cost ℓ (using the Hessian a ﬁnite constant and gradient). The following derivation is an extension of the c = sup λ (ℓ (r, t, y)). approach in [6] to dynamic/periodic trajectories. Furthermore, max ξξ t∈N,r∈Zr(t),y∈Y we extend the approach to online optimized/changing refer- ence trajectories by extending the reference generic ofﬂine Thus the matrix S = (max{c, 0})I is positive semi- n+m computation from [37] to an economic stage cost ℓ. In ad- deﬁnite and satisﬁes S ℓ . The construction in (26), the ξξ dition, this online computation involves an online computed deﬁnition of the Hessian ℓ and S ℓ directly imply xx ξξ adjoint for periodic trajectories, similar to the local gradient Q (r , t) ℓ (x , r , t) + 2ǫI . Similar to [6, Lemma 22], T xx r T n correction employed in [39]. there exists a small enough constant α > 0 (uniform in r , t, y), such that Q (r , t) ℓ (x, r , t, y) + ǫI , ∀t ∈ N, T T xx T n Linear-quadratic local auxiliary stage cost: The following y ∈ Y, r ∈ Z (t), V (x, r , t) ≤ α . Abbreviate Δx = T T δ T 1 Lemma extends the results in [6, Lemma 22-23] to compute x − x , which implies ℓ = kΔxk + ℓ Δx. Convexity of r q x an auxiliary stage cost ℓ , that locally upper bounds the stage Q the sublevel sets of V implies that V (x + sΔx, r , t) ≤ α δ δ r T 1 cost ℓ, which will later be used to derive sufﬁcient conditions for all s ∈ [0, 1] and any V (x + Δx, r , t) ≤ α . Hence, δ r T 1 for inequality (8c). we can use the mean value theorem for vector functions [1, Lemma 3. Suppose there exists some V , k satisfying δ f Prop. A.11 (b)], similar to [6, Lemma 23], to obtain the conditions in Assumption 3. Suppose further that the ℓ (x, r , t, y) − ℓ(x, r , t, y) (27) q T T sublevel sets of V are convex in x, the controller k is δ f twice continuously differentiable in x, continuous in r , and ⊤ ∗ = (1 − s)Δx (Q (r , t) − ℓ (x + s · Δx, r , t, y))Δxds T xx r T satisﬁes k (x , r , t) = u . Suppose that the stage cost ℓ f r T r and the dynamics f are locally Lipschitz continuous and 2 2 ≥ (1 − s)ǫkΔxk ds = ǫ/2kΔxk . twice continuously differentiable w.r.t (x, u). Then the function ℓ(x, r , t, y) = ℓ(x, k (x, r , t), t, y) − ℓ(r, t, y) is twice con- T f T tinuously differentiable with respect to x. For any ǫ > 0, there exists a constant α > 0 and a positive semi-deﬁnite matrix n+m×n+m S(r, t) ∈ R , such that the following conditions hold Basically, ℓ is a local linear-quadratic over approximation q 10 of the stage cost ℓ. Hence, we will formulate a sufﬁcient approximation at x = x yields condition for (8c) using the auxiliary stage cost ℓ . We point Δx =f(x, k (x, r , t), t) − f(x , u , t) f T r r out that Lemma 3 does not impose any deﬁniteness conditions =A (r , t)Δx + Φ (Δx), cl T r ,t on the Hessian of the stage cost ℓ, but instead upper bounds T the Hessian using the positive semi-deﬁnite matrix S. with the remainder term Φ . Twice continuous differentia- r ,t bility of f and compact constraints imply that the remainder Sufﬁcient conditions based on the linearization: We denote term is uniformly Lipschitz continuous in the terminal set, the Jacobian of f evaluated around an arbitrary point r ∈ i.e., kΦ (Δx)k ≤ L kΔk for all t ≥ 0, with a constant Z (t) at some time t ∈ N by r r ,t Φ,α T 1 L arbitrary small for α arbitrary small. Using this bound Φ,α 1 ∂f ∂f in combination with condition (30) implies that there exists A(r, t) := , B(r, t) := . (28) ∂x ∂u (x,u)=r (x,u)=r a sufﬁciently small constant α > 0, such that the nonlinear system (locally) satisﬁes Given some periodic trajectory r (·|t) ∈ Z (t), we denote T T the Jacobian w.r.t. x of the system f in closed loop with the + 2 2 2 kΔx k − kΔxk ≤ −kΔxk ∗ (33) P (R r ,t+1) P (r ,t) Q (r ,t) T T T T terminal control law k by for all x ∈ X (r , t), compare [37, Lemma 1] for details. f T A (r (·|t), t) := A(r (0|t), t) + B(r (0|t), t)k (r , t). cl T T T f,x T Given that T is ﬁnite, ℓ uniformly bounded and condition (32) holds, the vector p admits a uniform bound. Using the deﬁni- In the following, we introduce a corresponding adjoint periodic tion of p, we get trajectory p(k|t), which can be computed online based on the following set of n · T linear (in p) equality constraints ⊤ + p (1)Δx (34) ⊤ j A (R r (·|t), t + j)p(j + 1|t) (29) ≤p (1)A (r , t)Δx + kp(1)kkΦ (Δx)k cl T cl T r ,t (29) =p(j|t) − ℓ (x (j|t),R r (·|t), t + j, y), j = 0, . . . , T − 1, r T = p (0)Δx − ℓ (x , r , t, y)Δx + kp(1)kkΦ (Δx)k x T T x r T r ,t ⊤ 2 ≤p (0)Δx − ℓ (x , r , t, y)Δx + ǫ/2kΔxk , with p(N|t) = p(0|t). In the setpoint case (T = 1), this x r T reduces to p (A − I) = −ℓ , similar to [6], [27]. Similar cl where the last inequality holds for a sufﬁciently small constant to the adjoints used in [39], this vector p corrects the effect α > 0, given the uniform bound on kpk and the properties of of ℓ , the gradient of the stage cost. the remainder term Φ . By combining (33) with (34) and r ,t The following proposition shows that such an online com- using the auxiliary stage cost from Lemma 3, the terminal puted adjoint vector p in combination with an ofﬂine computed cost (31) satisﬁes matrix valued function P provides a suitable terminal cost for V (x ,R r , t + 1, y) − V (x, r , t, y) f T T f T dynamic operation with economic cost. 2 2 ≤ − kΔxk − ℓ (x , r , t, y)Δx + kΔxk ∗ x r T Q (r ,t) Proposition 4. Suppose the conditions in Lemma 3 hold. (24) Assume further that there exists a positive deﬁnite T -periodic ≤ − ℓ(x, r , t, y), matrix P (r , t), continuous in r, such that for any t ∈ N, and hence condition (8c). r ∈ Z (t), the following matrix inequality is satisﬁed T T A (r , t)P (R r, t + 1)A (r , t) − P (r , t) (30) T T cl T T cl Corollary 2. Suppose that the stage cost ℓ and the dynamics − Q (r , t) − ǫ˜I , f are locally Lipschitz continuous and twice continuously T n differentiable w.r.t (x, u). Assume further that there exists a with ˜ǫ > 0. Then for any periodic reference r ∈ Z (t), T T positive deﬁnite matrix P (r , t) and a matrix k (r , t), both T f,x T t ∈ N, the conditions (29) have a unique solution p(·|t). In continuous in r and T-periodic in t, such that for any t ∈ N, addition, there exists a constant α , such that the terminal r ∈ Z (t), the matrix inequality (30) is satisﬁed with some T T cost ǫ˜ > 0. Then there exists a function α(r ), such that the termi- 2 ⊤ nal controller k (x, r , t) = u + k (x − x ), the terminal V (x, r , t, y) := kx − xk + p (0|t)(x − x ), (31) f T r f,x r f T r r P (r ,t) n 2 set X (r , t) = {x ∈ R | kx − x k ≤ α(r )} and the f T r T P (r ,t) with p according to (29) and x = x (0), satisﬁes condi- r r terminal cost V according to (31) satisfy Assumptions 1 and tion (8c) with X (r , t) := {x ∈ R | V (x, r , t) ≤ α }. f T δ T 1 Proof. Part I. Condition (30) ensures that the linearized Proof. Given that P and Q + ǫ˜I are positive deﬁnite, the (time-varying) dynamics along the periodic trajectory r are T conditions (17a)–(17b) in Assumption 3 are satisﬁed with the (uniformly) exponentially stable, which implies incremental Lyapunov function V (x, r , t) = kx−x k δ T r P (r ,t) j and quadratic functions α , α , α ∈ K . Convexity of T−1 ⊤ 1 2 3 ∞ det(I − Π A (R r , t + j)) > 0. (32) n T cl j=0 T the terminal set X w.r.t. x (compare conditions Lemma 3) Thus, the constraints (29) have a unique solution p for any follows from V quadratic in x. Condition (17c) follows from r ∈ Z (t), compare also the reformulation of (29) in V quadratic and the assumed continuity of P w.r.t. r . T T δ T Remark 4. Conditions (8a), (8c), (17) hold for any α ≤ α , using Prop. 4. Part II. Denote Δx = x − x . The ﬁrst order Taylor Furthermore, given that Z (t) ∈ int(Z(t)) and k bounded, r r f,x 11 Algorithm 1 Ofﬂine computation there exists a small enough constant α > 0, such that 1: Compute Jacobian, Hessian A, B, ℓ , ℓ . conditions (8b), (18) hold for any α ≤ α . Hence, choosing 2 ξ ξξ 2: Determine matrix S 0, such that S ℓ (23). the constant terminal set size α(r ) = min{α , α } satisﬁes T 1 2 ξξ 3: Compute matrix P (and possibly k ) such that (30) holds all the conditions (condition (18) is trivially satisﬁed). f using LMIs, compare [37, App. D]. 4: Compute maximal terminal set size α using [37, Alg. 1]. 5: Derive α(r) ∈ (0, α ] for constraint satisfaction: a) Compute constant α > 0 using [37, Equation (24)] With this result, we can directly specify a procedure to com- and set α = min{α , α }. 1 2 pute suitable terminal ingredients. First, a symbolic expression b) Compute α(r ) online with a scalar optimization for the Jacobian A, B, ℓ and the Hessian ℓ are computed. variable, compare [25, Sec. 3.3]. ξ ξξ Then a positive semi-deﬁnite matrix S is computed, which satisﬁes (23). This can either be achieved with a constant The proposed procedure is a combination and extension of matrix S (c.f. proof Lemma 3) or by computing a suitably the reference generic ofﬂine computations in [37], the termi- parametrized matrix S using linear matrix inequalities (LMIs). nal ingredients for economic MPC [6], [27] and the online computation of p using (29). Regarding the online operation, Given S, we have to compute a parametrized matrix P , we simply include the constraints (29) to compute p(·|t) and such that condition (30) holds. Suppose we want to compute possibly constraints to compute α(r ) online (compare [25, a feedback of the form k = u + k Δx with some f r f,x Sec. 3.3.] for details) in the MPC optimization problem (6). parametrized feedback gain k (k = 0). In this case, f,x f,xx This procedure signiﬁcantly simpliﬁes in the special case of condition (30) with Q according to (26) is equivalent to [37, linear systems with linear/quadratic stage costs ℓ, which is Inequality (36)] with the following (output) tracking stage cost discussed in Section IV-C. Furthermore, in the special case of 2 2 ℓ = k(C + Dk )Δxk + (2ǫ + ǫ˜)kΔxk (35) f,x S artiﬁcial setpoints (T = 1), we recover the schemes in [26], I 0 [27], [28] and Algorithm 1 provides a corresponding procedure n n (n+m)×n (n+m)×m C = ∈ R , D = ∈ R . to derive suitable terminal ingredients. 0 I m m Hence, we can use the result in [37, Lemma 6, Prop. 6] to Remark 3. The matrices S, Q and P can also be compute suitable matrices k and P . In particular, in [37] parametrized by y, to yield a terminal cost V that depends f,x f the matrices P, k , A, B, S are parametrized based on a on the price signal y. However, the incremental Lyapunov f,x quasi-LPV (linear-parameter-varying system) approach and the function V used for the terminal set X (Ass. 3) may not δ f conditions are transformed into LMIs, that can be efﬁciently depend on y to ensure recursive feasibility independent of computed ofﬂine. online changes in the price signal y. Thus, the choice of X in Corollary 2 is only valid for P independent of y. Given that the vector p needs to satisfy condition (29) with equality, a similar parametrized ofﬂine computation for p Remark 4. As already discussed, the vector p needs to be seems intractable (with the exception of linear systems, com- computed online using (29), which adds nT optimization pare Section IV-C). Hence, we simply add the constraint (29) variables and n·T equality constraints (linear in p) to the op- to the MPC optimization problem (6) and compute p(·|t) timization problem (6). Abbreviate A(j|t) = A(r (j|t), t+j), online. B(j|t) = B(r (j|t), t + j), ℓ (j|t) = ℓ (r (j|t), t + j, y(t)) T ξ ξ T and suppose the feedback k is parametrized in the form f,x Finally, regarding the terminal set size α, we ﬁrst compute −1 k (j|t) = Y (j|t)X(j|t) with matrices X, Y (as is the f,x the constant α > 0, such that (8c) holds for all V ≤ α , 1 δ 1 case with the LMIs considered in [37]). Then, multiplying e.g. using Algorithm 1 from [37]. There are two options to the constraints (29) by X from the left yields the following compute a terminal set size α that also ensures constraint equivalent constraint satisfaction (8b). The deﬁnition of Z ⊆ int(Z) can be used to compute a constant α ∈ (0, α ], similar to the optimization (A(j|t)X(j|t) + B(j|t)Y (j|t)) p(j + 1|t) (36) problem (24) in [37]. However, such a constant α depends on ⊤ ⊤ =X(j|t)p(j|t) − X(j|t) Y (j|t) ℓ (j|t), the choice of Z and thus can yield arbitrary small values α (and thus slow convergence of r , compare Lemma 1), or re- T where we use k X = Y and the formula for ℓ from the f,x x quires restrictive constraints on the set of periodic trajectories proof of Lemma 3. The resulting constraint can be imple- Z . This problem can be alleviated by computing a reference T mented directly in terms of X, Y . The constraints (29) can trajectory dependent terminal set size α(r ) ∈ [α, α] online, T also be compressed into one n dimensional equality constraint which can be done by using an additional scalar optimization with only the n dimensional optimization variable p(0|t). In variable α in (6), compare [25, Sec. 3.3] for details. particular, denote ℓ (j|t) = ℓ (x (j|t), r (·|t), t + j, y(t)) x x r T k−1 j and A (k|t) = Π A (R r (·|t), t + j). Then p(0|t) cl cl T j=0 T The overall design procedure is summarized in Algorithm 1. satisfying (29) can be equivalently computed using T−1 ⊤ ⊤ ⊤ (I − A (T|t))p(0|t) = A (j|t)ℓ (j|t). (37) cl cl x 2 j=0 Due to the prediction horizon N the time index t changes to t+N in (29). 12 Furthermore, if the period length T is very large, an approx- (8b) and Assumption 3. Condition (8c) follows with imate solution can be obtained by assuming A (j|t) ≈ 0 cl + + V (x , r , t + 1) − V (x, r , t) f f T for j ≥ T with some T < T , which results in p(0|t) ≈ c c ⊤ ⊤ T −1 (40) k k + + k A (j|t)ℓ (j|t). The fact that we need to take the full cl x j=0 = (V (x , r , t + 1) − V (x, r , t)) δ δ k T c (1 − ρ ) trajectory r into account to compute the correct gradient k=1 correction p indicates that the computation of a terminal cost ν (38a)X k k for nonperiodic time-varying trajectories may be non-trivial. ≤ (ρ − 1)V (x, r , t) δ T k k c (1 − ρ ) k=1 ν ν X X (38b) V (x, r , t) 2) Given incremental Lyapunov function V : In the follow- δ δ T = − a ≤ − a kx − x k k k r ing, we consider the case, where some incremental Lyapunov k=1 l=1 function V with a corresponding feedback k is available δ f (39) and focus on computing a simple (positive deﬁnite) terminal ≤ ℓ(r, t, y) − ℓ(x, k (x, r , t), t, y). f T cost V . For simplicity, we consider exponential stability, as a special case of Ass. 3. This terminal cost is easy to compute, but can also be Assumption 6. There exists an incremental Lyapunov function quite conservative. In particular, an interesting feature of this V (x, t, r ), a controller k (x, r , t), a terminal set size δ T f T approach is that the terminal cost V is a sum of incremental α(r ) and constants c , c > 0, ρ ∈ (0, 1), such that at T l u Lyapunov functions and as such also an incremental Lyapunov any time t ∈ N, for any reference r ∈ Z (t) and any T T function, which is positive deﬁnite w.r.t. to the reference x ∈ X (r , t) := {x ∈ R | V (x, r , t) ≤ α(r )}, the f T δ T T trajectory r . Correspondingly, the terminal cost incentives following conditions hold regulation towards the reference trajectory r . In general the + + reference r will have a suboptimal performance, which is V (x , r , t + 1) ≤ρV (x, r , t), (38a) δ δ T why a purely economic (not positive deﬁnite) terminal cost as c kx − x k ≤ V (x, r , t) ≤c kx − x k, (38b) l r δ T u r in Proposition 4 may be advantageous. (x, u) ∈Z(t), (38c) Remark 5. Note, that if a matrix P is computed ofﬂine + + with x = f(x, u, t), r = R r ∈ Z (t + 1), u = T T T that satisﬁes condition (30), the results in Corollary 2 and k (x, r , t). In addition, Inequalities (17c) and (18) from f T Prop. 5 can be combined to show that the terminal cost V = Assumption 3 hold with some α , α ∈ K . 4 5 ∞ kΔxk +ckΔxk locally satisﬁes condition (8c), by choosing c > 0 suitably. In particular, consider V = kΔxk satisfying δ P Assumption 6 and the following local linear quadratic bound Such an incremental Lyapunov function can for example be 2 ∗ ℓ ≤ a kΔxk +kΔxk ∗, with Q according to Lemma 3 and 1 P computed using quasi linear parameter varying methods [37], Q a suitable constant a > 0 using continuous differentiability control contraction metrics [40], back stepping [41] or feed- of ℓ. Then choosing c := a /(1 − ρ) ensures satisfaction of back linearization. In addition, we assume that the stage cost (8c) using can be locally bounded by some polynomial, similar to [38, Ass. 5], [18, Ass. 26]. (33),(38a) + 2 V − V ≤ −kΔxk ∗ − c(1 − ρ)kΔxk ≤ −ℓ. f P Assumption 7. Consider the incremental Lyapunov function 3) Terminal equality constraints: In the following, we V , the feedback k and the terminal set X from Assump- δ f f discuss how to replace the general terminal set (Assumption 3) tion 6. There exists constants a ≥ 0, k = 1, . . . , ν, such that k with a simple terminal equality constraint (TEC, X (r , t) = f T the stage cost ℓ satisﬁes x ). In principle, the conditions (17a)–(17c) are quite general ν and not restrictive, but the explicit knowledge of V (which ℓ(x, k (x, r , t), t, y) − ℓ(r, t, y) ≤ a kx − x k , (39) characterizes the terminal set X ) can pose challenges. f T k r k=1 The following analysis is similar to [28] and [24], which also considered TEC in the steady-state case. To this end, we for all t ∈ N, y ∈ Y, r ∈ Z (t), x ∈ X (r , t). T T f T consider the following ﬁnite-time local incremental controlla- bility condition. The following proposition follows the arguments from [38, Prop. 2], compare also [18, Prop. 27]. Assumption 8. There exist constants ν ∈ N, ǫ > 0, such that at any time t ∈ N, for any references r , r˜ ∈ Z (t) T T T with kr − r˜ k ≤ ǫ, there exists a state and input sequence T T Proposition 5. Let Assumptions 6–7 hold. Then the following ν×(n+m) (x, u) ∈ R , such that terminal cost V satisﬁes Assumptions 1 and 3 x(0) =x (0), x(k + 1) = f(x(k), u(k), t + k), V (x, r , t) := V (x, r , t). (40) f T T k x(ν) =x ˜ (ν), (x(k), u(k)) ∈ Z(t + k), k = 0, . . . , ν − 1. c (1 − ρ ) T k=1 This condition is for example satisﬁed with ν ≤ n if the lin- Proof. The terminal set X directly satisﬁes conditions (8a), earization along any feasible periodic trajectory is (uniformly) f 13 controllable [1, Ass. 2.37], [8, Ass. 2], compare also [28, constraints. However, instead of a ν-step MPC implementa- Ass. 7] [19, Ass. 10]. Typically, an additional continuity tion (41), in [28, Algorithm 3] it was suggested to augment bound on ℓ is used (compare for example [24, Ass. 4]), the MPC with an algorithm that decides at each time t if which is, however, not necessary in the considered setup with the candidate solution or the standard MPC feedback (7a) is the bounded stage cost ℓ and the self-tuning weight β. The applied. In particular, if the cost of the artiﬁcial reference following result is an adaptation of Lemmas 1–2 to terminal r does not improve by a minimal amount ǫ˜, the candidate equality constraints (TEC) using Assumption 8 and a multi- solution is applied. Given Assumption 8, after at most ν steps, step implementation. it is possible to incrementally move the reference trajectory and thus improve the cost. Thus, by augmenting the MPC with Lemma 4. Let Assumptions 5 and 8 hold and assume that y(t) such an algorithm, it may not be necessary to apply the ﬁrst is constant. Consider the terminal ingredients X (r , t) = f T ν steps of the computed input trajectory, which can speed up {x }, V (x, r , t, y) = 0 and a prediction horizon N ≥ ν. r f T convergence. Suppose that the optimization problem (6) is feasible at time t with some reference trajectory r (·|t), which is not a local B. Modiﬁed reference cost minimum to (2). Then under the ν-step closed-loop system The proposed formulation (6) uses standard conditions for ∗ ∗ u(t + k) = u (k|t), x(t + k + 1) = x (k + 1|t), (41) the terminal ingredients (Ass. 1) and contains many economic MPC formulations as special cases, compare [26], [27], [28], k = 0, . . . , ν − 1, [21], [9], [17], [18], [2], [6]. However, the formulation also there exists a reference r˜ which is a feasible candidate requires the additional constraints (6h)–(6i), based on the con- solution to (6) at t + ν and satisﬁes tinuity condition (Ass. 5). In the following, we brieﬂy discuss ν ∗ an alternative solution to this problem, based on a modiﬁed J (r˜ , t + N + ν, y(t)) < J (R r (·|t), t + N + ν, y(t)). T T T T T cost for the artiﬁcial reference trajectory. The following result ν ∗ Proof. Given that R r (·|t) ∈ Z (t + N + ν) is not a local T T is based on [42, Prop. 1], which in turn is motivated by the minimum, Assumption 5 ensures that there exists a reference analysis of non-monotonic Lyapunov functions [43]. ν ∗ r˜ = R r (·|t) + Δr ∈ Z (t + N + ν), that satisﬁes T T T T T Lemma 5. Consider the terminal ingredients from Assump- the posed constraints (6h)–(6i), improves the reference cost ν ∗ tion 1. For any t ∈ N, y ∈ Y, r ∈ Z (t), x ∈ X (r , t), the T T f T J and satisﬁes kr˜ − R r (·|t)k ≤ ǫ. Due to the multi- T T T T ∗ modiﬁed terminal cost step implementation (41), the sequence x(k|t + ν) = x (k + ν|t), u(k|t + ν) = u (k + ν|t) satisﬁes the dynamics (6d), the V (x, r , t, y) :=V (x, r , t, y) f T f T constraints (6e) and ends in the reference (6f), i.e., T−2 T − 1 − k ∗ ∗ + ℓ(r (k), t + k, y) x(N − ν|t + ν) = x (N|t) = x (0|t). k=0 Correspondingly, for any r˜ ∈ Z (t + N + ν) with T T satisﬁes ν ∗ kR r (·|t) − r˜ k ≤ ǫ, we can append the state and input T T ˜ ˜ sequence x(·|t+ν), u(·|t+ν) with the candidate solution from V (x ,R r , t + 1, y) − V (x, r , t, y) f T T f T Assumption 8, which satisﬁes the constraints (6e)–(6g). ≤ − ℓ(x, u, t, y) + J (r , t, y)/T, T T Compared to the results in Lemmas 1–2 based on terminal + with x = f(x, u, t), u = k (x, r , t). f T sets, the resulting properties with terminal equality constraints Proof. Abbreviate ℓ(k) = ℓ(r (k), t + k, y), V = T f are only valid if we apply the ﬁrst ν parts of the computed ˜ ˜ ˜ V (x, r , t, y), V = V (x ,R r , t + 1, y). The modiﬁed f T f T T input sequence. For comparison, in tracking MPC with positive terminal cost satisﬁes deﬁnite stage cost ℓ, such a multi-step implementation is not needed, since the closed-loop system eventually converges + ˜ ˜ T (V − V ) to an ǫ neighbourhood of the reference trajectory r , com- T−2 pare [24, Thm. 2], [8, Thm. 3] [23, Thm. 2]. =T (V − V ) + (T − 1 − k)(ℓ(k + 1) − ℓ(k)) The main beneﬁt of the terminal equality constraint im- k=0 plementation is the simple design. Although we need to use T−2 (8c) a multi-step implementation with ν steps, we would like ≤ − T ℓ(x, u, t, y) + (T + 1 − T )ℓ(0) + ℓ(T − 1) + ℓ(k) to point out that ν is independent of T , and hence this k=1 method does not suffer from the same limitations as the = − T ℓ(x, u, t, y) + J (r , t, y). T T approaches based on T -step systems, such as [19], [35]. On the other hand, an implementation with a suitable terminal cost (Ass. 3) can use any prediction horizon N, requires no multi- We point out, that the modiﬁcation of the cost in Lemma 5 is step implementation and typically yields better closed-loop applicable both to terminal equality constraints (Lemma 4) and performance and (inherent) robustness properties, compare the terminal cost/sets (Ass. 3). The following proposition shows numerical example in Section V-B. that this modiﬁed terminal cost can ensure the same theoretical Remark 6. In [28] a similar economic MPC scheme for properties (Prop. 2) as the proposed scheme (6), without using setpoints (T = 1) has been considered with terminal equality the continuity condition (Ass. 5). 14 Proposition 6. Let Assumption 1 hold and assume that y(t) is in Lemma 3 is independent of r and we can consider a time- constant. Consider the MPC formulation (6) with V replaced varying matrix P (t) to satisfy condition (30) in Proposition 4. by V (Lemma 5) and the constraints (6h)–(6i) replaced by Matrices P (t), K(t) satisfying condition (30) can be computed by solving T coupled LMIs similar to [44]. Alternatively, the T−1 computation of K(t), P (t) can be achieved using the discrete- J (r (·|t), t + N, y(t)) ≤ κ(t) := κ (t). (42) T T j time LQR for a suitably deﬁned T -step system with x˜ ∈ R j=0 Tm and u ˜ ∈ R . The resulting closed-loop system satisﬁes the performance Using the reformulation of the constraints (29) discussed bound (10), if the update rule B satisﬁes Assumption 2. in Remark 4, the possibly nonlinear constraints (29) can be dropped by adding the explicit nonlinear term for p(0|t) in Proof. Similar to Prop. 2 the candidate solution from Prop. 1 the cost function. For the terminal set X , we can either use with the modiﬁed terminal cost implies an ellipsoidal set X (r , t) = {x| kx − x k ≤ α} or a f T r P (t) W (t + 1) − W (t) + ℓ(x(t), u(t), t, y(t)) polytopic (periodically time-varying) invariant set X . Thus, ≤J (r (·|t), t + N, y(t))/T + γ(t)κ(t + 1) T ℓ convex implies that the constraints in (6) are convex. (9) In case ℓ quadratic, Lemma 3 and Prop. 4 contain no =κ(t + 1)/T + γ(t)κ(t + 1). nonlinear terms that need to be locally over-approximated Correspondingly, the T -step bound (15) holds with c = 0, and hence we can set ǫ = ǫ˜ = 0 and α arbitrary large. since κ(t + 1) ≤ κ(t). The remainder of the proof follows Furthermore, for ℓ quadratic, the problem (6) is a quadrati- from the arguments in Prop. 2, similar to [26, Thm. 1][27, cally constrained quadratic program (QCQP). If we drop the Thm. 1]. constraints (6i) (compare Section IV-B) and use a polytopic terminal set X , the optimization problem (6) is a (linearly The properties in Prop. 3 and Theorem 1 hold equally with constrained) quadratic program (QP). the modiﬁed terminal cost V and a terminal set (Ass. 3), with In the special case that ℓ is linear, i.e., ℓ = x q(t, y), the the simpler constraint (42) (without requiring Assumption 5). vector p can be explicitly computed (independent of the online The main advantage of using this modiﬁed terminal cost V optimized reference r ) for a given price y. Furthermore, is that the technical continuity condition Assumption 5 is since S = 0, the terminal cost is linear (P = 0). Thus, if not required and the number of constraints in (6) is smaller, a polyhedral terminal set is chosen, the proposed scheme with while the theoretical properties are the same. However, this a linear cost ℓ only requires the solution to a linear program modiﬁed terminal cost yields an objective function, which to (LP), which can be done efﬁciently. Furthermore, with a the best knowledge of the authors differs from any existing polytopic incremental Lyapunov function V , the terminal cost MPC formulation (for T > 1). Correspondingly, it is unclear in Proposition 5 can be formulated in the MPC problem (6) what the practical effect on the closed-loop performance is, using linear constraints, resulting in an LP (or QP in case of which will be studied in the numerical example in Section V-B. ℓ quadratic). For the example considered in Section II-C with a terminal For this special case with a prediction horizon of N = 0, the equality constraint and T = 2, the modiﬁed terminal cost is proposed MPC scheme is almost equivalent to the periodicity V = ℓ(r). With this modiﬁed cost the closed loop also does constraint MPC proposed in [21], [9]. The main difference the right thing, i.e., converges to the T -periodic orbit {1, 2}. is that we use a linear terminal cost and the polyhedral constraint, instead of a terminal equality constraint (and the constraints (6i) which can typically be neglected). This small C. Convex problem difference in the design allows us to derive the desired In the following, we discuss the special case, when the peri- performance guarantees, while the periodicity constraint MPC odic optimal problem (2) is convex. Suppose that the dynamics can lead to suboptimal performance, compare [9, Example 6]. f are afﬁne, i.e. f(x, u, t) = A(t)x + B(t)u + c(t), and the We can get the same theoretical properties with a terminal constraint sets Z are polytopes, which implies that Z is a r T equality constraint (TEC) and N ≥ ν, if we use Lemma 4. convex polytope. For ℓ convex, this implies that the periodic In case that some of the input variables u are also subject optimal problem (2) is convex and Theorem 1/Corollary 1 to integer constraints (c.f. for example periodic scheduling guarantee that the closed-loop performance is no worse than problems with discrete decisions [15] and the HVAC numerical operation at an optimal T -periodic orbit. example in Section V-A), the problem can be formulated as a In Section IV-C1 we discuss how the construction of the mixed-integer linear program (MILP). terminal ingredients (Sec. IV-A) and the online optimization 2) Continuity condition - Assumption 5: In the following, simpliﬁes in the convex case. In Section IV-C2, we discuss we discuss sufﬁcient conditions for Assumption 5. Suppose the continuity condition (Ass. 5) for periodic optimal control. that ℓ is continuously differentiable and Z is a convex 1) Ofﬂine design and online optimization: In the following, we discuss how the design procedure (Sec. IV-A) and the 3 The optional consideration of an online optimized terminal set size α(r ) online optimization simpliﬁes for the considered special case. can be expressed using linear constraints, for both cases. In addition to the possibly ellipsoidal terminal set, the constraints (6i) Since we have a linear (time-varying) system, we consider a are quadratic, leading to a non negligible increase in the online computation. linear time-varying feedback k = K(t) and a time-varying f,x Given that p(0|t) in (37) is linear in r , the terminal cost V is quadratic in T f matrix S(t) satisfying condition (23). Thus, the matrix Q (t) the decision variables. 15 polytope. Given a reference r , the direction of feasible satisﬁes W (t) ≤ W , which is equivalent to changes Δr , which imply a decrease in J , i.e. {Δr ∇J ≤ T T T β(J (r (·|t), t + N, y) − J (t)) T T,min 0, r + Δr ∈ Z }, is a polytope. The condition in T T T N−1 Assumption 5 reduces to the existence of a direction Δr in ∗ ∗ ≤ ℓ(x ˜(k), u ˜(k), t + k, y) − ℓ(x (k|t), u (k|t), t + k, y) this set, such that the directional derivative of ℓ is uniformly k=0 bounded relative to the directional derivative of J . + V (x ˜(N), r (t), t + N, y) f min Although this condition is reasonable in the context of ∗ ∗ − V (x (N|t), r (·|t), t + N, y) optimal periodic control, it is not necessarily satisﬁed for ≤η, any convex problem. In particular, it is not valid if the (directional) derivative of J vanishes, but the gradient of T with some constant η > 0. The last inequality follows from ℓ is (uniformly) lower bounded. This is for example the boundedness of ℓ, V and N ﬁnite. This inequality directly case, if the economic cost J is quadratic and the inequality T implies constraints are not active at the optimal periodic orbit, yielding κ(t + 1) = J (r (·|t), t + N, y) ≤ J (t) + ǫ, T T,min a vanishing gradient of J . If this problem occurs, we need to use the reformulation in Section IV-B to guarantee optimal for β ≥ β(ǫ) := η/ǫ. performance without Assumption 5. Thus, for a large enough weight β, the cost of the artiﬁcial Suppose that there exists a constant ǫ > 0, such that for periodic orbit r is arbitrarily (ǫ) close to the cost of the any r ∈ Z which is not a local minimum, there exists T T optimal reachable periodic orbit (20). Combining this result a feasible direction Δr with ∇J Δr ≤ −ǫkΔr k. Then T T T with Lemma 2 and the stronger terminal ingredients (Ass. 3), Assumption 5 is satisﬁed with some ﬁnite c , if ℓ is Lipschitz J is a local minimizer to (2). Correspondingly, it is pos- T,min continuous. Note that such a directional derivative always sible to derive performance bounds similar to Theorem 1 with exists if, e.g., ℓ is linear. an additional suboptimality term ǫ, compare [28, Thm. 2]. V. NUMERICAL EXAMPLES The following examples demonstrate the applicability of the proposed framework to dynamic operation and online chang- ing conditions. We ﬁrst consider a simple HVAC systems [33], where dynamics, cost and constraints are periodically time- D. Constant parameter β varying and discrete inputs are considered. Then we consider the classical problem of increasing the yield of continuous stirred-tank reactors (CSTR) with dynamic operation, com- In the following, we brieﬂy discuss how the performance pare [13]. In this example, we compare the performance of bounds in Proposition 3 can be approximately guaranteed the proposed approach with periodicity constraint MPC [21], with a constant weight β. In particular, in [28] a competing [9], tracking MPC formulations [8], [25] and MPC without approach to [26], [27] has been considered with a constant terminal constraints [7], [19], [20]. In addition, we study the weight β. Instead of changing the weight β online to achieve effect of the various degrees of freedoms in the formulation (locally) optimal performance, a ﬁxed weight β is considered (terminal ingredients (Sec. IV-A), alternative cost formulations and a suboptimality bound on the performance is established. (Sec. IV-B), continuous-time formulations (Remark 2)) on The following proposition shows that the same result applies closed-loop performance. here, as an alternative to Proposition 3, similar to [28, Prop. 2]. A. Building cooling Proposition 7. Let Assumptions 1 and 5 hold with a bounded terminal cost V and assume that y(t) and β(t) are constant. In the following, we show the applicability of the proposed Assume that the optimization problem (6) is feasible at time t. framework to periodic optimal control problems subject to There exists a function β, such that for any ǫ > 0, β ≥ β(ǫ) online changing performance measures. We consider a simple implies κ(t + 1) ≤ J (x(t), t, y, κ (t)) + ǫ. T,min j building temperature evolution example from [33, Sec. IV.A] governed by Proof. Denote the minimizer and minimum of (20) at time t by r (t) and J (t), respectively. By deﬁnition, there m T (t) = −k(T (t) − T (t)) + q (t) − q(t), min T,min amb amb dt exists a feasible input sequence u ˜ with corresponding state with air temperature T , cooling rate q, ambient temperature sequence x ˜ such that r (t) satisﬁes the constraints in (6). min T , rate of direct heat by the ambient q and model amb amb Due to optimality, the cost constants m, k > 0. The cooling rate q is generated using N−1 N = 2 chillers and is subject to the following (time- chiller W = ℓ(x ˜(k), u˜(k), t + k, y) invariant) disjoint constraint set k=0 + V (x ˜(N), r (t), t + N, y) + βJ (t), q ∈ U = {0} × [0.75, 1] × [1.5, 2], f min T,min 16 1.5 which is implemented using an additional discrete decision variable v ∈ {0, 1, 2}, corresponding to the number of active chillers. The state is subject to time-varying comfort bounds 1 centred around T = 0: 0.5 T (t) ≤ T ≤ T (t). min max The corresponding discrete-time system is given by x(t + 1) = Ax(t) + Bu(t) + e(t), (x(t), u(t)) ∈ Z(t), -0.5 with x = T , u = q and periodically time-varying e(t) based on q , T . The economic cost function is to minimize the amb amb -1 electricity cost given by ℓ(x, u, t, y) = ρ(t, y) · u, with the 50 100 150 price proﬁle ρ(t, y). In [33], for ﬁxed periodic price signals ρ(t), it was shown that periodic economic formulations [17], [18], [15] outper- form tracking formulations [8], [25]. We consider the more challenging problem, where the price proﬁle ρ changes each 1.5 day. Furthermore, we assume that only the price proﬁle for the current day is available as a forecast, which is modelled using the external variable y that changes every 24 hours. The considered price proﬁle ρ is taken from the real data considered in [12] over the span of one week. 0.5 We implemented the proposed approach using N = 2, β = 10 with the modiﬁed cost from Prop. 6 and a terminal equality constraint (TEC), which also satisﬁes the properties in Lemma 1 for the considered scalar stable system. The resulting 0 50 100 150 MPC optimization problem (6) is a small scale mixed-integer linear program (MILP), which was solved to optimality using intlinprog from MATLAB. The resulting closed loop can be Fig. 2. Transient performance under online changing price signals ρ for seen in Figure 2. The closed loop yields a periodic like HVAC. Top: Closed-loop temperature x = T of the proposed approach (blue, solid) and the optimal operation (red, dotted), with time-varying constraints operation for each day, with small changes between each day T (black, dashed). Bottom: Closed-loop applied cooling rate u = q min / max based on the different price proﬁle ρ. The adjustment of the for the proposed approach (blue, solid) and optimal operation (red, dotted); closed-loop response based on the price ρ can be directly seen and the price signal ρ (green, dotted). with the applied input u, which is always at a maximum when the electricity price ρ is low. We also compared the proposed framework to the optimal operation in Figure 2, assuming full System model: We consider a continuous-time model of a knowledge of the future price proﬁle ρ(t) for all coming days. continuous stirred tank reactor (CSTR)   The proposed framework results in state and input trajectories   −1 −0.55 4 2 1 − x − 10 x exp − 400x exp 1 1 x˙ 1 very similar to the optimal operation, resulting in a minimal x x 1 3 3       −1 x˙ = 4 2 , increase of 0.1% in the overall electricity price. 10 x exp − x   x˙ u − x where u ∈ R is related to the heat ﬂux and x = ⊤ 3 (x , x , x ) ∈ R correspond to the concentration of the re- 1 2 3 B. Continuous stirred-tank reactor action, the desired product and the temperature, compare [13], [45], [5, Sec. 3.4]. The constraints are In the following time-invariant example, we ﬁrst demon- strate average performance improvement of the proposed Z =[0.05, 0.4] × [0.05, 0.2] × [0.059, 0.439], framework compared to ﬁxed periodic operation or steady- Z =[0.03, 1] × [0.049, 0.449], state operation. Then we show reliable economic performance and we consider the economic stage cost ℓ(x, u, y) = −x + under online changing dynamic operation due to changing cost 2 y · (u − u ) with u = 0.1491 and y ∈ Y = [0, 1]. If the functions. s s external parameter is y = 0, the stage cost ℓ tries to maximize the production of the desired product x . The online tunable The proposed framework can also consider peak-demand prices using part in the cost function is a regularization of the input u the formulation in [33] or unpredictably changing constraint sets (reﬂecting relative to the optimal steady-state input u . For y = 1 the comfort levels set by a user) using soft constraints (c.f. Remark 1). system is optimally operated at a steady-state (x , u ), while s s To allow for a consistent comparison, the same initial and ﬁnal state is considered. for y = 0 dynamic operation signiﬁcantly outperforms steady- 17 state operation, compare [5, Sec. 3.4]. Hence, treating y as an external variable allows a user to smoothly transition between steady-state and dynamic operation. The discrete-time model is deﬁned with a fourth order Runge-Kutta discretization and a sampling time of h = 0.05. For the following simulations, the initial condition is always chosen as the optimal steady-state. Average performance improvement: We ﬁrst consider the problem of maximizing the concentration x (y = 0) to show average performance improvements. In the absence of transient changes (y constant), the average performance of periodic- ity constraint MPC [21], [9] and tracking formulations [8], [25] are equivalent (assuming that convergence is achieved). Similarly, the proposed framework yields the same asymptotic 0 10 20 30 40 50 performance as [17], [18], [15] assuming that r converges. We implemented the proposed approach with T ∈ {1, 10, 20} and horizons N ∈ [1, 50] and tested dif- Fig. 3. Average performance improvement due to dynamic operation ferent proposed designs regarding the terminal ingredients relative to the optimal steady-state x - CSTR. Periodic operation (dotted) (V ,X ,Corollary 2, Remark 5, Lemma 4) and the cost f f vs. proposed economic MPC scheme (solid with circles) for T = 1 (blue), function (V , Lemma 5, Prop. 6). The detailed numerical T = 10 (green) and T = 20 (red). results for all the considered implementations can be found in Appendix A. In the following, we only consider the approach utilizing the positive deﬁnite terminal cost V from Remark 5 (based on Alg. 1) in combination with the modiﬁed cost V from Lemma 5, which seems most suitable for practi- 0.5 cal applications (in terms of computational complexity and performance). Figure 3 exemplarily shows the performance of this approach with T ∈ {1, 10, 20} for increasing N in comparison to the average cost at the optimal periodic orbit of length T = {10, 20} and the optimal steady-state (T = 1). We note that, neglecting small initial deviations , -0.5 the proposed EMPC outperforms optimal periodic operation with the same period length T , even though a constant value β is used (Ass. 4 does not hold). We can see in general that -1 the performance increases (for both purely periodic operation 0 100 200 300 400 500 and the proposed approach) if we increase T and N. This implies that the proposed framework utilizing periodic orbits (T > 1) and additional predictions (N ≥ 1) with a purely Fig. 4. Dynamic operation under online changing conditions: deviation in economic formulation can outperform periodicity constrained production x −x (blue, solid), deviation in heat ﬂux u−u (red, dashed), 2 2,s s formulations [21], [9] (N = 0), steady-state formulations [26], price signal y (green, dotted). All signals are normalized to |x| ≤ 1. [27], [28], [29] (T = 1) and periodic tracking formulations [8], [23], [24], [25] (ℓ positive deﬁnite), even in case of ﬁxed optimal operation (y constant). (e.g. t ∈ [185, 246], t ∈ [400, 470]) to the new optimal Transient performance under online changing conditions: mode of operation, i.e., the steady-state x . In this scenario, In the following, we study the performance of the proposed the MPC scheme on average still increases production by scheme under online changing conditions, i.e., y unpre- 2.8% compared to steady-state operation, while a 5% increase dictably time-varying. The resulting closed loop for the pro- was achieved for y ≡ 0 (c.f. Fig. 3). For comparison, the posed MPC scheme with N = 10, T = 20 can be seen performance of the proposed approach, the tracking MPC [8], in Figure 4. As y → 0 (e.g. t = 15 or t = 246), the [25] with T = 20, N ∈ [0, 50] and the periodicity constraint system operates dynamically to increase production x and MPC [21], [9] with T ∈ [0, 90] can be seen in Figure 5. once the weight y on input deviations increases the system First, note that the number of decision variables in a condensed quickly minimizes the control effort and smoothly converges formulation are n + m · (T + N) for the proposed approach and the tracking MPC [8], [25], and m · T for the periodicity In [45], [5] a sampling time of h = 0.1 is used. However, with the constraint MPC [21], [9] (N = 0). Thus, the x-axis (N + T ) considered fourth order explicit Runge-Kutta discretization, a sampling time in Figure 5 is a measure for the computational complexity. of h = 0.1 does not preserve stability of the continuous-time system. In addition, we consider x ≥ 0.03 instead of x ≥ 0, to avoid discretization i i First, note that we can further improve the performance of errors for x ≈ 0. the proposed approach by increasing N. Similarly, the perfor- The average performance is computed in the interval t ∈ [1000, 2000] mance of the periodicity constraint MPC [21], [9] improves starting with initial condition x = x . Thus, for very short horizons N, κ has not yet converged. for a larger period length T , but at a smaller pace. Thus, given 18 the same number of decision variables, the proposed approach or modiﬁed versions thereof [28], [21], [9], [18], [34] are can achieve a better performance. If we consider the tracking contained as special cases. MPC, for small values of N, the performance is similar to The practical application of the proposed framework to large the proposed economic formulation. However, in contrast to scale HVAC systems is part of future work. A theoretical anal- the proposed formulation, the economic performance of the ysis of (bounded) online changing constraint sets/dynamics tracking MPC does not improve signiﬁcantly with a large with robust performance guarantees is an open problem. horizon N. Additional numerical results can be found in Appendix A. REFERENCES To summarize, in the considered example we have shown [1] J. B. Rawlings, D. Q. Mayne, and M. Diehl, Model Predictive Control: the applicability of the proposed approach to nonlinear eco- Theory, Computation, and Design. Nob Hill Publishing, 2017. [2] D. Angeli, R. Amrit, and J. B. Rawlings, “On average performance nomic optimal control problems. In particular, the proposed and stability of economic model predictive control,” IEEE Trans Autom approach: (i) improves performance compared to (ﬁxed) Control, vol. 57, pp. 1615–1626, 2012. [3] M. Ellis, J. Liu, and P. D. Christoﬁdes, Economic Model Predictive steady-state or periodic operation, (ii) reliably operates under Control: Theory, Formulations and Chemical Process Applications. online changing conditions, (iii) in general achieves better Springer, 2016. performance than periodicity constraint formulations [21], [9] [4] M. A. Mu¨ller and F. Allgo¨wer, “Economic and distributed model predictive control: Recent developments in optimization-based control,” or tracking formulations [8], [25]. SICE J. of Control, Measurement, and System Integration, vol. 10, pp. 39–52, 2017. [5] T. Faulwasser, L. Gru¨ne, and M. A. Mu¨ller, “Economic nonlinear model predictive control,” Foundations and Trends® in Systems and Control, vol. 5, pp. 1–98, 2018. -1 [6] R. Amrit, J. B. Rawlings, and D. Angeli, “Economic optimization using model predictive control with a terminal cost,” Annual Reviews in -2 Control, vol. 35, pp. 178–186, 2011. [7] L. Gru¨ne, “Economic receding horizon control without terminal con- -3 straints,” Automatica, vol. 49, pp. 725–734, 2013. [8] D. Limon, M. Pereira, D. M. De La Pen˜a, T. Alamo, and J. M. Grosso, -4 “Single-layer economic model predictive control for periodic operation,” J. Proc. Contr., vol. 24, pp. 1207–1224, 2014. -5 [9] Y. Wang, J. R. Salvador, D. M. de la Pen˜a, V. Puig, and G. Cembrano, “Economic model predictive control based on a periodicity constraint,” J. Proc. Contr., vol. 68, pp. 226–239, 2018. -6 [10] M. Pereira, D. Limon, D. M. de la Pen˜a, L. Valverde, and T. Alamo, “Periodic economic control of a nonisolated microgrid,” IEEE Trans Ind -7 Electron, vol. 62, no. 8, pp. 5247–5255, 2015. [11] G. Serale, M. Fiorentini, A. Capozzoli, D. Bernardini, and A. Bemporad, 20 40 60 80 “Model predictive control (MPC) for enhancing building and HVAC system energy efﬁciency: Problem formulation, applications and oppor- tunities,” Energies, vol. 11, no. 3, p. 631, 2018. 2 [12] J. B. Rawlings, N. R. Patel, M. J. Risbeck, C. T. Maravelias, M. J. Fig. 5. Transient performance ℓ(x, u, y) = −x + y · (u − u ) in dynamic 2 s Wenzel, and R. D. Turney, “Economic MPC and real-time decision operation under online changing conditions for CSTR relative to steady-state making with application to large-scale hvac energy systems,” Computers operation ℓ = ℓ(x , u ): Proposed approach (blue, solid), tracking MPC [8], s s s & Chemical Engineering, vol. 114, pp. 89–98, 2018. [25] (red, dashed) with T = 20 and N ∈ [0, 50] and periodicity constraint [13] J. Bailey, F. Horn, and R. Lin, “Cyclic operation of reaction systems: MPC [21], [9] (green, dotted) with T ∈ [0, 90]. Effects of heat and mass transfer resistance,” AIChE Journal, vol. 17, pp. 818–825, 1971. [14] H. Budman and P. L. Silveston, “Control of periodically operated reactors,” in Periodic Operation of Chemical Reactors. Elsevier, 2013, VI. CONCLUSION pp. 543–567. [15] M. J. Risbeck, C. T. Maravelias, and J. B. Rawlings, “Uniﬁcation of We have presented an economic MPC framework that is closed-loop scheduling and control: State-space formulations, terminal applicable to nonlinear (periodically) time-varying problems constraints, and nominal theoretical properties,” Computers & Chemical Engineering, vol. 129, p. 106496, 2019. and online changing operation conditions. We have shown [16] M. Diehl, L. Magni, and G. De Nicolao, “Efﬁcient NMPC of unstable recursive feasibility, constraint satisfaction and derived per- periodic systems using approximate inﬁnite horizon closed loop costing,” formance guarantees relative to periodic (locally) optimal Annual Reviews in Control, vol. 28, no. 1, pp. 37–45, 2004. [17] M. Zanon, L. Gru¨ne, and M. Diehl, “Periodic optimal control, dissipa- operation. Interestingly, the problem of economic periodic tivity and MPC,” IEEE Trans. Autom. Control, vol. 62, pp. 2943–2949, operation requires additional techniques compared to opti- mal steady-state operation, which was also the case in [19]. [18] A. Alessandretti, A. P. Aguiar, and C. N. Jones, “On convergence and performance certiﬁcation of a continuous-time economic model predic- In particular, we used a novel continuity condition on the tive control scheme with time-varying performance index,” Automatica, economic cost of periodic orbits, to reformulate the update vol. 68, pp. 305–313, 2016. scheme and constraints on the artiﬁcial periodic reference [19] M. A. Mu¨ller and L. Gru¨ne, “Economic model predictive control without terminal constraints for optimal periodic behavior,” Automatica, vol. 70, trajectory. In addition, we proposed novel ofﬂine computations pp. 128–139, 2016. to obtain suitable terminal ingredients. We have demonstrated [20] L. Gru¨ne and S. Pirkelmann, “Economic model predictive control for the applicability and practicality of the proposed framework time-varying system: Performance and stability results,” Optimal Control Applications and Methods, 2018. for economic periodic control with an HVAC and a CSTR [21] B. Houska and M. A. Mu¨ller, “Cost-to-travel functions: A new per- example. The strength of the framework comes from the fact spective on optimal and model predictive control,” Systems & Control that many existing schemes [26], [27], [17], [6], [38], [2] Letters, vol. 106, pp. 79–86, 2017. 19 [22] D. Limon, I. Alvarado, T. Alamo, and E. F. Camacho, “MPC for [47] J. Andersson, J. Akesson, and M. Diehl, “Casadi: A symbolic package tracking piecewise constant references for constrained linear systems,” for automatic differentiation and optimal control,” in Recent advances Automatica, vol. 44, pp. 2382–2387, 2008. in algorithmic differentiation. Springer, 2012, pp. 297–307. [48] P. O. Scokaert, D. Q. Mayne, and J. B. Rawlings, “Suboptimal model [23] D. Limon, M. Pereira, D. M. de la Pen˜a, T. Alamo, C. N. Jones, and predictive control (feasibility implies stability),” IEEE Trans. Autom. M. N. Zeilinger, “MPC for tracking periodic references,” IEEE Trans. Control, vol. 44, pp. 648–654, 1999. Autom. Control, vol. 61, pp. 1123–1128, 2016. [24] D. Limon, A. Ferramosca, I. Alvarado, and T. Alamo, “Nonlinear MPC for tracking piece-wise constant reference signals,” IEEE Trans. Autom. Control, vol. 63, pp. 3735–3750, 2018. [25] J. Ko¨hler, M. A. Mu¨ller, and F. Allgo¨wer, “A nonlinear tracking model predictive control scheme for unreachable dynamic target signals,” Automatica, vol. 118, p. 109030, 2020. [26] M. A. Mu¨ller, D. Angeli, and F. Allgo¨wer, “Economic model predictive control with self-tuning terminal cost,” European Journal of Control, vol. 19, pp. 408–416, 2013. [27] ——, “On the performance of economic model predictive control with self-tuning terminal cost,” J. Proc. Contr., vol. 24, pp. 1179–1186, 2014. [28] L. Fagiano and A. R. Teel, “Generalized terminal state constraint for model predictive control,” Automatica, vol. 49, pp. 2622–2631, 2013. [29] A. Ferramosca, D. Limon, and E. F. Camacho, “Economic MPC for a changing economic criterion for linear systems,” IEEE Trans. Autom. Control, vol. 59, pp. 2657–2667, 2014. [30] T. J. Broomhead, C. Manzie, R. C. Shekhar, and P. Hield, “Robust periodic economic MPC for linear systems,” Automatica, vol. 60, pp. 30–37, 2015. [31] J. Gutekunst, H. G. Bock, and A. Potschka, “Economic NMPC for averaged inﬁnite horizon problems with periodic approximations,” Au- tomatica, vol. 117, p. 109001, 2020. [32] J. B. Rawlings, D. Bonne´, J. B. Jorgensen, A. N. Venkat, and S. B. Jorgensen, “Unreachable setpoints in model predictive control,” IEEE Trans. Autom. Control, vol. 53, pp. 2209–2215, 2008. [33] M. J. Risbeck and J. B. Rawlings, “Economic model predictive control for time-varying cost and peak demand charge optimization,” IEEE Transactions on Automatic Control, 2019. [34] D. Angeli, A. Casavola, and F. Tedesco, “Theoretical advances on economic model predictive control with time-varying costs,” Annual Reviews in Control, vol. 41, pp. 218–224, 2016. [35] K. P. Wabersich, F. A. Bayer, M. A. Mu¨ller, and F. Allgo¨wer, “Economic model predictive control for robust periodic operation with guaranteed closed-loop performance,” in Proc. European Control Conf. (ECC), 2018, pp. 507–513. [36] M. A. Mu¨ller, L. Gru¨ne, and F. Allgo¨wer, “On the role of dissipativity in economic model predictive control,” in Proc. 5th IFAC Conf. Nonlinear Model Predictive Control, 2015, pp. 110–116. [37] J. Ko¨hler, M. A. Mu¨ller, and F. Allgo¨wer, “A nonlinear model predictive control framework using reference generic terminal ingredients,” IEEE Trans. Autom. Control, vol. 65, no. 8, pp. 3576–3583, 2020, extended version online: arXiv:1909.12765. [38] A. Alessandretti, A. P. Aguiar, and C. N. Jones, “On the design of discrete-time economic model predictive controllers,” in Proc. 55th IEEE Conf. Decision and Control (CDC). IEEE, 2016, pp. 3196–3201. [39] M. Zanon and T. Faulwasser, “Economic MPC without terminal con- straints: Gradient-correcting end penalties enforce asymptotic stability,” J. Proc. Contr., vol. 63, pp. 1–14, 2018. [40] I. R. Manchester and J.-J. E. Slotine, “Control contraction metrics: Convex and intrinsic criteria for nonlinear feedback design,” IEEE Trans. Autom. Control, vol. 62, pp. 3046–3053, 2017. [41] M. Zamani, N. van de Wouw, and R. Majumdar, “Backstepping con- troller synthesis and characterizations of incremental stability,” Systems & Control Letters, vol. 62, no. 10, pp. 949–962, 2013. [42] J. Ko¨hler, M. A. Mu¨ller, and F. Allgo¨wer, “On periodic dissipativity notions in economic model predictive control,” IEEE Control Systems Letters, vol. 2, pp. 501–506, 2018. [43] A. A. Ahmadi and P. A. Parrilo, “Non-monotonic Lyapunov functions for stability of discrete time nonlinear and switched systems,” in Proc. 47th IEEE Conf. Decision and Control (CDC), 2008, pp. 614–621. [44] E. Aydiner, M. A. Mu¨ller, and F. Allgo¨wer, “Periodic reference tracking for nonlinear systems via model predictive control,” in Proc. European Control Conf. (ECC), 2016, pp. 2602–2607. [45] M. A. Mu¨ller, D. Angeli, F. Allgo¨wer, R. Amrit, and J. B. Rawlings, “Convergence in economic model predictive control with average con- straints,” Automatica, vol. 50, pp. 3100–3111, 2014. [46] J. F. Sturm, “Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones,” Optimization methods and software, vol. 11, pp. 625–653, 1999. 20 APPENDIX ﬁrst 1000 steps are discarded to better reﬂect the asymptotic average performance. A. Continuous stirred-tank reactor - details First, Figure 6 shows that we can in general achieve better In the following, we provide additional details regarding performance by using a dynamic operation. We can see that the implementation of the CSTR example in Section V-B increasing the length T of the optimal T -periodic orbit (2) and investigate the economic performance of the different can increase the average production by approximately 8%. We considered implementations in more detail. would like to point out, that the “optimal” T-periodic orbits Implementation details: In the implementation, we consider where computed using CasADi and may thus also represent c = 100 β = 10. The ofﬂine computation is done with an local minima. In addition to the periodic orbit, we also consid- Intel Core i7 using the semideﬁnite programming (SDP) solver ered and implemented an economic MPC scheme without any SeDuMi-1.3 [46] and the online optimization is done with terminal ingredients or artiﬁcial periodic trajectories [7], [19], CasADi [47]. The iterations in CasADi were stopped after [20]. We would ﬁrst like to point out again, that in general 1000 iterations, although typically the standard tolerance was for this setup (nonlinear system, unknown mode of optimal satisﬁed. Since the optimization problem (6) is not solved to dynamic operation), guaranteeing a priori desired performance optimality but a ﬁxed number of iterations are done online, bounds with such an implementation is difﬁcult. Neverthe- the resulting reference r is not necessarily a feasible periodic less, this implementation supports two important arguments orbit. Hence, the primal infeasibility of the reference r should of the proposed approach. First, we can see that for short be taken into account when deﬁning κ in (7c) (e.g. in terms horizons N (e.g. N ≤ 20 here) the performance of such of a penalty factor), to avoid persistent feasibility issues. Thus, a simple implementation without suitable modiﬁcations can we replaced (7c) by be worse than steady-state performance and thus worse than ∗ 3 implementing a simple tracking/stabilizing MPC. Second, we κ (t + 1) =ℓ(r (j + 1|t), t + N + 1 + j, y(t + 1)) + 10 ǫ, can see for large horizons N, e.g. N ≥ 50, that it is possible to where ǫ is the largest constraint violation (measured in terms of obtain signiﬁcantly better performance than periodic operation the inﬁnity norm) in the periodicity constraint (6g). In addition, with a ﬁxed period length T , which can be obtained with we replaced the resulting optimized trajectory by the feasible periodic tracking MPC schemes like [8], [23], [25]. We also −4 candidate solution in case it has a worse cost (up to 10 ) or included a scheme with multi-step implementation, since there −3 does not satisfy the constraints (up to 10 ), which ensures exist theoretical reasons to expect better performance if the that the performance guarantees in Proposition 2 remain valid optimal period length is used [19]. However, as it is not clear independent of numerical issues . if the system is optimally periodically operated and/or with Ofﬂine computations: For the design of the terminal cost which period length, we simply implemented a small value V and the terminal set X , we proceed along the lines of f f ν = 3, which, however, did not show any performance beneﬁts Algorithm 1. We consider S = diag(0, I ) ℓ and 2ǫ+ǫ˜ = m rr (except for the reduced computational demand). 0.1 in (35). The matrices P (r), K(r) are computed using the LMIs from [37, Lemma 2] and gridding (x , x , x , u, u ) 1 2 3 using 20 · 2 = 16.000 points, which takes 37 minutes. The terminal set is X = {x| kx − x k ≤ α}, with α = f r P (r) −8 −8 1.2 · 10 . The veriﬁcation of α = α = 1.2 · 10 along 1 2 the lines of [37, Alg. 1] took approximately 45 minutes. For this example, the convex formulation [37, Prop. 1] can only be used if the sampling time h is reduced and the resulting terminal cost tends to be very conservative. For details on the different formulations, compare also [37, Example 1]. In addition, we computed a constant c ≈ 21 ofﬂine, such that -5 the simpler terminal cost proposed in Remark 5 also satisﬁes 0 50 100 150 Assumptions 1 and 3. Fig. 6. Average performance improvement due to dynamic operation relative Average performance improvement to optimal steady-state x - CSTR. Periodic orbit (blue, solid, periodic) with We ﬁrst consider the general economic performance with period length T , unconstrained economic MPC (red, dashed, UCON) with horizon N and unconstrained economic MPC with multi-step implementation the ﬁxed economic stage cost ℓ(x, u) = −x , using y = using ν = 3 (green, dotted, UCON - ν = 3). 0 and compare the performance relative to the perfor- mance at the optimal steady-state x = (x , x , x ) = s 1,s 2,s 3,s We implemented the proposed approach (Problem (6)) using (0.0832; 0.0846; 0.1491). The performance is speciﬁed as im- N ∈ [0, 50], T ∈ {1, 10, 20}, β = 10 and c = 100. provement (in %) of the average concentration x and is For the terminal ingredients; we implemented the design computed over the time-interval t ∈ [1000, 2000], where the in Corollary 2 (QINF), the positive deﬁnite terminal cost V While ideally any solver should guarantee that the resulting solution is no worse than any provided feasible initial solution [48], this was not always We did not formally verify satisfaction of Assumption 5 with c = 100. the case in the considered implementation with IPOPT, probably due to the Instead ,we simply checked numerically that the proposed approach was able difﬁculty of initializing the dual variables. to converge to (local) optimal periodic orbits r with the chosen value. T 21 from Remark 5 (QINF-pdf), and a terminal equality constraint (TEC) with ν = 3 (Lemma 4). The corresponding results can be seen in Figure 7. For the economic terminal cost V from Corollary 2, the constraints (29) to compute p(·|t) are implemented using (36) from Remark 4. We also tested the alternative explicit formulation (37), however, except for the trivial case T = 1, this lead to numerical difﬁculties and a signiﬁcant increase in online iterations without any major beneﬁt. We also implemented the terminal equality constraint (TEC) MPC using a one-step implementation (ν = 1), which resulted in an improved performance for T, N large. If we use an artiﬁcial setpoint (T = 1, top ﬁgure), we 0 10 20 30 40 50 can a) clearly see that the performance further improves if we increase the prediction horizon N; b) we can see that a terminal cost/set (QINF) improves the performance relative to a terminal equality constraint (TEC). For example, the performance with N = 50, T = 1 with TEC is worse than the performance with a terminal cost using N = 40, T = 1, thus showing the potential for performance improve- ment/computational saving using suitable terminal ingredients. On the other hand, the difference between the positive deﬁnite terminal cost (QINF-pdf) and the economic terminal cost (QINF) is often small and thus the simple design in Remark 5 may be most favourable for practical implementations. For T = 10 (middle ﬁgure) and T = 20 (bottom ﬁgure) we can see 0 10 20 30 40 50 similar performance differences for larger horizons N: We can achieve better performance with a large horizon N or larger period length T , and we can achieve better performance if we use a terminal cost (QINF, QINF-pdf) instead of a terminal equality constraint (TEC). We also implemented the different terminal ingredients with the modiﬁed cost V from Lemma 5, compare Figure 8. If we compare the results in Figure 7 and Figure 8, the performance is essentially similar. Thus, the modiﬁed formula V (Lemma 5) does not seem to have any particularly negative impact on the performance, while reducing the computational demand by replacing the constraints (6h)–(6i) with (42). 0 10 20 30 40 50 We also implemented the continuous-time formulation dis- cussed in Remark 2 with a variable sampling time h ∈ −2 −2 [h , h ] = [10 , 5 · 10 ]. The resulting performance min max Fig. 7. Average performance improvement relative to optimal steady-state x for T = 1 (top), T = 10 (middle) and T = 20 (bottom). Approach for the optimal T -periodic orbit (2), and an economic MPC s with economic terminal cost (red, solid, QINF); positive deﬁnite terminal cost scheme without any terminal ingredients (UNCON) can be (blue, dashed, QINF-pdf); terminal equality constraint (black, dotted, TEC) seen in Figure 9. For comparison, we also implemented the with prediction horizon N ∈ [1, 50]. Performance at the optimal T -periodic orbit is show in green, dashed for comparison. proposed approach with a terminal equality constraint (TEC, Lemma 4), ν = 3 and T = 20. We can see that the variable sampling time h signiﬁcantly improves the overall perfor- We also tested the naive implementation discussed in Sec- mance compared to the ﬁxed sampling time implementation tion II-C. Although this approach is in general not satisfactory shown in Figure 6. Obtaining improved terminal ingredients (as shown in Sec. II-C), in the considered example this similar to Corollary 2 for this scenario, however, requires 11 approach often performs similar to the proposed formulation. further research . However, for small horizons N and a small weighting β, ne- glecting the constraints (6h)–(6i) can yield worse performance As the continuous-time analogue to (30) in Prop. 4, we can design 2 2 than steady-state operation, similar to the implementation a quadratic term satisfying kx − x k ≤ −kx − x k r r dt P(r ,t) Q (r ,t) T T using [37, Lemma 4/Prop. 5], which only requires 10s and 53s, respectively without terminal ingredients (UCON) in Figure 6. (compared to 37 min for the discrete-time formulation). However, a suitable Summary: Improved performance: In this scenario, we terminal cost V also requires a simple means to compute a corresponding have only considered the case of ﬁxed known stage cost ℓ gradient correction term p in continuous-time. In addition, due to the average dV (y constant). In this case, the proposed framework reduces to cost function proposed in Remark 2, we need ≤ −ℓ /h, which might dt only hold by scaling the terminal cost conservatively using . the periodic economic MPC approaches in [17], [18], [15], min 22 0 50 100 150 200 0 10 20 30 40 50 Fig. 9. Average performance improvement relative to optimal steady-state x with ﬂexible sampling time h. Periodic orbit (blue, solid, periodic cont.) with period length T , unconstrained economic MPC (red, dashed, UCON cont.) with horizon N and terminal equality constraint using ν = 3 and T = 20 (green, dotted, TEC T = 20 cont.) (y constant). The second scenario investigates this difference in performance for the transient case of changing operation conditions (y not constant). Different formulations: A second key ﬁnding in this scenario is the comparison of the different design choices 0 10 20 30 40 50 possible in the proposed framework, in particular the termi- nal ingredients (V ,X , Ass. 1, 3 Corollary 2, Remark 5, f f Lemma 4) and the modiﬁed cost function (V , Lemma 5, Prop. 6). First, even though the cost formulation V proposed in Lemma 5 may be unconventional, we have not found any dis- advantages in the performance comparison. At the same time, this formulation enjoys stronger theoretical properties (does not requires Ass. 5) and is easier to implement (T nonlinear constraints (6h)–(6i) are replaced by one constraint (42)). Regarding the terminal cost V , we can see that a properly designed terminal cost V (Corollary 2, Remark 5) allows 0 us to achieve a better performance with a shorter horizon 0 10 20 30 40 50 N compared to a simple terminal equality constraint (TEC, Lemma 4). However, the difference in performance between the terminal cost V in Corollary 2 and Remark 5 seems rather Fig. 8. Average performance improvement relative to optimal steady-state x s f for T = 1 (top), T = 10 (middle) and T = 20 (bottom) using the modiﬁed small, while the design in Corollary 2 requires T ·n additional cost V . Approach with economic terminal cost (red, solid, QINF); positive nonlinear constraints (36), which does not seem appropriate deﬁnite terminal cost (blue, dashed, QINF-pdf); terminal equality constraint for T >> 1 given the small performance improvement. (black, dotted, TEC) with prediction horizon N ∈ [1, 50]. Performance at the optimal T -periodic orbit is show in green, dashed for comparison. Given this comparison, the most suitable formulation seems to be the positive deﬁnite terminal cost V from Remark 5 combined with the modiﬁed cost V in Lemma 5, although with a ﬁxed periodic trajectory r (assuming convergence the simple terminal equality constraint may also be appropriate of r and considering the special case of periodic operation if the design in Alg. 1 is not applicable. In addition, further in [18]). We have seen that the proposed fully economic investigations into continuous-time formulations for dynamic formulation yields a better performance than operating the operations along the lines of Remark 2 may yield signiﬁcant system at the optimal steady-state (T = 1) or optimal periodic performance improvements. orbit (T > 1). In particular, this implies that the proposed framework utilizing periodic orbits (T > 1) and additional Transient performance under online changing conditions predictions (N ≥ 1), both with the economic stage cost ℓ, outperforms periodicity constrained formulations [21], [9] In the following, we provide additional details regarding (N = 0), steady-state formulations [26], [27], [28], [29] the transient performance comparison. The tracking MPC uses (T = 1) and periodic tracking formulations [8], [23], [24], [25] the quadratic part of the economic terminal cost from Prop. 4 (ℓ positive deﬁnite), even in case of ﬁxed optimal operation V (x, r) = kx− x k (similar to [37]) and the quadratic f,tr r P (r) 23 2 2 tracking cost ℓ (x, u, r) = kx − x k + ku − u k with tr r r Q R Q = 0.05, R = 1. First, we consider T = 20 and N = 10 as in Figure 4. In Figure 10, we see the closed-loop cost ℓ in the transition periods t ∈ [182, 220] and t ∈ [398, 440] (y → 1), for the proposed approach, the periodicity-constraint MPC [21], [9] and the tracking MPC [8], [25]. We can see that all approaches quickly converge to the optimal steady-state. Crucially, in the transient period, while the system converges to the optimal steady-state, the proposed approach and the tracking MPC [8], [25] seem to be able to utilize the transition period to further minimize the stage cost ℓ, while the periodicity constraint MPC [21], [9] does not seem to have the necessary degrees of freedom. The periodicity constraint MPC cannot really take advantage of the transient changes yielding faster convergence but overall small improvement (only 0.9%) compared to steady-state. For comparison, the proposed approach and the tracking MPC [8], [25] can improve overall performance by 3.4% and 2.9%, respectively. Thus, the proposed approach achieves better performance than the periodicity constraint MPC [21], [9] and the tracking MPC [8], [25], even in case of online changing operation conditions (y ) by utilizing a purely economic formulation with additional degrees of freedom. -0.084 -0.086 -0.088 -0.09 -0.092 -0.094 -0.096 190 200 210 220 -0.055 -0.06 -0.065 -0.07 -0.075 -0.08 -0.085 -0.09 400 410 420 430 440 Fig. 10. Transient performance in dynamic operation under online changing conditions for CSTR: Proposed approach (blue, solid), tracking MPC [8], [25] (red, dashed) with T = 20 and N = 10 and periodicity constraint MPC [21], [9] (green, dotted) with T = 20 at t ∈ [182, 220] (top) and t ∈ [398, 440] (bottom).

Journal

Mathematics – arXiv (Cornell University)

Published: May 11, 2020

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Periodic optimal control of nonlinear constrained systems using economic model predictive control

Periodic optimal control of nonlinear constrained systems using economic model predictive control

Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Periodic optimal control of nonlinear constrained systems using economic model predictive control

Periodic optimal control of nonlinear constrained systems using economic model predictive control

References (48)

Abstract

Journal

Recommended Articles

There are no references for this article.

Our policy towards the use of cookies