HIERARCHICAL MULTILEVEL OPTIMIZATION WITH MULTIPLE-LEADERS MULTIPLE-FOLLOWERS SETTING AND NONSEPARABLE OBJECTIVES

Hierarchical multilevel multi-leader multi-follower problems are non-cooperative decision problems in which multiple decision-makers of equal status in the upper-level and multiple decisionmakers of equal status are involved at each of the lower-levels of the hierarchy. Much of solution methods proposed so far on the topic are either model specific which may work only for a particular sub-class of problems or are based on some strong assumptions and only for two level cases. In this paper, we have considered hierarchical multilevel multi-leader multi-follower problems in which the objective functions contain separable and non-separable terms (but the non-separable terms can be written as a factor of two functions, a function which depends on other level decision variables and a function which is common to all objectives across the same level) and shared constraint. We have proposed a solution algorithm to such problems by equivalent reformulation as a hierarchical multilevel problem involving single decision maker at all levels of the hierarchy. Then, we applied a multi-parametric algorithm to solve the resulting single leader single followers problem. Mathematics Subject Classification. 91A65, 91A06, 91A10, 90C26. Received November 1, 2020. Accepted September 18, 2021.

efficient algorithms. The main objective of this paper is to propose a solution approach for a more general class of multilevel MLMF problems.
The remainder of the paper is organized into six sections. The next section presents the mathematical formulation of general multilevel-MLMF games. Section 3 describes the proposed equivalent reformulations of bilevel-MLMF problems into a hierarchical bilevel problem involving a single decision maker at each level of the hierarchy. In Section 4, a multi-parametric programming problems are described. Section 5 presents an algorithm to solve multilevel-MLMF problems using equivalent reformulation and multi-parametric approach. Numerical examples are provided in Section 6 to illustrate the procedures of the algorithms and the paper ends with concluding remarks in Section 7.

General formulations of multilevel-MLMF games
Consider a -level hierarchical multi-leader multi-follower game involving 1 decision makers at the first-level, 2 decision makers at the second-level, . . . , and decision makers at the ℎ -level. Under the assumptions that (i) there is a shared constraint common to all problems at the same level, (ii) the reaction of all followers are consistent across all the leaders at all hierarchical levels, and (iii) leaders and followers at all levels have their own decision variables, objective functions and constraints, the problem can be formulated mathematically as follows.
Continuing the same way as above, finally, at th-level, = ( 1 , . . . , ) is a decision vector for the ℎ -level problem and ∀ ∈ {1, 2, . . . , }, ∈ is a decision vector of the th-player at the th-level problem and − = ( 1 , . . . , −1 , +1 , . . . , ) is a vector of the decision variables of all th-level players without the vector ; and = ( ; − ). The shared constraint ℎ is common to all th-level players whereas, the constraint is assumed to affect only the th-player of the th-level problem.
In problem (2.1), if = 2 the problem is called a bilevel-MLMF game. As a bilevel problem, all leaders in the upper-level compete with each other in a non-cooperative Nash game and make their decisions first by anticipating the responses of all the followers. Upon receipt of the leaders' decisions, all followers compete with each other in a parametric non-cooperative Nash game in the lower level with the strategies of leaders as exogenous parameters [10]. This makes the problem of solving a multilevel problem having multiple decision makers at each decision level quite difficult and complex, particularly when there is information exchange between followers. In many applications, for instance in supply chain management models, however there are more than two decision agents, like manufacturers, distributers, retailers (or vendors), etc. If we follow the traditional approach, there will appear non-convex terms and shared variables across the followers at the ( − 1)th-level due to complementarity conditions from the th-level followers. This may result in challenging task in the process of solving multilevel-MLMF problem having shared resources and information.
At each levels of decision hierarchy in multilevel-MLMF problems, one need to solve parametric generalized Nash Equilibrium problems, where the variables from upper levels are considered as parameters. It is well known that solving such problems can be a tedious and error-prone task [23]. Many of the solution approaches for solving problems with MLMF nature apply a reformulation of the Nash equilibrium problem and Stackelberg equilibrium problem as an 'equivalent' variational inequality (VI) problems and mathematical problems with equilibrium constraints (MPECs), respectively. But such reformulations have several limitations as mentioned in [16]. In addition, a standard approach in MPECs requires ascertaining when the reaction map admits fixed points. But this is difficult due to the lack of continuity in the solution set associated with the equilibrium constraints capturing the follower equilibrium. Due to these challenges most of the solution methods work only for particular subclasses of multilevel-MLMF games which satisfy some strict conditions (such as, strict convexity, separability, etc.) for two levels. In the next section, we present a reformulation of a class of multiple leaders and multiple followers problems into an equivalent bilevel problem involving only a single decision maker at both levels of the hierarchy, which can be extended later to any finite hierarchical levels.

Equivalent reformulation of bilevel-MLMF games
Consider the following bilevel-MLMF game in which leaders compete in a non-cooperative game subject to the equilibrium conditions of followers competing in a lower-level game given leaders' level decisions. If we denote , ∈ {1, . . . , }, the decision variables vector for leader and , ∈ {1, . . . , }, the decision variables for follower . The leaders' and followers' variables are abbreviated, respectively by = ( 1 , . . . , ) and = ( 1 , . . . , ). For each of the leaders, = 1, . . . , , the Stackelberg game played by leader is given by: ( , ) ≤ 0, and for all = 1, . . . , , Let us assume that, for all and for each , the functions , , , ℎ, , in (3.1) are twice continuously differentiable, and that the followers' constraint functions satisfy the Guignard constraint qualifications conditions [7]. Let us define the following sets that can be used in the preceding sections to characterize solutions of the problem (3.1).
(i) The feasible set of problem (3.1) is given by (ii) For any given leaders' strategy , the feasible set for the ℎ -follower is defined as (iii) The Nash rational reaction set for the th-follower, is defined by, (iv) The feasible set for the th-leader, is defined as (v) The Nash rational reaction set for the th-leader is given by (vi) The set of Stackelberg-Nash equilibrium points of problem (3.1) is given by Using the reaction sets, the leaders Nash problem of (3.1) is given by subject to ( , ) ∈ ℛ ( − ). Given a strategy of the leaders, each of the followers play the Nash game: subject to ∈ ℛ ( , − ). When the Nash problem (3.2) has objective functions with separable terms and a non-separable term which is common to all leaders (i.e., for each if the objective functions are written as ( , − , ) =ˆ( ) +ˇ( − ) + ( , )), by defining a quasi-potential function Kulkarni and Shanbhag [15] equivalently reformulated problem (3.2) as a single optimization problem And they have shown that, the global minimizers of problem (3.4) are global equilibria of the problem (3.2). In general, it has been shown in [15] that if the objectives of the leaders and the followers admit a quasi-potential function formulation, then the global minimizers of this quasi-potential function problem are global equilibria of the MLMF game. The quasi-potential function problem of Kulkarni and Shanbhag considered in [15] covers only a class of games whose objective functions are written as separable terms and a non-separable term which is common to all decision makers at that level. In this work we have considered a game in which the objective functions contain a separable and non-separable terms; but the non-separable terms can be written as a factor of two functions where the first one is a function of other level decision variables and the second factor is common to all objectives across the same level.
We make the following assumptions on the structure of the objective functions of the multi-leader multifollower game that will enable us to reformulate a bilevel-MLMF game as a hierarchical bilevel game involving a single decision maker at both levels: (A1) The leaders objective can be written as where, for any ∈ , (∀ )(0 < ( ) < ∞).  Proof. For ( , ) ∈ × and 0 < ( ) < ∞, define the function Note that, because of Assumption (A1), the function is well defined. Then, for any we have Implying that˘( Similarly, for ( , ) ∈ × and 0 < ( ) < ∞ by defining we will have˘( Hence, the conclusion of the Lemma follows.
For a given strategy , define the following optimization problem, subject to ( , ) ∈ ℛ ( − ) (∀ ). (3.5) Equivalence between optimal solution of (3.5) and equilibrium point of the Nash problem (3.2) can be established based on the following theorem.
For a given strategy , define the following optimization problem, Then the relation between (3.7) and (3.3) can be established as follows.
Note that when all the functions ( ) and ( ) are always equal to a constant 1, then problem (3.1) reduces to the quasi-potential game considered in [15]. Therefore, our result improves the one given in [15].
Remark 3.8. The idea described above can be extended to any finite -level multi-leader multi-follower game by reformulating the problems on the same level with a single objective as stated above. That is, any multilevel multi-leader multi-follower game which satisfies assumptions (A1)-(A3) can be equivalently reformulated as a multilevel single-leader single-follower problem without increasing the vertical hierarchical levels. However the resulting multilevel optimization problem requires a solution approach that is different from the traditional Karush-Kuhn-Tucker (KKT) reformulation to avoid the effect of the complementarity conditions in the middle level problems.

Multi-parametric programming formulations and methods
A multi-parametric solution approach for multilevel optimization is a global solution strategy that works by rewriting the most inner level optimization problem as a multi-parametric problem, where the upper level optimization variables are considered as parameters. The resulting problem can be solved globally and the parametric solutions can be substituted into the most nearby upper level optimization problem. The key advantage of multi-parametric programming approach is that it provides a complete map of the optimal solution in the space of the varying parameters [20].
Multi-parametric programming techniques systematically subdivide the parameter space into characteristic regions where the optimal value and an optimizer are given as explicit functions of the parameters. A typical multi-parametric nonlinear program (mp-NLP) is generally defined as [5]: where ∈ ⊆ R is the parameters vector, ∈ ⊆ R is the vector of the decision variables, and , , ℎ : R × R −→ R are parametric nonlinear functions.
The assumptions (C1)-(C4) ensure that the inverse of 0 exists and hence for problems involving convex , and ℎ, the parametric solutions within the corresponding critical regions, are necessary and sufficient. The main question here is how to find a parametric solution ( ) which remains stable in some subregion of the parameter space. We will consider this in the following two subsections.

Multi-parametric problems with linear constraints
Based on Corollary 4.2, Dua et al. [3] proposed an algorithm to solve (4.1) in the entire range of the varying parameters for general convex problems. The space of where solution (4.4) remains optimal to (4.1) is defined as the critical region, ℛ, and can be obtained by using feasibility and optimality conditions [3]. Each piecewise linear approximation is confined to regions defined by feasibility and optimality conditions. Ifˇcorresponds to the inactive polyhedral constraints andˇto the Lagrangian multipliers of the active constraints, then the critical regions can be defined as, After defining the critical region ℛ on which the parametric solution is valid, if ℛ has not covered the parametric region, we repeat again the same mathematical procedure as in above with any new feasible parameter ( = 0 ) taken from the rest of parametric regions until the parametric region has been explored successfully as Table 1. Definition of the rest regions. [13]. To define the rest of the parametric region, consider ℛ = [ , ] to be the overall parametric region (where and represent the lower and upper bounds of the parametric region) and let the inequalities, labeled by 1 ≤ 0, 2 ≤ 0, 3 ≤ 0 define ℛ. Now the rest of the parameter region ℛ Rest = ℛ − ℛ can be characterized by considering each of the inequalities which comprise ℛ 0 , reversing their signs one by one and removing redundant constraints [4]. For example, consider inequality 1 ≤ 0, the rest of the region can be addressed by reversing the sign of inequality 1 ≤ 0 and removing redundant constraints in ℛ , which is ℛ Rest  Table 1.

Region Inequalities
Finally, the optimal solution ( ) can be expressed explicitly as a conditional piecewise linear function [3]: where are column vectors and are real matrices, whereas ℛ ⊆ R are critical regions and note that ℛ denotes the th critical region. For multi-parametric linear and quadratic problems, exact solutions can be computed using the first-order estimation.
From the above arguments we can observe that whenever nonlinear convex constraints appear in the parametric problem (4.1), we apply the barrier method procedure to transform the nonlinear functional expression into the objective part and apply the procedure in Section 4.1 to obtain the required parametric solutions.

Algorithm to solve multilevel-MLMF problems using equivalent reformulation and multi-parametric approach
In this section we propose an appropriate solution approach to solve a classes of multilevel-MLMF games with a property that every objective function in the problem consists of separable terms and non-separable terms (but each of the non-separable terms can be written as a factor of two functions one of the factor being common across all players of the same level) and with non-degenerate polyhedral constraints in addition to possible convex non-linear ones. For the sake of clarity in presentation, the methodology is described using a general trilevel-MLMF game. However, the same approach can be extended to a general -level case with appropriate adjustments.

Equivalent reformulation of multilevel-MLMF games
Consider a trilevel-MLMF game involving decision makers at the first-level, decision makers at the second-level and decision makers at the third-level which can be formulated mathematically as follows. For = 1, . . . , , the vector ( 1 , 2 , 3 ) solves an optimization problem, , and for all = 1, . . . , , , and for all = 1, . . . , , Assume that each of the objective functions of the third and second level followers is convex with respect to its own decision variable vector and the Guignard constraint qualifications [7] hold for the followers constraints.
We make the following assumptions on the structure of the objective functions of (5.1): (B1) The objective functions at the first-level can be written as The objective functions at the second-level can be written as The objective functions at the third-level can be written as (B4) 1 (·), 2 (·) and 3 (·) are twice continuously differentiable functions and uniformly bounded away from zero. If (5.1) satisfies the assumptions (B1)-(B4), then it can be equivalently formulated as a trilevel optimization problem having a single decision maker at all levels as follows. The vector ( 1 , 2 , 3 ) solves an optimization problem, ( 1, 2, 3) s.t.  Then the following statement is a direct consequence of Proposition 3.7. Hence we state it here without proof. Proposition 5.1. Suppose that (5.1) satisfies the assumptions (B1)-(B4). If ( * 1 , * 2 , * 3 ) is a Stackelberg equilibrium point of (5.2), then ( * 1 , * 2 , * 3 ) is a Stackelberg-Nash equilibrium point of (5.1).
Note that the approach by Kulkarni and Shanbhag [15] leads to an MPEC for two level problems whose global solution provides an equilibrium to a bilevel-MLMF game. When we have more than two hierarchical levels, the nonconvex expression resulted from the lower-level problems makes it difficult to solve. For example, in tri-level programs the complementarity condition of the third-level KKT conditions makes the second-level problem nonconvex. Therefore, using the same transformation will result in a problem which is very difficult to solve (if it is tractable at all). To avoid this situation, we apply multi-parametric procedures for transforming the lower level problems. In this procedure, instead of embedding lower level problems into the middle level through the KKT conditions, we will equivalently transform the tri-level problem into a single-level problem by sequentially substituting the parametric solutions in problems of the middle and upper levels.

Multi-parametric based algorithm to solve multilevel-MLMF
The algorithm starts by reformulating the hierarchical multi-leader multi-follower problem as a multilevel optimization problem involving a single decision maker over the hierarchy as discussed in Sections 3 and 5.1. Then, each of the optimization problems in lower levels can be recast as multi-parametric programming problem where the variables from upper level problems are considered as parameters, and hence obtain an analytical parametric solution for the rational reaction set for each of the sub problems in the corresponding critical (stability) region of the parameter space. The basic steps of the proposed algorithm for tri-level MLMF problem (5.1) are described as follows.
Step 1. Reformulate the hierarchical multi-leader multi-follower problem (5.1) as a multilevel optimization problem as discussed in Subsection 5.1 to obtain (5.2).
Step 2. The third-level problem of (5.2) is treated as a multi-parametric problem with 3 being the optimization variable and the first-level and second-level decision variables (i.e. 1, 2) the parameters; and solved by a multi-parametric approach discussed in Section 4.

Step 3. Substitute the parametric solution from
Step 2 in the second-level optimization problem of (5.2) and solve the resulting multiparametric problem with 2 being the optimization variable and the first-level decision variable ( 1) the parameters.
Step 4. Substitute the parametric solution from Step 3 in the first-level optimization problem of (5.2) and use a standard nonlinear optimization algorithms to solve the resulting optimization problem with decision variable 1.
Remark 5.2. Since the number of partitions of the critical regions are finite as shown in [3,13], the algorithmic procedures described in Steps 2 and 3 terminate after a finite number of iterations. Hence, the above algorithm requires only a finite number of iterative procedures to arrive at the required solution. Moreover, if all the involved functions are linear or quadratically convex, the solutions obtained through the above algorithm will be an exact global solution to the original problem.

Illustrative Examples
To apply the proposed solution algorithm, it is to be noted that each of the transformed problems at the lower levels must satisfy the required four conditions (C1)-(C4). These conditions seem to be very strong and restrictive. However, many practical application problems satisfy these conditions. For example, the oligopoly market problem with divisible homogeneous products, that is described and analyzed in [11], satisfies all of the assumptions in our reformulation in Section 3. Therefore, it can be considered as a particular example of our proposed formulation.
The second example is the supply chain management problem. Supply chains are systems with multiple components such as supplier, manufacturer, distributer, retailer and customer, that exchange information with one another [2]. Mathematical models formulated to analyse the overall economic process of the supply chain management are usually described by MLMF games with at least three hierarchical levels (see for instance [9], where all the involved functions are assumed to be linear). Most of the deterministic versions of such problems satisfy the conditions required in our model formulation and solution algorithm.
Here below, we will illustrate the proposed method using selectively constructed numerical examples.
Now, comparing all the values of the objective of the leader in each of the critical regions, we can see that the objective value obtained in ℛ 2 gives a better result. Hence we take the solution in ℛ 2 as an optimal solution to the upper-problem of (6.8).

Conclusion
This work proposes a solution procedure for a class of hierarchical multi-leader multi-follower games whose objective functions at each level have non-separable terms and some polyhedral constraints. The solution procedure transforms the given problem into an equivalent multilevel hierarchical problems having single decision maker at each level of the hierarchy and without increasing the level of hierarchy in the problem. The equivalent reformulation maintains the equilibrium points of the original problem, so that we can use any of the existing methods to solve the resulting multilevel problem with one player at each level. In this article the multi-parametric approach is employed to solve the resulting multilevel optimization problem. The proposed equivalence reformulation does not require the smoothness of the involved functions. However, due to the requirements of the multi-parametric solution methods (that is used to solve multilevel hierarchical problems in this article), we additionally imposed second order smoothness conditions as well as convexity of the lower level problems. These conditions may not be necessary if one uses other methods (like the heuristic methods) to solve the resulting equivalent multilevel hierarchical optimization problems.