AN AXIOMATIC AND NON-COOPERATIVE APPROACH TO THE MULTI-STEP SHAPLEY VALUE

Inspired by the two-step Shapley value, in this paper we introduce and axiomatize the multi-step Shapley value for cooperative games with levels structures. Moreover, we design a multistep bidding mechanism, which implements the value strategically in subgame perfect Nash equilibrium for superadditve games. Mathematics Subject Classification. 91A10, 91A12. Received September 7, 2020. Accepted April 30, 2021.

In this paper, we develop a distributive analysis of cooperative surplus among players when they have organized themselves into a levels structure before realizing cooperation. We define a new solution concept, called a multi-step Shapley value, for cooperative games with levels structures. It can be interpreted as the result of bargaining of the lower-level unions on the worth of the higher-level unions they join. Analogous to the various axiomatizations [10,13,19] of the Shapley value, we provide several axiomatizations of the multi-step Shapley value, too.
After a solution concept for cooperative games has been defined, it seems natural to ask question whether there exists a non-cooperative framework to give rise to this value. Pérez-Castrillo and Wettstein [12] provided a non-cooperative bidding mechanism and showed that the subgame perfect Nash equilibrium (SP N E) outcomes of this mechanism always coincide with the vector of the Shapley value payoffs for zero-monotonic cooperative games.
Later, some variants of the bidding mechanism emerge for implementing other solution concepts. Ju et al. [5] designed a two-level bidding mechanism to implement the consensus value [5]. Ju and Wettstein [6] provided a non-cooperative foundation to several cooperative solution concepts, such as the Shapley value and consensus value, by using a class of bidding mechanisms that differ in the power awarded to the proposer chosen through a bidding process. Van den Brink et al. [15] incorporated the possibility of breakdown of the negotiations into the bidding mechanism in Pérez-Castrillo and Wettstein [12] and showed that the α-egalitarian Shapley value defined by Joosten [4] is implemented as the equilibrium payoff. Van den Brink and Funaki [14] introduced discounting in the bidding mechanism and the modified bidding mechanism yields the corresponding discounted Shapley value in Joosten [4] as the payoff distribution in every subgame perfect Nash equilibrium. Vidal-Puga and Bergantiños [16] gave a coalitional bidding mechanism which has two rounds and implements the Owen value. Vidal-Puga [17] proposed a levels bidding mechanism which is played in several rounds and implements the levels structure value. Based on these research, we proceed by establishing a multi-step bidding mechanism such that players behave strategically and obtain as the final outcome the multi-step Shapley value in every subgame perfect Nash equilibrium for superadditive games. In some respects, we provide a new non-cooperative approach to the two-step Shapley value different to Kamijo [8] as well, since the multi-step Shapley value is equal to the two-step Shapley value associated with the induced coalition structure when the levels structure has the degree 2.
This paper is organized as follows. Section 2 is the basic information about cooperative games and levels structures. In Section 3.1, we define a multi-step Shapley value and give its axiomatizations. In Section 4, we give a multi-step bidding mechanism which implements the multi-step Shapley value for superadditive game. Section 5 concludes.

Cooperative games
A cooperative game can be described by a pair (N, v), where N = {1, 2, . . . . , n} is the set of players and v : 2 N → R with v(∅) = 0 is the corresponding characteristic function. Any subset S ⊆ N is called a coalition and v(S) the worth of S. If there is no ambiguity, we identify the game (N, v) with its characteristic function v. The set of all cooperative games over N is denoted by G(N ). We use the notation | · | to represent the number of members in a set.
The restriction of a game (N, v) to a coalition S ⊆ N , denoted by ( A value for cooperative game (N, v) is a mapping φ : G n → R n . The famous Shapley value for cooperative games is defined as for any i ∈ N .

Levels structures
Aumann and Dreze [1] introduced a simplest model of limited cooperation, coalition structure, which is specified by a partition of the player set.
A coalition structure on N is a partition Kamijo [7] defined a solution, called the two-step Shapley value for cooperative games with coalition structures.
The two-step Shapley value allocates the cooperative surplus by using the Shapley value in a two-step bargaining process: one step within coalitions and one between coalitions. Moreover, the bargaining surplus, i.e., Sh k (M, v M ) − v(B k ) of each coalition B k is allocated among its members in an egalitarian way.
Winter [18] took into account the cooperation between coalitions and extends games with coalition structure to the so called games with levels structure of cooperation. The levels structure means a sequence of coalition structures, each obtained from the previous by unification of coalitions and each representing the various cooperations between coalitions of the previous coalition structure. Each coalition structure is called a "level".
A levels structure over N of degree h is a sequence of partitions In this paper, we only consider the case that the grand coalition will eventually form, i.e., B h = {N }. The set of levels structures on N is denoted by LS N . The levels structure B has h + 1 levels. B k is called the kth level of B and each S ∈ B k is a union at level k.
The restriction of B to a union U ∈ B k at level k is a levels structure B(U ) on the player set U with B(U ) = (B 1 (U ), B 2 (U ), . . . , B k (U )), where B l (U ), ∀l ∈ {1, 2, . . . , k} is the restriction to U of coalition structure B l . For any T ∈ B k and S ⊆ T , we use the notation B k S to represent the union at level k that contains the subset S, i.e., B k S = T . When S = {i}, we omit the bracket in B k {i} , i.e., B k i is the union at level k that includes player i.
We call the elements in a partition B of N that constitute a coalition by acting as bargaining units the direct members of this coalition, i.e., for any S = We call B on N a trivial levels structure, denoted by N , if h = 1.
A triple (N, v, B), where (N, v) ∈ G N and B is a levels structure on N , represents a cooperative game with levels structure, which describes the following cooperative situation. The players first organize themselves into a coalition structure B 1 (the first level of B) as pressure groups for the division of v(N ); then the coalitions in B 1 as players organize themselves again into the coalition structure B 2 (the second level of B); and so until the final level of B is reached. The set of cooperative games with levels structures on N is denoted by LG N .  A solution on N for cooperative games with levels structures is a real-valued function Ψ : LG N → R n which assigns to any cooperative game with levels structure an n-dimensional payoff vector.
For each k ∈ {0, 1, . . . , h}, we define a game with levels structure (B k , v/B k , B/B k ) on B k , induced from (N, v, B) by viewing unions at level k as individual players. Indeed, B k is the set of players at level k in the k-level game (B k , v/B k ). The worth of a coalition of players at level k in the game (B k , v/B k ) is defined in a natural way, as the worth of the subset of all original players that it contains. Formally, the worth of the subset

The multi-step Shapley value and its axiomatizations
In this section we extend the two-step Shapley value [7] to cooperative games with levels structures and define the multi-step Shapley value. Without special instructions, the levels structure B we mentiond in the following context has the degree h, denoted as B = (B 0 , B 1 , . . . , B h ).
We can interpret the multi-step Shapley value by the following multi-step bargaining process.
In the first step, all the players forming the union S 1 ∈ B 1 at the first level bargain for the worth of S 1 and agree that v(S 1 ) is distributed by the Shapley value of the game restricted to the subset S 1 . So, each player i ∈ S 1 receives Sh i (S 1 , v) in this step.
In the step k (k ∈ {2, 3, . . . , h − 1}), all the unions at level k − 1 act like single players. They agree that the worth of each formed union S k ∈ B k at level k is distributed among its direct members by the Shapley value, i.e., each member is allocated among all the original players in S k−1 in a way that each union at a certain level is treated equally, i.e., each player gets the proportion Finally, the sum of the payments received by each player over all steps is its multi-step Shapley value.
Remark 3.2. In particular, when the degree of the levels structure B on N is 2, i.e., B = (B 0 , B 1 , B 2 ), B corresponds to the coalition structure B 1 . The difference is that in the model of coalition structure [11], B 1 is a partition that divides the player set N into disjoint groups, which can cooperate, but in the levels structure, the cooperation between these disjoint groups needs to be described by the last level B 2 = {N }. Hence, the multi- , coincides with the two-step Shapley value of the game In the following proposition, we state that the sum of the multi-step Shapley values of players in a union at a level is the payoff of this union when we treat it as a unit. holds.
On the other hand, where the first equality holds due to Definition 3.1. Therefore, Now we present several properties for a solution Ψ on LG N as below. Efficiency.
is the restriction to the subset N k−1 (S) of players in the game of level k − 1, N k−1 (S) is the trivial levels structure on the set N k−1 (S) and N k−1 (S)\{T 1 } (N k−1 (S)\{T 2 }) is the trivial levels structure on the set N k−1 (S) after deleting T 1 (T 2 ) for all (N, v, B) ∈ LG N . In the following context, we will write ( This property is a generalization of the coalitional balanced contributions defined by Calvo and Gutiérrez [2]. It says that for each union at a level, the amount that a direct member would obtain or lose from the withdrawal of another direct member is equal.
Internal surplus equality. Let k ∈ {1, 2, . . . , h} and S ∈ B k . For any This property states that the direct members of a union at a level must have an equal share of the surplus earned by the union as a unit.
Level strong monotonicity in unions. Let k ∈ {1, . . . , h} and T be a direct member of S ∈ B k with respect This property states that when the direct players of a union at some level bargain for its worth, the division of the direct member whose marginal contribution increases should not be decreased.
Internal symmetry in unions. Let k ∈ {1, 2, . . . , h} and S ∈ B k . For any This property says that the symmetric direct members of a union at any level receive equal payoffs.
Level null coalition property. Let k ∈ {1, 2, . . . , h} and S ∈ B k . For some This property claims that the direct member of a union at a level, who has no contribution to the worth of this union, receives zero from this union.
The theorem below gives a characterization of the multi-step Shapley value for cooperative games with levels structures. Efficiency. According to Definition 3.1 and Proposition 3.3, we have Internal surplus equality. Let k ∈ {1, 2, . . . , h} and S ∈ B k . We have for any T ∈ N k−1 (S) by the proof of Proposition 3.3. Therefore, for any S 1 , where the first equality holds because of Equation (3.1) and the second equality holds by Definition 3.1. Next, we prove the uniqueness of the allocation rule Ψ satisfying the efficiency, level balanced contributions, and internal surplus equality. Define |N k−1 (S)| = l, we first prove that Ψ (N k−1 (S), v, B) = Sh(N k−1 (S), v) by induction on l.
If l = 1, we can easily verify that Suppose |N k−1 (S)| = l + 1. It follows from level balanced contributions that Using this equation and efficiency iteratively, we obtain that Ψ (N k−1 (S), v, B) is unique and equal to Sh(N k−1 (S), v, B). Further, in the case of k = h, using internal surplus equality and efficiency iteratively, we have that i∈S Ψ i (N, v, B) for each S ∈ B h−1 is unique. Continue this operation from k = h to k = 1, we can finally prove that Ψ (N, v, B) is unique.
Therefore, the multi-step Shapley value is the unique allocation rule for cooperative games with levels structures satisfying efficiency, level balanced contributions, and internal surplus equality.
By replacing level balanced contributions with level strong monotonicity in unions and internal symmetry in unions in Theorem 3.4, we derive another characterization of the multi-step Shapley value. Internal symmetry in unions. Let k ∈ {1, 2, . . . , h}, S ∈ B k and S 1 , S 2 ∈ N k−1 (S). On one hand, because of symmetry of the Shapley value [13], if for all On the other hand, by Proposition 3.3,  Further, we can adopt the same method in Theorem 1 using internal surplus equality and efficiency repeatedly to show that Ψ (N, v, B) is unique and equal to M S (N, v, B).

So by level strong monotonicity and induction it follows that
Therefore, the multi-step Shapley value is the unique allocation rule satisfying the efficiency, internal surplus equality, level strong monotonicity in unions and internal symmetry in unions for cooperative games with levels structures.
If additivity and level null coalition property in Theorem 3.5 are substituted by level strong monotonicity in unions, we have the following theorem.
Theorem 3.6. The multi-step Shapley value is the unique allocation rule for cooperative games with levels structures satisfying efficiency, internal surplus equality, internal symmetry in unions, additivity, and level null coalition property.
Proof. Efficiency, internal surplus equality and internal symmetry in unions of the multi-step Shapley value have been proven in Theorem 3.4. Additivity can be easily observed from Definition 3.1. Level null coalition property is obtained by the fact of M S(N k−1 (S), v, B) = Sh(N k−1 (S), v) for any S ∈ B k , k ∈ {1, 2, . . . , h} and null player property (see Shapely [13]) of the Shapley value.
Given an allocation rule Ψ for (N, v, B) that satisfies the properties in this theorem. It is not difficult to prove that for any S ∈ B k , k ∈ {1, 2, . . . , h}, Ψ (N k−1 (S), v, B) = Sh(N k−1 (S), v) holds from efficiency, internal symmetry in unions, additivity and level null coalition property by the aid of the axiomatization of the Shapley value in Shapley [13], and we omit the proof. Thus, applying the internal surplus equality for all S ∈ B k , k ∈ {1, 2, . . . , h}, and efficiency, we can derive that Ψ (N, v, B) is equal to the multi-step Shapley value.
Therefore, the multi-step Shapley value is the unique allocation rule that satisfies efficiency, internal surplus equality, internal symmetry in unions, additivity and level null coalition property for cooperative games with levels structures.

A non-cooperative approach the multi-step Shapley value
In this section, we design a non-cooperative multi-step bidding mechanism that implements the multi-step Shapley value for games with levels structures, inspired by the bidding mechanism of Pérez-Castrillo and Wettstein [12], and the levels bidding mechanism of Vidal-Puga [17].
Our mechanism is played in several rounds and defined recursively. Once we know the rules of multi-step bidding mechanism when the levels structure has at most degree h, we know define the mechanism for levels structure with degree h. In the first round, the players in each union at the first level simultaneously bargain for the resources of their own coalition. Let B p ∈ B 1 . Each player i ∈ B p announces a bid b i j ∈ R for any j ∈ B p \{i}. The bid b i j can be seen as the amount that player i is willing to pay to player j as the cost of being elected as the proposer. The net bid of a player is the difference between what it gives to others and what it receives from others. The player, say α p , in B p who makes the highest net bid will be chosen as the proposer. If several players make the highest net bid, the proposer is chosen randomly among them. The proposer α p pays the promised bids to players in B p \{i}. Then, the proposer α p makes an offer y αp to the other players in B p . If all the players in B p \{α p } accept the offer, the proposer α p becomes a representative of the coalition B p , i.e., α p goes to Round 2 with all the resources of B p . If at least one player in B p \{α p } rejects the offer, the proposer α p leaves the game and other players in B p \α p continue to play the game (B p \{α p }, v, B p \{α p }) (If there is no confusion, in the following context, we abbreviate (B p \{α p }, v, B p \{α p }) as (B p \{α p }, v, B)). When there is only one player in B p , the player becomes the representative of itself.
After finishing Round 1, denote by R the set of the remaining players after removing all the players in the unions at the first level that do not have their own representatives.
Let U be a maximal coalition that can be formed by the players in R, i.e., U ∈ B k for some k ∈ {1, 2, . . . , h} is a subset of R and there exists no U ∈ B m for some m ∈ {1, 2, . . . , h} such that U U , since the formation of a coalition requires the participation of all its members.
In the second round, all the coalitions in N 1 (U ) act as individual players and play (B 1 (U ), v/B 1 , B(U )/B 1 (U )) following the same procedure as before. Proceed this until in some round, there is only one player who plays alone and obtains its individual worth. Now we formally describe the multi-step bidding mechanism and proceed by the induction on the degree h of levels structure B as follows.
If there is only one player, then this player obtains the worth of its stand-alone coalition.
If h = 1, the players play a single round. This round comprises the bidding mechanism (Pérez-Castrillo and Wettstein [12]) associated with the game (N, v).
Suppose that we know the multi-step bidding mechanism when the degree of B is less than h (h > 1). For the levels structure with degree h, the multi-step bidding mechanism proceeds as follows: Stage 1. Each player i ∈ B p makes bids b i j ∈ R for each j ∈ B p \{i}. Hence, at this stage, a strategy for player i ∈ B p is a vector b i ∈ R |Bp|−1 . Calculate the net bid for each i ∈ B p . Let α p be the player with the highest net bid among players in B p (α p is randomly chosen in case of a non-unique maximizer). Then α p becomes the proposer in the next stage.
Stage 2. The proposer α p makes an offer y αp i to each player i ∈ B p \{α p }. Therefore, at this stage a strategy for α p is a vector y αp ∈ R |Bp|−1 .
Stage 3. The players of B p \{α p } either accept or reject the offer. If at least one player rejects it, then we call the offer rejected. Otherwise, we call the offer accepted.
If the offer is rejected, player α p pays b Round 2 through h. The representatives involved in the coalition U * play the multi-step bidding mechanism associated with where the representative α p of B p ∈ B 1 (U * ) takes the role of B p . These rounds are well defined by induction on h. Denote by p 2 αp ,. . . , p h αp , the outcome of the representative α p in these rounds. The final payoff of the proposer α p is the sum of its obtained payoffs over all rounds, i.e., p 1 αp + p 2 αp + p h αp . In order to characterize the SP N E outcomes of the multi-step bidding mechanism, the following result will be helpful.  (N, v, B), Proof. By efficiency of the multi-step Shapely value, we have Then we only need to prove that Since superadditivity of (N, v) implies the superadditivity of the game (N k−1 (S), v) for any S ∈ B k , k ∈ {1, 2, . . . , h}, and the Shapley value of a superadditive game satisfies individual rationality, we have that for any Applying superadditivity of (N, v), we obtain that In order to implement the multi-step Shapley value, throughout the rest of this section, inspired by Moldovanu and Winter [9] and Hart and Mas-Colell [3], we give the following two assumptions.
(a) Each player prefers to be a member of a larger coalition.
(b) A player will choose to accept if he is indifferent to accepting or rejecting a offer.
Theorem 4.2. The multi-step bidding mechanism implements the multi-step Shapley value in every SP N E for superadditive games.
Proof. The proof relies on induction on the degree of the levels structure. When h = 1, the multi-step Shapley value is equal to the Shapley value and the multi-step bidding mechanism is reduced to the bidding mechanism of Pérez-Castrillo and Wettstein [12]. Thus, the multi-step bidding mechanism implements the multi-step Shapley value in SP N E by the fact that the bidding mechanism implements the Shapley value for zero-monotonic games by Theorem 1 in Pérez-Castrillo and Wettstein [12]. Assume the result is true when the levels structure has the degree h − 1 (h > 1) or less, we now prove that the result is still true when the degree is h. First, we prove that the multi-step Shapley value is indeed an SP N E outcome. We explicitly construct an SP N E that yields the multi-step Shapley value. Consider the following strategies: for every j ∈ B p \{i}.

Stage 2.
A proposer, player α p , offers to every j ∈ B p \{α p }. Now, we prove that these strategies yield the multi-step Shapley value. Note that the offers of α p for any B p ∈ B 1 in Round 1 are accepted by players in B p \{α p }, then the payoff of i ∈ B p \{α p } is (N, v, B).
Next, we prove that these strategies are an SP N E. It has been easily known that these strategies yields an SP N E by induction on h after Round 2. Therefore, we only need to prove that these strategies induce an SP N E in Round 1. Let B p ∈ B 1 .
Stage 3. Assume that some player i ∈ B p \{α p } rejects the offer of the proposer α p . The game (B p \{α p }, v, B) is played and the outcome is by the induction hypothesis on h because the degree of B p \{α p } is less than h. Hence, it is optimal for player i to accept any offer not less than Stage 2. If α p makes an offer to some i ∈ B p \{α p } less than M S i (B p \{α p }, v, B), the offer will be rejected and therefore, α p finally obtains By Proposition 4.1, this payoff is not greater than M S αp (N, v, B), saying that the proposer α p can not improve. If α p offers at least M S i (B p \{α p }, v, B) to any player i ∈ B p \{α p }, the offer will be accepted and α p gets at most M S i (B p \{α p }, v, B). Stage 1. We first get that B i = 0 for any i ∈ B p since where the second equality holds due to Definition 3.1 and the last equality holds because the fact that Shapley value satisfies the balanced contributions property in Myerson [10]. Assume that player i ∈ B p makes different bids and B * i is its net bid. If B * i > 0, player i becomes a proposer, but he must pay extra B * i to other players of B p \{i}, which leads to its final payoff lower than M S i (N, v, B). If B * i ≤ 0, player i cannot improve. Therefore, the multi-step Shapley value is indeed an SP N E outcome. Next, we prove that the outcome in any SP N E of the multi-step bidding mechanism is equal to the multi-step Shapley value. Let B p ∈ B 1 . because of the assumption (a). Therefore, the Claim 1 holds Claim 2. In any SP N E outcome, the offer of the proposer α p will be accepted by all other players in B p .
Assume that the offer y αp of the proposer α p is rejected by some player in B p \{α p }. In this case, α p will Now assume the proposer proposes the offer to players in B p \{α p }, the offer z αp will be accepted by all players in B p \{α p } by Claim 1 and then the proposer α p will obtain the payment According to Proposition 4.1, we know that w ≤w. If w <w, it says that to offer a rejected offer is not an optimal strategy for the proposer. If w =w, it says that the proposer is indifferent to offering y αp or z αp and then by the assumption (b), the proposer α p will prefer to offer z αp . Therefore, each player in B p \{α p } will accept the offer of the proposer α p in every SP N E outcome. Claim 3. In any SP N E, the offer from the proposer α p to each i ∈ B p \{α p } is According to Claim 2, the proposer α p ∈ B p will give the offer to player i ∈ B p \{α p }. If for some j ∈ B p \{α p }, y αp j > M S j (B p \{α p }, v, B), that must induce a lower payoff for the proposer because he has to pay more to others. Supposing, to the contrary, that Ω = B p , it implies that B i > 0 for some i ∈ Ω. Take some player j ∈ B p \Ω and B j = max k∈Bp\Ω B k . Let player i change its bids by announcingb i k = b i k + δ for any k ∈ Ω\{i},b i j = b i j − |Ω|δ, andb i k = b i k for any k ∈ B p \(Ω ∪ {j}), where δ is a positive real number. Then, the new net bids are: for all k ∈ Ω,B k = B k − δ,B j = B j + |Ω|δ, and for all l ∈ B p \(Ω ∪ {j}),B l = B l .
Since B j < B i , we can find a δ > 0 which is small enough such that B j + |Ω|δ < B i − δ. Further, Ω does not change becauseΩ = {k ∈ B p :B k = max l∈BpB l } = Ω. Each player in Ω has the equal probability to be chosen as the proposer. So, by making the new bidb i , if player i is not the proposer, then his payoff does not change and if player i is the proposer, then he will receive δ units more than making the bid b i . Claim 5. In Round 1 of any SP N E, the payoff of each player i ∈ B p is the same regardless of who is chosen as the proposer.
By Claim 4, we know that B i = 0 for all i ∈ B p in any SP N E. However, in order (not) to be a proposer, player i have to slightly increase (decrease) its bids to others, which is impossible in an SP N E. Claim 6. In any SP N E outcome, the final payment of each player coincides with its multi-step Shapley value.
By Claims 1-3, we know that if |B p | = 1, each player i ∈ B p obtains the final payoff M S B 1 i (B 1 , v, B) = M S i (N, v, B). If |B p | > 1 and player i ∈ B p is the proposer in Round 1, it will obtain the final payoff

Conclusions
In this paper, we first define and axiomatize multi-step Shapley values for cooperative games with levels structures by extending the two-step Shapley value for cooperative games with coalition structures. Second, we develop a non-cooperative mechanism to achieve the multi-step Shapley values for superadditive games, which is different from the implementation of the two-step Shapley value in Kamijo [8] when the levels structure has the degree 2.