STRATEGIC BEHAVIOR IN A VACATION QUEUE WITH DELAYED OBSERVATIONS

We study an emerging computer network model of delayed observations in which the system is unobservable for the customers at their arrival instants, but after a while, they are informed about their current positions and they may renege. We develop a queueing-game-theoretic vacation model to explore customers’ equilibrium strategy, stationary system behavior and social welfare based on a reward-cost structure. Our main results are as follows. First, we determine a closed form of the customers’ equilibrium strategy, the expected net benefit of a customer and social welfare in the service system. Second, extensive numerical experiments that demonstrate the effect of vacation rate θ and system announcement rate δ on the equilibrium strategy and social welfare. We find that the impact of announcement rate δ is greater than vacation rate θ in some cases. Finally, we show that the equilibrium strategy can give customers more information and reduce the cost of waiting. Moreover, our results can also provide more precise information to the system administrators. Mathematics Subject Classification. 90B22, 60K25. Received August 7, 2020. Accepted July 21, 2021.


Introduction
The information asymmetry between customers and administrators is a very important element in most service systems. In many situations, it is possible for system administrators to provide some valuable information to their customers at specified moments, which can directly affect customers' behavior and reduce congestion in the system. The study of delayed observations is essential and has attracted the attention in some burgeoning computer networks in recent years, which is the arriving customers decide whether to join or balk without knowing the state of the system, but after a while, the administrator of the system announces to customers of their current positions and they may renege.
The literature on strategic customers behavior influenced by information begins with Naor [18], who analyzes an observable M/M/1 model with a simple linear reward-cost structure in which arriving customers are informed about the queue length before making their decisions on whether to join the queue or balk. After that, Edelson and Hilderbrand [9] complement this study and introduce unobservable queue cases, where customers make decisions without observing the system state. Since then, there has been a large volume of literature in the Keywords. Strategic customers, vacation queue, equilibrium analysis, delayed observations, observable queue, unobservable queue. economic and equilibrium analysis of queueing systems with customers' strategic behavior. Guo and Zipkin [12] consider the models with different levels of delay information, where the state-space of the queue length is divided into subsets and the arriving customers are not informed about the exact queue length. Koshman-Kaz and Hassin [16] deal with high-low delay announcements in a queueing system. Recently, Burnetas et al. [5] investigate a queueing system with a delayed observation structure. Yu et al. [25] do research on delay announcement and retrials in fluid queueing models. The main approaches and recent advances about various models in this direction can be summarized in bibliographical references [2,3,6,15,17,19,20]. The fundamental results on this subject can be found in the comprehensive monographs of Hassin and Haviv [14], Hassin [13] and Stidham [21].
Despite the boom of the work on delay observations in computer networks, little is known about delay observations in vacation queues, which are also quite common in both manufacturing and service systems. Recent work in vacations has also been very active, which is widely used in complex computer and communication networks. The books by Takagi [23], Tian and Zhang [24] are devoted to such systems and their applications. We can find general models in vacation queues from Burnetas and Economou [4]. They study customers' strategic behavior in a single server Markovian queue with set up times under different levels of information. Economou et al. [8] extend the model to general distributions of service and vacation lengths. Guo and Hassin [10] analyze a vacation queue with N-policy and exhaustive service, and the equilibrium and social optimal strategies are derived for both the unobservable case and the observable case. This work is extended by Guo and Hassin [11] to heterogeneous customers case. Sun et al. [22] explore equilibrium and optimal strategies in Markovian queues with multiple vacations and N-policy. A multiserver case with setup times is presented in Artalejo et al. [1].
In practice, web-based systems and electronic notification policies have already benn implemented in a large number of service systems, and customers' decisions are affected by the announcements. We are motivated by a situation occurs when some people submit their applications through university admissions platforms online, without knowing the state of the system. After submission, the customer will receive a confirmation message with a registration number, which is the number of pending applications in front of them, and they may renege. This can be done by periodically refreshing the web page and indicates the registration number of the application currently being processed. However, sometimes the web-based systems may require maintenance and there may be a cost associated with sending announcements. For example, virus attacks have an adverse effect on the service system, virus scan and system upgrades are essential and effective maintenance methods for the webbased servers. These virus scans and system upgrades can be regarded as system vacations. When the system is in the working state, the web-based server handles applications one by one. And if the system is in the vacation state, the server does not handle any applications, but to check if the system is infected with virus and upgrade the system, and the system enters the working state if the system is nonempty.
Another example in practice of the proposed model occurs when the information-providing mechanism has to respect some periodicity of the mode of operation of the systems. For example, in a health care system or a inventory management system, if the system is in the working state, administrator of the system provides announcements to customers properly. But if the system is in the vacation state, the administrator of the system intentionally stops to send announcements for economic or other reasons. This is reasonable under various scenarios, for example, the information-providing mechanism is costly.
In this paper, to study the equilibrium and social welfare on customer behavior in such systems, we aim to extend the aforementioned studies by considering a vacation queueing system with a delayed observations structure. To the best of our knowledge, there is no previous study has dealing with the equilibrium strategies and social welfare in vacation queues with delayed observations in the queueing literature. We consider a singleserver vacation queueing system, upon customers' arrival, they should decide whether to join or balk without knowing the state of the system. After a while, they are informed about the number of customers currently in the queue and they may renege. We explore the equilibrium strategy, stationary system behavior and social welfare. Furthermore, extensive numerical experiments that demonstrate the effect of several parameters on the equilibrium strategy and social welfare.
There are three lightspots in our study. First, our paper is the first to study the equilibrium strategy of customers and equilibrium social welfare with the combination of a vacation queueing system and delayed observations in the queueing literature. Second, we derive the equilibrium strategy, stationary system behavior and social welfare in our model, which fills a gap in the rich literatures on the analysis of the strategic behavior of customers in vacation queues. Finally, the study of our model may provide more precise information to system administrators and improve decision-making, which has both theoretical and practical significance.
The balance of this paper is organized as follows. In Section 2, we briefly introduce the model and the rewardcost structure. Equilibrium strategies of the customers and equilibrium social welfare are studied in Section 3. We derive the corresponding equilibrium strategies and explore the stationary behavior for the model. Section 4 presents numerical experiments and investigate how the effect of vacation rate δ and system announcement rate θ on the behavior of the system and social benefit. Finally, conclusions are made in Section 5 and future research directions are also outlined.

System model
Consider an M/M/1 service system with first-come-first-served queue discipline. Customers need for the service arise according to a Poisson process with rate λ. The service times of all customers are independent and exponentially distributed with parameter µ. The server is deactivated as soon as the queue becomes empty. When a new customer arrives at an empty system, a setup process starts for the server to be reactivated. The time required for setup is also exponentially distributed with rate θ. Each customer, upon arrival, decides whether to join or balk, without observing the number of customers in the system. However, the administrator of the system announces to the customers their positions in the system at the points of a Poisson process with rate δ. The customers, after an announcement, reevaluate their expected benefit of staying in the system and will renege if it is negative. Assuming that inter arrival times, service times, setup times and announcement processes are mutually independent.
We establish a certain reward-cost structure that quantifies the customers' desire for service and their dislike for waiting. Every customer receives a reward of R unit after service. This reward R may represent his satisfaction or the added value of the customer obtains from being served. On the other hand, there also exists a waiting cost of C per unit time when the customers remain in the system. At the epoch of the first announcement following his entrance, the system becomes observable to him. Thus, the customer is willing to stay of his position n in the system is such that R − C n µ ≥ 0. Otherwise, they will leave the system. It can be shown that customers who wish to maximize their individual welfare will follow a pure threshold strategy [7]. This means that there exists an integer n e such that newly-arrived customers will join the queue if and only if the number of other customers already present in the system is smaller than n e , where n e = Rµ C . We assume that customers are risk-neutral, if they choose to join the queue and does not renege after the first announcement, they will not renege later. Let ρ = λq µ denote the maximum load of the queue, which q is the join probability of customers. We assume that n represents their reneging threshold at the time of the first announcement after their arrival, customers follow a strategy (n, q). Under such a strategy, we represent the state of system at time t is characterised by the pair {I(t), J(t)}, where I(t) and J(t) denote the number of customers in system and state of the server (0: the server is in vacation period; 1: the server is in busy period) respectively. It is clear that the process {I(t), J(t), t ≥ 0} is a two-dimensional continuous time Markov chain with state space indicates that the server is in the vacation period and there are i customers; State (i, 1), i ≥ 1, indicates that the server is in the busy period and there are i customers. The graphical description of this system is presented in Figure 1.
In this paper, we consider an M/M/1 queue with vacation where the administrator makes periodic announcements to customers about their current positions at announcement rate δ. We analyze the stationary behavior of the system, determine a closed form of the customers' equilibrium strategy, the expected net benefit of a customer and social welfare in the following section, which make sense in applicative contexts.

Equilibrium analysis of threshold policies
Theorem 3.1. The stationary distribution of customers in the system for the vacation queue with delayed observations, when customer follows an (n, q) strategy with n > 0, q > 0, and λq = µ, are given as follows: Proof. From Figure 1, the stationary distribution can be obtained by the following balance equations of the system: Let α = λq λq+θ . Thus α < 1. By(3.1) and (3.2), we obtain We observe that π i1 is a solution of the nonhomogeneous linear difference equation (3.4) with constant coefficients. Using the standard approach to solve such equations, see e.g. [11], we consider the corresponding characteristic equation which has two roots at 1 and ρ, then the general solution of the homogeneous version of (3.4) is where Dα i is a specific solution of (3.9). Substituting x i = Dα i into (3.9), we obtain Furthermore, substituting i = 1 and i = 2 into (3.11), we obtain Then Similarly, when i ≥ n + 1, from (3.6), we obtain We consider the corresponding characteristic equation which has a root at The general solution of (3.6) is given as It follows from (3.12) that Ex n + F α n = βπ 0,0 α n − βπ 0,0 ρ n .
(3.13) From (3.5), we have Solving the Equations (3.13) and (3.14), we obtain the expressions of E and F presented in this theorem. Then, we obtain The remaining probability π 00 can be found from the normalization equation Therefore, we have expressed all stationary probabilities in terms of π 00 in relations see (3.6), (3.12) and (3.15).
The corresponding mean number of customers in the system E(N ) is Lemma 3.2. When customers follows an (n, q) strategy, the mean number of customers in the vacation queue with delayed observations is a decreasing function of the announcement rate δ, and the throughout is a decreasing function of the announcement rate δ.
Proof. The throughout of the system is given as µ(1 − π 00 (n, q)). It can be easy proved that the number of customers in the system is stochastically decreasing in δ by using direct coupling arguments, so the stationary probability of an empty system π 00 (n, q) is increasing with respect to δ.
Let I ((i, j)|n) be the conditional expected net benefit of a customer who enters the system at the state (i, j), and all customers follow the reneging threshold n. Lemma 3.3. The conditional expected net benefit of a customer who finds the server in state (i, j) can be expressed as follows:

17)
Proof. When all customers follow an (n, q) strategy in the vacation queue with delayed observations, we will take the server in busy period as an example, and consider the following two situations. Suppose a tagged customer who enters the system at the state (i, j). When the server is free, the tagged customer will be immediately served.
When i ≤ n − 1, the customer will receive the service reward R, his sojourn time in the system will be the sum of (i + 1) service times. Therefore, we conclude with the first branch of (3.16).
When i ≥ n, the proof is similar to the one of Burnetas et al. [5] and we omit it here. Moreover, for any fixed i ≤ Rµ C + µ θ , I ((i, j)|n) is a strictly decreasing function of i.
π i1 (n, q)I(i|n). Using Theorem 3.1 and Lemma 3.2, when the other customers follow an (n, q) strategy, we can compute the expected net benefit of a customer who decides to join.
Theorem 3.4. In the vacation queue with delayed observations, when the customers follows an (n, q) strategy, the expected net benefit of a customer who decides to join is given by Proof. We have that Let β = µ µ+δ . When n, q > 0, λq = µ, by using Theorem 3.1 and Lemma 3.3 and evaluating the corresponding geometric sums yields Theorem 3.5. Consider the vacation queue with delayed observations, for n e = µR C , a unique equilibrium strategy (n e , q e ) exists.
(n e , q e ) =    (n e , 0) I(n e , 0) ≤ 0, (n e , q) I(n e , 1) < 0 < I(n e , 0), (n e , 1) I(n e , 1) ≥ 0. Proof. Supposing a tagged customer arrives at the system and decides to join, the expected net benefit of the customer is I(n, q). Consider the first branch of (3.19). When I(n e , 0) ≤ 0, the maximum benefit of the customer is a non-positive number, and the customer will not enter the system whether there are customers in the front or not. Therefore, (n e , 0) is the unique equilibrium strategy.
In the second branch of (3.19), when I(n e , 1) < 0 < I(n e , 0), there is a unique root q e ∈ (0, 1), so that I(n e , q) = 0. Because of the monotonicity and continuity of I(n e , q), thus for any (n e , q) with q ∈ [0, q e ), the best strategy of the tagged is (n e , 1). Such a strategy cannot be equilibrium. Similarly, for any (n e , q) with q ∈ [q e , 1), the best strategy of the tagged is (n e , 0). Such a strategy cannot be equilibrium, either. Therefore, only (n e , q e ) is the unique equilibrium strategy.
Considering the third branch of (3.19), when I(n e , 0) ≥ 0, the minimum benefit of the customer is a nonpositive number, and the customer will enter the system whether there are customers in the front or not. Therefore, (n e , 1) is the unique equilibrium strategy.
When customers follows an (n, q) strategy, we use the notation S(n, q) to denote the social mean net benefit per time, which is given by expected S(n, q) = λqI(n, q). The closed form of S(n, q) is too complicated, so it seems that the maximization of S(n, q) can be done only numerically.
Remark 3.6. For the partial information case, the customers are informed about the state of the server upon arrival. And a mixed strategy is specified by the joining probability of an arriving customer that finds the server on vacation or not. Suppose that all customers follow a mixed strategy, and the system behaves as the original, but with arrival rate λ i = λq(i) for states where the server is in state i instead of λ, where q(i) is the join probability of customers where the server is in state i. Based on this information state, the customer estimates the expected waiting time in the system. We also assume a linear reward-cost utility function for customers. The same problem can be solved in our unobservable vacation queue with delayed observations. We have explored the stationary behavior and the corresponding equilibrium strategies for the model. However, the function of expected net benefit of a customer is too complicated to get a closed form solution. So, for such case, we will focus on finding some new techniques to derive the equilibrium strategies and social optimal strategies, which will be reported in our forthcoming paper.

Numerical examples
Our objective is to study effects on strategic customer behavior and to derive managerial insight on optimizing system design. In this section, according to the analysis above, we present numerical studies to investigate how the effect of the several parameters on the behavior of the system. For this analysis, we consider three sets of numerical experiments. Specifically, we first present examples to show the effect of announcement rate δ on equilibrium join probability. Whereas the second set studies the effect of the expected net benefit of a customer I(n e , q) with respect to the vacation rate δ in Section 4.2. Moreover, in Section 4.3, we investigate sensitivity of equilibrium strategies and social welfare with respect to vacation rate θ and announcement rate δ.

Optimization of δ on equilibrium strategy (n e , q e )
In Section 3, we have obtained the equilibrium strategy of the customers. Here, we try to explore the effect of the announcement rate δ on the equilibrium strategies (n e , q e ). Recall that (n e , q e ) =    (n e , 0) I(n e , 0) ≤ 0, (n e , q) I(n e , 1) < 0 < I(n e , 0), (n e , 1) I(n e , 1) ≥ 0.
The monotonicity with respect to δ of (n e , q e ) under the equilibrium strategy cannot be carried out analytically. However, we have also elaborated on these issues by numerical experiments. Now that we have introduced the model and the equilibrium concept, we will move on to analyze the relationships between the unique equilibrium joining probability and the announcement rate. First, we consider the effect of δ on the expected net benefit of a tagged customer I(n e , q). An increase in δ may has two positive effects on customers. On one hand, the more frequently the information is announced, the less congestion is seems intuitive. On the other hand, the tagged customer will be notified more quickly about the number of customers in the system and will balk if his expected net benefit is a non-positive number. Therefore, we predict that the expected net benefit of a customer will increase in δ. This fact is established formally in Figures 2-4. Figures 2-4 show the effect of expected net benefit of the customer respect to the customer's joining probability q under different announcement rate δ. In each figures, we plot three curves for the announcement rate δ → 0, 0 < δ < ∞, δ → ∞, respectively, the rest of the parameters are kept fixed. Specifically, in Figure 2, we have that I(n, q) ≤ 0. Therefore, (n e , 0) is a best response against itself. In Figure 3, the monotonicity of I(n, q) with respect to the announcement rate δ implies that the root of I(n, q) = 0 is increasing in δ. So in this situation, the function q that gives the unique equilibrium join probability is an increasing function of δ, reaching q = 1 and staying at 1 after that. In Figure 4, by using Lemma 3.1, we have that I(n, q) ≥ 0. The monotonicity of I(n, q) with respect to the announcement rate δ shows that I(n e , 1) > 0, therefore for all δ > 0, q = 1 is the unique equilibrium join probability.
Moreover, we observe that when the value of δ is fixed, the expected net benefit of the customer decreases in customer's joining probability q. In Figures 2-4, on one hand, with regard to the expected net benefit of the customer, we observe that they are decreasing with respect to q. Therefore when the arrival rate increases, customers are less willing to enter the system. Therefore, as q keeps increasing, there will be congestion in the  system, which has a negative effect on the expected net benefit of the customer, and customers are less willing to enter the system. On the other hand, when taking three different values of δ, we find that the curves have the same shape and trend. The expected net benefit of the customer increases as δ increases. This indicates that the more frequently the information announcement of the system, the more expected net benefit of the customer has, so the customer will choose to enter the system.

Optimization of θ on equilibrium strategy (n e , q e )
According to the analysis above, in this subsection, we investigate the effect of expected net benefit of the customer respect to the customer's joining probability q under different vacation rate θ while keeping δ constant. We use the same analysis method as in Subsection 4.1.   Figure 5 shows that when we have I(n, q) ≤ 0, (n e , 0) is also a best response against itself. In Figure 6, the monotonicity of I(n, q) with respect to the vacation rate θ implies that the root of I(n, q) = 0 is increasing in θ. So the function q that gives the unique equilibrium join probability is an increasing function of θ. In Figure 7, we have that I(n, q) ≥ 0. The monotonicity of I(n, q) with respect to the vacation rate θ shows that I(n e , 1) > 0, therefore for all θ > 0, q = 1 is the unique equilibrium join probability.
From Figures 5-7, the explanation is based again on the previous discussion: as q increases, it influences the expected net benefit of the customer towards the same direction, which means that when the number of arriving customers increases, customers will realize that the load rate of system is relatively large and the system is congested, and they are reluctant to enter the queueing system. When taking three different values of θ, which are θ → 0, 0 < θ < ∞, θ → ∞, respectively, we find that the curves have the same shape and trend. Evidently,  this is intuitive, because when the server vacation gets shorter, customers generally have a greater incentive to enter the system.

4.3.
Optimization of θ and δ on q and social welfare S(n, q) According to Subsections 4.1 and 4.2, we find that both announcement rate δ and vacation rate θ make a great impact on the system. So, this subsection compares announcement rate δ and vacation rate θ in terms of the customer's joining probability and social welfare. In Figures 8 and 9, we depict sensitivity of equilibrium strategies and social welfare respect to vacation rate θ and announcement rate δ. More concretely, in Figure 8, the equilibrium joining probability is increasing in both δ and θ, and eventually reaches 1 for a lager value of δ, and for any value of θ. As the vacation rate θ and announcement rate δ increase, customers will know the system state quickly, so they are more likely to enter the system. Moreover, in Figure 9, we notice the different trends of the increase in social welfare. As θ and δ increase, it is very helpful for increasing equilibrium social welfare for a lager value of δ, and for any value of θ. Especially the trend of increase is more obviously at some certain moments(e.g when θ = 2, δ = 200). Those observations indicate that δ and θ are two beneficial factors to increase the social welfare. And increasing θ and δ appropriately can reduce system congestion and improve the service system revenue. An interesting finding is that δ has much more influence than θ on the customer behavior and social welfare.

Conclusions
In our research, we investigate the booming computer networks with vacation and delayed observations in a service system and explore its economic impact. We first explore customers' equilibrium strategy, stationary system behavior and social welfare in the service system. Extensive numerical experiments that demonstrate the effect of several parameters on the equilibrium strategy and social welfare. Second, our model can be seen as a continuous generalization that bridges unobservable and observable vacation systems, and customers have the right to make decisions according to the accurate situation, which is more sensible than the classical viewpoint in queueing theory. To the best of our knowledge, there is no work concerning the equilibrium strategies and social welfare in queues with vacation and delayed observations, and this is the first time that both vacation and delayed observations are introduced into the economics of queues. For the service system with vacation and delayed observations, we derive the equilibrium threshold strategies and also discussed the effect of information structure on the equilibrium thresholds and social welfare of all customers as well as the sensitivity of them with respect to other parameters. A main conclusion is to derive the influence of both announcement rate δ and vacation rate θ on strategic customer behavior and system performances. And the impact of announcement rate δ is greater than vacation rate θ in some cases.
Our results show that sufficient flexibility to control the information structure and vacation of the system improves its performance, both in terms of system management and social perspective. The equilibrium strategy can give customers more information and reduce the cost of waiting. Moreover, our results can also provide more comprehensive and precise information to the system administrators to improve the system and reduce congestion.
It would be interesting to extend our results in more complicated queueing models. It seems quite difficult from an analytical point of view, but a numerical study is worthy of attention. Another interesting direction for future research is to discuss the pricing issues in queuing systems, such as the entrance fee, the priority fee and so on, since our study provides the managers with a good reference.
This journal is currently published in open access with no charge for authors under a Subscribe-to-Open model (S2O). Open access is the free, immediate, online availability of research articles combined with the rights to use these articles fully in the digital environment.
S2O is one of the transformative models that aim to move subscription journals to open access. Every year, as long as the minimum amount of subscriptions necessary to sustain the publication of the journal is attained, the content for the year is published in open access.
Ask your library to support open access by subscribing to this S2O journal.
Please help to maintain this journal in open access! Encourage your library to subscribe or verify its subscription by contacting subscribers@edpsciences.org We are thankful to our subscribers and sponsors for making it possible to publish the journal in open access, free of charge for authors. More information and list of sponsors: https://www.edpsciences.org/en/maths-s2o-programme