Age-of-information minimization via opportunistic sampling by an energy harvesting source

Herein, minimization of time-averaged age-of-information (AoI) in an energy harvesting (EH) source equipped remote sensing setting is considered. The EH source opportunistically samples one or multiple processes over discrete time instants, and sends the status updates to a sink node over a time-varying wireless link. At any discrete time instant, the EH node decides whether to ascertain the link quality using its stored energy (called probing) and then, decides whether to sample a process and communicate the data based on the channel probe outcome. The trade-off is between the freshness of information available at the sink node and the available energy at the energy buffer of the source node. To this end, infinite horizon Markov decision process (MDP) theory is used to formulate the problem of minimization of time-averaged expected AoI for a single energy harvesting source node. The following two scenarios are considered: (i) energy arrival process and channel fading process are independent and identically distributed (i.i.d.) across time, (ii) energy arrival process and channel fading process are Markovian. In the i.i.d. setting, after probing a channel, the optimal source node sampling policy is shown to be a threshold policy involving the instantaneous age of the process, the available energy in the buffer and the instantaneous channel quality as the decision variables. Also, for unknown channel state and energy harvesting characteristics, a variant of the Q-learning algorithm is proposed for the two-stage action taken by the source, that seeks to learn the optimal status update policy over time. For Markovian channel and Markovian energy arrival processes, the problem is again formulated as an MDP, and a learning algorithm is provided to handle unknown dynamics. Finally, numerical results are provided to demonstrate the policy structures and performance trade-offs.