Applying Q-learning approach to CSMA Scheme to dynamically tune the contention probability

Many CSMA based MACs can abruptly degrade their performance when the optimized design parameters do not fit with the considered scenarios. The main issue is the lack of optimal adaptation of the MAC strategies to dynamic network conditions. Novel approaches based on learning, deep-learning, nature-inspired learning are gaining interest for their robustness to dynamic network conditions. In this paper, a simple adaptive MAC strategy based on the Q-learning has been proposed. Our proposal, called Q-CSMA, is able to dynamically tune the contention probability in a slotted CSMA to decrease the number of collisions, also reducing the packet latency. Q-CSMA has been compared with optimal p-persistent CSMA (p-pers-CSMA/p*) and a sift-based CSMA (sift-CSMA).