TRANSNET researchers propose a novel application for reinforcement learning

In a paper published today in the Journal of Optical Communications and Networking, TRANSNET researchers propose a novel application of reinforcement learning (RL) with invalid action masking and a novel training methodology for routing and wavelength assignment (RWA) in fixed-grid optical networks.

Due to the well-known NP hard problem associated with the routing and wavelength assignment (RWA) in optical networks, there has been a widespread interest in finding new methods that improve on the efficacy of existing heuristic approaches. Through the introduction of invalid action masking and a new training method, the paper demonstrates that the applicability of reinforcement learning (RL) to routing and wavelength assignment (RWA) in fixed-grid networks can be extended from considering connection requests between nodes to servicing demands of a given bit rate. Thus, lightpaths can be used to service multiple demands, as long it has sufficient remaining capacity.

The paper highlights additional challenges involved for this routine and wavelength (RWA) assignment problem and thus recommends invalid action masking and a novel training method to improve the efficacy of reinforcement learning (RL) agent in optical networks.

The proposed training method reduces the difficulty of credit assignment during the training stage, increasing the efficacy of the reinforcement learning (RL) agent through allowing the agent to learn a policy that generalises to a realistic target case with a higher efficacy as opposed to training directly on the target problem.

Moreover, the team also performs interpretation of the learned routing and wavelength assignment (RWA) policy via visualisation of the distribution of the services across channels and the distribution across links. The paper demonstrates that the proposed reinforcement learning (RL) model shows potential for real-world applicability in terms of its routing and wavelength assignment (RWA) runtime, which is comparable to that of the considered heuristic approaches.

The research is led by TRANSNET investigators Prof. Georgios Zervas and Prof. Seb Savory, along with TRANSNET members from UCL and Cambridge and is published in the September 2022 issue of the Journal of Optical Communications and Networking.

Full paper can be accessed here: