DSP-Based Physical Layer Security for Legacy and Next-Generation Networks

Abstract

Driven by ambitious 6G targets, next-generation networks must satisfy increasingly stringent key performance indicators (KPIs), including ultra-high connection densities, ultra-low latencies, and ultra-high reliability, while accommodating heterogeneous devices. To meet these requirements, seamless network convergence has emerged as a promising solution, enabling signals to continuously flow across heterogeneous network segments, i.e., optical fibre and millimetre-wave (mmWave) links, without electrical-optical/optical-electrical (E-O/O-E) conversions and digital signal processing (DSP) at intermediate nodes. Such convergence reduces latency, improves resource utilisation efficiency, and enhances network operation flexibility. As converged networks increasingly rely on multi-vendor deployments, open network solutions are adopted to enable interoperability and reduce both capital and operational expenditures. In the open and converged networks, it is essential to extend security beyond high-layer protocols to the physical layer, thus providing additional protection against emerging advanced cybersecurity threats, such as those enabled by quantum computing.

However, existing physical layer security (PLS) techniques, such as quantum key distribution, optical chaos-secure communications, radio frequency (RF)-based PLS, and digital chaotic systems, often suffer from high system complexity, limited compatibility with heterogeneous networks, or reliance on specific signal characteristics. Their applicability is therefore strongly restricted in the open and converged networks. To address these challenges, this dissertation research proposes and demonstrates, for the first time, two cost-effective, low-complexity, key-transmission-free DSP-based PLS techniques applicable to both legacy and next-generation networks: (i) optical domain encryption/decryption using dispersive elements and key-driven phase modulators, and (ii) digital domain encryption/decryption using chaotic digital filters (CDFs).

The first technique, primarily targeting legacy optical networks, secures data by introducing optical signal distortions through dispersive elements and DSP-generated private-security key-driven phase modulators. A theoretical model is established, based on which numerical simulations are conducted for coherent optical transmission systems. Optimal system parameters are identified, which maximise security while preserve the transmission performance. By employing the optimal parameters, sensitivity to encryption/decryption parameter offsets and maximum transmission distances are systematically analysed using signals with different modulation formats and baud rates. Results confirm that the encryption-induced performance penalty is almost negligible. The results also confirm the practical applicability of the proposed PLS technique. This has been validated by field trials of a corresponding prototype developed using commercial devices in a 10Gb/s@57km deployed fibre loop.

To enable PLS in heterogeneous, open and resource-constrained networks, this dissertation proposes a fundamentally new concept, termed chaotic digital filters (CDFs), by introducing noise-like, key-dependent variations to standard digital filters. Consequently, the CDFs exhibit chaotic amplitude and phase frequency responses with high unpredictability characterised by their permutation entropies of >0.99. Based on the CDFs with security key-dependent responses, the PLS technique is thus created by employing paired chaotic shaping filters for encryption and paired chaotic matching filters for decryption. This technique provides tri-level signal protection, i.e., i) signals are distorted in the frequency and time domains, ii) data-assisted protection is realised by mutual signal interference during their transmissions, and iii) enhanced signal distortions occur if incorrect security keys are applied.

Comprehensive experimental investigations of the CDF-based PLS technique are performed in: i) a 25km IMDD fibre system, ii) a converged system comprising 25km fibre and 5m 36GHz mmWave link based on free-running laser and data-carrying optical signal beating-generated mmWave carriers and their down-conversion using envelope detection. Both single-channel and multi-channel transmissions are investigated to demonstrate its suitability for next-generation open and converged networks. Across these scenarios, experimental results confirm that the CDF-based PLS technique supports continuously secure dynamic channel transmissions across different network segments without changing the security technique. In addition, it also provides flexible optical and RF access with identical secure transmission performance, while still maintaining an encryption-induced power penalty of <1dB. Furthermore, the technique also exhibits utmost security against eavesdropping due to its high sensitivity to minor mismatches between encryption and decryption private security keys. Moreover, optimum CDF parameters are identified that are compatible with different network types, thereby simplifying its practical implementation. The above results indicate that the CDF-based PLS technique has four unique features, including security-by-design, openness-by-design, dynamic security at the traffic level and universal network compatibility.

Date of Award	11 Aug 2026
Original language	English
Awarding Institution	Bangor University
Supervisor	Roger Giddings (Supervisor) & Jianming Tang (Supervisor)