The potential of the Non-Orthogonal Multiple Access (NOMA) approach for wireless communications in the fifth generation (5G) and beyond can not be underestimated. This is because users with favorable channel conditions can serve as relays to improve system performance by employing Successive Interference Cancellation (SIC). Lately, the combination of NOMA and the cooperative relay has attracted the interest of researchers. The analysis of cooperative relay NOMA (CR-NOMA) with a massive

more »

... input multiple-output (mMIMO) system is mainly based on theoretical channel models such as the correlated-based stochastic channel model (CBSM) even though the geometric-based stochastic channel model (GBSM) has been found to provide better, practical and realistic channel properties. This, in our view, is due to computational challenges. Again, the performance of CR-NOMA systems using the GBSM channel model with large antenna transmitters and network coding schemes has attracted little attention in academia. Therefore, the need to study mMIMO CR-NOMA that considers channel properties such as path-loss, delay profile and tilt angle has become vital. Furthermore, the co-existing of large antenna transmitters with coding schemes needs further investigation. In this paper, we study the performance of a two-stage mMIMO CR-NOMA network where the transmitter is represented as a uniform rectangular array (URA) or cylindrical array (CA). The communication channel from the transmitter (TX) to the user equipment (UE) through a relay station (RS) is modeled with a 3GPP's three-dimensional (3D) GBSM mMIMO channel model. To improve the analytical tractability of 3D GBSM, we defined the antenna element location vectors using the physical dimension of the antenna array and incorporated them into the 3D channel model. Bit-error rates, achievable rates and outage probabilities (OP) are examined using amplify-and-forward (AF) and decode-and-forward (DF) coding schemes. Results obtained show with fixed power allocation and SNR of 20 dB, far or weak users can attain a high achievable rate using DF and URA. Again, from the results, the combination of AF and CA presents better outage probabilities. Finally, the results indicate that the performance difference between CBSM and GBSM is marginal, even though the proposed 3D GBSM channel model has a higher degree of random parameters and computational complexities.