Attracted by the advantages of Non-Orthogonal Multiple Access (NOMA) in accommodating multiple users within the same resources, this paper jointly addresses the resource allocation and power control problem for Machine Type Devices (MTDs) in a Hybrid NOMA system. Particularly, we model the problem using a Mean Field Game (MFG) framework underlying a Multi-Armed Bandit (MAB) approach. Firstly, the devices invoke the MAB tool to arrange themselves into multiple NOMA coalitions. Then, within each coalition, the MTDs apply the MFG approach to autonomously adjust their transmit power based on limited feedback received from the Base Station (BS). Simulation results are given to illustrate the equilibrium behavior of the proposed resource allocation algorithm and to underline its robustness compared to existing works in the literature.