We consider the utility maximization problem under convex constraints with regard to theoretical results which allow the formulation of algorithmic solvers which make use of deep learning techniques. In particular for the case of random coefficients, we prove a stochastic maximum principle (SMP) generalizing the SMP proved by Li and Zheng (2018). We use this SMP together with the strong duality property for defining a new algorithm, which we call deep primal SMP algorithm. Numerical examples illustrate the effectiveness of the proposed algorithm. Moreover, our numerical experiments for constrained problems show that the novel deep primal SMP algorithm overcomes the deep SMP algorithm's (see Davey and Zheng (2021)) weakness of erroneously producing the value of the corresponding unconstrained problem. Furthermore, in contrast to the deep controlled 2BSDE algorithm from Davey and Zheng (2021), this algorithm is also applicable to problems with path dependent coefficients. Finally, we propose a learning procedure based on epochs, which improved the results of our algorithm even further. Implementing a semi-recurrent network architecture for the control process turned out to be also a valuable advancement.