The error message indicates that the size of the generated model is too large. You will have to use model splitting techniques to numerically decouple different parts of your system. Refer to the "forum question. The PLECS Manual and the tutorials contain guidance on what is required if a modulator is implemented using C-Scripts.
On the RT Box you are not using the PWM Capture input to supply the switching signals to your series connected full bridges. This will reduce the model fidelity substantially.