If this is a mold, and the foam is going to be removed before you use the molded part, you could try a thick even coat of "setting type" drywall joint compound. It is the type that comes in a powder form and sets in a specific amount of time...the 45 min works as a good all around time frame...they make 10 minute and 15 min but unless you are doing a really small area it will set up way too fast.

You can get a smooth surface...definitely need to have plenty of release agent on it though as resin will bond tightly to it. The down side is that if you get even a small hole in it...or you sand it too thin polyester will melt the Styrofoam under it in a heart beat.

If the foam is going to be left in place I would go with the Epoxy/glass bubbles "bog".