Superparameterization is a multiscale numerical method that models the impacts of unresolved subgridscale dynamics on resolved large scales by embedding high-resolution, periodic simulations of fine-scale phenomena into the computational grid of a large-scale model. The conventional framework for superparameterization is computationally demanding, and fails to capture certain small-scale instabilities like baroclinic instability. We provide a new stochastic framework that captures an increased range of small-scale instabilities while significantly decreasing the computational cost. The framework defines both a nonlinear deterministic closure for subgridscale terms and a stochastic closure whose mean equals the deterministic closure. The framework is tested in a one-dimensional model of wave turbulence (deterministic closure) and in a two-layer model of quasigeostrophic turbulence (stochastic closure).