We present a methodology for modeling and optimally managing the demand of an aggregator with deferrable (flexible) loads (e.g., electric vehicles and HVACs) under uncertainty. We propose a unified framework for treating different types of flexible loads, that captures uncertainties in their parameters, and environmental conditions they are exposed to. Our optimization formulation minimizes the total expected cost, whose goal is to optimally balance two terms: user discomfort cost (regret), and cost paid to the utility. The main contribution lies in the estimation of the impact of uncertainty in temperature forecasts and load parameters on optimal program selection with utilities, and, consequently, optimal demand side management (DSM). We propose a cost-efficient procedure for risk estimation, and provide guidelines for its consideration in cost-effective program selection.