Bayesian inference provides a natural framework for quantifying uncertainty in PDE-constrained inverse problems, for fusing heterogeneous sources of information, and for conditioning successive predictions on data. In this setting, simulating from the posterior via Markov chain Monte Carlo (MCMC) constitutes a fundamental computational bottleneck. We present a new technique that entirely avoids Markov chain-based simulation, by constructing a map under which the posterior becomes the pushforward measure of the prior. Existence and uniqueness of a suitable map is established by casting our algorithm in the context of optimal transport theory. The proposed maps are analytically and efficiently computed using various optimization methods.