This thesis focuses on the development and application of Bayesian inference techniques for early-Universe signals and on the advancement of mathematical tools for information retrieval. A crucial quantity required to gain information from the early Universe is the primordial scalar potential and its statistics. We reconstruct this scalar potential from cosmic microwave background data. Technically, the inference is done by splitting the large inverse problem of such a reconstruction into many, each of them solved by an optimal linear filter. Once the primordial scalar potential and its correlation structure have been obtained the underlying physics can be directly inferred from it. Small deviations of the scalar potential from Gaussianity, for instance, can be used to study parameters of inflationary models. A method to infer such parameters from non-Gaussianity is presented. To avoid expensive numerical techniques the method is kept analytical as far as possible. This is achieved by introducing an approximation of the desired posterior probability including a Taylor expansion of a matrix determinant. The calculation of a determinant is also essential in many other Bayesian approaches, both apart from and within cosmology. In cases where a Taylor approximation fails, its evaluation is usually challenging. The evaluation is in particular difficult, when dealing with big data, where matrices are to huge to be accessible directly, but need to be represented indirectly by a computer routine implementing the action of the matrix. To solve this problem, we develop a method to calculate the determinant of a matrix by using well-known sampling techniques and an integral representation of the log-determinant. The prerequisite for the presented methods as well as for every data analysis of scientific experiments is a proper calibration of the measurement device. Therefore we advance the theory of self-calibration at the beginning of the thesis to infer signal and calibration simultaneously from data. This is achieved by successively absorbing more and more portions of calibration uncertainty into the signal inference equations. The result, the Calibration-Uncertainty Renormalized Estimator, follows from the solution of a coupled differential equation. Advisors/Committee Members: Enßlin, Torsten (advisor).