Abstract: Gaussian processes (GPs) are a powerful statistical tool for modeling and predicting complex phenomena, and they are frequently employed as the foundation for Bayesian optimization and active learning techniques. However, when it comes to the direct application of GPs to real-world experiments, practitioners often encounter significant challenges. These include the incorporation of existing domain knowledge, uncertainties in inputs, and the integration of dissimilar data modalities. To tackle these issues, we introduced GPax, a software package developed in close coordination with experimentalists.
In this tutorial, I will provide an in-depth discussion of fully Bayesian implementations of GPs. We'll start with the basics, ensuring a solid understanding of the fundamental principles underpinning GPs. Then, we'll move on to more advanced topics, including examples of how to incorporate prior knowledge into GPs. This can take the form of phenomenological models or data from prior simulations, both of which can significantly enhance the predictive power of the GPs and allow for more efficient active learning and optimization.
A common scenario in experimental sciences involves uncertainty in the input parameter values. We will explore how GPax can handle this issue, allowing for more accurate and reliable predictions even when the input data is uncertain/stochastic.
Finally, I will discuss a hybrid approach that combines the strengths of Gaussian processes and deep learning. This approach is particularly suited for active learning in multi-modal experiments, where it can leverage the rich, hierarchical representations learned by deep learning models to guide the GPs in making more informed and effective decisions.
By the end of this tutorial, you'll have a comprehensive understanding of how Gaussian processes work, how GPax can help overcome some of the traditional challenges associated with applying GPs to real-world data, and how a hybrid GP-deep learning approach can be used to enhance active learning in complex experimental settings.
GitHub link: https://github.com/ziatdinovmax