Speaker
Description
Dimension reduction (DR) for data visualization provides unique insights into the structure of high-dimensional data. DR provides a bird's eye view of a dataset, showing clusters and their relationships, manifolds, branching patterns, and even possible errors in the data. It is extremely effective for scientific discovery and hypothesis generation.
One tension that has always faced DR methods is the apparent trade-off between the preservation of local structure (i.e., neighborhoods) and the preservation of global structure (i.e., placement of clusters or manifolds). In the past, DR methods were either global or local, but not both. This changed with the PaCMAP algorithm, published in 2021. PaCMAP is based on an understanding of how local and global structure are preserved in DR; we discovered that these are separate mechanisms, which is why PaCMAP can preserve both simultaneously with no inherent trade-off between them. I will discuss interesting elements of DR algorithms leading to PaCMAP's derivation. I will also discuss applications to bioinformatics, name-ethnicity classification, finance, and neurology. The original PaCMAP paper will be discussed paper and recent extensions.
Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization
Journal of Machine Learning Research (JMLR), 2021
Our code for PaCMAP is the winner of the 2023 John M. Chambers Statistical Software Award and the 2024 Award for Innovation in Statistical Programming and Analytics, both from the American Statistical Association. It regularly has >20K downloads per month from PyPI and its github repository has >1K stars.