Federated Learning Is Not Private for Google GBoard Next Word Prediction [pdf] (arxiv.org)

A recent study has raised significant concerns regarding the privacy of federated learning in Google's GBoard app, particularly in its next word prediction feature. The research reveals that the model is susceptible to attacks that can accurately recover the words a user types, even in diverse contextual scenarios, undermining the intended privacy safeguards of federated learning. Despite the implementation of techniques such as mini-batches and local noise addition, the researchers found these countermeasures to be ineffective, highlighting a substantial vulnerability in the app's architecture. This revelation is particularly troubling for the AI/ML community as it challenges the perceived security of federated learning—a technique often touted for its ability to enhance privacy without compromising model quality. The ability to reconstruct not just individual words, but entire sentences poses serious implications for user confidentiality and trust in AI-driven applications. As GBoard is in active use by millions, this study emphasizes the need for further research into more robust privacy-preserving methods in machine learning, especially for applications that handle sensitive user data.