Federated learning of neural networks

Federated learning is a method for training ML models where original data physically remains with its owner and is never shared with any external contractors. Compliant with the Russian Federal Law 152-FZ on personal data, this approach is particularly suitable for handling such sensitive information as medical records.

A key advantage is that you do not need to keep the data structure identical across sources. Different healthcare institutions may collect different patient data, yet the technology enables efficient model training even on such heterogeneous datasets.

Federated learning allows pooling resources from multiple organizations without centralized data exchange: the models train locally on each contributor’s data, regardless of format differences.

In an experiment by Yandex Cloud in collaboration with Sechenov University and ISP RAS, the method demonstrated its real-world viability by successfully detecting cardiac pathologies in ECG data. This partnership aims to validate federated learning’s efficiency in healthcare, where data privacy is paramount.

In this article, we will share key highlights of our experiment, showing how federated learning helped address the challenges of both data heterogeneity and scarcity. We will also discuss the results of applying this approach and explore its opportunities and prospects in the nearest future of healthcare.

How federated learning works

Typically, when training AI models, you collect data from a vast range of sources, such as archives from different medical institutions, healthcare information systems, or public datasets. Then you send all this data to a single central server for processing, where your model trains on them.

This traditional approach, however, poses serious privacy risks, especially in such sensitive domains as healthcare. Among such risks, one may mention possible data leaks or unauthorized access to patients' personal medical information.

This approach preserves data privacy since any raw data never leaves its owner’s system. Thus, medical institutions are free to collaborate and pool their resources without compromising patient privacy, ultimately improving the performance of the AI models.

Unlike traditional centralized systems, this central federated server does not process or store any source data. Instead, it coordinates the learning process by aggregating model parameters to improve the overall model, while ensuring data privacy.

Once the model parameters from all clients are combined, an updated version of the model goes back to its client devices to keep learning based on their local data.

By adapting to changes and unique features of each client, the model gets more accurate and efficient over time. The process repeats until it reaches the desired performance metrics, such as prediction accuracy and general stability.

The federated server sends certain tasks to its clients; such tasks include the model code and configuration settings. However, not every user can send such a task, since there are different roles with different permissions. Once created, a task gets validated both on the server side and on each client side. Client devices run only those tasks that pass such validations. After that, the client runs another verification to determine whether the training results can be safely sent back to the server. These multi-layered validation steps help ensure the security and integrity of the data throughout the process.

Once a task is validated successfully, the model begins training on the local data; if failed to validate, the task is rejected. This prevents data leaks and ensures that learning happens in a secure and trusted environment.

Federated learning types

Federated learning can be classified into horizontal, vertical, and transfer.

In a federated learning setup, the model trains directly on the data owner’s infrastructure, while the costs are typically covered by the customer or organization commissioning the project. If a team lacks servers equipped with graphical processing units (GPUs), which are essential for intensive computations, they can rent the required infrastructure from a cloud provider. GPUs significantly accelerate the training of such complex models as neural networks. A hybrid approach is also possible, where some data is stored and processed locally, and other parts are handled in the cloud.

Cloud providers offer per-second billing for computing resources, allowing developers to pay only for what they actually use and avoid unnecessary expenses.

Training an ML model on medical data: a case by Yandex Cloud, ISP RAS, and Sechenov University

The experiment aimed at training an ML model to recognize cardiac pathologies based on ECG data, adopting the NVFlare framework for GPU-accelerated federated learning. The Yandex Cloud team of architects thought through the architecture and implemented a technical solution, and Sechenov University provided medical support and expertise.

The project came to life on a hybrid infrastructure, with part of resources deployed in the ISP RAS own data center, while Sechenov University, lacking sufficient in-house computing power, relied on Yandex Cloud.

Software engineers from IMPRUV IT developed a setup to deploy the framework in the cloud and configure it with a single click, ensuring that computing resources were used only during training, and not wasted otherwise.

Armed with this approach, the team successfully tested the infrastructure using low-cost VMs, only bringing in expensive GPU resources when actually needed for model training. This significantly reduced overall computing costs.

The experiment involved two datasets: one publicly available open dataset and a second, proprietary dataset provided by the Sechenov University. Both datasets were labeled in a unified format.

ISP RAS adapted its existing ECG classifier so that it could function within the federated learning framework, and then initiated the training process. The train dataset consisted of 47,000 twelve-lead ECG recordings, 30,000 from ISP RAS and 17,000 from Sechenov University.

During the experiment, engineers trained the model to diagnose atrial fibrillation, having achieved a sensitivity of 99% and a specificity of 95%. To verify the accuracy and clinical applicability of the model, its results were reviewed by three functional diagnostic physicians with over decade of experience each.

Prospects of federated learning in modern healthcare

Today, federated learning is seen globally as a groundbreaker with a promising future for AI development in the domains involving sensitive data. Healthcare has emerged as one of the most interested sectors, accounting for 36% of the market. This high level of interest stems from the technology’s potential in areas such as medical image analysis and drug development. Federated learning can also be applied to train predictive models using patient electronic medical records (EMRs), and even to develop large language models (LLMs) based on medical data, thus furthering and expanding AI-driven healthcare innovation.

“Neural networks have not yet achieved complete perfection in medical diagnostics, and that is not even necessary. After all, the goal is to use AI as a tool to assist doctors, not replace them. Clinical decision support systems (CDSS’s) take over tasks that can be automated, especially routine procedures that consume time and resources. However, the final diagnosis remains in the hands of the physician, who can always manually review key indicators, account for factors the system may have missed, and form a final, well-informed opinion.”

The Institute of Personalized Cardiology at Sechenov University, together with researchers from ISP RAS and the Institute for Design-Technological Informatics of the Russian Academy of Sciences (IDTI RAS), is developing devices that integrate with CDSS’s to aid in remote heart monitoring.

These devices detect cardiovascular diseases and help process ECG data, as well as HR and pulse wave measurements. The model assists doctors in interpreting the collected data and provides a preliminary diagnosis; if needed, one can share results with colleagues for advice.

“At ISP RAS, developing federated learning algorithms is part of our broader effort to build trusted AI systems. With support from the Ministry of Science and Higher Education of Russia, we have launched a youth lab this year, focused on training large-scale models for healthcare-related tasks. We also prioritize data protection and preventing threats that could compromise the quality of learning. This research has significant scientific and practical value, confirmed by our successful collaboration with Sechenov University and Yandex Cloud.”

Federated learning is not limited to healthcare: it offers extensive prospects in other domains as well. For example, in finance, it can be used for fraud detection and credit risk assessment. Importantly, customer data remains secure, which is crucial when handling sensitive information.