NN Modules Tutorial

In FATE-1.10，the whole NN framework is re-designed for highly customizable machine learning. With this framework, you can create your own models, datasets, trainers, and aggregators to meet your specific needs. This tutorial introduces you to our brand new framework. With links below, you can learn how to use our Homo & Hetero(Horizontal and Vertical) NN step by step. Please notice that this tutorial is based on the standalone version, if you are using the cluster version, you may have to deploy the codes/data on every party respectively.

In FATE-1.11, we support federated large language model training with parameter efficient methods(Adapters).

Quick Start

To get yourself be familiar with FATE-NN and pipeline, we recommend completing these two quick start tutorials. If you are using tabular data and do not require any customizations, these tutorials should be sufficient for your needs

• Homo-NN Quick Start: A Binary Classification Task
• Hetero-NN Quick Start: A Binary Classification Task

Homo-NN Customization

1.Dataset

Customizing a dataset in PyTorch is similar to using the built-in PyTorch Dataset class. In this chapter, we will use a simple image classification task - the MNIST Handwritten Recognition dataset - as an example to demonstrate how to customize and use your own dataset in FATE.

• Customize your Dataset
• Using FATE Built-In Dataset

2.Loss Function

In this section, we show you how to customize loss functions

• Customize loss function

3.Model

In FATE 1.10, you can create a PyTorch model by subclassing nn.Module. Using the example from the previous chapter, we will demonstrate how to develop, deploy, and use a more complex model in FATE.

• Customize Model

4.Trainer

In order to show you how to develop your own Trainer, here we try to develop a simple Trainer to implement the FedProx algorithm (this is just an exsample, please do not use it for production). In addition, we will also show you how to use the interface that comes with TrainerBase to save the model, set the checkpoint, output the prediction results, and use the interface to display some data on FateBoard.

• Customize trainer to control the training process

• Using FATE-interfaces

5.Aggregator

• Develop Aggregator: Introduce Basics

Hetero-NN Customization

Dataset

• Customize your Dataset

Top Model, Bottom Model & Interactive Layer & Loss

• Use CustModel to Set Top, Bottom Model

Advanced Examples

Here we offer some advanced examples of using FATE-NN framework.

FATE-LLM(Federated Large Language Models) Training

• FATE-LLM Training

Resnet classification(Homo-NN)

• Federated Rensnet training on CIFAR-10

Recommendation models(Homo-NN)

• Federated classic CTR model training on Criteo

Federated Text Classification Using Bert(Homo-NN)

• Using Frozen Parameters Bert for Sentiment Classification

Node Classification Using GNN(Homo-NN)

• Homo Graph Sage for Node Classification Task

Training on vertical-split heterogeneous data(Hetero-NN)

• A federated task with guest using image data and host using text data

---

最后更新: 2023-05-31