Hetero-NN Customize Models & Loss¶
We can customize the top model and the bottom model in the Hetero NN. The model_zoo module was introduced in FATE 1.10, located under federatedml.nn.model_zoo. This module allows you to customize your own PyTorch model, provided that it is developed based on torch.nn.Module and implements the forward interface. For more information, see the PyTorch documentation Pytorch Module on custom modules. To use your custom model in a federated task, simply place it in the federatedml/nn/model_zoo directory and specify the module and model class through the interface when submitting the task. Hetero-NN components will automatically search and import the model you have implemented.
You can also define your own loss class in a similar way. The you can place your loss class under the loss module, located under federatedml.nn.loss. We recommend you read these two tutorials before reading this tutorial: Customize loss function, Customize Model
As an example, we consider reuse the task of MNIST handwriting recognition of last hetero-nn tutorial.
Prepare MNIST Data¶
Please download the guest/host MNIST dataset from the link below and place it in the project examples/data folder:
guest data: https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate/examples/data/mnist_guest.zip
host data: https://webank-ai-1251170195.cos.ap-guangzhou.myqcloud.com/fate/examples/data/mnist_host.zip
The mnist_guest is a simplified version of the MNIST dataset, with a total of ten categories, which are classified into 0-9 10 folders according to labels. The mnist_host has the same images as the mnist_guest, but it is not labeled.
! ls ../../../../examples/data/mnist_guest
0 1 2 3 4 5 6 7 8 9
! ls ../../../../examples/data/mnist_host
not_labeled
Dataset¶
In version FATE-1.10, FATE introduces a new base class for datasets called Dataset, which is based on PyTorch's Dataset class. This class allows users to create custom datasets according to their specific needs. The usage is similar to that of PyTorch's Dataset class, with the added requirement of implementing two additional interfaces when using FATE-NN for data reading and training: load() and get_sample_ids().
To create a custom dataset in Hetero-NN, users need to:
- Develop a new dataset class that inherits from the Dataset class
- Implement the __len__() and __getitem__() methods, which are consistent with PyTorch's Dataset usage. The __len__() method should return the length of the dataset, while the __getitem__() method should return the corresponding data at the specified index. However, please notice that different __getitem__() methods may have different behaviors between different parties. In the guest party(party with labels), __getitem__() method return features and labels, while in the host parties(parties without label), __getitem__() method return features only.
- Implement the load(), get_sample_ids(), get_classes() methods
For those unfamiliar with PyTorch's Dataset class, more information can be found in the PyTorch documentation: Pytorch Dataset Documentation
Customize Bottom/Top Model¶
Name the model code bottom_net.py, you can put it directly under federatedml/nn/model_zoo or use the shortcut interface of jupyter notebook: save_to_fate, to save it directly to federatedml/nn/model_zoo. This is the bottom model structure we define for feature extraction.
from pipeline.component.nn import save_to_fate
%%save_to_fate model bottom_net.py
import torch as t
from torch import nn
from torch.nn import Module
class BottomNet(nn.Module):
def __init__(self):
super(BottomNet, self).__init__()
self.seq = t.nn.Sequential(
nn.Conv2d(in_channels=3, out_channels=12, kernel_size=5),
nn.MaxPool2d(kernel_size=3),
nn.Conv2d(in_channels=12, out_channels=12, kernel_size=3),
nn.AvgPool2d(kernel_size=3)
)
self.fc = t.nn.Sequential( # extracted feature is a 8-dim embedding
nn.Linear(48, 32),
nn.ReLU(),
nn.Linear(32, 8),
nn.ReLU()
)
def forward(self, x):
x = self.seq(x)
x = x.flatten(start_dim=1)
x = self.fc(x)
return x
And this is the top model we define for classification, we named it as top_model.py.
%%save_to_fate model top_net.py
import torch as t
from torch import nn
from torch.nn import Module
class TopNet(nn.Module):
def __init__(self):
super(TopNet, self).__init__()
self.fc = t.nn.Sequential(
nn.Linear(8, 10)
)
self.softmax = t.nn.Softmax(dim=1)
def forward(self, x):
x = self.fc(x)
return self.softmax(x)
Use Cust Loss¶
Using Cust Loss is exactly the same as Homo-NN, see: Customize loss function. Here we use a new CrossEntropyLoss.
%%save_to_fate loss ce.py
import torch as t
from federatedml.util import consts
from torch.nn.functional import one_hot
def cross_entropy(p2, p1, reduction='mean'):
p2 = p2 + consts.FLOAT_ZERO # to avoid nan
assert p2.shape == p1.shape
if reduction == 'sum':
return -t.sum(p1 * t.log(p2))
elif reduction == 'mean':
return -t.mean(t.sum(p1 * t.log(p2), dim=1))
elif reduction == 'none':
return -t.sum(p1 * t.log(p2), dim=1)
else:
raise ValueError('unknown reduction')
class CrossEntropyLoss(t.nn.Module):
"""
A CrossEntropy Loss that will not compute Softmax
"""
def __init__(self, reduction='mean'):
super(CrossEntropyLoss, self).__init__()
self.reduction = reduction
def forward(self, pred, label):
one_hot_label = one_hot(label.flatten())
loss_ = cross_entropy(pred, one_hot_label, self.reduction)
return loss_
Then, we can use our models & loss in the Hetero-NN MNIST task! The usage is the same as Homo-NN: we specify our model and loss by nn.CustModel and nn.CustLoss interfaces.
pipeline initialization¶
Here we define the pipeline to run a hetero task
import os
import torch as t
from torch import nn
from pipeline import fate_torch_hook
from pipeline.component import HeteroNN
from pipeline.component.hetero_nn import DatasetParam
from pipeline.backend.pipeline import PipeLine
from pipeline.component import Reader, Evaluation, DataTransform
from pipeline.interface import Data, Model
from pipeline.component.nn import save_to_fate
fate_torch_hook(t)
# bind path to fate name&namespace
fate_project_path = os.path.abspath('../../../../')
guest = 10000
host = 9999
pipeline_img = PipeLine().set_initiator(role='guest', party_id=guest).set_roles(guest=guest, host=host)
guest_data = {"name": "mnist_guest", "namespace": "experiment"}
host_data = {"name": "mnist_host", "namespace": "experiment"}
guest_data_path = fate_project_path + '/examples/data/mnist_guest/'
host_data_path = fate_project_path + '/examples/data/mnist_host/'
pipeline_img.bind_table(name='mnist_guest', namespace='experiment', path=guest_data_path)
pipeline_img.bind_table(name='mnist_host', namespace='experiment', path=host_data_path)
{'namespace': 'experiment', 'table_name': 'mnist_host'}
guest_data = {"name": "mnist_guest", "namespace": "experiment"}
host_data = {"name": "mnist_host", "namespace": "experiment"}
reader_0 = Reader(name="reader_0")
reader_0.get_party_instance(role='guest', party_id=guest).component_param(table=guest_data)
reader_0.get_party_instance(role='host', party_id=host).component_param(table=host_data)
hetero_nn_0 = HeteroNN(name="hetero_nn_0", epochs=3,
interactive_layer_lr=0.01, batch_size=512, task_type='classification', seed=100
)
guest_nn_0 = hetero_nn_0.get_party_instance(role='guest', party_id=guest)
host_nn_0 = hetero_nn_0.get_party_instance(role='host', party_id=host)
# define model
# use cust model here
# our simple classification model:
guest_bottom = t.nn.CustModel(module_name='bottom_net.py', class_name='BottomNet')
# use cust model here
host_bottom = t.nn.CustModel(module_name='bottom_net.py', class_name='BottomNet')
# use new top model here
guest_top = t.nn.CustModel(module_name='top_net.py', class_name='TopNet')
# interactive layer define
interactive_layer = t.nn.InteractiveLayer(out_dim=8, guest_dim=8, host_dim=8)
# add models
guest_nn_0.add_top_model(guest_top)
guest_nn_0.add_bottom_model(guest_bottom)
host_nn_0.add_bottom_model(host_bottom)
# opt, loss
optimizer = t.optim.Adam(lr=0.01)
loss = t.nn.CustLoss(loss_module_name='ce', class_name='CrossEntropyLoss')
# use DatasetParam to specify dataset and pass parameters
guest_nn_0.add_dataset(DatasetParam(dataset_name='image', return_label=True))
host_nn_0.add_dataset(DatasetParam(dataset_name='image', return_label=False))
hetero_nn_0.set_interactive_layer(interactive_layer)
hetero_nn_0.compile(optimizer=optimizer, loss=loss)
pipeline_img.add_component(reader_0)
pipeline_img.add_component(hetero_nn_0, data=Data(train_data=reader_0.output.data))
pipeline_img.add_component(Evaluation(name='eval_0', eval_type='multi'), data=Data(data=hetero_nn_0.output.data))
pipeline_img.compile()
<pipeline.backend.pipeline.PipeLine at 0x7f3f363f5970>
pipeline_img.fit()
2022-12-24 21:26:06.571 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:83 - Job id is 202212242126060352040 2022-12-24 21:26:06.584 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:98 - Job is still waiting, time elapse: 0:00:00 2022-12-24 21:26:07.598 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:98 - Job is still waiting, time elapse: 0:00:01 m2022-12-24 21:26:08.622 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:125 - 2022-12-24 21:26:08.624 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component reader_0, time elapse: 0:00:02 2022-12-24 21:26:09.648 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component reader_0, time elapse: 0:00:03 2022-12-24 21:26:10.671 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component reader_0, time elapse: 0:00:04 2022-12-24 21:26:11.697 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component reader_0, time elapse: 0:00:05 2022-12-24 21:26:12.720 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component reader_0, time elapse: 0:00:06 2022-12-24 21:26:13.738 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component reader_0, time elapse: 0:00:07 m2022-12-24 21:26:15.888 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:125 - 2022-12-24 21:26:15.890 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:09 2022-12-24 21:26:17.144 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:10 2022-12-24 21:26:18.180 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:11 2022-12-24 21:26:19.238 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:12 2022-12-24 21:26:20.274 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:13 2022-12-24 21:26:21.303 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:14 2022-12-24 21:26:22.334 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:15 2022-12-24 21:26:23.365 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:16 2022-12-24 21:26:24.389 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:17 2022-12-24 21:26:25.420 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:18 2022-12-24 21:26:26.460 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:19 2022-12-24 21:26:27.501 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:20 2022-12-24 21:26:28.587 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:22 2022-12-24 21:26:29.624 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:23 2022-12-24 21:26:30.651 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:24 2022-12-24 21:26:31.674 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:25 2022-12-24 21:26:32.721 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:26 2022-12-24 21:26:33.741 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:27 2022-12-24 21:26:34.774 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:28 2022-12-24 21:26:35.799 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:29 2022-12-24 21:26:36.825 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:30 2022-12-24 21:26:37.847 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:31 2022-12-24 21:26:38.870 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:32 2022-12-24 21:26:39.902 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:33 2022-12-24 21:26:40.935 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:34 2022-12-24 21:26:41.970 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:35 2022-12-24 21:26:42.998 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:36 2022-12-24 21:26:44.044 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:37 2022-12-24 21:26:45.076 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:38 2022-12-24 21:26:46.100 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:39 2022-12-24 21:26:47.123 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:40 2022-12-24 21:26:48.142 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:41 2022-12-24 21:26:49.202 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:42 2022-12-24 21:26:50.223 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:43 2022-12-24 21:26:51.245 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:44 2022-12-24 21:26:52.269 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:45 2022-12-24 21:26:53.289 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:46 2022-12-24 21:26:54.327 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:47 2022-12-24 21:26:55.348 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:48 2022-12-24 21:26:56.372 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:49 2022-12-24 21:26:57.393 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:50 2022-12-24 21:26:58.415 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:51 2022-12-24 21:26:59.437 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:52 2022-12-24 21:27:00.460 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:53 2022-12-24 21:27:01.480 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:54 2022-12-24 21:27:02.502 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:55 2022-12-24 21:27:03.568 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:56 2022-12-24 21:27:04.591 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:58 2022-12-24 21:27:05.618 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:00:59 2022-12-24 21:27:06.638 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:00 2022-12-24 21:27:07.659 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:01 2022-12-24 21:27:08.678 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:02 2022-12-24 21:27:09.700 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:03 2022-12-24 21:27:10.723 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:04 2022-12-24 21:27:11.746 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:05 2022-12-24 21:27:12.768 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:06 2022-12-24 21:27:13.788 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:07 2022-12-24 21:27:14.853 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:08 2022-12-24 21:27:15.883 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:09 2022-12-24 21:27:16.935 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:10 2022-12-24 21:27:17.973 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:11 2022-12-24 21:27:19.000 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:12 2022-12-24 21:27:20.022 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:13 2022-12-24 21:27:21.043 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:14 2022-12-24 21:27:22.071 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:15 2022-12-24 21:27:23.107 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:16 2022-12-24 21:27:24.144 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:17 2022-12-24 21:27:25.163 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:18 2022-12-24 21:27:26.187 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:19 2022-12-24 21:27:27.210 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:20 2022-12-24 21:27:28.234 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:21 2022-12-24 21:27:29.258 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:22 2022-12-24 21:27:30.324 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:23 2022-12-24 21:27:31.355 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:24 2022-12-24 21:27:32.388 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:25 2022-12-24 21:27:33.408 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:26 2022-12-24 21:27:34.431 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:27 2022-12-24 21:27:35.454 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:28 2022-12-24 21:27:36.567 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:29 2022-12-24 21:27:37.609 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:31 2022-12-24 21:27:38.641 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:32 2022-12-24 21:27:39.689 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:33 2022-12-24 21:27:40.716 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:34 2022-12-24 21:27:41.741 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:35 2022-12-24 21:27:42.764 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:36 2022-12-24 21:27:43.785 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:37 2022-12-24 21:27:44.805 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:38 2022-12-24 21:27:45.826 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:39 2022-12-24 21:27:46.844 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component hetero_nn_0, time elapse: 0:01:40 m2022-12-24 21:27:47.923 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:125 - 2022-12-24 21:27:47.925 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component eval_0, time elapse: 0:01:41 2022-12-24 21:27:48.950 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component eval_0, time elapse: 0:01:42 2022-12-24 21:27:50.007 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component eval_0, time elapse: 0:01:43 2022-12-24 21:27:51.033 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component eval_0, time elapse: 0:01:44 2022-12-24 21:27:52.053 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component eval_0, time elapse: 0:01:45 2022-12-24 21:27:53.076 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component eval_0, time elapse: 0:01:46 2022-12-24 21:27:54.122 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:127 - Running component eval_0, time elapse: 0:01:47 2022-12-24 21:27:56.202 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:89 - Job is success!!! Job id is 202212242126060352040 2022-12-24 21:27:56.204 | INFO | pipeline.utils.invoker.job_submitter:monitor_job_status:90 - Total time: 0:01:49
pipeline_img.get_component('hetero_nn_0').get_output_data() # get result
id | label | predict_result | predict_score | predict_detail | type | |
---|---|---|---|---|---|---|
0 | img_1 | 0 | 1 | 0.12622389197349548 | {'0': 0.07662956416606903, '1': 0.126223891973... | train |
1 | img_3 | 4 | 1 | 0.12622389197349548 | {'0': 0.07662956416606903, '1': 0.126223891973... | train |
2 | img_4 | 0 | 1 | 0.12622389197349548 | {'0': 0.07662956416606903, '1': 0.126223891973... | train |
3 | img_5 | 0 | 1 | 0.12622389197349548 | {'0': 0.07662956416606903, '1': 0.126223891973... | train |
4 | img_6 | 7 | 1 | 0.12622389197349548 | {'0': 0.07662956416606903, '1': 0.126223891973... | train |
... | ... | ... | ... | ... | ... | ... |
1304 | img_32537 | 1 | 1 | 0.12622389197349548 | {'0': 0.07662956416606903, '1': 0.126223891973... | train |
1305 | img_32558 | 1 | 1 | 0.12622389197349548 | {'0': 0.07662956416606903, '1': 0.126223891973... | train |
1306 | img_32563 | 1 | 1 | 0.12622389197349548 | {'0': 0.07662956416606903, '1': 0.126223891973... | train |
1307 | img_32565 | 1 | 1 | 0.12622389197349548 | {'0': 0.07662956416606903, '1': 0.126223891973... | train |
1308 | img_32573 | 1 | 1 | 0.12622389197349548 | {'0': 0.07662956416606903, '1': 0.126223891973... | train |
1309 rows × 6 columns