Skip to content

FATE Test Tutorial

A collection of useful tools to running FATE tests and 📁examples.

quick start

  1. install

    pip install -e python/fate_test
    2. edit default fate_test_config.yaml

    # edit priority config file with system default editor
    # filling some field according to comments
    fate_test config edit
  2. configure FATE-Pipeline and FATE-Flow Commandline server setting

    # configure FATE-Pipeline server setting
    pipeline init --port 9380 --ip
    # configure FATE-Flow Commandline server setting
    flow init --port 9380 --ip
  3. run some fate_test suite

    fate_test suite -i <path contains *testsuite.json>
  4. run some fate_test benchmark

    fate_test benchmark-quality -i <path contains *benchmark.json>
  5. useful logs or exception will be saved to logs dir with namespace shown in last step

command types

  • suite: used for running testsuites, collection of FATE jobs

    fate_test suite -i <path contains *testsuite.json>
  • data: used for upload, delete, and generate dataset

    fate_test data [upload|delete] -i <path1 contains *testsuite.json | *benchmark.json>
    - upload example data of min_test/all_examples command:

    fate_test data upload -t min_test
    fate_test data upload -t all_examples
    fate_test data generate -i <path1 contains *testsuite.json | *benchmark.json>
  • benchmark-quality: used for comparing modeling quality between FATE and other machine learning systems, as specified in benchmark job configuration

    fate_test bq -i <path contains *benchmark.json>
  • benchmark-performance: used for checking FATE algorithm performance; user should first generate and upload data before running performance testsuite

    fate_test data generate -i <path contains *benchmark.json> -ng 10000 -fg 10 -fh 10 -m 1.0 --upload-data
    fate_test performance -i <path contains *benchmark.json> --skip-data
  • op-test: used for testing FATE's mpc protocol.

    fate_test op-test paillier
    fate_test op-test spdz
  • convert: used for converting pipeline to dsl&conf.

    fate_test convert pipeline-to-dsl -i ${your pipeline file}
    fate_test convert pipeline-testsuite-to-dsl-testsuite -i {your pipeline testsuite folder}



Last update: 2022-04-07