Skip to content

0.2.0 - 2023-05-15

👩‍💻 User Interfaces

  • Added two new types of user interfaces:
    • Multiple runs, multiple models with DB writer. This interface has the same functionality as the previous one, but result metrics are stored in a user database. For that, users need to pass a DB writer function to the interface that can write result metrics to their database. After each metrics computation run, the interface will use this function to save results in the database.
    • Multiple runs, multiple models with several test sets. Except for a traditional flow with one test set, Virny has an interface to work with many test sets that could be used for model stress testing. This interface uses the same estimators in the bootstrap for inference on the input test sets and saves metrics for each test set in a user database. In such a way, a model comparison on different test sets is faster and more accurate.

🗃 New Benchmark Fair-ML Datasets

  • Added 5 new data loaders for all tasks in the folktables benchmark
    • ACSIncomeDataset. A data loader for the income task from the folktables dataset. Target: binary classification, predict if a person has an annual income > $50,000.
    • ACSEmploymentDataset. A data loader for the employment task from the folktables dataset. Target: binary classification, predict if a person is employed.
    • ACSMobilityDataset. A data loader for the mobility task from the folktables dataset. Target: binary classification, predict whether a young adult moved addresses in the last year.
    • ACSPublicCoverageDataset. A data loader for the public coverage task from the folktables dataset. Target: binary classification, predict whether a low-income individual, not eligible for Medicare, has coverage from public health insurance.
    • ACSTravelTimeDataset. A data loader for the travel time task from the folktables dataset. Target: binary classification, predict whether a working adult has a travel time to work of greater than 20 minutes.

💠 Analyzers and Metrics

  • Developed an ability to define subgroups based on a list of values, e.g., create a subgroup based on values from 30 to 45 for the age column.

  • Extended the ability to define intersectional groups based on 3 or more columns and conditions.