Codebase for "Paid with Models: Optimal Contract Design for Collaborative Machine Learning". Implementation and experiments of optimal contracting design for incentivizing machine learning collaboration.

• Name: Bingchen Wang
• Email: bw2506@columbia.edu
• Institution/Organization: National University of Singapore (NUS)
• Website/Portfolio: Bingchen's Google Site

Feel free to contact me if you have any code-related question.

Ensure Python dependencies are met by installing the following:

pip install -r requirements.txt

Top: Optimal contracts under incomplete information for varied probability of high-cost type $p_1 \in (0,1)$ and total number of participants $N \in [2, 100]$, with $c = \{0.02, 0.01\}$.
Bottom: Information costs for the coordinator and information rents for the parties under incomplete information vis-à-vis complete information.

You can replicate Section 6.1 by running the script 📜 Experiment_Twotype_Case.ipynb.

Optimal contract designs for multi-type scenarios.
Scenario 1: All types would train a model on their own.
Scenario 2: All types would not train a model on their own due to prohibitive costs.
Scenario 3: Some types would train the model on their own and others would not.

You can replicate Section 6.2 by running the script 📜 Experiment_Multitype_Case.ipynb.

Functions $f$ and $g$ and the $m_1$'s that give the maximum values.

You can replicate Appendix A.1 by running the script 📜 Appendix_A_Related_Work.ipynb.

In addition to the experiments presented in the paper, we provide two additional experiments in the repository for interested readers.

The contribution of constraint analysis in the paper comes in two fronts. First, it helps establish clean properties of the optimal contract. Second, it may help speed up computation (~15% for Sec. 6.2.). However, the effect depends significantly on the choice of optimization algorithm—it is expected to be more pronounced for active-set methods [1] and likely to matter less so for others.

The scripts 📜 Experiment_Twotype_Case.ipynb and 📜 Experiment_Multitype_Case_noSimp.ipynb run through the experiments in Sec. 6.1 and Sec. 6.2 without the constraint simplication.

Our preliminary experiments show that the algorithm’s running time increases at roughly a factor of 2 per additional type added, while increments in $N$ has less effect. This corroborates our discussion on the combinatorial challenge in Appendix B.9. Our current model is best suited for cases where $N$ and $I$ are reasonably finite, as is the case of cross-silos collaboration. For reference, it took 8.03 seconds (wall-clock) to run the experiments in Sec 6.2 on a Macbook Pro with M2 chip. We posit that scalability can be improved with relaxation on the distribution assumption, which is an active research area [2]. It can also be improved with better approximation algorithms, though with a trade-off between welfare and speed of computation.

Interested reader can implement 📜 Scalability_check_increasing_N.py and 📜 Scalability_check_increasing_types.py to see the increase in running time.

1. Nocedal, J., & Wright, S. J. (2006). Numerical Optimization. Springer, 2006, p.424. Available through Springer.

2. Dütting, P., et al. (2019). Simple versus Optimal Contracts. Presented at EC'19. Available through ACM Digital Library.