a framework for communication-efficient distributed optimization
[New!] ProxCoCoA+ provides support for L1-regularized objectives. See paper and code.
We've added support for faster additive udpates with CoCoA+. See more information here.
This code performs a comparison of 5 distributed algorithms for training of machine learning models, using Apache Spark. The implemented algorithms are
The present code trains a standard SVM (hinge-loss, l2-regularized) using SDCA as a local solver, and reports training and test error, as well as the duality gap certificate if the method is primal-dual. The code can be easily adapted to include other internal solvers or solve other objectives.
How to run the code locally:
sbt/sbt assembly
./run-demo-local.sh
(For the sbt script to run, make sure you have downloaded CoCoA into a directory whose path contains no spaces.)
The ProxCoCoA+, CoCoA+, and CoCoA algorithmic frameworks are described in more detail in the following papers:
Smith, V., Forte, S., Jordan, M. I., Jaggi, M. L1-Regularized Distributed Optimization: A Communication-Efficient Primal-Dual Framework.
Ma, C., Smith, V., Jaggi, M., Jordan, M. I., Richtarik, P., & Takac, M. Adding vs. Averaging in Distributed Primal-Dual Optimization. International Conference on Machine Learning (ICML), 2015.
Jaggi, M., Smith, V., Takac, M., Terhorst, J., Krishnan, S., Hofmann, T., & Jordan, M. I. Communication-Efficient Distributed Dual Coordinate Ascent. Advances in Neural Information Processing Systems (NIPS), 2014.