a framework for communication-efficient distributed optimization

View the Project on GitHub gingsmith/cocoa

[New!] ProxCoCoA+ provides support for L1-regularized objectives. See paper and code.

We've added support for faster additive udpates with CoCoA+. See more information here.

This code performs a comparison of 5 distributed algorithms for training of machine learning models, using Apache Spark. The implemented algorithms are

The present code trains a standard SVM (hinge-loss, l2-regularized) using SDCA as a local solver, and reports training and test error, as well as the duality gap certificate if the method is primal-dual. The code can be easily adapted to include other internal solvers or solve other objectives.

Getting Started

How to run the code locally:

sbt/sbt assembly

(For the sbt script to run, make sure you have downloaded CoCoA into a directory whose path contains no spaces.)


The ProxCoCoA+, CoCoA+, and CoCoA algorithmic frameworks are described in more detail in the following papers:

Smith, V., Forte, S., Jordan, M. I., Jaggi, M. L1-Regularized Distributed Optimization: A Communication-Efficient Primal-Dual Framework.

Ma, C., Smith, V., Jaggi, M., Jordan, M. I., Richtarik, P., & Takac, M. Adding vs. Averaging in Distributed Primal-Dual Optimization. International Conference on Machine Learning (ICML), 2015.

Jaggi, M., Smith, V., Takac, M., Terhorst, J., Krishnan, S., Hofmann, T., & Jordan, M. I. Communication-Efficient Distributed Dual Coordinate Ascent. Advances in Neural Information Processing Systems (NIPS), 2014.