Yahoo has opened the gates for open-source developers to take advantage of CaffeOnSpark, deep learning software which powers Flickr’s image capabilities.
Based on the Apache Spark open source cluster computing framework, Yahoo says the system bolsters the Spark framework, which already comes with Spark MLib, a package of non-deep learning algorithms for classifying data. CaffeOnSpark takes this further, giving Spark applications the ability to use dataframes to extract predictions and models based on collections of user-generated data.