Machine learning is becoming more accessible as the technology advances and companies release more machine learning APIs. In a recent blog post, Louis Dorard decided to compare some of the bigger names in machine learning (ML) with a few lesser-known APIs. The ML services being compared are Google Prediction API, Amazon ML, PredicSis and BigML.
The comparison was based on the Kaggle “Give Me Some Credit” challenge for assigning financial credit scores based on the predicted probability of default. The predictions are drawn from a dataset of input-output values corresponding to individual applicants who later either did or didn’t default. The goal is to use ML to predict whether a new applicant will default on any awarded credit.
The author used a dataset of 150,000 instances to train the model and create an evaluation set used for testing. Results were based on 3 metrics; accuracy of predictions, time it took to train, time it took to make predictions.
Of the APIs compared, Google performed the worst, being the slowest and least accurate of the four. Amazon ML was the most accurate, but this came at the expense of time to train and make predictions. BigML proved to be the fastest in both training and predictions, but compromised on accuracy.
That left PredicSis as the best overall performer, being second fastest behind BigML and second-most accurate behind Amazon ML. Dorard didn’t include pricing or features in the assessment, so this comparison is based purely on performance of this one task, and results from another dataset may vary. However, this does show that the bigger companies don’t always offer the best products.