A preview of the PDF is not available
Learning Orchestra: building Machine Learning workflows on scalable containers
Abstract and Figures
Interesting efforts were done to construct tools to facilitate and streamline the development of Machine Learning (ML) workflows composed of several pipelines in the last two decades. From Unix scripts to Web based ML components and solutions to automate and orchestrate ML and Data Mining (DM) pipes, we have tried many high level services for the data scientist iterative process. On the other hand, we have the low level services being investigated, like cloud environments, container orchestration, fault tolerance service and so forth. Normally, scripts are produced to simplify such low level services operations. Unfortunately, no existing solution put both low and high level services on a unique service stack. Furthermore, none of them enables the utilization of different existing tools during the construction of a single pipeline, i.e., they are not flexible enough to permit a tool to build pre-processing pipes, another tool to build parameters tuning steps and a third different tool to perform the training step of a single pipeline. To address these limitations, we present the Learning Orchestra system, a tool to construct complex workflows using different ML tools or players transparently, i.e., from a single interoperable API we can build interesting analytical flows. The workflows can be deployed on a containerized cloud environment capable to scale and be resilient. Initial experiments demonstrated that our system is a promising and innovative alternative for the problem of simplify and streamline the ML iterative process.
Figures - uploaded by Joubert de Castro Lima
All figure content in this area was uploaded by Joubert de Castro Lima
Content may be subject to copyright.