Tools & techniques for brining machine learning research to production
PyData Chicago February Meetup youtube, https://github.com/dconathan/mtg-search
As a data scientist on a machine learning research team, I often find myself treading the line between the wild west of research and the unrelenting demands of production-quality software development. Because of the nature of machine learning research, traditional software development methodologies don’t necessarily apply, and new approaches are needed. These issues are compounded by the fact that the worlds of machine learning, DevOps and cloud computing are always changing and it seems impossible to keep up-to-date. In this talk, I will discuss these challenges and present some lessons learned and best practices. In particular I will demonstrate how to leverage Python’s packaging system as a valuable tool to tackle these issues and conclude with an example research-to-production CI/CD pipeline.