What data scientists and data engineers can do with current generation serverless technologies

O'Reilly Data Show Podcast

In this episode of the Data Show, I spoke with Avner Braverman, co-founder and CEO of Binaris, a startup that aims to bring serverless to web-scale and enterprise applications. This conversation took place shortly after the release of a seminal paper from UC Berkeley (“Cloud Programming Simplified: A Berkeley View on Serverless Computing”), and this paper seeded a lot of our conversation during this episode.

Serverless is clearly on the radar of data engineers and architects. In a recent survey, we found 85% of respondents already had parts of their data infrastructure in one of the public clouds, and 38% were already using at least one of the serverless offerings we listed. As more serverless offerings get rolled out—e.g., things like PyWren that target scientists—I expect these numbers to rise.

We had a great conversation spanning many topics, including:

• A short history of cloud computing.
• The fundamental differences between serverless and conventional cloud computing.
• The reasons serverless—specifically AWS Lambda—took off so quickly.
• What can data scientists and data engineers do with the current generation serverless offerings.
• What is missing from serverless today and what should users expect in the near future.

Related resources:

• “The evolution and expanding utility of Ray”
• Results of a new survey: “Evolving Data Infrastructure: Tools and Best Practices for Advanced Analytics and AI”
• Eric Jonas on “Building accessible tools for large-scale computation and machine learning”
• “7 data trends on our radar”
• “Handling real-time data operations in the enterprise”
• “Progress for big data in Kubernetes”