Microservices made easy with Mesos

About Me

March 2010

Platform Engineering Lead @

Apache Mesos

March 2010

An open-source cluster manager that abstracts CPU, memory, storage, and other compute resources away from machines, enabling fault-tolerant and elastic distributed systems to easily be built and run effectively.

 

http://mesos.apache.org/

 

Mesos : Origins

Project "Nexus" created

by Ben Hindman, Andy Konwinski, Matei Zaharia at UC Berkley*

Spring 2009

March 2010

September 2010

December 2010

Twitter begins using Mesos

 Ben Hindman gives tech talk to twitter engineers

Mesos Published

 Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center**

Mesos enters Apache Incubator

March 2010

Meanwhile over at Google

March 2010

Mesos enters Apache Incubator

10 years experience deploying containers at scale on Borg

 

Google Faculty Summit (John Wilkes) on Omega (2011)

https://www.youtube.com/watch?v=0ZFMlO98Jkc&feature=youtu.be

 

Omega: flexible, scalable schedulers for large compute clusters (2013)

http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/41684.pdf

 

Large-scale cluster management at Google with Borg (2015)

http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43438.pdf

Types of schedulers

March 2010

Monolithic

 

Two-level

 

Shared-state

Uses single, centralised scheduling for all jobs (Fleet, Docker Swarm, Borg, Kubernetes)

Single resource manager, offers compute resources to multiple, parallel, independent frameworks (Mesos)

Lock-free optimistic concurrency control (Omega)

Mesos : Architecture

March 2010

Source: https://www.antonlindstrom.com/images/mesosmasterquorum-al-150324-w1024_1x.png

Mesos : Frameworks

March 2010

Source: http://maauso.com/wp-content/uploads/2015/02/frameworks.png

Or build your own

March 2010

Mesos : Adopters

March 2010

Apache Hadoop + Spark, contributed Chronos framework

Powers Apple Siri. Custom framework (Jarvis)

Powers Twitter production services using Apache Aurora

Large Scale CI using Jenkins Framework

Custom framework (Fenzo) powering Netflix services

What we've done : Apollo

March 2010

OS

Cluster scheduling

Cluster services

YOUR MICROSERVICE

Frameworks + Apps

What others are doing

March 2010

Mesos : What's new and coming

March 2010

Oversubscription (http://mesos.apache.org/documentation/latest/oversubscription/)

Maintenance Primitives (http://mesos.apache.org/documentation/latest/maintenance/)

Dynamic reservation & persistent volumes (https://mesosphere.github.io/marathon/docs/persistent-volumes.html)

Network isolation (https://github.com/mesosphere/net-modules)

 

Optimistic Offers (MESOS-1607)

Windows based agent (MESOS-3094)

Cluster wide resource constraints (MESOS-2728)

A 1.0.0 release at some point

March 2010

microservices made easy with mesos

By Graham Taylor

microservices made easy with mesos

  • 844
Loading comments...

More from Graham Taylor