June 9

With great self-served platform comes a greater need for multi-tenancy

This talk will be curated with the use cases, challenges, concerns and mitigation strategies experienced by the Lowe's Platform engineering team while creating, maintaining, and scaling a real-time data reporting platform.
Talk abstract

The Engineering Platforms & Architecture (EPA) team at Lowe's ensures critical engineering platforms operate and scale for our 1700+ stores which act as mini datacenters constituting a distributed hybrid cloud. To support the real-time data reporting needs of internal business teams, we built a platform on a shared tenancy model. To save crucial engineering time spent to onboard tenants, we made the platform self-serve which catapulted adoption to 2x YoY (8TB data with 54 billion events). However, when some tenants started issuing long-running analytical queries, it created a noisy neighbor situation and several tenants witnessed a degradation in performance. Upon further analysis, we decided to bring in multi-tenancy and resource isolation to avoid such "noisy neighbor" situations. 

In this session, attendees will get to know:

  • How to achieve a self-served state for a real-time stream analytics platform.
  • The pain points of a self-served platform without guard rails and the emergence of noisy neighbor situations.
  • QoS and SLA: How can we bring in multi-tenancy to the infrastructure by means of resource isolation, to resolve some of the challenges originating from shared tenancy.