With great self-served platform comes a greater need for multi-tenancy
The Engineering Platforms & Architecture (EPA) team at Lowe's ensures critical engineering platforms operate and scale for our 1700+ stores which act as mini datacenters constituting a distributed hybrid cloud. To support the real-time data reporting needs of internal business teams, we built a platform on a shared tenancy model. To save crucial engineering time spent to onboard tenants, we made the platform self-serve which catapulted adoption to 2x YoY (8TB data with 54 billion events). However, when some tenants started issuing long-running analytical queries, it created a noisy neighbor situation and several tenants witnessed a degradation in performance. Upon further analysis, we decided to bring in multi-tenancy and resource isolation to avoid such "noisy neighbor" situations.
In this session, attendees will get to know:
- How to achieve a self-served state for a real-time stream analytics platform.
- The pain points of a self-served platform without guard rails and the emergence of noisy neighbor situations.
- QoS and SLA: How can we bring in multi-tenancy to the infrastructure by means of resource isolation, to resolve some of the challenges originating from shared tenancy.
