The client
NBC.com is the online presence of NBC, the broadcast television network and flagship brand of NBCUniversal, a multi-national media conglomerate. NBC is responsible for some of your favorite entertainment, such as The Tonight Show, Saturday Night Live, and Law & Order: SVU.
The problem
NBC is expanding their user interactions with highly personalized site sections. These new features are powered by many different data sources and tailored to an individual’s viewing preferences. Throughout the day, NBC’s traffic is generally predictable. However, large spikes occur with on-air callouts or special events (any fans of The Voice out there?) so the system needs to be robust and able to scale very quickly in response to these events. To top it all off, responses from the system need to be made in around 100 milliseconds.
The team gained a very deep understanding of how to reap the most performance out of serverless functions running in AWS.
The solution
Responding quickly to highly variant requests backed by many data sources is very challenging. Most requests cannot be cached or pre-calculated. In order to handle this we started with performant data stores: MarkLogic acts as the primary content store. All video metadata and user activity is stored in documents here and can be quickly and efficiently retrieved when needed. DynamoDB acts as a secondary store for user recommendations which are computed daily by NBC’s data science team based on user activity and can be quickly scaled in size and throughput to accommodate changes in user behavior
In order to access this data efficiently while scaling for bursty traffic the team employed serverless functions deployed on AWS Lambda. Lambda functions scale up when you need them and down when you don’t. The result is a highly available stack that doesn’t cost a fortune to run during quiet periods. Serverless functions also allow the team to quickly build, test, and deploy new functionality by joining the environment configuration with the application code – the same configuration used in development can be used in production and additional resources will be added on demand
Lambda’s ability to quickly launch new application servers is not perfect. A fair amount of load testing, tweaking, and debugging was done to push the system to its limits. Through this exploration, the team gained a very deep understanding of how to reap the most performance out of serverless functions running in AWS. This includes very high-level and perhaps obvious changes like enabling API Gateway caching, using aggressive timeouts to short circuit expensive work, and employing an application cache. Our other tweaks included very low-level and AWS specific solutions such as cold start mitigation (the time taken for a fresh application container to launch) and side effects of certain types of application caches causing network limitations.
NBC’s new personalization layer was deployed in AWS and regularly responds in 150 milliseconds under peak load.
The results
NBC’s new personalization layer was deployed in AWS and regularly responds in 150 milliseconds under peak load. New personalized components can be quickly added and time to market is reduced by moving business logic out of each application and into the personalization layer. The system was benchmarked against a traditional stand-alone application and performed as well or better under load, cementing the value in the serverless approach.
The details
In the two years since we first starting working with NBC, we’ve built a relationship of trust. As NBC’s digital presence grows, our team is helping them rely less and less on Drupal, making solutions less custom-fit and more reusable across platforms, finding ways to manage their content once and then present it everywhere
Highlights
- Decoupled architecture with centralized content API
- Supports multiple devices: iPhone, Android, Apple TV, Roku, Samsung TV, Alexa