MongoDB at Shutterfly
WHAT IS MONGODB?
MongoDB is a cross-platform, document oriented database that provides, high performance, high availability, and easy scalability. MongoDB works on concept of collection and document.
Database
Database is a physical container for collections. Each database gets its own set of files on the file system. A single MongoDB server typically has multiple databases.
Collection
Collection is a group of MongoDB documents. It is the equivalent of an RDBMS table. A collection exists within a single database. Collections do not enforce a schema. Documents within a collection can have different fields. Typically, all documents in a collection are of similar or related purpose.
Document
A document is a set of key-value pairs. Documents have dynamic schema. Dynamic schema means that documents in the same collection do not need to have the same set of fields or structure, and common fields in a collection’s documents may hold different types of data.
MongoDB is one of the newest competitors in the field of Data Storage. Still, it has become very popular with its document-oriented NoSQL Features, Map Reduce calculation capability, and distributed key-value store. It has got the title of “Database Management System of the Year” by DB-Engine. It is one of the necessary languages full stack developers need to work with. The versatile knowledge make full-stack developer salary sky high in India. These days many companies are looking for certified MongoDB professionals to make use of MongoDB Database.
SHUTTERFLY
Shutterfly, a leading Internet-based social expression and personal publishing service, safeguards more than six billion images for millions of customers. The company’s premium online photo service and products enable consumers to share memories, in the form of photos and videos, and turn everyday pictures into creative keepsakes, such as photo books and personalized websites.
As the only photo sharing site that does not down-sample, compress, or force delete photos, Shutterfly faced massive data growth that pushed the performance limits of its existing Oracle database. After an extensive analysis of open source relational and non-relational alternatives, Shutterfly chose MongoDB as its persistent data store. MongoDB delivers high performance, scalability and fast time to market — a trifecta that helps Shutterfly deliver unrivaled service to customers.
The Problem at Shutterfly
To stay ahead of the competition and provide the highest quality customer experience, Shutterfly continually implements new features to improve the various ways in which users interact with their products and the web site. After a decade of developing on Oracle, Shutterfly’s code base was brittle, bringing many engineering use cases to a painful crawl. Applications took a long time to build properly; or, once built, they wouldn’t perform quickly enough. Oracle proved too costly an expense (between software licenses and hardware) and offered sub-par performance. In 2009, Shutterfly began searching for a replacement database that could grow with the business and sought an open source solution that would enable them to scale horizontally across commodity hardware, speed up application performance and improve time to market. MongoDB fit the bill.
Solution with MongoDB
MongoDB provided Shutterfly with an agile, high performance, scalable solution at a low cost. MongoDB works seamlessly with Shutterfly’s services-based architecture, providing a clean and fast way to build applications, as well as the flexibility to match multiple use cases. To date, Shutterfly has developed nearly a dozen projects on MongoDB, which now stores more than 20TB of data.
ACCELERATED TIME TO MARKET
The “really killer reason” for using MongoDB is its rich JSON-based data structure, which offers Shutterfly an agile approach to develop software, explained Kenny Gorman, director of data services. With MongoDB, the Shutterfly team can quickly develop and deploy new applications, especially Web 2.0 and social features (e.g. tagging, likes, comments), which were difficult and costly to implement on their legacy system. MongoDB’s document model enriched Shutterfly’s data modeling abilities, allowing for new query patterns previously unavailable, while the flexible schema cut development cycles and accelerated time to market.
IMPROVED PERFORMANCE
Leaving behind complex legacy code simplified the Shutterfly architecture, allowing the company to inexpensively improve individual page and site performance. Average latency for inserts dropped to 2ms from 400ms, and Shutterfly customers now get the speed they expect when performing tasks like sharing and commenting on photos.
RELIABLE SERVER SUPPORT
Shutterfly satisfied its extremely high uptime requirements (more than 10,000 ex/sec) with MongoDB’s replica sets, which enable high availability through automatic failover of nodes and recovery of member nodes. Their unique architecture comprises four members for every replica set, and the ability to write data to more than one member has been critical for data quality and safety.
AT A GLANCE MONITORING
Shutterfly uses the MongoDB Management Service (MMS) as the primary tool for gaining system visibility and insights. Not only has it saved on DBA resources, but MMS has provided them with the confidence to move forward on projects more quickly. Plus, “it’s enabled us to have a tight link with MongoDB support,” said Gorman.
OPEN-SOURCE COMMUNITY
Shutterfly predicted early on that MongoDB had the passion, vision and know-how to build both a top-notch product and a large community. And they were right: the MongoDB community has been a great asset to their projects. “With so much energy behind MongoDB, we’ve been very pleased with the overall direction and velocity of development,” said Gorman.
Results
Shutterfly’s switch from Oracle to MongoDB resulted in a 9x performance improvement and substantial cost reduction. Additionally, Shutterfly can now develop applications in a couple of sprints — a timeframe of weeks rather than tens of months.