Ten years ago, a database was needed to scale out for both reads and writes to meet the long-term needs of a growing business. The Amazon Dynamo database was developed and WERNER VOGELS, CTO of Amazon.com explains how its has grown from then.
It all started in 2004 when Amazon was running Oracle’s enterprise edition with clustering and replication. We had an advanced team of database administrators and access to top experts within Oracle. We were pushing the limits of what was a leading commercial database at the time and were unable to sustain the availability, scalability and performance needs that our growing Amazon business demanded.
Our straining database infrastructure on Oracle led us to evaluate if we could develop a purpose-built database that would support our business needs for the long term. We prioritized focusing on requirements that would support high-scale, mission-critical services like Amazon’s shopping cart, and questioned assumptions traditionally held by relational databases such as the requirement for strong consistency. Our goal was to build a database that would have the unbounded scalability, consistent performance and the high availability to support the needs of our rapidly growing business.
A deep dive on how we were using our existing databases revealed that they were frequently not used for their relational capabilities. About 70 percent of operations were of the key-value kind, where only a primary key was used and a single row would be returned. About 20 percent would return a set of rows, but still operate on only a single table.
With these requirements in mind, and a willingness to question the status quo, a small group of distributed systems experts came together and designed a horizontally scalable distributed database that would scale out for both reads and writes to meet the long-term needs of our business. This was the genesis of the Amazon Dynamo database.
The success of our early results with the Dynamo database encouraged us to write Amazon’s Dynamo whitepaper and share it at the 2007 ACM Symposium on Operating Systems Principles (SOSP conference), so that others in the industry could benefit. The Dynamo paper was well-received and served as a catalyst to create the category of distributed database technologies commonly known today as “NoSQL.”
Of course, no technology change happens in isolation, and at the same time NoSQL was evolving, so was cloud computing. As we began growing the AWS business, we realized that external customers might find our Dynamo database just as useful as we found it within Amazon.com. So, we set out to build a fully hosted AWS database service based upon the original Dynamo design.
The requirements for a fully hosted cloud database service needed to be at an even higher bar than what we had set for our Amazon internal system. The cloud-hosted version would need to be:
- Scalable – The service would need to support hundreds of thousands, or even millions of AWS customers, each supporting their own internet-scale applications.
- Secure – The service would have to store critical data for external AWS customers which would require an even higher bar for access control and security.
- Durable and Highly-Available – The service would have to be extremely resilient to failure so that all AWS customers could trust it for their mission-critical workloads as well.
- Performant – The service would need to be able to maintain consistent performance in the face of diverse customer workloads.
- Manageable – The service would need to be easy to manage and operate. This was perhaps the most important requirement if we wanted a broad set of users to adopt the service.
With these goals in mind, In January, 2012 we launched Amazon DynamoDB, our cloud-based NoSQL database service designed from the ground up to support extreme scale, with the security, availability, performance and manageability needed to run mission-critical workloads.
Today, DynamoDB powers the next wave of high-performance, internet-scale applications that would overburden traditional relational databases. Many of the world’s largest internet-scale businesses such as Lyft, Tinder and Redfin as well as enterprises such as Comcast, Under Armour, BMW, Nordstrom and Toyota depend on DynamoDB’s scale and performance to support their mission-critical workloads.
DynamoDB is used by Lyft to store GPS locations for all their rides, Tinder to store millions of user profiles and make billions of matches, Redfin to scale to millions of users and manage data for hundreds of millions of properties, Comcast to power their XFINITY X1 video service running on more than 20 million devices, BMW to run its car-as-a-sensor service that can scale up and down by two orders of magnitude within 24 hours, Nordstrom for their recommendations engine reducing processing time from 20 minutes to a few seconds, Under Armour to support its connected fitness community of 200 million users, Toyota Racing to make real time decisions on pit-stops, tire changes, and race strategy, and another 100,000+ AWS customers for a wide variety of high-scale, high-performance use cases.
With all the real-world customer use, DynamoDB has proven itself on those original design dimensions:
- Scalable – DynamoDB supports customers with single tables that serve millions of requests per second, store hundreds of terabytes, or contain over 1 trillion items of data. In support of Amazon Prime Day 2017, the biggest day in Amazon retail history, DynamoDB served over 12.9 million requests per second. DynamoDB operates in all AWS regions (16 geographic regions now with announced plans for six more Regions in Bahrain, China, France, Hong Kong, Sweden), so you can have a scalable database in the geographic region you need.
- Secure – DynamoDB provides fine-grained access control at the table, item, and attribute level, integrated with AWS Identity and Access Management. VPC Endpoints give you the ability to control whether network traffic between your application and DynamoDB traverses the public Internet or stays within your virtual private cloud. Integration with AWS CloudWatch, AWS CloudTrail, and AWS Config enables support for monitoring, audit, and configuration management. SOC, PCI, ISO, FedRAMP, HIPAA BAA, and DoD Impact Level 4 certifications allows customers to meet a wide range of compliance standards.
- Durable and Highly-Available – DynamoDB maintains data durability and 99.99 percent availability in the event of a server, a rack of servers, or an Availability Zone failure. DynamoDB automatically re-distributes your data to healthy servers to ensure there are always multiple replicas of your data without you needing to intervene.
- Performant – DynamoDB consistently delivers single-digit millisecond latencies even as your traffic volume increases. In addition, DynamoDB Accelerator (DAX) a fully managed, highly available, in-memory cache further speeds up DynamoDB response times from milliseconds to microseconds and can continue to do so at millions of requests per second.
- Manageable – DynamoDB eliminates the need for manual capacity planning, provisioning, monitoring of servers, software upgrades, applying security patches, scaling infrastructure, monitoring, performance tuning, replication across distributed datacenters for high availability, and replication across new nodes for data durability. All of this is done for you automatically and with zero downtime so that you can focus on your customers, your applications, and your business.
- Adaptive Capacity –DynamoDB intelligently adapts to your table’s unique storage needs, by scaling your table storage up by horizontally partitioning them across many servers, or down with Time To Live (TTL) that deletes items that you marked to expire. DynamoDB provides Auto Scaling, which automatically adapts your table throughput up or down in response to actual traffic to your tables and indexes. Auto Scaling is on by default for all new tables and indexes.
Ten years ago, we never would have imagined the lasting impact our efforts on Dynamo would have. What started out as an exercise in solving our own needs in a customer obsessed way, turned into a catalyst for a broader industry movement towards non-relational databases, and ultimately, an enabler for a new class of internet-scale applications.
As we say at AWS, It is still Day One for DynamoDB. We believe we are in the midst of a transformative period for databases, and the adoption of purpose-built databases like DynamoDB is only getting started. We expect that the next ten years will see even more innovation in databases than the last ten. I know the team is working on some exciting new things for DynamoDB – I can’t wait to share them with you over the upcoming months.
How we use phones to avoid human contact
A recent study by Kaspersky Lab has found that 75% of people pick up their connected device to avoid conversing with another human being.
Connected devices are becoming essential to keeping people in contact with each other, but for many they are also a much-needed comfort blanket in a variety of social situations when they do not want to interact with others. A recent survey from Kaspersky Lab has confirmed this trend in behaviour after three-quarters of people (75%) admitted they use a device to pretend to be busy when they don’t want to talk to someone else, showing the importance of keeping connected devices protected under all circumstances.
Imagine you’ve arrived at a bar and you’re waiting for your date. The bar is busy, and people are chatting all around you. What do you do now? Strike up a conversation with someone you don’t know? Grab your phone from your pocket or handbag until your date arrives to keep yourself busy? Why talk to humans or even make eye-contact with someone else when you can stare at your connected device instead?
The truth is, our use of devices is making it much easier to avoid small talk or even be polite to those around us, and new Kaspersky Lab research has found that 72% of people use one when they do not know what to do in a social situation. They are also the ‘go-to’ distraction for people even when they aren’t trying to look busy or avoid someone’s eye. 46% of people admit to using a device just to kill time every day and 44% use it as a daily distraction.
In addition to just being a distraction, devices are also a lifeline to those who would rather not talk directly to another person in day-to-day situations, to complete essential tasks. In fact, nearly a third (31%) of people would prefer to carry out tasks such as ordering a taxi or finding directions to where they need to go via a website and an app, because they find it an easier experience than speaking with another person.
Whether they are helping us avoid direct contact or filling a void in our daily lives, our constant reliance on devices has become a cause for panic when they become unusable. A third (34%) of people worry that they will not be able to entertain themselves if they cannot access a connected device. 12% are even concerned that they won’t be able to pretend to be busy if their device is out of action.
Dmitry Aleshin, VP for Product Marketing, Kaspersky Lab said, “The reliance on connected devices is impacting us in more ways than we could have ever expected. There is no doubt that being connected gives us the freedom to make modern life easier, but devices are also vital to help people get through different and difficult social situations. No matter what your ‘connection crutch’ is, it is essential to make sure your device is online and available when you need it most.”
To ensure your device lifeline is always there and in top health – no matter what the reason or situation – Kaspersky Security Cloud keeps your connection safe and secure:
· I want to use my device while waiting for a friend – is it secure to access the bar’s Wi-Fi?
With Kaspersky Security Cloud, devices are protected against network threats, even if the user needs to use insecure public Wi-Fi hotspots. This is done through transferring data via an encrypted channel to ensure personal data safety, so users’ devices are protected on any connection.
· Oh no! I’m bored but my phone’s battery is getting low – what am I going to do?
Users can track their battery level thanks to a countdown of how many minutes are left until their device shuts down in the Kaspersky Security Cloud interface. There is also a wide-range of portable power supplies available to keep device batteries charged while on-the-go.
· I’ve lost my phone! How will I keep myself entertained now?
Should the unthinkable happen and you lose or have your phone stolen, Kaspersky Security Cloud can track and protect your device from data breaches, for complete peace of mind. Remote lock and locate features ensure your device remains secure until you are reunited.
Five key biometric facts
Due to their uniqueness, fingerprints are being used more and more to quickly identify and ensure the security of customers. CLAUDE LANGLEY, Regional Sales Manager, for Africa at HID Global Biometrics, outlines five facts about the technology.
How many times in a day are you expected to identify yourself? From when you arrive at work you are required to sign in, visiting your bank, receiving healthcare services… The list is endless. When a system knows who you are, you are able to do any number common, everyday activities. Your identity is unique and precious. It is also easily stolen and the target of many hackers across the globe. Technology is constantly evolving alongside the criminal element, always looking for ways to protect data and identity. One such solution happens to be biometrics and it is rapidly gaining traction in our increasingly complex modern world.
Reliable, secure and fundamentally YOU, unique biometric traits such as fingerprints are being used by banks, enterprises and consumers to verify identity. Biometric solutions offer significant identity protection because they use unique biological details to ensure an account is only accessed by the account holder, a door only opened by the owner. Here are five things that are little known about this technology…
- The uncut identity. Your fingerprint is unique to you. Nobody can use a copy of it to impersonate you. Good technology is capable of scanning down into the layers of the fingertip to differentiate unique elements of a person’s fingerprint, this data is then encrypted and used as a key to unlocking whichever physical or virtual door that the biometric system protects.
- The living proof. No, there is nothing to the stories of fingerprints being used without their owner’s knowledge or permission. Biometric solutions can use specific variables to determine if the finger used to access the system is that of a present, living person. A copy or a fake cannot be used to access a cutting-edge biometric solution.
- Easy and convenient. Queues and documents and paperwork may well be a thing of the past should biometrics take a firmer grip of government and banking systems. The process of registering is easy, and access to identity documents and records is yours alone.
- Security blanket. A thousand passwords and a hundred post-it notes stuck on walls and drawers. An excel file with a list of sites and applications and their corresponding passwords, all a thing of the past. Nobody needs to remember their password with biometrics, they only need to show up.
- Anywhere is cool. Schools, airports, networks, offices, homes, toilets, banks, libraries, governments, border controls, immigration services, call centres, hospitals and even clubs and pubs – knowing “who” matters and biometrics can quickly and conveniently confirm your identity where needed.