Amazon Web Services (AWS) this week announced Amazon Security Lake, a service that automatically centralises an organisation’s security data from cloud and on-premises sources into a purpose-built data lake in a customer’s AWS account. This will allow customers to act on security data faster, said AWS CEO Adam Selipsky when he made the announcement during his keynote address at the AWS re:Invent conference in Las Vegas on Tuesday.,
Amazon Security Lake manages data throughout its lifecycle with customisable data retention settings, converts incoming security data to the efficient Apache Parquet format, and conforms it to the Open Cybersecurity Schema Framework (OCSF) open standard. This will make it easier to automatically normalise security data from AWS and combine it with dozens of pre-integrated third-party enterprise security data sources.
Security analysts and engineers can use Amazon Security Lake to aggregate, manage, and optimise large volumes of disparate log and event data to enable faster threat detection, investigation, and incident response. This will allow them to address potential issues quickly, while continuing to utilise their preferred analytics tools.
Initial customers of the service include Salesforce and Tinder.
Tinder is the world’s most popular app for meeting new people, available in 190 countries and more than 40 languages. It’s been downloaded more than 530-million times and led to more than 75-billion matches, see here.
“Because our users entrust Tinder with their information, the security of our application and the privacy of our customers’ data is our top priority,” said Jonathan Walker, DevSecOps manager at Tinder. “Ensuring that we maintain a robust, transparent, and accountable security program is core to our commitment to our customers. Amazon Security Lake has drastically reduced time and money in our efforts to query security events at scale across regions, sources, and events. This has allowed our team to shift our focus away from data engineering to analysing security events within the cloud.”
Customers want greater visibility into security activity across their entire organizations to proactively identify potential threats and vulnerabilities, assess security alerts, respond accordingly, and help prevent future security events. To do this, most organisations rely on log and event data from many different sources (e.g., applications, firewalls, and identity systems) running in the cloud and on premises, each using a unique and often incompatible data format.
To uncover security-related insights, like spotting unauthorized external data transfers for sensitive information or identifying the installation of malware across employee devices, organisations must first aggregate and normalize all this data into a consistent format. Once the data is formatted consistently, customers can analyse it and understand the current level of vulnerability, and then correlate and monitor threats for improved observability.
Customers typically use different security solutions to address specific use cases, such as incident response and security analytics, which often means they duplicate and process the same data multiple times because each solution has its own data stores and format. This is time consuming and costly, slowing down security teams’ ability to detect and respond to issues.
As customers add new users, tools, and data sources, security teams must also spend time managing a complex set of data-access rules and security policies to track how data is used and ensure people can get the information they need. Some security teams create a central repository for all their security data in a data lake, but these systems require specialised skills and can take months to build due to the large amount of log data from different sources, which can run into petabyte scale.
Amazon Security Lake is a purpose-built security data lake that can be created in a few clicks and enables customers to aggregate, normalise, and store data so they can respond to security events faster using their preferred tools. After setup and connections to selected data sources, Amazon Security Lake automatically builds a security data lake in a customer-selected region, which can help customers meet regional data compliance requirements.
After customers choose their data sources, Amazon Security Lake automatically aggregates and normalises data from AWS, combines it with third-party sources that support the OCSF open standard, and optimises it into a format that is easy to store and query. Amazon Security Lake automatically orchestrates the end-to-end process from data lake creation and data aggregation to normalisation and integration.
The new service builds the security data lake using Amazon Simple Storage Service (Amazon S3) and AWS Lake Formation to automatically set up security data lake infrastructure in a customer’s AWS account, providing full control and ownership over security data. Once ingested and normalised, customers can use their preferred security and analytics tools, including Amazon Athena, Amazon OpenSearch, and Amazon SageMaker, along with leading third-party solutions (e.g., IBM, Splunk, or Sumo Logic) to make it faster and easier to capture broader and deeper analytics from AWS and more than 50 third-party (e.g., Cisco, CrowdStrike, and Palo Alto Networks) and customer data sources.
Jon Ramsey, vice president for security services at AWS, said: “Customers must be able to quickly detect and respond to security risks so they can take swift action to secure data and networks, but the data they need for analysis is often spread across multiple sources and stored in a variety of formats. Customers tell us they want to take action on this data faster to improve their security posture, but the process of collecting, normalizing, storing, and managing this data is complex and time consuming.
“Amazon Security Lake lets customers of all sizes securely set up a security data lake with just a few clicks to aggregate logs and event data from dozens of sources, normalise it to conform with the OCSF standard, and make it more broadly usable so customers can take action quickly using their security tools of choice. With Amazon Security Lake, customers get superior visibility and control, with help from the largest ecosystem of security partners and solutions.”
Salesforce, the global CRM leader, empowers companies of every size and industry to digitally transform and create a 360° view of their customers. Vikram Rao, chief trust officer at Salesforce, said: “Salesforce builds security into everything we do. As we scale to support the growth of our global customer base, our Detection and Response teams analyse petabytes of security logs to catch malicious activity and protect customer data.
“Amazon Security Lake streamlines that work by unifying security logs and events from AWS and other cloud providers—reducing time spent on log onboarding and coverage so that our engineers can focus on proactive prevention and incident response.”
* For more information about Amazon Security Lake, visit aws.amazon.com/security-lake.