With advances in in-memory processing, it is no wonder that the SAP HANA database system is topping the charts for speed and agility. However, SAP IQ, with it’s more traditional columnar storage, is still massively respected for its capability of processing big data and analytics workloads. So respected in fact, that with it’s 2014 update, 45 of the top 50 banks and security firms were using SAP ASE and SAP IQ to manage their data! As years go by and the digital economy grows, cloud compatibility has become a necessity, and SAP has predictably responded. SAP IQ is now available in the cloud with a new name – SAP HANA Data Lake.
In this blog, we’ll discuss the benefits of SAP IQ technology and it’s move to the cloud with SAP HANA Cloud.
WHAT IS SAP IQ?
Originally a Sybase product, SAP acquired the relational database management system (RDBMS) in 2010. A relational database management system is simply a place to store and process structured information. Data are organised into tables according to their type or attribute, and allows for relationships to be created between data types. For example, a relational database can link data from a customer information table like the customer’s name, address and billing information, to data from a customer transaction table like product orders, based on the shared unique ID attributed to the specific customer. When a company’s order processing application submits an order to the database, the database can go to the customer order table, pull the correct information about the product order, and use the customer ID from that table to look up the customer’s billing and shipping information in the customer information table. The warehouse can then pull the correct product, the customer can receive timely delivery of the order, and the company can get paid. Many of the documents businesses run to track inventory, sales, finance, or even perform financial projections come from a relational database operating behind the scenes.
SAP IQ, with its impressive speed, power, and security, supports extreme-scale enterprise data warehousing and Big Data analytics, with the following features:
- Proven columnar relational database server
- Compliant with SQL standards
- Designed to handle petabyte-scale data volumes
- Reduces storage volumes by more than 70% for lower storage costs and improved performance
SAP IQ TECHNOLOGY BENEFITS
Read and write performance
SAP IQ’s query response and data load times are exceptionally fast because read and write operations are directed to the memory caches, not directly to disk. Through Distributed Query Processing, the multi-threaded bulk loader and automatic data indexing on load, read and write operations are further optimised and accelerated.
In the more recent versions of SAP IQ, an in-memory, write-optimised store, known as the Row-Level Versioning (RLV) store allows for concurrent updates to different rows of the same table to make data available quickly for real-time analysis.
Compression
As soon as data is loaded into SAP IQ, it is automatically compressed up to 50% less than the raw data size, which means you need a lot less storage space than other databases.
SAP IQ IN THE CLOUD
SAP IQ finally made its appearance in the cloud in April 2020 when SAP launched SAP HANA Data Lake, which is in essence a SAP IQ database deployed in the cloud.
A data lake is a repository for all types of data. From this repository, data can be examined, accessed, and used to make data-backed business decisions.
The SAP HANA Cloud data architecture follows a tiered approach in which mission-critical data is stored in-memory in the top tier for real-time access, and below that, data not needed as frequently, but still important, is stored in the HANA Data Lake for easy access when it is needed.
HANA Data Lake offers exceptional compression for massive amounts of data, making it a very cost-effective solution for storing data. It’s scalability (you can add nodes and other components as your requirements grow) and elasticity (you can increase CPUs for peak periods of usage through the year), coupled with HANA cloud security, and its ability to store structured, semi-structured, and unstructured data files increases its flexibility. Additionally, the close integration between SAP HANA Cloud and SAP HANA Data Lake means that you can easily combine data within the two systems whilst benefiting from the strengths of both.
SAP IQ based HANA Data Lake can also be implemented as a standalone solution. This allows existing customers to transition easily from on-premise SAP IQ to SAP Hana Data Lake.
HANA DATA LAKE CAPABILITIES
- Integrated into HANA Cloud Instance
- Automatically provisioned and administered with HANA Cloud
- Based on SAP IQ technology
- Elastic scale, independent of HANA DB
- Designed to scale up petabytes of data
- High speed ingestion enablement
- Ability to analyse data with excellent performance
- Access to cloud storage (e.g. AWS S3, GCP Cloud Storage)
HANA DATA LAKE BENEFITS
- Ingest any data from cloud or on-premise data sources
- Easy to setup and use (single access layer in HANA Cloud)
- Low TCO
- Fast analytic processing through columnar architecture
In summary, businesses looking for a cost-effective data lake solution in the cloud that is able to store and process massive volumes of data and still perform optimally, SAP HANA Data Lake is the perfect solution.