Skip to content

hana cloud hdl getting started tutorials update #22998

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -14,43 +14,41 @@ primary_tag: software-product>sap-hana-cloud
- If you don't have a trial account yet, sign up [here](https://www.sap.com/cmp/td/sap-hana-cloud-trial.html).

## You will learn
- What the components Data Lake, IQ and Data Lake Files are
- What the components Data Lake, Data Lake Relational Engine and Data Lake Files are
- How to differentiate between managed and standalone data lakes
- How to provision a standalone data lake
- How to change the storage capacity of data lake instances
---

### Introduction to SAP HANA Cloud, data lake

SAP HANA Cloud, data lake is one of the components that make up SAP HANA Cloud. It's composed of two different components, the **Data Lake, IQ** and **Data Lake Files**. The Data Lake, IQ enables efficient storage and high-performance analytics of relational data at petabyte scale. It leverages the technology of SAP IQ, which many customers already know from the on-premise world.
SAP HANA Cloud, data lake is one of the components that make up SAP HANA Cloud. It's composed of two different components, the **Data Lake Relational Engine** and **Data Lake Files**.

With SAP HANA Cloud, data lake, you can ingest data from multiple sources, as well as non-SAP cloud storage providers, at high speed. It's an integrated part of SAP HANA Cloud, providing common security mechanisms, tenancy models, and tools operating within SAP HANA Cloud.

SAP HANA Cloud, data lake was built to be scalable and accommodate increases in data volume, in user count, and the complexity of workloads. This tutorial will focus on using SAP HANA Cloud, data lake as a standalone component of SAP HANA Cloud.

<!-- border -->![Standalone vs. Managed data lake](ss-01-standalone-vs-managed-dl.png)
![Standalone vs. Managed data lake](ss-01-standalone-vs-managed-dl.png)


### Data Lake, IQ and Data Lake Files
### Data Lake, Data Lake Relational Engine and Data Lake Files

The two main components of SAP HANA Cloud, data lake are:


1. **Data Lake, IQ**: Data Lake, IQ is an efficient disk-optimized relational store, based on SAP IQ on-premise. It's enabled by default when you provision a data lake instance, whether it's standalone or managed by an SAP HANA database instance within SAP HANA Cloud.
1. **Data Lake Relational Engine**: Data Lake Relational Engine enables efficient storage of structured data and high-performance analytics of relational data at petabyte scale. It's enabled by default when you provision a data lake instance, whether it's standalone or managed by an SAP HANA database instance within SAP HANA Cloud.

2. **Data Lake Files**: Data Lake Files service provides a secure, managed object storage to host structured, semi-structured and unstructured data files. You can query files in a relational format stored in data lake files by using the Data Lake's **SQL on Files** feature. This allows you to analyze the data with a low-cost strategy, given that this data has an unknown value. It is also easy to share this data with other processing tools. It's enabled by default when you provision a data lake instance, whether it's a standalone or managed by an SAP HANA database instance.

The Data Lake Files component is currently not available in the free tier model or trial.




### How to use a data lake

SAP HANA Cloud, data lake can be provisioned and used in two different ways:


- **Managed data lake**: the data lake is provisioned as part of the SAP HANA Cloud, SAP HANA database provisioning. A remote connection between the SAP HANA database and the Data Lake, IQ is then automatically created. The easiest way to access the data in a managed data lake is to use SAP HANA virtual tables using the SAP HANA Database Explorer. You can, however, also access the data lake independently.
- **Managed data lake**: the data lake is provisioned as part of the SAP HANA Cloud, SAP HANA database provisioning. A remote connection between the SAP HANA database and the data lake Relational Engine is then automatically created. The easiest way to access the data in a managed data lake is to use SAP HANA virtual tables using the SAP HANA Database Explorer. You can, however, also access the data lake independently.

- **Standalone data lake**: the data lake is provisioned independently of any other SAP HANA Cloud services, and therefore it is not automatically connected to any other SAP HANA Cloud instances you might have. You can access your data within the data lake with SAP HANA Database Explorer, dbisql, isql, or any of the supported data lake client interfaces.

Expand All @@ -73,15 +71,15 @@ To create a standalone data lake instance in SAP HANA Cloud,

4. Click on **Create**, then on **Data Lake**.

<!-- border -->![Create data lake](ss-02-create-data-lake.png)
![Create data lake](ss-02-create-data-lake.png)

5. A new tab will open after finishing the last step - the **SAP HANA Cloud Central** provisioning wizard.

6. Here in **Step 1** you must choose the **Type** of instance to create **SAP HANA Cloud, Data Lake** instance.

7. Click on **Next Step**.

<!-- border -->![Create data lake](ss-03-create-data-lake_1.png)
![Create data lake](ss-03-create-data-lake_1.png)


### Describe the instance
Expand All @@ -94,12 +92,13 @@ Here you will be configuring the general preferences.

2. You can insert a description of this instance on the **Description** field.

<!-- border -->![General](ss-04-general.png)
![General](ss-04-general.png)

3. Next, choose which connections are allowed to access your data lake instance. That means choosing if you allow access to your instance from outside of the SAP Business Technology Platform.

> You can either limit it to SAP Business Technology Platform by denying all IP addresses, or allow specific applications to access it by inserting one or more specific IP addresses. Finally, you can allow all connections from all IP addresses.

![Connections](ss-connections.png)

4. Click on **Next Step** on the left-hand side of the wizard.

Expand All @@ -108,14 +107,14 @@ Here you will be configuring the general preferences.
### Define storage settings


In this step, you can choose to activate the Data Lake, IQ component. Once you enable the option, you will see more options to customize the data lake.
In this step, you can choose to enable the data lake Relational Engine component. Once you enable the option, you will see more options to customize the data lake.


1. Click on the toggle button to **Enable** the Data Lake, IQ component.
1. Click on the toggle button to **Enable** the data lake Relational Engine component.

2. Insert a password in the **Administrator Password** field. Please make sure to use a strong password. This password will be associated with the automatically generated HDLADMIN user, which has full powers within your data lake, so make sure not to forget it.

<!-- border -->![Storage settings]( ss-05-storage setting.png)
![Storage settings](ss-05-storage-setting.png)

3. Confirm it by typing it again on the **Confirm Administrator Password** field.

Expand All @@ -128,17 +127,17 @@ In this step, you can choose to activate the Data Lake, IQ component. Once you e
7. Click on **Next Step** to continue.


### Define advanced data lake IQ settings
### Define advanced data lake Relational Engine settings

1. Now you need to choose the initialization mode, which is whether your standalone data lake will be more compatible with SAP HANA or with SAP IQ.

<!-- border -->![Data lake Settings](ss-06-Data_Lake_IQ.png)
![Data lake Settings](ss-06-Data_Lake_RE.png)

>
>The right choice of compatibility depends on how you wish to use your data lake. If most of your use will be associated with SAP HANA, then choose that option. On the other hand, if you wish to move data from SAP IQ, then you should choose that option.


<!-- border -->![Advanced DL Settings1](ss-07-Data lake-advanced-settings-1.png)
![Advanced DL Settings1](ss-07-Data-lake-advanced-settings-1.png)

2. If you choose compatibility with SAP IQ, then you can also choose the **Collation**.

Expand All @@ -148,7 +147,7 @@ In this step, you can choose to activate the Data Lake, IQ component. Once you e

5. Click on **Review and Create** and you are done! Your first standalone data lake will be created, and you can monitor its status to see when it will be ready to be used. This process usually takes a few minutes.

<!-- border -->![Creation Complete](ss-08-data-lake-creation-complete.png)
![Creation Complete](ss-08-data-lake-creation-complete.png)


### Edit storage size
Expand All @@ -161,13 +160,12 @@ To resize your data lake, follow these steps:

1. In the SAP HANA Cloud Central, locate your existing data lake.

2. Click on the three dots button on the right side of the same line to open the menu.
2. Click on the three dots button on the right side of the same line to open the menu and then click on **Manage Configuration**.

3. Click on **Edit**.
3. Click on **Edit** in the top right corner.

4. The editing wizard will open and show you the options currently available.


By changing storage capacity, the instance will be forced to restart if it was previously running. When changing compute size, single-worker-node systems will require a restart. Multi-worker-node systems will require a restart when the worker-node's size changes, but do not require a restart when the number of workers changes if the size per worker-node is held constant.

>**Storage service level and the additional settings found under the initialization section cannot be modified after provisioning and must be configured appropriately at the time when the data lake is created.**
Expand All @@ -179,7 +177,4 @@ To resize your data lake, follow these steps:






---
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tutorials/hana-cloud-hdl-getting-started-1/ss-04-general.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ Let's get started. Follow these steps:



>**Attention**: You should make a slight adjustment in the above query if you have provisioned the Data Lake, IQ instance in maximum compatibility mode with SAP HANA Cloud, SAP HANA database. This mode does not support **CHAR** data type. Instead, you can alter the query by using **VARCHAR** data type.
>**Attention**: You should make a slight adjustment in the above query if you have provisioned the data lake Relational Engine instance in maximum compatibility mode with SAP HANA Cloud, SAP HANA database. This mode does not support **CHAR** data type. Instead, you can alter the query by using **VARCHAR** data type.
>
>For more information on database compatibility modes, please see our [technical documentation](https://help.sap.com/viewer/a896c6a184f21015b5bcf4c7a967df07/LATEST/en-US/82cdda1398644f4a95ec402f3144bcea.html).

Expand Down Expand Up @@ -258,7 +258,7 @@ Let's get started. Follow these steps:

```

>**Attention**: You should make a slight adjustment in the above query if you have provisioned the Data Lake, IQ instance in maximum compatibility mode with SAP HANA Cloud, SAP HANA database. This mode does not support **CHAR** data type. Instead, you can alter the query by using **VARCHAR** data type.
>**Attention**: You should make a slight adjustment in the above query if you have provisioned the data lake Relational Engine instance in maximum compatibility mode with SAP HANA Cloud, SAP HANA database. This mode does not support **CHAR** data type. Instead, you can alter the query by using **VARCHAR** data type.
>
>For more information on database compatibility modes, please see our [technical documentation](https://help.sap.com/viewer/a896c6a184f21015b5bcf4c7a967df07/LATEST/en-US/82cdda1398644f4a95ec402f3144bcea.html).

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Let's see in detail the procedures to access different monitoring views.

### Explore workload monitoring using SAP HANA Database Explorer

Using **SAP HANA Database Explorer** connected to your data lake, you can display a menu with list of all available Data Lake, IQ monitoring views. This can be done by running the command in the SQL Console:
Using **SAP HANA Database Explorer** connected to your data lake, you can display a menu with a list of all available data lake Relational Engine monitoring views. This can be done by running the command in the SQL Console:

```SQL
SELECT * FROM IQMONVIEWS
Expand Down Expand Up @@ -97,7 +97,7 @@ Similarly, you can go to the SAP HANA Cloud Central when the remaining storage s

### Differentiate between sp_iqsysmon and monitoring views

The monitoring views and **`sp_iqsysmon`** are intended for very different reporting timescales. The **`sp_iqsysmon`** is a system stored procedure that allows you to define the time interval for monitoring using **Start** and **Stop** parameters. Data lake, IQ continuously monitors performance and displays the latest performance data on the monitoring views. Rather than monitoring over a defined period, monitoring views start gathering performance metrics at server startup and reports the current state of the system.
The monitoring views and **`sp_iqsysmon`** are intended for very different reporting timescales. The **`sp_iqsysmon`** is a system stored procedure that allows you to define the time interval for monitoring using **Start** and **Stop** parameters. Data lake Relational Engine continuously monitors performance and displays the latest performance data on the monitoring views. Rather than monitoring over a defined period, monitoring views start gathering performance metrics at server startup and reports the current state of the system.
>For example, you can monitor your system at a 30-second time interval for two hours using **`sp_iqsysmon`**. On the other hand, monitoring views provide a current snapshot of system heath and performance metrics.

It is beneficial to monitor changes in performance over time and this can be achieved by regularly checking the monitoring views at regular intervals. Any metrics moving out of their normal ranges should be evaluated.
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ primary_tag: products>sap-hana-cloud
---

# Perform Backup and Recovery of Standalone Data Lake in SAP HANA Cloud
<!-- description --> Learn to backup and recover the data in your standalone data lake in SAP HANA Cloud using a trial account.
<!-- description --> Learn to backup and recover the data in your standalone data lake in SAP HANA Cloud using a production account.

## You will learn
- The backup schedule for the database
Expand Down Expand Up @@ -81,7 +81,7 @@ When creating a service request for data lake recovery, be sure to complete the

>The **Service ID** is the same as the **Instance ID** for your data lake. This can be found from the SAP HANA Cloud Central.

<!-- border -->![Instance ID](ss-04-instance-ID.png)
![Instance ID](ss-04-instance-ID.png)


>In this tutorial, you have learned about the backup and recovery process of your database using SAP HANA Cloud, data lake. Make sure to take note of the frequency of the backups, their time of occurrence and the process for recovery.
Expand All @@ -93,5 +93,4 @@ For more learning materials on **SAP HANA Cloud**, [click here](https://communit




---
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.