SAP Business Data Cloud SDK

Table of Contents

• Introduction
• Installation
• Usage
  • Creating a Databricks Client
  • Create or Update Share
  • Create or Update Share CSN
  • Publish a Data Product
  • Unpublish a Data Product

Introduction

With this SDK you can perform Business Data Cloud Connect operations such as:

• Create or update shares
• Create or update shares CSN
• Publish or unpublish Data Products

Open Resource Discovery (ORD) is a protocol that allows applications and services to self-describe their resources and capabilities. It facilitates cross consumption of resources and capabilities between different systems through a common standard interface.

Delta Sharing (DS) is an open protocol for secure sharing of Delta tables. SAP Business Data Cloud support Delta Sharing and allows user data sets, represented as shares, to be shared securely.

This document focus on describing how SAP Business Data Cloud SDK should be used to publish shares and ORD Data Products for downstream consumption within SAP Business Data Cloud.

Installation

To install this SDK use the following pip command:

pip install sap-bdc-connect-sdk

Usage

Create a client

from bdc_connect_sdk.auth import BdcConnectClient
from bdc_connect_sdk.auth import DatabricksClient

databricks_client = DatabricksClient(dbutils, "<recipient-name>")
bdc_connect_client = BdcConnectClient(databricks_client)

• DatabricksClient receives:
  • dbutils parameter, which is a Databricks utility that can be used inside the Databricks notebooks
  • recipient-name parameter, which is the recipient name representing the Business Data Cloud in which the Data Product should be published
• BdcConnectClient receives:
  • DatabricksClient parameter to get information from the Databricks environment (e.g. secrets, api_token, workspace_url_base)

Create or update share

A share is a mechanism for distributing and accessing data across different systems. Creating or updating a share involves including specific attributes, such as @openResourceDiscoveryV1, in the request body, aligning with the Open Resource Discovery protocol. This procedure ensures that the share is properly structured and described according to specified standards, facilitating effective data sharing and management.

from bdc_connect_sdk.auth import BdcConnectClient
from bdc_connect_sdk.auth import DatabricksClient

bdc_connect_client = BdcConnectClient(DatabricksClient(dbutils, "<recipient-name>"))

share_name = "<share-name>"

open_resource_discovery_information = {
    "@openResourceDiscoveryV1": {
        "title": "<title>",
        "shortDescription": "<short-description>",
        "description": "<description>"
    }
}

response = bdc_connect_client.create_or_update_share(
    share_name,
    open_resource_discovery_information
)
print(f"[REQUEST] Create or update share request was executed and returned {response}")

Create or update share CSN

The CSN serves as a standardized format for configuring and describing shares within a network. To create or update the CSN for a share, it's advised to prepare the CSN content in a separate file and include this content in the request body. This approach ensures accuracy and compliance with the CSN interoperability specifications, facilitating consistent and effective share configuration across systems.

Note: The CSN schema, can be efficiently generated using a dedicated script generate_csn_template available in csn_generator file. This script automates the creation of CSN content for Databricks environments based on a Databricks share.

from bdc_connect_sdk.auth import BdcConnectClient
from bdc_connect_sdk.auth import DatabricksClient
from bdc_connect_sdk.utils import csn_generator

bdc_connect_client = BdcConnectClient(DatabricksClient(dbutils, "<recipient-name>"))

share_name = "<share-name>"

csn_schema = csn_generator.generate_csn_template(share_name)

response = bdc_connect_client.create_or_update_share_csn(
    share_name,
    csn_schema
)
print(f"[REQUEST] Create or update CSN request was executed and returned {response if response else 'OK'}")

If the generated CSN needs to be modified for any reason, it can be written to a file for editing, for example.

from bdc_connect_sdk.utils import csn_generator
import json

share_name = "<share-name>"
csn_schema = csn_generator.generate_csn_template(share_name)

file_path = f"csn_{share_name}.json"
with open(file_path, "w") as csn_file:
    csn_file.write(json.dumps(csn_schema, indent=2))

# change the file

with open(file_path, "r") as csn_file:
    changed_csn_schema = csn_file.read()

csn_schema = changed_csn_schema

Publish a Data Product

A Data Product is an abstraction that represents a type of data or data set within a system, facilitating easier management and sharing across different platforms. It bundles resources or API endpoints to enable efficient data access and utilization by integrated systems. Publishing a Data Product allows these systems to access and consume the data, ensuring seamless communication and resource sharing.

from bdc_connect_sdk.auth import BdcConnectClient
from bdc_connect_sdk.auth import DatabricksClient

bdc_connect_client = BdcConnectClient(DatabricksClient(dbutils, "<recipient-name>"))

share_name = "<share-name>"

response = bdc_connect_client.publish_data_product(
    share_name
)
print(f"[REQUEST] Publish Data Product request was executed and returned {response if response else 'OK'}")

Unpublish a Data Product

Unpublishing a Data Product involves withdrawing access to the data, typically when it has been sufficiently consumed or is no longer desired to be shared. This action helps maintain control over data accessibility and ensures that sharing aligns with security and usage policies. By unpublishing, the system can effectively manage the lifecycle of data and its availability to other integrated platforms.

from bdc_connect_sdk.auth import BdcConnectClient
from bdc_connect_sdk.auth import DatabricksClient

bdc_connect_client = BdcConnectClient(DatabricksClient(dbutils, "<recipient-name>"))

share_name = "<share-name>"

response = bdc_connect_client.delete_share(
    share_name
)
print(f"[REQUEST] Delete share request was executed and returned {response if response else 'OK'}")