--- name: alibabacloud-dataworks-infra-manage description: | DataWorks Infrastructure Management: Create and query operations for Data Sources (50 types), Compute Resources, and Serverless Resource Groups, plus connectivity testing and resource group binding/unbinding. Uses aliyun CLI to call dataworks-public OpenAPI (2024-05-18). Trigger keywords: DataWorks data source, compute resource, resource group, datasource, data source, compute resource, resource group, mysql/hologres/maxcompute data source, holo/mc/flink resource, Serverless resource group, DataWorks infra, create/list datasource, DW environment config, infrastructure initialization, connect database to DataWorks, database connection failure, configure holo/mc resource. Not triggered: data development tasks, scheduling configuration, MaxCompute table management, data integration tasks, ECS/RDS/OSS operations, workspace member management, data quality monitoring, data lineage, data preview. --- # DataWorks Infrastructure Management Unified management of **Data Sources**, **Compute Resources**, and **Resource Groups** in Alibaba Cloud DataWorks workspaces, supporting create and query operations. ## Architecture ``` DataWorks ├── Workspaces ─── Query and search workspaces │ ├── Data Sources ─── 50 types: MySQL, Hologres, MaxCompute, ... │ └── Compute Resources ─── Hologres, MaxCompute, Flink, Spark └── Resource Groups ─── Serverless resource group management (cross-workspace) Dependencies: Workspace ◀── Data Sources, Compute Resources (must belong to a workspace) Workspace ◀── Resource Groups (associated via binding; one resource group can bind to multiple workspaces) Connectivity Test ──depends on──▶ Resource Group (must be bound to the workspace of the data source) Standard Mode ──requires──▶ Dev (Development) + Prod (Production) dual data sources and compute resources ``` --- ## Global Rules ### Prerequisites 1. **Aliyun CLI >= 3.3.1**: `aliyun version` (Installation guide: [references/cli-installation-guide.md](references/cli-installation-guide.md)) 2. **First-time use**: `aliyun configure set --auto-plugin-install true` 3. **jq** (required for resource group operations): `which jq` 4. **Credential status**: `aliyun configure list`, verify valid credentials exist 5. **DataWorks edition**: Basic edition or above required > **Security Rules**: **DO NOT** read/print/echo AK/SK values, **DO NOT** let users input AK/SK directly, **ONLY** use `aliyun configure list` to check credential status. ### Command Formatting - **User-Agent (mandatory)**: All `aliyun` CLI commands **must** include the `--user-agent AlibabaCloud-Agent-Skills` parameter to identify the source. - **Single-line commands**: When executing Bash commands, **must** construct as a **single-line string**; do not use `\` for line breaks. - **jq step-by-step execution**: First execute the `aliyun` command to get JSON, then format with `jq` (to avoid multi-line security prompts). - **Endpoint mandatory**: When specifying the `--region` parameter, you **must** also add `--endpoint dataworks..aliyuncs.com`. Not needed when `--region` is not specified. ### Parameter Confirmation > Before executing any command, all user-customizable parameters **must** be confirmed by the user. Do not assume or use default values. > **Exception**: When the user has **explicitly specified** parameter values in the conversation, use them directly without re-confirmation. **Resource group related parameters (mandatory user selection)**: VPC, VSwitch, Resource Group ID (for binding/connectivity testing) — involve networking and billing, **DO NOT auto-select**; must display a list for the user to explicitly choose. Confirm even if there is only one option. ### ⚠️ Write API Execution Gate — MUST Check Before Every Write Operation > **MANDATORY**: Before calling **any** Write API (Create / Update / Delete / Bind / Unbind / Associate / Dissociate / Test), you **MUST** perform the following checks in order: > > 1. **Scan the entire SKILL.md** for a Security Restriction or Disabled Operations notice that mentions the target API or module. > 2. **If a restriction exists**: **BLOCK the operation immediately**. Do NOT call the API. Respond to the user with: > - What operation is blocked and why > - The recommended alternative (e.g., use the DataWorks console, contact administrator) > 3. **If no restriction exists**: Proceed normally with parameter confirmation and execution. > > **This check is NOT optional.** It applies to every single write operation without exception. Never skip this step. > > **Quick Reference — Blocked APIs in this skill**: > | Module | Blocked APIs | Reason | > |--------|-------------|--------| > | Data Sources (Module 1) | `UpdateDataSource`, `DeleteDataSource` | Prevent accidental data loss, credential exposure, disruption of running tasks | > | Compute Resources (Module 2) | `UpdateComputeResource`, `DeleteComputeResource` | Prevent disruption of running development and scheduling tasks | > > **Allowed Write APIs**: `CreateDataSource`, `CreateComputeResource`, `CreateResourceGroup`, `AssociateProjectToResourceGroup`, `DissociateProjectFromResourceGroup`, `TestDataSourceConnectivity` ### RAM Permissions All operations require `dataworks:` permissions. Creating resource groups additionally requires `AliyunBSSOrderAccess` and `vpc:DescribeVpcs`, `vpc:DescribeVSwitches`. > Full permission matrix: [references/ram-policies.md](references/ram-policies.md) --- ## Quick Start: New Workspace Infrastructure Initialization When the user is **unsure about specific operations** or has vague requirements, guide them through the following process: 1. **Environment check** — Check CLI and credentials per Prerequisites 2. **Confirm workspace** — Use `ListProjects` to locate the workspace, `GetProject` to confirm the mode (Simple/Standard) 3. **Create compute resources** — Guide engine type selection; the system will **automatically create corresponding data sources**. Standard Mode requires Dev+Prod pairs. Only pure storage-type data sources (MySQL, Kafka, etc.) need separate data source creation 4. **Create/bind resource groups** — Query existing resource groups → let user select → bind. Guide creation when no resource groups are available 5. **Test connectivity** — Test with bound resource groups; when all pass, inform "Infrastructure configuration complete" > After each step, proactively suggest the next action. --- ## Next Step Guidance After each write operation is completed and verified, **proactively suggest** follow-up actions: | Completed Operation | Recommended Next Step | |-----------|-----------| | Create compute resource | Standard Mode: "Create the corresponding Dev resource?"; "Test connectivity?" | | Create data source separately | "Test connectivity?"; Standard Mode: "Create Dev/Prod environment data sources?" | | Create resource group | "Bind to a workspace?" | | Bind resource group | "Test data source connectivity?" | | Connectivity test passed | "Infrastructure is ready." | | Connectivity test failed | Analyze the error cause, guide the fix | | Unbind resource group | "Bind to another workspace?" | --- ## Trigger Rules **Trigger scenarios**: Data source create/query, compute resource create/query, resource group management, infrastructure initialization, colloquial aliases (DW database connection failure, configure holo/mc resources, create rg) **Not triggered**: Data development tasks, scheduling configuration, MaxCompute table management, data integration tasks, ECS/RDS/OSS, workspace member management, data quality/lineage/preview. Standalone workspace queries are handled by the `alibabacloud-dataworks-workspace-manage` skill. ## Interaction Flow All operations follow: **Identify module → Environment check → Collect parameters → Execute command → Verify result → Guide next step** Common aliases: DW=DataWorks, holo=Hologres, mc/MC/odps=MaxCompute, pg=PostgreSQL, rg=Resource Group, ds=Data Source, RDS=InstanceMode MySQL/PG/SQLServer, ADB=AnalyticDB Naming suggestions: Data source `{type}_{business}_{purpose}`, Compute resource `{type}_{business}`, Resource group `dw_{purpose}_rg_{env}` --- # Module 0: Workspace Query > If the `alibabacloud-dataworks-workspace-manage` skill is available, prefer using it for workspace queries. The following is only a fallback. ```bash aliyun dataworks-public ListProjects --user-agent AlibabaCloud-Agent-Skills --Status Available --PageSize 100 ``` When searching by name, first get the full list then filter `.PagingInfo.Projects[]` by Name/DisplayName using `jq`. --- # Module 1: Data Source Management Supports **50** data source types. See [references/data-sources/README.md](references/data-sources/README.md) for details. > **When do you need to create a data source separately?** Creating a compute resource (Module 2) will **automatically create the corresponding data source**. Only pure storage-type databases (MySQL, PostgreSQL, Kafka, MongoDB, etc.) need separate creation. > **Note**: The following types do not currently support OpenAPI: `hdfs` Connection modes: **UrlMode** (self-hosted databases, requires host/port) or **InstanceMode** (Alibaba Cloud managed instances, requires instanceId). When unsure, proactively ask the user. InstanceMode is preferred. > Instance query APIs: [references/data-sources/instance-apis.md](references/data-sources/instance-apis.md) ## ⚠️ Security Restriction — See Write API Execution Gate (Global Rules) for mandatory pre-check > **IMPORTANT**: The `DeleteDataSource` and `UpdateDataSource` APIs are supported by the DataWorks service, but this skill has **disabled** modifying or deleting data sources for security reasons. **Before attempting any write operation, the agent MUST check the Write API Execution Gate section.** > If you need to modify or delete a data source, please use the DataWorks console directly or contact your administrator. ### Connection Mode Quick Reference `ConnectionPropertiesMode` selection determines required fields. InstanceMode is preferred when both are available. | Mode | Types | Count | |------|-------|-------| | **Both** | mysql, postgresql, sqlserver, polardb, polardbo, polardb-x-2-0, apsaradb_for_oceanbase, drds, starrocks, analyticdb_for_mysql, analyticdb_for_postgresql, milvus, mongodb, redis, elasticsearch, kafka | 16 | | **InstanceMode only** | hologres, dlf, opensearch | 3 | | **UrlMode only** | oracle, mariadb, dm, db2, tidb, vertica, gbase8a, kingbasees, saphana, snowflake, maxcompute, hive, clickhouse, doris, selectdb, redshift, hbase, lindorm, oss, s3, ftp, ssh, tablestore, memcache, graph_database, datahub, loghub, restapi, salesforce, httpfile, bigquery | 31 | > `hdfs` — not supported via OpenAPI. > Full details: [references/data-sources/README.md](references/data-sources/README.md) ### Workspace Mode > **Environment note**: **Prod (Production)** is for production data processing; **Dev (Development)** is for development and debugging, physically isolated from production. `aliyun dataworks-public GetProject --user-agent AlibabaCloud-Agent-Skills --Id ` — check `DevEnvironmentEnabled`: - `false` → Simple Mode (1 data source, envType=Prod) - `true` → Standard Mode (2 data sources, Dev + Prod, physically isolated) > Full mode comparison: [references/data-sources/README.md](references/data-sources/README.md) ## Task 1.1: Create Data Source (CreateDataSource) ```bash aliyun dataworks-public CreateDataSource --user-agent AlibabaCloud-Agent-Skills [--region --endpoint dataworks..aliyuncs.com] --ProjectId --Name --Type --ConnectionPropertiesMode --ConnectionProperties '' --Description "" ``` **ConnectionProperties common structure**: - **UrlMode**: `{"envType":"Prod","address":[{"host":"","port":}],"database":"","username":"","password":""}` - **InstanceMode**: `{"envType":"Prod","instanceId":"","regionId":"","database":"","username":"","password":""}` > Special type structures (Oracle, MaxCompute, HBase, etc.): see [references/data-sources/](references/data-sources/) per-type docs > Cross-account data source configuration: [references/cross-account-datasources.md](references/cross-account-datasources.md) ## Task 1.2: Get Data Source (GetDataSource) ```bash aliyun dataworks-public GetDataSource --user-agent AlibabaCloud-Agent-Skills --Id [--region --endpoint dataworks..aliyuncs.com] ``` ## Task 1.3: List Data Sources (ListDataSources) ```bash aliyun dataworks-public ListDataSources --user-agent AlibabaCloud-Agent-Skills --ProjectId [--Types '["mysql"]'] [--EnvType ] [--PageNumber 1] [--PageSize 20] ``` > Returns nested structure `DataSources[].DataSource[]`; Name/Type are in the outer layer, Id/Description in the inner layer. ## Task 1.4: Test Connectivity (TestDataSourceConnectivity) **Process**: Query resource group list → **Let user select** a resource group → Execute test. ```bash # Step 1: Query project resource groups aliyun dataworks-public ListResourceGroups --user-agent AlibabaCloud-Agent-Skills --ProjectId # Step 2: Execute test after user selects a resource group aliyun dataworks-public TestDataSourceConnectivity --user-agent AlibabaCloud-Agent-Skills --DataSourceId --ProjectId --ResourceGroupId "" ``` > If error `"resourceGroupId is not in the project"`, the resource group needs to be bound first (confirm with user, then execute `AssociateProjectToResourceGroup`). --- # Module 2: Compute Resource Management Supports Hologres, MaxCompute, Flink, Spark, and other types. The system will **automatically create corresponding data sources** upon creation. ## ⚠️ Security Restriction — See Write API Execution Gate (Global Rules) for mandatory pre-check > **IMPORTANT**: For security reasons, this skill does **NOT** support **modifying** or **deleting** compute resources. **Before attempting any write operation, the agent MUST check the Write API Execution Gate section.** These operations are disabled to prevent: > - Accidental data loss or service interruption > - Disruption of running data development and scheduling tasks > - Unintended changes to production compute resource configurations > > If you need to modify or delete a compute resource, please use the DataWorks console directly or contact your administrator. ### authType Rules - **Dev environment**: `authType` is fixed as `Executor` - **Prod environment**: Options are `PrimaryAccount` (recommended), `TaskOwner`, `SubAccount`, `RamRole`. Default recommendation is `PrimaryAccount` unless user has special requirements > authType details and guidance: [references/compute-resources/README.md](references/compute-resources/README.md) ### Type-Specific Notes - **Hologres**: Only supports **InstanceMode**, requires `instanceId`, `securityProtocol` - **MaxCompute**: Only supports **UrlMode**, requires `project`, `endpointMode` > Full ConnectionProperties examples: [references/compute-resources/README.md](references/compute-resources/README.md) ## Task 2.1: Create Compute Resource (CreateComputeResource) ```bash aliyun dataworks-public CreateComputeResource --user-agent AlibabaCloud-Agent-Skills [--region --endpoint dataworks..aliyuncs.com] --ProjectId --Name --Type --ConnectionPropertiesMode --ConnectionProperties '' [--Description ""] ``` > After creation, use `ListDataSources` to verify the corresponding data source was auto-generated. ## Task 2.2: Get Compute Resource (GetComputeResource) ```bash aliyun dataworks-public GetComputeResource --user-agent AlibabaCloud-Agent-Skills --Id --ProjectId ``` ## Task 2.3: List Compute Resources (ListComputeResources) ```bash aliyun dataworks-public ListComputeResources --user-agent AlibabaCloud-Agent-Skills --ProjectId [--Name ] [--EnvType ] [--PageSize 20] [--SortBy CreateTime] [--Order Desc] ``` > Returns nested structure `ComputeResources[].ComputeResource[]`; Name/Type are in the outer layer, Id in the inner layer. --- # Module 3: Resource Group Management Manages the full lifecycle of Serverless resource groups. ## Task 3.1: Create Resource Group (CreateResourceGroup) > Requires `AliyunBSSOrderAccess` permission. **Interaction flow** (let user choose at each step, DO NOT auto-select): 1. **Query and select VPC**: ```bash aliyun vpc DescribeVpcs --user-agent AlibabaCloud-Agent-Skills --RegionId "" --PageSize 50 ``` If the list is empty, guide the user to create a VPC; **DO NOT** auto-create. 2. **Query and select VSwitch**: ```bash aliyun vpc DescribeVSwitches --user-agent AlibabaCloud-Agent-Skills --RegionId "" --VpcId "" --PageSize 50 ``` 3. **Confirm name and specification** → Execute creation: ```bash aliyun dataworks-public CreateResourceGroup --user-agent AlibabaCloud-Agent-Skills [--region --endpoint dataworks..aliyuncs.com] --Name "" --PaymentType PostPaid --VpcId "" --VswitchId "" --ClientToken "$(uuidgen 2>/dev/null || echo "token-$(date +%s)")" --Remark "Created by Agent" ``` After creation, poll `GetResourceGroup` until status becomes `Normal` (every 10 seconds, up to 10 minutes). ## Task 3.2: Get Resource Group (GetResourceGroup) ```bash aliyun dataworks-public GetResourceGroup --user-agent AlibabaCloud-Agent-Skills --Id "" ``` ## Task 3.3: List Resource Groups (ListResourceGroups) ```bash aliyun dataworks-public ListResourceGroups --user-agent AlibabaCloud-Agent-Skills [--ProjectId ] [--Statuses '["Normal"]'] --PageSize 100 ``` ## Task 3.4: Bind Resource Group (AssociateProjectToResourceGroup) **Process**: Query available resource groups → **Display list for user to select** → Bind after user confirms. ```bash aliyun dataworks-public AssociateProjectToResourceGroup --user-agent AlibabaCloud-Agent-Skills --ResourceGroupId "" --ProjectId "" ``` ## Task 3.5: Query Binding Relationships ```bash aliyun dataworks-public ListResourceGroupAssociateProjects --user-agent AlibabaCloud-Agent-Skills --ResourceGroupId "" ``` ## Task 3.6: Unbind Resource Group (DissociateProjectFromResourceGroup) ```bash aliyun dataworks-public DissociateProjectFromResourceGroup --user-agent AlibabaCloud-Agent-Skills --ResourceGroupId "" --ProjectId "" ``` --- ## Success Verification After all write operations, use the corresponding Get/List command to verify the result. ## Common Errors | Error Code | Solution | |--------|----------| | Forbidden.Access / PermissionDenied | Check RAM permissions, see [references/ram-policies.md](references/ram-policies.md) | | InvalidParameter | Check ConnectionProperties JSON and required parameters | | EntityNotExists | Verify the ID and Region are correct | | QuotaExceeded | Delete unused resources or request a quota increase | | Duplicate* | Use a different name | ## Region Common: `cn-hangzhou`, `cn-shanghai`, `cn-beijing`, `cn-shenzhen`. Endpoint: `dataworks..aliyuncs.com` > Full list: [references/related-apis.md](references/related-apis.md) ## Best Practices 1. **Query before action** — Confirm current state before create operations 2. **Manage by environment** — Manage Dev and Prod resources separately 3. **Verify operations** — Use Get/List to verify after each write operation 4. **Proactive guidance** — Suggest the next step after each step completes 5. **Protect data sources and compute resources** — Never modify or delete data sources or compute resources via this skill; use the DataWorks console for such operations ## Reference Links | Reference | Description | |-----------|-------------| | [references/data-sources/README.md](references/data-sources/README.md) | Data source type list and ConnectionProperties examples | | [references/data-sources/](references/data-sources/) | Detailed configuration docs for each data source type (50 files) | | [references/cross-account-datasources.md](references/cross-account-datasources.md) | Cross-account data source configuration guide | | [references/compute-resources/README.md](references/compute-resources/README.md) | Compute resource ConnectionProperties examples | | [references/cli-installation-guide.md](references/cli-installation-guide.md) | Aliyun CLI installation guide | | [references/ram-policies.md](references/ram-policies.md) | RAM permission configuration and policy examples | | [references/related-apis.md](references/related-apis.md) | API parameter details and Region Endpoints | FILE:references/cli-installation-guide.md # Aliyun CLI Installation & Configuration Guide > **Aliyun CLI 3.3.1+** required for full plugin ecosystem support. ## Installation ### macOS (Homebrew) ```bash brew install aliyun-cli brew upgrade aliyun-cli aliyun version # verify >= 3.3.1 ``` ### macOS (Binary) ```bash wget https://aliyuncli.alicdn.com/aliyun-cli-macosx-latest-amd64.tgz tar -xzf aliyun-cli-macosx-latest-amd64.tgz sudo mv aliyun /usr/local/bin/ aliyun version ``` ### Linux ```bash wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz tar -xzf aliyun-cli-linux-latest-amd64.tgz sudo mv aliyun /usr/local/bin/ aliyun version ``` ### Windows (PowerShell) ```powershell Invoke-WebRequest -Uri "https://aliyuncli.alicdn.com/aliyun-cli-windows-latest-amd64.zip" -OutFile "aliyun-cli.zip" Expand-Archive -Path aliyun-cli.zip -DestinationPath C:\aliyun-cli $env:Path += ";C:\aliyun-cli" aliyun version ``` ## Configuration ```bash # Interactive configuration aliyun configure # Or set via environment variables (see official docs for details) # Note: AK/SK should be configured through `aliyun configure` or credential files, # not echoed/printed in chat. See SKILL.md security rules. export ALIBABA_CLOUD_REGION_ID=cn-hangzhou ``` ## Enable Auto Plugin Install ```bash aliyun configure set --auto-plugin-install true ``` ## Verification ```bash aliyun ecs describe-regions --user-agent AlibabaCloud-Agent-Skills # test authentication aliyun configure list # show current configuration ``` ## References - Official Documentation: https://help.aliyun.com/zh/cli/ - RAM Console: https://ram.console.aliyun.com/ - Access Key Management: https://ram.console.aliyun.com/manage/ak FILE:references/compute-resources/README.md # Compute Resource ConnectionProperties Reference ## Workspace Mode and Compute Resource Configuration > **Important**: Workspace mode (Simple Mode / Standard Mode) affects the quantity and authType configuration of compute resources. ### Query Workspace Mode ```bash aliyun dataworks-public GetProject --user-agent AlibabaCloud-Agent-Skills --Id 2>/dev/null | jq '.Project | {Id, Name, DevEnvironmentEnabled}' ``` - `DevEnvironmentEnabled: false` → Simple Mode - `DevEnvironmentEnabled: true` → Standard Mode ### Simple Mode vs Standard Mode | Dimension | Simple Mode | Standard Mode | |------|---------|---------| | Number of environments | 1 (Production environment) | 2 (Development + Production) | | Number of compute resources | 1 | 2 | | envType | `Prod` only | `Dev` + `Prod` | ### Compute Resource Creation Strategy #### Simple Mode Workspace Create **1 compute resource** with `envType: Prod`: ```json { "envType": "Prod", "authType": "PrimaryAccount", ... } ``` #### Standard Mode Workspace Create **2 compute resources**. Different environments use different authType: **Production environment**: ```json { "envType": "Prod", "authType": "PrimaryAccount", ... } ``` **Development environment**: ```json { "envType": "Dev", "authType": "Executor", ... } ``` --- ## authType Rules (by environment) - **Dev environment** (`envType: "Dev"`): `authType` is fixed as **`Executor`** - **Prod environment** (`envType: "Prod"`): `authType` supports the following values: | AuthType | Description | |----------|------| | PrimaryAccount | Access using primary account identity (**Recommended**) | | TaskOwner | Access using task owner identity | | SubAccount | Access using specified sub-account identity | | RamRole | Access using specified RAM role identity | --- ## Hologres (InstanceMode only) **Prod environment example:** ```json { "envType": "Prod", "regionId": "cn-hangzhou", "instanceId": "hgprecn-cn-xxxxx", "database": "mydb", "securityProtocol": "authTypeNone", "authType": "PrimaryAccount" } ``` **Dev environment example:** ```json { "envType": "Dev", "regionId": "cn-hangzhou", "instanceId": "hgprecn-cn-xxxxx", "database": "mydb", "securityProtocol": "authTypeNone", "authType": "Executor" } ``` ## MaxCompute (UrlMode only) **Prod environment example:** ```json { "envType": "Prod", "regionId": "cn-hangzhou", "project": "my_maxcompute_project", "authType": "PrimaryAccount", "endpointMode": "SelfAdaption" } ``` **Dev environment example:** ```json { "envType": "Dev", "regionId": "cn-hangzhou", "project": "my_maxcompute_project_dev", "authType": "Executor", "endpointMode": "SelfAdaption" } ``` ## Flink (InstanceMode) ```json { "envType": "Prod", "regionId": "cn-hangzhou", "instanceId": "flink-xxxxx", "workspaceName": "my_flink_workspace", "authType": "PrimaryAccount" } ``` ## Spark (InstanceMode) ```json { "envType": "Prod", "regionId": "cn-hangzhou", "clusterId": "spark-xxxxx", "authType": "PrimaryAccount" } ``` --- ## Important Notes - All compute resource types use **camelCase** for ConnectionProperties fields - Dev environment authType is fixed as `Executor`; Prod environment supports `PrimaryAccount`/`TaskOwner`/`SubAccount`/`RamRole` - Hologres compute resources support InstanceMode only - MaxCompute compute resources support UrlMode only FILE:references/cross-account-datasources.md # Cross-Account Data Source Configuration Guide Use cross-account mode when you need to access resources under other Alibaba Cloud accounts. --- ## Cross-Account Configuration Process ### Step 1: Resource Account (Target Account) Preparation In the account where the resource resides (e.g., ``): 1. Create a RAM role (e.g., ``) 2. Configure trust policy to allow the source account (e.g., ``) to assume 3. Grant the role permissions to access target resources (e.g., MaxCompute project access) **Trust Policy Example:** ```json { "Statement": [{ "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "RAM": ["acs:ram:::root"], "Service": ["@engine.dataworks.aliyuncs.com"] } }], "Version": "1" } ``` ### Step 2: Create Data Source in the DataWorks Account (Source Account) Create a data source in the DataWorks account and configure cross-account parameters: - `crossAccountOwnerId`: Alibaba Cloud UID of the resource account - `crossAccountRoleName`: RAM role name in the resource account - `authType`: `RamRole` (required for MaxCompute/Hologres cross-account scenarios) --- ## Cross-Account Data Source Type Quick Reference | Data source type | Type | Connection mode | authType | Required parameters | Special notes | |-----------|------|---------|----------|---------|---------| | MaxCompute | maxcompute | UrlMode | RamRole | project, regionId, endpointMode | Does not require username/password | | Hologres | hologres | InstanceMode | RamRole | instanceId, regionId, database | Does not require username/password | | MySQL | mysql | InstanceMode | - | instanceId, regionId, database, username, password | Standard cross-account | | PostgreSQL | postgresql | InstanceMode | - | instanceId, regionId, database, username, password | Standard cross-account | | PolarDB | polardb | InstanceMode | - | clusterId, regionId, database, dbType, username, password | Uses clusterId instead of instanceId | | SQLServer | sqlserver | InstanceMode | - | instanceId, regionId, database, username, password | Standard cross-account | | AnalyticDB MySQL | analyticdb_for_mysql | InstanceMode | - | instanceId, regionId, database, username, password | Standard cross-account | | AnalyticDB PostgreSQL | analyticdb_for_postgresql | InstanceMode | - | instanceId, regionId, database, username, password | Standard cross-account | | StarRocks | starrocks | InstanceMode | - | instanceId, instanceType, regionId, database, username, password | Must provide `instanceType` (`emr-olap` or `serverless`) | > **Note**: Only MaxCompute and Hologres require explicitly setting `authType: RamRole` for cross-account. Other types do not need to set authType. --- ## Cross-Account Configuration Examples ### MaxCompute Cross-Account ```json { "project": "target_mc_project", "regionId": "cn-zhangjiakou", "endpointMode": "SelfAdaption", "authType": "RamRole", "crossAccountOwnerId": "", "crossAccountRoleName": "", "envType": "Prod" } ``` ### Hologres Cross-Account ```json { "instanceId": "hgpostcn-cn-xxxxx", "regionId": "cn-zhangjiakou", "database": "mydb", "authType": "RamRole", "crossAccountOwnerId": "", "crossAccountRoleName": "", "envType": "Prod" } ``` ### MySQL Cross-Account ```json { "instanceId": "rm-xxxxx", "regionId": "cn-zhangjiakou", "database": "mydb", "username": "myuser", "password": "", "crossAccountOwnerId": "", "crossAccountRoleName": "", "envType": "Prod" } ``` ### PolarDB Cross-Account ```json { "clusterId": "pc-xxxxx", "regionId": "cn-zhangjiakou", "database": "mydb", "dbType": "mysql", "username": "myuser", "password": "", "crossAccountOwnerId": "", "crossAccountRoleName": "", "envType": "Prod" } ``` ### PostgreSQL Cross-Account ```json { "instanceId": "pgm-xxxxx", "regionId": "cn-zhangjiakou", "database": "postgres", "username": "myuser", "password": "", "crossAccountOwnerId": "", "crossAccountRoleName": "", "envType": "Prod" } ``` ### SQLServer Cross-Account ```json { "instanceId": "rm-xxxxx", "regionId": "cn-zhangjiakou", "database": "mydb", "username": "myuser", "password": "", "crossAccountOwnerId": "", "crossAccountRoleName": "", "envType": "Prod" } ``` ### AnalyticDB MySQL Cross-Account ```json { "instanceId": "am-xxxxx", "regionId": "cn-zhangjiakou", "database": "mydb", "username": "myuser", "password": "", "crossAccountOwnerId": "", "crossAccountRoleName": "", "envType": "Prod" } ``` ### AnalyticDB PostgreSQL Cross-Account ```json { "instanceId": "gp-xxxxx", "regionId": "cn-zhangjiakou", "database": "postgres", "username": "myuser", "password": "", "crossAccountOwnerId": "", "crossAccountRoleName": "", "envType": "Prod" } ``` ### StarRocks Cross-Account ```json { "instanceId": "c-xxxxx", "instanceType": "serverless", "regionId": "cn-zhangjiakou", "database": "mydb", "username": "sr_user", "password": "", "crossAccountOwnerId": "", "crossAccountRoleName": "", "envType": "Prod" } ``` --- ## CLI Command Examples Complete command for creating a cross-account data source: ```bash aliyun dataworks-public CreateDataSource --user-agent AlibabaCloud-Agent-Skills \ --region cn-zhangjiakou \ --ProjectId 12345 \ --Name cross_account_mysql \ --Type mysql \ --ConnectionPropertiesMode InstanceMode \ --ConnectionProperties '{"envType":"Prod","instanceId":"rm-xxxxx","regionId":"cn-zhangjiakou","database":"mydb","username":"myuser","password":"","crossAccountOwnerId":"","crossAccountRoleName":""}' \ --Description "Cross-account MySQL datasource" ``` --- ## Detailed Configuration Documentation For detailed cross-account configuration of each data source type, please refer to: - [data-sources/maxcompute.md](data-sources/maxcompute.md) - [data-sources/hologres.md](data-sources/hologres.md) - [data-sources/mysql.md](data-sources/mysql.md) - [data-sources/postgresql.md](data-sources/postgresql.md) - [data-sources/polardb.md](data-sources/polardb.md) - [data-sources/sqlserver.md](data-sources/sqlserver.md) - [data-sources/analyticdb_for_mysql.md](data-sources/analyticdb_for_mysql.md) - [data-sources/analyticdb_for_postgresql.md](data-sources/analyticdb_for_postgresql.md) - [data-sources/starrocks.md](data-sources/starrocks.md) FILE:references/data-sources/README.md # DataWorks Data Source Reference Supports a total of **50** data source types (verified through API testing). > **Note**: The following types do not currently support OpenAPI, please configure through the console (Support will be added in future versions): `hdfs` --- ## Workspace Mode and Data Source Configuration > **Important**: Workspace mode (Simple Mode / Standard Mode) directly affects the number and configuration of data sources. Workspace mode must be confirmed before creating data sources. ### Query Workspace Mode ```bash aliyun dataworks-public GetProject --user-agent AlibabaCloud-Agent-Skills --Id 2>/dev/null | jq '.Project | {Id, Name, DevEnvironmentEnabled}' ``` - `DevEnvironmentEnabled: false` → Simple Mode - `DevEnvironmentEnabled: true` → Standard Mode ### Simple Mode vs Standard Mode | Dimension | Simple Mode | Standard Mode | |-----------|-------------|---------------| | Number of environments | 1 (Production environment) | 2 (Development + Production) | | Number of data sources | 1 | 2 (Physically isolated) | | Data source name | Any | Dev/Prod can share the same name | | envType | Only `Prod` | `Dev` + `Prod` | | Code editing | Edit production code directly | Edit only in development environment | | Release process | Submit and schedule directly | Submit → Publish → Schedule | ### Data Source Creation Strategy #### Simple Mode Workspace Only need to create **1 data source**, `envType` fixed as `Prod`: ```json { "envType": "Prod", "instanceId": "rm-xxxxx", "database": "prod_db", "username": "root", "password": "" } ``` #### Standard Mode Workspace Must create **2 data sources** (Physically isolated), can use the same name: **Production environment data source**: ```json { "envType": "Prod", "instanceId": "rm-prod-xxxxx", "database": "prod_db", "username": "root", "password": "" } ``` **Development environment data source** (same name): ```json { "envType": "Dev", "instanceId": "rm-dev-xxxxx", "database": "dev_db", "username": "dev_user", "password": "" } ``` ### Physical Isolation Approaches (Standard Mode) Standard Mode requires dev/prod data sources to be **physically isolated**. Recommended approaches: | Isolation approach | Description | Applicable scenarios | |--------------------|-------------|----------------------| | Different instances | Dev uses `rm-dev-xxx`, Prod uses `rm-prod-xxx` | **Recommended**, Complete isolation | | Same instance, different databases | Dev uses `dev_db`, Prod uses `prod_db` | Cost-sensitive scenarios | > **Best practice**: Before creating data sources, confirm the workspace mode first. For Standard Mode, guide users to create both development and production data sources. --- ## Cross-Account Data Source Quick Reference ### Data Source Types Supporting Cross-Account | Type | Type | Connection Mode | authType | Cross-account specific parameters | |------|------|-----------------|----------|-----------------------------------| | MaxCompute | maxcompute | UrlMode | RamRole | project, endpointMode | | Hologres | hologres | InstanceMode | RamRole | instanceId, database | | MySQL | mysql | InstanceMode | - | instanceId, username, password | | PostgreSQL | postgresql | InstanceMode | - | instanceId, username, password | | PolarDB | polardb | InstanceMode | - | clusterId, dbType, username, password | | SQLServer | sqlserver | InstanceMode | - | instanceId, username, password | | AnalyticDB MySQL | analyticdb_for_mysql | InstanceMode | - | instanceId, username, password | | AnalyticDB PostgreSQL | analyticdb_for_postgresql | InstanceMode | - | instanceId, username, password | | StarRocks | starrocks | InstanceMode | - | instanceId, instanceType, username, password | ### Cross-Account Common Parameters All cross-account data sources require: - `crossAccountOwnerId`: Alibaba Cloud UID of the target account - `crossAccountRoleName`: RAM role name in the target account ### Configuration Differences **MaxCompute / Hologres**: - Must set `authType: RamRole` - Does not require `username` and `password` **Other types** (MySQL/PostgreSQL/SQLServer/PolarDB/AnalyticDB/StarRocks): - Does not require setting `authType` - Must provide `username` and `password` - PolarDB uses `clusterId` instead of `instanceId` - StarRocks must provide `instanceType` (`emr-olap` or `serverless`) ### Detailed Configuration Documentation For detailed cross-account configuration of each data source type, please refer to the corresponding documentation: - [maxcompute.md](maxcompute.md) - [hologres.md](hologres.md) - [mysql.md](mysql.md) - [postgresql.md](postgresql.md) - [polardb.md](polardb.md) - [sqlserver.md](sqlserver.md) - [analyticdb_for_mysql.md](analyticdb_for_mysql.md) - [analyticdb_for_postgresql.md](analyticdb_for_postgresql.md) - [starrocks.md](starrocks.md) --- ## Type List ### Relational Databases (18) | Type | Name | UrlMode | InstanceMode | Notes | |------|------|---------|-------------|-------| | mysql | MySQL | Yes | Yes | | | postgresql | PostgreSQL | Yes | Yes | | | oracle | Oracle | Yes | No | Uses jdbcUrl format | | sqlserver | SQL Server | Yes | Yes | | | polardb | PolarDB MySQL | Yes | Yes | Requires dbType | | polardbo | PolarDB for Oracle | Yes | Yes | InstanceMode requires instanceId/regionId | | polardb-x-2-0 | PolarDB-X 2.0 | Yes | Yes | InstanceMode requires instanceId/regionId | | apsaradb_for_oceanbase | OceanBase | Yes | Yes | InstanceMode requires instanceId/tenant/regionId, UrlMode requires dbMode | | mariadb | MariaDB | Yes | No | | | dm | DM (Dameng) | Yes | No | Does not require the database field | | db2 | DB2 | Yes | No | Requires jdbcDriver parameter | | tidb | TiDB | Yes | No | | | vertica | Vertica | Yes | No | | | gbase8a | GBase 8a | Yes | No | | | kingbasees | KingbaseES | Yes | No | | | saphana | SAP HANA | Yes | No | | | drds | DRDS | Yes | Yes | | | snowflake | Snowflake | Yes | No | Requires accountUrl | ### Big Data Engines (13) | Type | Name | UrlMode | InstanceMode | Notes | |------|------|---------|-------------|-------| | maxcompute | MaxCompute | Yes | No | Requires project parameter | | hologres | Hologres | No | Yes | Only supports InstanceMode | | hive | Hive | Yes | No | Requires version/metaType/metastoreUris | | clickhouse | ClickHouse | Yes | No | | | starrocks | StarRocks | Yes | Yes | InstanceMode requires instanceType | | doris | Doris | Yes | No | Requires loadAddress | | selectdb | SelectDB | Yes | No | Requires loadAddress | | analyticdb_for_mysql | AnalyticDB MySQL | Yes | Yes | | | analyticdb_for_postgresql | AnalyticDB PostgreSQL | Yes | Yes | | | redshift | Amazon Redshift | Yes | No | | | hbase | HBase | Yes | No | Uses hbaseConfig | | lindorm | Lindorm | Yes | No | Uses seedserver/namespace | | dlf | Data Lake Formation | No | Yes | Requires catalogId/catalogName/catalogType | ### Storage Services (4) | Type | Name | UrlMode | InstanceMode | Notes | |------|------|---------|-------------|-------| | oss | OSS | Yes | No | RamRole authentication recommended | | s3 | Amazon S3 | Yes | No | | | ftp | FTP | Yes | No | protocol uses lowercase | | ssh | SSH | Yes | No | Only supports connection string mode | ### NoSQL Databases (7) | Type | Name | UrlMode | InstanceMode | Notes | |------|------|---------|-------------|-------| | tablestore | Tablestore | Yes | No | Requires regionId | | memcache | Memcache | Yes | No | Uses proxy/port | | milvus | Milvus | Yes | Yes | UrlMode uses endpoint | | mongodb | MongoDB | Yes | Yes | Requires authDb and engineVersion | | redis | Redis | Yes | Yes | Supports InstanceMode and UrlMode, SSL authentication optional | | elasticsearch | Elasticsearch | Yes | Yes | Supports InstanceMode and UrlMode, anonymous authentication optional | | graph_database | Graph Database | Yes | No | Connection string mode (host/port) | ### Message Services (3) | Type | Name | UrlMode | InstanceMode | Notes | |------|------|---------|-------------|-------| | datahub | DataHub | Yes | No | Requires regionId | | loghub | LogHub (SLS) | Yes | No | Requires regionId | | kafka | Kafka | Yes | Yes | version uses "2.0" or "3.4" | ### SaaS Services (5) | Type | Name | UrlMode | InstanceMode | Notes | |------|------|---------|-------------|-------| | restapi | REST API | Yes | No | | | opensearch | OpenSearch | No | Yes | Only supports InstanceMode | | salesforce | Salesforce | Yes | No | Uses OAuth, requires refreshToken | | httpfile | HttpFile | Yes | No | HTTP file data source | | bigquery | BigQuery | Yes | No | Requires bigQueryProjectId, bigQueryAuth | --- ## Connection Modes | Mode | Applicable Scenarios | Common Required Fields | |------|----------------------|------------------------| | **UrlMode** | Self-hosted database, Known IP/port | address, database, username, password | | **InstanceMode** | Alibaba Cloud managed instance (RDS, etc.) | instanceId, regionId, database, username, password | > **Special types**: Oracle uses `jdbcUrl` (not address+database), MaxCompute uses `project`+`endpointMode`, OSS uses `bucket`+`endpoint`+`authType`. See examples for each type below. --- ## ConnectionProperties Examples ### MySQL (UrlMode) ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": 3306}], "database": "mydb", "username": "root", "password": "" } ``` ### MySQL (InstanceMode - Same-account) ```json { "envType": "Prod", "instanceId": "rm-xxxxx", "regionId": "cn-shanghai", "database": "mydb", "username": "root", "password": "" } ``` ### MySQL (Cross-account) ```json { "envType": "Prod", "instanceId": "rm-xxxxx", "regionId": "cn-shanghai", "database": "mydb", "username": "root", "password": "", "crossAccountOwnerId": "1234567890", "crossAccountRoleName": "CrossAccountRole" } ``` ### PostgreSQL (UrlMode) ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": 5432}], "database": "mydb", "username": "postgres", "password": "" } ``` ### Oracle (UrlMode) > **Note**: Oracle uses `jdbcUrl` format, not the common `address` + `database` structure. ```json { "envType": "Prod", "jdbcUrl": "jdbc:oracle:thin:@192.168.1.100:1521:ORCL", "username": "system", "password": "" } ``` ### Hologres (InstanceMode) ```json { "envType": "Prod", "instanceId": "hgpostcn-cn-xxxxx", "regionId": "cn-shanghai", "database": "mydb", "authType": "PrimaryAccount" } ``` ### MaxCompute > **Note**: The MaxCompute data source field name is `project` (not `maxComputeProject`), and does not support `Executor` identity. It is recommended to use `endpointMode: SelfAdaption` for adaptive mode. ```json { "envType": "Prod", "project": "my_mc_project", "regionId": "cn-shanghai", "endpointMode": "SelfAdaption", "authType": "PrimaryAccount" } ``` ### ClickHouse (UrlMode) ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": 8123}], "database": "default", "username": "default", "password": "" } ``` ### OSS > **Note**: OSS field names are `bucket` (not `bucketName`), `endpoint` (not `endPoint`), and requires `authType` and `regionId`. **RamRole authentication is recommended** (not Ak mode). ```json { "envType": "Prod", "regionId": "cn-shanghai", "endpoint": "http://oss-cn-shanghai.aliyuncs.com", "bucket": "my-bucket", "authType": "RamRole", "authIdentity": "123456789" } ``` ### Hive (UrlMode) > **Note**: Hive UrlMode requires `version`, `metaType`, `metastoreUris`, `loginMode` and other mandatory fields. ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": 10000}], "database": "default", "version": "2.3.9", "metaType": "HiveMetastore", "metastoreUris": "thrift://192.168.1.100:9083", "loginMode": "Anonymous", "securityProtocol": "authTypeNone" } ``` ### PolarDB MySQL (UrlMode) > **Note**: PolarDB requires the `dbType` parameter to specify the database type. ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": 3306}], "database": "mydb", "username": "root", "password": "", "dbType": "mysql" } ``` ### DM (Dameng) (UrlMode) > **Note**: DM does not require the `database` field. ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": 5236}], "username": "mockuser", "password": "" } ``` ### DB2 (UrlMode) > **Note**: DB2 requires the `jdbcDriver` parameter. Possible values: `db2_1`, `db2_2`, `as400_1`. ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": 50000}], "database": "mydb", "username": "db2admin", "password": "", "jdbcDriver": "db2_1" } ``` ### StarRocks/Doris/SelectDB (UrlMode) > **Note**: These types require the `loadAddress` parameter for data import. ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": 9030}], "loadAddress": [{"host": "192.168.1.100", "port": 8030}], "database": "mydb", "username": "root", "password": "" } ``` ### HBase (UrlMode) > **Note**: HBase uses the `hbaseConfig` object instead of `address`. ```json { "envType": "Prod", "hbaseConfig": { "hbase.zookeeper.quorum": "192.168.1.100:2181", "hbaseVersion": "0.9.4" }, "securityProtocol": "authTypeNone" } ``` ### Lindorm (UrlMode) > **Note**: Lindorm uses `seedserver`/`namespace` instead of `address`/`database`. ```json { "envType": "Prod", "seedserver": "ld-xxx.lindorm.rds.aliyuncs.com:30020", "namespace": "default", "username": "root", "password": "" } ``` ### Memcache (UrlMode) > **Note**: Memcache uses `proxy`/`port` instead of `address`. ```json { "envType": "Prod", "proxy": "192.168.1.100", "port": "11211", "username": "mockuser", "password": "" } ``` ### Milvus (UrlMode) > **Note**: Milvus UrlMode uses `endpoint` instead of `host`/`port`. ```json { "envType": "Prod", "endpoint": "http://192.168.1.100:19530", "database": "default", "username": "root", "password": "", "authType": "USERNAME_PASSWORD" } ``` ### Tablestore (UrlMode) > **Note**: Tablestore requires the `regionId` parameter. ```json { "envType": "Prod", "regionId": "cn-shanghai", "endpoint": "https://myinstance.cn-shanghai.ots.aliyuncs.com", "instanceName": "myinstance", "accessId": "", "accessKey": "" } ``` ### DataHub (UrlMode) > **Note**: DataHub requires the `regionId` parameter. ```json { "envType": "Prod", "regionId": "cn-shanghai", "endpoint": "http://dh-cn-shanghai.aliyuncs.com", "project": "my_project", "accessId": "", "accessKey": "" } ``` ### LogHub (UrlMode) > **Note**: LogHub requires the `regionId` parameter. ```json { "envType": "Prod", "regionId": "cn-shanghai", "endpoint": "cn-shanghai.log.aliyuncs.com", "project": "my_project", "accessId": "", "accessKey": "" } ``` ### S3 (UrlMode) > **Note**: S3 uses `accessId`/`accessKey` (not accessKey/secretKey). ```json { "envType": "Prod", "regionId": "us-east-1", "endpoint": "http://s3.amazonaws.com", "bucket": "my-bucket", "accessId": "", "accessKey": "" } ``` ### FTP (UrlMode) > **Note**: The `protocol` parameter uses lowercase values: `ftp`, `sftp`, `ftps`. ```json { "envType": "Prod", "protocol": "ftp", "host": "192.168.1.100", "port": "21", "username": "ftpuser", "password": "" } ``` ### OpenSearch (InstanceMode) > **Note**: OpenSearch only supports InstanceMode. ```json { "envType": "Prod", "regionId": "cn-shanghai", "instanceType": "vectorSearchVersion", "instanceId": "ha-xxxxx", "username": "admin", "password": "" } ``` ### AnalyticDB MySQL (UrlMode) ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": 3306}], "database": "mydb", "username": "root", "password": "" } ``` ### AnalyticDB PostgreSQL (UrlMode) ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": 5432}], "database": "mydb", "username": "postgres", "password": "" } ``` ### Snowflake (UrlMode) > **Note**: Snowflake requires the `accountUrl` parameter with the full account URL. ```json { "envType": "Prod", "accountUrl": "xy12345.snowflakecomputing.com", "database": "mydb", "securityProtocol": "authTypeClientPassword", "username": "myuser", "password": "", "warehouseName": "my_warehouse" } ``` ### MongoDB (UrlMode) > **Note**: MongoDB requires `authDb` (authorization database) and `engineVersion` parameters. ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": "27017"}], "database": "mydb", "username": "root", "password": "", "authDb": "admin", "engineVersion": "5.x" } ``` ### Kafka (UrlMode) > **Note**: Kafka UrlMode uses the `address` parameter, and `version` only supports "2.0" or "3.4". ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": "9092"}], "version": "2.0", "securityProtocol": "authTypeNone" } ``` ### DRDS (UrlMode) ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": "3306"}], "database": "mydb", "username": "root", "password": "" } ``` ### DLF (InstanceMode) > **Note**: DLF only supports InstanceMode, requires catalog-related parameters. ```json { "envType": "Prod", "authType": "PrimaryAccount", "database": "db1", "catalogId": "clg-paimon-xxx", "catalogName": "xxx", "catalogType": "Paimon", "endpoint": "http://cn-hangzhou-vpc.dlf.aliyuncs.com" } ``` ### Graph Database (UrlMode) > **Note**: Graph Database uses connection string mode with host/port. ```json { "host": "127.0.0.1", "port": "5432", "username": "xxxxx", "password": "xxxxx", "envType": "Dev" } ``` ### BigQuery (UrlMode) > **Note**: BigQuery requires `bigQueryProjectId` (project ID) and `bigQueryAuth` (credential file ID). ```json { "bigQueryProjectId": "bigquery_id", "bigQueryAuth": "123", "envType": "Prod" } ``` ### PolarDB-O (InstanceMode / UrlMode) > **Note**: PolarDB-O supports both InstanceMode and UrlMode. InstanceMode requires `instanceId` and `regionId`. ```json { "envType": "Prod", "regionId": "cn-beijing", "instanceId": "pc-xxxxx", "database": "my_database", "username": "my_username", "password": "" } ``` ### PolarDB-X 2.0 (InstanceMode / UrlMode) > **Note**: PolarDB-X 2.0 supports both InstanceMode and UrlMode. InstanceMode requires `instanceId` and `regionId`. ```json { "envType": "Prod", "regionId": "cn-beijing", "instanceId": "pxc-xxxxx", "database": "my_database", "username": "my_username", "password": "" } ``` ### OceanBase (apsaradb_for_oceanbase) (InstanceMode / UrlMode) > **Note**: OceanBase supports both modes. InstanceMode requires `instanceId`, `tenant`, `regionId`. UrlMode requires `dbMode` (mysql/oracle). ```json { "envType": "Prod", "instanceId": "ob5nj51ns6qjr4", "tenant": "t5nnecr8dppi8", "regionId": "cn-shanghai", "database": "my_database", "username": "aliyun", "password": "" } ``` --- ## Environment Types | Value | Description | |-------|-------------| | Dev | Development environment | | Prod | Production environment | --- > For full configuration of each data source type, please refer to `.md` (e.g., `mysql.md`, `hologres.md`) FILE:references/data-sources/analyticdb_for_mysql.md # AnalyticDB MySQL Datasource Documentation ## Overview **Last Updated:** October 17, 2024, 10:22:06 ## Property Definition - **Datasource Type:** `analyticdb_for_mysql` - **Supported Configuration Modes (ConnectionPropertiesMode):** - UrlMode (Connection String Mode) - InstanceMode (Instance Mode) --- ## Same Account Instance Mode | Name | Type | Example Value | Required | Description and Notes | |------|------|---------------|----------|----------------------| | regionId | String | cn-shanghai | Yes | The Region where the ADB MySQL instance is located. | | instanceId | String | am-xxxxx | Yes | The ADB MySQL instance ID. | | database | String | database_demo | Yes | ADB MySQL database name. | | username | String | xxxxx | Yes | Username for ADB MySQL database access. | | password | String | xxxxx | Yes | Password for ADB MySQL database access. | | envType | String | Dev | Yes | envType indicates the datasource environment information.
• **Dev:** Development environment.
• **Prod:** Production environment. | ### Configuration Example - Same Account Instance Mode ```json { "database": "database", "password": "***", "instanceId": "am-xxxxx", "regionId": "cn-shanghai", "envType": "Dev", "username": "username" } ``` --- ## Cross-Account Instance Mode | Name | Type | Example Value | Required | Description and Notes | |------|------|---------------|----------|----------------------| | regionId | String | cn-shanghai | Yes | The Region where the ADB MySQL instance is located. | | instanceId | String | am-xxxxx | Yes | The ADB MySQL instance ID. | | database | String | database_demo | Yes | ADB MySQL database name. | | username | String | xxxxx | Yes | Username for ADB MySQL database access. | | password | String | xxxxx | Yes | Password for ADB MySQL database access. | | crossAccountOwnerId | String | | Yes | The cross-AliCloud master account ID of the other party; required in cross-account scenarios. | | crossAccountRoleName | String | dw-ds2.0-role | Yes | The role name under the other party's account in cross-account scenarios. | | envType | String | Dev | Yes | envType indicates the datasource environment information.
• **Dev:** Development environment.
• **Prod:** Production environment. | ### Configuration Example - Cross-Account Instance Mode ```json { "database": "database", "password": "***", "instanceId": "am-xxxxx", "regionId": "cn-shanghai", "envType": "Dev", "username": "username", "crossAccountOwnerId": "", "crossAccountRoleName": "dw-ds2.0-role" } ``` --- ## Connection String Mode (UrlMode) | Name | Type | Example Value | Required | Description and Notes | |------|------|---------------|----------|----------------------| | database | String | database_demo | Yes | ADB MySQL database name. | | username | String | xxxxx | Yes | Username for ADB MySQL database access. | | password | String | xxxxx | Yes | Password for ADB MySQL database access. | | address | JSON Array | `[{"host": "127.0.0.1", "port": 3306}]` | Yes | Structured as an array, but only allows configuration of 1 set of host and port. | | properties | JSONObject | - | No | JDBC connection advanced parameters. | | envType | String | Dev | Yes | envType indicates the datasource environment information.
• **Dev:** Development environment.
• **Prod:** Production environment. | ### Configuration Example - Connection String Mode ```json { "address": [ { "host": "127.0.0.1", "port": 5432 } ], "database": "db", "properties": { "connectTimeout": "2000" }, "username": "aliyun", "password": "xxx", "envType": "Dev" } ``` --- ## Summary | Configuration Mode | Use Case | Key Parameters | |-------------------|----------|----------------| | **Same Account Instance Mode** | Connect to ADB MySQL within the same AliCloud account | regionId, instanceId, database, username, password, envType | | **Cross-Account Instance Mode** | Connect to ADB MySQL in another AliCloud account | regionId, instanceId, database, username, password, crossAccountOwnerId, crossAccountRoleName, envType | | **Connection String Mode** | Connect using direct host/port connection | address, database, username, password, properties (optional), envType | FILE:references/data-sources/analyticdb_for_postgresql.md # AnalyticDB PostgreSQL DataSource Documentation ## Overview - **DataSource Type**: `analyticdb_for_postgresql` - **Supported Configuration Modes**: - `UrlMode` (Connection String Mode) - `InstanceMode` (Instance Mode) --- ## Configuration Modes ### 1. Same-Account Instance Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `regionId` | String | `cn-shanghai` | Yes | ADB PostgreSQL instance Region. | | `instanceId` | String | `gp-xxxxx` | Yes | ADB PostgreSQL instance ID. | | `database` | String | `database_demo` | Yes | ADB PostgreSQL database name. | | `username` | String | `xxxxx` | Yes | ADB PostgreSQL database access username. | | `password` | String | `xxxxx` | Yes | ADB PostgreSQL database access password. | | `envType` | String | `Dev` | Yes | Datasource environment info. Values: `Dev` (Development), `Prod` (Production). | **Example:** ```json { "database": "dw", "password": "***", "instanceId": "gp-xxxxx", "regionId": "cn-hangzhou", "envType": "Prod", "username": "dw" } ``` --- ### 2. Cross-Account Instance Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `regionId` | String | `cn-shanghai` | Yes | ADB PostgreSQL instance Region. | | `instanceId` | String | `gp-xxxxx` | Yes | ADB PostgreSQL instance ID. | | `database` | String | `database_demo` | Yes | ADB PostgreSQL database name. | | `username` | String | `xxxxx` | Yes | ADB PostgreSQL database access username. | | `password` | String | `xxxxx` | Yes | ADB PostgreSQL database access password. | | `crossAccountOwnerId` | String | `` | Yes | Cross-account target Alibaba Cloud main account ID (required for cross-account scenarios). | | `crossAccountRoleName` | String | `dw-ds2.0-role` | Yes | Role name under the target account for cross-account scenarios. | | `envType` | String | `Dev` | Yes | Datasource environment info. Values: `Dev` (Development), `Prod` (Production). | **Example:** ```json { "crossAccountOwnerId": "", "crossAccountRoleName": "dw-role", "database": "dw", "password": "***", "instanceId": "gp-xxxxx", "regionId": "cn-shanghai", "envType": "Prod", "username": "dw" } ``` --- ### 3. Connection String Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `database` | String | `database_demo` | Yes | ADB PostgreSQL database name. | | `username` | String | `xxxxx` | Yes | ADB PostgreSQL database access username. | | `password` | String | `xxxxx` | Yes | ADB PostgreSQL database access password. | | `address` | JSON Array | `[{"host": "127.0.0.1", "port": 3306}]` | Yes | Array format, but only allows 1 set of host and port configuration. | | `properties` | JSONObject | - | No | JDBC connection advanced parameters. | | `envType` | String | `Dev` | Yes | Datasource environment info. Values: `Dev` (Development), `Prod` (Production). | **Example:** ```json { "address": [ { "host": "127.0.0.1", "port": 5432 } ], "database": "db", "properties": { "connectTimeout": "2000" }, "username": "aliyun", "password": "xxx", "envType": "Dev" } ``` --- ## Summary Table | Mode | Required Parameters | |------|---------------------| | Same-Account Instance | `regionId`, `instanceId`, `database`, `username`, `password`, `envType` | | Cross-Account Instance | `regionId`, `instanceId`, `database`, `username`, `password`, `crossAccountOwnerId`, `crossAccountRoleName`, `envType` | | Connection String | `address`, `database`, `username`, `password`, `envType` (optional: `properties`) | FILE:references/data-sources/apsaradb_for_oceanbase.md # OceanBase ConnectionProperties Documentation ## Property Definition - **Datasource Type**: `apsaradb_for_oceanbase` - **Supported Configuration Modes (ConnectionPropertiesMode)**: - UrlMode (Connection String Mode) - InstanceMode (Instance Mode) --- ## Same-Account Instance Mode | Name | Type | Example Value | Required | Description and Notes | |------|------|---------------|----------|----------------------| | regionId | String | cn-shanghai | Yes | The Region where the instance belongs. | | instanceId | String | ob5nj51ns6qjr4 | Yes | OceanBase cluster instance ID. | | tenant | String | t5nnecr8dppi8 | Yes | OceanBase tenant ID. | | database | String | ob_database | Yes | Database name. | | username | String | xxxxx | Yes | Username. | | password | String | xxxxx | Yes | Password. | | readOnlyDBInstance | String | t62zwpvyehmps-ro0.cn-beijing.oceanbase.aliyuncs.com:1521 | No | Standby database address. | | envType | String | Dev | Yes | envType indicates the datasource environment information.
- `Dev`: Development environment
- `Prod`: Production environment | --- ## Cross-Account Instance Mode | Name | Type | Example Value | Required | Description and Notes | |------|------|---------------|----------|----------------------| | regionId | String | cn-shanghai | Yes | The Region where the instance belongs. | | instanceId | String | ob5nj51ns6qjr4 | Yes | OceanBase cluster instance ID. | | tenant | String | t5nnecr8dppi8 | Yes | OceanBase tenant ID. | | database | String | ob_database | Yes | Database name. | | username | String | xxxxx | Yes | Username. | | password | String | xxxxx | Yes | Password. | | crossAccountOwnerId | String | 1 | Yes | Cross-account target cloud account ID. | | crossAccountRoleName | String | cross-role | Yes | Cross-account target RAM role name. | | readOnlyDBInstance | String | t62zwpvyehmps-ro0.cn-beijing.oceanbase.aliyuncs.com:1521 | No | Standby database address. | | envType | String | Dev | Yes | envType indicates the datasource environment information.
- `Dev`: Development environment
- `Prod`: Production environment | --- ## Connection String Mode | Name | Type | Example Value | Required | Description and Notes | |------|------|---------------|----------|----------------------| | dbMode | String | mysql | No | Database mode. Valid values:
- `mysql`
- `oracle` | | address | Array | `[{"host": "127.0.0.1", "port": 3306}]` | Yes | Only a single address is allowed. | | database | String | ob_database | Yes | Database name. | | username | String | xxxxx | Yes | Username. | | password | String | xxxxx | Yes | Password. | | properties | JSON Object | `{"useSSL": "false"}` | No | Driver properties. | | envType | String | Dev | Yes | envType indicates the datasource environment information.
- `Dev`: Development environment
- `Prod`: Production environment | --- ## Datasource Configuration Examples ### Same-Account Instance Mode ```json { "envType": "Prod", "instanceId": "obxxxxxxxxxx", "tenant": "txxxxxxxxxx", "regionId": "cn-shanghai", "database": "db", "username": "aliyun", "password": "xxx" } ``` ### Cross-Account Instance Mode ```json { "envType": "Prod", "instanceId": "obxxxxxxxxxx", "tenant": "txxxxxxxxxx", "regionId": "cn-shanghai", "database": "db", "username": "aliyun", "password": "xxx", "crossAccountOwnerId": "1234567890", "crossAccountRoleName": "my_ram_role" } ``` ### Connection String Mode ```json { "envType": "Prod", "address": [ { "host": "127.0.0.1", "port": "5432" } ], "dbMode": "mysql", "database": "db", "properties": { "connectTimeout": "2000" }, "username": "aliyun", "password": "xxx" } ``` --- **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/oceanbase FILE:references/data-sources/bigquery.md # BigQuery Datasource Documentation ## Overview - **Datasource Type**: `bigquery` - **Supported Configuration Mode**: `UrlMode` (Connection String Mode) --- ## ConnectionProperties Parameters ### Connection String Mode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `bigQueryProjectId` | String | `bigquery_id` | **Yes** | The ID of the BigQuery Project. | | `bigQueryAuth` | String | `123` | **Yes** | BigQuery authentication credentials. Enter the file ID. | | `envType` | String | `Dev` | **Yes** | Indicates the datasource environment information.
• `Dev`: Development environment
• `Prod`: Production environment | --- ## Configuration Example ### Connection String Mode ```json { "bigQueryProjectId": "bigquery_id", "bigQueryAuth": "123", "envType": "Dev" } ``` --- ## API Reference Summary | Property | Value | |----------|-------| | Datasource Type (`type`) | `bigquery` | | ConnectionPropertiesMode | `UrlMode` | | Required Parameters | `bigQueryProjectId`, `bigQueryAuth`, `envType` | --- **Source**: [Aliyun DataWorks Developer Reference - BigQuery](https://help.aliyun.com/zh/dataworks/developer-reference/bigquery) **Last Updated**: 2024-10-15 FILE:references/data-sources/clickhouse.md # ClickHouse Datasource Documentation ## Property Definition - **Datasource Type**: `clickhouse` - **Supported Configuration Mode (ConnectionPropertiesMode)**: `UrlMode` (Connection String Mode) --- ## Connection String Mode Parameters | Name | Type | Example Value | Required | Description & Notes | |------|------|---------------|----------|---------------------| | `address` | JSON Array | `[{"host": "127.0.0.1", "port": 8123}]` | Yes | Only a single address is allowed. ClickHouse HTTP port is typically 8123, native port is 9000. | | `database` | String | `xxx_db` | Yes | ClickHouse database name. | | `properties` | JSONObject | `{"useSSL": "false"}` | No | Driver properties. | | `username` | String | `xxx_username` | Yes | Username for ClickHouse database access. | | `password` | String | `xxx_password` | Yes | Password for ClickHouse database access. | | `securityProtocol` | String | `authTypeNone` | No | Authentication method. Enum values:
• `authTypeNone`: No SSL configuration
• `authTypeSsl`: SSL enabled
**Default**: `authTypeNone` | | `sslRootCertificateFile` | String | `123` | No | SSL certificate ID. Required when `securityProtocol` is `authTypeSsl`. | | `envType` | String | `Dev` | Yes | Datasource environment information.
• `Dev`: Development environment
• `Prod`: Production environment | --- ## Datasource Configuration Example ### Connection String Mode ```json { "address": [ { "host": "127.0.0.1", "port": 8123 } ], "securityProtocol": "authTypeNone", "database": "db", "properties": { "connectTimeout": "2000" }, "username": "aliyun", "password": "xxx", "envType": "Dev" } ``` FILE:references/data-sources/datahub.md # DataHub ConnectionProperties Documentation ## Overview - **Data Source Type**: `datahub` - **Supported Configuration Mode**: `UrlMode` (Connection String Mode) --- ## Connection String Mode Parameters | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `regionId` | String | `cn-shanghai` | Yes | The region where DataHub is located. | | `endpoint` | String | `http://dh-cn-shanghai.aliyuncs.com` | Yes | DataHub access endpoint. | | `project` | String | `project-name` | Yes | DataHub project name. | | `accessId` | String | `xxxxx` | Yes | The accessId used to access the data source in AK mode. Required when using AK mode. | | `accessKey` | String | `xxxxx` | Yes | The accessKey used to access the data source in AK mode. Required when using AK mode. | | `envType` | String | `Dev` | Yes | Indicates the data source environment information. Valid values:
- `Dev`: Development environment
- `Prod`: Production environment | --- ## Configuration Example ### Connection String Mode ```json { "envType": "Prod", "endpoint": "http://dh-cn-shanghai.aliyuncs.com", "project": "jiangcheng-test1", "accessId": "xxx", "accessKey": "xxx" } ``` --- *Source: https://help.aliyun.com/zh/dataworks/developer-reference/datahub* FILE:references/data-sources/db2.md # DB2 ConnectionProperties Documentation ## Overview - **Datasource Type**: `db2` - **Supported Configuration Mode**: `UrlMode` (Connection String Mode) --- ## Connection String Mode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `address` | JSON Array | `[{"host": "127.0.0.1", "port": 3306}]` | Yes | Formally an array, but only allows configuration of one set of host and port. | | `database` | String | `mysql_database` | Yes | Database name. | | `jdbcDriver` | String | `db2_1` | Yes | DB2 model type. Valid values: `db2_1`, `as400_1`, `db2_2` | | `username` | String | `xxxxx` | Yes | Username. | | `password` | String | `xxxxx` | Yes | Password. | | `properties` | JSON Object | `{"currentSchema": "abc"}` | No | Driver properties. | | `envType` | String | `Dev` | Yes | Represents the datasource environment information. Valid values: `Dev` (Development environment), `Prod` (Production environment). | --- ## Configuration Example ### Connection String Mode ```json { "address": [ { "host": "127.0.0.1", "port": 5432 } ], "database": "db", "properties": { "currentSchema": "abc" }, "jdbcDriver": "db2_1", "username": "xxxxx", "password": "xxxxx", "envType": "Dev" } ``` --- **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/db2 FILE:references/data-sources/dlf.md # Data Lake Formation Datasource Documentation ## Property Definition - **Datasource type**: `dlf` - **Supported configuration mode (ConnectionPropertiesMode)**: - `InstanceMode` (Instance Mode) --- ## Property Parameters | Name | Type | Example Value | Required | Description and Notes | |------|------|---------------|----------|----------------------| | `regionId` | String | `cn-hangzhou` | Yes | Region ID. | | `catalogId` | String | `clg-paimon-xxx` | Yes | DLF catalog ID. | | `catalogName` | String | `xxx` | Yes | DLF catalog name. | | `catalogType` | String | `Paimon` | Yes | DLF catalog type. Only supports `Paimon`. | | `database` | String | `db1` | Yes | Database name. | | `authType` | String | `Executor` | Yes | DLF access identity. Enumerated values:
• `Executor`: Executor (Development environment)
• `PrimaryAccount`: Primary account (Production environment)
• `SubAccount`: Specified sub-account (Production environment)
• `RamRole`: Specified RAM role (Production environment) | | `authIdentity` | String | `123123` | No | Sub-account ID or Role ID. Required when `authType` is `SubAccount` or `RamRole`. | | `envType` | String | `Dev` | Yes | Datasource environment information.
• `Dev`: Development environment
• `Prod`: Production environment | | `endpoint` | String | `http://cn-hangzhou-vpc.dlf.aliyuncs.com` | Yes | DLF access endpoint. | --- ## Datasource Configuration Example ```json { "envType": "Prod", "authType": "SubAccount", "database": "testdb01", "catalogId": "clg-paimon-xx", "catalogName": "xx", "catalogType": "Paimon", "endpoint": "http://cn-hangzhou-vpc.dlf.aliyuncs.com", "authIdentity": "xxx" } ``` FILE:references/data-sources/dm.md # DM (Dameng) Datasource ConnectionProperties Documentation ## Overview - **Datasource Type**: `dm` - **Supported Configuration Mode**: `UrlMode` (Connection String Mode) --- ## Connection String Mode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `address` | JSON Array | `[{"host": "127.0.0.1", "port": 3306}]` | Yes | Formally an array, but only allows configuration of **1 set** of host and port. | | `username` | String | `xxxxx` | Yes | Username. | | `password` | String | `xxxxx` | Yes | Password. | | `properties` | JSON Object | `{"schema": "db1"}` | No | Driver properties. | | `envType` | String | `Dev` | Yes | Indicates the datasource environment information.
- `Dev`: Development environment
- `Prod`: Production environment | --- ## Configuration Example ### Connection String Mode ```json { "address": [ { "host": "127.0.0.1", "port": 5432 } ], "properties": { "schema": "db1" }, "username": "xxxxx", "password": "xxxxx", "envType": "Dev" } ``` --- **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/dm FILE:references/data-sources/doris.md # Doris Datasource Documentation ## Overview - **Datasource Type**: `doris` - **Supported Configuration Mode**: `UrlMode` (Connection String Mode) --- ## Connection Properties (UrlMode) | Name | Type | Required | Description | |------|------|----------|-------------| | `address` | Array | Yes | Query address. **Only a single address is allowed.** | | `loadAddress` | String | Yes | FE Endpoint. **Multiple addresses are allowed.** | | `database` | String | Yes | Database name. | | `username` | String | Yes | Username for authentication. | | `password` | String | Yes | Password for authentication. | | `properties` | JSON Object | No | Driver properties. | | `envType` | String | Yes | Datasource environment information. Values: `Dev` (Development environment) or `Prod` (Production environment). | --- ## Field Details ### address - **Type**: Array - **Required**: Yes - **Example**: ```json [ { "host": "127.0.0.1", "port": 3306 } ] ``` - **Note**: Only a single address is allowed. ### loadAddress - **Type**: String - **Required**: Yes - **Example**: ```json [ { "host": "127.0.0.1", "port": 3306 } ] ``` - **Note**: FE Endpoint; multiple addresses are allowed. ### database - **Type**: String - **Required**: Yes - **Example**: `dbName` - **Description**: The database name to connect to. ### username - **Type**: String - **Required**: Yes - **Example**: `xxxxx` - **Description**: Username for authentication. ### password - **Type**: String - **Required**: Yes - **Example**: `xxxxx` - **Description**: Password for authentication. ### properties - **Type**: JSON Object - **Required**: No - **Example**: ```json { "useSSL": "false" } ``` - **Description**: Additional driver properties. ### envType - **Type**: String - **Required**: Yes - **Example**: `Dev` - **Description**: Represents the datasource environment information. - `Dev`: Development environment - `Prod`: Production environment --- ## Configuration Example (UrlMode) ```json { "envType": "Prod", "address": [ { "host": "127.0.0.1", "port": "3306" } ], "loadAddress": [ { "host": "127.0.0.2", "port": "8031" } ], "database": "my_database", "username": "my_username", "password": "", "properties": { "socketTimeout": "2000" } } ``` --- ## API Reference Summary | Property | Type | Required | Description | |----------|------|----------|-------------| | `address` | Array | Yes | Query endpoint (single address only) | | `loadAddress` | String | Yes | FE endpoint (multiple addresses allowed) | | `database` | String | Yes | Target database name | | `username` | String | Yes | Authentication username | | `password` | String | Yes | Authentication password | | `properties` | JSON Object | No | Optional driver properties (e.g., `useSSL`, `socketTimeout`) | | `envType` | String | Yes | Environment type: `Dev` or `Prod` | --- **Source**: [Aliyun DataWorks Developer Reference - Doris](https://help.aliyun.com/zh/dataworks/developer-reference/doris) **Last Updated**: 2024-10-15 FILE:references/data-sources/drds.md # DRDS Datasource Documentation ## Property Definition - **Datasource Type**: `drds` - **Supported Configuration Mode (ConnectionPropertiesMode)**: `InstanceMode`, `UrlMode` --- ## InstanceMode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `regionId` | String | `cn-shanghai` | Yes | DRDS instance region. | | `instanceId` | String | `drdsfacbzeu9f7z2` | Yes | DRDS instance ID. | | `database` | String | `db1` | Yes | Database name. | | `username` | String | `user1` | Yes | Username. | | `password` | String | `pass1` | Yes | Password. | | `crossAccountOwnerId` | String | `` | No | Cross-account target Alibaba Cloud main account ID. | | `crossAccountRoleName` | String | `role-name` | No | Cross-account role name. | | `envType` | String | `Prod` | Yes | Environment type. Values: `Dev`, `Prod`. | --- ## UrlMode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `address` | JSON Array | `[{"host": "127.0.0.1", "port": "3306"}]` | Yes | Connection address. Only single host/port configuration allowed. | | `database` | String | `db1` | Yes | Database name. | | `username` | String | `user1` | Yes | Username. | | `password` | String | `pass1` | Yes | Password. | | `properties` | JSON Object | `{"prop1": "value1"}` | No | Driver properties. | | `envType` | String | `Prod` | Yes | Environment type. Values: `Dev`, `Prod`. | --- ## Configuration Examples ### InstanceMode ```json { "envType": "Prod", "regionId": "cn-shanghai", "instanceId": "drds-xxxxx", "database": "mydb", "username": "root", "password": "" } ``` ### InstanceMode (Cross-Account) ```json { "envType": "Prod", "regionId": "cn-shanghai", "instanceId": "drds-xxxxx", "database": "mydb", "username": "root", "password": "", "crossAccountOwnerId": "", "crossAccountRoleName": "cross-account-role" } ``` ### UrlMode ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": "3306"}], "database": "mydb", "username": "root", "password": "" } ``` FILE:references/data-sources/elasticsearch.md # Elasticsearch Datasource Documentation ## Property Definition - **Datasource Type**: `elasticsearch` - **Supported Configuration Mode (ConnectionPropertiesMode)**: `InstanceMode`, `UrlMode` --- ## InstanceMode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `regionId` | String | `cn-shanghai` | Yes | Instance region. | | `instanceId` | String | `es-xxxxx` | Yes | Elasticsearch instance ID. | | `instanceType` | String | `cloudNative` | Yes | Instance type. Values:
• `cloudNative`: Cloud-native
• `serverless`: Serverless | | `username` | String | `elastic` | Yes | Username. | | `password` | String | `xxx` | Yes | Password. | | `envType` | String | `Prod` | Yes | Environment type. Values: `Dev`, `Prod`. | --- ## UrlMode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `endpoint` | String | `http://esxxx.elasticsearch.aliyuncs.com:9200` | Yes | Elasticsearch connection endpoint. | | `authEnable` | String | `enable` | Yes | Enable authentication. Values: `enable`, `disable`. Default: `enable`. | | `username` | String | `elastic` | Conditional | Username. Required when `authEnable=enable`. | | `password` | String | `xxx` | Conditional | Password. Required when `authEnable=enable`. | | `envType` | String | `Prod` | Yes | Environment type. Values: `Dev`, `Prod`. | --- ## Configuration Examples ### InstanceMode ```json { "envType": "Prod", "regionId": "cn-shanghai", "instanceId": "es-xxxxx", "instanceType": "cloudNative", "username": "elastic", "password": "" } ``` ### UrlMode (With Authentication) ```json { "envType": "Prod", "endpoint": "http://esxxx.elasticsearch.aliyuncs.com:9200", "authEnable": "enable", "username": "elastic", "password": "" } ``` ### UrlMode (Anonymous) ```json { "envType": "Prod", "endpoint": "http://192.168.1.100:9200", "authEnable": "disable" } ``` FILE:references/data-sources/ftp.md # FTP Data Source Connection Properties Documentation ## Overview - **Data Source Type**: `ftp` - **Supported Configuration Mode**: `UrlMode` (Connection String Mode) --- ## Connection String Mode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `protocol` | String | `ftp` | Yes | Protocol type. Valid values: `ftp`, `sftp`, `ftps`. | | `host` | String | `10.0.0.1` | Yes | Host address. | | `port` | String | `22` | Yes | Port number. | | `baseDir` | String | `/root/` | No | Base path/directory. | | `securityProtocol` | String | `passwordAuth` | No | Required when using SFTP protocol. Authentication option, valid values: `passwordAuth`, `authTypeSshKey`. | | `username` | String | `xxxxx` | Yes | Username. | | `password` | String | `xxxxx` | No | Password. Required when: protocol is SFTP with `passwordAuth` authentication, or protocol is FTP/FTPS. | | `sshKeyFile` | String | `1` | No | Authentication file ID. Required when authentication option is `authTypeSshKey`. | --- ## Configuration Example ### Connection String Mode ```json { "host": "127.0.0.1", "port": "5432", "protocol": "sftp", "securityProtocol": "passwordAuth", "username": "xxxxx", "password": "xxxxx", "envType": "Dev" } ``` --- *Source: https://help.aliyun.com/zh/dataworks/developer-reference/ftp* *Last Updated: 2024-10-17* FILE:references/data-sources/gbase8a.md # GBASE 8A ConnectionProperties Documentation ## Data Source Type - **Type**: `gbase8a` ## Supported Configuration Modes - **ConnectionPropertiesMode**: `UrlMode` (Connection String Mode) --- ## Connection String Mode Parameters | Name | Type | Example Value | Required | Description & Notes | |------|------|---------------|----------|---------------------| | `address` | JSON Array | `[{"host": "127.0.0.1", "port": 3306}]` | Yes | Formally an array, but only 1 set of host and port is allowed. | | `database` | String | `mysql_database` | Yes | Database name. | | `username` | String | `xxxxx` | Yes | Username. | | `password` | String | `xxxxx` | Yes | Password. | | `properties` | JSON Object | `{"useSSL": "false"}` | No | Driver properties. | | `envType` | String | `Dev` | Yes | Represents the data source environment information.
- `Dev`: Development environment.
- `Prod`: Production environment. | --- ## Configuration Example ### Connection String Mode ```json { "address": [ { "host": "127.0.0.1", "port": 5432 } ], "database": "db", "properties": { "connectTimeout": "2000" }, "username": "xxxxx", "password": "xxxxx", "envType": "Dev" } ``` --- **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/gbase-8a FILE:references/data-sources/graph_database.md # Graph Database Connection Properties Documentation ## Overview - **Data Source Type**: `graph_database` - **Supported Configuration Mode**: `UrlMode` (Connection String Mode) --- ## Connection String Mode Parameters | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `host` | String | `10.0.0.1` | Yes | Graph instance domain name. | | `port` | String | `22` | Yes | Graph instance port number. | | `username` | String | `xxxxx` | No | Graph instance account. | | `password` | String | `xxxxx` | No | Graph instance password. | | `envType` | String | `Dev` | Yes | Indicates the data source environment information. Possible values:
- `Dev`: Development environment
- `Prod`: Production environment | --- ## Configuration Example ### Connection String Mode ```json { "host": "127.0.0.1", "port": "5432", "username": "xxxxx", "password": "xxxxx", "envType": "Dev" } ``` --- *Source: https://help.aliyun.com/zh/dataworks/developer-reference/graph-database* *Last Updated: 2024-10-15 09:31:48* FILE:references/data-sources/hbase.md # HBase Datasource Documentation ## Overview **Datasource Type:** `hbase` **Supported Configuration Mode:** `UrlMode` (Connection String Mode) --- ## ConnectionProperties Parameters (UrlMode) | Name | Type | Example Value | Required | Description & Notes | |------|------|---------------|----------|---------------------| | `hbaseConfig` | JSON Object | `{"hbase.zookeeper.quorum":"localhost:2181", "hbaseVersion":"0.9.4"}` | Yes | HBase configuration. | | `securityProtocol` | String | `authTypeNone` | No | Authentication option. Valid values: `authTypeNone`, `authTypeClientPassword`, `authTypeKerberos` | | `username` | String | `xxxx` | No | Username. Required when `authTypeClientPassword` is used. | | `password` | String | `xxxx` | No | Password. Required when `authTypeClientPassword` is used. | | `kerberosFileConf` | String | `` | No | Kerberos authentication Conf file (reference). Required when `authTypeKerberos` is used. | | `kerberosFileKeytab` | String | `` | No | Kerberos authentication Keytab file (reference). Required when `authTypeKerberos` is used. | | `principal` | String | `xxx@com` | No | Principal. | | `envType` | String | `Dev` | Yes | Indicates the datasource environment. Valid values: `Dev` (Development environment), `Prod` (Production environment). | --- ## Configuration Example (UrlMode) ```json { "hbaseConfig": { "hbase.zookeeper.quorum": "localhost:2181", "hbaseVersion": "0.9.4" }, "securityProtocol": "authTypeClientPassword", "username": "xxxxx", "password": "xxxxx", "envType": "Dev" } ``` --- ## Authentication Types Summary | Type | Description | Required Additional Parameters | |------|-------------|-------------------------------| | `authTypeNone` | No authentication | None | | `authTypeClientPassword` | Username/password authentication | `username`, `password` | | `authTypeKerberos` | Kerberos authentication | `kerberosFileConf`, `kerberosFileKeytab`, `principal` | --- **Last Updated:** 2024-11-06 FILE:references/data-sources/hive.md # Hive Datasource Documentation ## Property Definition - **Datasource Type**: `hive` - **Supported Configuration Modes (ConnectionPropertiesMode)**: - `UrlMode` (Connection String Mode) - `InstanceMode` (Instance Mode) - `CdhMode` (CDH Cluster Mode) --- ## 1. Same-Account Instance Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `regionId` | String | `cn-shanghai` | Yes | Region ID | | `clusterId` | String | `c-d1a993bbcd298315` | Yes | Instance ID | | `database` | String | `db1` | Yes | Database name | | `version` | String | `2.3.9` | Yes | Hive version | | `authType` | String | `Executor` | Yes | OSS access identity. Options: `Executor` (development environment), `PrimaryAccount` (production environment), `SubAccount` (specified sub-account), `RamRole` (specified RAM role) | | `authIdentity` | String | `123123` | No | Sub-account ID or Role ID. Required when `authType` is `SubAccount` or `RamRole` | | `loginMode` | String | `Anonymous` | Yes | Hive login method. **Only supports**: `Anonymous`, `LDAP` | | `username` | String | `xxx` | No | Username. Required when using username/password login | | `password` | String | `xxx` | No | Password. Required when using username/password login | | `securityProtocol` | String | `authTypeNone` | No | SSL authentication. Options: `authTypeNone` (no auth), `authTypeSsl` (enable SSL), `authTypeKerberos` (enable Kerberos) | | `truststoreFile` | String | `1` | No | Truststore certificate file (reference) | | `truststorePassword` | String | `apasara` | No | Truststore password | | `keystoreFile` | String | `2` | No | Keystore certificate file (reference) | | `keystorePassword` | String | `apasara` | No | Keystore password | | `kerberosFileConf` | String | `123123` | No | Kerberos configuration file (reference) | | `kerberosFileKeytab` | String | `123123` | No | Kerberos Keytab file (reference) | | `principal` | String | `xxx@com` | No | Kerberos principal | | `hiveConfig` | JSON Object | `{"fs.oss.accessKeyId": "xxx"}` | No | Extended parameters | | `envType` | String | `Dev` | Yes | Environment type. Options: `Dev` (development), `Prod` (production) | --- ## 2. Cross-Account Instance Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `crossAccountOwnerId` | String | `11111` | Yes | Target Alibaba Cloud primary account UID | | `crossAccountRoleName` | String | `xx-role` | Yes | Target RAM role name | | `regionId` | String | `cn-shanghai` | Yes | Region ID | | `clusterId` | String | `c-d1a993bbcd298315` | Yes | Instance ID | | `database` | String | `db1` | Yes | Database name | | `version` | String | `2.3.9` | Yes | Hive version | | `authType` | String | `RamRole` | Yes | OSS access identity (fixed as `RamRole`) | | `loginMode` | String | `Anonymous` | Yes | Hive login method. **Only supports**: `Anonymous`, `LDAP` | | `username` | String | `xxx` | No | Username. Required when using username/password login | | `password` | String | `xxx` | No | Password. Required when using username/password login | | `securityProtocol` | String | `authTypeNone` | No | SSL authentication. Options: `authTypeNone`, `authTypeSsl`, `authTypeKerberos` | | `truststoreFile` | String | `1` | No | Truststore certificate file (reference) | | `truststorePassword` | String | `apasara` | No | Truststore password | | `keystoreFile` | String | `2` | No | Keystore certificate file (reference) | | `keystorePassword` | String | `apasara` | No | Keystore password | | `kerberosFileConf` | String | `123123` | No | Kerberos configuration file (reference) | | `kerberosFileKeytab` | String | `123123` | No | Kerberos Keytab file (reference) | | `principal` | String | `xxx@com` | No | Kerberos principal | | `hiveConfig` | JSON Object | `{"fs.oss.accessKeyId": "xxx"}` | No | Extended parameters | | `envType` | String | `Dev` | Yes | Environment type. Options: `Dev`, `Prod` | --- ## 3. Connection String Mode (UrlMode) | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `address` | JSON Array | `[{"host":"127.0.0.1","port":"1234"}]` | Yes | Single host address and port only | | `database` | String | `hive_database` | Yes | Database name | | `metaType` | String | `HiveMetastore` | Yes | Metadata type. Options: `HiveMetastore`, `DLF1.0` | | `metastoreUris` | String | `thrift://123:123` | Yes | Metastore URIs | | `version` | String | `2.3.9` | Yes | Hive version | | `accessId` | String | `xxxxx` | No | AccessKey ID. Required when `metaType` is `DLF` | | `accessKey` | String | `xxxxx` | No | AccessKey Secret. Required when `metaType` is `DLF` | | `properties` | JSON Object | `{"useSSL": "false"}` | No | Driver properties | | `defaultFS` | String | `xxx` | No | Default FS | | `loginMode` | String | `Anonymous` | Yes | Hive login method. **Only supports**: `Anonymous`, `LDAP` | | `username` | String | `xxx` | No | Username. Required when using username/password login | | `password` | String | `xxx` | No | Password. Required when using username/password login | | `securityProtocol` | String | `authTypeNone` | No | SSL authentication. Options: `authTypeNone`, `authTypeSsl`, `authTypeKerberos` | | `truststoreFile` | String | `1` | No | Truststore certificate file (reference) | | `truststorePassword` | String | `apasara` | No | Truststore password | | `keystoreFile` | String | `2` | No | Keystore certificate file (reference) | | `keystorePassword` | String | `apasara` | No | Keystore password | | `kerberosFileConf` | String | `123123` | No | Kerberos configuration file (reference) | | `kerberosFileKeytab` | String | `123123` | No | Kerberos Keytab file (reference) | | `principal` | String | `xxx@com` | No | Kerberos principal | | `hiveConfig` | JSON Object | `{"fs.oss.accessKeyId": "xxx"}` | No | Extended parameters | | `envType` | String | `Dev` | Yes | Environment type. Options: `Dev`, `Prod` | --- ## 4. CDH Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `clusterIdentifier` | String | `cdh_cluster` | Yes | CDH cluster identifier | | `database` | String | `db1` | Yes | Database name | | `defaultFS` | String | `xxx` | No | Default FS | | `loginMode` | String | `Anonymous` | Yes | Hive login method. **Only supports**: `Anonymous`, `LDAP` | | `username` | String | `xxx` | No | Username. Required when using username/password login | | `password` | String | `xxx` | No | Password. Required when using username/password login | | `securityProtocol` | String | `authTypeNone` | No | SSL authentication. Options: `authTypeNone`, `authTypeSsl` | | `truststoreFile` | String | `1` | No | Truststore certificate file (reference) | | `truststorePassword` | String | `apasara` | No | Truststore password | | `keystoreFile` | String | `2` | No | Keystore certificate file (reference) | | `keystorePassword` | String | `apasara` | No | Keystore password | | `kerberosFileConf` | String | `123123` | No | Kerberos configuration file (reference) | | `kerberosFileKeytab` | String | `123123` | No | Kerberos Keytab file (reference) | | `principal` | String | `xxx@com` | No | Kerberos principal | | `hiveConfig` | JSON Object | `{"fs.oss.accessKeyId": "xxx"}` | No | Extended parameters | | `envType` | String | `Dev` | Yes | Environment type. Options: `Dev`, `Prod` | --- ## Configuration Examples ### Same-Account Instance Mode ```json { "clusterId": "c-xxxxxxxxx", "regionId": "cn-shanghai", "database": "db", "loginMode": "Anonymous", "version": "2.3.9", "authType": "Executor", "securityProtocol": "authTypeNone", "envType": "Dev" } ``` ### Cross-Account Instance Mode ```json { "clusterId": "c-xxxxxxxxx", "regionId": "cn-shanghai", "database": "db", "loginMode": "LDAP", "version": "2.3.9", "authType": "Executor", "securityProtocol": "authTypeNone", "envType": "Dev" } ``` ### Connection String Mode ```json { "address": [ { "host": "127.0.0.1", "port": 10000 } ], "database": "db", "properties": { "connectTimeout": "2000" }, "username": "aliyun", "password": "xxx", "metastoreUris": "thrift://127.0.0.1:9083", "metaType": "HiveMetastore", "version": "2.3.9", "loginMode": "Anonymous", "securityProtocol": "authTypeNone", "envType": "Dev" } ``` ### CDH Mode ```json { "clusterIdentifier": "c-xxxxxxxxx", "database": "db", "loginMode": "Anonymous", "authType": "Executor", "securityProtocol": "authTypeNone", "envType": "Dev" } ``` --- ## Important Notes > ⚠️ **loginMode Parameter Note**: The `loginMode` parameter for Hive data sources **only supports** the following two values: > - `Anonymous`: Anonymous login > - `LDAP`: LDAP authentication login > > Other values such as `simple` and `disable` are **not supported** and will cause an error: `Unsupported loginMode=simple for Hive data source, only Anonymous and LDAP are supported` FILE:references/data-sources/hologres.md # Hologres Datasource Documentation ## Property Definition - **Datasource Type**: `hologres` - **Supported Configuration Mode (ConnectionPropertiesMode)**: `InstanceMode` (Instance Mode) --- ## Same Account Access Identity Mode | Name | Type | Example Value | Required | Description and Notes | |------|------|---------------|----------|----------------------| | `regionId` | String | `cn-shanghai` | Yes | The Region where the Hologres instance belongs. | | `instanceId` | String | `hgpostcn-cn-xxxxx` | Yes | The Hologres instance ID. | | `database` | String | `holo_db` | Yes | The Hologres database name. | | `warehouseName` | String | `init_warehouse` | No | Hologres compute group information, e.g., the default `init_warehouse`. Note: If a compute group is configured, the database parameter in downstream `findConnection` will append `@` with compute group info, e.g., `"database": "holo_demo@compute_group_name"`. | | `authType` | String | `Executor` | Yes | Datasource access identity. Enum values:
• `Executor`: Executor (Development environment)
• `PrimaryAccount`: Primary account (Production environment)
• `SubAccount`: A specified sub-account (Production environment)
• `RamRole`: A specified RAM role (Production environment) | | `authIdentity` | String | `` | No | In the same account scenario, the cloud account ID of the corresponding task submitter. | | `securityProtocol` | String | `authTypeNone` | No | Whether to enable SSL transmission for datasource access. Enum values:
• `authTypeNone`: Do not use SSL authentication
• `authTypeSsl`: Use SSL authentication | | `sslMode` | String | `require` | No | Verification requirement during SSL transmission:
• `require`: Indicates verification | | `envType` | String | `Dev` | Yes | `envType` indicates the datasource environment information.
• `Dev`: Development environment
• `Prod`: Production environment | --- ## Cross-Account Mode | Name | Type | Example Value | Required | Description and Notes | |------|------|---------------|----------|----------------------| | `regionId` | String | `cn-shanghai` | Yes | The Region where the Hologres instance belongs. Note: Historical data for non-engine datasources may not have this value. | | `instanceId` | String | `hgpostcn-cn-xxxxx` | Yes | The Hologres instance ID. | | `database` | String | `holo_db` | Yes | The Hologres database name. | | `warehouseName` | String | `init_warehouse` | No | Hologres compute group information, e.g., the default `init_warehouse`. Note: If a compute group is configured, the database parameter in downstream `findConnection` will append `@` with compute group info. | | `authType` | String | `RamRole` | Yes | Fixed as `RamRole`. | | `crossAccountOwnerId` | String | `` | Yes | The other party's primary account ID for cross-Alibaba Cloud primary account. Required for cross-account scenarios. | | `crossAccountRoleName` | String | `holo-accross-role-name` | Yes | The role name under the other party's account in cross-account scenarios. | | `securityProtocol` | String | `authTypeNone` | No | Whether to enable SSL transmission for datasource access. Enum values:
• `authTypeNone`: Do not use SSL authentication
• `authTypeSsl`: Use SSL authentication | | `sslMode` | String | `require` | No | Verification requirement during SSL transmission:
• `null`: Indicates no verification
• `require`: Indicates verification | | `envType` | String | `Dev` | Yes | `envType` indicates the datasource environment information.
• `Dev`: Development environment
• `Prod`: Production environment | --- ## Datasource Configuration Examples ### Executor Access Identity Mode ```json { "database": "database", "instanceId": "hgpostcn-cn-xxxxx", "securityProtocol": "authTypeNone", "regionId": "cn-beijing", "envType": "Dev", "authType": "Executor" } ``` ### Cross-Account Mode ```json { "crossAccountOwnerId": "", "crossAccountRoleName": "holo-accross-role-name", "database": "database", "instanceId": "hgpostcn-cn-xxxxx", "securityProtocol": "authTypeNone", "regionId": "cn-beijing", "envType": "Dev", "authType": "RamRole" } ``` FILE:references/data-sources/httpfile.md # HttpFile Datasource Documentation ## Property Definition - **Datasource Type**: `httpfile` - **Supported Configuration Mode (ConnectionPropertiesMode)**: `UrlMode` (Connection String Mode) --- ## UrlMode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `urlPrefix` | String | `http://127.0.0.1` | Yes | URL domain/prefix. | | `defaultHeaders` | String | `{}` | Yes | HTTP request headers as JSON string. Each key is header name, value is header value. | | `envType` | String | `Prod` | Yes | Environment type. Values: `Dev`, `Prod`. | --- ## Configuration Examples ### Basic Configuration ```json { "envType": "Prod", "urlPrefix": "http://example.com", "defaultHeaders": "{}" } ``` ### With Custom Headers ```json { "envType": "Prod", "urlPrefix": "http://example.com", "defaultHeaders": "{\"Authorization\": \"Bearer \", \"Content-Type\": \"application/json\"}" } ``` FILE:references/data-sources/instance-apis.md # InstanceMode Data Source Instance Query API Reference When creating InstanceMode data sources, you can call the corresponding cloud product's OpenAPI to query the instance list for user selection. > **Important**: The user may not have permissions to call these APIs. If the instance list query fails, **do not block the subsequent process**; prompt the user to manually input the instance ID. --- ## Instance Query API Summary | Data source type | Product | API | CLI Command | |-----------|------|-----|----------| | `mysql` | RDS MySQL | DescribeDBInstances | `aliyun rds DescribeDBInstances --user-agent AlibabaCloud-Agent-Skills --Engine MySQL` | | `postgresql` | RDS PostgreSQL | DescribeDBInstances | `aliyun rds DescribeDBInstances --user-agent AlibabaCloud-Agent-Skills --Engine PostgreSQL` | | `sqlserver` | RDS SQL Server | DescribeDBInstances | `aliyun rds DescribeDBInstances --user-agent AlibabaCloud-Agent-Skills --Engine SQLServer` | | `polardb` | PolarDB MySQL | DescribeDBClusters | `aliyun polardb DescribeDBClusters --user-agent AlibabaCloud-Agent-Skills --DBType MySQL` | | `drds` | DRDS | DescribeDrdsInstances | `aliyun drds DescribeDrdsInstances --user-agent AlibabaCloud-Agent-Skills` | | `hologres` | Hologres | ListInstances | `aliyun hologram ListInstances --user-agent AlibabaCloud-Agent-Skills` | | `analyticdb_for_mysql` | AnalyticDB MySQL | DescribeDBClusters | `aliyun adb DescribeDBClusters --user-agent AlibabaCloud-Agent-Skills` | | `analyticdb_for_postgresql` | AnalyticDB PostgreSQL | DescribeDBInstances | `aliyun gpdb DescribeDBInstances --user-agent AlibabaCloud-Agent-Skills` | | `starrocks` | EMR StarRocks | ListCluster | `aliyun emr ListCluster --user-agent AlibabaCloud-Agent-Skills --ClusterType OLAP` | | `milvus` | Milvus | ListInstances | `aliyun milvus ListInstances --user-agent AlibabaCloud-Agent-Skills` | | `mongodb` | MongoDB | DescribeDBInstances | `aliyun dds DescribeDBInstances --user-agent AlibabaCloud-Agent-Skills` | | `kafka` | Kafka | GetInstanceList | `aliyun alikafka GetInstanceList --user-agent AlibabaCloud-Agent-Skills` | | `opensearch` | OpenSearch | ListInstances | `aliyun searchengine ListInstances --user-agent AlibabaCloud-Agent-Skills` | --- ## Detailed API Description ### RDS MySQL (`mysql`) **API**: DescribeDBInstances - Query Instance List **Documentation**: https://help.aliyun.com/zh/rds/apsaradb-rds-for-mysql/api-rds-2014-08-15-describedbinstances-mysql ```bash aliyun rds DescribeDBInstances --user-agent AlibabaCloud-Agent-Skills --RegionId "" --Engine MySQL --PageSize 50 2>/dev/null | jq -r ' .Items.DBInstance[] | "ID: \(.DBInstanceId) | Name: \(.DBInstanceDescription // .DBInstanceId) | Status: \(.DBInstanceStatus)" ' ``` --- ### RDS PostgreSQL (`postgresql`) **API**: DescribeDBInstances - Query Instance List **Documentation**: https://help.aliyun.com/zh/rds/apsaradb-rds-for-postgresql/api-rds-2014-08-15-describedbinstances-postgresql ```bash aliyun rds DescribeDBInstances --user-agent AlibabaCloud-Agent-Skills --RegionId "" --Engine PostgreSQL --PageSize 50 2>/dev/null | jq -r ' .Items.DBInstance[] | "ID: \(.DBInstanceId) | Name: \(.DBInstanceDescription // .DBInstanceId) | Status: \(.DBInstanceStatus)" ' ``` --- ### RDS SQL Server (`sqlserver`) **API**: DescribeDBInstances - Query Instance List **Documentation**: https://help.aliyun.com/zh/rds/apsaradb-rds-for-sql-server/api-rds-2014-08-15-describedbinstances-sqlserver ```bash aliyun rds DescribeDBInstances --user-agent AlibabaCloud-Agent-Skills --RegionId "" --Engine SQLServer --PageSize 50 2>/dev/null | jq -r ' .Items.DBInstance[] | "ID: \(.DBInstanceId) | Name: \(.DBInstanceDescription // .DBInstanceId) | Status: \(.DBInstanceStatus)" ' ``` --- ### PolarDB MySQL (`polardb`) **API**: DescribeDBClusters - Query Cluster List **Documentation**: https://help.aliyun.com/zh/polardb/api-polardb-2017-08-01-describedbclusters ```bash aliyun polardb DescribeDBClusters --user-agent AlibabaCloud-Agent-Skills --RegionId "" --DBType MySQL 2>/dev/null | jq -r ' .Items.DBCluster[] | "ID: \(.DBClusterId) | Name: \(.DBClusterDescription // .DBClusterId) | Status: \(.DBClusterStatus)" ' ``` > **Note**: When creating a data source, use the `clusterId` parameter (not `instanceId`), and specify the `dbType` parameter (`mysql` or `postgresql`). --- ### DRDS (`drds`) **API**: DescribeDrdsInstances - Query Instance List **Documentation**: https://help.aliyun.com/zh/drds/api-drds-2019-01-23-describedrdsinstances ```bash aliyun drds DescribeDrdsInstances --user-agent AlibabaCloud-Agent-Skills --RegionId "" 2>/dev/null | jq -r ' .Instances.Instance[] | "ID: \(.DrdsInstanceId) | Name: \(.Description // .DrdsInstanceId) | Status: \(.Status)" ' ``` > **Note**: The returned JSON path is `.Instances.Instance[]`, not `.Data.Instances.Instance[]`. --- ### Hologres (`hologres`) **API**: ListInstances - Get Instance List **Documentation**: https://help.aliyun.com/zh/hologres/developer-reference/api-hologram-2022-06-01-listinstances ```bash aliyun hologram POST /api/v1/instances --user-agent AlibabaCloud-Agent-Skills --region "" --body '{}' 2>/dev/null | jq -r ' .InstanceList[] | "ID: \(.InstanceId) | Name: \(.InstanceName) | Status: \(.InstanceStatus)" ' ``` > **Note**: The Hologres API requires a POST request with an empty body `--body '{}'`. When creating a data source, `username`/`password` are not required; the `authType` parameter is needed. --- ### AnalyticDB MySQL (`analyticdb_for_mysql`) **API**: DescribeDBClusters - Query Instance List **Documentation**: https://help.aliyun.com/zh/analyticdb/analyticdb-for-mysql/developer-reference/api-adb-2019-03-15-describedbclusters ```bash aliyun adb DescribeDBClusters --user-agent AlibabaCloud-Agent-Skills --RegionId "" 2>/dev/null | jq -r ' .Items.DBCluster[] | "ID: \(.DBClusterId) | Name: \(.DBClusterDescription // .DBClusterId) | Status: \(.DBClusterStatus)" ' ``` > **Note**: This API does not support the `--PageSize` parameter; it returns 30 records by default. --- ### AnalyticDB PostgreSQL (`analyticdb_for_postgresql`) **API**: DescribeDBInstances - Query Database Instance List **Documentation**: https://help.aliyun.com/zh/analyticdb/analyticdb-for-postgresql/developer-reference/api-gpdb-2016-05-03-describedbinstances ```bash aliyun gpdb DescribeDBInstances --user-agent AlibabaCloud-Agent-Skills --RegionId "" --PageSize 50 2>/dev/null | jq -r ' .Items.DBInstance[] | "ID: \(.DBInstanceId) | Name: \(.DBInstanceDescription // .DBInstanceId) | Status: \(.DBInstanceStatus)" ' ``` --- ### StarRocks (`starrocks`) **API**: DescribeInstances - Query Instance List **Documentation**: https://help.aliyun.com/zh/emr-serverless-starrocks/developer-reference/api-starrocks-2022-10-19-describeinstances ```bash aliyun starrocks DescribeInstances --user-agent AlibabaCloud-Agent-Skills --region "" 2>/dev/null | jq -r ' .Data[] | "ID: \(.InstanceId) | Name: \(.InstanceName) | Status: \(.InstanceStatus)" ' ``` > **Note**: The returned JSON path is `.Data[]`, and field names are lowercase. When creating a data source, specify the `instanceType` parameter (`serverless` or `emr-olap`). --- ### Milvus (`milvus`) **API**: list-instances - Get Instance List **Documentation**: https://help.aliyun.com/zh/milvus/developer-reference/api-milvus-2023-10-12-listinstances > **Prerequisites**: You need to install the Milvus CLI plugin first: > ```bash > aliyun plugin install --names aliyun-cli-milvus > ``` ```bash aliyun milvus list-instances --user-agent AlibabaCloud-Agent-Skills --region "" 2>/dev/null | jq -r ' .Data[] | "ID: \(.InstanceId) | Name: \(.ClusterName) | Status: \(.InstanceStatus)" ' ``` > **Note**: The command and parameters are lowercase (`list-instances`, `--region`). The returned JSON path is `.Data[]`. --- ### MongoDB (`mongodb`) **API**: DescribeDBInstances - Query MongoDB Instance List **Documentation**: https://help.aliyun.com/zh/mongodb/developer-reference/api-dds-2015-12-01-describedbinstances ```bash aliyun dds DescribeDBInstances --user-agent AlibabaCloud-Agent-Skills --RegionId "" 2>/dev/null | jq -r ' .DBInstances.DBInstance[] | "ID: \(.DBInstanceId) | Name: \(.DBInstanceDescription // .DBInstanceId) | Status: \(.DBInstanceStatus)" ' ``` > **Note**: This API returns 30 records by default; the `--PageSize` parameter is not needed. --- ### Kafka (`kafka`) **API**: GetInstanceList - Query Instance Information for a Specific Region **Documentation**: https://help.aliyun.com/zh/apsaramq-for-kafka/cloud-message-queue-for-kafka/developer-reference/api-alikafka-2019-09-16-getinstancelist ```bash aliyun alikafka GetInstanceList --user-agent AlibabaCloud-Agent-Skills --RegionId "" 2>/dev/null | jq -r ' .InstanceList.InstanceVO[] | "ID: \(.InstanceId) | Name: \(.InstanceName) | Status: \(.Status)" ' ``` --- ### OpenSearch (`opensearch`) **API**: ListInstances - Get Instance List **Documentation**: https://help.aliyun.com/zh/open-search/developer-reference/api-searchengine-2021-10-25-listinstances ```bash aliyun searchengine ListInstances --user-agent AlibabaCloud-Agent-Skills --RegionId "" 2>/dev/null | jq -r ' .result[] | "ID: \(.instanceId) | Name: \(.description // .instanceId) | Type: \(.edition)" ' ``` > **Note**: The returned JSON fields are lowercase: `result`, `instanceId`, etc. --- ## Recommended Process When creating an InstanceMode data source: 1. **Try to query the instance list** ```bash aliyun --RegionId "" ... ``` 2. **Query successful** → Display list for user to select instance ID 3. **Query failed** → Prompt user: "Unable to retrieve the instance list, please enter the instance ID directly" --- ## Error Handling Common Errors and Handling: | Error | Cause | Handling | |------|------|----------| | `Forbidden.Access` | No API permission for this product | Prompt user to input instance ID | | `InvalidRegionId` | Region does not exist | Check region parameter | | `ServiceUnavailable` | Product service unavailable | Prompt user to retry later or input manually | **Key Principle**: When instance list query fails, **do not block the data source creation process**; fall back to letting the user manually input the instance ID. FILE:references/data-sources/kafka.md # Kafka Datasource Documentation ## Property Definition - **Datasource Type**: `kafka` - **Supported Configuration Mode (ConnectionPropertiesMode)**: `InstanceMode`, `UrlMode` --- ## UrlMode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `address` | JSON Array | `[{"host": "192.168.1.100", "port": "9092"}]` | Yes | Kafka connection address. | | `version` | String | `2.0` | Yes | Client version. Only supports `"2.0"` or `"3.4"`. | | `securityProtocol` | String | `authTypeNone` | Yes | Authentication method. Values:
• `authTypeNone`: No authentication
• `authTypeSaslSsl`: SASL SSL
• `authTypeSaslPlaintext`: SASL Plaintext
• `authTypeSsl`: SSL | | `saslMechanism` | String | `plain` | Conditional | SASL mechanism. Required when `securityProtocol=authTypeSaslSsl` or `authTypeSaslPlaintext`. Values: `gssapi`, `plain`, `scram-sha-256`, `scram-sha-512`. | | `saslUsername` | String | `user` | No | SASL username. | | `saslPassword` | String | `xxx` | No | SASL password. | | `truststoreFile` | String | `` | Conditional | Truststore certificate file ID. Required when `securityProtocol=authTypeSsl` or `authTypeSaslSsl`. | | `truststorePassword` | String | `xxx` | No | Truststore password. | | `keystoreFile` | String | `` | No | Keystore certificate file ID. | | `keystorePassword` | String | `xxx` | No | Keystore password. | | `keyPassword` | String | `xxx` | No | Private key password. | | `kerberosFileKeytab` | String | `` | Conditional | Keytab authentication file. Required when using Kerberos. | | `kerberosFileConf` | String | `` | Conditional | krb5.conf configuration file. Required when using Kerberos. | | `jaasFileConf` | String | `` | No | JAAS configuration file. | | `kafkaConfig` | JSON Object | `{}` | No | Kafka extended parameters. | | `envType` | String | `Prod` | Yes | Environment type. Values: `Dev`, `Prod`. | --- ## InstanceMode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `regionId` | String | `cn-shanghai` | Yes | Region where the instance belongs. | | `instanceId` | String | `alikafka-xxxxx` | Yes | Kafka cluster ID. | | `version` | String | `2.0` | Yes | Client version. Only supports `"2.0"` or `"3.4"`. | | `securityProtocol` | String | `authTypeNone` | Yes | Authentication method. | | `crossAccountOwnerId` | String | `` | No | Cross-account target Alibaba Cloud main account ID. | | `crossAccountRoleName` | String | `role-name` | No | Cross-account role name. | | `envType` | String | `Prod` | Yes | Environment type. Values: `Dev`, `Prod`. | --- ## Configuration Examples ### UrlMode (No Authentication) ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": "9092"}], "version": "2.0", "securityProtocol": "authTypeNone" } ``` ### UrlMode (SASL Plaintext) ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": "9092"}], "version": "2.0", "securityProtocol": "authTypeSaslPlaintext", "saslMechanism": "plain", "saslUsername": "user", "saslPassword": "" } ``` ### InstanceMode ```json { "envType": "Prod", "regionId": "cn-shanghai", "instanceId": "alikafka-xxxxx", "version": "2.0", "securityProtocol": "authTypeNone" } ``` FILE:references/data-sources/kingbasees.md # KingbaseES Datasource ConnectionProperties Documentation ## Overview - **Datasource Type**: `kingbasees` - **Supported Configuration Mode**: `UrlMode` (Connection String Mode) --- ## Connection String Mode Parameters | Name | Type | Required | Description | |------|------|----------|-------------| | `address` | JSON Array | Yes | An array format, but only allows configuration of 1 set of host and port. | | `database` | String | Yes | Database name. | | `username` | String | Yes | Username. | | `password` | String | Yes | Password. | | `properties` | JSON Object | No | Driver properties. | | `envType` | String | Yes | Indicates the datasource environment information. Options: `Dev` (Development environment) or `Prod` (Production environment). | --- ## Parameter Details ### address **Type**: JSON Array **Example**: ```json [ { "host": "127.0.0.1", "port": 3306 } ] ``` **Note**: Although it's an array format, only one set of host and port is allowed. --- ### database **Type**: String **Example**: `mysql_database` **Description**: The name of the database. --- ### username **Type**: String **Example**: `xxxxx` **Description**: The username for authentication. --- ### password **Type**: String **Example**: `xxxxx` **Description**: The password for authentication. --- ### properties **Type**: JSON Object **Example**: ```json { "useSSL": "false" } ``` **Description**: Driver properties (optional). --- ### envType **Type**: String **Example**: `Dev` **Description**: Indicates the datasource environment information. **Valid Values**: - `Dev`: Development environment - `Prod`: Production environment --- ## Configuration Example ### Connection String Mode ```json { "address": [ { "host": "127.0.0.1", "port": 5432 } ], "database": "db", "properties": { "connectTimeout": "2000" }, "username": "xxxxx", "password": "xxxxx", "envType": "Dev" } ``` --- **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/kingbasees FILE:references/data-sources/lindorm.md # Lindorm Datasource Documentation **Last Updated:** 2024-10-15 09:31:57 ## Property Definition | Property | Value | |----------|-------| | **Data Source Type (type)** | `lindorm` | | **Supported Configuration Mode (ConnectionPropertiesMode)** | `UrlMode` (Connection String Mode), `InstanceMode` (Instance Mode) | > **Note:** > - **Wide Table Engine**: Only supports `UrlMode` (Connection String Mode). > - **Compute Engine**: Supports both `UrlMode` and `InstanceMode`. `InstanceMode` supports cross-account access. --- ## Connection String Mode Parameters (UrlMode) | Name | Type | Example Value | Required | Description and Notes | |------|------|---------------|----------|----------------------| | `seedserver` | String | `ld-xxxxxxxxxxxxx.rds.aliyuncs.com:30020` | Yes | Connection address. | | `username` | String | `user` | Yes | Username for accessing Lindorm. | | `password` | String | `pass` | Yes | Password for accessing Lindorm. | | `namespace` | String | `default` | Yes | Namespace / Database name. | | `envType` | String | `Dev` | Yes | Environment type: `Dev` (Development), `Prod` (Production). | --- ## Instance Mode Parameters (InstanceMode) > **Applicable to:** Compute Engine only. | Name | Type | Example Value | Required | Description and Notes | |------|------|---------------|----------|----------------------| | `regionId` | String | `cn-xxxxx` | Yes | Region ID. | | `instanceId` | String | `ld-xxxxxxxxxxxxx` | Yes | Lindorm instance ID. | | `namespace` | String | `db` | Yes | Namespace / Database name. | | `username` | String | `user` | Yes | Username for accessing Lindorm. | | `password` | String | `pass` | Yes | Password for accessing Lindorm. | | `engineType` | String | `compute` | Yes | Engine type. Fixed value: `compute` (Compute Engine). | | `envType` | String | `Prod` | Yes | Environment type: `Dev` (Development), `Prod` (Production). | | `crossAccountOwnerId` | String | `111111111` | No | Target Alibaba Cloud primary account UID (cross-account only). | | `crossAccountRoleName` | String | `xx-role` | No | Target RAM role name (cross-account only). | --- ## Configuration Examples ### Connection String Mode (UrlMode) ```json { "seedserver": "ld-xxxxxxxxxxxxx.rds.aliyuncs.com:30020", "username": "user", "password": "pass", "namespace": "default", "envType": "Dev" } ``` ### Instance Mode (InstanceMode) - Compute Engine ```json { "envType": "Prod", "engineType": "compute", "regionId": "cn-xxxxx", "instanceId": "ld-xxxxxxxxxxxxx", "namespace": "db", "username": "user", "password": "pass" } ``` ### Instance Mode (InstanceMode) - Cross-Account Compute Engine ```json { "envType": "Prod", "engineType": "compute", "regionId": "cn-xxxxx", "instanceId": "ld-xxxxxxxxxxxxx", "namespace": "db", "username": "user", "password": "pass", "crossAccountOwnerId": "111111111", "crossAccountRoleName": "xx-role" } ``` --- ## API Reference Summary - **Source:** DataWorks Developer Reference > Appendix > Data Source Connection Information (ConnectionProperties) - **Platform:** Alibaba Cloud DataWorks - **Navigation:** Previous: KingbaseES | Next: LogHub ## Related Lindorm OpenAPIs | API | Description | Documentation | |-----|-------------|---------------| | `GetLindormInstance` | Get detailed information about a Lindorm instance. | [GetLindormInstance](https://help.aliyun.com/zh/lindorm/developer-reference/api-hitsdb-2020-06-15-getlindorminstance) | | `GetLindormInstanceEngineList` | Get the list of engine types supported by a Lindorm instance. | [GetLindormInstanceEngineList](https://help.aliyun.com/zh/lindorm/developer-reference/api-hitsdb-2020-06-15-getlindorminstanceenginelist) | | `GetLindormInstanceList` | Get the list of Lindorm instances. | [GetLindormInstanceList](https://help.aliyun.com/zh/lindorm/developer-reference/api-hitsdb-2020-06-15-getlindorminstancelist) | FILE:references/data-sources/loghub.md # LogHub Data Source ConnectionProperties ## Property Definition - **Data source type**: `loghub` - **Supported configuration mode**: UrlMode (Connection String Mode) --- ## Connection String Mode Parameters | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | regionId | String | cn-sanghai | Yes | Region where LogHub is located. | | endpoint | String | http://cn-beijing.log.aliyuncs.com | Yes | LogHub access endpoint. | | project | String | project-name | Yes | LogHub project name. | | accessId | String | xxxxx | Yes | AccessId used for accessing the data source in AK mode. Required in AK mode. | | accessKey | String | xxxxx | Yes | AccessKey used for accessing the data source in AK mode. Required in AK mode. | | envType | String | Dev | Yes | envType indicates data source environment information.
- **Dev**: Development environment
- **Prod**: Production environment | --- ## Configuration Examples ### Connection String Mode ```json { "envType": "Prod", "endpoint": "http://cn-beijing.log.aliyuncs.com", "project": "jiangcheng-test1", "accessId": "xxx", "accessKey": "xxx" } ``` --- *Source: https://help.aliyun.com/zh/dataworks/developer-reference/loghub* FILE:references/data-sources/mariadb.md # MariaDB DataSource ConnectionProperties Documentation ## Attribute Definition - **Datasource type**: `mariadb` - **Supported configuration mode**: `UrlMode` (Connection String Mode) --- ## Connection String Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | address | JSON Array | `[{"host": "127.0.0.1", "port": 3306}]` | Yes | Formally an array, but only allows configuration of 1 set of host and port. | | database | String | `mysql_database` | Yes | Database name. | | username | String | `xxxxx` | Yes | Username. | | password | String | `xxxxx` | Yes | Password. | | properties | JSON Object | `{"useSSL": "false"}` | No | Driver properties. | | envType | String | `Dev` | Yes | envType indicates datasource environment information.
- **Dev:** Development environment
- **Prod:** Production environment | --- ## Datasource Configuration Example ### Connection String Mode ```json { "address": [ { "host": "127.0.0.1", "port": 5432 } ], "database": "db", "properties": { "connectTimeout": "2000" }, "username": "xxxxx", "password": "xxxxx", "envType": "Dev" } ``` --- **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/mariadb FILE:references/data-sources/maxcompute.md # MaxCompute Datasource Documentation ## Overview | Property | Value | |----------|-------| | **Datasource Type** | `maxcompute` | | **Supported Configuration Mode** | `UrlMode` (Connection String Mode) | --- ## Query MaxCompute Projects Before creating a MaxCompute data source, you can query the list of available MaxCompute projects. > **Reference**: [MaxCompute ListProjects API](https://help.aliyun.com/zh/maxcompute/user-guide/api-maxcompute-2022-01-04-listprojects) ```bash aliyun maxcompute ListProjects --user-agent AlibabaCloud-Agent-Skills --RegionId "" --maxItem 50 2>/dev/null | jq -r ' .data.projects[] | "Project Name: \(.name) | Status: \(.status)" ' ``` --- ## ConnectionProperties Parameters ### 1. Same-Account Access Identity Mode | Name | Type | Required | Example | Description | |------|------|----------|---------|-------------| | `project` | String | Yes | `hello_mc_project` | MaxCompute project name | | `regionId` | String | Yes | `cn-shanghai` | Region identifier where the MaxCompute project resides. Examples: `cn-shanghai` (Shanghai), `cn-beijing` (Beijing) | | `endpointMode` | String | Yes | `SelfAdaption` | Access address configuration mode. Values: `SelfAdaption` (adaptive), `Custom` (custom access address) | | `endpoint` | String | No | `http://service.cn-shanghai.maxcompute.aliyun-inc.com/api` | MaxCompute Service access endpoint (classic network). Format: `service.${regionId}.maxcompute.aliyun-inc.com/api` or `service.${regionId}-intranet.maxcompute.aliyun-inc.com/api` | | `tunnelServer` | String | No | `http://dt.cn-shanghai.maxcompute.aliyun-inc.com` | MaxCompute Tunnel access endpoint (classic network). Format: `dt.${regionId}.maxcompute.aliyun-inc.com` or `dt.${regionId}-intranet.maxcompute.aliyun-inc.com` | | `authType` | String | Yes | `Executor` | Datasource access identity. Values (case-insensitive): `Executor`, `TaskOwner`, `PrimaryAccount`, `SubAccount`, `RamRole`, `HadoopUser` | | `authIdentity` | String | No | `` | Cloud account ID of the task submitter (same-account scenario) | | `envType` | String | Yes | `Dev` | Datasource environment. Values: `Dev` (development), `Prod` (production) | --- ### 2. Cross-Account Mode | Name | Type | Required | Example | Description | |------|------|----------|---------|-------------| | `project` | String | Yes | `hello_mc_project` | MaxCompute project name | | `regionId` | String | Yes | `cn-shanghai` | Region identifier. Examples: `cn-shanghai`, `cn-beijing` | | `endpointMode` | String | Yes | `SelfAdaption` | Access address configuration mode: `SelfAdaption` or `Custom` | | `endpoint` | String | No | `http://service.cn-shanghai.maxcompute.aliyun-inc.com/api` | MaxCompute Service access endpoint (classic network) | | `tunnelServer` | String | No | `http://dt.cn-shanghai.maxcompute.aliyun-inc.com` | MaxCompute Tunnel access endpoint (classic network) | | `authType` | String | Yes | `RamRole` | Fixed value: `RamRole` | | `crossAccountOwnerId` | String | Yes | `` | Target Alibaba Cloud primary account ID (required for cross-account) | | `crossAccountRoleName` | String | Yes | `mc-accross-role-name` | Role name under the target account | | `envType` | String | Yes | `Dev` | Datasource environment: `Dev` or `Prod` | --- ## Configuration Examples ### PrimaryAccount Access Identity Mode (Recommended) > **Note**: MaxCompute data source does not support `Executor` identity. Please use `PrimaryAccount`. ```json { "envType": "Prod", "project": "skyfire_20221228", "regionId": "cn-shanghai", "endpointMode": "SelfAdaption", "authType": "PrimaryAccount" } ``` ### Custom Endpoint Mode ```json { "endpoint": "http://service.cn-shanghai.maxcompute.aliyun-inc.com/api", "envType": "Prod", "project": "skyfire_20221228", "authType": "PrimaryAccount", "endpointMode": "Custom", "regionId": "cn-shanghai" } ``` ### Cross-Account Mode ```json { "project": "skyfire_20221228", "crossAccountOwnerId": "", "crossAccountRoleName": "mc-accross-role-name", "endpointMode": "SelfAdaption", "envType": "Prod", "authType": "RamRole", "regionId": "cn-shanghai" } ``` ## AuthType Enum Values > **Note**: MaxCompute data source **does not support** `Executor` identity. `PrimaryAccount` is recommended. | Value | Description | MaxCompute Support | |-------|-------------|-------------------| | `Executor` | Executor identity | **Not Supported** | | `TaskOwner` | Task owner identity | Supported | | `PrimaryAccount` | Primary account identity | **Recommended** | | `SubAccount` | Specified sub-account identity | Supported | | `RamRole` | Specified RAM role identity | Supported (cross-account scenario) | | `HadoopUser` | Identity mapping for EMR/Hadoop scenarios | Supported | ## Endpoint Format Notes - **Service Endpoint Formats:** - Classic: `service.${regionId}.maxcompute.aliyun-inc.com/api` - New format: `service.${regionId}-intranet.maxcompute.aliyun-inc.com/api` - **Tunnel Endpoint Formats:** - Classic: `dt.${regionId}.maxcompute.aliyun-inc.com` - New format: `dt.${regionId}-intranet.maxcompute.aliyun-inc.com` FILE:references/data-sources/memcache.md # Memcache Data Source ConnectionProperties ## Basic Information - **Data source type**: `memcache` - **Supported configuration mode**: `UrlMode` (Connection String Mode) --- ## Connection String Mode Parameters | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | proxy | String | `127.0.0.1` | Yes | Proxy Host. | | port | String | `22` | Yes | Port number. | | username | String | `xxxxx` | Yes | Username. | | password | String | `xxxxx` | Yes | Password. | | envType | String | `Dev` | Yes | envType indicates data source environment information.
- `Dev`: Development environment
- `Prod`: Production environment | --- ## Configuration Examples ### Connection String Mode ```json { "proxy": "127.0.0.1", "port": "5432", "username": "xxxxx", "password": "xxxxx", "envType": "Dev" } ``` --- **Last updated**: 2024-10-15 09:32:01 **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/memcache FILE:references/data-sources/milvus.md # Milvus Data Source ConnectionProperties **Data source type**: `milvus` **Supported configuration modes**: - InstanceMode (Instance Mode) - UrlMode (Connection String Mode) --- ## Instance Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | regionId | String | cn-shanghai | Yes | Region where the instance is located. | | ConnectionPropertiesMode | String | InstanceMode | Yes | Configuration mode | | database | String | xxxxx | No | Database name | | instanceId | String | c-dd8f**** | Yes | Instance ID | | username | String | xxxxx | Yes | Username | | password | String | xxxxx | Yes | Password | | envType | String | Dev | Yes | envType indicates data source environment information.
- Dev: Development environment.
- Prod: Production environment. | ### Instance Mode Configuration Example ```json { "envType": "Prod", "regionId": "cn-beijing", "instanceId": "c-dd8f71372xxxx", "database": "default", "username": "root", "password": "xxxxxxx" } ``` --- ## Connection String Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | endpoint | String | http://cmilvusxxxx.com:19530 | Yes | Connection endpoint | | ConnectionPropertiesMode | String | UrlMode | Yes | Configuration mode | | database | String | xxxxx | No | Database name | | username | String | xxxxx | Yes | Username | | password | String | xxxxx | Yes | Password | | authType | String | USERNAME_PASSWORD | Yes | Authentication method | | envType | String | Dev | Yes | envType indicates data source environment information.
- Dev: Development environment.
- Prod: Production environment. | ### Connection String Mode Configuration Example ```json { "envType": "Prod", "endpoint": "http://c-dd8xxxxx.milvus.aliyuncs.com:19530", "database": "default", "username": "root", "password": "xxxx", "authType": "USERNAME_PASSWORD" } ``` --- **Last updated**: 2025-03-27 14:05:17 **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/milvus FILE:references/data-sources/mongodb.md # MongoDB Datasource Documentation ## Property Definition - **Datasource Type**: `mongodb` - **Supported Configuration Mode (ConnectionPropertiesMode)**: `InstanceMode`, `UrlMode` --- ## InstanceMode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `regionId` | String | `cn-shanghai` | Yes | The region where the MongoDB instance belongs. | | `instanceId` | String | `dds-2zebc89f45b238b4` | Yes | MongoDB instance ID. | | `database` | String | `my_db` | Yes | Database name. | | `username` | String | `root` | Yes | Username. | | `password` | String | `xxx` | Yes | Password. | | `authDb` | String | `admin` | Yes | Authentication database. | | `engineVersion` | String | `4.x` | No | MongoDB engine version. Values: `4.x`, `5.x`, `6.x`, `7.x`. | | `authType` | String | `authTypeNone` | No | Authentication method. Values:
• `authTypeNone`: No SSL
• `authTypeSsl`: SSL authentication | | `truststoreFile` | String | `` | Conditional | Truststore certificate file ID. Required when `authType=authTypeSsl`. | | `truststorePassword` | String | `xxx` | No | Truststore password. | | `properties` | JSON Object | `{"ssl": "true"}` | No | Advanced properties. | | `envType` | String | `Prod` | Yes | Environment type. Values: `Dev`, `Prod`. | --- ## UrlMode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `address` | JSON Array | `[{"host": "127.0.0.1", "port": "27017"}]` | Yes | MongoDB connection address. | | `database` | String | `my_db` | Yes | Database name. | | `username` | String | `root` | Yes | Username. | | `password` | String | `xxx` | Yes | Password. | | `authDb` | String | `admin` | Yes | Authentication database. | | `engineVersion` | String | `4.x` | No | MongoDB engine version. Values: `4.x`, `5.x`, `6.x`, `7.x`. | | `authType` | String | `authTypeNone` | No | Authentication method. Values:
• `authTypeNone`: No SSL
• `authTypeSsl`: SSL authentication | | `truststoreFile` | String | `` | Conditional | Truststore certificate file ID. Required when `authType=authTypeSsl`. | | `truststorePassword` | String | `xxx` | No | Truststore password. | | `properties` | JSON Object | `{"ssl": "true"}` | No | Advanced properties. | | `envType` | String | `Prod` | Yes | Environment type. Values: `Dev`, `Prod`. | --- ## Configuration Examples ### InstanceMode ```json { "envType": "Prod", "instanceId": "dds-xxxxx", "regionId": "cn-shanghai", "database": "my_db", "username": "root", "password": "", "authDb": "admin", "engineVersion": "5.x" } ``` ### UrlMode ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": "27017"}], "database": "my_db", "username": "root", "password": "", "authDb": "admin", "engineVersion": "5.x" } ``` ### UrlMode with SSL ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": "27017"}], "database": "my_db", "username": "root", "password": "", "authDb": "admin", "engineVersion": "5.x", "authType": "authTypeSsl", "truststoreFile": "" } ``` FILE:references/data-sources/mysql.md # MySQL DataSource ConnectionProperties Documentation ## Overview - **Data Source Type**: `mysql` - **Supported Configuration Modes**: - `UrlMode` (Connection String Mode) - `InstanceMode` (Instance Mode) --- ## 1. Same-Account Instance Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `regionId` | String | `cn-shanghai` | Yes | The region where the instance belongs. Note: Historical non-engine data sources may not have this value. | | `instanceId` | String | `rm-xxxxxxxxx` | Yes | Instance ID. | | `database` | String | `mysql_database` | Yes | Database name. | | `username` | String | `xxxxx` | Yes | Username. | | `password` | String | `xxxxx` | Yes | Password. | | `securityProtocol` | String | `authTypeNone` | No | SSL authentication setting. Values: `authTypeNone` (no authentication), `authTypeSsl` (enable SSL authentication). | | `truststoreFile` | String | `1` | No | Truststore certificate file (reference). | | `truststorePassword` | String | `apasara` | No | Truststore password. | | `keystoreFile` | String | `2` | No | Keystore certificate file (reference). | | `keystorePassword` | String | `apasara` | No | Keystore password. | | `driverVersion` | String | `8.2.0` | No | Driver version. Enumerated values: (empty), `8.2.0`, `5.1.49`, `5.1.46`. | | `authType` | String | `PrimaryAccount` | No | Required for OSS Binlog reading support. Values: `PrimaryAccount`, `SubAccount`, `RamRole`. | | `authIdentity` | String | `123456` | No | Required when `authType` is `SubAccount` (specify sub-account ID) or `RamRole` (specify role ID). Not needed for cross-account scenarios. | | `readOnlyDBInstance` | String | `rm-uf65l3bwae8w8r35` | No | Standby instance ID. | | `envType` | String | `Dev` | Yes | Environment type. Values: `Dev` (development environment), `Prod` (production environment). | ### Example ```json { "instanceId": "rm-xxxxxxxxx", "regionId": "cn-shanghai", "database": "db", "username": "aliyun", "password": "xxx", "securityProtocol": "authTypeNone", "envType": "Dev" } ``` --- ## 2. Cross-Account Instance Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `regionId` | String | `cn-shanghai` | Yes | The region where the instance belongs. Note: Historical non-engine data sources may not have this value. | | `instanceId` | String | `rm-xxxxxxxxx` | Yes | Instance ID. | | `database` | String | `mysql_database` | Yes | Database name. | | `username` | String | `xxxxx` | Yes | Username. | | `password` | String | `xxxxx` | Yes | Password. | | `crossAccountOwnerId` | String | `1` | Yes | Cross-account target cloud account ID. | | `crossAccountRoleName` | String | `mysql-role` | Yes | Cross-account target RAM role name. | | `securityProtocol` | String | `authTypeNone` | No | SSL authentication setting. Values: `authTypeNone` (no authentication), `authTypeSsl` (enable SSL authentication). | | `truststoreFile` | String | `1` | No | Truststore certificate file (reference). | | `truststorePassword` | String | `apasara` | No | Truststore password. | | `keystoreFile` | String | `2` | No | Keystore certificate file (reference). | | `keystorePassword` | String | `apasara` | No | Keystore password. | | `driverVersion` | String | `8.2.0` | No | Driver version. Enumerated values: (empty), `8.2.0`, `5.1.49`, `5.1.46`. | | `authType` | String | `PrimaryAccount` | No | Required for OSS Binlog reading support. Values: `RamRole`. | | `readOnlyDBInstance` | String | `rm-uf65l3bwae8w8r35` | No | Standby instance ID. | | `envType` | String | `Dev` | Yes | Environment type. Values: `Dev` (development environment), `Prod` (production environment). | ### Example ```json { "instanceId": "rm-xxxxxxxxx", "regionId": "cn-shanghai", "database": "db", "username": "aliyun", "password": "xxx", "crossAccountOwnerId": "1234567890", "crossAccountRoleName": "my_ram_role", "securityProtocol": "authTypeNone", "envType": "Dev" } ``` --- ## 3. Connection String Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `address` | Array | `[{"host": "127.0.0.1", "port": 3306}]` | Yes | Allows configuration of multiple host addresses and ports. | | `database` | String | `mysql_database` | Yes | Database name. | | `username` | String | `xxxxx` | Yes | Username. | | `password` | String | `xxxxx` | Yes | Password. | | `securityProtocol` | String | `authTypeNone` | No | SSL authentication setting. Values: `authTypeNone` (no authentication), `authTypeSsl` (enable SSL authentication). | | `truststoreFile` | String | `1` | No | Truststore certificate file (reference). | | `truststorePassword` | String | `apasara` | No | Truststore password. | | `keystoreFile` | String | `2` | No | Keystore certificate file (reference). | | `keystorePassword` | String | `apasara` | No | Keystore password. | | `properties` | JSON Object | `{"useSSL": "false"}` | No | Driver properties. | | `envType` | String | `Dev` | Yes | Environment type. Values: `Dev` (development environment), `Prod` (production environment). | ### Example ```json { "address": [ { "host": "127.0.0.1", "port": 3306 } ], "database": "db", "properties": { "connectTimeout": "2000" }, "username": "aliyun", "password": "xxx", "envType": "Dev" } ``` --- **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/mysql FILE:references/data-sources/opensearch.md # OpenSearch ConnectionProperties Documentation ## Overview **Data Source Type**: `opensearch` **Supported Configuration Mode (ConnectionPropertiesMode)**: - `InstanceMode` (Instance Mode) --- ## Instance Mode Parameters | Name | Type | Example Value | Required | Description & Notes | |------|------|---------------|----------|---------------------| | `instanceType` | String | `vectorSearchVersion` | Yes | OpenSearch engine type. Options:
- `vectorSearchVersion` (Vector Search Version)
- `recallEnginVersion` (Recall Engine Version) | | `regionId` | String | `cn-shanghai` | Yes | The region where the instance is located. | | `ConnectionPropertiesMode` | String | `InstanceMode` | Yes | Configuration mode. | | `instanceId` | String | `ha-cn-kve****` | Yes | Instance ID. | | `username` | String | `xxxxx` | Yes | Username. | | `password` | String | `xxxxx` | Yes | Password. | ### Important Notes For **Vector Search Version** and **Recall Engine Version**, both username/password and AccessKey are required: - **AccessKey**: Used by plugins to retrieve table schema and pull table structure information in wizard mode. - **Username/Password**: Used for actual runtime data writing. --- ## Configuration Example ### Instance Mode ```json { "envType": "Prod", "regionId": "cn-beijing", "instanceType": "vectorSearchVersion", "instanceId": "ha-xxxx", "username": "admin", "password": "xxxx" } ``` --- ## Document Information - **Last Updated**: 2024-12-30 18:28:58 - **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/opensearch FILE:references/data-sources/oracle.md # Oracle DataSource ConnectionProperties Documentation ## Property Definition - **Data source type**: `oracle` - **Supported configuration mode (ConnectionPropertiesMode)**: `UrlMode` (Connection String Mode) --- ## Connection String Mode Parameters | Name | Type | Example Value | Required | Description and Notes | |------|------|---------------|----------|----------------------| | `jdbcUrl` | String | `jdbc:oracle:thin:@host:port:SID` | Yes | Oracle's jdbcUrl. | | `username` | String | `xxxxx` | Yes | Username. | | `password` | String | `xxxxx` | Yes | Password. | | `securityProtocol` | String | `authTypeNone` | No | Authentication option. Valid values: `authTypeNone`, `authTypeSsl`. | | `truststoreFile` | String | `123` | No | Truststore file ID. Required when `securityProtocol` is `authTypeSsl`. | --- ## DataSource Configuration Example ### Connection String Mode ```json { "jdbcUrl": "jdbc:oracle:thin:@host:port:SID", "username": "xxxxx", "password": "xxxxx", "securityProtocol": "authTypeNone", "envType": "Dev" } ``` --- **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/oracle FILE:references/data-sources/oss.md # OSS Data Source Documentation ## Property Definition - **Data source type**: `oss` - **Supported configuration mode (ConnectionPropertiesMode)**: `UrlMode` (Connection string mode) ## Connection String Mode Parameters | Name | Type | Example | Required | Description and Notes | |------|------|---------|----------|----------------------| | regionId | String | cn-shanghai | Yes | The region where OSS is located. | | endpoint | String | http://oss-beijing.aliyuncs.com | Yes | OSS access endpoint. | | bucket | String | test-oss-sh | Yes | The OSS bucket name. | | authType | String | RamRole | Yes | Access identity type. Only supports `RamRole` and `Ak`. | | authIdentity | String | 112345 | No | Role ID. Required when using `RamRole` access identity. | | accessId | String | xxxxx | No | AccessId used for accessing the data source in AK mode. Required in AK mode. | | accessKey | String | xxxxx | No | AccessKey used for accessing the data source in AK mode. Required in AK mode. | | envType | String | Dev | Yes | envType indicates the data source environment information.
- `Dev`: Development environment
- `Prod`: Production environment | ## Data Source Configuration Examples ### Connection String Mode (RamRole Recommended) > **Note**: OSS data source supports both `RamRole` and `Ak` authentication methods. **RamRole is recommended**. **Create via RAM Role Mode (Recommended):** ```json { "envType": "Prod", "endpoint": "http://oss-beijing.aliyuncs.com", "bucket": "test-oss-sh", "authType": "RamRole", "authIdentity": "1123455" } ``` **Create via AK Mode (Alternative):** ```json { "envType": "Prod", "endpoint": "http://oss-beijing.aliyuncs.com", "bucket": "test-oss-sh", "authType": "Ak", "accessId": "xxx", "accessKey": "xxx" } ``` --- *Source: https://help.aliyun.com/zh/dataworks/developer-reference/oss* FILE:references/data-sources/polardb-x-2-0.md # PolarDB-X 2.0 ConnectionProperties Documentation ## Attribute Definition - **Datasource type**: `polardbx20` - **Supported configuration modes (ConnectionPropertiesMode)**: - UrlMode (Connection String Mode) - InstanceMode (Instance Mode) --- ## Same-Account Instance Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | regionId | String | `cn-shanghai` | Yes | The region where the instance belongs. Note: Historical non-engine datasource data does not have this value. | | instanceId | String | `pc-xxxxx` | Yes | PolarDB-X cluster ID. | | database | String | `mysql_database` | Yes | Database name. | | username | String | `xxxxx` | Yes | Username. | | password | String | `xxxxx` | Yes | Password. | | properties | JSON Object | `{"useSSL": "false"}` | No | Driver properties. | | envType | String | `Dev` | Yes | Datasource environment type. Options: `Dev` (Development environment), `Prod` (Production environment). | --- ## Cross-Account Instance Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | regionId | String | `cn-shanghai` | Yes | The region where the instance belongs. Note: Historical non-engine datasource data does not have this value. | | instanceId | String | `pc-xxxxx` | Yes | PolarDB-X cluster ID. | | database | String | `polardbx_database` | Yes | Database name. | | username | String | `xxxxx` | Yes | Username. | | password | String | `xxxxx` | Yes | Password. | | crossAccountOwnerId | String | `1` | Yes | Cross-account target cloud account ID. | | crossAccountRoleName | String | `cross-role` | Yes | Cross-account target RAM role name. | | properties | JSON Object | `{"useSSL": "false"}` | No | Driver properties. | | envType | String | `Dev` | Yes | Datasource environment type. Options: `Dev` (Development environment), `Prod` (Production environment). | --- ## Connection String Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | address | Array | `[{"host": "127.0.0.1", "port": 3306}]` | Yes | Only a single address is allowed. | | database | String | `polardbx_database` | Yes | Database name. | | username | String | `xxxxx` | Yes | Username. | | password | String | `xxxxx` | Yes | Password. | | properties | JSON Object | `{"useSSL": "false"}` | No | Driver properties. | | envType | String | `Dev` | Yes | Datasource environment type. Options: `Dev` (Development environment), `Prod` (Production environment). | --- ## Datasource Configuration Examples ### Same-Account Instance Mode ```json { "envType": "Prod", "ownerId": "", "regionId": "cn-beijing", "instanceId": "pxc-bjrikym49rir1s", "database": "my_database", "username": "my_username", "password": "", "properties": { "socketTimeout": "2000" } } ``` ### Cross-Account Instance Mode ```json { "envType": "Prod", "regionId": "cn-beijing", "instanceId": "pxc-bjrikym49rir1s", "database": "my_database", "username": "my_username", "password": "", "properties": { "socketTimeout": "2000" }, "crossAccountOwnerId": "123123123", "crossAccountRoleName": "cross-role" } ``` ### Connection String Mode ```json { "envType": "Prod", "address": [ { "host": "127.0.0.1", "port": "3306" } ], "database": "my_database", "username": "my_username", "password": "", "properties": { "socketTimeout": "2000" } } ``` --- **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/polardb-x-2-0 FILE:references/data-sources/polardb.md # PolarDB MySQL & PostgreSQL ConnectionProperties ## Data Source Type: `polardb` PolarDB supports both MySQL and PostgreSQL engine types. --- ## Mode 1: Same-Account Instance Mode | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `regionId` | String | Yes | Region where the instance belongs | | `clusterId` | String | Yes | PolarDB cluster ID | | `database` | String | Yes | Database name | | `dbType` | String | Yes | PolarDB engine type: `mysql` or `postgresql` | | `username` | String | Yes | Username | | `password` | String | Yes | Password | | `securityProtocol` | String | No | SSL authentication: `authTypeNone` or `authTypeSsl` | | `truststoreFile` | String | No | Truststore certificate file (reference) | | `truststorePassword` | String | No | Truststore password | | `keystoreFile` | String | No | Keystore certificate file (reference) | | `keystorePassword` | String | No | Keystore password | | `readOnlyDBInstance` | String | No | Read replica connection address | | `envType` | String | Yes | Environment: `Dev` or `Prod` | ### Example (InstanceMode - MySQL): ```json { "envType": "Prod", "regionId": "cn-beijing", "clusterId": "pc-xxxxx", "dbType": "mysql", "database": "mydb", "username": "root", "password": "xxxxxx" } ``` --- ## Mode 2: Cross-Account Instance Mode | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `regionId` | String | Yes | Region where the instance belongs | | `clusterId` | String | Yes | PolarDB cluster ID | | `database` | String | Yes | Database name | | `dbType` | String | Yes | PolarDB engine type: `mysql` or `postgresql` | | `username` | String | Yes | Username | | `password` | String | Yes | Password | | `crossAccountOwnerId` | String | Yes | Cross-account target cloud account ID | | `crossAccountRoleName` | String | Yes | Cross-account target RAM role name | | `securityProtocol` | String | No | SSL authentication: `authTypeNone` or `authTypeSsl` | | `envType` | String | Yes | Environment: `Dev` or `Prod` | ### Example (Cross-Account): ```json { "envType": "Prod", "regionId": "cn-beijing", "clusterId": "pc-xxxxx", "dbType": "mysql", "database": "mydb", "username": "root", "password": "xxxxxx", "crossAccountOwnerId": "1234567890123456", "crossAccountRoleName": "cross-account-role" } ``` --- ## Mode 3: Connection String Mode | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `dbType` | String | Yes | PolarDB engine type: `mysql` or `postgresql` | | `address` | Array | Yes | Host addresses. Format: `[{"host": "127.0.0.1", "port": 3306}]` | | `database` | String | Yes | Database name | | `username` | String | Yes | Username | | `password` | String | Yes | Password | | `securityProtocol` | String | No | SSL authentication: `authTypeNone` or `authTypeSsl` | | `properties` | JSON Object | No | Driver properties. Example: `{"useSSL": "false"}` | | `envType` | String | Yes | Environment: `Dev` or `Prod` | ### Example (UrlMode - MySQL): ```json { "envType": "Prod", "dbType": "mysql", "address": [{"host": "pc-xxxxx.rwlb.rds.aliyuncs.com", "port": 3306}], "database": "mydb", "username": "root", "password": "xxxxxx" } ``` ### Example (UrlMode - PostgreSQL): ```json { "envType": "Prod", "dbType": "postgresql", "address": [{"host": "pc-xxxxx.rwlb.rds.aliyuncs.com", "port": 5432}], "database": "mydb", "username": "postgres", "password": "xxxxxx" } ``` FILE:references/data-sources/polardbo.md # PolarDB-O ConnectionProperties Documentation ## Overview | Attribute | Value | |-----------|-------| | **Datasource Type** | `polardbo` | | **Supported ConnectionPropertiesMode** | `UrlMode` (Connection String Mode), `InstanceMode` (Instance Mode) | --- ## Configuration Modes ### 1. Same-Account Instance Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `regionId` | String | `cn-shanghai` | Yes | The region where the instance is located. **Note:** Historical non-engine datasource records may not have this value. | | `instanceId` | String | `pc-xxxxx` | Yes | PolarDB O cluster ID. | | `database` | String | `mysql_database` | Yes | Database name. | | `username` | String | `xxxxx` | Yes | Username. | | `password` | String | `xxxxx` | Yes | Password. | | `envType` | String | `Dev` | Yes | Datasource environment information. Values: `Dev` (Development environment), `Prod` (Production environment). | **Example:** ```json { "envType": "Prod", "ownerId": "", "regionId": "cn-beijing", "instanceId": "pxc-bjrikym49rir1s", "database": "my_database", "username": "my_username", "password": "", "properties": { "socketTimeout": "2000" } } ``` --- ### 2. Cross-Account Instance Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `regionId` | String | `cn-shanghai` | Yes | The region where the instance is located. **Note:** Historical non-engine datasource records may not have this value. | | `instanceId` | String | `pc-xxxxx` | Yes | PolarDB O cluster ID. | | `database` | String | `polardbx_database` | Yes | Database name. | | `username` | String | `xxxxx` | Yes | Username. | | `password` | String | `xxxxx` | Yes | Password. | | `crossAccountOwnerId` | String | `1` | Yes | Cross-account target cloud account ID. | | `crossAccountRoleName` | String | `cross-role` | Yes | Cross-account target RAM role name. | | `envType` | String | `Dev` | Yes | Datasource environment information. Values: `Dev` (Development environment), `Prod` (Production environment). | **Example:** ```json { "envType": "Prod", "regionId": "cn-beijing", "instanceId": "pxc-bjrikym49rir1s", "database": "my_database", "username": "my_username", "password": "", "properties": { "socketTimeout": "2000" }, "crossAccountOwnerId": "123123123", "crossAccountRoleName": "cross-role" } ``` --- ### 3. Connection String Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `address` | Array | `[{"host": "127.0.0.1", "port": 3306}]` | Yes | Only a single address is allowed. | | `database` | String | `polardbx_database` | Yes | Database name. | | `username` | String | `xxxxx` | Yes | Username. | | `password` | String | `xxxxx` | Yes | Password. | | `properties` | JSON Object | `{"useSSL": "false"}` | No | Driver properties. | | `envType` | String | `Dev` | Yes | Datasource environment information. Values: `Dev` (Development environment), `Prod` (Production environment). | **Example:** ```json { "envType": "Prod", "address": [ { "host": "127.0.0.1", "port": "3306" } ], "database": "my_database", "username": "my_username", "password": "", "properties": { "socketTimeout": "2000" } } ``` --- **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/polardb-o FILE:references/data-sources/postgresql.md # PostgreSQL ConnectionProperties Documentation ## Attribute Definition - **Datasource type**: `postgresql` - **Supported configuration modes (ConnectionPropertiesMode)**: - `UrlMode` (Connection String Mode) - `InstanceMode` (Instance Mode) --- ## Same-Account Instance Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | regionId | String | cn-shanghai | Yes | The region where the instance belongs. Note: Historical non-engine datasources may not have this value. | | instanceId | String | pgm-xxxxxxxxx | Yes | Instance ID. | | database | String | postgresql_database | Yes | Database name. | | username | String | xxxxx | Yes | Username. | | password | String | xxxxx | Yes | Password. | | securityProtocol | String | AuthTypeNone | No | SSL authentication setting. Values: `AuthTypeNone` (no authentication), `AuthTypeSsl` (enable SSL authentication). | | truststoreFile | String | | No | Truststore certificate file (reference). | | keystoreFile | String | | No | Keystore certificate file (reference). | | keyFile | String | | No | Private key file (reference). | | clientPassword | String | abc | No | Private key password. | | readOnlyDBInstance | String | pgr-uf65l3bwae8w8r35 | No | Read-only replica instance ID. | | envType | String | Dev | Yes | Datasource environment information. Values: `Dev` (development environment), `Prod` (production environment). | --- ## Cross-Account Instance Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | regionId | String | cn-shanghai | Yes | The region where the instance belongs. Note: Historical non-engine datasources may not have this value. | | instanceId | String | pgm-xxxxxxxxx | Yes | Instance ID. | | database | String | postgresql_database | Yes | Database name. | | username | String | xxxxx | Yes | Username. | | password | String | xxxxx | Yes | Password. | | crossAccountOwnerId | String | 1 | Yes | Cross-account target cloud account ID. | | crossAccountRoleName | String | postgresql-role | Yes | Cross-account target RAM role name. | | securityProtocol | String | AuthTypeNone | No | SSL authentication setting. Values: `AuthTypeNone` (no authentication), `AuthTypeSsl` (enable SSL authentication). | | truststoreFile | String | | No | Truststore certificate file (reference). | | keystoreFile | String | | No | Keystore certificate file (reference). | | keyFile | String | | No | Private key file (reference). | | clientPassword | String | abc | No | Private key password. | | readOnlyDBInstance | String | pgr-uf65l3bwae8w8r35 | No | Read-only replica instance ID. | | envType | String | Dev | Yes | Datasource environment information. Values: `Dev` (development environment), `Prod` (production environment). | --- ## Connection String Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | address | Array | `[{"host": "127.0.0.1", "port": 5432}]` | Yes | Allows configuration of multiple host addresses and ports. | | database | String | postgresql_database | Yes | Database name. | | username | String | xxxxx | Yes | Username. | | password | String | xxxxx | Yes | Password. | | securityProtocol | String | AuthTypeNone | No | SSL authentication setting. Values: `AuthTypeNone` (no authentication), `AuthTypeSsl` (enable SSL authentication). | | truststoreFile | String | | No | Truststore certificate file (reference). | | keystoreFile | String | | No | Keystore certificate file (reference). | | keyFile | String | | No | Private key file (reference). | | clientPassword | String | abc | No | Private key password. | | properties | JSON Object | `{"useSSL": "false"}` | No | Driver properties. | | envType | String | Dev | Yes | Datasource environment information. Values: `Dev` (development environment), `Prod` (production environment). | --- ## Datasource Configuration Examples ### Same-Account Instance Mode ```json { "envType": "Prod", "instanceId": "pgm-xxxxxxxxx", "regionId": "cn-shanghai", "database": "db", "username": "aliyun", "password": "xxx", "securityProtocol": "AuthTypeNone" } ``` ### Cross-Account Instance Mode ```json { "envType": "Prod", "instanceId": "pgm-xxxxxxxxx", "regionId": "cn-shanghai", "database": "db", "username": "aliyun", "password": "xxx", "securityProtocol": "AuthTypeNone", "crossAccountOwnerId": "1234567890", "crossAccountRoleName": "my_ram_role" } ``` ### Connection String Mode ```json { "envType": "Prod", "address": [ { "host": "127.0.0.1", "port": 5432 } ], "database": "db", "properties": { "connectTimeout": "2000" }, "username": "aliyun", "password": "xxx" } ``` --- **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/postgresql FILE:references/data-sources/redis.md # Redis Datasource Documentation ## Property Definition - **Datasource Type**: `redis` - **Supported Configuration Mode (ConnectionPropertiesMode)**: `InstanceMode`, `UrlMode` --- ## InstanceMode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `instanceId` | String | `r-2zelhj0qqp6tvxyxql` | Yes | Redis instance ID. | | `password` | String | `xxx` | Yes | Password. | | `regionId` | String | `cn-beijing` | Yes | Region where the instance belongs. | | `securityProtocol` | String | `authTypeNone` | Yes | Authentication method. Values:
• `authTypeNone`: No SSL
• `authTypeSsl`: SSL authentication | | `truststoreFile` | String | `` | Conditional | Truststore certificate file ID. Required when `securityProtocol=authTypeSsl`. | | `truststorePassword` | String | `xxx` | No | Truststore password. | | `keystoreFile` | String | `` | No | Keystore certificate file ID. | | `keystorePassword` | String | `xxx` | No | Keystore password. | | `envType` | String | `Prod` | Yes | Environment type. Values: `Dev`, `Prod`. | --- ## UrlMode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `address` | JSON Array | `[{"host": "127.0.0.1", "port": "6379"}]` | Yes | Redis connection address. Only single host/port configuration allowed. | | `password` | String | `xxx` | Yes | Password. | | `securityProtocol` | String | `authTypeNone` | Yes | Authentication method. Values:
• `authTypeNone`: No SSL
• `authTypeSsl`: SSL authentication | | `truststoreFile` | String | `` | Conditional | Truststore certificate file ID. Required when `securityProtocol=authTypeSsl`. | | `truststorePassword` | String | `xxx` | No | Truststore password. | | `keystoreFile` | String | `` | No | Keystore certificate file ID. | | `keystorePassword` | String | `xxx` | No | Keystore password. | | `envType` | String | `Prod` | Yes | Environment type. Values: `Dev`, `Prod`. | --- ## Configuration Examples ### InstanceMode ```json { "envType": "Prod", "regionId": "cn-beijing", "instanceId": "r-xxxxx", "password": "", "securityProtocol": "authTypeNone" } ``` ### InstanceMode with SSL ```json { "envType": "Prod", "regionId": "cn-beijing", "instanceId": "r-xxxxx", "password": "", "securityProtocol": "authTypeSsl", "truststoreFile": "" } ``` ### UrlMode ```json { "envType": "Prod", "address": [{"host": "192.168.1.100", "port": "6379"}], "password": "", "securityProtocol": "authTypeNone" } ``` FILE:references/data-sources/redshift.md # Redshift Datasource Documentation ## Overview - **Datasource Type:** `redshift` - **Supported Configuration Mode:** `UrlMode` (Connection String Mode) --- ## ConnectionProperties Parameters (UrlMode) | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `address` | JSON Array | `[{"host": "127.0.0.1", "port": 3306}]` | Yes | Formally an array, but only **one set** of host and port is allowed. | | `database` | String | `mysql_database` | Yes | Database name. | | `username` | String | `xxxxx` | Yes | Username. | | `password` | String | `xxxxx` | Yes | Password. | | `properties` | JSON Object | `{"useSSL": "false"}` | No | Driver properties. | | `envType` | String | `Dev` | Yes | Indicates the datasource environment. Valid values: `Dev` (Development environment), `Prod` (Production environment). | --- ## Configuration Example (UrlMode) ```json { "address": [ { "host": "127.0.0.1", "port": 5432 } ], "database": "db", "properties": { "connectTimeout": "2000" }, "username": "xxxxx", "password": "xxxxx", "envType": "Dev" } ``` --- ## Field Details ### `address` - **Type:** JSON Array - **Required:** Yes - **Notes:** Although formatted as an array, it only accepts a single host/port configuration pair. ### `database` - **Type:** String - **Required:** Yes - **Description:** The name of the database to connect to. ### `username` - **Type:** String - **Required:** Yes - **Description:** Authentication username. ### `password` - **Type:** String - **Required:** Yes - **Description:** Authentication password. ### `properties` - **Type:** JSON Object - **Required:** No - **Description:** Additional JDBC driver properties (e.g., `useSSL`, `connectTimeout`). ### `envType` - **Type:** String - **Required:** Yes - **Valid Values:** - `Dev` - Development environment - `Prod` - Production environment FILE:references/data-sources/restapi.md # RESTAPI ConnectionProperties Documentation ## Property Definition - **Data Source Type**: `restapi` - **Supported Configuration Mode**: `UrlMode` (Connection String Mode) --- ## Connection String Mode Parameters | Name | Type | Example Value | Required | Description and Notes | |------|------|---------------|----------|----------------------| | `url` | String | `http://test-ots-sh-shanghai.ots.aliyuncs.com` | Yes | URL. | | `defaultHeader` | String | `{}` | No | Default request headers. | | `securityProtocol` | String | `authTypeNone` | No | Authentication method. Supported values:
- `authTypeNone`
- `basic`
- `token`
Default value: `authTypeNone`. | | `username` | String | `my-username` | No | Username. Required when `securityProtocol` is `basic`. | | `password` | String | `` | No | Password. Required when `securityProtocol` is `basic`. | | `authToken` | String | `my-token` | No | Token authentication. Required when `securityProtocol` is `token`. | | `envType` | String | `Dev` | Yes | `envType` indicates the data source environment information.
- `Dev`: Development environment.
- `Prod`: Production environment. | --- ## Data Source Configuration Example ### Connection String Mode ```json { "envType": "Prod", "url": "http://127.0.0.1/get", "securityProtocol": "basic", "username": "xxx", "password": "xxx" } ``` --- *Source: https://help.aliyun.com/zh/dataworks/developer-reference/restapi* FILE:references/data-sources/s3.md # S3 Data Source ConnectionProperties ## Property Definition - **Data source type**: `s3` - **Supported configuration mode**: `UrlMode` (Connection String Mode) --- ## Connection String Mode Parameters | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | regionId | String | cn-shanghai | Yes | Region where S3 is located. | | endpoint | String | http://s3-endpoint.com | Yes | S3 access endpoint. | | bucket | String | test-s3 | Yes | S3 bucket name. | | accessId | String | xxxxx | Yes | AccessId used for accessing the data source in AK mode. Required in AK mode. | | accessKey | String | xxxxx | Yes | AccessKey used for accessing the data source in AK mode. Required in AK mode. | | envType | String | Dev | Yes | envType indicates data source environment information.
- **Dev**: Development environment
- **Prod**: Production environment | --- ## Configuration Examples ### Connection String Mode ```json { "envType": "Prod", "endpoint": "http://s3-endpoint.com", "regionId": "cn-shanghai", "bucket": "s3-test", "accessId": "xxx", "accessKey": "xxx" } ``` --- *Source: https://help.aliyun.com/zh/dataworks/developer-reference/s3* FILE:references/data-sources/salesforce.md # Salesforce Datasource Documentation ## Property Definition - **Datasource Type**: `salesforce` - **Supported Configuration Mode (ConnectionPropertiesMode)**: `UrlMode` (Connection String Mode) --- ## UrlMode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `type` | String | `rest` | Yes | API type. Currently only supports `rest`. | | `instanceUrl` | String | `https://xxxxx.my.salesforce.com` | Yes | Salesforce instance URL. | | `refreshToken` | String | `xxx` | Yes | Refresh token for long-term authentication. | | `apiVersion` | String | `v58.0` | Yes | API version. Default: `v58.0`. Available versions: `v31.0`~`v58.0`. | | `envType` | String | `Prod` | Yes | Environment type. Values: `Dev`, `Prod`. | --- ## Configuration Example ```json { "envType": "Prod", "type": "rest", "instanceUrl": "https://xxxxx.my.salesforce.com", "refreshToken": "", "apiVersion": "v58.0" } ``` --- ## Notes > Salesforce data source uses OAuth authentication. You need to complete the authorization process through the console to obtain the refreshToken. The refreshToken is long-lived and used to automatically refresh access tokens. FILE:references/data-sources/saphana.md # SAP HANA ConnectionProperties Documentation ## Property Definition - **Datasource type (`type`)**: `saphana` - **Supported configuration mode (`ConnectionPropertiesMode`)**: `UrlMode` (Connection String Mode) --- ## Connection String Mode | Name | Type | Example Value | Required | Description and Notes | |------|------|---------------|----------|----------------------| | `address` | JSON Array | `[{"host": "127.0.0.1", "port": 3306}]` | Yes | Formally an array, but only **1 set of host and port** is allowed. | | `database` | String | `mysql_database` | No | Database name. | | `username` | String | `xxxxx` | Yes | Username. | | `password` | String | `xxxxx` | Yes | Password. | | `properties` | JSON Object | `{"useSSL": "false"}` | No | Driver properties. | | `envType` | String | `Dev` | Yes | Represents the datasource environment information.
- `Dev`: Development environment
- `Prod`: Production environment | --- ## Datasource Configuration Example ### Connection String Mode ```json { "address": [ { "host": "127.0.0.1", "port": 5432 } ], "database": "db", "properties": { "connectTimeout": "2000" }, "username": "xxxxx", "password": "xxxxx", "envType": "Dev" } ``` --- **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/sap-hana FILE:references/data-sources/selectdb.md # SelectDB Datasource Documentation ## Overview - **Data Source Type**: `selectdb` - **Supported Configuration Mode**: `UrlMode` (Connection String Mode) - **Last Updated**: 2024-11-06 --- ## Connection String Mode Parameters | Name | Type | Required | Description | |------|------|----------|-------------| | `address` | Array | Yes | Only a single address is allowed. | | `loadAddress` | String | Yes | FE Endpoint, multiple addresses can be configured. | | `database` | String | Yes | Database name. | | `username` | String | Yes | Username. | | `password` | String | Yes | Password. | | `properties` | JSON Object | No | Driver properties. | | `envType` | String | Yes | Indicates the datasource environment information. Values: `Dev` (Development environment) or `Prod` (Production environment). | --- ## Parameter Examples ### address ```json [ { "host": "127.0.0.1", "port": 3306 } ] ``` ### loadAddress ```json [ { "host": "127.0.0.1", "port": 3306 } ] ``` ### properties ```json { "useSSL": "false" } ``` --- ## Complete Configuration Example ```json { "envType": "Prod", "address": [ { "host": "127.0.0.1", "port": "3306" } ], "loadAddress": [ { "host": "127.0.0.2", "port": "8031" } ], "database": "my_database", "username": "my_username", "password": "", "properties": { "socketTimeout": "2000" } } ``` --- ## API Reference Summary | Field | Type | Required | Default | Notes | |-------|------|----------|---------|-------| | `envType` | String | Yes | - | `Dev` or `Prod` | | `address[].host` | String | Yes | - | Host IP address | | `address[].port` | String/Number | Yes | - | Port number | | `loadAddress[].host` | String | Yes | - | FE Endpoint host | | `loadAddress[].port` | String/Number | Yes | - | FE Endpoint port | | `database` | String | Yes | - | Target database name | | `username` | String | Yes | - | Authentication username | | `password` | String | Yes | - | Authentication password | | `properties` | Object | No | - | Additional JDBC driver properties | FILE:references/data-sources/snowflake.md # Snowflake Datasource Documentation ## Property Definition - **Datasource Type**: `snowflake` - **Supported Configuration Mode (ConnectionPropertiesMode)**: `UrlMode` (Connection String Mode) --- ## UrlMode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `accountUrl` | String | `xy12345.snowflakecomputing.com` | Yes | Complete account URL. Format: `account_locator.cloud_region_id` or `account_locator.cloud_region_id.cloud` or `account_locator.gov_compliance.cloud_region_id.cloud` | | `warehouseName` | String | `my_warehouse` | No | Compute resource, similar to Hologres compute group. | | `database` | String | `my_db` | Yes | Database name to access. | | `securityProtocol` | String | `authTypeClientPassword` | Yes | Authentication method. Values:
• `authTypeClientPassword`: Password authentication (default)
• `authTypePrivateKey`: Private key authentication | | `username` | String | `myuser` | Yes | Username. | | `password` | String | `xxx` | Conditional | Password. Required when `securityProtocol=authTypeClientPassword`. | | `privateKeyFileId` | Long | `123` | Conditional | PEM format private key file ID. Required when `securityProtocol=authTypePrivateKey`. | | `privateKeyPassword` | String | `xxx` | No | Password for PEM format private key. | | `role` | String | `my_role` | No | Role for data access. | | `envType` | String | `Prod` | Yes | Environment type. Values: `Dev`, `Prod`. | --- ## Configuration Examples ### Password Authentication Mode ```json { "envType": "Prod", "accountUrl": "xy12345.snowflakecomputing.com", "database": "my_db", "securityProtocol": "authTypeClientPassword", "username": "myuser", "password": "", "warehouseName": "my_warehouse" } ``` ### Private Key Authentication Mode ```json { "envType": "Prod", "accountUrl": "xy12345.snowflakecomputing.com", "database": "my_db", "securityProtocol": "authTypePrivateKey", "username": "myuser", "privateKeyFileId": "", "privateKeyPassword": "", "role": "my_role" } ``` FILE:references/data-sources/sqlserver.md # SQL Server ConnectionProperties Documentation ## Overview - **Datasource Type**: `sqlserver` - **Supported Configuration Modes (ConnectionPropertiesMode)**: - UrlMode (Connection String Mode) - InstanceMode (Instance Mode) --- ## Same-Account Instance Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | regionId | String | cn-shanghai | Yes | SQL Server instance region | | instanceId | String | rm-xxxxx | Yes | SQL Server instance ID | | database | String | db1 | Yes | Database name | | username | String | user1 | Yes | Username | | password | String | pass1 | Yes | Password | | securityProtocol | String | authTypeNone | No | SSL authentication option. Values: `authTypeNone` (no authentication), `authTypeSsl` (enable SSL authentication). Default: `authTypeNone` | | truststoreFile | String | 1 | No | Truststore certificate file (reference). Required when `securityProtocol=authTypeSsl` | | truststorePassword | String | apasara | No | Truststore password. Required when `securityProtocol=authTypeSsl` | | readOnlyDBInstance | String | rm-xxxxx | No | Read-only replica instance ID | | envType | String | Dev | Yes | Datasource environment information. `Dev` for development environment, `Prod` for production environment | --- ## Cross-Account Instance Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | regionId | String | cn-shanghai | Yes | SQL Server instance region | | instanceId | String | rm-xxxxx | Yes | SQL Server instance ID | | database | String | db1 | Yes | Database name | | username | String | user1 | Yes | Username | | password | String | pass1 | Yes | Password | | crossAccountOwnerId | String | 11111 | No | Cross-account owner's primary account ID. Required for cross-account scenarios | | crossAccountRoleName | String | role-name | No | Role name in the target account for cross-account scenarios | | securityProtocol | String | authTypeNone | No | SSL authentication option. Values: `authTypeNone` (no authentication), `authTypeSsl` (enable SSL authentication). Default: `authTypeNone` | | truststoreFile | String | 1 | No | Truststore certificate file (reference). Required when `securityProtocol=authTypeSsl` | | truststorePassword | String | apasara | No | Truststore password. Required when `securityProtocol=authTypeSsl` | | readOnlyDBInstance | String | rm-xxxxx | No | Read-only replica instance ID | | envType | String | Dev | Yes | Datasource environment information. `Dev` for development environment, `Prod` for production environment | --- ## Connection String Mode | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | address | JSONArray | `[{"host": "127.0.0.1", "port": "1234"}]` | Yes | Only single host address and single port configuration is allowed | | database | String | db1 | Yes | Database name | | username | String | user1 | Yes | Username | | password | String | pass1 | Yes | Password | | properties | JSON Object | `{"queryTimeout":"1000"}` | No | Driver properties | | securityProtocol | String | authTypeNone | No | SSL authentication option. Values: `authTypeNone` (no authentication), `authTypeSsl` (enable SSL authentication). Default: `authTypeNone` | | truststoreFile | String | 1 | No | Truststore certificate file (reference). Required when `securityProtocol=authTypeSsl` | | truststorePassword | String | apasara | No | Truststore password. Required when `securityProtocol=authTypeSsl` | | envType | String | Dev | Yes | Datasource environment information. `Dev` for development environment, `Prod` for production environment | --- ## Configuration Examples ### Same-Account Instance Mode ```json { "envType": "Prod", "instanceId": "rm-xxxxx", "regionId": "cn-shanghai", "database": "db", "username": "aliyun", "password": "xxx", "securityProtocol": "authTypeNone" } ``` ### Cross-Account Instance Mode ```json { "envType": "Prod", "instanceId": "rm-xxxxx", "regionId": "cn-shanghai", "database": "db", "username": "aliyun", "password": "xxx", "securityProtocol": "authTypeNone", "crossAccountOwnerId": "1234567890", "crossAccountRoleName": "my_ram_role" } ``` ### Connection String Mode ```json { "envType": "Prod", "address": [ { "host": "127.0.0.1", "port": 5432 } ], "database": "db", "properties": { "queryTimeout": "2000" }, "username": "aliyun", "password": "xxx" } ``` --- **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/sql-server FILE:references/data-sources/ssh.md # SSH Datasource Documentation ## Property Definition - **Datasource Type**: `ssh` - **Supported Configuration Mode (ConnectionPropertiesMode)**: `UrlMode` (Connection String Mode) --- ## UrlMode Parameters | Name | Type | Example Value | Required | Description | |------|------|---------------|----------|-------------| | `host` | String | `192.168.1.100` | Yes | Host address. | | `port` | String | `22` | Yes | Host port. | | `username` | String | `root` | Yes | Username. | | `securityProtocol` | String | `passwordAuth` | Yes | Authentication mode. Values:
• `passwordAuth`: Password authentication
• `authTypeSshKey`: SSH key authentication
• `authTypeSshPublicKey`: Public key authentication | | `password` | String | `xxx` | Conditional | Password. Required when `securityProtocol=passwordAuth`. | | `sshKeyFile` | String | `` | Conditional | Private key file ID. Required when `securityProtocol=authTypeSshKey` or `authTypeSshPublicKey`. | | `sshKeyPassword` | String | `xxx` | No | Private key password. Only available when `securityProtocol=authTypeSshKey`. | | `publicKey` | String | `ssh-rsa xxx` | Conditional | Public key text. Required and read-only when `securityProtocol=authTypeSshPublicKey`. | | `envType` | String | `Prod` | Yes | Environment type. Values: `Dev`, `Prod`. | --- ## Configuration Examples ### Password Authentication Mode ```json { "envType": "Prod", "host": "192.168.1.100", "port": "22", "username": "root", "securityProtocol": "passwordAuth", "password": "" } ``` ### SSH Key Authentication Mode ```json { "envType": "Prod", "host": "192.168.1.100", "port": "22", "username": "root", "securityProtocol": "authTypeSshKey", "sshKeyFile": "", "sshKeyPassword": "" } ``` ### SSH Public Key Authentication Mode ```json { "envType": "Prod", "host": "192.168.1.100", "port": "22", "username": "root", "securityProtocol": "authTypeSshPublicKey", "sshKeyFile": "", "publicKey": "ssh-rsa xxxxx" } ``` FILE:references/data-sources/starrocks.md # StarRocks Datasource - ConnectionProperties Documentation ## Overview - **Datasource Type:** `starrocks` - **Last Updated:** 2024-10-15 09:30:16 ## Supported Configuration Modes | Mode | Description | |------|-------------| | UrlMode | Connection string mode | | InstanceMode | Instance mode | --- ## Query StarRocks Instances Before creating a StarRocks data source, you can query the list of available instances. ### OLAP Type (Semi-Managed Mode) > **Note**: OLAP type uses EMR API to query cluster list. ```bash aliyun emr ListClusters --user-agent AlibabaCloud-Agent-Skills --RegionId "" 2>/dev/null | jq -r ' .Clusters[] | select(.ClusterType == "OLAP") | "Cluster ID: \(.ClusterId) | Name: \(.ClusterName) | Status: \(.ClusterState)" ' ``` ### Serverless Type > **Note**: The current aliyun CLI does not support querying the StarRocks Serverless instance list. Please guide users to obtain the instance ID manually through the [EMR Serverless StarRocks Console](https://emr.console.aliyun.com/). **Steps to Get Instance ID**: 1. Log in to the [EMR Serverless StarRocks Console](https://emr.console.aliyun.com/) 2. Select the corresponding region 3. View the **Instance ID** in the instance list (format: `c-xxxxx`) 4. Use the instance ID for the `instanceId` parameter when creating the data source --- ## 1. Same-Account Instance Mode | Field | Type | Example | Required | Description | |-------|------|---------|----------|-------------| | `regionId` | String | `cn-shanghai` | Yes | The region where the instance resides | | `instanceType` | String | `serverless` | Yes | Instance type. Options: `emr-olap` (semi-managed mode), `serverless` (serverless mode) | | `instanceId` | String | `c-12345` | Yes | Instance mode instance ID | | `database` | String | `dbName` | Yes | Database name | | `username` | String | `srUser` | Yes | Username | | `password` | String | `srPassword` | Yes | Password | | `envType` | String | `Dev` | Yes | Datasource environment info. Options: `Dev` (development), `Prod` (production) | ### Example ```json { "envType": "Prod", "instanceType": "serverless", "regionId": "cn-shanghai", "instanceId": "c-107e2047ef787c2e", "database": "my_database", "username": "xxxxx", "password": "xxxxx" } ``` --- ## 2. Cross-Account Instance Mode | Field | Type | Example | Required | Description | |-------|------|---------|----------|-------------| | `regionId` | String | `cn-shanghai` | Yes | The region where the instance resides | | `instanceId` | String | `c-12345` | Yes | Instance mode instance ID | | `instanceType` | String | `serverless` | Yes | Instance type. Options: `emr-olap` (semi-managed mode), `serverless` (serverless mode) | | `database` | String | `dbName` | Yes | Database name | | `crossAccountOwnerId` | String | `` | Yes | The target Alibaba Cloud main account ID for cross-account scenarios | | `crossAccountRoleName` | String | `mc-accross-role-name` | Yes | Role name under the target account for cross-account scenarios | | `username` | String | `srUser` | Yes | Username | | `password` | String | `srPassword` | Yes | Password | | `envType` | String | `Dev` | Yes | Datasource environment info. Options: `Dev` (development), `Prod` (production) | ### Example ```json { "envType": "Prod", "instanceType": "serverless", "regionId": "cn-shanghai", "instanceId": "c-107e2047ef787c2e", "database": "my_database", "username": "xxxxx", "password": "xxxxx", "crossAccountRoleName": "cross-role", "crossAccountOwnerId": "123123123" } ``` --- ## 3. Connection String Mode | Field | Type | Example | Required | Description | |-------|------|---------|----------|-------------| | `database` | String | `dbName` | Yes | Database name | | `username` | String | `srUser` | Yes | Username | | `password` | String | `srPassword` | Yes | Password | | `properties` | JSON | `{"connectTimeout": "2000"}` | No | Connection string mode connection properties (key-value pairs) | | `address` | JSON Array | `[{"host": "127.0.0.1", "port": 3306}]` | Yes | Array format, but **only allows 1 set of host and port** | | `loadAddress` | JSON Array | `[{"host": "127.0.0.1", "port": 3306}]` | Yes | Array format, **allows 1 or more sets of host and port** | | `envType` | String | `Dev` | Yes | Datasource environment info. Options: `Dev` (development), `Prod` (production) | ### Example ```json { "envType": "Prod", "address": [ { "host": "127.0.0.1", "port": 3306 } ], "loadAddress": [ { "host": "127.0.0.1", "port": 3306 } ], "database": "asdf", "username": "xxxxx", "password": "xxxxx", "properties": { "socketTimeout": 2000 } } ``` --- ## Quick Reference Summary | Parameter | Instance Mode | Cross-Account Mode | Connection String Mode | |-----------|---------------|-------------------|----------------------| | `regionId` | Required | Required | - | | `instanceId` | Required | Required | - | | `instanceType` | Required | Required | - | | `database` | Required | Required | Required | | `username` | Required | Required | Required | | `password` | Required | Required | Required | | `envType` | Required | Required | Required | | `address` | - | - | Required | | `loadAddress` | - | - | Required | | `properties` | - | - | Optional | | `crossAccountOwnerId` | - | Required | - | | `crossAccountRoleName` | - | Required | - | FILE:references/data-sources/tablestore.md # TableStore ConnectionProperties Documentation ## Property Definition - **Data Source Type (type):** `tablestore` - **Supported Configuration Mode (ConnectionPropertiesMode):** `UrlMode` (Connection String Mode) --- ## Connection String Mode Parameters | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `regionId` | String | `cn-shanghai` | Yes | The region where TableStore is located. | | `endpoint` | String | `http://test-ots-sh-shanghai.ots.aliyuncs.com` | Yes | TableStore access endpoint. | | `instanceName` | String | `test-ots-sh` | Yes | TableStore instance name. | | `accessId` | String | `xxxxx` | Yes | The accessId used to access the data source under AK mode. Required when using AK mode. | | `accessKey` | String | `xxxxx` | Yes | The accessKey used to access the data source under AK mode. Required when using AK mode. | | `envType` | String | `Dev` | Yes | Indicates the data source environment information.

`Dev`: Development environment
`Prod`: Production environment

| --- ## Configuration Example ### Connection String Mode ```json { "envType": "Prod", "endpoint": "http://test-ots-sh-shanghai.ots.aliyuncs.com", "instanceName": "test-ots-sh", "accessId": "xxx", "accessKey": "xxx" } ``` --- *Source: https://help.aliyun.com/zh/dataworks/developer-reference/tablestore* FILE:references/data-sources/tidb.md # TiDB Datasource ConnectionProperties Documentation ## Property Definition - **Datasource Type**: `tidb` - **Supported Configuration Mode (ConnectionPropertiesMode)**: - `UrlMode` (Connection String Mode) --- ## Connection String Mode Parameters | Name | Type | Example Value | Required | Description and Notes | |------|------|---------------|----------|----------------------| | `address` | JSONArray | `[{"host": "127.0.0.1", "port": "1234"}]` | Yes | Only allowed to configure as a single host address and single port format | | `database` | String | `db1` | Yes | Database name | | `username` | String | `user1` | Yes | Username | | `password` | String | `pass1` | Yes | Password | | `properties` | JSON Object | `{"queryTimeout": "1000"}` | No | Driver properties | | `securityProtocol` | String | `authTypeNone` | No | Whether to use SSL authentication. Values:
- `authTypeNone` (No authentication)
- `authTypeSsl` (Enable SSL authentication)
Default: `authTypeNone` | | `truststoreFile` | String | `1` | No | Truststore certificate file (reference). Required when `securityProtocol=authTypeSsl` | | `truststorePassword` | String | `apasara` | No | Truststore password. Required when `securityProtocol=authTypeSsl` | | `keystoreFile` | String | `2` | No | Keystore certificate file (reference) | | `keystorePassword` | String | `apasara` | No | Keystore password | | `envType` | String | `Dev` | Yes | `envType` indicates the datasource environment information:
- Development environment: `Dev`
- Production environment: `Prod` | --- ## Datasource Configuration Example ### Connection String Mode Configuration via host address and port number: ```json { "envType": "Prod", "securityProtocol": "authTypeNone", "address": [ { "host": "127.0.0.1", "port": 5432 } ], "database": "db", "properties": { "queryTimeout": "2000" }, "username": "aliyun", "password": "xxx" } ``` --- **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/tidb FILE:references/data-sources/vertica.md # Vertica DataSource ConnectionProperties Documentation ## Attribute Definition - **Datasource type**: `vertica` - **Supported configuration mode**: `UrlMode` (Connection String Mode) --- ## Connection String Mode Parameters | Name | Type | Example | Required | Description | |------|------|---------|----------|-------------| | `address` | JSON Array | `[{"host": "127.0.0.1", "port": 3306}]` | Yes | Formally an array, but only allows configuration of 1 set of host and port. | | `database` | String | `mysql_database` | Yes | Database name. | | `username` | String | `xxxxx` | Yes | Username. | | `password` | String | `xxxxx` | Yes | Password. | | `properties` | JSON Object | `{"useSSL": "false"}` | No | Driver properties. | | `envType` | String | `Dev` | Yes | envType indicates datasource environment information.
- `Dev`: Development environment
- `Prod`: Production environment | --- ## Datasource Configuration Example ### Connection String Mode ```json { "address": [ { "host": "127.0.0.1", "port": 5432 } ], "database": "db", "properties": { "connectTimeout": "2000" }, "username": "xxxxx", "password": "xxxxx", "envType": "Dev" } ``` --- **Source**: https://help.aliyun.com/zh/dataworks/developer-reference/vertica FILE:references/ram-policies.md # DataWorks Infrastructure RAM Policies ## Permission Matrix ### Data Source Permissions | API Action | RAM Permission | Access Level | |------------|----------------|--------------| | CreateDataSource | dataworks:CreateDataSource | Write | | GetDataSource | dataworks:GetDataSource | Read | | ListDataSources | dataworks:ListDataSources | List | | TestDataSourceConnectivity | dataworks:TestDataSourceConnectivity | Read | ### Compute Resource Permissions | API Action | RAM Permission | Access Level | |------------|----------------|--------------| | CreateComputeResource | dataworks:CreateComputeResource | Write | | GetComputeResource | dataworks:GetComputeResource | Read | | ListComputeResources | dataworks:ListComputeResources | List | ### Resource Group Permissions | API Action | RAM Permission | Access Level | |------------|----------------|--------------| | CreateResourceGroup | dataworks:CreateResourceGroup | Write | | GetResourceGroup | dataworks:GetResourceGroup | Read | | ListResourceGroups | dataworks:ListResourceGroups | List | | AssociateProjectToResourceGroup | dataworks:AssociateProjectToResourceGroup | Write | | DissociateProjectFromResourceGroup | dataworks:DissociateProjectFromResourceGroup | Write | | ListResourceGroupAssociateProjects | dataworks:ListResourceGroupAssociateProjects | List | ### Workspace Member Permissions | API Action | RAM Permission | Access Level | |------------|----------------|--------------| | ListProjectRoles | dataworks:ListProjectRoles | List | | ListProjectMembers | dataworks:ListProjectMembers | List | | GetProjectMember | dataworks:GetProjectMember | Read | | CreateProjectMember | dataworks:CreateProjectMember | Write | | DeleteProjectMember | dataworks:DeleteProjectMember | Write | | GrantMemberProjectRoles | dataworks:GrantMemberProjectRoles | Write | | RevokeMemberProjectRoles | dataworks:RevokeMemberProjectRoles | Write | ### VPC Permissions (required for resource group operations) | API Action | RAM Permission | Access Level | |------------|----------------|--------------| | DescribeVpcs | vpc:DescribeVpcs | Read | | DescribeVSwitches | vpc:DescribeVSwitches | Read | ## Role Requirements | Operation | Required DataWorks Roles | |------|---------------------| | Create resource | Tenant Owner, Workspace Admin, Project Owner, Operator | | View resource | Tenant Owner, Workspace Admin, Deployer, Developer, Project Owner, Operator | | List resources | All roles | ## RAM Policy Examples ### Full Access ```json { "Version": "1", "Statement": [ { "Effect": "Allow", "Action": [ "dataworks:CreateDataSource", "dataworks:GetDataSource", "dataworks:ListDataSources", "dataworks:TestDataSourceConnectivity", "dataworks:CreateComputeResource", "dataworks:GetComputeResource", "dataworks:ListComputeResources", "dataworks:CreateResourceGroup", "dataworks:GetResourceGroup", "dataworks:ListResourceGroups", "dataworks:AssociateProjectToResourceGroup", "dataworks:DissociateProjectFromResourceGroup", "dataworks:ListResourceGroupAssociateProjects", "dataworks:ListProjects", "dataworks:ListProjectRoles", "dataworks:ListProjectMembers", "dataworks:GetProjectMember", "dataworks:CreateProjectMember", "dataworks:DeleteProjectMember", "dataworks:GrantMemberProjectRoles", "dataworks:RevokeMemberProjectRoles" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "vpc:DescribeVpcs", "vpc:DescribeVSwitches" ], "Resource": "*" } ] } ``` ### Read-Only ```json { "Version": "1", "Statement": [ { "Effect": "Allow", "Action": [ "dataworks:GetDataSource", "dataworks:ListDataSources", "dataworks:GetComputeResource", "dataworks:ListComputeResources", "dataworks:GetResourceGroup", "dataworks:ListResourceGroups", "dataworks:ListResourceGroupAssociateProjects", "dataworks:ListProjectMembers", "dataworks:GetProjectMember", "dataworks:ListProjectRoles" ], "Resource": "*" } ] } ``` ### Resource Group Management > Creating resource groups requires additionally attaching the `AliyunBSSOrderAccess` system policy. ```json { "Version": "1", "Statement": [ { "Effect": "Allow", "Action": [ "dataworks:CreateResourceGroup", "dataworks:GetResourceGroup", "dataworks:ListResourceGroups", "dataworks:AssociateProjectToResourceGroup", "dataworks:DissociateProjectFromResourceGroup", "dataworks:ListResourceGroupAssociateProjects", "dataworks:ListProjects" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "vpc:DescribeVpcs", "vpc:DescribeVSwitches" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "bss:ViewOrder", "bss:PayOrder" ], "Resource": "*" } ] } ``` ## System Policies Reference | Policy Name | Description | |----------|------| | AliyunDataWorksFullAccess | DataWorks full access | | AliyunVPCReadOnlyAccess | VPC read-only access | | AliyunBSSOrderAccess | BSS order access (required for creating resource groups) | ## References - [DataWorks RAM Permission Documentation](https://help.aliyun.com/zh/dataworks/user-guide/manage-members-and-roles) - [RAM Console](https://ram.console.aliyun.com/) FILE:references/related-apis.md # DataWorks Infrastructure Related APIs ## Data Source APIs ### CreateDataSource - Create Data Source **Request Parameters:** | Name | Type | Required | Description | Example | |------|------|----------|------|--------| | ProjectId | long | Yes | DataWorks workspace ID | 17820 | | Name | string | Yes | Data source name | my_mysql | | Type | string | Yes | Data source type | mysql | | ConnectionPropertiesMode | string | Yes | Connection mode: UrlMode / InstanceMode | UrlMode | | ConnectionProperties | string | Yes | Connection configuration JSON | See examples | | Description | string | No | Description | MySQL production | **Response Parameters:** | Name | Type | Description | |------|------|------| | RequestId | string | Request ID | | Id | long | Data source ID | ### GetDataSource - Get Data Source Details **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | Id | long | Yes | Data source ID | **Response Parameters:** | Name | Type | Description | |------|------|------| | RequestId | string | Request ID | | DataSource | object | Data source details | ### ListDataSources - List Data Sources **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | ProjectId | long | Yes | Workspace ID | | Types | array | No | Data source type filter | | EnvType | string | No | Environment type: Dev / Prod | | PageNumber | integer | No | Page number, default 1 | | PageSize | integer | No | Page size, default 10, max 100 | ### TestDataSourceConnectivity - Test Connectivity **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | Id | long | Yes | Data source ID | | ProjectId | long | Yes | Workspace ID | | ResourceGroupId | string | Yes | Resource group ID | > **Note**: UpdateDataSource and DeleteDataSource APIs are intentionally excluded from this skill for security reasons. To modify or delete data sources, please use the DataWorks console. --- ## Compute Resource APIs ### CreateComputeResource - Create Compute Resource **Request Parameters:** | Name | Type | Required | Description | Example | |------|------|----------|------|--------| | ProjectId | long | Yes | DataWorks workspace ID | 10001 | | Name | string | Yes | Compute resource name (letters, digits, underscores; cannot start with digit or underscore; max 255 chars) | my_holo_resource | | Type | string | Yes | Compute resource type | hologres | | ConnectionPropertiesMode | string | Yes | Connection mode: InstanceMode / UrlMode | InstanceMode | | ConnectionProperties | string | Yes | Connection configuration JSON | See examples | | Description | string | No | Description (max 3000 chars) | Hologres resource | **Response Parameters:** | Name | Type | Description | |------|------|------| | RequestId | string | Request ID | | Id | long | Compute resource ID | ### GetComputeResource - Get Compute Resource Details **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | Id | long | Yes | Compute resource ID | | ProjectId | long | Yes | Workspace ID | **Response Parameters:** | Name | Type | Description | |------|------|------| | RequestId | string | Request ID | | ComputeResource | object | Compute resource details (Id, Name, Type, ConnectionProperties, CreateTime, ModifyTime, etc.) | ### ListComputeResources - List Compute Resources **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | ProjectId | long | Yes | Workspace ID | | Name | string | No | Name filter | | Types | array | No | Type filter | | EnvType | string | No | Environment type: Dev / Prod | | PageNumber | integer | No | Page number, default 1 | | PageSize | integer | No | Page size, default 10, max 100 | | SortBy | string | No | Sort field | | Order | string | No | Sort order: Desc / Asc | > **Note**: UpdateComputeResource and DeleteComputeResource APIs are intentionally excluded from this skill for security reasons. To modify or delete compute resources, please use the DataWorks console. --- ## Resource Group APIs ### CreateResourceGroup - Create Resource Group **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | Name | string | Yes | Resource group name | | PaymentType | string | Yes | Payment type: PostPaid | | VpcId | string | Yes | VPC ID | | VswitchId | string | Yes | VSwitch ID | | ClientToken | string | No | Idempotent token | | Remark | string | No | Remark | | Spec | integer | No | Specification | ### GetResourceGroup - Get Resource Group **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | Id | string | Yes | Resource group ID | ### ListResourceGroups - List Resource Groups **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | PageSize | integer | No | Page size | | Statuses | array | No | Status filter | ### AssociateProjectToResourceGroup - Bind Workspace **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | ResourceGroupId | string | Yes | Resource group ID | | ProjectId | long | Yes | Workspace ID | ### DissociateProjectFromResourceGroup - Unbind Workspace **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | ResourceGroupId | string | Yes | Resource group ID | | ProjectId | long | Yes | Workspace ID | ### ListResourceGroupAssociateProjects - Query Binding Relationships **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | ResourceGroupId | string | Yes | Resource group ID | --- ## Workspace Member APIs ### ListProjectRoles - Query Role List **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | ProjectId | long | Yes | Workspace ID | | Type | string | No | Role type: System / Custom | | PageSize | integer | No | Page size | ### ListProjectMembers - Query Member List **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | ProjectId | long | Yes | Workspace ID | | RoleCodes | array | No | Role filter | | PageSize | integer | No | Page size | ### GetProjectMember - Get Member Details **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | ProjectId | long | Yes | Workspace ID | | UserId | string | Yes | User ID | ### CreateProjectMember - Add Member **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | ProjectId | long | Yes | Workspace ID | | UserId | string | Yes | User ID | | RoleCodes | array | Yes | Role list | ### DeleteProjectMember - Remove Member **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | ProjectId | long | Yes | Workspace ID | | UserId | string | Yes | User ID | ### GrantMemberProjectRoles - Grant Roles **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | ProjectId | long | Yes | Workspace ID | | UserId | string | Yes | User ID | | RoleCodes | array | Yes | Roles to grant | ### RevokeMemberProjectRoles - Revoke Roles **Request Parameters:** | Name | Type | Required | Description | |------|------|----------|------| | ProjectId | long | Yes | Workspace ID | | UserId | string | Yes | User ID | | RoleCodes | array | Yes | Roles to revoke | --- ## Region and Endpoints DataWorks OpenAPI uses region-specific endpoints. When specifying `--region`, you **must** also add the matching `--endpoint`. ### Usage | Scenario | Parameters | |----------|-----------| | Public network | `--region --endpoint dataworks..aliyuncs.com` | | VPC internal network | `--region --endpoint dataworks-vpc..aliyuncs.com` | ### Supported Regions | Region Name | Region ID | Public Endpoint | VPC Endpoint | |---|---|---|---| | China (Hangzhou) | `cn-hangzhou` | `dataworks.cn-hangzhou.aliyuncs.com` | `dataworks-vpc.cn-hangzhou.aliyuncs.com` | | China (Shanghai) | `cn-shanghai` | `dataworks.cn-shanghai.aliyuncs.com` | `dataworks-vpc.cn-shanghai.aliyuncs.com` | | China (Beijing) | `cn-beijing` | `dataworks.cn-beijing.aliyuncs.com` | `dataworks-vpc.cn-beijing.aliyuncs.com` | | China (Zhangjiakou) | `cn-zhangjiakou` | `dataworks.cn-zhangjiakou.aliyuncs.com` | `dataworks-vpc.cn-zhangjiakou.aliyuncs.com` | | China (Ulanqab) | `cn-wulanchabu` | `dataworks.cn-wulanchabu.aliyuncs.com` | `dataworks-vpc.cn-wulanchabu.aliyuncs.com` | | China (Shenzhen) | `cn-shenzhen` | `dataworks.cn-shenzhen.aliyuncs.com` | `dataworks-vpc.cn-shenzhen.aliyuncs.com` | | China (Chengdu) | `cn-chengdu` | `dataworks.cn-chengdu.aliyuncs.com` | `dataworks-vpc.cn-chengdu.aliyuncs.com` | | China (Hong Kong) | `cn-hongkong` | `dataworks.cn-hongkong.aliyuncs.com` | `dataworks-vpc.cn-hongkong.aliyuncs.com` | | Singapore | `ap-southeast-1` | `dataworks.ap-southeast-1.aliyuncs.com` | `dataworks-vpc.ap-southeast-1.aliyuncs.com` | | Malaysia (Kuala Lumpur) | `ap-southeast-3` | `dataworks.ap-southeast-3.aliyuncs.com` | `dataworks-vpc.ap-southeast-3.aliyuncs.com` | | Indonesia (Jakarta) | `ap-southeast-5` | `dataworks.ap-southeast-5.aliyuncs.com` | `dataworks-vpc.ap-southeast-5.aliyuncs.com` | | Japan (Tokyo) | `ap-northeast-1` | `dataworks.ap-northeast-1.aliyuncs.com` | `dataworks-vpc.ap-northeast-1.aliyuncs.com` | | South Korea (Seoul) | `ap-northeast-2` | `dataworks.ap-northeast-2.aliyuncs.com` | `dataworks-vpc.ap-northeast-2.aliyuncs.com` | | US (Virginia) | `us-east-1` | `dataworks.us-east-1.aliyuncs.com` | `dataworks-vpc.us-east-1.aliyuncs.com` | | US (Silicon Valley) | `us-west-1` | `dataworks.us-west-1.aliyuncs.com` | `dataworks-vpc.us-west-1.aliyuncs.com` | | UK (London) | `eu-west-1` | `dataworks.eu-west-1.aliyuncs.com` | `dataworks-vpc.eu-west-1.aliyuncs.com` | | Germany (Frankfurt) | `eu-central-1` | `dataworks.eu-central-1.aliyuncs.com` | `dataworks-vpc.eu-central-1.aliyuncs.com` | | UAE (Dubai) | `me-east-1` | `dataworks.me-east-1.aliyuncs.com` | `dataworks-vpc.me-east-1.aliyuncs.com` | | Mexico (Querétaro) | `na-south-1` | `dataworks.na-south-1.aliyuncs.com` | `dataworks-vpc.na-south-1.aliyuncs.com` | | Saudi Arabia (Riyadh) | `me-central-1` | `dataworks.me-central-1.aliyuncs.com` | `dataworks-vpc.me-central-1.aliyuncs.com` | | China South 1 Finance | `cn-shenzhen-finance-1` | `dataworks.cn-shenzhen-finance-1.aliyuncs.com` | `dataworks-vpc.cn-shenzhen-finance-1.aliyuncs.com` | | China East 2 Finance | `cn-shanghai-finance-1` | `dataworks.cn-shanghai-finance-1.aliyuncs.com` | `dataworks-vpc.cn-shanghai-finance-1.aliyuncs.com` | | China East 1 Finance | `cn-hangzhou-finance` | `dataworks.aliyuncs.com` | *not available* | ### Notes - **Endpoint naming rule**: If a region is not explicitly listed above, the endpoint follows the standard naming pattern: - Public: `dataworks..aliyuncs.com` - VPC: `dataworks-vpc..aliyuncs.com` - **Endpoint selection**: Always use the **public endpoint** by default. Only use the VPC endpoint when the user explicitly specifies it (e.g., the API call is being made from within an Alibaba Cloud VPC). - **Finance Cloud (`cn-hangzhou-finance`)**: Uses the global endpoint `dataworks.aliyuncs.com` and does **not** support VPC endpoints. - **Official documentation**: [DataWorks Service Access Points](https://help.aliyun.com/zh/dataworks/developer-reference/api-dataworks-public-2024-05-18-endpoint) --- ## Official Documentation Links ### Data Sources - [CreateDataSource](https://help.aliyun.com/zh/dataworks/developer-reference/api-dataworks-public-2024-05-18-createdatasource) - [GetDataSource](https://help.aliyun.com/zh/dataworks/developer-reference/api-dataworks-public-2024-05-18-getdatasource) - [ListDataSources](https://help.aliyun.com/zh/dataworks/developer-reference/api-dataworks-public-2024-05-18-listdatasources) - [TestDataSourceConnectivity](https://help.aliyun.com/zh/dataworks/developer-reference/api-dataworks-public-2024-05-18-testdatasourceconnectivity) ### Compute Resources - [CreateComputeResource](https://help.aliyun.com/zh/dataworks/developer-reference/api-dataworks-public-2024-05-18-createcomputeresource) - [GetComputeResource](https://help.aliyun.com/zh/dataworks/developer-reference/api-dataworks-public-2024-05-18-getcomputeresource) - [ListComputeResources](https://help.aliyun.com/zh/dataworks/developer-reference/api-dataworks-public-2024-05-18-listcomputeresources) ### Resource Groups - [CreateResourceGroup](https://help.aliyun.com/zh/dataworks/developer-reference/api-dataworks-public-2024-05-18-createresourcegroup) - [GetResourceGroup](https://help.aliyun.com/zh/dataworks/developer-reference/api-dataworks-public-2024-05-18-getresourcegroup) - [ListResourceGroups](https://help.aliyun.com/zh/dataworks/developer-reference/api-dataworks-public-2024-05-18-listresourcegroups)

Alibabacloud Dataworks Infra Manage

Comments (0)