--- name: alibabacloud-pai-eas-service-deploy description: | Deploy AI models as PAI-EAS inference services. Supports LLMs (Qwen, Llama), image gen (SD, SDXL), speech synthesis, and more. When to use: deploy models, create inference services, EAS deployment, model serving, deploy vLLM/SGLang/ComfyUI. license: Apache-2.0 metadata: version: "1.0.0" domain: aiops owner: pai-eas-team contact: pai-eas-agent@alibaba-inc.com tags: - pai-eas - model-deployment - inference-service - llm - vllm - sglang required_tools: - aliyun - jq prerequisites: - "Aliyun CLI >= 3.3.1" - "jq command-line JSON processor" required_permissions: - "eas:CreateService" - "eas:DescribeService" - "eas:ListServices" - "eas:DescribeMachineSpec" - "eas:ListResources" - "eas:ListGateway" - "eas:DescribeGateway" - "nlb:ListLoadBalancers" - "aiworkspace:ListImages" - "aiworkspace:ListWorkspaces" - "vpc:DescribeVpcs" - "vpc:DescribeVSwitches" - "ecs:DescribeSecurityGroups" --- # PAI-EAS Service Deployment ## ⚠️ TOP RULES (read first) **1. 🔴 NO DUPLICATE SERVICE NAMES** 🔴 If a service with the target name already exists: STOP and inform the user. Do NOT delete and recreate. Do NOT reuse it either. **2. Mandatory API Calls** — Execute ALL of these in order: | # | API | CLI | Purpose | |---|-----|-----|---------| | 1 | ListImages | `aliyun aiworkspace list-images` | Validate image | | 2 | describe-machine-spec | `aliyun eas describe-machine-spec` | Validate GPU type | | 3 | create-service | `aliyun eas create-service` | Create service | | 4 | describe-service | `aliyun eas describe-service` | Check status (once) | | 5 | describe-service-endpoints | `aliyun eas describe-service-endpoints` | Get endpoints | Execute #1 and #2 ALWAYS, even if user provided the info. `describe-machine-spec` ≠ `list-resources`. `describe-service` ≠ `ListServices`. **3. Prohibited** — ❌ Reuse existing service ❌ Write bash scripts (run CLI directly) ❌ CPU+vLLM/SGLang ❌ `file://` in create-service ❌ Skip mandatory APIs ❌ Change the service name the user specified ❌ Poll describe-service in a loop (call once only) **4. Autonomous Execution** — Do NOT ask user for info discoverable via APIs. Do NOT ask "should I proceed?" Execute directly. Timeout? Retry with `--read-timeout 60`. Error? Inform user and CONTINUE. Missing param? Pick reasonable default. If any pre-check or resource discovery step fails, log the failure and continue to the next step. Only STOP for the specific conditions listed in Self-Verify Checkpoints (duplicate service name, missing NLB/GW/dedicated resource group). **5. Self-Verify Checkpoints:** ``` Before Step 2: Did Step 1.5 confirm no duplicate service name? If duplicate → STOP, inform user, do NOT proceed. Before Step 5: Have I run list-images AND describe-machine-spec? If NO → STOP and run them NOW. Before Step 6: Did Step 1.5 confirm no duplicate service name? If duplicate exists → STOP, inform user, do NOT proceed. Step 4 resources: If NLB/GW/Resource Group not found → inform user and STOP. Do NOT block or attempt workarounds. After Step 7: Did I call describe-service once and report the status? ``` **6. Run CLI commands DIRECTLY** — Use `execute_shell_command` to run each aliyun CLI command one at a time. Do NOT write bash scripts, deployment scripts, or shell files. Do NOT use `retry_command()` or any wrapper functions. Each command = one direct execution. --- ## Pre-checks ```bash aliyun configure ai-mode enable aliyun configure ai-mode set-user-agent \ --user-agent "AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy" aliyun plugin update aliyun configure set --auto-plugin-install true ``` **⚠️ MANDATORY: Install and update required plugins BEFORE proceeding:** ```bash aliyun plugin update aliyun plugin install aiworkspace --force aliyun plugin install eas --force ``` **Verify plugins are installed:** ```bash aliyun plugin list 2>/dev/null | grep -E "aiworkspace|eas" ``` If either plugin is missing after install, retry with: ```bash aliyun plugin install --force ``` See [CLI Guide](references/cli-installation-guide.md). Never read/echo AK/SK. Timeout? Retry with `--read-timeout 60`. **⚠️ MANDATORY CHECKPOINT: After pre-checks, you MUST proceed to Step 1 regardless of any warnings during plugin installation. Plugin install failures are non-blocking — the plugins may already be installed. NEVER stop execution at the pre-check stage.** --- ## Step 1: Collect Parameters | Param | Required | Default | |-------|----------|---------| | RegionId | Yes | - | | Service Name | Yes | - | | Workspace ID | No | Auto-select | | Image URI | Yes | - | | Instance Type | Yes | - | | Replicas | No | 1 | | Port | No | 8000 | | OSS Path | No | - | **Service name**: lowercase/digits/underscores only. No hyphens. 3-63 chars. **IMPORTANT**: Use the EXACT service name the user specifies. Do NOT rename. If the user specifies a prefix (e.g. "skill_qwen_开头"), generate a random suffix of 6 digits (e.g. `skill_qwen_482917`). **Set profile region** — Set the CLI profile region to match the deployment region. This avoids "Region mismatch" errors when `--cluster-id` differs from the profile's default region: ```bash aliyun configure set --region ``` **Workspace ID**: Required in `metadata.workspace_id`. If user does not specify a workspace, query available workspaces and pick one: ```bash aliyun aiworkspace list-workspaces --region \ --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy | \ jq '.Workspaces[] | select(.Status == "ENABLED") | {WorkspaceId, WorkspaceName}' ``` If multiple workspaces exist, list them and let the user choose. If only one exists, use it directly. ## Step 1.5: Check for Duplicate Service Name ```bash aliyun eas list-services --region --cluster-id \ --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy | \ jq '.ServiceList[] | select(.ServiceName == "") | {ServiceName, Status}' ``` **If a service with the same name already exists → STOP and inform the user: "A service named already exists (Status: ). Please choose a different name." Do NOT delete or reuse it.** **If no duplicate → proceed to Step 2.** ## Step 2: ListImages (🚧 BLOCKING GATE — NEVER SKIP) Execute even if user provided image URI. Purpose = VALIDATION. **⚠️ If you see "parse error" or "Exit Code 4", the plugin failed to install. You MUST retry with explicit install:** ```bash aliyun plugin install aiworkspace --force ``` Then retry the list-images command. Do NOT skip this step. ```bash aliyun aiworkspace list-images --region --verbose true \ --labels 'system.official=true,system.supported.eas=true' \ --page-size 50 --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy | \ jq '.Images[] | {ImageId, ImageUri, ImageCategory}' > /tmp/eas_images.json echo "Saved $(jq length /tmp/eas_images.json) images" ``` **If still fails after plugin reinstall, retry with timeout:** ```bash aliyun aiworkspace list-images --region --verbose true \ --labels 'system.official=true,system.supported.eas=true' \ --page-size 50 --read-timeout 60 \ --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy | \ jq '.Images[] | {ImageId, ImageUri, ImageCategory}' > /tmp/eas_images.json echo "Saved $(jq length /tmp/eas_images.json) images" ``` See [Model-Image](references/model-image-matching.md). **This output is REQUIRED by Step 5. Do NOT skip.** **❌ NEVER skip this step saying "user already provided the image".** ## Step 3: describe-machine-spec (🚧 BLOCKING GATE — NEVER SKIP) Execute even if user specified instance type. NOT `list-resources`. **⚠️ If this command fails, retry with `--read-timeout 60`. NEVER proceed to Step 5 without /tmp/eas_specs.json existing.** ```bash aliyun eas describe-machine-spec --region \ --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy | \ jq '.InstanceTypes[] | select(.GPU > 0) | {InstanceType, CPU, GPU, Memory}' \ > /tmp/eas_specs.json echo "Available GPU specs:"; cat /tmp/eas_specs.json ``` CPU→GPU auto-switch: If vLLM/SGLang + CPU, silently pick GPU. Do NOT ask. **This output is REQUIRED by Step 5. Do NOT skip.** ## Step 4: Network & Resource Config | Type | VPC | Config | |------|-----|--------| | Shared | No | (default, no networking fields) | | Dedicated GW | Yes | `networking.gateway` + `cloud.networking` | | NLB | Yes | `networking.nlb` + `cloud.networking` | **⚠️ If a required resource does not exist → STOP and inform the user. Do NOT block or attempt workarounds. This is a valid outcome.** **Dedicated Gateway** — Call `list-gateway`. If no gateway exists → inform user and STOP. Otherwise call `describe-gateway` to get VPC/VSwitch, then query security group under that VPC. If no security group found → inform user and STOP. ```bash aliyun eas list-gateway --region \ --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy ``` If gateway found, get details: ```bash aliyun eas describe-gateway --region --cluster-id \ --gateway-id \ --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy ``` Extract VPC and comma-separated VSwitch ID: ```bash aliyun eas describe-gateway --region --cluster-id \ --gateway-id \ --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy | \ jq '{vpc_id: .LoadBalancerList[0].VpcId, vswitch_id: (.LoadBalancerList[0].VSwitchIds | join(","))}' ``` **NLB** — Requires VPC/VSwitch/SecurityGroup. If user does not provide them, query via APIs. If any required resource not found → inform user and STOP. **⚠️ NLB requires ≥2 VSwitches across different availability zones.** Use comma-separated format: `"vswitch_id": "vsw-zone-a,vsw-zone-b"`. **⚠️ NLB Plugin Bug (aliyun-cli-eas v0.2.0):** If create-service with NLB config returns 400 with `'vswitch can not be null'` or `'vpcId, vswId and securityGroupId are required'`, this is a known CLI plugin bug (not a resource issue). **Fallback strategy:** 1. Retry create-service with NLB config once more (max 2 attempts). 2. If both fail → Remove `networking.nlb` and `cloud.networking` from service.json, redeploy with shared gateway. 3. Inform user: "NLB config failed due to CLI plugin limitation. Deployed with shared gateway instead." **EAS Dedicated Resource Group** — Call `list-resources`. Filter for `ResourceType == "Dedicated"` and `Status == "ResourceReady"`. ```bash aliyun eas list-resources --region \ --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy | \ jq '.Resources[] | select(.ResourceType == "Dedicated" and .Status == "ResourceReady") | {ResourceId, ResourceType, Status}' ``` - If exists → Set `"metadata": {"resource": ""}`. Do NOT set `cloud.computing`. - If NOT exists → Inform the user and STOP. Do NOT fall back to public resource group. ## Step 5: Build Service JSON **⚠️ BEFORE building JSON, you MUST read these reference files:** - `references/config-patterns.md` — Complete JSON templates for all 8 patterns - `references/config-schema.md` — Field descriptions and validation rules - `references/storage-mount.md` — OSS/NAS mount configuration details - `references/network-config.md` — NLB/Gateway network configuration details **⚠️ HARD GATE: Before writing service.json, VERIFY these files exist and have content. If either is missing → STOP and run that Step NOW.** ``` test -s /tmp/eas_images.json || echo "MISSING: Run Step 2 NOW" test -s /tmp/eas_specs.json || echo "MISSING: Run Step 3 NOW" ``` **⚠️ JSON format rules:** - Allowed top-level keys: `metadata`, `containers`, `storage`, `cloud`, `autoscaler`, `networking` - ❌ NEVER use as top-level keys: `spec`, `ServiceName`, `Image`, `Cpu`, `Memory`, `Gpu`, `processor_path`, `resourceGroupId`, `instance`, `port`, `command`, `access` - ❌ FORBIDDEN fields: `processor_path`, `resourceGroupId`, `spec`, `access` - `metadata.name` = service name, `metadata.workspace_id` = workspace (REQUIRED) - `containers[].image` = image URI, `containers[].command` = start command, `containers[].port` = port - `cloud.computing.instance_type` = instance type (MANDATORY for shared gateway) ### Quick Reference — JSON Skeletons Below are minimal skeletons. **Read `references/config-patterns.md` for complete templates with all fields and examples.** **Base (Shared Gateway):** ```json {"metadata":{"name":"","instance":1,"workspace_id":""}, "containers":[{"image":"","port":

,"command":""}], "cloud":{"computing":{"instance_type":""}}} ``` **+ OSS** → add `"storage":[{"mount_path":"/dir","oss":{"path":"oss:///

/","readOnly":true}}]` **+ Autoscaling** → add `"autoscaler":{"min":1,"max":4,"scaleStrategies":[{"metricName":"qps","threshold":20}]}` **+ Health Check** → add `startup_check` to `containers[]` (see config-patterns.md Pattern 4) **NLB** — full template (read `references/network-config.md` for details): ```json {"metadata":{"name":"","instance":1,"workspace_id":""}, "containers":[{"image":"","port":

,"command":""}], "cloud":{"computing":{"instance_type":""}, "networking":{"vpc_id":"","vswitch_id":",","security_group_id":""}}, "networking":{"nlb":[{"id":"default","listener_port":

,"netType":"intranet"}]}} ``` ⚠️ `vswitch_id` must be **comma-separated with ≥2 VSwitches across different zones** **Dedicated Resource Group** — `"metadata.resource"` instead of `cloud.computing`: ```json {"metadata":{"name":"","instance":1,"resource":"","workspace_id":""}, "containers":[{"image":"","port":

,"command":""}]} ``` **Dedicated Gateway** — `networking.gateway` + `cloud.networking`: ```json {"metadata":{"name":"","instance":1,"workspace_id":""}, "containers":[{"image":"","port":

,"command":""}], "networking":{"gateway":""}, "cloud":{"computing":{"instance_type":""}, "networking":{"vpc_id":"","vswitch_id":",","security_group_id":""}}} ``` ⚠️ `vswitch_id` comma-separated if gateway returns multiple VSwitches ### Validate Before Writing ```bash jq -r '.[] | select(.ImageUri | contains("vllm")) | .ImageUri' /tmp/eas_images.json jq -r '.[] | select(.InstanceType == "") | .InstanceType' /tmp/eas_specs.json ``` ## Step 6: Create Service (MANDATORY) **🔴 CONFIRM: Did Step 1.5 confirm no duplicate service name? If a service with this name already exists → STOP. Inform the user and do NOT proceed with create-service.** **Use `$(cat service.json)` NOT `file://service.json`.** **Run this DIRECTLY via execute_shell_command, do NOT write a bash script.** ```bash aliyun eas create-service --region \ --body "$(cat service.json)" \ --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy ``` **409 Conflict** → Service already exists. Inform the user and STOP. **400 BadRequest** with `'vswitch can not be null'` or `'vpcId, vswId and securityGroupId are required'` → NLB CLI plugin bug (see Step 4 fallback). Remove `networking.nlb` and `cloud.networking` from service.json and retry. ## Step 7: Verify Deployment **Call describe-service ONCE to check the current status. Do NOT poll. Do NOT loop. Do NOT wait for Running.** ```bash aliyun eas describe-service --region --cluster-id \ --service-name \ --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy | \ jq '{Status, ServiceName, ServiceId}' ``` **Report whatever status you get (Running, Waiting, Creating, etc.) and proceed to Step 8 immediately. create-service returning 200 = success.** ## Step 8: Report Result (MANDATORY) **Get endpoint info via DescribeServiceEndpoint:** ```bash aliyun eas describe-service-endpoints --region --cluster-id \ --service-name --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy | \ jq '{AccessToken, Endpoints: [.Endpoints[] | { Type: .EndpointType, Port: .Port, InternetEndpoints: .InternetEndpoints, IntranetEndpoints: .IntranetEndpoints }]}' ``` **Use the status from Step 7 and the endpoints above to report.** **Copy the ENTIRE output into your final response. Format:** ``` Deployment Summary ================== Service Name: Status: Endpoints: - : InternetEndpoint: IntranetEndpoint: Port: Service Invocation Examples: curl /api/predict/ \ -H "Authorization: " curl /api/predict/ \ -H "Authorization: " curl :/api/predict/ \ -H "Authorization: " ``` **`InternetEndpoint` and `IntranetEndpoint` MUST appear in your response, even if null.** If null: `(not available for this network type)` **Always include a service invocation example using the AccessToken and endpoint URL.** **Success criteria: create-service returning 200 with ServiceId = success. Any status (Running, Waiting, Creating) is acceptable.** --- When done, disable AI-Mode: `aliyun configure ai-mode disable` ## References (read when needed) | Doc | When to Read | |-----|-------------| | [Config Patterns](references/config-patterns.md) | **Step 5** — Complete JSON templates for all 8 patterns | | [Config Schema](references/config-schema.md) | **Step 5** — Field descriptions and validation rules | | [Storage Mount](references/storage-mount.md) | **Step 5** — OSS/NAS mount details | | [Network Config](references/network-config.md) | **Step 4/5** — NLB/Gateway config details | | [Model-Image](references/model-image-matching.md) | **Step 2** — Image selection guide | | [Related APIs](references/related-apis.md) | **Any step** — CLI command reference | | [Workflow](references/deployment-workflow.md) | Overview — Full deployment flow | | [CLI Guide](references/cli-installation-guide.md) | Pre-checks — Plugin install | | [RAM Policies](references/ram-policies.md) | Pre-checks — Required permissions | | [Service Features](references/service-features.md) | Step 5 — Advanced features | FILE:references/acceptance-criteria.md # Acceptance Criteria: alibabacloud-pai-eas-service-deploy **Scenario**: PAI-EAS Service Deployment **Purpose**: Skill test acceptance criteria **Table of Contents** - [CLI Command Patterns](#correct-cli-command-patterns) - [Service Config Validation](#service-config-validation) - [Authentication Patterns](#authentication-patterns) - [Parameter Confirmation Requirements](#parameter-confirmation-requirements) - [Resource Cleanup](#resource-cleanup) --- # Correct CLI Command Patterns ## 1. EAS Service Operations ### ✅ Correct: Create Service ```bash aliyun eas create-service --region cn-hangzhou --body "$(cat service.json)" --user-agent AlibabaCloud-Agent-Skills ``` ### ❌ Wrong: Missing --user-agent ```bash aliyun eas create-service --region cn-hangzhou --body "$(cat service.json)" ``` ### ❌ Wrong: Using API format instead of plugin mode ```bash aliyun eas create-service --region cn-hangzhou --body "$(cat service.json)" ``` ## 2. AIWorkSpace Operations ### ✅ Correct: List Images ```bash aliyun aiworkspace list-images --verbose true --labels 'system.official=true,system.supported.eas=true' --page-size 50 --user-agent AlibabaCloud-Agent-Skills ``` ### ❌ Wrong: Labels format error ```bash aliyun aiworkspace list-images --verbose true --labels 'system.official=true' --user-agent AlibabaCloud-Agent-Skills ``` ## 3. OSS Operations ### ✅ Correct: List Buckets ```bash ossutil ls ``` ### ✅ Correct: List Objects ```bash ossutil ls oss://bucket-name/path/ ``` ### ❌ Wrong: Missing oss:// prefix ```bash ossutil ls bucket-name/path/ ``` ## 4. VPC Operations ### ✅ Correct: Query VPC ```bash aliyun vpc describe-vpcs --biz-region-id cn-hangzhou --vpc-id vpc-xxx --user-agent AlibabaCloud-Agent-Skills ``` ### ❌ Wrong: Missing user-agent ```bash aliyun vpc describe-vpcs --biz-region-id cn-hangzhou --vpc-id vpc-xxx ``` --- # Service Config Validation ## 1. metadata Config ### ✅ Correct: Service Name Format ```json { "metadata": { "name": "my-vllm-service", "instance": 1 } } ``` ### ❌ Wrong: Service name contains uppercase letters ```json { "metadata": { "name": "My-VLLM-Service", "instance": 1 } } ``` ### ❌ Wrong: Service name contains special characters ```json { "metadata": { "name": "my.vllm.service", "instance": 1 } } ``` ## 2. containers Config ### ✅ Correct: Container Config ```json { "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.14.0-gpu", "port": 8000, "script": "vllm serve /models --port 8000" }] } ``` ### ❌ Wrong: Missing port config ```json { "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.14.0-gpu", "script": "vllm serve /models --port 8000" }] } ``` ## 3. storage Config ### ✅ Correct: OSS Mount ```json { "storage": [{ "mount_path": "/models", "oss": { "path": "oss://bucket/models/", "readOnly": true } }] } ``` ### ❌ Wrong: OSS path missing trailing slash ```json { "storage": [{ "mount_path": "/models", "oss": { "path": "oss://bucket/models", "readOnly": true } }] } ``` ### ❌ Wrong: Missing oss:// prefix ```json { "storage": [{ "mount_path": "/models", "oss": { "path": "bucket/models/", "readOnly": true } }] } ``` ## 4. cloud Config ### ✅ Correct: Public Resource Group ```json { "cloud": { "computing": { "instance_type": "ecs.gn7-c12g1.12xlarge" } } } ``` ### ✅ Correct: Multi-spec Instances ```json { "cloud": { "computing": { "instances": [ {"type": "ecs.gn7-c12g1.12xlarge"}, {"type": "ecs.gn8is.2xlarge"} ] } } } ``` ### ❌ Wrong: Wrong field name ```json { "cloud": { "computing": { "instanceType": "ecs.gn7-c12g1.12xlarge" } } } ``` ## 5. networking Config ### ✅ Correct: Dedicated Gateway ```json { "networking": { "gateway": "gw-xxx" } } ``` ### ✅ Correct: NLB ```json { "networking": { "nlb": [{ "id": "default", "listener_port": 9000, "netType": "intranet" }] } } ``` ### ❌ Wrong: NLB port is 8080 (not allowed) ```json { "networking": { "nlb": [{ "id": "default", "listener_port": 8080, "netType": "intranet" }] } } ``` --- # Authentication Patterns ## ✅ Correct: Using CredentialClient (Python SDK) ```python from alibabacloud_credentials.client import Client as CredentialClient from alibabacloud_eas20210701.client import Client as EasClient from alibabacloud_tea_openapi import models as open_api_models credential = CredentialClient() config = open_api_models.Config(credential=credential) config.region_id = "cn-hangzhou" config.user_agent = "AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy" client = EasClient(config) ``` ## ❌ Wrong: Hardcoded AK/SK ```python config = open_api_models.Config( access_key_id="LTAIxxx", access_key_secret="xxx" ) ``` --- # Parameter Confirmation Requirements ## ✅ Correct: All user parameters must be confirmed The following parameters must be confirmed before deployment: - RegionId (region) - Service name - Workspace ID - Image URI - Instance type - OSS path (if mounting) - Gateway ID (if using dedicated gateway) - VPC/VSwitch/Security group (if using ALB/NLB) ## ❌ Wrong: Using default values without confirmation ```bash # Wrong: Using default region without asking user aliyun eas create-service --region cn-hangzhou ... ``` --- # Resource Cleanup ## ✅ Correct: Cleanup after deployment failure ```bash aliyun eas delete-service \ --cluster-id cn-hangzhou \ --service-name \ --user-agent AlibabaCloud-Agent-Skills ``` ## ❌ Wrong: Not cleaning up failed services Failure to delete after service creation failure leads to resource waste. FILE:references/api-reference.md # API 参考手册 ## API 返回结构 **阿里云 API 返回结构不一致，使用 jq 时需注意**： | API | jq 路径 | 结构 | |-----|---------|------| | `AIWorkSpace ListWorkspaces` | `.Workspaces[]` | 单层 | | `AIWorkSpace ListImages` | `.Images[]` | 单层 | | `eas DescribeMachineSpec` | `.InstanceMetas[]` | 单层 | | `eas list-gateway` | `.Gateways[]` | 单层 | | `eas ListResources` | `.Resources[]` | 单层 | | `eas DescribeService` | `.Service` | 单个对象 | | `eas describe-service-event` | `.Events[]` | 单层 | | `vpc DescribeVpcs` | `.Vpcs.Vpc[]` | ⚠️ 双层 | | `vpc DescribeVSwitches` | `.VSwitches.VSwitch[]` | ⚠️ 双层 | | `ecs DescribeSecurityGroups` | `.SecurityGroups.SecurityGroup[]` | ⚠️ 双层 | | `nlb ListLoadBalancers` | `.LoadBalancers[]` | 单层 | ## jq 示例 ```bash aliyun aiworkspace list-workspaces --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills | jq -r '.Workspaces[] | "\(.WorkspaceId)\t\(.WorkspaceName)"' aliyun eas list-gateway --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills | jq -r '.Gateways[] | "\(.GatewayId)\t\(.GatewayName)"' aliyun vpc describe-vpcs --biz-region-id cn-hangzhou --user-agent AlibabaCloud-Agent-Skills | jq -r '.Vpcs.Vpc[] | "\(.VpcId)\t\(.VpcName)"' ``` ## CLI 命令参考 ### 参数命名规则 | 命令类型 | 参数名 | 示例 | |---------|--------|------| | 列表类/创建类 | `--region` | `ListServices`, `CreateService` | | 针对单个服务 | `--cluster-id` | `DescribeService`, `DeleteService` | ### 常用命令 ```bash aliyun aiworkspace list-workspaces --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills aliyun aiworkspace list-images --verbose true --labels 'system.official=true,system.supported.eas=true' --page-size 100 --user-agent AlibabaCloud-Agent-Skills aliyun eas describe-machine-spec --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills aliyun eas list-resources --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills aliyun eas list-gateway --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills aliyun eas describe-gateway --cluster-id cn-hangzhou --gateway-id gw-xxx --user-agent AlibabaCloud-Agent-Skills aliyun eas create-service --region cn-hangzhou --body "$(cat service.json)" --user-agent AlibabaCloud-Agent-Skills aliyun eas describe-service --cluster-id cn-hangzhou --service-name my_service --user-agent AlibabaCloud-Agent-Skills ``` ## 权限列表详见 [RAM 权限策略](ram-policies.md) ## 常用 GPU 规格 | 规格 | GPU | CPU | 内存 | 适用场景 | |------|-----|-----|------|---------| | `ecs.gn6i-c4g1.xlarge` | 1× T4 | 4 | 16Gi | 小模型推理 | | `ecs.gn6i-c8g1.2xlarge` | 1× T4 | 8 | 32Gi | 中等模型 | | `ecs.gn7-c12g1.12xlarge` | 4× A10 | 12 | 192Gi | 大模型推理 | FILE:references/cli-installation-guide.md # Aliyun CLI Installation & Configuration Guide Complete guide for installing and configuring Aliyun CLI. > **Aliyun CLI 3.3.1+**: Supports installing and using all published Alibaba Cloud product plugins. Make sure to upgrade to 3.3.1 or later for full plugin ecosystem coverage. **Table of Contents** - [Installation](#installation) - [Configuration](#configuration) - [Verification](#verification) - [Security Best Practices](#security-best-practices) - [Troubleshooting](#troubleshooting) - [Advanced Configuration](#advanced-configuration) - [Next Steps](#next-steps) - [References](#references) ## Installation ### macOS **Using Homebrew (Recommended)** ```bash brew install aliyun-cli # Upgrade to latest brew upgrade aliyun-cli # Verify version (>= 3.3.1) aliyun version ``` **Using Binary** ```bash # Download wget https://aliyuncli.alicdn.com/aliyun-cli-macosx-latest-amd64.tgz # Extract tar -xzf aliyun-cli-macosx-latest-amd64.tgz # Move to PATH sudo mv aliyun /usr/local/bin/ # Verify aliyun version ``` ### Linux **Debian/Ubuntu** ```bash # Download wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz # Extract and install tar -xzf aliyun-cli-linux-latest-amd64.tgz sudo mv aliyun /usr/local/bin/ # Verify aliyun version ``` **CentOS/RHEL** ```bash # Download wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-amd64.tgz # Extract and install tar -xzf aliyun-cli-linux-latest-amd64.tgz sudo mv aliyun /usr/local/bin/ # Verify aliyun version ``` **ARM64 Architecture** ```bash # Download ARM64 version wget https://aliyuncli.alicdn.com/aliyun-cli-linux-latest-arm64.tgz # Extract and install tar -xzf aliyun-cli-linux-latest-arm64.tgz sudo mv aliyun /usr/local/bin/ ``` ### Windows **Using Binary** 1. Download from: https://aliyuncli.alicdn.com/aliyun-cli-windows-latest-amd64.zip 2. Extract the ZIP file 3. Add the directory to your PATH environment variable 4. Open new Command Prompt or PowerShell 5. Verify: `aliyun version` **Using PowerShell** ```powershell # Download Invoke-WebRequest -Uri "https://aliyuncli.alicdn.com/aliyun-cli-windows-latest-amd64.zip" -OutFile "aliyun-cli.zip" # Extract Expand-Archive -Path aliyun-cli.zip -DestinationPath C:\aliyun-cli # Add to PATH (requires admin privileges) $env:Path += ";C:\aliyun-cli" [Environment]::SetEnvironmentVariable("Path", $env:Path, [System.EnvironmentVariableTarget]::Machine) # Verify aliyun version ``` ## Configuration ### Quick Start ```bash aliyun configure set \ --mode AK \ --access-key-id \ --access-key-secret \ --region cn-hangzhou ``` All `aliyun configure` commands support non-interactive flags, which is the recommended approach — it works in scripts, CI/CD pipelines, and agent-driven automation without hanging on stdin prompts. **Where to Get Access Keys** 1. Log in to Aliyun Console: https://ram.console.aliyun.com/ 2. Navigate to: AccessKey Management 3. Create a new AccessKey pair 4. Save the secret immediately — it's only shown once ### Configuration Modes Aliyun CLI supports 6 authentication modes. All examples below use non-interactive flags. #### 1. AK Mode (Access Key) Most common mode for personal accounts and scripts. ```bash aliyun configure set \ --mode AK \ --access-key-id LTAI5tXXXXXXXX \ --access-key-secret 8dXXXXXXXXXXXXXXXXXXXXXXXX \ --region cn-hangzhou ``` Configuration is stored in `~/.aliyun/config.json`: ```json { "current": "default", "profiles": [ { "name": "default", "mode": "AK", "access_key_id": "LTAI5tXXXXXXXX", "access_key_secret": "8dXXXXXXXXXXXXXXXXXXXXXXXX", "region_id": "cn-hangzhou", "output_format": "json", "language": "en" } ] } ``` #### 2. StsToken Mode (Temporary Credentials) For short-lived access (tokens expire in 1-12 hours). ```bash aliyun configure set \ --mode StsToken \ --access-key-id LTAI5tXXXXXXXX \ --access-key-secret 8dXXXXXXXXXXXXXXXXXXXXXXXX \ --sts-token v1.0:XXXXXXXXXXXXXXXX \ --region cn-hangzhou ``` Use cases: CI/CD pipelines, temporary access for external contractors, cross-account access. #### 3. RamRoleArn Mode (Assume RAM Role) Assume a RAM role for elevated or cross-account access. ```bash aliyun configure set \ --mode RamRoleArn \ --access-key-id LTAI5tXXXXXXXX \ --access-key-secret 8dXXXXXXXXXXXXXXXXXXXXXXXX \ --ram-role-arn acs:ram::123456789012:role/AdminRole \ --role-session-name my-session \ --region cn-hangzhou ``` Use cases: cross-account resource access, temporary elevated privileges, role-based access control. #### 4. EcsRamRole Mode (ECS Instance RAM Role) Use the RAM role attached to an ECS instance — no credentials needed. ```bash aliyun configure set \ --mode EcsRamRole \ --ram-role-name MyEcsRole \ --region cn-hangzhou ``` Requirements: must be running on an ECS instance with a RAM role attached. Use cases: scripts and automation running on ECS instances. #### 5. RsaKeyPair Mode (RSA Key Pair) Use RSA key pair for authentication (generate key pair in Aliyun Console first). ```bash aliyun configure set \ --mode RsaKeyPair \ --private-key /path/to/private-key.pem \ --key-pair-name my-key-pair \ --region cn-hangzhou ``` #### 6. RamRoleArnWithEcs Mode (ECS + RAM Role) Combine ECS instance role with RAM role assumption for cross-account access from ECS. ```bash aliyun configure set \ --mode RamRoleArnWithEcs \ --ram-role-name MyEcsRole \ --ram-role-arn acs:ram::123456789012:role/TargetRole \ --role-session-name my-session \ --region cn-hangzhou ``` ### Environment Variables **Highest priority** - overrides config file **Access Key Mode** ```bash export ALIBABA_CLOUD_ACCESS_KEY_ID=your_access_key_id export ALIBABA_CLOUD_ACCESS_KEY_SECRET=your_access_key_secret export ALIBABA_CLOUD_REGION_ID=cn-hangzhou ``` **STS Token Mode** ```bash export ALIBABA_CLOUD_ACCESS_KEY_ID=your_access_key_id export ALIBABA_CLOUD_ACCESS_KEY_SECRET=your_access_key_secret export ALIBABA_CLOUD_SECURITY_TOKEN=your_sts_token export ALIBABA_CLOUD_REGION_ID=cn-hangzhou ``` **ECS RAM Role Mode** ```bash export ALIBABA_CLOUD_ECS_METADATA=role_name ``` **Use Case**: - CI/CD pipelines - Docker containers - Temporary credential override ### Managing Multiple Profiles **Create Named Profiles** ```bash aliyun configure set --profile projectA \ --mode AK \ --access-key-id LTAI5tAAAAAAAA \ --access-key-secret 8dAAAAAAAAAAAAAAAAAAAAAAAA \ --region cn-hangzhou aliyun configure set --profile projectB \ --mode AK \ --access-key-id LTAI5tBBBBBBBB \ --access-key-secret 8dBBBBBBBBBBBBBBBBBBBBBBBB \ --region cn-shanghai ``` **Use Specific Profile** ```bash aliyun ecs describe-instances --profile projectA export ALIBABA_CLOUD_PROFILE=projectA aliyun ecs describe-instances # Uses projectA ``` **List and Switch Profiles** ```bash aliyun configure list # List all profiles aliyun configure set --current projectA # Switch default profile ``` ### Credential Priority Credentials are loaded in this order (first found wins): 1. **Command-line flag**: `--profile ` 2. **Environment variable**: `ALIBABA_CLOUD_PROFILE` 3. **Environment credentials**: `ALIBABA_CLOUD_ACCESS_KEY_ID`, etc. 4. **Configuration file**: `~/.aliyun/config.json` (current profile) 5. **ECS Instance RAM Role**: If running on ECS with attached role ## Verification ### Test Authentication ```bash # Basic test - list regions aliyun ecs describe-regions # Expected output: JSON array of regions ``` **If successful**, you'll see: ```json { "Regions": { "Region": [ { "RegionId": "cn-hangzhou", "RegionEndpoint": "ecs.cn-hangzhou.aliyuncs.com", "LocalName": "华东 1（杭州）" }, ... ] }, "RequestId": "..." } ``` **If failed**, you'll see error messages: - `InvalidAccessKeyId.NotFound` - Wrong Access Key ID - `SignatureDoesNotMatch` - Wrong Access Key Secret - `InvalidSecurityToken.Expired` - STS token expired (for StsToken mode) - `Forbidden.RAM` - Insufficient permissions ### Debug Configuration ```bash # Show current configuration aliyun configure get # Test with debug logging aliyun ecs describe-regions --log-level=debug # Check credential provider aliyun configure get mode ``` ## Security Best Practices ### 1. Use RAM Users (Not Root Account) ❌ **Don't**: Use Aliyun root account credentials ✅ **Do**: Create RAM users with specific permissions ```bash # Create RAM user in console # Attach only necessary policies # Use RAM user's access keys ``` ### 2. Principle of Least Privilege Grant only the minimum permissions needed: ```bash # Example: Read-only ECS access # Attach policy: AliyunECSReadOnlyAccess ``` ### 3. Rotate Access Keys Regularly ```bash # Create new access key in RAM Console, then update configuration aliyun configure set --access-key-id NEW_KEY --access-key-secret NEW_SECRET # Delete old access key from console ``` ### 4. Use STS Tokens for Temporary Access ```bash aliyun configure set --mode StsToken \ --access-key-id XXXX --access-key-secret XXXX \ --sts-token XXXX --region cn-hangzhou ``` ### 5. Use ECS RAM Roles When Possible ```bash aliyun configure set --mode EcsRamRole --ram-role-name MyRole --region cn-hangzhou ``` ### 6. Never Commit Credentials ```bash # Add to .gitignore echo "~/.aliyun/config.json" >> .gitignore # Use environment variables in CI/CD instead ``` ### 7. Secure Config File ```bash # Restrict permissions chmod 600 ~/.aliyun/config.json ``` ## Troubleshooting ### Issue: Command Not Found ```bash # Check installation which aliyun # Check PATH echo $PATH # Reinstall or add to PATH ``` ### Issue: Authentication Failed ```bash # Verify configuration aliyun configure get # Test with debug aliyun ecs describe-regions --log-level=debug # Check credentials in console # Verify access key is active ``` ### Issue: Permission Denied ```bash # Error: Forbidden.RAM # Check RAM user permissions # Attach necessary policies in RAM console # Example: AliyunECSFullAccess for ECS operations ``` ### Issue: STS Token Expired ```bash # Error: InvalidSecurityToken.Expired # Reconfigure with new token aliyun configure set --mode StsToken \ --access-key-id XXXX --access-key-secret XXXX \ --sts-token NEW_TOKEN --region cn-hangzhou ``` ### Issue: Wrong Region ```bash # Some resources may not exist in the specified region # Check available regions aliyun ecs describe-regions # Update default region aliyun configure set region cn-shanghai ``` ## Advanced Configuration ### Custom Endpoint ```bash # Use custom or private endpoint export ALIBABA_CLOUD_ECS_ENDPOINT=ecs-vpc.cn-hangzhou.aliyuncs.com ``` ### Proxy Settings ```bash # HTTP proxy export HTTP_PROXY=http://proxy.example.com:8080 export HTTPS_PROXY=http://proxy.example.com:8080 # No proxy for specific domains export NO_PROXY=localhost,127.0.0.1,.aliyuncs.com ``` ### Timeout Settings ```bash # Connection timeout (default: 10s) export ALIBABA_CLOUD_CONNECT_TIMEOUT=30 # Read timeout (default: 10s) export ALIBABA_CLOUD_READ_TIMEOUT=30 ``` ## Next Steps After installation and configuration: 1. **Install plugins** for services you need (v3.3.1+ supports all published product plugins): ```bash aliyun plugin install --names ecs vpc rds # List all available plugins aliyun plugin list-remote ``` 2. **Explore commands**: ```bash aliyun ecs --help aliyun fc --help ``` 3. **Read documentation**: - [Command Syntax Guide](./command-syntax.md) - [Global Flags Reference](./global-flags.md) - [Common Scenarios](./common-scenarios.md) ## References - Official Documentation: https://help.aliyun.com/zh/cli/ - RAM Console: https://ram.console.aliyun.com/ - Access Key Management: https://ram.console.aliyun.com/manage/ak - Plugin Repository: https://github.com/aliyun/aliyun-cli FILE:references/config-examples.md # 服务配置示例 **目录** - [JSON 字段规范](#json-字段规范重要) - [容器模式配置](#容器模式配置重要) - [基础配置](#基础配置) - [完整配置](#完整配置) - [公共资源组配置](#公共资源组配置) - [专属资源组配置](#专属资源组配置) - [ALB 网关配置](#alb-网关配置) - [NLB 配置](#nlb-配置) - [自动扩缩容配置](#自动扩缩容配置) - [存储挂载配置](#存储挂载配置) ## ⚠️ JSON 字段规范（重要） **服务名称必须放在 `metadata.name` 字段**，不是顶层字段： ```json { "metadata": { "name": "my-service", // ✅ 正确：服务名称在这里 "instance": 1 } } ``` **错误示例**： ```json { "service_name": "my-service", // ❌ 错误：这是无效字段 "name": "my-service" // ❌ 错误：不在 metadata 中 } ``` ## 容器模式配置（重要）使用镜像部署时，必须配置 `containers` 字段： ```json { "metadata": { "name": "my-service", "instance": 1 }, "containers": [{ "image": "镜像地址", "port": 8000, "command": "启动命令" }], "storage": [{ "mount_path": "/model_dir", "oss": { "path": "oss://bucket/models/" } }], "cloud": { "computing": { "instance_type": "ecs.gn6i-c8g1.2xlarge" } } } ``` **关键字段**： - `metadata.name` - 服务名称（必填） - `containers[].image` - 镜像地址（必填） - `containers[].port` - 服务端口（必填） - `containers[].command` - 启动命令（可选） **⚠️ 注意**：使用 `containers` 字段，不要使用 `processor` 或 `processor_path`。 ## 基础配置 ```json { "metadata": { "name": "simple_service", "instance": 1 }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.14.0-gpu", "port": 8000 }], "cloud": { "computing": { "instance_type": "ecs.gn6i-c8g1.2xlarge" } } } ``` ## 完整配置 ```json { "metadata": { "name": "myservice", "instance": 2, "workspace_id": "368951", "disk": "30Gi", "shm_size": 100, "enable_grpc": true }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.14.0-gpu", "port": 8000, "env": [ {"name": "NCCL_P2P_DISABLE", "value": "1"} ] }], "cloud": { "computing": { "instance_type": "ecs.gn6e-c12g1.12xlarge" }, "networking": { "vpc_id": "vpc-xxx", "vswitch_id": "vsw-xxx", "security_group_id": "sg-xxx" } }, "storage": [{ "mount_path": "/models", "oss": { "path": "oss://my-bucket/models/llama-7b", "readOnly": true } }], "networking": { "gateway": "gw-xxx" }, "autoscaler": { "min": 1, "max": 10, "scaleStrategies": [{ "metricName": "qps", "threshold": 100 }] } } ``` ## 公共资源组配置 ```json { "metadata": { "name": "public-resource-service", "instance": 1 }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.14.0-gpu", "port": 8000 }], "cloud": { "computing": { "instance_type": "ecs.gn6i-c8g1.2xlarge" } }, "storage": [{ "mount_path": "/models", "oss": { "path": "oss://my-bucket/models/" } }] } ``` ## 专属资源组配置 ```json { "metadata": { "name": "dedicated-resource-service", "instance": 1 }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.14.0-gpu", "port": 8000 }], "cloud": { "computing": { "instance_type": "ecs.gn6i-c8g1.2xlarge" } }, "resource": "eas-r-xxx", "storage": [{ "mount_path": "/models", "oss": { "path": "oss://my-bucket/models/" } }] } ``` ## ALB 网关配置 ```json { "metadata": { "name": "alb-gateway-service", "instance": 1 }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.14.0-gpu", "port": 8000 }], "cloud": { "computing": { "instance_type": "ecs.gn6i-c8g1.2xlarge" }, "networking": { "vpc_id": "{从网关获取}", "vswitch_id": "{从网关获取}", "security_group_id": "sg-xxx" } }, "networking": { "gateway": "gw-xxx" } } ``` ## NLB 配置 ```json { "metadata": { "name": "nlb-service", "instance": 1 }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.14.0-gpu", "port": 8000 }], "cloud": { "computing": { "instance_type": "ecs.gn6i-c8g1.2xlarge" }, "networking": { "vpc_id": "vpc-xxx", "vswitch_id": "vsw-xxx", "security_group_id": "sg-xxx" } }, "networking": { "nlb": [{ "id": "default", "listener_port": 8080, "netType": "intranet" }] } } ``` ## 自动扩缩容配置 ```json { "autoscaler": { "min": 1, "max": 10, "scaleStrategies": [ { "metricName": "qps", "threshold": 100 }, { "metricName": "cpu", "threshold": 80 } ] } } ``` ## 存储挂载配置 ```json { "storage": [ { "mount_path": "/models", "oss": { "path": "oss://my-bucket/models/", "readOnly": true } }, { "mount_path": "/data", "nfs": { "server": "xxx.cn-hangzhou.nas.aliyuncs.com", "path": "/share" } }, { "mount_path": "/dataset", "dataset": { "id": "d-xxx", "version": "v1" } } ] } ``` FILE:references/config-patterns.md # Complete Config Pattern Examples This document contains 8 complete JSON config patterns for different deployment scenarios. ## Config Pattern Decision Table | User Requirement | Pattern | Key Fields | |-----------------|---------|------------| | vLLM + OSS + Public Resource + Shared GW | Pattern 1 | `storage[].oss`, `cloud.computing` | | vLLM + Autoscaling | Pattern 2 | `autoscaler` (camelCase!) | | vLLM + NLB | Pattern 3 | `networking.nlb` | | SGLang + Health Check | Pattern 4 | `containers[].startup_check` | | ComfyUI + OSS | Pattern 5 | `storage[].oss` | | Custom Image / CPU→GPU Auto-switch | Pattern 6 | Only `containers[].image` | | EAS Resource Group | Pattern 7 | `metadata.resource` | | Dedicated Gateway | Pattern 8 | `networking.gateway`, `cloud.networking` | ## Steps to Build JSON 1. Select the best matching pattern based on user requirements 2. Copy the JSON template for that pattern 3. Replace actual values (service name, image URI, OSS path, instance type, etc.) 4. If extra requirements exist (e.g. autoscaling + NLB), merge corresponding fields 5. Save as `service.json` --- ## Pattern 1: vLLM + OSS Mount + Public Resource Group + Shared Gateway ```json { "metadata": { "name": "qwen35_7b_prod", "instance": 1, "workspace_id": "" }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.13.0rc2.a8ec486.20260305pai-gpu", "port": 8000, "command": "vllm serve /model_dir --port 8000 --trust-remote-code", "startup_check": { "http_get": {"path": "/health", "port": 8000}, "initial_delay_seconds": 120, "period_seconds": 10, "failure_threshold": 30 } }], "storage": [{ "mount_path": "/model_dir", "oss": { "path": "oss://yqtest-model/qwen2.5-0.5b-instruct/", "readOnly": true } }], "cloud": { "computing": { "instance_type": "ecs.gn7i-c16g1.4xlarge" } } } ``` --- ## Pattern 2: vLLM + Autoscaling ```json { "metadata": { "name": "qwen_autoscaling_test", "instance": 1, "workspace_id": "" }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.13.0rc2.a8ec486.20260305pai-gpu", "port": 8000, "command": "vllm serve /model_dir --port 8000 --trust-remote-code" }], "storage": [{ "mount_path": "/model_dir", "oss": { "path": "oss://yqtest-model/qwen2.5-0.5b-instruct/", "readOnly": true } }], "cloud": { "computing": { "instance_type": "ecs.gn7i-c16g1.4xlarge" } }, "autoscaler": { "min": 1, "max": 4, "scaleStrategies": [ {"metricName": "qps", "threshold": 20} ] } } ``` --- ## Pattern 3: vLLM + NLB ```json { "metadata": { "name": "qwen_nlb_test", "instance": 1, "workspace_id": "" }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.13.0rc2.a8ec486.20260305pai-gpu", "port": 8000, "command": "vllm serve /model_dir --port 8000 --trust-remote-code" }], "storage": [{ "mount_path": "/model_dir", "oss": { "path": "oss://yqtest-model/qwen2.5-0.5b-instruct/", "readOnly": true } }], "cloud": { "computing": { "instance_type": "ecs.gn7i-c16g1.4xlarge" }, "networking": { "vpc_id": "vpc-xxx", "vswitch_id": "vsw-zone-a,vsw-zone-b", "security_group_id": "sg-xxx" } }, "networking": { "nlb": [{ "id": "default", "listener_port": 8000, "netType": "intranet" }] } } ``` --- ## Pattern 4: SGLang + Health Check ```json { "metadata": { "name": "llama3_8b_api", "instance": 1, "workspace_id": "" }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/sglang:0.5.8-acclep1.2.1-gpu", "port": 8000, "command": "python -m sglang.launch_server --model-path /model_dir --host 0.0.0.0 --port 8000", "startup_check": { "http_get": {"path": "/health", "port": 8000}, "initial_delay_seconds": 120, "period_seconds": 10, "failure_threshold": 30 } }], "storage": [{ "mount_path": "/model_dir", "oss": { "path": "oss://yqtest-model/llama3-8b-instruct/", "readOnly": true } }], "cloud": { "computing": { "instance_type": "ecs.gn7i-c16g1.4xlarge" } } } ``` --- ## Pattern 5: ComfyUI + OSS Mount ```json { "metadata": { "name": "sdxl_inference", "instance": 1, "workspace_id": "" }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/comfyui:2.2-api", "port": 8000, "command": "python main.py --listen 0.0.0.0 --port 8000", "startup_check": { "http_get": {"path": "/", "port": 8000}, "initial_delay_seconds": 60, "period_seconds": 10, "failure_threshold": 30 } }], "storage": [{ "mount_path": "/models", "oss": { "path": "oss://yqtest-model/sdxl-v1.0/", "readOnly": true } }], "cloud": { "computing": { "instance_type": "ecs.gn7i-c16g1.4xlarge" } } } ``` --- ## Pattern 6: vLLM + Small Model + GPU Instance (auto-switch from CPU when user requests CPU) ```json { "metadata": { "name": "qwen_cpu_service", "instance": 2, "workspace_id": "" }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.13.0rc2.a8ec486.20260305pai-gpu", "port": 8000, "command": "vllm serve /model_dir --port 8000 --trust-remote-code" }], "storage": [{ "mount_path": "/model_dir", "oss": { "path": "oss://yqtest-model/qwen2.5-0.5b-instruct/", "readOnly": true } }], "cloud": { "computing": { "instance_type": "ecs.gn6i-c4g1.xlarge" } } } ``` --- ## Pattern 7: EAS Resource Group Deployment ```json { "metadata": { "name": "my_vllm_service", "instance": 1, "resource": "eas-r-d29k8ytqxzxmqi7l0s", "workspace_id": "", "gpu": 1, "cpu": 4, "memory": 8000 }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.13.0rc2.a8ec486.20260305pai-gpu", "port": 8000, "command": "vllm serve /model_dir --port 8000 --trust-remote-code" }], "storage": [{ "mount_path": "/model_dir", "oss": { "path": "oss://yqtest-model/qwen2.5-0.5b-instruct/", "readOnly": true } }] } ``` --- ## Pattern 8: Dedicated Gateway Deployment ```json { "metadata": { "name": "my_gateway_service", "instance": 1, "workspace_id": "" }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.13.0rc2.a8ec486.20260305pai-gpu", "port": 8000, "command": "vllm serve /model_dir --port 8000 --trust-remote-code" }], "storage": [{ "mount_path": "/model_dir", "oss": { "path": "oss://yqtest-model/qwen2.5-0.5b-instruct/", "readOnly": true } }], "networking": { "gateway": "gw-48hmbdt00fi6x90gft" }, "cloud": { "computing": { "instance_type": "ecs.gn7i-c16g1.4xlarge" }, "networking": { "vpc_id": "vpc-bp13kiflgde6v9dc9smc8", "vswitch_id": "vsw-bp1bhmnwqdh1ta9z9klms,vsw-bp1lz95xtmjiwqcpq31ng", "security_group_id": "sg-bp1e36bfv61nfy1yudyc" } } } ``` FILE:references/config-schema.md # Service Config Field Reference > This document lists all JSON config fields for PAI-EAS services. **Table of Contents** - [Config Structure Overview](#config-structure-overview) - [metadata (required)](#metadata-required) - [containers (required)](#containers-required) - [cloud (public resource group)](#cloud-public-resource-group) - [storage (mount)](#storage-mount) - [networking](#networking) - [autoscaler](#autoscaler) - [runtime](#runtime) - [features](#features) - [Full Example](#full-example) ## Config Structure Overview ```json { "metadata": { ... }, // Service metadata (required) "containers": [ ... ], // Container config (required) "cloud": { ... }, // Public resource group config "storage": [ ... ], // Storage mount "networking": { ... }, // Network config "autoscaler": { ... }, // Autoscaling "runtime": { ... }, // Runtime config "features": { ... } // Feature config } ``` --- ## metadata (required) Service metadata defining basic service information. | Field | Type | Required | Default | Description | |-------|------|----------|---------|-------------| | `name` | string | ✅ | - | Service name, lowercase letters/digits/underscores, 3-63 chars | | `instance` | int | ❌ | 1 | Number of replicas | | `workspace_id` | string | ❌ | - | Workspace ID | | `resource` | string | ❌ | - | Dedicated resource group ID (mutually exclusive with cloud.computing) | | `disk` | string | ❌ | - | Temp disk size, e.g. "30Gi" | | `shm_size` | int | ❌ | 64 | Shared memory size (GB) | | `rdma` | int | ❌ | - | Number of RDMA NICs | | `enable_grpc` | bool | ❌ | false | Enable GRPC protocol | | `rolling_strategy` | object | ❌ | - | Rolling update strategy | | `eas` | object | ❌ | - | EAS advanced config | ### rolling_strategy | Field | Type | Description | |-------|------|-------------| | `max_surge` | int | Max new instances during rolling update | | `max_unavailable` | int | Max unavailable instances during rolling update | --- ## containers (required) Container config array, must contain at least one container. | Field | Type | Required | Default | Description | |-------|------|----------|---------|-------------| | `image` | string | ✅ | - | Image URI | | `port` | int | ✅ | 8000 | Service port | | `script` | string | ❌ | - | Startup script | | `command` | string | ❌ | - | Startup command | | `args` | []string | ❌ | - | Command arguments | | `env` | []EnvVar | ❌ | - | Environment variables | | `prepare` | Prepare | ❌ | - | Pre-install config | | `startup_check` | Probe | ❌ | - | Startup probe | | `liveness_check` | Probe | ❌ | - | Liveness probe | | `health_check` | Probe | ❌ | - | Health check | | `resources` | ResourceRequirements | ❌ | - | Resource limits | ### EnvVar ```json {"name": "ENV_NAME", "value": "env_value"} ``` ### Prepare ```json { "pythonRequirements": ["numpy==1.6.4", "pandas"], "pythonRequirementsPath": "/path/to/requirements.txt" } ``` ### Probe (Health Check) | Field | Type | Default | Description | |-------|------|---------|-------------| | `http_get` | object | - | HTTP check config | | `initial_delay_seconds` | int | 15 | Initial delay in seconds | | `period_seconds` | int | 10 | Check interval in seconds | | `timeout_seconds` | int | 1 | Timeout in seconds | | `success_threshold` | int | 1 | Success threshold | | `failure_threshold` | int | 1 | Failure threshold | ```json { "http_get": {"path": "/health", "port": 8000}, "initial_delay_seconds": 15, "period_seconds": 10, "timeout_seconds": 1, "success_threshold": 1, "failure_threshold": 3 } ``` --- ## cloud (public resource group) Config when using public resource group. ### cloud.computing | Field | Type | Required | Description | |-------|------|----------|-------------| | `instance_type` | string | either/or | Instance type, e.g. "ecs.gn6i-c8g1.2xlarge" | | `instances` | []object | either/or | Multi-spec instance list | ```json { "cloud": { "computing": { "instance_type": "ecs.gn6i-c8g1.2xlarge" } } } ``` Or: ```json { "cloud": { "computing": { "instances": [ {"type": "ecs.gn6i-c8g1.2xlarge"}, {"type": "ecs.gn7-c12g1.12xlarge"} ] } } } ``` ### cloud.networking VPC network config (required for ALB/NLB). | Field | Type | Required | Description | |-------|------|----------|-------------| | `vpc_id` | string | ✅ | VPC ID | | `vswitch_id` | string | ✅ | VSwitch ID, comma-separated for multi-zone (e.g. `"vsw-a,vsw-b"`) | | `security_group_id` | string | ✅ | Security group ID | --- ## storage (mount) Storage mount config array. ### OSS Mount ```json { "mount_path": "/models", "oss": { "path": "oss://my-bucket/models/", "readOnly": true } } ``` ### NAS/NFS Mount ```json { "mount_path": "/data", "nfs": { "server": "xxx.cn-hangzhou.nas.aliyuncs.com", "path": "/share" } } ``` ### Dataset Mount ```json { "mount_path": "/dataset", "dataset": { "id": "d-xxx", "version": "v1", "read_only": true } } ``` --- ## networking ### Shared Gateway No networking field needed. ### ALB Dedicated Gateway ```json { "networking": { "gateway": "gw-xxx" } } ``` ### NLB ```json { "networking": { "nlb": [ { "id": "default", // or "nlb-xxx" "listener_port": 8000, "netType": "intranet" } ] } } ``` | Field | Type | Description | |-------|------|-------------| | `id` | string | "default" for system-created, or actual NLB ID | | `listener_port` | int | Listener port (cannot be 8080) | | `netType` | string | "intranet" or "internet" | --- ## autoscaler ### ⚠️ Important: Field Naming Convention **EAS API uses camelCase**: | ✅ Correct Field Name | ❌ Wrong Field Name | Description | |----------------------|---------------------|-------------| | `min` | ~~`min_replica`~~ | Min replicas | | `max` | ~~`max_replica`~~ | Max replicas | | `scaleStrategies` | ~~`scale_strategies`~~ | Scaling strategy array | | `metricName` | ~~`metric_name`~~ | Metric name | ```json { "autoscaler": { "min": 1, "max": 10, "scaleStrategies": [ {"metricName": "qps", "threshold": 100}, {"metricName": "cpu", "threshold": 80} ] } } ``` | Field | Type | Required | Default | Description | |-------|------|----------|---------|-------------| | `min` | int | ❌ | 1 | Min replicas | | `max` | int | ❌ | 10 | Max replicas | | `scaleStrategies` | []object | ❌ | - | Scaling strategies | ### scaleStrategies | Field | Description | |-------|-------------| | `metricName` | Metric name: qps, cpu, gpu, memory | | `threshold` | Trigger threshold | --- ## runtime ```json { "runtime": { "termination_grace_period": 30 } } ``` | Field | Type | Default | Description | |-------|------|---------|-------------| | `termination_grace_period` | int | 30 | Graceful shutdown wait time (seconds) | --- ## features ```json { "features": { "eas.aliyun.com/gpu-driver-version": "550.54.15" } } ``` Common features: - `eas.aliyun.com/gpu-driver-version`: GPU driver version --- ## Full Example ```json { "metadata": { "name": "my-llm-service", "instance": 2, "workspace_id": "", "disk": "30Gi", "shm_size": 64, "enable_grpc": true, "rolling_strategy": { "max_surge": 1, "max_unavailable": 0 } }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.14.0-gpu", "port": 8000, "script": "vllm serve /models/qwen-7b --port 8000 --tensor-parallel-size 2", "env": [ {"name": "NCCL_P2P_DISABLE", "value": "1"} ], "startup_check": { "http_get": {"path": "/health", "port": 8000}, "initial_delay_seconds": 60, "period_seconds": 10, "failure_threshold": 30 } }], "cloud": { "computing": { "instance_type": "ecs.gn7-c12g1.12xlarge" }, "networking": { "vpc_id": "vpc-xxx", "vswitch_id": "vsw-zone-a,vsw-zone-b", "security_group_id": "sg-xxx" } }, "storage": [{ "mount_path": "/models", "oss": { "path": "oss://my-bucket/models/qwen-7b", "readOnly": true } }], "networking": { "gateway": "gw-xxx" }, "autoscaler": { "min": 1, "max": 5, "scaleStrategies": [ {"metricName": "qps", "threshold": 50} ] }, "runtime": { "termination_grace_period": 60 } } ``` FILE:references/deployment-workflow.md # Deployment Workflow Detailed interaction flow and display format for PAI-EAS service deployment. **Table of Contents** - [Core Interaction Principles](#core-interaction-principles) - [Phase 1: Basic Info](#phase-1-basic-info) - [Phase 2: Environment](#phase-2-environment) - [Phase 3: Resources](#phase-3-resources) - [Phase 4: Network](#phase-4-network) - [Phase 5: Service Features](#phase-5-service-features) - [Phase 6: Deploy](#phase-6-deploy) --- ## Core Interaction Principles ### 1. Step-by-Step Guidance Show current step → Wait for user input → Show next step ### 2. List Pagination When more than 10 items: `Enter number to select | 'n' next page | keyword filter | 'all' show all` ### 3. Resource Selection Pattern For optional resources (e.g. EAS resource group, gateway): `1. Select from existing 2. Skip` For workspaces: Auto-query; select if found, skip immediately if empty ### 4. Default Value Handling ``` Port [8000]: 1. Use default 2. Custom Select (enter 1 or 2): ``` Defaults: mount path `/model_dir`, port `8000`, replicas `1`, initial delay `60`, check interval `10` --- ## Phase 1: Basic Info ### Step 1.1: Service Name (required) Lowercase letters, digits, underscores, 3-63 characters. ### Step 1.2: Workspace (optional, auto-handled) ```bash aliyun aiworkspace list-workspaces --region --page-size 100 --verbose true --user-agent AlibabaCloud-Agent-Skills ``` **Logic**: - **Has results** → Show list for user to select, record workspace_id - **Empty list** → **Skip immediately, proceed to next step! Never block asking user!** Simply omit `metadata.workspace_id` from JSON **⚠️ Never** stop deployment or ask user to create a workspace in the console **Display format (with results, paginated, max 10)**: ``` | # | Workspace ID | Name | |---|-------------|------------------| | 1 | 312319 | aiworkspace_test | | 2 | 545434 | ccjtest | Total: 20, showing 1-10 Enter number to select | 'n' next page | keyword filter | 'all' show all ``` --- ## Phase 2: Environment ### Step 2.1: Select Image **⚠️ MUST call `ListImages` to query official image list, even if user already provided image address (eval checkpoint)** | # | Category | Target Models | |---|----------|--------------| | 1 | LLM Inference | Qwen, Llama, Mistral | | 2 | Image Generation | Stable Diffusion | | 3 | Speech Synthesis | CosyVoice | | 4 | RAG | RAG Applications | | 5 | General Inference | PyTorch Models | | 6 | Custom Image | User-provided image | **LLM inference requires secondary selection**: vLLM / SGLang **Query images**: ```bash aliyun aiworkspace list-images --verbose true \ --labels 'system.official=true,system.supported.eas=true' \ --page-size 50 --user-agent AlibabaCloud-Agent-Skills | \ jq -r '.Images[] | "\(.ImageId)\t\(.Name)\t\(.Labels[] | select(.Key == "system.chipType") | .Value)"' ``` **Display format (paginated, max 10)**: ``` | # | Image Name | Version | Chip | |---|-----------------|-----------|------| | 1 | vllm:0.13.0rc2 | 0.13.0rc2 | GPU | | 2 | vllm:0.14.0-xpu | 0.14.0 | XPU | Total: 14, showing 1-10 Enter number to select | 'n' next page | keyword filter ``` **Query image chip type** (query immediately after selection, record CHIP_TYPE): ```bash aliyun aiworkspace get-image --image-id --user-agent AlibabaCloud-Agent-Skills | \ jq '{ChipType: (.Labels[] | select(.Key == "system.chipType") | .Value), ImageUri: (.Labels[] | select(.Key == "system.imageUri") | .Value)}' ``` **Output example**: ```json {"ChipType": "GPU", "ImageUri": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.14.0-gpu"} ``` **Variable**: `CHIP_TYPE=GPU` (subsequent resource selection will auto-filter compatible instances) **⚠️ Image must match model type**, see [Model-Image Matching Guide](model-image-matching.md) **⚠️ Chip compatibility (never block)**: Image chip type must match instance type! - GPU image → `ecs.gn*` series (NVIDIA GPU instances) - CPU image → any non-GPU instance - **If user requests CPU instance + GPU image (e.g. vLLM/SGLang), you MUST auto-switch to a GPU instance (e.g. ecs.gn7i-c16g1.4xlarge) silently. vLLM and SGLang REQUIRE GPU — NEVER deploy them on CPU. Do NOT ask the user, just switch and explain your choice.** ### Step 2.2: Storage Mount **Do you need to mount storage?** | # | Option | Description | |---|--------|-------------| | 1 | Yes | Mount model files or data | | 2 | No (skip) | No storage mount | **If "Yes", select storage type**: | # | Storage Type | Description | |---|-------------|-------------| | 1 | OSS | Mount OSS bucket (most common) | | 2 | NAS | Mount NAS file system | | 3 | CPFS | Mount CPFS | | 4 | Dataset | Use PAI dataset | **OSS mount flow**: 1. **Query bucket list**: ```bash ossutil ls ``` 2. **Display bucket list (paginated, max 10)**: ``` | # | Bucket Name | Region | |---|---------------|-------------| | 1 | yqtest-model | cn-hangzhou | | 2 | my-bucket | cn-shanghai | Total: 15, showing 1-10 Enter number to select | 'n' next page | keyword filter ``` 3. **List directory**: ```bash ossutil ls oss://bucket-name/ ``` 4. **Display directory list**: ``` Selected Bucket: yqtest-model | # | Path | Description | |---|-----------------|-------------| | 1 | Qwen3.5-0.8B/ | LLM model | | 2 | llama-7b/ | LLM model | Select model directory (enter number): ``` 5. **Configure mount path**: ``` Mount path [/model_dir]: 1. Use default 2. Custom Select (enter 1 or 2): ``` **⚠️ Important**: OSS path format must be `oss://bucket/path/` (note trailing `/`) **⚠️ Important**: OSS models must be mounted via storage as local paths (e.g. `/model_dir`). Never pass oss:// URL directly to vllm/sglang commands! See [Storage Mount Guide](storage-mount.md) ### Step 2.3: Startup Command ```bash aliyun aiworkspace get-image --image-id --user-agent AlibabaCloud-Agent-Skills | jq -r '.EasConfig.script' ``` **Logic**: - If image has preset command → Show to user for confirmation or modification - If no preset command → Use typical command as default **Typical commands**: | Image Type | Typical Command | |-----------|----------------| | vLLM | `vllm serve /model_dir --port 8000 --trust-remote-code` | | SGLang | `python -m sglang.launch_server --model-path /model_dir --port 8000` | | ComfyUI | `python main.py --listen 0.0.0.0 --port 8000` | **Interaction example**: ``` Startup command: 1. Use recommended: vllm serve /model_dir --port 8000 --trust-remote-code 2. Custom command Select (enter 1 or 2): ``` ### Step 2.4: Port ``` Port [8000]: 1. Use default 2. Custom Select (enter 1 or 2): ``` Default: `8000` --- ## Phase 3: Resources ### Step 3.1: Resource Type | # | Type | Description | |---|------|-------------| | 1 | Public Resource | On-demand, pay-as-you-go | | 2 | EAS Resource Group | Dedicated resource group | ### Step 3.2: Public Resource Group **Auto-filter logic** (based on image chip type from step 2.1): ``` Selected image chip type: GPU Filtering compatible instance types... ``` **Filter rules**: | Image Chip Type | jq Filter | Instance Type Pattern | |----------------|-----------|----------------------| | GPU | `.GPUAmount > 0` | `ecs.gn*` series (NVIDIA GPU) | | CPU | `.GPUAmount == 0` | Non-GPU instances | | PPU | `.InstanceType \| startswith("ecs.ebmppu")` | Hanguang instances | | XPU | `.InstanceType \| startswith("ecs.egs")` | XPU instances | **One-command query** (auto-select based on chip type): ```bash # Assuming CHIP_TYPE variable obtained from image labels aliyun eas describe-machine-spec --region --user-agent AlibabaCloud-Agent-Skills | \ jq -r --arg chip "$CHIP_TYPE" ' .InstanceMetas[] | select(.IsAvailable == true) | if $chip == "GPU" then select(.GPUAmount > 0) | "\(.InstanceType)\t\(.GPUAmount)x\(.GPU)\t\(.CPU) cores\t\(.Memory)GB" elif $chip == "CPU" then select(.GPUAmount == 0) | "\(.InstanceType)\t-\t\(.CPU) cores\t\(.Memory)GB" elif $chip == "PPU" then select(.InstanceType | startswith("ecs.ebmppu")) | "\(.InstanceType)\tPPU\t\(.CPU) cores\t\(.Memory)GB" elif $chip == "XPU" then select(.InstanceType | startswith("ecs.egs")) | "\(.InstanceType)\tXPU\t\(.CPU) cores\t\(.Memory)GB" else empty end ' ``` **GPU image query** (chip type = GPU): ```bash aliyun eas describe-machine-spec --region --user-agent AlibabaCloud-Agent-Skills | \ jq -r '.InstanceMetas[] | select(.IsAvailable == true and .GPUAmount > 0) | "\(.InstanceType)\t\(.GPUAmount)x\(.GPU)\t\(.CPU) cores\t\(.Memory)GB"' ``` **CPU image query** (chip type = CPU): ```bash aliyun eas describe-machine-spec --region --user-agent AlibabaCloud-Agent-Skills | \ jq -r '.InstanceMetas[] | select(.IsAvailable == true and .GPUAmount == 0) | "\(.InstanceType)\t-\t\(.CPU) cores\t\(.Memory)GB"' ``` **Display format (paginated, max 10)**: ``` | # | Instance Type | GPU | CPU | Memory | |---|-------------------------|--------|---------|--------| | 1 | ecs.gn6i-c4g1.xlarge | 1×T4 | 4 cores | 16GB | | 2 | ecs.gn7-c12g1.12xlarge | 4×A10 | 12 cores| 192GB | Total: 45, showing 1-10 Enter number to select | 'n' next page | keyword filter ``` ### Step 3.3: EAS Resource Group **⚠️ MUST call `list-resources` to query resource group list, even if user already provided resource group ID:** ```bash # MUST call: list all resource groups aliyun eas list-resources --region --user-agent AlibabaCloud-Agent-Skills # Query specific resource group details aliyun eas list-resources --region --user-agent AlibabaCloud-Agent-Skills | \ jq -r '.Resources[] | "\(.ResourceId)\t\(.ResourceName)\t\(.GpuCount)\t\(.GpuUsed)\t\(.CpuCount)\t\(.CpuUsed)"' ``` **Display format**: ``` | # | Resource ID | Name | GPU (total/used/free) | CPU (total/used/free) | |---|------------|------------|----------------------|----------------------| | 1 | eas-r-xxx | production | 16/2/14 | 128/20/108 | ``` **Calculation**: free = total - used ### Step 3.4: Replicas ``` Replicas [1]: 1. Use default 2. Custom Select (enter 1 or 2): ``` --- ## Phase 4: Network ### Step 4.1: Gateway Type | # | Type | VPC Config | Description | |---|------|-----------|-------------| | 1 | Shared Gateway | ❌ Not needed | Free, suitable for testing | | 2 | ALB Dedicated Gateway | ✅ Required | Recommended for production | | 3 | NLB | ✅ Required | High-performance load balancing | ### Step 4.2: ALB Dedicated Gateway Config **⚠️ MUST call `list-gateway` first to list gateways (even if user provided gateway ID), then call `describe-gateway` to get VPC info:** ```bash # MUST call: list all gateways aliyun eas list-gateway --region --user-agent AlibabaCloud-Agent-Skills ``` **Display format**: ``` | # | Gateway Name | Gateway ID | |---|-------------|-----------| | 1 | prod-gw | gw-xxx | | 2 | test-gw | gw-yyy | Enter number to select: ``` **Get gateway VPC info**: ```bash aliyun eas describe-gateway --cluster-id cn-hangzhou --gateway-id gw-xxx --user-agent AlibabaCloud-Agent-Skills | \ jq '{VpcId: .LoadBalancerList[0].VpcId, VSwitchIds: .LoadBalancerList[0].VSwitchIds}' ``` **Select VSwitch**: ``` Obtained gateway VPC: vpc-xxx | # | VSwitch ID | Availability Zone | |---|-------------|-------------------| | 1 | vsw-xxx | Zone A | | 2 | vsw-yyy | Zone B | Enter number to select: ``` **Select security group**: ```bash aliyun ecs describe-security-groups --biz-region-id cn-hangzhou --vpc-id --user-agent AlibabaCloud-Agent-Skills ``` See [Network Config](network-config.md) --- ## Phase 5: Service Features ### Step 5.1: Feature Selection **Do you need to configure service features?** | # | Option | Description | |---|--------|-------------| | 1 | Configure features | Select features to enable | | 2 | Skip | No features enabled (default) | **If "Configure features"**: ``` Available features (multi-select with comma): | # | Feature | Description | |---|---------|-------------| | 1 | Health Check | Configure startup/liveness probes | | 2 | Rolling Update | Graceful shutdown + rolling strategy | | 3 | GRPC | Enable GRPC protocol | | 4 | Autoscaling | Auto-adjust replicas based on load | Enter feature numbers to enable (e.g. 1,2,4), enter 'done' to finish: ``` **When modifying feature status**: ``` Current service feature config: | # | Feature | Status | |---|---------|--------| | 1 | Health Check | ✅ Enabled | | 2 | Rolling Update | ❌ Disabled | | 3 | GRPC | ❌ Disabled | | 4 | Autoscaling | ❌ Disabled | Enter number to toggle (e.g. enter 1 to disable health check), enter 'done' to finish: ``` See [Service Features Config](service-features.md) --- ## Phase 6: Deploy ### Step 6.1: Config Preview **Compatibility validation** (must pass before deployment): ``` ✅ Image chip type: GPU ✅ Instance type: ecs.gn7-c12g1.12xlarge (GPU instance) ✅ Compatibility: Passed ``` If compatibility check fails: ``` ❌ Image chip type: XPU ❌ Instance type: ecs.gn7-c12g1.12xlarge (GPU instance) ❌ Compatibility: Failed - XPU image requires XPU instance, please change image or instance type ``` ``` Generated service configuration: ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ { "metadata": { "name": "mytest_006", "instance": 1 }, "containers": [{ "image": "...", "port": 8000 }], ... } ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Config Summary: • Service Name: mytest_006 • Image: vllm:0.14.0-gpu (Chip: GPU) • Instance: ecs.gn7-c12g1.12xlarge (4×A10, GPU) • Network: Shared Gateway • Compatibility: ✅ Passed Select: 1. Confirm deploy 2. Modify config 3. Save JSON to file 4. Cancel ``` ### Step 6.2: Modify Config **Select config item to modify**: | # | Config Item | Current Value | |---|------------|---------------| | 1 | Service Name | mytest_006 | | 2 | Workspace | aiworkspace_test | | 3 | Image | vllm:0.13.0rc2 | | 4 | Storage Mount | Not configured | | 5 | Startup Command | vllm serve ... | | 6 | Port | 8000 | | 7 | Instance Type | ecs.gn5-c8g1.2xlarge | | 8 | Replicas | 1 | | 9 | Network | ALB Dedicated Gateway | | 10 | Features | Health Check | | 11 | Go back | - | Enter number to select: ### Step 6.3: Execute Deployment **Save config to file, then deploy**: ```bash # Save config to file cat > /tmp/eas_service.json << 'EOF' { "metadata": { "name": "my-service", "instance": 1 }, "containers": [{ "image": "...", "port": 8000 }], "cloud": { "computing": { "instance_type": "ecs.gn6i-c4g1.xlarge" } } } EOF # Execute deployment (may take 1-2 minutes) # ⚠️ Do NOT use file:// prefix, use $(cat) to read file content aliyun eas create-service --region --body "$(cat /tmp/eas_service.json)" --user-agent AlibabaCloud-Agent-Skills ``` **⚠️ Note**: Use `$(cat file)` to read file content for `--body`, do NOT use `file://` prefix (may cause JSON parsing errors) **⚠️ CRITICAL: If service with same name exists**: You MUST delete it first using `aliyun eas delete-service --cluster-id --service-name --user-agent AlibabaCloud-Agent-Skills`, wait for deletion to complete, then recreate following ALL steps (ListImages, describe-machine-spec, create-service, describe-service). NEVER just report the existing service status — you MUST always go through the full deployment workflow. ### Step 6.4: Wait for Service Ready ```bash for i in $(seq 1 6); do STATUS=$(aliyun eas describe-service --cluster-id cn-hangzhou --service-name \ --user-agent AlibabaCloud-Agent-Skills | jq -r '.Status') case $STATUS in Running) echo "✅ Service ready"; break ;; Failed) echo "❌ Service startup failed"; break ;; *) echo "⏳ Status: $STATUS ($((i*30))s/180s)"; sleep 30 ;; esac done ``` ### Step 6.5: Display Deployment Result **MUST include InternetEndpoint and IntranetEndpoint from describe-service response.** ``` ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Deployment Result Service Name: mytest_006 Status: Running InternetEndpoint: http://xxx.cn-hangzhou.pai-eas.aliyuncs.com/api/predict/xxx IntranetEndpoint: http://xxx.vpc.cn-hangzhou.pai-eas.aliyuncs.com/api/predict/xxx ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ``` **⚠️ You MUST output the exact field names "InternetEndpoint" and "IntranetEndpoint" with their actual values from the describe-service API response.** ### Step 6.6: Service Invocation Example ``` ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Service Invocation Examples Endpoint: http://xxx.cn-hangzhou.pai-eas.aliyuncs.com/api/predict/mytest_006 [curl] curl http://xxx/api/predict/mytest_006 -H "Content-Type: application/json" -d '{"input": "Hello"}' [OpenAI SDK] client = OpenAI(base_url="http://xxx/v1", ...) See [Service Invocation Examples](service-invoke-examples.md) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ ``` ### Step 6.7: Failure Handling **Query events**: ```bash aliyun eas describe-service-event --cluster-id --service-name --user-agent AlibabaCloud-Agent-Skills | \ jq -r '.Events[-3:][] | "[\(.Type)] \(.Reason): \(.Message)"' ``` **Common errors**: | Error Type | Cause | Solution | |-----------|-------|---------| | ImagePullBackOff | Image pull failed | Check image address and permissions | | CrashLoopBackOff | Container startup failed | Check startup command and model path | | Instance crashed | Chip type mismatch | Image chip type must match instance type (GPU image → GPU instance) | | InsufficientResources | Resource shortage | Choose different instance type or resource group | | ModelNotFound | Wrong model path | Check OSS mount path | FILE:references/image-categories.md # Official Image Categories > **Usage**: After user selects a category, call the API to query images in that category. ## Category List ``` | # | Category | Description | Frameworks | |---|-------------------|-------------------------------|----------------------------------| | 1 | LLM Inference | Large language model inference | vLLM, SGLang | | 2 | Image Generation | Text-to-image, image processing| ComfyUI, Stable Diffusion, Kohya | | 3 | Speech Synthesis | TTS voice synthesis | CosyVoice | | 4 | RAG | Retrieval-augmented generation | PaiRag | | 5 | General Inference | General deep learning inference| PyTorch, Triton, TF Serving | | 6 | Tools | General tool images | OpenClaw, CoPaw, Python | ``` ## Category Details ### 1. LLM Inference **Flow**: Let user select framework first, then query **Framework selection**: ``` | # | Framework | Description | |---|-----------|------------------------| | 1 | vLLM | High-performance LLM inference | | 2 | SGLang | Efficient LLM serving framework | ``` **Query commands** (call once after user selects framework): ```bash # vLLM - fuzzy search by Name aliyun aiworkspace list-images --verbose true --name vllm \ --labels 'system.official=true,system.supported.eas=true' \ --page-size 50 --user-agent AlibabaCloud-Agent-Skills | jq -r '.Images[] | "\(.ImageId)\t\(.Name)\t\(.Labels[] | select(.Key == "system.chipType") | .Value)"' # SGLang - fuzzy search by Name aliyun aiworkspace list-images --verbose true --name sglang \ --labels 'system.official=true,system.supported.eas=true' \ --page-size 50 --user-agent AlibabaCloud-Agent-Skills | jq -r '.Images[] | "\(.ImageId)\t\(.Name)\t\(.Labels[] | select(.Key == "system.chipType") | .Value)"' ``` **Image info**: | Framework | Versions | Chips | |-----------|----------|-------| | vLLM | 0.7.x - 0.14.0 | GPU, PPU | | SGLang | 0.5.2 - 0.5.8 | GPU, PPU | ### 2. Image Generation **ComfyUI**: Node-based image generation workflow - Versions: 2.1, 2.2 - Chip: GPU **Stable Diffusion WebUI**: Classic SD interface - Versions: 4.1, 4.2 - Chip: GPU **Kohya**: Model training tool - Versions: 2.2, 25.0.3 - Chip: GPU **EasyAnimate**: Video generation - Versions: 1.1.4, 1.1.5 **Query command**: ```bash # ComfyUI - fuzzy search by Name aliyun aiworkspace list-images --verbose true --name comfyui \ --labels 'system.official=true,system.supported.eas=true' \ --page-size 50 --user-agent AlibabaCloud-Agent-Skills | jq -r '.Images[] | "\(.ImageId)\t\(.Name)"' ``` ### 3. Speech Synthesis **CosyVoice**: Alibaba speech synthesis - Versions: 0.1.5, 2.2.6, 3.0.6 - Components: webui, backend, frontend - Chips: GPU, PPU **Query command**: ```bash aliyun aiworkspace list-images --verbose true \ --labels 'system.official=true,system.supported.eas=true' \ --page-size 100 | jq '.Images[] | select(.Name | test("cosyvoice"; "i"))' ``` ### 4. RAG **PaiRag**: PAI RAG framework - Versions: 0.3.5, 0.4.3 - Chips: CPU, PPU **Query command**: ```bash # PaiRag - fuzzy search by Name aliyun aiworkspace list-images --verbose true --name pairag \ --labels 'system.official=true,system.supported.eas=true' \ --page-size 50 --user-agent AlibabaCloud-Agent-Skills | jq -r '.Images[] | "\(.ImageId)\t\(.Name)"' ``` ### 5. General Inference **PyTorch**: General deep learning framework - Versions: 2.2, 2.3.1, 2.5.1, 2.7.1 - Chips: GPU, CPU, PPU **Triton Server**: NVIDIA inference server - Versions: 21.09 - 25.03 - Chip: GPU **TensorFlow Serving**: TF model serving - Versions: 1.15.0, 2.11.1, 2.14.1, 2.17.1, 2.18.1 **Query command**: ```bash # PyTorch - fuzzy search by Name aliyun aiworkspace list-images --verbose true --name pytorch \ --labels 'system.official=true,system.supported.eas=true' \ --page-size 50 --user-agent AlibabaCloud-Agent-Skills | jq -r '.Images[] | "\(.ImageId)\t\(.Name)"' ``` ### 6. Tools **OpenClaw**: PAI tool image - Version: 2026.3.13 - Chip: CPU **CoPaw**: CoPaw tool image - Version: v0.0.7 - Chip: CPU **Query command**: ```bash # OpenClaw - fuzzy search by Name aliyun aiworkspace list-images --verbose true --name openclaw \ --labels 'system.official=true,system.supported.eas=true' \ --page-size 50 --user-agent AlibabaCloud-Agent-Skills | jq -r '.Images[] | "\(.ImageId)\t\(.Name)"' ``` --- ## Query Parameter Reference **Labels filter syntax**: ```bash # Multiple labels separated by comma, format: Key=Value --labels 'system.official=true,system.supported.eas=true' # Fuzzy search image name via Name parameter --name vllm ``` **Common labels**: | Label Key | Description | |-----------|-------------| | `system.official` | `true` = official image | | `system.supported.eas` | `true` = supports EAS deployment | | `system.chipType` | `GPU`/`CPU`/`PPU`/`XPU` = chip type | | `system.framework.xxx` | Framework type label, value = version | FILE:references/model-image-matching.md # Model-Image Matching Guide > **Important**: Before selecting an image, confirm your model type matches the image type. Mismatch will cause deployment failure! ## Quick Matching Table | Model Type | Recommended Image Category | Specific Image | Model Format | |-----------|--------------------------|---------------|-------------| | **LLM** (Qwen, Llama, Mistral, Baichuan, etc.) | LLM Inference | vLLM, SGLang | HuggingFace (safetensors) | | **LLM** (quantized) | LLM Inference | llama.cpp | GGUF | | **Image Generation** (Stable Diffusion, SDXL) | Image Generation | ComfyUI, SD WebUI | .safetensors, .ckpt | | **Speech Synthesis** (TTS) | Speech Synthesis | CosyVoice | Model-specific format | | **RAG Applications** | RAG | PaiRag | - | | **Custom Inference** | General Inference | PyTorch, Triton | .pt, .pth, .onnx | ## Common Errors ### ❌ Wrong Example ``` Model: Qwen3.5-0.8B (LLM) Image: ComfyUI (Image Generation) Result: Deployment failed, image does not support LLM inference ``` ### ✅ Correct Example ``` Model: Qwen3.5-0.8B (LLM) Image: vLLM or SGLang Result: Deployment successful ``` ## Detailed Matching Rules ### 1. LLM Models **HuggingFace Format** (.safetensors, model.bin): ``` Image: vLLM, SGLang Model directory structure: ├── config.json ├── model.safetensors (or model.bin) ├── tokenizer.json ├── tokenizer_config.json └── vocab.json ``` **GGUF Format**: ``` Image: llama.cpp Model file: *.gguf ``` **Note**: Do not use vLLM for GGUF models; do not use llama.cpp for HuggingFace format. ### 2. Image Generation Models **Stable Diffusion Series**: ``` Image: ComfyUI, Stable Diffusion WebUI Model format: .safetensors, .ckpt Storage path: /models/checkpoints/ or /models/stable-diffusion/ ``` **Note**: LLM models cannot be used with image generation images. ### 3. Speech Synthesis Models **CosyVoice**: ``` Image: CosyVoice Requires: Pre-trained model files ``` ## Model Download Sources | Source | Format | Description | |--------|--------|-------------| | HuggingFace | safetensors | Most common, natively supported by vLLM/SGLang | | ModelScope | safetensors | Faster access in China, HuggingFace compatible | | HuggingFace GGUF | GGUF | Quantized models, requires llama.cpp | ## How to Determine Model Type 1. **Check file extension**: - `.safetensors` + `config.json` → HuggingFace LLM - `.gguf` → GGUF LLM - `.ckpt` / `.safetensors` (no config.json) → Image model 2. **Check model name**: - Contains `llama`, `qwen`, `mistral`, `baichuan` → LLM - Contains `sd`, `stable-diffusion`, `sdxl` → Image generation - Contains `cosyvoice`, `tts` → Speech synthesis 3. **Check source page**: - HuggingFace model page indicates model type ## Chip Type Compatibility (Important) > **⚠️ Image chip type must match instance type, otherwise deployment fails!** ### Image Chip Types Images use `system.chipType` label to indicate supported chips: | Chip Type | Description | Instance Type Prefix | |-----------|-------------|---------------------| | **GPU** | NVIDIA GPU | `ecs.gn`, `ecs.gn6`, `ecs.gn7`, `ecs.gn8` | | **CPU** | CPU only | Non-GPU instances | | **PPU** | Alibaba Hanguang chip | `ecs.ebmppu` | | **XPU** | Alibaba XPU | `ecs.egs` | ### Compatibility Matrix | Image Chip Type | Compatible Instance Types | |----------------|--------------------------| | GPU | gn6i, gn6v, gn7, gn6e, gn8 (NVIDIA GPU instances) | | CPU | Any non-GPU instance | | PPU | ebmppu (Hanguang instances) | | XPU | egs (XPU instances) | ### Query Image Chip Type ```bash aliyun aiworkspace get-image --image-id --user-agent AlibabaCloud-Agent-Skills | \ jq -r '.Labels[] | select(.Key == "system.chipType") | .Value' ``` ### Query Instance Type GPU Info ```bash aliyun eas describe-machine-spec --region --user-agent AlibabaCloud-Agent-Skills | \ jq -r '.InstanceMetas[] | select(.InstanceType == "") | {InstanceType, GPUAmount, GPU}' ``` ### Common Errors ``` ❌ Wrong: Image: vllm:0.14.0-xpu (XPU chip) Instance: ecs.gn7-c12g1.12xlarge (GPU instance) Result: Instance crashed - chip type mismatch ✅ Correct: Image: vllm:0.14.0-gpu (GPU chip) Instance: ecs.gn7-c12g1.12xlarge (GPU instance) Result: Deployment successful ``` --- ## Pre-deployment Checklist ``` □ Confirm model type (LLM / Image / Speech / Custom) □ Select matching image category □ Confirm model format is compatible with image □ Confirm image chip type matches instance type □ Confirm model file completeness (config.json, tokenizer, etc.) □ Confirm model path is mounted correctly ``` FILE:references/network-config.md # Network Configuration Rules **Table of Contents** - [Network Config Requirements](#network-config-requirements) - [Consistency Requirements](#consistency-requirements) - [ALB Dedicated Gateway](#alb-dedicated-gateway) - [NLB Configuration](#nlb-configuration) - [Shared Gateway](#shared-gateway) ## Network Config Requirements | Gateway Type | `cloud.networking` | Description | |-------------|-------------------|-------------| | **Shared Gateway** | ❌ Not required | For testing, not recommended for production, no VPC config needed | | **ALB Dedicated Gateway** | ✅ **Required** | VPC/VSwitch obtained from gateway | | **NLB** | ✅ **Required** | VPC/VSwitch must be consistent with NLB | ## Consistency Requirements ``` When deploying with ALB/NLB: ├── cloud.networking.vpc_id ← Must match gateway/NLB VPC ├── cloud.networking.vswitch_id ← Must match gateway/NLB VSwitch └── cloud.networking.security_group_id ← Must be in the same VPC as VSwitch ⚠️ Important: VPC, VSwitch, and security group must all be under the same VPC! ``` ## ALB Dedicated Gateway ### Config Flow 1. Select gateway (from `list-gateway`) 2. Call `DescribeGateway` to get VPC/VSwitch 3. Select security group under the gateway VPC ### Config Example ```json { "networking": { "gateway": "gw-abc123" }, "cloud": { "networking": { "vpc_id": "{from gateway}", "vswitch_id": "{from gateway, comma-separated if multiple}", "security_group_id": "{user selected}" } } } ``` ### API Calls ```bash # List gateways aliyun eas list-gateway --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills # Get gateway details (includes VPC and VSwitch info) aliyun eas describe-gateway --cluster-id cn-hangzhou --gateway-id gw-xxx --user-agent AlibabaCloud-Agent-Skills # Key fields in response: # .LoadBalancerList[0].VpcId → VPC ID # .LoadBalancerList[0].VSwitchIds → VSwitch list ``` **Correct jq command to get VPC and comma-separated VSwitch ID**: ```bash aliyun eas describe-gateway --cluster-id cn-hangzhou --gateway-id gw-xxx --user-agent AlibabaCloud-Agent-Skills | \ jq '{ vpc_id: .LoadBalancerList[0].VpcId, vswitch_id: (.LoadBalancerList[0].VSwitchIds | join(",")) }' ``` **Response example**: ```json { "vpc_id": "vpc-bp13kiflgde6v9dc9smc8", "vswitch_id": "vsw-bp1bhmnwqdh1ta9z9klms,vsw-bp1lz95xtmjiwqcpq31ng" } ``` **Query security groups** (after getting VPC): ```bash aliyun ecs describe-security-groups --biz-region-id cn-hangzhou --vpc-id {gateway_vpc} --user-agent AlibabaCloud-Agent-Skills | \ jq '.SecurityGroups[] | "\(.SecurityGroupId)\t\(.SecurityGroupName)"' ``` --- ## NLB Configuration ### Two Modes | Mode | ID | Description | |------|-----|-------------| | **System NLB** | `"default"` | System auto-creates, lifecycle follows service (recommended) | | **Custom NLB** | Actual NLB ID | Associate with user's existing NLB instance | ### System NLB (Recommended) System automatically creates an NLB instance under your account. The NLB lifecycle follows the service. **Config example**: ```json { "networking": { "nlb": [{ "id": "default", "listener_port": 9090, "netType": "intranet" }] }, "cloud": { "networking": { "vpc_id": "vpc-xxx", "vswitch_id": "vsw-zone-a,vsw-zone-b", "security_group_id": "sg-xxx" } } } ``` **Parameter description**: | Parameter | Description | |-----------|-------------| | `id` | Fixed as `"default"` | | `listener_port` | Listener port (cannot be 8080) | | `netType` | `intranet` or `internet` | | `vswitch_id` | **Comma-separated, must include at least 2 VSwitches across different zones** (e.g. `"vsw-xxx-a,vsw-xxx-b"`) | **Multi-port config supported**: ```json { "networking": { "nlb": [ {"id": "default", "listener_port": 9090, "netType": "intranet"}, {"id": "default", "listener_port": 9091, "netType": "internet"} ] } } ``` **Config flow**: 1. Select "System NLB" 2. Configure VPC info (reuse high-speed direct connect VPC/VSwitch) 3. Select network type (intranet/internet/both) 4. Configure listener port **⚠️ Important rules**: - `vswitch_id` must be **comma-separated with at least 2 VSwitches across different availability zones** (e.g. `"vsw-zone-a,vsw-zone-b"`) - Port 8080 is reserved by EAS engine, cannot be used - VPC, VSwitch, and security group must all be under the same VPC - System NLB lifecycle follows the service ### Custom NLB Associate with user's existing NLB instance. **Config example**: ```json { "networking": { "nlb": [{ "id": "nlb-fpj3530zrbt7x5zkhk", "listener_port": 9090 }] }, "cloud": { "networking": { "vpc_id": "vpc-xxx", "vswitch_id": "vsw-zone-a,vsw-zone-b", "security_group_id": "sg-xxx" } } } ``` **Requirements**: - NLB must be in the same VPC as the service - `vswitch_id` must be comma-separated with at least 2 VSwitches across different zones - Port 8080 cannot be used - Port must not conflict with existing NLB listeners - Custom NLB and system NLB are mutually exclusive (modifying will disassociate the old one) ### NLB API Calls ```bash # Query existing NLBs aliyun nlb list-load-balancers --biz-region-id cn-hangzhou --user-agent AlibabaCloud-Agent-Skills # Get NLB details (includes VPC and VSwitch info) aliyun nlb get-load-balancer-attribute --load-balancer-id nlb-xxx --user-agent AlibabaCloud-Agent-Skills | \ jq '{vpc_id: .VpcId, vswitch_id: ([.ZoneMappings[].VswitchId] | join(","))}' ``` **Response example**: ```json { "vpc_id": "vpc-xxx", "vswitch_id": "vsw-xxx-a,vsw-xxx-b" } ``` **Query security groups** (after getting NLB VPC): ```bash aliyun ecs describe-security-groups --biz-region-id cn-hangzhou --vpc-id {nlb_vpc} --user-agent AlibabaCloud-Agent-Skills | \ jq '.SecurityGroups[] | "\(.SecurityGroupId)\t\(.SecurityGroupName)"' ``` **⚠️ Important rules**: - VPC, VSwitch, and security group must all be under the NLB's VPC - Security group must be in the **same VPC** as NLB - Port 8080 is reserved by EAS engine, cannot be used ### Access URL ``` Access URL: :/api/predict/ Get domain: https://nlb.console.aliyun.com/ ``` ## Shared Gateway No `cloud.networking` config needed, uses default mode. ```json { "metadata": { "name": "my-service" }, "containers": [...] // No networking or cloud.networking fields needed } ``` FILE:references/ram-policies.md # RAM Policies Minimum RAM permissions required for PAI-EAS service deployment. ## Minimum Permission Policy ```json { "Version": "1", "Statement": [ { "Effect": "Allow", "Action": [ "eas:CreateService", "eas:DescribeService", "eas:ListServices", "eas:DescribeServiceEndpoints", "eas:DescribeServiceEvent", "eas:DescribeMachineSpec", "eas:ListResources", "eas:ListGateway", "eas:DescribeGateway" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "aiworkspace:ListWorkspaces", "aiworkspace:ListImages", "aiworkspace:GetImage", "aiworkspace:ListDatasets" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "oss:ListBuckets", "oss:GetBucketLocation", "oss:ListObjects" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "vpc:DescribeVpcs", "vpc:DescribeVSwitches" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "ecs:DescribeSecurityGroups" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "nlb:ListLoadBalancers" ], "Resource": "*" } ] } ``` ## Permission Details ### EAS Service Permissions | Action | Description | Use Case | |--------|-------------|----------| | `eas:CreateService` | Create service | Deploy new service | | `eas:DescribeService` | Query service details | View status, get endpoints | | `eas:ListServices` | List services | View existing services | | `eas:DescribeServiceEndpoints` | Query service endpoints | Get access URL | | `eas:DescribeServiceEvent` | Query service events | Diagnose issues | | `eas:DescribeMachineSpec` | Query machine specs | Select instance type | | `eas:ListResources` | List resource groups | Query dedicated resources | | `eas:ListGateway` | List gateways | Query available gateways | | `eas:DescribeGateway` | Query gateway details | Get gateway VPC info | ### AIWorkSpace Permissions | Action | Description | Use Case | |--------|-------------|----------| | `aiworkspace:ListWorkspaces` | List workspaces | Select workspace | | `aiworkspace:ListImages` | List images | Query available images | | `aiworkspace:GetImage` | Get image details | View image config | | `aiworkspace:ListDatasets` | List datasets | Select dataset mount | ### OSS Permissions | Action | Description | Use Case | |--------|-------------|----------| | `oss:ListBuckets` | List buckets | Select OSS storage | | `oss:GetBucketLocation` | Get bucket region | Verify cross-region access | | `oss:ListObjects` | List objects | Browse model files | ### VPC Permissions | Action | Description | Use Case | |--------|-------------|----------| | `vpc:DescribeVpcs` | Query VPCs | Network config | | `vpc:DescribeVSwitches` | Query VSwitches | Network config | ### ECS Permissions | Action | Description | Use Case | |--------|-------------|----------| | `ecs:DescribeSecurityGroups` | Query security groups | Network config | ### NLB Permissions | Action | Description | Use Case | |--------|-------------|----------| | `nlb:ListLoadBalancers` | List load balancers | NLB network config | ## Extended Operations Permissions (Optional) For more complete PAI-EAS operations (update, delete services, etc.), add these permissions: ```json { "Version": "1", "Statement": [ { "Effect": "Allow", "Action": [ "eas:UpdateService", "eas:DeleteService", "eas:ScaleService", "eas:DescribeServiceLog", "eas:ResumeService", "eas:SuspendService" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "aiworkspace:ListImages", "aiworkspace:ListImageLabels", "aiworkspace:GetImage", "aiworkspace:ListWorkspaces", "aiworkspace:ListDatasets" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "vpc:DescribeVpcs", "vpc:DescribeVSwitches", "ecs:DescribeSecurityGroups" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "oss:ListBuckets", "oss:GetBucketLocation", "oss:ListObjects" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "nlb:ListLoadBalancers", "nlb:GetLoadBalancerAttribute" ], "Resource": "*" } ] } ``` ## Permission Check Check current user permissions: ```bash aliyun ram get-login-profile --user-name ``` Or check user authorization policies via the RAM Console. FILE:references/related-apis.md # Related API List All APIs and CLI commands involved in PAI-EAS service deployment. ## EAS Service APIs | API | CLI Command | Description | |-----|------------|-------------| | CreateService | `aliyun eas create-service` | Create service | | DescribeService | `aliyun eas describe-service` | Query service details | | ListServices | `aliyun eas list-services` | List services | | DescribeServiceEndpoints | `aliyun eas describe-service-endpoints` | Query service endpoint | | DescribeServiceEvent | `aliyun eas describe-service-event` | Query service events | | DescribeMachineSpec | `aliyun eas describe-machine-spec` | Query machine specs | | ListResources | `aliyun eas list-resources` | List resource groups | | ListGateway | `aliyun eas list-gateway` | List gateways | | DescribeGateway | `aliyun eas describe-gateway` | Query gateway details | ## AIWorkSpace APIs | API | CLI Command | Description | |-----|------------|-------------| | ListWorkspaces | `aliyun aiworkspace list-workspaces` | List workspaces | | ListImages | `aliyun aiworkspace list-images` | List images | | GetImage | `aliyun aiworkspace get-image` | Get image details | | ListDatasets | `aliyun aiworkspace list-datasets` | List datasets | ## OSS APIs | API | CLI Command | Description | |-----|------------|-------------| | ListBuckets | `ossutil ls` | List buckets | | GetBucketLocation | `ossutil stat` | Get bucket region | | ListObjects | `ossutil ls oss://bucket/` | List objects | ## VPC APIs | API | CLI Command | Description | |-----|------------|-------------| | DescribeVpcs | `aliyun vpc describe-vpcs` | Query VPCs | | DescribeVSwitches | `aliyun vpc describe-vswitches` | Query VSwitches | ## ECS APIs | API | CLI Command | Description | |-----|------------|-------------| | DescribeSecurityGroups | `aliyun ecs describe-security-groups` | Query security groups | ## NLB APIs | API | CLI Command | Description | |-----|------------|-------------| | ListLoadBalancers | `aliyun nlb list-load-balancers` | List load balancers | --- ## CLI Command Details ### EAS Service Operations ```bash # ⚠️ Do NOT use file:// prefix, use $(cat) to read file content aliyun eas create-service --region cn-hangzhou --body "$(cat service.json)" --user-agent AlibabaCloud-Agent-Skills aliyun eas describe-service --cluster-id cn-hangzhou --service-name my-service --user-agent AlibabaCloud-Agent-Skills aliyun eas list-services --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills aliyun eas describe-machine-spec --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills aliyun eas list-resources --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills aliyun eas list-gateway --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills aliyun eas describe-gateway --cluster-id cn-hangzhou --gateway-id gw-xxx --user-agent AlibabaCloud-Agent-Skills ``` ### AIWorkSpace Operations ```bash aliyun aiworkspace list-workspaces --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills aliyun aiworkspace list-images --verbose true --labels 'system.official=true,system.supported.eas=true' --page-size 50 --user-agent AlibabaCloud-Agent-Skills aliyun aiworkspace get-image --image-id image-xxx --user-agent AlibabaCloud-Agent-Skills aliyun aiworkspace list-datasets --region cn-hangzhou --user-agent AlibabaCloud-Agent-Skills ``` ### OSS Operations ```bash ossutil ls ossutil ls oss://bucket-name/path/ ``` ### VPC Operations ```bash aliyun vpc describe-vpcs --biz-region-id cn-hangzhou --user-agent AlibabaCloud-Agent-Skills aliyun vpc describe-vswitches --biz-region-id cn-hangzhou --user-agent AlibabaCloud-Agent-Skills ``` ### ECS Operations ```bash aliyun ecs describe-security-groups --biz-region-id cn-hangzhou --user-agent AlibabaCloud-Agent-Skills ``` ### NLB Operations ```bash aliyun nlb list-load-balancers --biz-region-id cn-hangzhou --user-agent AlibabaCloud-Agent-Skills ``` --- ## SDK Call Metadata If you need to use Python Common SDK instead of CLI, here are the API metadata: | Service | API | popCode | popVersion | |---------|-----|---------|------------| | EAS | CreateService | eas | 2021-07-01 | | EAS | DescribeService | eas | 2021-07-01 | | EAS | ListServices | eas | 2021-07-01 | | EAS | DescribeServiceEndpoint | eas | 2021-07-01 | | EAS | ListServiceEvents | eas | 2021-07-01 | | EAS | DescribeMachineSpec | eas | 2021-07-01 | | EAS | ListResources | eas | 2021-07-01 | | EAS | ListGateway | eas | 2021-07-01 | | EAS | DescribeGateway | eas | 2021-07-01 | | NLB | ListLoadBalancers | nlb | 2022-04-30 | | AIWorkSpace | ListWorkspaces | aiworkspace | 2020-04-20 | | AIWorkSpace | ListImages | aiworkspace | 2020-04-20 | | AIWorkSpace | GetImage | aiworkspace | 2020-04-20 | | AIWorkSpace | ListDatasets | aiworkspace | 2020-04-20 | | OSS | ListBuckets | oss | 2019-05-17 | | OSS | GetBucketLocation | oss | 2019-05-17 | | OSS | ListObjects | oss | 2019-05-17 | | VPC | DescribeVpcs | vpc | 2016-04-28 | | VPC | DescribeVSwitches | vpc | 2016-04-28 | | ECS | DescribeSecurityGroups | ecs | 2014-05-26 | FILE:references/service-features.md # Service Features Configuration Guide Detailed configuration guide for health check, rolling update, GRPC, and autoscaling. **Table of Contents** - [Feature Selection Interaction](#feature-selection-interaction) - [Health Check](#1-health-check) - [Rolling Update](#2-rolling-update) - [GRPC](#3-grpc) - [Autoscaling](#4-autoscaling) - [Combined Feature Example](#5-combined-feature-example) --- ## Feature Selection Interaction ``` | # | Feature | Default Status | |---|---------|---------------| | 1 | Health Check | ✅ Enabled | | 2 | Rolling Update | ✅ Enabled | | 3 | GRPC | ❌ Disabled | | 4 | Autoscaling | ❌ Disabled | Select features to enable (multi-select with comma, e.g. 1,2,4), or press Enter for defaults: ``` **User input handling**: - Enter numbers (e.g. `1,3,4`) → Enable corresponding features - Press Enter → Use defaults (health check + rolling update) - Enter `none` → Disable all features --- ## 1. Health Check Health check monitors whether the service is running normally, including startup and liveness probes. ### Startup Probe (startup_check) Checks whether the service is ready after startup. ### Parameter Description | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | http_get.path | string | `/` | HTTP check path | | http_get.port | int | 8000 | Check port | | initial_delay_seconds | int | 60 | Initial delay (how long after startup to begin checking) | | period_seconds | int | 10 | Check interval | | timeout_seconds | int | 1 | Single check timeout | | success_threshold | int | 1 | Success threshold (consecutive successes to consider healthy) | | failure_threshold | int | 30 | Failure threshold (consecutive failures to consider unhealthy) | ### JSON Config ```json { "containers": [{ "startup_check": { "http_get": { "path": "/health", "port": 8000 }, "initial_delay_seconds": 60, "period_seconds": 10, "timeout_seconds": 1, "success_threshold": 1, "failure_threshold": 30 } }] } ``` ### TCP Check ```json { "containers": [{ "startup_check": { "tcp_socket": { "port": 8000 }, "initial_delay_seconds": 60, "period_seconds": 10 } }] } ``` ### Interaction Flow ``` Configure health check parameters: Check type: 1. HTTP (recommended) 2. TCP HTTP check path [/health]: 1. Use default 2. Custom Initial delay seconds [60]: 1. Use default 2. Custom Check interval seconds [10]: 1. Use default 2. Custom Failure threshold [30]: 1. Use default 2. Custom ``` --- ## 2. Rolling Update Rolling update ensures zero-downtime service upgrades through graceful shutdown and rolling strategy. ### Parameter Description | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | termination_grace_period | int | 30 | Graceful shutdown wait time (seconds) | | max_surge | int | 1 | Max new instances during rolling | | max_unavailable | int | 0 | Max unavailable instances during rolling | | enable_sigterm | bool | true | Enable SIGTERM signal | ### JSON Config ```json { "metadata": { "rolling_strategy": { "max_surge": 1, "max_unavailable": 0 } }, "eas": { "termination_grace_period": 30 }, "rpc": { "enable_sigterm": true } } ``` ### Parameter Details **termination_grace_period**: - How long to wait after receiving stop signal before force termination - Recommended: 30-120 seconds - Should be longer than time to process current requests **max_surge**: - How many new instances can be started during rolling update - Higher value = faster update, but more resource consumption - Recommended: 1-2 **max_unavailable**: - Max instances that can be unavailable during rolling update - Set to 0 for zero-downtime update - Recommended: 0 (production) ### Interaction Flow ``` Configure rolling update parameters: Graceful shutdown wait (seconds) [30]: 1. Use default 2. Custom Max new instances [1]: 1. Use default 2. Custom Max unavailable instances [0]: 1. Use default 2. Custom ``` --- ## 3. GRPC Enable GRPC protocol support. Service will support both HTTP and GRPC calls. ### JSON Config ```json { "metadata": { "enable_grpc": true } } ``` ### Description - Once enabled, service listens for both HTTP and GRPC requests - GRPC port defaults to same as HTTP port - Suitable for high-performance RPC scenarios ### Interaction Flow ``` GRPC protocol enabled. Service will support both HTTP and GRPC calls. ``` --- ## 4. Autoscaling Auto-adjust service replicas based on load. ### ⚠️ Important: Field Naming Convention **EAS API uses camelCase**, ensure correct field names: | ✅ Correct Field Name | ❌ Wrong Field Name | Description | |----------------------|---------------------|-------------| | `min` | ~~`min_replica`~~ | Min replicas | | `max` | ~~`max_replica`~~ | Max replicas | | `scaleStrategies` | ~~`scale_strategies`~~ | Scaling strategy array | | `metricName` | ~~`metric_name`~~ | Metric name | **Using wrong field names will cause config to be ignored, falling back to defaults (min=0, max=0)!** ### Parameter Description | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | min | int | 1 | Min replicas | | max | int | 10 | Max replicas | | metricName | string | qps | Scaling metric: qps, cpu, gpu, memory | | threshold | int | 100 | Scaling threshold | ### JSON Config ```json { "autoscaler": { "min": 1, "max": 10, "scaleStrategies": [ {"metricName": "qps", "threshold": 100} ] } } ``` ### Scaling Metrics | Metric | Description | Recommended Threshold | |--------|-------------|----------------------| | qps | Queries per second | 50-200 | | cpu | CPU utilization (%) | 70-85 | | gpu | GPU utilization (%) | 70-85 | | memory | Memory utilization (%) | 70-85 | ### Interaction Flow ``` Configure autoscaling parameters: Min replicas (min) [1]: 1. Use default 2. Custom Max replicas (max) [10]: 1. Use default 2. Custom Scaling metric (metricName): 1. qps (recommended) 2. cpu 3. gpu 4. memory Select metric (enter number): Scaling threshold [100]: 1. Use default 2. Custom ``` ### Config Examples **Example 1: QPS-based autoscaling** ```json { "autoscaler": { "min": 1, "max": 5, "scaleStrategies": [ {"metricName": "qps", "threshold": 50} ] } } ``` **Example 2: Multi-metric autoscaling** ```json { "autoscaler": { "min": 2, "max": 10, "scaleStrategies": [ {"metricName": "qps", "threshold": 100}, {"metricName": "cpu", "threshold": 80} ] } } ``` ### ⚠️ Common Mistakes **Mistake 1: Using snake_case naming** ```json // ❌ Wrong - fields will be ignored { "autoscaler": { "min_replica": 1, "max_replica": 10, "scale_strategies": [ {"metric_name": "qps", "threshold": 100} ] } } // ✅ Correct - use camelCase { "autoscaler": { "min": 1, "max": 10, "scaleStrategies": [ {"metricName": "qps", "threshold": 100} ] } } ``` **Mistake 2: Setting min and max to same value** ```json // ❌ Wrong - cannot scale with equal min and max { "autoscaler": { "min": 5, "max": 5 } } // ✅ Correct - max > min { "autoscaler": { "min": 1, "max": 5 } } ``` --- ## 5. Combined Feature Example Enable health check, rolling update, and GRPC together: ```json { "metadata": { "name": "my-service", "instance": 2, "enable_grpc": true, "rolling_strategy": { "max_surge": 1, "max_unavailable": 0 } }, "eas": { "termination_grace_period": 30 }, "rpc": { "enable_sigterm": true }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.14.0-gpu", "port": 8000, "startup_check": { "http_get": {"path": "/health", "port": 8000}, "initial_delay_seconds": 60, "period_seconds": 10, "failure_threshold": 30 } }], "autoscaler": { "min": 1, "max": 5, "scaleStrategies": [ {"metricName": "qps", "threshold": 50} ] } } ``` --- *References*: - [Health Check Docs](https://help.aliyun.com/zh/pai/user-guide/advanced-configuration-health-check) - [Rolling Update Docs](https://help.aliyun.com/zh/pai/user-guide/scrolling-updates-with-graceful-exit) - [Autoscaling Docs](https://help.aliyun.com/zh/pai/user-guide/autoscaling) *Last updated*: 2025-03-21 - Fixed autoscaler field naming FILE:references/service-invoke-examples.md # Service Invocation Examples PAI-EAS service invocation examples. --- ## Endpoint Formats | Type | Format | Description | |------|--------|-------------| | Public | `http://{service_name}.cn-hangzhou.pai-eas.aliyuncs.com/api/predict/{service_name}` | Public internet access | | Internal | `http://{service_name}.gw-xxx-vpc.cn-hangzhou.pai-eas.aliyuncs.com/` | VPC internal access (faster and more stable) | --- ## Invocation Methods ### Method 1: HTTP Invocation (curl) ```bash curl http://xxx.cn-hangzhou.pai-eas.aliyuncs.com/api/predict/my_service \ -H "Content-Type: application/json" \ -d '{"input": "Hello, please introduce yourself"}' ``` ### Method 2: OpenAI SDK Compatible Invocation For vLLM, SGLang and other OpenAI API compatible images. ```python from openai import OpenAI client = OpenAI( base_url="http://xxx.cn-hangzhou.pai-eas.aliyuncs.com/api/predict/my_service/v1", api_key="xxx" # Fill in if Token auth is configured, otherwise omit ) response = client.chat.completions.create( model="/model_dir", # Model mount path messages=[{"role": "user", "content": "Hello, please introduce yourself"}], max_tokens=100 ) print(response.choices[0].message.content) ``` ### Method 3: Python requests ```python import requests url = "http://xxx.cn-hangzhou.pai-eas.aliyuncs.com/api/predict/my_service" headers = {"Content-Type": "application/json"} data = {"input": "Hello, please introduce yourself"} response = requests.post(url, json=data, headers=headers) print(response.json()) ``` ### Method 4: Internal Network Invocation Use internal endpoint within the same VPC for faster and more stable access: ```bash curl http://xxx.gw-xxx-vpc.cn-hangzhou.pai-eas.aliyuncs.com/api/predict/my_service \ -H "Content-Type: application/json" \ -d '{"input": "Hello"}' ``` --- ## Image-specific Endpoints | Image Type | Endpoint | Description | |-----------|----------|-------------| | vLLM | `/v1/chat/completions` | OpenAI compatible API | | SGLang | `/v1/chat/completions` | OpenAI compatible API | | ComfyUI | `/prompt` | See image docs | | SD WebUI | `/sdapi/v1/txt2img` | See image docs | | Custom | `/` | Depends on implementation | --- ## Authentication | Gateway Type | Authentication Method | |-------------|----------------------| | Shared Gateway | No auth by default, publicly accessible | | ALB/NLB | Token auth can be configured, add `Authorization: Bearer ` header | --- ## vLLM Complete Example ```python from openai import OpenAI # Service endpoint base_url = "http://my-service.cn-hangzhou.pai-eas.aliyuncs.com/api/predict/my_service/v1" client = OpenAI( base_url=base_url, api_key="your-token" # Optional ) # Chat completion response = client.chat.completions.create( model="/model_dir", # Model mount path messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Introduce Alibaba Cloud PAI-EAS"} ], temperature=0.7, max_tokens=500 ) print(response.choices[0].message.content) ``` --- ## FAQ **Q: Getting 404 response?** A: Check service name and endpoint are correct, ensure service status is Running. **Q: Getting 401 response?** A: Check if Token auth is configured, add `Authorization: Bearer ` header. **Q: Cannot access internal endpoint?** A: Ensure the caller and service are in the same VPC. FILE:references/storage-mount.md # Storage Mount Configuration Guide > Reference: https://help.aliyun.com/zh/pai/user-guide/mount-storage-to-services EAS services support multiple storage mount methods to mount model files, code files, or other config files into service instances. **⚠️ Important: OSS models MUST be mounted via storage config as local paths (e.g. `/model_dir`), then use local paths in startup commands. Never pass `oss://` URL directly to vllm/sglang commands!** **Table of Contents** - [Storage Type Selection Guide](#storage-type-selection-guide) - [Supported Storage Types](#supported-storage-types) - [OSS Mount](#oss-mount) - [Dataset Mount](#dataset-mount) - [NAS Mount](#nas-mount) - [CPFS Mount](#cpfs-mount) - [Full Config Example](#full-config-example) - [Notes](#notes) --- ## Storage Type Selection Guide | Data Type | Recommended Storage | Description | |-----------|-------------------|-------------| | Models, images, videos (primarily read) | **OSS** | Most common | | Frequent small file I/O, shared read/write across instances | NAS | General-purpose NAS | | HPC, AI training, ultra-low latency needed | Ultra-fast NAS / CPFS | High throughput | | PAI registered datasets | **Dataset** | Public AI assets | | Git repositories | Git mount | Read-only | | PAI registered code | Code config | Public AI assets | | PAI registered models | PAI models | Read-only | --- ## Supported Storage Types | Type | Status | Description | |------|--------|-------------| | OSS | ✅ Supported | Most common, mount OSS bucket | | NAS | ✅ Supported | General-purpose NAS, ultra-fast NAS | | CPFS | ✅ Supported | CPFS (requires Lingjun quota) | | Dataset | ✅ Supported | PAI dataset (OSS type only) | | Git mount | ❌ Not supported | Git repositories | | Code config | ❌ Not supported | PAI code sets | | PAI models | ❌ Not supported | PAI registered models | | Image mount | ❌ Not supported | Docker Image mount | | EmptyDir | ❌ Not supported | Local temp directory | --- ## OSS Mount ### Config Example ```json { "storage": [{ "mount_path": "/mnt/data/", "oss": { "path": "oss://bucket-name/path/", "readOnly": true } }] } ``` ### Parameter Description | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `mount_path` | string | ✅ | Target path in service instance, e.g. `/mnt/data/` | | `oss.path` | string | ✅ | OSS path, format `oss://bucket/path/` (note trailing `/`) | | `oss.readOnly` | bool | ❌ | Read-only, default false | ### Interaction Flow ``` Step 2.2: Storage Mount 1. Do you need to mount storage? | # | Option | Description | |---|--------|-------------| | 1 | Yes | Mount model files or data | | 2 | No (skip) | No storage mount | 2. Select storage type: | # | Storage Type | Description | |---|-------------|-------------| | 1 | OSS | Mount OSS bucket (most common) | | 2 | Dataset | Use PAI dataset | 3. OSS config: a. Query bucket list: ossutil ls b. Select bucket (paginated, max 10) c. List directory: ossutil ls oss://bucket-name/ d. Select model directory e. Mount path [/model_dir]: 1. Use default 2. Custom f. Permissions [read-only]: 1. Read-only (recommended) 2. Read-write ``` ### Multiple OSS Mounts ```json { "storage": [ { "mount_path": "/models", "oss": { "path": "oss://my-bucket/models/", "readOnly": true } }, { "mount_path": "/data", "oss": { "path": "oss://my-bucket/data/", "readOnly": false } } ] } ``` ### FAQ **Q: Mounted OSS but getting file not found error?** A: Usually a path issue. For example: - Mount `oss://my-bucket/` to `/mnt/data` - File in OSS: `oss://my-bucket/subfolder/myfile.txt` - Container access path: `/mnt/data/subfolder/myfile.txt` (not `/mnt/data/myfile.txt`) **Q: Can I mount OSS from a different region?** A: No. EAS cannot cross-region mount OSS. Use OSS cross-region replication to sync data to the same region. --- ## Dataset Mount Mount PAI datasets as public AI assets. **⚠️ Only OSS-type custom datasets are supported for mounting.** ### Config Example ```json { "storage": [{ "mount_path": "/mnt/dataset/", "dataset": { "id": "d-pcsah1t86bm8xxxx", "version": "v1", "read_only": true } }] } ``` ### Parameter Description | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `mount_path` | string | ✅ | Target path in service instance | | `dataset.id` | string | ✅ | Dataset ID | | `dataset.version` | string | ❌ | Dataset version, e.g. `v1` | | `dataset.read_only` | bool | ❌ | Read-only, default true | ### Query Datasets ```bash aliyun aiworkspace list-datasets --region cn-hangzhou ``` ### Create Datasets Reference: [Create and manage datasets](https://help.aliyun.com/zh/pai/user-guide/create-and-manage-datasets) --- ## NAS Mount NAS mount (general-purpose and ultra-fast NAS) only supports same-region intranet mount. Requires direct network connectivity to the NAS VSwitch. ### Config Example ```json { "storage": [{ "mount_path": "/mnt/data/", "nfs": { "path": "/", "server": "06ba74****-a****.cn-hangzhou.nas.aliyuncs.com", "readOnly": false } }] } ``` ### Parameter Description | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `mount_path` | string | ✅ | Target path in service instance, e.g. `/mnt/data/` | | `nfs.server` | string | ✅ | NAS mount point address | | `nfs.path` | string | ✅ | Source path in NAS, e.g. `/` | | `nfs.readOnly` | bool | ❌ | Read-only, default false | | `nfs.resourceGroup` | string | ❌ | File system resource group | ### Interaction Flow ``` NAS Mount Config: 1. NAS mount point: 06ba74****-a****.cn-hangzhou.nas.aliyuncs.com 2. File system path [/]: 1. Use default 2. Custom 3. Mount path [/mnt/data/]: 1. Use default 2. Custom 4. Permissions [read-only]: 1. Read-only 2. Read-write ``` ### Notes - **Same-region only**: NAS only supports same-region intranet mount - **Network config**: Requires direct network connectivity to the NAS VSwitch - **Console**: View file system ID and mount point at [NAS Console](https://nasnext.console.aliyun.com/) - **Network setup**: See [Network Configuration](https://help.aliyun.com/zh/pai/user-guide/configure-network-connectivity) --- ## CPFS Mount CPFS file system mount, suitable for HPC and AI training scenarios. ### Config Example ```json { "storage": [{ "mount_path": "/mnt/data/", "nfs": { "path": "/", "server": "cpfs-xxx.cn-hangzhou.cpfs.aliyuncs.com", "readOnly": false } }] } ``` ### Notes - **Quota requirement**: Only supported when deploying EAS services with Lingjun quota - **Same-region only**: CPFS only supports same-region intranet mount - **Network config**: Requires direct network connectivity to the CPFS VSwitch --- ## Git Mount (Not Supported) ```json { "storage": [{ "mount_path": "/mnt/data/", "git": { "repo": "https://codeup.aliyun.com/xxx/eas/aitest.git", "branch": "master", "commit": "xxx", "username": "username", "password": "password or access token" } }] } ``` --- ## EmptyDir Mount (Not Supported) Read/write to local disk during instance runtime. Content persists across instance restarts. ```json { "storage": [{ "mount_path": "/data_image", "empty_dir": {} }] } ``` ### Shared Memory Config ```json { "storage": [{ "mount_path": "/dev/shm", "empty_dir": { "medium": "memory", "size_limit": 20 } }] } ``` --- ## Full Config Example ```json { "metadata": { "name": "my-service", "instance": 1 }, "containers": [{ "image": "eas-registry-vpc.cn-hangzhou.cr.aliyuncs.com/pai-eas/vllm:0.14.0-gpu", "port": 8000 }], "storage": [ { "mount_path": "/model_dir", "oss": { "path": "oss://my-bucket/models/qwen/", "readOnly": true } }, { "mount_path": "/mnt/dataset/", "dataset": { "id": "d-xxx", "version": "v1", "read_only": true } } ] } ``` --- ## Notes 1. **Mount paths cannot overlap** 2. **Mount paths cannot be system reserved paths**: `/`, `/bin`, `/etc`, `/usr`, etc. 3. **OSS path format must be correct**: `oss://bucket/path/`, note the trailing `/` 4. **Permissions**: Model files should use `readOnly: true` to prevent accidental modification 5. **Without storage mount**: Files downloaded to the instance are stored on system disk and will be cleared on restart or update FILE:references/verification-method.md # Verification Methods How to verify PAI-EAS service deployment success. ## Deployment Verification Flow ### Step 1: Verify Service Creation ```bash aliyun eas describe-service \ --cluster-id cn-hangzhou \ --service-name \ --user-agent AlibabaCloud-Agent-Skills ``` **Expected result**: - Response JSON contains `ServiceId` - `Status` field is `Creating` or `Running` ### Step 2: Wait for Service Ready ```bash for i in $(seq 1 6); do STATUS=$(aliyun eas describe-service \ --cluster-id cn-hangzhou \ --service-name \ --user-agent AlibabaCloud-Agent-Skills | jq -r '.Status') case $STATUS in Running) echo "✅ Service ready"; break ;; Failed) echo "❌ Service startup failed"; break ;; *) echo "⏳ Status: $STATUS ($((i*30))s/180s)"; sleep 30 ;; esac done ``` **Expected result**: - Service status becomes `Running` ### Step 3: Get Service Endpoints ```bash aliyun eas describe-service \ --cluster-id cn-hangzhou \ --service-name \ --user-agent AlibabaCloud-Agent-Skills | jq '{ Status: .Status, InternetEndpoint: .InternetEndpoint, IntranetEndpoint: .IntranetEndpoint, ServiceId: .ServiceId }' ``` **Expected result**: - `InternetEndpoint` or `IntranetEndpoint` has value ### Step 4: Verify Service Accessibility **HTTP service verification**: ```bash curl -s -o /dev/null -w "%{http_code}" \ "http:///health" ``` **Expected result**: - HTTP status code 200 **vLLM service verification**: ```bash curl http:///v1/models ``` **Expected result**: - Returns model list JSON ### Step 5: View Service Events (optional) If service startup fails, check events: ```bash aliyun eas describe-service-event \ --cluster-id cn-hangzhou \ --service-name \ --user-agent AlibabaCloud-Agent-Skills ``` --- ## Config Verification ### Verify Image Config ```bash aliyun aiworkspace get-image \ --image-id \ --user-agent AlibabaCloud-Agent-Skills | jq '.EasConfig' ``` ### Verify Resource Config ```bash aliyun eas describe-machine-spec \ --cluster-id cn-hangzhou \ --user-agent AlibabaCloud-Agent-Skills | jq ".InstanceMetas[] | select(.InstanceType == \"\")" ``` ### Verify Storage Config ```bash ossutil ls oss://// ``` --- ## Service Invocation Verification ### Service Console After successful deployment, view details in PAI Console: ``` https://pai.console.aliyun.com/?regionId=&workspaceId=#/eas/serviceDetail//detail ``` **Example**: ``` https://pai.console.aliyun.com/?regionId=cn-hangzhou&workspaceId=1111#/eas/serviceDetail/qwen35_7b_prod/detail ``` ### Get Service Token Service invocation requires Token authentication. Get from console service detail page, or via API: ```bash aliyun eas describe-service \ --cluster-id \ --service-name \ --user-agent AlibabaCloud-Agent-Skills | jq -r '.Token' ``` ### Shared Gateway Invocation ```bash curl "http:///api/predict/" \ -H "Content-Type: application/json" \ -H "Authorization: " \ -d '{"input": "test"}' ``` **Example** (token redacted): ```bash curl "http://1227512831780489.cn-hangzhou.pai-eas.aliyuncs.com/api/predict/qwen35_7b_prod" \ -H "Content-Type: application/json" \ -H "Authorization: ZmYxMmMz***YWY4ZDU=" \ -d '{"input": "Hello"}' ``` ### Dedicated Gateway Invocation ```bash curl "http://.-vpc..pai-eas.aliyuncs.com/" \ -H "Content-Type: application/json" \ -H "Authorization: " \ -d '{"input": "test"}' ``` ### OpenAI Compatible Interface ```python from openai import OpenAI client = OpenAI( base_url="http:///v1", api_key="" ) response = client.chat.completions.create( model="", messages=[{"role": "user", "content": "Hello"}] ) print(response.choices[0].message.content) ``` ### vLLM Service Invocation ```bash curl "http:///v1/chat/completions" \ -H "Content-Type: application/json" \ -H "Authorization: " \ -d '{ "model": "/models", "messages": [{"role": "user", "content": "Hello"}], "max_tokens": 100 }' ``` --- ## Troubleshooting ### Service Status Check ```bash aliyun eas describe-service \ --cluster-id cn-hangzhou \ --service-name \ --user-agent AlibabaCloud-Agent-Skills | jq '{Status, Message, Reason}' ``` ### View Service Logs ```bash aliyun eas describe-service-log \ --cluster-id cn-hangzhou \ --service-name \ --user-agent AlibabaCloud-Agent-Skills ``` ### Check Resource Config - Confirm instance type is available - Confirm resource group has sufficient quota - Confirm VPC/VSwitch/security group config is correct ### Check Image Config - Confirm image URI is correct - Confirm startup command is correct - Confirm port config is correct ### Check Storage Config - Confirm OSS path exists - Confirm mount paths don't conflict - Confirm permissions are correct FILE:scripts/create-service-from-json.sh #!/bin/bash # ============================================================================== # PAI-EAS Service Deployment Script # Create PAI-EAS service using JSON configuration # # Usage: # ./create-service-from-json.sh --config service.json # ./create-service-from-json.sh --config-json '{"metadata":{"name":"xxx"...}}' # ./create-service-from-json.sh --config service.json --region cn-hangzhou # # Output: # JSON format, containing service creation result # ============================================================================== set -euo pipefail # Parameter parsing CONFIG_FILE="" CONFIG_JSON="" REGION="" WAIT_READY=true TIMEOUT=300 while [[ $# -gt 0 ]]; do case $1 in --config) CONFIG_FILE="$2" shift 2 ;; --config-json) CONFIG_JSON="$2" shift 2 ;; --region) REGION="$2" shift 2 ;; --no-wait) WAIT_READY=false shift ;; --timeout) TIMEOUT="$2" shift 2 ;; -h|--help) cat <&2 exit 1 ;; esac done # Check environment variables if [[ -z "$REGION" ]]; then REGION="${ALIBABACLOUD_REGION:-}" fi if [[ -z "$REGION" ]]; then echo '{"success":false,"error":"Please specify --region or set ALIBABACLOUD_REGION environment variable"}' exit 1 fi # Get script directory SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" # Read config if [[ -n "$CONFIG_FILE" ]]; then if [[ ! -f "$CONFIG_FILE" ]]; then echo "{\"success\":false,\"error\":\"Config file does not exist: $CONFIG_FILE\"}" exit 1 fi CONFIG=$(cat "$CONFIG_FILE") elif [[ -n "$CONFIG_JSON" ]]; then CONFIG="$CONFIG_JSON" else echo '{"success":false,"error":"Please provide --config or --config-json parameter"}' exit 1 fi # Validate config echo "Validating config..." >&2 VALIDATION=$("$SCRIPT_DIR/validate-service-config.sh" --config-json "$CONFIG" --fix 2>/dev/null) if [[ $? -ne 0 ]]; then ERROR=$(echo "$VALIDATION" | jq -r '.errors | join("; ")') echo "{\"success\":false,\"error\":\"Config validation failed: $ERROR\"}" exit 1 fi # Extract fixed config FIXED_CONFIG=$(echo "$VALIDATION" | jq -c '.fixed_config') # Extract service name SERVICE_NAME=$(echo "$FIXED_CONFIG" | jq -r '.metadata.name') echo "Service name: $SERVICE_NAME" >&2 echo "Config validation passed" >&2 # Create service echo "Creating service..." >&2 CREATE_RESULT=$(aliyun eas create-service \ --cluster-id "$REGION" \ --body "$FIXED_CONFIG" \ --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy \ 2>&1) # Check creation result if echo "$CREATE_RESULT" | jq -e '.ServiceId' > /dev/null 2>&1; then SERVICE_ID=$(echo "$CREATE_RESULT" | jq -r '.ServiceId') echo "Service created successfully: $SERVICE_ID" >&2 # Wait for service ready if [[ "$WAIT_READY" == "true" ]]; then echo "Waiting for service to be ready..." >&2 START_TIME=$(date +%s) while true; do CURRENT_TIME=$(date +%s) ELAPSED=$((CURRENT_TIME - START_TIME)) if [[ $ELAPSED -gt $TIMEOUT ]]; then echo "{\"success\":false,\"error\":\"Timeout waiting for service to be ready\",\"service_id\":\"$SERVICE_ID\",\"service_name\":\"$SERVICE_NAME\"}" exit 1 fi # Query service status SERVICE_INFO=$(aliyun eas describe-service \ --cluster-id "$REGION" \ --service-name "$SERVICE_NAME" \ --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy \ 2>&1) if ! echo "$SERVICE_INFO" | jq -e '.Status' > /dev/null 2>&1; then echo "Failed to query service status, retrying..." >&2 sleep 5 continue fi STATUS=$(echo "$SERVICE_INFO" | jq -r '.Status') MESSAGE=$(echo "$SERVICE_INFO" | jq -r '.Message // empty') case $STATUS in "Running") echo "Service is ready" >&2 ENDPOINT=$(echo "$SERVICE_INFO" | jq -r '.InternetEndpoint // empty') INTRANET_ENDPOINT=$(echo "$SERVICE_INFO" | jq -r '.IntranetEndpoint // empty') echo "{\"success\":true,\"service_id\":\"$SERVICE_ID\",\"service_name\":\"$SERVICE_NAME\",\"status\":\"Running\",\"internet_endpoint\":\"$ENDPOINT\",\"intranet_endpoint\":\"$INTRANET_ENDPOINT\"}" exit 0 ;; "Failed") echo "Service startup failed: $MESSAGE" >&2 echo "{\"success\":false,\"error\":\"Service startup failed: $MESSAGE\",\"service_id\":\"$SERVICE_ID\",\"service_name\":\"$SERVICE_NAME\"}" exit 1 ;; *) echo "Waiting for service to be ready... ($ELAPSED/$TIMEOUT sec) Status: $STATUS" >&2 sleep 5 ;; esac done else # Do not wait, return immediately echo "{\"success\":true,\"service_id\":\"$SERVICE_ID\",\"service_name\":\"$SERVICE_NAME\",\"status\":\"Creating\"}" exit 0 fi else # Creation failed ERROR_MSG=$(echo "$CREATE_RESULT" | jq -r '.Message // .') echo "Service creation failed: $ERROR_MSG" >&2 echo "{\"success\":false,\"error\":\"Service creation failed: $ERROR_MSG\",\"service_name\":\"$SERVICE_NAME\"}" exit 1 fi FILE:scripts/list-images.sh #!/bin/bash # PAI-EAS Official Image List Query Script # Uses ListImages and ListImageLabels APIs to fetch image information set -e # Default parameters REGION="${REGION:-cn-hangzhou}" FRAMEWORK="${FRAMEWORK:-}" CHIP_TYPE="${CHIP_TYPE:-}" VERBOSE="${VERBOSE:-false}" # Color output RED='\033[0;31m' GREEN='\033[0;32m' YELLOW='\033[1;33m' BLUE='\033[0;34m' NC='\033[0m' # No Color # Help information show_help() { cat < Region (default: cn-hangzhou) -f, --framework Filter by framework (vLLM, SGLang, PyTorch, TensorFlow, PaiRag, CosyVoice) -c, --chip Filter by chip type (CPU, GPU, PPU) -v, --verbose Show detailed information -h, --help Show help information Examples: # List all official EAS-supported images $0 # List vLLM images $0 -f vLLM # List GPU type images $0 -c GPU # List vLLM GPU images (verbose) $0 -f vLLM -c GPU -v Framework types: vLLM - LLM inference acceleration SGLang - Structured output inference PyTorch - Deep learning framework TensorFlow - Deep learning framework PaiRag - RAG knowledge QA CosyVoice - Voice synthesis ModelScope - ModelScope community images EOF } # Parse parameters while [[ $# -gt 0 ]]; do case $1 in -r|--region) REGION="$2" shift 2 ;; -f|--framework) FRAMEWORK="$2" shift 2 ;; -c|--chip) CHIP_TYPE="$2" shift 2 ;; -v|--verbose) VERBOSE="true" shift ;; -h|--help) show_help exit 0 ;; *) echo -e "${RED}Unknown parameter: $1${NC}" show_help exit 1 ;; esac done echo -e "${BLUE}=== PAI-EAS Official Image List ===${NC}" echo "" # Build label filter conditions LABELS="system.official=true,system.supported.eas=true" FRAMEWORK_FILTER="" if [ -n "$FRAMEWORK" ]; then # Framework filter - filter in results FRAMEWORK_FILTER="$FRAMEWORK" fi if [ -n "$CHIP_TYPE" ]; then LABELS="$LABELS,system.chipType=$CHIP_TYPE" fi echo -e "${YELLOW}Query parameters:${NC}" echo " Region: $REGION" [ -n "$FRAMEWORK" ] && echo " Framework: $FRAMEWORK" [ -n "$CHIP_TYPE" ] && echo " Chip: $CHIP_TYPE" echo "" # Call ListImages API echo -e "${YELLOW}Querying image list...${NC}" IMAGES=$(aliyun aiworkspace list-images \ --verbose true \ --labels "$LABELS" \ --page-size 100 \ --user-agent AlibabaCloud-Agent-Skills/alibabacloud-pai-eas-service-deploy) # Check if successful if [ -z "$IMAGES" ] || [ "$(echo "$IMAGES" | jq -r '.RequestId' 2>/dev/null)" = "null" ]; then echo -e "${RED}Query failed, please check network connection and permissions${NC}" echo "Error: $IMAGES" exit 1 fi # Count images TOTAL=$(echo "$IMAGES" | jq -r '.Images | length') if [ "$TOTAL" -eq 0 ]; then echo -e "${YELLOW}No matching images found${NC}" exit 0 fi echo -e "${GREEN}Found $TOTAL images${NC}" echo "" # Display image list if [ "$VERBOSE" = "true" ]; then # Verbose mode if [ -n "$FRAMEWORK_FILTER" ]; then # Filter by framework echo "$IMAGES" | jq -r --arg fw "$FRAMEWORK_FILTER" ' [.Images[] | select(.Labels[] | select(.Key | contains($fw))) | { name: .Name, uri: .ImageUri, description: .Description, chipType: (([.Labels[] | select(.Key == "system.chipType") | .Value] | first) // "N/A"), framework: (([.Labels[] | select(.Key | startswith("system.framework.")) | .Value] | first) // "N/A"), port: (([.Labels[] | select(.Key == "system.eas.default.port") | .Value] | first) // "8000"), script: (([.Labels[] | select(.Key == "system.eas.default.script") | .Value] | first) // ""), latest: (([.Labels[] | select(.Key == "system.eas.deploy.latest") | .Value] | first) // "false") }] | sort_by(.latest == "false", .name) | .[] | "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n" + "Name: \(.name)\n" + "Type: [\(.chipType)] \(.framework)\n" + "Image: \(.uri)\n" + "Description: \(.description)\n" + "Port: \(.port)\n" + (if .script != "" then "Script: \(.script)\n" else "" end) + (if .latest == "true" then "Tag: ⭐ Latest\n" else "" end)' else echo "$IMAGES" | jq -r ' [.Images[] | { name: .Name, uri: .ImageUri, description: .Description, chipType: (([.Labels[] | select(.Key == "system.chipType") | .Value] | first) // "N/A"), framework: (([.Labels[] | select(.Key | startswith("system.framework.")) | .Value] | first) // "N/A"), port: (([.Labels[] | select(.Key == "system.eas.default.port") | .Value] | first) // "8000"), script: (([.Labels[] | select(.Key == "system.eas.default.script") | .Value] | first) // ""), latest: (([.Labels[] | select(.Key == "system.eas.deploy.latest") | .Value] | first) // "false") }] | sort_by(.latest == "false", .name) | .[] | "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n" + "Name: \(.name)\n" + "Type: [\(.chipType)] \(.framework)\n" + "Image: \(.uri)\n" + "Description: \(.description)\n" + "Port: \(.port)\n" + (if .script != "" then "Script: \(.script)\n" else "" end) + (if .latest == "true" then "Tag: ⭐ Latest\n" else "" end)' fi else # Compact mode if [ -n "$FRAMEWORK_FILTER" ]; then # Filter by framework echo -e "${BLUE}Image list:${NC}" echo "$IMAGES" | jq -r --arg fw "$FRAMEWORK_FILTER" ' [.Images[] | select(.Labels[] | select(.Key | contains($fw))) | { name: .Name, chipType: (([.Labels[] | select(.Key == "system.chipType") | .Value] | first) // "N/A"), framework: (([.Labels[] | select(.Key | startswith("system.framework.")) | .Value] | first) // "N/A"), latest: (([.Labels[] | select(.Key == "system.eas.deploy.latest") | .Value] | first) // "false"), description: .Description }] | sort_by(.latest == "false", .name) | .[] | " [\(.chipType)] \(.framework) - \(.name)" + (if .latest == "true" then " ⭐" else "" end)' else echo -e "${BLUE}Image list:${NC}" echo "$IMAGES" | jq -r ' [.Images[] | { name: .Name, chipType: (([.Labels[] | select(.Key == "system.chipType") | .Value] | first) // "N/A"), framework: (([.Labels[] | select(.Key | startswith("system.framework.")) | .Value] | first) // "N/A"), latest: (([.Labels[] | select(.Key == "system.eas.deploy.latest") | .Value] | first) // "false"), description: .Description }] | sort_by(.latest == "false", .name) | .[] | " [\(.chipType)] \(.framework) - \(.name)" + (if .latest == "true" then " ⭐" else "" end)' fi fi echo "" echo -e "${BLUE}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" echo -e "${YELLOW}Tips:${NC}" echo " Use -v for detailed information" echo " Use -f to filter by framework" echo " Use -c to filter by CPU/GPU/PPU" echo "" FILE:scripts/validate-service-config.sh #!/bin/bash # ============================================================================== # PAI-EAS Service Config Validator # Validate service JSON config, apply defaults, output validation result # # Usage: # ./validate-service-config.sh --config service.json # ./validate-service-config.sh --config service.json --fix # ./validate-service-config.sh --config-json '{"metadata":{"name":"xxx"...}}' # ./validate-service-config.sh --config service.json --output service-fixed.json # # Output: # JSON format, containing validation result and fixed config # ============================================================================== set -euo pipefail # Parameter parsing CONFIG_FILE="" CONFIG_JSON="" FIX_MODE=false OUTPUT_FILE="" while [[ $# -gt 0 ]]; do case $1 in --config) CONFIG_FILE="$2" shift 2 ;; --config-json) CONFIG_JSON="$2" shift 2 ;; --fix) FIX_MODE=true shift ;; --output) OUTPUT_FILE="$2" shift 2 ;; -h|--help) cat <&2 exit 1 ;; esac done # Read config if [[ -n "$CONFIG_FILE" ]]; then if [[ ! -f "$CONFIG_FILE" ]]; then echo '{"valid":false,"errors":["Config file does not exist: '"$CONFIG_FILE"'"],"warnings":[]}' exit 1 fi CONFIG=$(cat "$CONFIG_FILE") elif [[ -n "$CONFIG_JSON" ]]; then CONFIG="$CONFIG_JSON" else echo '{"valid":false,"errors":["Please provide --config or --config-json parameter"],"warnings":[]}' exit 1 fi # Check JSON format if ! echo "$CONFIG" | jq empty 2>/dev/null; then echo '{"valid":false,"errors":["Invalid JSON format"],"warnings":[]}' exit 1 fi # Extract fields NAME=$(echo "$CONFIG" | jq -r '.metadata.name // empty') INSTANCE=$(echo "$CONFIG" | jq -r '.metadata.instance // empty') WORKSPACE_ID=$(echo "$CONFIG" | jq -r '.metadata.workspace_id // empty') RESOURCE=$(echo "$CONFIG" | jq -r '.metadata.resource // empty') CONTAINERS=$(echo "$CONFIG" | jq -c '.containers // []') CLOUD_COMPUTING=$(echo "$CONFIG" | jq -c '.cloud.computing // empty') CLOUD_NETWORKING=$(echo "$CONFIG" | jq -c '.cloud.networking // empty') STORAGE=$(echo "$CONFIG" | jq -c '.storage // []') NETWORKING=$(echo "$CONFIG" | jq -c '.networking // empty') # Initialize errors and warnings ERRORS="[]" WARNINGS="[]" # 1. Required field check if [[ -z "$NAME" ]]; then ERRORS=$(echo "$ERRORS" | jq '. + ["metadata.name is a required field"]') fi CONTAINER_COUNT=$(echo "$CONTAINERS" | jq 'length') if [[ "$CONTAINER_COUNT" -eq 0 ]]; then ERRORS=$(echo "$ERRORS" | jq '. + ["containers is a required field, at least one container config is needed"]') else # Check image field for each container for i in $(seq 0 $((CONTAINER_COUNT - 1))); do IMAGE=$(echo "$CONTAINERS" | jq -r ".[$i].image // empty") if [[ -z "$IMAGE" ]]; then ERRORS=$(echo "$ERRORS" | jq ". + [\"containers[$i].image is a required field\"]") else # Simple image format check if [[ ! "$IMAGE" =~ ^[a-zA-Z0-9._/-]+(:[a-zA-Z0-9._-]+)?$ ]]; then WARNINGS=$(echo "$WARNINGS" | jq ". + [\"containers[$i].image format may be incorrect: $IMAGE\"]") fi fi done fi # 2. Service name format check if [[ -n "$NAME" ]] && [[ ! "$NAME" =~ ^[a-z0-9_]+$ ]]; then ERRORS=$(echo "$ERRORS" | jq ". + [\"metadata.name format error, only lowercase letters, numbers, and underscores allowed: $NAME\"]") fi # 3. Resource config check HAS_RESOURCE=false HAS_CLOUD_COMPUTING=false if [[ -n "$RESOURCE" ]]; then HAS_RESOURCE=true fi if [[ -n "$CLOUD_COMPUTING" ]] && [[ "$CLOUD_COMPUTING" != "null" ]]; then HAS_CLOUD_COMPUTING=true INSTANCE_TYPE=$(echo "$CLOUD_COMPUTING" | jq -r '.instance_type // empty') INSTANCES=$(echo "$CLOUD_COMPUTING" | jq -c '.instances // []') if [[ -z "$INSTANCE_TYPE" ]] && [[ $(echo "$INSTANCES" | jq 'length') -eq 0 ]]; then ERRORS=$(echo "$ERRORS" | jq '. + ["cloud.computing requires instance_type or instances configuration"]') fi fi if [[ "$HAS_RESOURCE" == "false" ]] && [[ "$HAS_CLOUD_COMPUTING" == "false" ]]; then ERRORS=$(echo "$ERRORS" | jq '. + ["Either metadata.resource (dedicated resource group) or cloud.computing (public resource group) must be configured"]') fi # 4. Network config check (for dedicated gateway) GATEWAY=$(echo "$NETWORKING" | jq -r '.gateway // empty') if [[ -n "$GATEWAY" ]]; then # Check if "shared" is incorrectly set as gateway if [[ "$GATEWAY" == "shared" ]]; then ERRORS=$(echo "$ERRORS" | jq '. + ["Shared gateway does not need networking.gateway field, please remove it"]') else # Dedicated gateway requires VPC config VPC_ID=$(echo "$CLOUD_NETWORKING" | jq -r '.vpc_id // empty') VSWITCH_ID=$(echo "$CLOUD_NETWORKING" | jq -r '.vswitch_id // empty') SECURITY_GROUP_ID=$(echo "$CLOUD_NETWORKING" | jq -r '.security_group_id // empty') if [[ -z "$VPC_ID" ]] || [[ -z "$VSWITCH_ID" ]] || [[ -z "$SECURITY_GROUP_ID" ]]; then ERRORS=$(echo "$ERRORS" | jq '. + ["When using dedicated gateway, cloud.networking must include vpc_id, vswitch_id, security_group_id"]') fi fi fi # 5. Storage mount check STORAGE_COUNT=$(echo "$STORAGE" | jq 'length') if [[ "$STORAGE_COUNT" -eq 0 ]]; then WARNINGS=$(echo "$WARNINGS" | jq '. + ["No storage mount configured, official images require model files to be mounted"]') else # Check OSS path format for i in $(seq 0 $((STORAGE_COUNT - 1))); do OSS_PATH=$(echo "$STORAGE" | jq -r ".[$i].oss.path // empty") if [[ -n "$OSS_PATH" ]]; then if [[ ! "$OSS_PATH" =~ ^oss:// ]]; then ERRORS=$(echo "$ERRORS" | jq ". + [\"storage[$i].oss.path format error, must start with oss://: $OSS_PATH\"]") fi fi done fi # 6. Check unsupported config types PROCESSOR=$(echo "$CONFIG" | jq -r '.processor // empty') MODEL=$(echo "$CONFIG" | jq -r '.model // empty') if [[ -n "$PROCESSOR" ]]; then ERRORS=$(echo "$ERRORS" | jq '. + ["processor deployment is not supported, please use containers image deployment"]') fi if [[ -n "$MODEL" ]]; then ERRORS=$(echo "$ERRORS" | jq '. + ["model deployment is not supported, please use containers image deployment"]') fi # 7. Check quota config (not supported) QUOTA_ID=$(echo "$CONFIG" | jq -r '.metadata.quota_id // empty') if [[ -n "$QUOTA_ID" ]]; then WARNINGS=$(echo "$WARNINGS" | jq '. + ["Quota resource group is not supported, quota_id config will be ignored"]') fi # Build result ERROR_COUNT=$(echo "$ERRORS" | jq 'length') if [[ "$ERROR_COUNT" -gt 0 ]]; then VALID="false" else VALID="true" fi # Apply defaults function apply_defaults() { local config="$1" # Apply defaults echo "$config" | jq ' # Default instance count if .metadata.instance == null or .metadata.instance == "" then .metadata.instance = 1 else . end # Default container port | .containers = (.containers | map( if .port == null or .port == "" then .port = 8000 else . end )) # Default shared memory size | if .metadata.shm_size == null or .metadata.shm_size == "" then .metadata.shm_size = 64 else . end # Default graceful termination period | if .runtime.termination_grace_period == null or .runtime.termination_grace_period == "" then .runtime.termination_grace_period = 30 else . end # Default keepalive timeout | if .rpc.keepalive == null or .rpc.keepalive == "" then .rpc.keepalive = 600 else . end # Shared gateway: remove networking field (not needed for shared gateway) | if .networking.gateway == "shared" or .networking.gateway == "" or .networking.gateway == null then del(.networking) else . end ' } # Main logic if [[ "$FIX_MODE" == "true" ]] && [[ "$VALID" == "true" ]]; then FIXED_CONFIG=$(apply_defaults "$CONFIG") if [[ -n "$OUTPUT_FILE" ]]; then echo "$FIXED_CONFIG" | jq '.' > "$OUTPUT_FILE" echo "{\"valid\":$VALID,\"errors\":$ERRORS,\"warnings\":$WARNINGS,\"output_file\":\"$OUTPUT_FILE\"}" else echo "{\"valid\":$VALID,\"errors\":$ERRORS,\"warnings\":$WARNINGS,\"fixed_config\":$FIXED_CONFIG}" fi elif [[ -n "$OUTPUT_FILE" ]]; then # Output config to file (even with errors, output for user to fix) echo "$CONFIG" | jq '.' > "$OUTPUT_FILE" echo "{\"valid\":$VALID,\"errors\":$ERRORS,\"warnings\":$WARNINGS,\"output_file\":\"$OUTPUT_FILE\"}" else echo "{\"valid\":$VALID,\"errors\":$ERRORS,\"warnings\":$WARNINGS}" fi if [[ "$VALID" == "false" ]]; then exit 1 fi exit 0

Alibabacloud Pai Eas Service Deploy

Comments (0)