Lokasi ngalangkungan proxy:   [ UP ]  
[Ngawartoskeun bug]   [Panyetelan cookie]                
Skip to content

Commit 3d5c297

Browse files
authored
Merge pull request #7978 from MicrosoftDocs/release-sqlseattle-ctp2.1
Publish for CTP 2.1
2 parents 68432ac + 83c82bc commit 3d5c297

55 files changed

Lines changed: 1920 additions & 1221 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 193 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
---
2+
title: How to deploy an app on SQL Server big data cluster | Microsoft Docs
3+
description: Deploy a Python or R script as an application on SQL Server 2019 big data cluster (preview).
4+
author: TheBharath
5+
ms.author: bharaths
6+
manager: craigg
7+
ms.date: 11/06/2018
8+
ms.topic: conceptual
9+
ms.prod: sql
10+
---
11+
12+
# How to deploy an app on SQL Server 2019 big data cluster (preview)
13+
14+
This article describes how to deploy and manage R and Python script as an application inside a SQL Server 2019 big data cluster (preview).
15+
16+
R and Python applications are deployed and managed with the **mssqlctl-pre** command-line utility which is included in CTP 2.1. This article provides examples of how to deploy these R and Python scripts as apps from the command line.
17+
18+
## Prerequisites
19+
20+
You must have a SQL Server 2019 big data cluster configured. For more information, see [How to deploy SQL Server big data cluster on Kubernetes](deployment-guidance.md).
21+
22+
## Installation
23+
24+
The **mssqlctl-pre** command-line utility is provided to preview the Python and R application deployment feature. Use the following command to install the utility:
25+
26+
```cmd
27+
pip3 install --extra-index-url https://private-repo.microsoft.com/python/ctp-2.1 mssqlctl-pre
28+
```
29+
30+
## Capabilities
31+
32+
In CTP 2.1 you can create, delete, list, and run an R or Python application. The following table describes the application deployment commands that you can use with **mssqlctl-pre**.
33+
34+
| Command | Description |
35+
|---|---|
36+
| `mssqlctl-pre login` | Log into a SQL Server big data cluster |
37+
| `mssqlctl-pre app create` | Create an app |
38+
| `mssqlctl-pre app list` | List deployed apps |
39+
| `mssqlctl-pre app delete` | Delete an app |
40+
| `mssqlctl-pre app run` | List running apps |
41+
42+
You can get help with the `--help` parameter as in the following example:
43+
44+
```bash
45+
mssqlctl-pre app create --help
46+
```
47+
48+
The following sections describe these commands in more detail.
49+
50+
## Log in
51+
52+
Before configuring R and Python applications, first log into your SQL Server big data cluster with the `mssqlctl-pre login` command. Specify the IP address of the `service-proxy-lb` (for example: `https://ip-address:30777`) along with the user name and password to the cluster.
53+
54+
You can get the IP address of the service-proxy-lb service by running this command in a bash or cmd window:
55+
```bash
56+
kubectl get svc service-proxy-lb:30777 -n <name of your cluster>
57+
```
58+
59+
```bash
60+
mssqlctl-pre login -e https://<ip-address-of-service-proxy-lb> -u <user-name> -p <password>
61+
```
62+
63+
## Create an app
64+
65+
To create an application, you pass Python or R code files to **mssqlctl-pre** with the `app create` command. These files reside locally on the machine that you are creating the app from.
66+
67+
Use the following syntax to create a new app in your big data cluster:
68+
69+
```bash
70+
mssqlctl-pre app create -n <app_name> -v <version_number> -r <runtime> -i <path_to_code_init> -c <path_to_code> --inputs <input_params> --outputs <output_params>
71+
```
72+
73+
The following command shows an example of what this command might look like:
74+
75+
```py
76+
#add.py
77+
def add(x,y):
78+
result = x+y
79+
return result;
80+
result=add(x,y)
81+
```
82+
To try this, save the above lines of code to your local directory as `add.py` and run the command below
83+
84+
```bash
85+
mssqlctl-pre app create --name add-app --version v1 --runtime Python --code ./add.py --inputs x=int,y=int --outputs result=int
86+
```
87+
88+
You can check if the app is deployed using the list command:
89+
90+
```bash
91+
mssqlctl-pre app list
92+
```
93+
94+
If the deployment is not complete you should see the "state" show "Creating":
95+
96+
```
97+
[
98+
{
99+
"name": "add-app",
100+
"state": "Creating",
101+
"version": "v1"
102+
}
103+
]
104+
```
105+
106+
After the deployment is successful you should see the "state" change to "Ready" status:
107+
108+
```
109+
[
110+
{
111+
"name": "add-app",
112+
"state": "Ready",
113+
"version": "v1"
114+
}
115+
]
116+
```
117+
118+
## List an app
119+
120+
You can list any apps that were successfully created with the `app list` command.
121+
122+
The following command lists all available applications in your big data cluster:
123+
124+
```bash
125+
mssqlctl-pre app list
126+
```
127+
128+
If you specify a name and version, it will list that specific app and its state (Creating or Ready):
129+
130+
```bash
131+
mssqlctl-pre app list --name <app_name> --version <app_version>
132+
```
133+
134+
The following example demonstrates this command:
135+
136+
```bash
137+
mssqlctl-pre app list --name add-app --version v1
138+
```
139+
140+
You should see output similar to the following example:
141+
142+
```
143+
[
144+
{
145+
"name": "add-app",
146+
"state": "Ready",
147+
"version": "v1"
148+
}
149+
]
150+
```
151+
152+
## Run an app
153+
154+
If the app is in a "Ready" state, you can use it by running it with your specified input parameters. Use the following syntax to run an app:
155+
156+
```bash
157+
mssqlctl-pre app run --name <app_name> --version <app_version> --inputs <inputs_params>
158+
```
159+
160+
The following example command demonstrates the run command:
161+
162+
```bash
163+
mssqlctl-pre app run --name add-app --version v1 --inputs x=1,y=2
164+
```
165+
166+
If the run was successful, you should see your output as specified when you created the app. The following is an example.
167+
168+
```
169+
{
170+
"changedFiles": [],
171+
"consoleOutput": "",
172+
"errorMessage": "",
173+
"outputFiles": {},
174+
"outputParameters": {
175+
"result": 3
176+
},
177+
"success": true
178+
}
179+
```
180+
181+
## Delete an app
182+
183+
To delete an app from your big data cluster, use the following syntax:
184+
185+
```bash
186+
mssqlctl-pre app delete --name add-app --version v1
187+
```
188+
189+
## Next steps
190+
191+
You can also check out additional samples at [https://github.com/Microsoft/sql-server-samples/tree/master/samples/features/sql-big-data-cluster](https://github.com/Microsoft/sql-server-samples/tree/master/samples/features/sql-big-data-cluster).
192+
193+
For more information about SQL Server big data clusters, see [What are SQL Server 2019 big data clusters?](big-data-cluster-overview.md).

docs/big-data-cluster/big-data-cluster-overview.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: Learn about SQL Server 2019 big data clusters (preview) that run on
44
author: rothja
55
ms.author: jroth
66
manager: craigg
7-
ms.date: 10/25/2018
7+
ms.date: 11/06/2018
88
ms.topic: overview
99
ms.prod: sql
1010
---
@@ -13,6 +13,8 @@ ms.prod: sql
1313

1414
Starting with [!INCLUDE[SQL Server 2019](../includes/sssqlv15-md.md)], SQL Server big data clusters allow you to deploy scalable clusters of SQL Server, Spark, and HDFS containers running on Kubernetes. These components are running side by side to enable you to read, write, and process big data from Transact-SQL or Spark, allowing you to easily combine and analyze your high-value relational data with high-volume big data.
1515

16+
For more information about new features and known issues for latest release, see the [release notes](big-data-cluster-release-notes.md).
17+
1618
[!INCLUDE [Limited public preview note](../includes/big-data-cluster-preview-note.md)]
1719

1820
## Scenarios

docs/big-data-cluster/big-data-cluster-release-notes.md

Lines changed: 73 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,18 +4,88 @@ description: This article describes the latest updates and known issues for SQL
44
author: rothja
55
ms.author: jroth
66
manager: craigg
7-
ms.date: 10/04/2018
7+
ms.date: 11/06/2018
88
ms.topic: conceptual
99
ms.prod: sql
1010
---
1111

1212
# Release notes for SQL Server 2019 big data clusters
1313

14-
This article provides the latest updates and known issues for the latest release of SQL Server big data clusters.
14+
This article provides the latest updates and known issues for the latest release of SQL Server big data clusters. The following table links you to the section for the releases covered in this article.
15+
16+
| Release | Date |
17+
|---|---|
18+
| [CTP 2.1](#ctp21) | November 2018 |
19+
| [CTP 2.0](#ctp20) | October 2018 |
20+
1521

1622
[!INCLUDE [Limited public preview note](../includes/big-data-cluster-preview-note.md)]
1723

18-
## CTP 2.0 (October 2018)
24+
## <a id="ctp21"></a> CTP 2.1 (November 2018)
25+
26+
The following sections describe the new features and known issues for big data clusters in SQL Server 2019 CTP 2.1.
27+
28+
### What's in the CTP 2.1 release?
29+
30+
- [Deploy Python and R apps](big-data-cluster-create-apps.md) in a big data cluster.
31+
- New version of **mssqlctl** and updated images.
32+
- Miscellaneous bug fixes and improvements.
33+
34+
### Known issues
35+
36+
The following sections provide known issues for SQL Server big data clusters in CTP 2.1.
37+
38+
#### Deployment
39+
40+
- Upgrading a big data data cluster from a previous release is not supported. You must backup and delete any existing big data cluster before deploying the latest release. For more information, see [Upgrade to a new release](deployment-guidance.md#upgrade).
41+
42+
- After deploying on AKS, you might see the following two warning events from the deployment. Both of these events are known issues, but they do not prevent you from successfully deploying the big data cluster on AKS.
43+
44+
`Warning FailedMount: Unable to mount volumes for pod "mssql-storage-pool-default-1_sqlarisaksclus(c83eae70-c81b-11e8-930f-f6b6baeb7348)": timeout expired waiting for volumes to attach or mount for pod "sqlarisaksclus"/"mssql-storage-pool-default-1". list of unmounted volumes=[storage-pool-storage hdfs storage-pool-mlservices-storage hadoop-logs]. list of unattached volumes=[storage-pool-storage hdfs storage-pool-mlservices-storage hadoop-logs storage-pool-java-storage secrets default-token-q9mlx]`
45+
46+
`Warning Unhealthy: Readiness probe failed: cat: /tmp/provisioner.done: No such file or directory`
47+
48+
- If a big data cluster deployment fails, the associated namespace is not removed. This could result in an orphaned namespace on the cluster. A workaround is to delete the namespace manually before deploying a cluster with the same name.
49+
50+
#### Admin portal
51+
52+
- When you [create an app using msqlctl-ctp command](big-data-cluster-create-apps.md) and deploy it on a SQL Server big data cluster, the Cluster Admin Portal shows the pods where the application was deployed as "Unknown" in the Controller section of the Admin Portion.
53+
54+
#### External tables
55+
56+
- It is possible to create a data pool external table for a table that has unsupported column types. If you query the external table, you get a message similar to the following:
57+
58+
`Msg 7320, Level 16, State 110, Line 44 Cannot execute the query "Remote Query" against OLE DB provider "SQLNCLI11" for linked server "(null)". 105079; Columns with large object types are not supported for external generic tables.`
59+
60+
- If you query a storage pool external table, you might get an error if the underlying file is being copied into HDFS at the same time.
61+
62+
`Msg 7320, Level 16, State 110, Line 157 Cannot execute the query "Remote Query" against OLE DB provider "SQLNCLI11" for linked server "(null)". 110806;A distributed query failed: One or more errors occurred.`
63+
64+
#### Spark and notebooks
65+
66+
- POD IP addresses may change in the Kubernetes environment as PODs restarts. In the scenario where the master-pod restarts, the Spark session may fail with `NoRoteToHostException`. This is caused by JVM caches that don't get refreshed with new IP addresses.
67+
68+
- If you have Jupyter already installed and a separate Python on Windows, Spark notebooks might fail. To work around this issue, upgrade Jupyter to the latest version.
69+
70+
- In a notebook, if you click the **Add Text** command, the text cell is added in preview mode rather than edit mode. You can click on the preview icon to toggle to edit mode and edit the cell.
71+
72+
#### HDFS
73+
74+
- If you right-click on a file in HDFS to preview it, you might see the following error:
75+
76+
`Error previewing file: File exceeds max size of 30MB`
77+
78+
Currently there is no way to preview files larger than 30 MB in Azure Data Studio.
79+
80+
- Configuration changes to HDFS that involve changes to hdfs-site.xml are not supported.
81+
82+
#### Security
83+
84+
- The SA_PASSWORD is part of the environment and discoverable (for example in a cord dump file). You must reset the SA_PASSWORD on the master instance after deployment. This is not a bug but a security step. For more information on how to change the SA_PASSWORD in a Linux container, see [Change the SA password](../linux/quickstart-install-connect-docker.md#sapassword).
85+
86+
- AKS logs may contain SA password for big data cluster deployments.
87+
88+
## <a id="ctp20"></a> CTP 2.0 (October 2018)
1989

2090
The following sections describe the new features and known issues for big data clusters in SQL Server 2019 CTP 2.0.
2191

docs/big-data-cluster/cluster-admin-portal.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: Learn how to use the cluster administration portal to monitor SQL S
44
author: yualan
55
ms.author: alayu
66
manager: craigg
7-
ms.date: 10/16/2018
7+
ms.date: 11/06/2018
88
ms.topic: conceptual
99
ms.prod: sql
1010
---
@@ -27,7 +27,7 @@ Follow the [quickstart to deploy your big data cluster](quickstart-big-data-clus
2727
Once the controller pod is running, you can use the cluster administration portal to monitor the deployment. You can access the portal using the external IP address and port number for the `service-proxy-lb` (for example: **https://\<ip-address\>:30777**). Credentials for accessing the admin portal are the values of `CONTROLLER_USERNAME` and `CONTROLLER_PASSWORD` environment variables provided above.
2828

2929
> [!NOTE]
30-
> For CTP 2.0, There is a security warning when accessing the web page since it is using auto-generated SSL certificates.
30+
> For CTP 2.1, There is a security warning when accessing the web page since it is using auto-generated SSL certificates.
3131
3232
## Overview
3333

docs/big-data-cluster/concept-compute-pool.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: This article describes the compute pool in a SQL Server 2019 big da
44
author: rothja
55
ms.author: jroth
66
manager: craigg
7-
ms.date: 10/01/2018
7+
ms.date: 11/06/2018
88
ms.topic: conceptual
99
ms.prod: sql
1010
---
@@ -18,7 +18,7 @@ This article describes the role of *SQL Server compute pools* in a SQL Server 20
1818
A compute pool is made of one or more compute pods running in Kubernetes. The automated creation and management of these pods is coordinated by the [SQL Server master instance](concept-master-instance.md). Each pod contains a set of base services and an instance of the SQL Server database engine.
1919

2020
> [!NOTE]
21-
> CTP 2.0 only supports a single compute pool per cluster.
21+
> CTP 2.1 only supports a single compute pool per cluster.
2222
2323
## Scale-out groups
2424

docs/big-data-cluster/concept-controller.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: This article describes the controller of a SQL Server 2019 big data
44
author: mihaelablendea
55
ms.author: mihaelab
66
manager: craigg
7-
ms.date: 10/01/2018
7+
ms.date: 11/06/2018
88
ms.topic: conceptual
99
ms.prod: sql
1010
---
@@ -21,8 +21,8 @@ The controller service provides the following core functionality:
2121
- Expose monitoring tools to observe the state of the cluster
2222
- Expose troubleshooting tools to detect and repair unexpected issues
2323
- Manage cluster security: ensure secure cluster endpoints, manage users and roles, configure credentials for intra-cluster communication
24-
- Manage the workflow of upgrades so that they are implemented safely (not available in CTP 2.0)
25-
- Manage high availability and DR for statefull services in the cluster (not available in CTP 2.0)
24+
- Manage the workflow of upgrades so that they are implemented safely (not available in CTP 2.1)
25+
- Manage high availability and DR for statefull services in the cluster (not available in CTP 2.1)
2626

2727
## Deploying the controller service
2828

docs/big-data-cluster/concept-data-persistence.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: Learn about how data persistence works in a SQL Server 2019 big dat
44
author: rothja
55
ms.author: jroth
66
manager: craigg
7-
ms.date: 10/01/2018
7+
ms.date: 11/06/2018
88
ms.topic: conceptual
99
ms.prod: sql
1010
---
@@ -19,7 +19,7 @@ The way SQL Server big data cluster consumes these persistent volumes is by usin
1919

2020
> [!NOTE]
2121
22-
> For CTP 2.0, only `ReadWriteOnce` access mode for the whole cluster is supported.
22+
> For CTP 2.1, only `ReadWriteOnce` access mode for the whole cluster is supported.
2323
2424
## Deployment settings
2525

docs/big-data-cluster/concept-master-instance.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ description: This article describes the master instance in a SQL Server 2019 big
44
author: rothja
55
ms.author: jroth
66
manager: craigg
7-
ms.date: 10/01/2018
7+
ms.date: 11/06/2018
88
ms.topic: conceptual
99
ms.prod: sql
1010
---
@@ -21,7 +21,7 @@ The SQL Server master instance provides an externally accessible TDS endpoint fo
2121

2222
## Scale-out query management
2323

24-
The SQL Server master instance contains the scale-out query engine that is used to distribute queries across SQL Server instances on nodes in the [compute pool](concept-compute-pool.md). The scale-out query engine also provides access through Transact-SQL to all Hive tables in the cluster without any additional configuration. (Hive tables support is not in CTP 2.0)
24+
The SQL Server master instance contains the scale-out query engine that is used to distribute queries across SQL Server instances on nodes in the [compute pool](concept-compute-pool.md). The scale-out query engine also provides access through Transact-SQL to all Hive tables in the cluster without any additional configuration. (Hive tables support is not in CTP 2.1)
2525

2626
## Metadata and user databases
2727

0 commit comments

Comments
 (0)