Data Item

Description

A data item is a container of data that is registered with the server. The set of data items registered with the server comprises the server's data store.

A data item contains all the information required by the server to connect to a folder or database while serving out one or more GIS services. Data items are extensively used while publishing GIS services to the server, as they inform the publishing client (such as ArcMap) whether the data needs to be explicitly copied to the server or if it can be referred from the server. The server supports the following types of data items:

JSON Structure

A data item is represented in the server as a JSON object.

The type property of the JSON object describes the type of the data item, with egdb and folder as valid values.

The path property defines a unique identifier for each of the data items when registered with the server. While the server supports a hierarchical view of data items, this is not used often. Most file shares are registered under /fileShares and enterprise databases are registered under /enterpriseDatabases, respectively.

The info section stores the actual connection string or file system path to the data item. The dataStoreConnectionType property in the info section indicates to the server if the data item is shared, replicated, or serverOnly.

{
	"path": "<unique path on the server>",
	"type": "<folder|egdb>", 
	"clientPath": "<client paths for replicated folders>",
	"info": {
		"path": "<path to the folder>",
		"clientConnectionString": "<connection string for client to connect to shared enterprise database>",
		"connectionString": "<connection string for server to connect to enterprise database>",
		"dataStoreConnectionType": "<shared|replicated|serverOnly>",
		"isManaged": <true|false>
	}
}

Connection Strings

For enterprise geodatabases, you need to enter the connection string as the connectionString property inside the info section of the JSON object. A connection string consists of the information bits like the name of the database server, the database name and the credentials. The server will store this data item JSON inside of its configuration store "as is." It is recommended that you encrypt the credentials using the encryption scheme before storing it into the server. This can be achieved by invoking Get Database Connection String tool inside the Publishing Tools toolbox published as a system service on the server. The string returned by the tool can be used as the value of the connectionString property.

See the examples below for more information on the format of the different data items supported by the server.

Example—Folder where the publisher and server use the same shared path

{
	"path": "/fileShares/folder_shared", //a unique path on the server
	"type": "folder", //as this is a file share
	"clientPath": null, //not needed as this is a shared folder
	"info": {
		"path": "\\\\server\\data\\rest_data", //path to the share
		"dataStoreConnectionType": "shared" //this is a shared folder     
	}
}

Example—Folder where the publisher and server use local paths

{
	"path": "/fileShares/folder_replicated", //a unique path on the server
	"type": "folder", //as this is a folder
	"clientPath": "C:\\data", //the path to the folder from the client
	"info": {
		"path": "c:\\data", //the path to the folder from the server
		"dataStoreConnectionType": "replicated", //indicates that client and server are seeing the same datasets
		"hostName": "GRID3" //name of the client host
	}
}

Example—Enterprise database where the publisher and server use the same database

{
	"path": "/enterpriseDatabases/egdb_shared", //a unique path on the server
	"type": "egdb", //as this is a database
	"clientPath": null, //not needed as this is a database
	"info": {
		"connectionString": "ENCRYPTED_PASSWORD=00022e686f66464c762b6e2b31732b675a4e35667832547070513d3d2a00;
							SERVER=dbserver;INSTANCE=sde:sqlserver:dbserver;DBCLIENT=sqlserver;
							DB_CONNECTION_PROPERTIES=dbserver;DATABASE=vtest;
							USER=map;VERSION=sde.DEFAULT;AUTHENTICATION_MODE=DBMS", //an encrypted connection string
		"isManaged": false, //indicates that the database is not fully owned by the server
		"dataStoreConnectionType": "shared" //indicates that the database is shared with clients
	}
}

Example—Server-managed enterprise database (such as ArcGIS Server's Managed Database)

{
	"path": "/enterpriseDatabases/egdb_managed", //a unique path on the server
	"type": "egdb", //as this is a database
	"clientPath": null, //not needed as this is a database
	"info": {
		"connectionString": "ENCRYPTED_PASSWORD=00022e683671383653345a4c4f465743393131343544673045773d3d2a00;
							SERVER=dbserver;INSTANCE=sde:sqlserver:dbserver;DBCLIENT=sqlserver;
							DB_CONNECTION_PROPERTIES=dbserver;DATABASE=vtest;USER=map;
							AUTHENTICATION_MODE=DBMS", //an encrypted connection string for server to connect the database
		"isManaged": true, //indicates that the database is fully owned and managed by the server
		"dataStoreConnectionType": "serverOnly" //indicates that the database is not shared with clients 
	}
}

Example—Enterprise database where the publisher and server use different databases

{
	"path": "/enterpriseDatabases/egdb_replicated", //a unique path on the server
	"type": "egdb", //as this is a database
	"clientPath": null, //not needed as this is a database
	"info": {
		"clientConnectionString": "ENCRYPTED_PASSWORD=00022e686255754d75395162382f704c726a4d4d6456364b56673d3d2a00;
								SERVER=dbserver;INSTANCE=sde:sqlserver:dbserver;DBCLIENT=sqlserver;
								DB_CONNECTION_PROPERTIES=dbserver;DATABASE=vtest;USER=map;VERSION=sde.DEFAULT;
								AUTHENTICATION_MODE=DBMS", //an encrypted connection string for client to connect the database
		"connectionString": "ENCRYPTED_PASSWORD=00022e6847686f65752b4f503261623364533032445441544d773d3d2a00;
							SERVER=dbserver2;INSTANCE=sde:sqlserver:dbserver2;DBCLIENT=sqlserver;
							DB_CONNECTION_PROPERTIES=dbserver2;DATABASE=map;USER=ags;
							AUTHENTICATION_MODE=DBMS",//an encrypted connection string for server to connect the database
		"isManaged": false, //indicates that the database is not fully owned and managed by the server 
		"dataStoreConnectionType": "replicated" //indicates that the database is shared with clients 
	}
}

Big Data File Share

Big Data File Share data items are file shares, HDFS or Hive data stores that contain input data for Geoanalytics. Connection information for the data store item is stored within connectionString as a stringified JSON. ArcGIS Server encrypts connection string for storage. Connection strings that are encrypted will include a {crypt} prefix. You can get a data store item with decrypted connection string by passing a decrypt=true parameter in the request for a data store item. Data store with decrypted connection string will be returned only for requests made with https. The examples below show decrypted connectionString.The examples below show data stores with decrypted connectionString.

id for a data store item is auto-generated.

connectionType represents the type of big data file share item. Possible values are fileShare, hdfs and hive.

A bigDataFileShare can be successfully registered only on a Geoanalytics Server.

username is optional for HDFS and Hive file share.

Example—Big data file share item - fileShare

{
	"path": "/bigDataFileShares/myFileShare",
	"type": "bigDataFileShare",
	"id": "3f9b26d5-588a-499f-b8e6-f05c06fc3e74",
	"info": {
		"connectionString": "{\"path\":\"\\\\\\\\myFileShare\\\\data\\\\geoanalytics\"}", //backslashes in path should be double escaped.
		"connectionType": "fileShare"
	}
}

Example—Big data file share item - HDFS

{
	"path": "/bigDataFileShares/myhdfsShare",
	"type": "bigDataFileShare",
	"id": "4g7b36d5-588a-469f-a8e6-f05c06fc3e77",
	"info": {
		"connectionString": "{\"path\": \"/hdfs://hostname.domain.com:8020/data/geoanalytics/”\",\"username\":\"username\"}",
  "connectionType": "hdfs"
	}
}

Example—Big data file share item - Hive

{
	"path": "/bigDataFileShares/myhiveShare",
	"type": "bigDataFileShare",
	"id": "7c3a36d5-928c-463a-a8f6-a070c6cf3e43",
	"info": {
		"connectionString": "{\"metaStoreUris\" : \"thrift://myhost.domain.com:9083\",\"database\": \"default\",\"username\":\"username\"}",
  "connectionType": "hive"
	}
}

Cloud Store

Cloud Store data item represents a connection to a Amazon or Microsoft Azure store.

Connection information for the data store item is stored within connectionString as a stringified JSON. ArcGIS Server encrypts connection string for storage. Connection strings that are encrypted will include a {crypt} prefix. You can get a data store item with decrypted connection string by passing a decrypt=true parameter in the request for a data store item. Data store with decrypted connection string will be returned only for requests made with https. The examples below show data stores with decrypted connectionString. .

A valid objectStore (S3 bucket or Azure Blob store) is required. Folders within an object store are optional.

id for a data store item is auto-generated.

Example—Cloud store item - Amazon

{
	"path": "/cloudStores/myAmazonStore",
	"type": "cloudStore",
	"id": "e64eed8a-b8ce-4243-a1c3-715c6f27b976",
	"provider": "amazon",
	"info": {
		"isManaged": false,
		"connectionString": "{\"accessKeyId\":\"accessKeyId\",\"secretAccessKey\":\"secretAccessKey\",\"region\":\"us-east-1\",\"defaultEndpointsProtocol\":\"https\",\"credentialType\":\"accesskey\"}",
		"objectStore": "myS3Store\F1\F2"
	}
}

Example—Cloud store item - Azure

{
  "path": "/cloudStores/AzureTest",
  "type": "cloudStore",
  "id": "18501265-7546-48c4-8dc9-fc429e971a0c",
  "provider": "azure",
  "info": {
    "isManaged": false,
    "connectionString": "{\"accountKey\":\"accountKey\",\"accountName\":\"accountName\",\"defaultEndpointsProtocol\":\"https\",\"accountEndpoint\":\"core.windows.net\",\"credentialType\":\"accessKey\"}",
    "objectStore": "blobtest"
  }
}

Raster Store

Network accessible file system folders or cloud stores that are output stores for Raster Analysis.

Connection information for the data store item is stored within connectionString as a stringified JSON. ArcGIS Server encrypts connection string for storage. Connection strings that are encrypted will include a {crypt} prefix. You can get a data store item with decrypted connection string by passing a decrypt=true parameter in the request for a data store item. Data store with decrypted connection string will be returned only for requests made with https. The examples below show data stores with decrypted connectionString. .

id for a data store item is auto-generated.

connectionType can be fileShare or dataStore

Example—Raster store item - File share

{
  "path": "/rasterStores/rasterstore_fileshare",
  "type": "rasterStore",
  "id": "da89edbb-56a2-4f8d-87bb-ffbdb5b31696",
  "info": {
    "connectionString": "{\"path\":\"\\\\\\\\myserver\\\\data\"}", //backslashes in path should be double escaped.
    "connectionType": "fileShare"
  }
}

Example—Raster store item - Cloud Store

{
	"path": "/rasterStores/rasterStoreAzure",
	"type": "rasterStore",
	"id": "289c689d-6905-4efd-a7ca-82d6fbb7b1ed",
	"info": {
		"connectionString": "{\"path\":\"/cloudStores/AzureTest\"}",
		"connectionType": "dataStore"
	}
}

3/3/2017