ArcGIS Server in development, staging, and production environments

Many organizations use separate development, staging, and production environments to maintain the quality of their websites. Here’s how the environments look when ArcGIS Server is involved:

The development, staging, and production environments ideally use different databases and infrastructure. Each organization has its own rules for how changes are tested and approved across the sites.

Moving a change from site to site can present logistical challenges. This help topic provides patterns and scripts to guide you through the process.

Configuring each environment

In each environment, you’ll need to install ArcGIS Server, create a site, and configure security, server object extensions (SOEs), and other settings. Most of these tasks are faster if performed manually although you can use a script like Example: Create users and roles from two text files.

Get your development site working first. Then create the staging site, followed by the production site.

Deploying services

The key to deploying services in multiple environments is to register your folders and databases correctly with ArcGIS Server and use service definitions (SDs) for publishing.

Registering folders and databases with ArcGIS Server

When you register a folder or database with ArcGIS Server, you provide both the publisher’s path to the data and the server’s path.

  • The publisher’s path is the data path on the machine you’ll use to make the SD files. The publisher’s path is always the same when you register an item on the development, staging, and production servers.
  • The server’s path is the data path on the server. This path can vary when you register an item on the development, staging, and production servers.

If you have many data folders or databases to register, you may consider using scripts. Example: Register folders and databases listed in a text file uses the ArcPy AddDataStoreItem function to register a list of folders and database connections supplied in a text file. You would modify the text file for each environment.

Publishing services

Use SD files when deploying your services in multiple environments. The SD takes the information needed to publish a service and packages it into one convenient file. Although it’s possible to package the GIS data within the SD, you’ll probably find it easier to preload the data in each environment and use replication to keep it in sync.

Create connection-neutral SD files (choose the No available connection option in the Save a Service Definition wizard) so they are flexible enough to be published on any server. When you publish an SD file, ArcGIS Server automatically corrects the paths written in the SD so that your server paths are used. Thus, careful data registration allows you to deploy the same SD file in multiple environments.

Publishing services is a task well-suited for scripting. Example: Publish service definitions listed in a text file reads a text file and publishes all SDs listed. The script uses the ArcPy function Upload Service Definition to publish each SD.

SD files are deployed to the different environments using a script

After deploying your services from the SDs, you must enable any SOEs required by the services. This can be done manually or through scripting.

Applying permissions is another task that can be scripted. Example: Apply permissions from a text file uses the ArcGIS REST API to apply permissions to various services as listed in a text file.

Updating services

Sometimes you might want to update a service to use new properties or reflect changes in the source document, such as a set of permanent symbology edits in an ArcMap document (MXD). The recommended way to update a service in multiple environments is to save a new SD file then delete the service and publish the updated SD.

When you take this approach, the same example script used above for publishing can also perform service updates. Just modify the input file to include only the services you want to update. If an existing service is found, the script deletes it before uploading the SD.

After you update a service in this way, you need to reenable any SOEs used by the service.

You can alternatively script updates to service properties (but not the map or source document) using the Edit Service operation in the ArcGIS REST API.

Keeping data in sync

You need to make sure that your data is kept in sync across your multiple environments. Geodatabase replication can help you with this. Alternatively, you can completely replace the old dataset with a new dataset. For example, you might delete a file geodatabase and replace it with an updated file geodatabase.

If you decide to entirely replace tables or file geodatabases, keep in mind that ArcGIS Server services lock the schemas of the underlying datasets by default. If the schema is locked, you need to stop your service before you can replace the data. With some degree of caution you can disable schema locking for map services, but you can’t disable it for other service types.

Updating applications

To move an application between the development, staging, and production environments, you copy the application files from one site to the other, then update any web service URLs in your code so they point at the new site. Use configuration files to define the URLs for your services.

The script below helps you update the URLs in your code. The code recursively looks for files within a folder you provide, searches for a particular string such as http://myDevServer/arcgis, and replaces it with the location of your staging or production site, such as http://myProdServer/arcgis.

Before you launch this script, you should make a backup copy of the original application files. Also note that the script allows you to specify the extension of the text files you want to process, for example, .xml for your ArcGIS Viewer for Flex configuration files, .js and .html for your ArcGIS API for JavaScript files, and so on.

Once you’ve replaced the URLs using the script, you can copy the application files onto the next server in the workflow (staging or production).

import os
import sys
import shutil
import traceback

def dirEntries(dir_name, subdir, *args):
    '''Return a list of file names found in directory 'dir_name'
    If 'subdir' is True, recursively access subdirectories under 'dir_name'.
    Additional arguments, if any, are file extensions to match filenames. Matched
        file names are added to the list.
    If there are no additional arguments, all files found in the directory are
        added to the list.
    Example usage: fileList = dirEntries(r'H:\TEMP', False, 'txt', 'py')
        Only files with 'txt' and 'py' extensions will be added to the list.
    Example usage: fileList = dirEntries(r'H:\TEMP', True)
        All files and all the files in subdirectories under H:\TEMP will be added
        to the list.
    '''
    fileList = []
    for file in os.listdir(dir_name):
        dirfile = os.path.join(dir_name, file)
        if os.path.isfile(dirfile):
            if not args:
                fileList.append(dirfile)
            else:
                if os.path.splitext(dirfile)[1][1:] in args:
                    fileList.append(dirfile)
        # recursively access file names in subdirectories
        elif os.path.isdir(dirfile) and subdir:
            print "Accessing directory:", dirfile
            fileList.extend(dirEntries(dirfile, subdir, *args))
    return fileList

def updateString(infileName, srchString,rplString):
    bakFileName = os.path.splitext(infileName)[0] + ".bak"
    if not os.path.exists(bakFileName):
        # copy the original file
        shutil.copy(infileName, bakFileName)

    # Open the backup (= original) file to read
    inFileHndl = open(bakFileName,"r")
    outFileHndl = open(infileName,"w")  

    for line in inFileHndl:
        if line.find(searchString) > 0:
            line = line.replace(searchString, replaceString)
        outFileHndl.write(line)
        
    outFileHndl.close()
    inFileHndl.close()
    # remove the backup (=original content file)
    os.remove(bakFileName)

if __name__ == '__main__':

    try:
        inFolder = r"C:\inetpub\wwwroot\viewer"  # path to search
        searchString = "http://mydevserver"      # string to find
        replaceString = "http://mystgserver"     # string - replace searchString with replaceString
        
        fList = dirEntries(inFolder, True, 'xml')
        for f in fList:
            updateString(f, searchString, replaceString)
            
        print '\n\n\n' + 'Process Complete'

    except:
        # Return any python specific errors as well as any errors from the geoprocessor
        #
        tb = sys.exc_info()[2]
        tbinfo = traceback.format_tb(tb)[0]
        pymsg = "PYTHON ERRORS:\nTraceback Info:\n" + tbinfo + "\nError Info:\n    " + \
                str(sys.exc_type)+ ": " + str(sys.exc_value) + "\n"
        print '\n\n\n' + pymsg
9/1/2015