Extending geoprocessing through Python modules

By utilizing the Python Distribution Utilities (Distutils) and the information found in the Python documentation, Distributing Python Modules, Python toolboxes and custom toolboxes containing Models and/or Script Tools can be efficiently distributed to ArcGIS users in the form of Python modules. The process for building and distributing these toolboxes starts with the creation of the Python module. The module used in this example will be foo.py.

foo.py

Sample code to create the foo Python module:

import os

def hello():
    print 'Hello ' + os.getenv('username')

In order for the module to be built and distributed correctly, a specific directory structure must exist. A directory named foo must be created to store the foo module. Since distribution requires that the directory storing the foo module be within a parent directory, a directory named src is created to house the foo directory and the foo module. The directory structure should be the following:

Alt text is required

In order for the foo module to initialize and automatically execute certain code once it has been imported, it requires an __init__.py file. With the __init__.py in place, users will be able to access the foo module and import selected definitions.

__init__.py

Sample code to create __init__.py for foo:

from foo import hello

The directory structure should now be the following:

Alt text is required

With these files and directory structure in place, the foo module can be imported through import foo, and foo.hello() can be called and executed. The next step is to build a distribution package for the foo module so it can be installed into the Python site-packages directory in order to be easily shared. This is done by writing a setup.py script.

setup.py

Sample code to create setup.py:

from distutils.core import setup
setup(name='foo',
    version='1.0',
    packages=['foo'],
    package_dir={'foo': 'foo'},
    )

The setup.py file sets the module name and version, and points the build utility to the package directory. The setup.py file should be saved to the src directory. At this point, the directory structure should be the following:

Alt text is required

With the directory structure in place, an installer can be built for the foo module by running one of the commands below from within the src directory using the corresponding operating system command prompt. This example is built on the Windows OS.

NoteNote:

Make sure that the path to python.exe is set on your system. In the Windows OS, this can be done by adding C:\Python27\ArcGIS10.2 to the Path System Variable in the Windows OS Environment Variables.

Windows:

python setup.py bdist_wininst

Linux:

python setup.py bdist_rpm

The Windows builder creates dist and build directories in the src directory. In the dist directory, foo-1.0.win32.exe is created. This is an executable that can be distributed in order to install the foo module to the Python site-packages directory on a Windows machine. As an alternative to running the executable to install the foo module, the foo directory can also be copied directly from the build/lib directory into the Python site-packages directory. If there are user restrictions in place that prohibit running an executable, copying the foo directory from the build/lib directory to the site-packages directory will produce the same effect as installing it through the executable. Once the foo module is installed or copied into the site-packages directory, the structure should be the following:

Alt text is required

This process can be further implemented to extend geoprocessing functionality by adding custom toolboxes and/or Python toolboxes directly to the ArcGIS system toolboxes. As a system toolbox, it is readily accessible in the list of system toolboxes within ArcGIS and can have ArcPy wrappers created for extending ArcPy as well. In addition, this allows the custom toolbox module to take advantage of the well-established methodology that ArcGIS system toolboxes have for message distribution, language-based help, and response to localized settings. ArcGIS for Desktop will search within the Python site-packages location to see if a directory named esri exists within each module. The esri directory contains the custom toolboxes along with their associated help files. The following is the directory structure for the English language:

Alt text is required

Custom toolboxes (.tbx and .pyt) are placed in the esri/toolboxes directory along with any supporting scripts if using script tools. The esri/help/gp directory is where the toolbox and tool metadata (.xml) for custom toolboxes and tools are stored. The naming convention for the toolbox is <toolbox alias>_toolbox.xml and the naming convention for each tool is <toolname>_<toolbox alias>.xml. The esri/help/gp/messages directory is where any geoprocessing message (.xml) files are placed. These message files are used within the Python toolboxes for messages that need to be internationalized. The toolbox and tool labels categories override files are located in the esri/help/gp/toolboxes directory. Through the creation of a new Python toolbox named SamplePythonToolbox, the entire process of extending geoprocessing through Python modules can be demonstrated. For additional information on creating and working with Python toolboxes, see http://resources.arcgis.com/en/help/main/10.1/index.html#//001500000034000000.

NoteNote:

It is important to set an alias on the Python toolbox that will be used in this process.

SamplePythonToolbox.pyt

Sample code to create a Python toolbox:

import arcpy
import os
import foo

class Toolbox(object):
    def __init__(self):
        """Define the toolbox (the name of the toolbox is the name of the
        .pyt file)."""
        self.label = "Toolbox"
        self.alias = "SampleToolbox"

        # List of tool classes associated with this toolbox
        self.tools = [SampleTool]


class SampleTool(object):
    def __init__(self):
        """Define the tool (tool name is the name of the class)."""
        self.label = "Sample Tool"
        self.description = ""
        self.canRunInBackground = False

    def getParameterInfo(self):
        """Define parameter definitions"""
        params = None
        return params

    def isLicensed(self):
        """Set whether tool is licensed to execute."""
        return True

    def updateParameters(self, parameters):
        """Modify the values and properties of parameters before internal
        validation is performed.  This method is called whenever a parameter
        has been changed."""
        return

    def updateMessages(self, parameters):
        """Modify the messages created by internal validation for each tool
        parameter.  This method is called after internal validation."""
        return

    def execute(self, parameters, messages):
        """The source code of the tool."""
        messages.AddMessage(os.getenv("username") + " welcome to the sample tool")
        foo.hello()
        return

In the SamplePythonToolbox.pyt, the foo module has been imported and the execute method of the SampleTool class has called the hello function from the foo module. This is an effective way of distributing custom Python code as a module and exposing its functionality through ArcGIS geoprocessing tools. Once the SamplePythonToolbox.pyt is created and the side-panel help has been configured for the toolbox through the metadata edited in the Item Description context menu or a custom compiled help file (.chm) has been created, it and its accompanying .xml files must be copied over from the directory where it was created. Using ArcCatalog or the Catalog window, copy the files into the esri/toolboxes directory that exists in the distribution structure. The esri directory and file layout should be the following:

Alt text is required

The new directory structure for the distribution should be the following:

Alt text is required

In order to reflect these changes in our distribution, the setup.py file must be edited.

The new setup.py

Sample code to include setup.py directory changes:

from distutils.core import setup
setup(name='foo',
    version='1.0',
    packages=['foo'],
    package_dir={'foo': 'foo'},
    package_data={'foo': ['esri/toolboxes/*.*']},
    )

The new setup.py differs from the original by one line where the additional data found within the esri directory is added to the package. Now when the builder for the foo module is executed and installed, the following directory structure will be created in the Python site-packages directory:

Alt text is required

Using ArcGIS for Desktop and the Python Distribution Utilities (Distutils), it is possible to build and install a package that extends geoprocessing with custom tools in custom toolboxes that can be viewed and executed from within the ArcGIS system toolboxes. For English language distributions, this is all that is needed. The Internationalization topic expands on the process of extending geoprocessing to utilize the same methodology used by Esri to package the module for distribution in languages other than English.

Related Topics

3/3/2014