Geoprocessing overview

Defining geoprocessing

The fundamental purposes of geoprocessing are to allow you to automate your geographic information system (GIS) tasks and to perform spatial analysis and modeling. Geoprocessing is a universal capability (available in ArcGIS for Desktop, ArcGIS Engine, and ArcGIS for Server) that can be used and deployed by all GIS users to automate their work, build repeatable and well-defined methods and procedures, and model important geographic processes. Geoprocessing is cross-language (Python, .NET, Java, C++) and cross-platform (UNIX, Linux, Windows, depending on the language used).

Geoprocessing and ArcObjects

ArcObjects is the extensive library of low-level programming objects on which the ArcGIS software is built and are available for developers to use. Developers use ArcObjects to build new applications or extend the existing functionality of ArcGIS applications. In fact, most system tools and the entire geoprocessing framework were built using ArcObjects. Like geoprocessing, the ArcObjects software development kit (SDK) can be used to create software.

ArcObjects and geoprocessing are complementary (neither obsoletes the other). Generally, ArcObjects is used to extend ArcGIS with new behavior, while geoprocessing automates tasks. Use ArcObjects to, for example, add new user interfaces (UIs), add custom behavior to feature classes, or create a special cartographic renderer. Geoprocessing is used to create software (models and scripts) that automates tasks within the confines of a well-behaved framework. In addition, ArcObjects and geoprocessing can be used together, and you can leverage geoprocessing tools from within .NET code.

ArcObjects is meant to be used with a system programming language when the programmer needs to access low-level primitives to implement complex logic and algorithms. This is why ArcObjects contains thousands of different objects and requests, to allow the programmer the fine degree of control required. Because ArcObjects is used in concert with a system programming language, it requires a good deal of programming knowledge, much more than geoprocessing, with its models and scripts.

There are two parts to geoprocessing, the suite of tools and a framework for creating software that uses these tools.

Geoprocessing tools

Geoprocessing tools have been evolving since before modern GIS technology. In those days, when geographic analysts were trying to solve real-world problems, they would gather around a whiteboard (or a chalkboard) and create flowcharts and diagrams very similar to those that GIS analysts create today using ModelBuilder. The tasks in those early flowcharts evolved into software and have become the core set of geoprocessing tools you see today. This was an iterative process lasting many years (and is still ongoing) that has refined the tools into the set of simple and elemental operators that are installed with ArcGIS.

There are literally hundreds of tools, and it takes time to learn what tool to use and when to use it. In your code, you can use model and script tools developed by geoprocessing-savvy analysts. For example, you, or any GIS analyst, can create a model or Python script tool, and you can call the model or script tool from your .NET code. Finally, you can benefit from an analyst's knowledge to find the right combination of tools to use.

Geoprocessing framework

Geoprocessing is based on a framework of data transformation. A typical geoprocessing tool performs an operation on an ArcGIS dataset (such as a feature class, raster, or table) and produces a new dataset (or updates an input) as the result of the tool. Each geoprocessing tool performs a small, yet essential operation on geographic data, such as projecting a dataset from one map projection to another, adding a field to a table, or creating a buffer zone around features. ArcGIS includes hundreds of such geoprocessing tools.

The core idea behind the geoprocessing framework is to allow you to quickly and easily turn your ideas into new software that can be executed, managed, modified, documented, and shared with the ArcGIS user community. Software, in this case, means something that instructs ArcGIS to do what you want.

Using ModelBuilder

A geoprocessing model, for example, is new software built by you with an easy-to-use visual programming language called ModelBuilder. It does not require any programming experience. ModelBuilder is how GIS analysts quickly and easily turn your ideas into software by chaining together elements of the geoprocessing language (the tools) into a sequence. It is important to understand that models are software, since they instruct the computer to do something. The programming language is visual—rather than text-based like a traditional programming language. The power of ModelBuilder lies in creating generic tools. They behave exactly like other tools in the system. You can call these model tools in your .NET code. For more information to help you get started, see the ArcGIS Desktop Help topic A quick tour of creating tools with ModelBuilder.

Using Python

You can also use a scripting language to create useful software. A program that uses a scripting language is a script. In the world of software programming, languages can be divided into two basic categories, system languages and scripting languages. System languages, such as C++ and .NET, are used to create applications from scratch, using low-level primitives and the raw resources of the computer. Scripting languages, such as Python and Perl, are used to glue applications together, using built-in higher-level functions of the computer and masking the fine-grain details a system language programmer must deal with. Compared to system languages, scripting languages are easier to learn and use. A basic understanding of programming is all that is needed to be productive.

ModelBuilder is one way that ArcGIS for Desktop users create software. The other choice is to use Python scripting. Compared to ModelBuilder, Python is infinitely more powerful for the following reasons:

Python supports advanced programming logic, such as conditional execution and advanced error handling; more advanced data structures, such as dictionaries and lists; and more functionality, such as string, math, and file manipulation functions. Python has been extended with third-party packages for, for example, advanced math and statistics, Web automation, database queries, and advanced system utilities. For ArcGIS, Python has been extended with the ArcPy package.
The ArcPy site-package contains low-level geoprocessing functions. Cursors, for example, let you loop through records on a table, reading or writing rows, and inserting new rows. There are functions to access the properties of ArcGIS data, such as the extent of a feature class or the sundry properties of individual fields on a table.
Python is great for the gluing together of application components. For example, if you have a model written in ModelBuilder, you could use Python to wrap this program, introduce this script to a toolbox, and use it directly in ArcGIS for Desktop or in .NET.
Scripts can be executed outside ArcGIS for Desktop. That is, you can execute the script directly from the operating system prompt. You still need to have (at least) the ArcGIS Engine and the ArcPy site-package installed on the machine since you need access to the geoprocessing tools.
You can take advantage of the power of script tools by customizing tool behavior by enabling and disabling parameters, providing default values, and updating string keywords. For more information, see Customizing script tool behavior.

You can call a Python script from .NET without having the ArcGIS application open. For more information, see Leveraging ArcPy in a .NET application. Although you need ArcGIS for Desktop to create these models and scripts, you can use them outside of the desktop application if you have ArcGIS Engine installed.

In the geoprocessing framework, scripts can be used to create tools. For more information, see A quick tour of creating script tools.

These model and script tools can be added to a custom toolbox. You can also add these tools on menus and toolbars. For more information, see Adding and removing tools on menus and toolbars.

To use the model or script tools in a .NET application, add the tools to a custom toolbox in ArcMap. In .NET, add the toolbox to the geoprocessor using the AddToolbox method. Once the toolbox is included, you can call this model from a .NET application. For code examples in C# and VB .NET for this step, see the "Executing a custom tool" section in How to run a geoprocessing tool.

Summary

The geoprocessing framework is built to let you quickly and easily turn your ideas into new software that you can call from .NET using the ArcObjects SDK for the Microsoft .NET Framework.

Geoprocessing is a language consisting of operators or tools that operate on the data within ArcGIS (for example, tables, feature classes, rasters, triangulated irregular network [TINs], and so on), and perform tasks that are necessary for manipulating and analyzing geographic information across a wide range of disciplines.

You can quickly and easily create software in the form of models and scripts. These new tools perform tasks that are not part of the standard ArcGIS package.

Tools are managed by the geoprocessing framework. This is a subtle but important point that is not immediately obvious.

All tools, whether they are system tools or custom (user-written) tools, can be accessed from their toolbox or from a toolbar. Imagine a different situation where models, scripts, and system and custom tools were each accessed by different interfaces and methods (it would be a nightmare to use and manage). In geoprocessing, all things are created and managed equally, whether they are component tools, model tools, or script tools.
Tools are all documented the same way. Once you create a tool, you can document your tool in the Catalog window so it can be cataloged and searched by the system.
Tools have the same UI, the dialog box. These dialog boxes are automatically created based on the tool parameters. You do not have to do any UI programming. Consider the alternative where UI design and programming are left to the tool author.

Tools can be easily shared. A toolbox with all its tools and toolsets is contained in a file on disk with a .tbx extension or within a geodatabase. Anyone with access to the file or geodatabase can run its tools. For more information, see A quick tour of sharing tools.

The salient point is that your tools become full-fledged components of the geoprocessing framework where they have consistent documentation, UI, methods of access, and methods of sharing.