Managing intermediate (scratch) data in shared model and script tools

Scratch data is data used by your tool and deleted after your tool is run. In ModelBuilder, scratch data is known as intermediate data. Data that is marked as intermediate in ModelBuilder is automatically deleted after your model tool is run. In scripts, however, you are responsible for deleting scratch data within your script.

Note:

If you are sharing your tool using a geoprocessing package or service, you do not have to make any modifications to your models or scripts to manage scratch or intermediate data. The techniques described below only apply when you share your custom toolbox—that is, you are not sharing a package or service but are giving someone your toolbox (.tbx file). In this case, you need to ensure that your tools are portable by writing scratch and intermediate to a location that exists on any computer that executes your tool.

Whenever you share a toolbox containing your model and script tools with others, you need a location—a folder or geodatabase—where your tools can write your scratch or intermediate data. There are two read-only environments, Scratch GDB (Scratch Geodatabase) and Scratch Folder, available for you to write intermediate and scratch data. These two environments were introduced in ArcGIS 10.1.

Intermediate data in models

All intermediate data should be flagged as such and written to either the scratch folder or scratch geodatabase. The illustration below shows the Centroids Intersect data variable flagged as intermediate, and its output location is the scratch geodatabase (%scratchGDB%).The percent signs (%) denote variable substitution—the value of scratchGDB is expanded when the model tool is run and a feature class named poly_Intersect is written to the scratch geodatabase. Similarly, you can use %scratchFolder% to write file-based data, such as .lyr or .txt files.

Learn more about intermediate data in modelsLearn more about variable substitution in modelsExamples of inline model variable substitution

When writing intermediate feature classes, you may be tempted to write shapefiles to the scratch folder. You should avoid this practice and write feature data to the scratch geodatabase, as shapefiles have some fairly severe limitations that can affect portability of your tools. See Geoprocessing considerations for shapefile output for more information on shapefiles and their limitations.

You can also write intermediate data to the in-memory workspace.

Learn more about the in-memory workspace

Managing scratch data in script tools

Scratch data in script tools should be written to either the scratch geodatabase or scratch folder. The code below shows copying a feature class to the scratch geodatabase and deleting it when finished.

import arcpy
import os
inFC = arcpy.GetParameterAsText(0)

tempFC = arcpy.env.scratchGDB + os.path.sep + "tempFC"
arcpy.CopyFeatures_management(inFC, tempFC)

# Do some work here...

# Clean up when done...
#
arcpy.Delete_management(tempFC)

Within a script, you can also write data to the in-memory workspace. For example:

import arcpy

table = arcpy.CreateTable_management("in_memory", "table1")
arcpy.AddField_management(table, "Field1", "TEXT", field_length=20)

cursor = arcpy.da.InsertCursor(table, ["Field1"])
cursor.insertRow(["Hello World"])

The in_memory workspace is only valid for geoprocessing tools; it is not a general-purpose virtual directory where you can write any data.

Scratch workspace

The Scratch Workspace environment is used primarily by ModelBuilder as a location to write intermediate and output data. This environment can be set by you (or the user of your tool) to any location—a folder, a geodatabase, or even a feature dataset within a geodatabase. Using the Scratch Workspace environment with tools that are to be shared is not recommended, because the user of your tool can set their Scratch Workspace environment to a folder, geodatabase, or feature dataset. For example, you may expect Scratch Workspace to be set to a folder so you can output a layer file, but the user of your tool sets Scratch Workspace to a geodatabase. When your tool is run, it fails because you cannot write your layer file to a geodatabase. This is the main reason that Scratch Geodatabase and Scratch Folder were introduced at 10.1—to give you a known geodatabase and a known folder to write your data.