A quick tour of managing intermediate data
When a model is run, output data is created for each process in the model. Some of this output data is only created as a middle step to connect to other processes that will create the final output. The data generated from these middle steps, called intermediate data, is often (but not always) of no use once the model has finished running. You can think of intermediate data as temporary scratch data that should be deleted after the model has run. However, when you run a model from the ModelBuilder window, intermediate data is not deleted—it is up to you to delete it.
Learn how to delete intermediate data
All the data variables except existing data and the final output variables are automatically marked as intermediate in a model. To determine if a data variable in a model is considered intermediate or change the status of a variable that is intermediate, right-click the variable and click Intermediate to either add or remove the check mark next to it.
Note:To prevent the final output variables from being deleted, It is strongly recommended that you do not set the final output variable as intermediate data.
Learn more about preserving intermediate data
Controlling where intermediate data is written
Geoprocessing tools that create new output datasets autogenerate the name of the output dataset (as described in Using the current and scratch workspace environments). This autogenerated name includes the path to the workspace or folder containing the dataset. ModelBuilder uses this autogenerated output name for writing intermediate data. You can always override this output dataset name by entering another name and location for the output dataset, and there are ways you can specify the workspace used to construct the autogenerated output name.
When you're working in ModelBuilder, it is easy to ignore this autogenerated output name, especially for intermediate data. But there are situations where you want to control where intermediate data is written. You can write the intermediate data to any of the following locations:
- Any workspace (geodatabase, geodatabase feature dataset, or folder) by right-clicking intermediate data variables and changing the location where they are written. If you have a lot of intermediate data, you can make your intermediate data managed data, as described in Making intermediate data managed data.
- To the Scratch GDB (Scratch Geodatabase) or Scratch Folder. These two read-only environments were introduced in ArcGIS 10.1. The illustration below shows the Centroids Intersect data variable flagged as intermediate, and its output location is the scratch geodatabase (%scratchGDB%). The percent signs (%) denote variable substitution—the value of scratchGDB is expanded when the model tool is run, and a feature class named poly_Intersect is written to the scratch geodatabase. Similarly, you can use %scratchFolder% to write file-based data, such as .lyr or .txt files.
- To the In-memory workspace.
- To the Scratch Workspace using %scratchworkspace%. This environment can be set by you (or the user of your tool) to any location—a folder, a geodatabase, or even a feature dataset within a geodatabase. Using the Scratch Workspace environment with tools that are to be shared is not recommended, because the user of your tool can set the Scratch Workspace environment to a folder, geodatabase, or feature dataset. For example, you may expect Scratch Workspace to be set to a folder so you can output a layer file, but the user of your tool sets Scratch Workspace to a geodatabase. When your tool is run, it fails because you cannot write your layer file to a geodatabase. This is the main reason that Scratch Geodatabase and Scratch Folder environments were introduced at 10.1—to give you a known geodatabase and a known folder to write your data.
There are a few recommendations for writing the intermediate data in a model as described below:
- You never want to write intermediate data to an ArcSDE geodatabase. This causes performance problems, since you are writing temporary scratch data to an enterprise database across a local area network. It's suggested that you always use a file geodatabase (rather than a personal geodatabase or a shapefile workspace) for the scratch workspace.
- You want to avoid cluttering a working geodatabase with intermediate data. Rather, you should designate another workspace, preferably a file geodatabase, where intermediate data is written.
- When writing intermediate feature classes, you may be tempted to write shapefiles to the scratch folder. You should avoid this practice and write feature data to the scratch geodatabase, as shapefiles have some fairly severe limitations that can affect portability of your tools. See Geoprocessing considerations for shapefile output for more information on shapefiles and their limitations.