Inside the compact cache storage format

The compact cache storage format allows you to group tiles in large files rather than storing the tiles as individual files. Advantages of storing tiles in groups include the following:

• Caches are easier to copy because the number of files is reduced.
• The total size on disk of the cache is reduced.
• Tiles are generally created more quickly because disk I/O is reduced during tile creation.
• Scalability is improved when creating tiles with multiple-machine deployments, because of reduced network traffic.

How a compact cache works

The compact cache groups many tiles together in one large file called a bundle. A bundle contains up to 16,384 tiles. The result is a cache with dozens or hundreds of files, instead of thousands or millions. If you look at a compact cache on disk, you can see the bundle files with the .bundle extension. You'll also see some corresponding index files with the extension .bundlx.

While the cache is being created, you may see .lock and .done files in the cache folders. The .lock files are the server's way of keeping track of which bundles are currently being created; the presence of a .lock file doesn't mean that the bundle is inaccessible to clients. Similarly, the .done file is the server's way of understanding which bundles have finished. All the .lock and .done files should go away when the caching job has finished.

It's possible to have a small cache with just one bundle at each level. More common is that you have a bundle boundary crossing some portion of the geography, so you get multiple bundles in a level (although the bundles might not contain the full 16,000 tiles if the geography is small). Large caches encompass many bundles.

The bundle boundaries are determined by the tiling scheme origin and are not adjustable. For reference, at the neighborhood/street level scale of 1:4096, a full bundle covers about the area of a midsize county in the eastern United States.

Legacy:

In 10.0 and earlier versions, if you used a feature smaller than a bundle to define the boundaries of your cache job, you could see CPU underutilization. In 10.1 and later releases, the software has more intelligence about how to allocate CPU resources to the cache job, and the geographic size of the feature defining your cache job should not affect CPU utilization.

How updates occur on a compact cache

When you update tiles in a compact cache, the entire bundle is not re-created. Instead, a finer-grained area of 4096 x 4096 pixels (no antialiasing) or 2048 x 2048 pixels (antialiasing) is updated. In ArcGIS documentation, this unit of area is sometimes referred to as a supertile.

Getting tiles from the bundle

ArcGIS clients, including the web APIs, know how to read the bundle files produced by the compact cache format. In a web situation, the client issues a call to the server for the specific level, row, and column of the tile. The server receives the request and returns the appropriate tile from the bundle.

The internal architecture of the bundle is not publicly documented by Esri. If you've coded your own logic to pull tiles out of a virtual directory, you should continue to use the exploded format, which stores each tile as a single file and was the only option at ArcGIS Server 9.3.1 and previous.