Working with NumPy in ArcGIS
Numerical Python (NumPy) is a fundamental package for scientific computing in Python, including support for a powerful N-dimensional array object. NumPy provides an avenue to perform complex mathematical operations and has been part of the ArcGIS software installation since 9.2. For more information, see the NumPy website.
A Python NumPy array is designed to deal with large arrays. There are many existing Python functions that have been created to process NumPy arrays, the most noted being contained in the SciPy scientific computing package for Python.
Working with tables and feature data
Table and feature classes can be converted to and from NumPy arrays using functions in the data access (arcpy.da) module.
To convert NumPy arrays to tables and feature classes, the arrays must be structured arrays. Structured arrays include fields (or structs) that are used to map the data to field in ArcGIS table and feature classes. For more information on structured arrays, see Structured arrays.
Create a structured NumPy array.
import numpy array = numpy.array([(471316.383, 5000448.782), (470402.493, 5000049.216)], numpy.dtype([('X', '>f8'),('Y', '>f8')]))
Once created, a structured NumPy array can be converted to a feature class or table.
Convert a NumPy array to a geodatabase feature class.
import arcpy import numpy outFC = "C:/data/texas.gdb/fd/pointlocations" # Create a numpy array with an id field, and a field with a tuple # of x,y coordinates array = numpy.array([(1, (471316.3835861763, 5000448.782036674)), (2, (470402.49348005146, 5000049.216449278))], numpy.dtype([('idfield', numpy.int32),('XY', '<f8', 2)])) # Define a spatial reference for the output feature class SR = arcpy.Describe("C:/data/texas.gdb/fd").spatialReference # Export the numpy array to a feature class using the XY field to # represent the output point feature arcpy.da.NumPyArrayToFeatureClass(array, outFC, ['XY'], SR)
Join the contents of a NumPy structured array to a table based on a common attribute field.
Convert a feature class to NumPy structured array.
Convert a NumPy structured array to a feature class.
Convert a NumPy structured array to a table.
Convert a table to NumPy structured array.
Integer fields in NumPy arrays don't support nulls. If data converted using FeatureClassToNumPyArray or TableToNumPyArray contains nulls, the rows containing the nulls should either be skipped entirely or masked with a substitute value.
Skip all records that include a null.
array = da.FeatureClassToNumPyArray(fc, fields, skip_nulls=True)
Mask Nones in integer fields with different values using a dictionary.
fields = ['field1', 'field2'] arcpy.da.FeatureClassToNumPyArray(fc, fields, null_value=-9999)
The dtypes of the created array are determined from the field type of the input table or feature class.
<u1, <u10, and so on
String fields converted to an array will have the same width. For instance, a string field with a width of 20 will have a dtype of <u20.
Other field types not listed above, including date, raster, and BLOB fields are not supported. Geometry fields are also not supported, but multiple geometry properties can be added to the array using the special tokens listed below.
The feature's centroid x,y coordinates
The feature's true centroid x,y coordinates
The feature's x-coordinate
The feature's y-coordinate
The feature's z-coordinate
The feature's m-value
The feature's area
The feature's length
NumPy preallocates memory when creating arrays. An array that requires more memory than is available will fail with a MemoryError exception.
Tips to avoid MemoryError exceptions:
- Delete array objects after use; deleting the array will release the memory.
- Use only those fields you need, especially text fields; a text field converted to an array will consume 4 bytes for every character of width. For instance, a string field with a width of 100 will consume 400 bytes of memory for each value in the array.
Number of bytes per value
numpy.nbytes returns a dictionary of dtypes and number of bytes.
Working with rasters
Raster can be converted to and from NumPy arrays using the ArcPy functions RasterToNumPyArray and NumPyArrayToRaster. You may want to convert an ArcGIS raster to a NumPy array to:
- Implement one of the many existing Python functions that can be applied to a NumPy array (for example, run filters on the data, perform multidimensional analysis, or utilize optimization routines).
- Develop a custom function by accessing the individual cells within the NumPy array (for example, to implement neighborhood notation, change individual cell values, or run accumulative operators on an entire raster).
Convert a raster to a NumPy array.
Convert a NumPy array to a raster.
A raster is converted to a NumPy array to calculate the percentage of the cell value in the entire raster row. A new raster is then created.
import arcpy import numpy my_array = arcpy.RasterToNumPyArray('C:/data/inRaster') my_array_sum = my_array.sum(1) my_array_sum.shape = (my_array.shape, 1) my_array_perc = (my_array * 1.0) / my_array_sum new_raster = arcpy.NumPyArrayToRaster(my_array_perc) new_raster.save("C:/output/fgdb.gdb/PercentRaster")