Designing effective data models for field data collection

Geographic information system (GIS) maps consist of operational and basemap layers. Operational layers are those used for editing and to identify features. Basemap layers provide visual background reference information. For example, a field project that involves collecting the XY locations and attributes of newly placed water valves in a subdivision have a water valves operational layer. This is the layer that is edited in the field. You can also have a water pipes operational layer used to identify individual water pipes to help the field worker. Underneath these operational layers, you may want to include a basemap layer that involves aerial imagery on this operation area to show streets, parks, and other background points of interest.

For field projects, keep the map as simple as possible. Avoid unnecessary layers and other advanced ArcGIS for Desktop behaviors. Consider the constraints of your mobile device and the field conditions where the mobile map is used.

Geodatabase support

Mobile service and mobile cache created by the Create Mobile Cache tool, ArcGIS for Windows Mobile supports editing of data within file-based and multiuser ArcSDE geodatabases. Your mobile map can contain layers that reference other data sources, but to make a layer editable, its data source must at least be a feature class stored within a geodatabase. For more details about the requirements of an editable layer, read the Data schema requirements for purposes of operational layers section below.

NoteNote:

Also note that a feature class cannot carry field names that are reserved by the SQL language.

A hosted feature service has less restrictions than a mobile service in terms of requirements to the data source. Your data can come from a file-based geodatabase, an ArcSDE geodatabase, CSV, or shape files. Data from these sources is editable by the field applications if you publish it as hosted feature services. However, when a set of data is published as a hosted feature service, the data is copied to the ArcGIS Online for organizations or Portal for ArcGIS online storage; therefore, any edits made by field crew will not go back to the source database, but to the online storage.

Data schema and purposes of operational map layers

When designing schema for operational map layers, consider the purposes intended for each map layer and note the schema requirements of a layer. You also need to consider the properties of map layer fields.

Data schema requirements for purposes of operational layers

The purposes you can specify for an operational layer are described in the following list. A map layer can be used for only one of the following purposes:

  • Collecting data—For mobile services, the type of data repository used can potentially affect how your field crew performs data collection. If the data comes from a file geodatabase, after you host it on ArcGIS Server, the data becomes read-only even though you have added GlobalID to it. In this case, you can still use the Create Mobile Cache tool to generate a mobile cache and add the cache to your project for data collection. If, however, your data is from an ArcSDE database and you do not register the data to the server before publishing, your data becomes read-only as well. However, these restrictions do not apply to feature services, that is, data is still editable even it has no GlobalID or stored in a file geodatabase.
  • Providing read-only reference information—There's no restriction to the schema of a read-only layer.
  • Identifying the current user—An identity layer must be a point layer with two text fields: one for storing user identity and one for storing the user display name. If the layer is not to be edited by field workers, the GlobalID is not required. If the layer is to be edited, you can host it to an ArcGIS server as a mobile service, or on ArcGIS Online for organizations or Portal for ArcGIS as a feature service, or cache it using the Create Mobile Cache tool.
  • Logging layer—A logging layer must be an editable point layer, it must have a date/time field, a text field for a machine name (or identity information if you have an Identity layer). You can host the layer to an ArcGIS server as a mobile service, to ArcGIS Online for organizations or Portal for ArcGIS as a feature service, or cache it using the Create Mobile Cache tool.
  • Field crew layer—Similar to a logging layer, a field crew layer must be an editable point layer, it must have a date/time field, and a text field for the machine name or identity information. Since the layer synchronizes and refreshes all field workers' last known positions on a regular basis, it must come from a mobile service/feature service, not from a local mobile cache.
  • Hidden from user—There's no restriction to the schema of a hidden layer.
For more information about layer purpose, see Operational layer properties.

Map layer fields and field properties

Geodatabase behavior supported in ArcGIS for Windows Mobile applications include the following:

  • Subtypes—Used to classify features stored within a single feature class. For example, tree features can be classified by the tree type. When creating a new tree feature in the field, the field worker can choose the tree type to create, thereby eliminating the need to enter the type as a separate attribute.
  • Domains—Both coded value and range domains are supported. Each coded value domain appears as a pick list when collecting or updating attribute values. Range domains indicate the range of valid values that can be entered for a numeric and/or date attribute.
  • Default values—When a default value is assigned for an attribute field, field applications automatically populate the field with the default value. For example, a height attribute field for a tree can be set to 10. The height value for any new tree features will automatically be assigned 10. If the tree is shorter or taller than 10, your field crew can update the height field. Any attribute field of a feature class can be assigned a default value.
  • Attachments—Enabling a feature class to store attachments provides a way to manage information related to features. Attachments must be added to your geodatabase feature classes before you can use them in the field. A common use of attachments is to store a digital photo of a feature. You can collect and view multiple attachments for a single feature.
  • Raster/BLOB (not supported by hosted feature service)—Fields of type raster/BLOB are recognized as a special field type that can be used to store photographs. You can browse for photos and add them to your raster field, or use the embedded camera in your mobile device directly. Note that for each feature, you can only have one picture per raster/BLOB field.
    CautionCaution:

    To use a BLOB field to store images, use esri_picture or picture_ as the prefix for the field's name. For example, esri_picture and picture_WaterValve are acceptable.

  • Dates—Attribute fields can store date value. In the field, your field crew can edit a date/time field by selecting from the calendar that pops up.
  • Phone numbers—If you use a text field to store phone numbers and format your phone number to (XXX)XXX-XXXX so that it's recognized by a Windows Mobile device that has an embedded cellular chip, you can start the phone and call the number directly from within the ArcGIS application.
NoteNote:

Feature classes that are z- or m-enabled can be edited, but z- and m-values will not be maintained. For newly collected features, z- and m-values will not be set.

ArcGIS for Windows Mobile does not allow you to create a new map layer or alter the schema of attribute fields on a mobile device. If you anticipate the need to capture features that do not exist inside your current data model, or if you need to capture unstructured information such as field notes, it is recommended that you create an additional feature class inside your geodatabase schema to handle the storage of this information.

Geodatabase design considerations

NoteNote:

If you plan to publish your map as a feature service, skip the rest of this section.

If you have an existing multiuser geodatabase that you want to use for a field project, do one of the following:

The following sections describe the geodatabase replication and ETL models for managing field editing applications using existing geodatabases.

Geodatabase replication model

You can publish the contents of your multiuser geodatabase for use in the field and use versioning to isolate the edits that come in from the field from other edits that are made in the office. However, there are a number of challenges with this approach. For example, if you need to synchronize updates from the field, your production geodatabase needs to be accessible outside your company's firewall. For most organizations, this is not possible. A better approach is to use geodatabase replication and isolate the information that is captured in the field from the information that is continually updated in the office.

Using geodatabase replication, you can create a separate enterprise geodatabase instance that stores edits made in the field and periodically synchronizes updates with the parent geodatabase. Using this approach, you only need to replicate the information that you need to take to the field (not the entire information geodatabase), and you can isolate mobile web services from your master production geodatabase. Some examples of where replication is useful include a distributed system where remote offices have field workers and do not have connectivity to the main office or when a vehicle-mounted laptop contains a replicated geodatabase and field edits are synchronized when the device is docked in the vehicle. In both examples, metadata can be stored for each field editing task that does not belong in the enterprise geodatabase (for example, Global Positioning System [GPS] measurement data required for differential correction of collected data).

Editing a replicated geodatabase created for managing field edits

ETL model

Quite often, how you represent spatial information in an enterprise database is different than how you create and update it in the field. Modeling lake polygons as shoreline line features so they can be captured in pieces is one example. Another is to join normalized data tables or attribute fields stored in the enterprise geodatabase into one table or attribute in the field geodatabase. Street attribute information is another good example. Often the proper street name is stored in multiple attributes (number, prefix, name, suffix, and so on). In ArcMap, you use an expression to label the full street name. To display the street name on a mobile device, join those attributes together into one field that you can label against.

Geoprocessing models can be used to manage the ETL processes between mobile and enterprise geodatabases. You can also use the ArcGIS Data Interoperability extension to visually design these transformation processes. It's important to note that the processes you define may not be bidirectional. Define one set of geoprocessing models or custom spatial ETL tools for transforming your data model to a mobile device and a second set of models to reassemble field data to fit the enterprise schema.

Mobile transaction model

How you plan to manage field updates and integrate them with your geodatabase is embodied in your transaction model. In part, this is defined by the geodatabase design decisions you make, but it's also defined by the number of field editors you need to support and how isolated you want their edits to be.

Some of the transaction model considerations include the following:

The following sections outline some of the key functionality you should consider when designing your geodatabase for field use.

Editing a non-versioned geodatabase

If you have only a few field editors that are performing simple editing tasks such as updating attributes, and there is little to no chance that they will update the same feature in the field, a non-versioned transaction model may best fit your needs. One potential drawback of this approach is that direct update is the only option available to the field editor. If for some reason updates need to be synchronized but are not complete, synchronization becomes problematic. With versioning, you have more flexibility in how your field workers can synchronize updates.

Field edits directly update the geodatabase

Using a versioned transaction model, you can isolate field edits and add additional postprocessing and quality assurance checking before reconciling updates. Depending on how you want to isolate field edits, you could create a single version to store all field edits or you could create a version for each field editor. If you create a version for each editor, you'll need to publish a service for each editor. Once the editors have completed their data capture or maintenance, and synchronized their updates with the geodatabase, ArcGIS for Desktop is needed to reconcile the version edits with the parent version when in the office.

Diagram of reconciling the version edits

ArcGIS for Windows Mobile client editing framework

ArcGIS for Windows Mobile applications do not have a concept of start, stop, and save edits as in other ArcGIS applications. Each edit that's made is stored within the mobile cache on the device until the time to synchronize updates with the server. You can cancel an edit that you've made within the application, which rolls back all updates made in the field and restores the original state of the feature before editing, but you cannot undo one edit to a feature at a time.

Updating the feature layer does not synchronize your changes with the geodatabase directly.

Posting changes

The updates that you perform in the field are stored locally in the mobile service cache on the mobile device. This is important because your field workers may not be connected in the field at all times, or they may need to shut down and charge their devices, and it is important that updates are not lost. When connectivity with the server is established, you can then synchronize updates stored in the cache with the back-end database.

When posting changes from the mobile device, only changes, also known as "deltas," are sent to the server. For example, if you change an attribute on a feature, only the change to that specific field is recorded rather than marking the entire row as edited. This is done so that when synchronizing changes, only the information that has actually changed is sent back to the server.

Depending on the number of edits you expect and the type of data connection you have (general packet radio service [GPRS], for example), you may want to enable posting features only when the application and device are back in the office to ensure a stable high-speed connection.

1/7/2015