Recommendations when migrating large schematic datasets

Before migrating a schematic dataset containing a large number of schematic diagrams, it is recommended that you check the size of the schematic diagrams you want to migrate and analyze how the schematic dataset is configured.

Migration is time-consuming and memory-consuming in the following situations:

• You have a large number of diagrams to migrate.
• The diagrams you want to migrate contain thousands of schematic features.
• There are schematic attributes whose type are Query, Function, Formatted, or Script on your schematic feature classes.

For those situations, it is strongly recommended that you migrate your schematic dataset as explained in the Migrating large schematic datasets step by step section above.

The About performances when executing parallel processes for migrating schematic diagrams section gives you some recommendations about the number of machines you need to set up and the number of processes you can execute in parallel to migrating a large number of schematic diagrams within an acceptable amount of time.

Migrating large schematic datasets step by step

Phase#1—Preparing the schematic dataset you want to migrate

This first phase consists of analyzing how the pre-10 schematic dataset you want to migrate is configured and organized and prepare it for efficient migration:

1. Start ArcCatalog on a pre-10 client where the ArcGIS Schematics extension is installed.
2. Browse and select the pre-10 schematic dataset you want to migrate and edit it within Schematic Designer.
3. Analyze each schematic feature class definition and check whether there are Query, Function, Formatted, or Script type attributes on the schematic feature class.
4. As described in the Preparing pre-10 schematic datasets topic:
   • Verify the spatial reference set for each schematic feature class.
   • Configure the ContainerMark attribute for any node schematic feature class you want to be migrated as a container.
5. Optionally, create temporary schematic folders in your pre-10 schematic dataset and move subsets of schematic diagrams into the temporary folders.

Phase#2—Migrating the schematic dataset definitions

This second phase consists of migrating the schematic dataset definitions:

1. Start ArcCatalog.
2. Right-click the pre-10 schematic dataset you want to migrate in the Catalog tree and point to Migrate To The Current Schematics Version.
3. Specify the geometry type for the schematic node and drawing classes to migrate in the Select Geometry Type dialog box that opens and close the dialog box.
4. In the Migrate To The Current Schematics Version dialog box that opens:
   • Specify the output schematic dataset into which you want to migrate.
   • Check Definitions in the Output Activity section.
   • Click OK.

The migration process starts. When it completes, the schematic dataset definitions are migrated.

Phase#3—Migrating different subsets of schematic diagrams at the same time

This third phase focuses on the migration of the schematic diagrams contained in the pre-10 schematic dataset.

It consists of running several parallel migration processes on different machines, with each migration process working on different subsets of schematic diagrams.

On each 10.x machine, you must launch one or several migration processes as follows:

1. Start ArcCatalog.
2. Right-click your pre-10 schematic dataset in the Catalog tree and choose Migrate To The Current Schematics Version.
3. Close the Select Geometry Type dialog box that opens.
4. In the Migrate To The Current Schematics Version dialog box that opens:
   • Specify the output schematic dataset into which you want to migrate.
   • In the Output Activity section, do the following:
     • Check Diagrams.
     • Check Specify filters and configure the SQL filters to use to determine the subset of schematic diagrams this process will migrate; for example, build an SQL expression based on a FOLDERID corresponding to a temporary folder you created during Phase#1.
     • For those schematic diagrams that contain schematic features with Query, Function, Formatted, or Script-type attributes, uncheck With attribute evaluations.
       Nota:

       If you have no Query, Function, Formatted, or Script schematic attributes in your schematic dataset definition, the With attribute evaluations check box can be checked. Running migration with or without this option checked has no impact on the time required to migrate your diagrams in this case.

   • Click OK.

The migration process starts. When it completes, the subset of schematic diagrams is migrated in the output schematic dataset.

Nota:

During each migration process, when Schematics is migrating a particular diagram, it checks to see if this diagram already exists in the output schematic dataset. This diagram is only migrated when it is missing in the output schematic dataset.

Phase#4—Updating the attributes on the schematic features in the migrated schematic diagrams (optional)

If you uncheck the With attribute evaluations option when migrating a subset of diagrams, none of the Query, Function, Formatted, or Script-type attributes defined for the schematic features in the migrated diagrams are evaluated. In this case, you need to update those attributes afterward.

For schematic diagrams based on the Standard builder or configured from custom queries, the following Python sample script exemplifies how those attributes can be easily and quickly updated. It is based on the Update Diagrams GP tool and works in different situations:

• Case A—Refreshing attributes on all schematic diagrams you migrated into a particular schematic folder
• Case B—Refreshing attributes on all migrated schematic diagrams that are based on a specified diagram template
• Case C—Refreshing attributes on all migrated schematic diagrams that have not been updated for N days

# Name: RefreshingAttributesInMigratedDiagrams.py
# Description: Refreshing attributes on schematic features contained in migrated diagrams
# Requirement: ArcGIS Schematics extension

# import system modules
import arcpy
msgNoLicenseAvailable = "ArcGIS Schematics extension license required"

try:
    # Checks out the ArcGIS Schematics extension license
    if arcpy.CheckExtension("Schematics") == "Available":
        arcpy.CheckOutExtension("Schematics")
    else:
        raise Exception(msgNoLicenseAvailable)

    # Sets Schematics general settings
    dataLocation="C:/ArcGIS/ArcTutor/Schematics/Schematics_In_ArcMap"
    gdbName="ElecDemo.gdb"
    in_schDataset="ElecDemo"
    in_schFolder="MyTemporaryFolderName"
    in_diagTempName="GeoSchematic"
    updateOption="REFRESH"
    recOption1="RECURSIVE"
    nbDays="60"

    # Sets environment settings
    arcpy.env.overwriteOutput = True
    arcpy.env.workspace = dataLocation + "/" + gdbName

    # Case A - Refreshing attributes on schematic diagrams that were migrated on a particular schematic folder.
    arcpy.UpdateDiagrams_schematics(in_schDataset + "/" + in_schFolder1,"REFRESH")

    # Case B - Refreshing attributes on all migrated schematic diagrams that are based on a specified diagram template.
    arcpy.UpdateDiagrams_schematics(in_schDataset, "REFRESH", "RECURSIVE", in_diagTempName)

    # Case C - Refreshing attributes on all migrated schematic diagrams that have not been updated for N days.
    arcpy.UpdateDiagrams_schematics(in_schDataset, "REFRESH", "RECURSIVE", "#", nbDays)

    # Returns the ArcGIS Schematics extension license
    arcpy.CheckInExtension("Schematics")

    print "Script completed successfully"

except Exception as e:
    # If an error occurred, print line number and error message
    import traceback, sys
    tb = sys.exc_info()[2]
    print "An error occurred on line %i" % tb.tb_lineno
    print str(e)

This sample script may help you when writing your own script to refresh attributes on the schematic features contained in your migrated diagrams.

Nota:

The REFRESH builder_option is the key option to use in this script. It allows updating diagrams content so that only the schematic feature attributes associated with the network features or the records returned by the custom queries are refreshed. No other changes on these network features or queried records are reflected in the schematic diagram after the update.

Learn more about updating diagrams using the Refresh attributes option

To get the attributes refreshed in your migrated diagrams, do the following:

1. Start ArcCatalog.
2. Copy and paste your script in the Python window.
3. Press ENTER.

Phase#5—Reorganizing your migrated schematic dataset (optional)

In the case where you created temporary schematic folders in your pre-10 schematic dataset to easily migrate your schematic diagrams in each folder, the temporary folders also exist in the migrated schematic dataset. You can now move the schematic diagrams migrated to those folders into the schematic location you want and remove the temporary folders.

About performances when executing parallel processes for migrating schematic diagrams

To migrate a large number of schematic diagrams of considerable size—for example, diagrams containing 2,000 schematic features—it is recommended that you execute several parallel migration processes on different machines, each migration process working on different subsets of schematic diagrams. The following sections detail requirements for the client machines that will support the migration process and the database server into which the migration will be done including recommendations for dimensioning those processes.

ArcGIS Desktop 10.2 client machine requirements

• Processor: Dual-core minimum
• Installed memory (RAM): 4 GB minimum to support the migration of 1,000 large schematic diagrams by executing two parallel processes, each process working on 500 schematic diagrams.

  Running with 8 GB installed memory is recommended for the CPU consumption when executing four parallel processes and migrating 2,000 large diagrams, at 500 diagrams per process.

Database server machine requirements

• Installed memory (RAM):16 GB minimum
• Database size: Considering that 1,000 diagrams containing 2,000 schematic features each require around 1.5 GB disk space, the database must be correctly dimensioned to support the size of the diagrams that will be migrated.

Dimensioning the migration processes to execute in parallel

Migrating pre-10 schematic diagrams contained in file geodatabases into file or SDE geodatabases is quite fast—500 large schematic diagrams are migrated in about an hour. However, migrating diagrams contained in a pre-10 schematic dataset stored in SDE geodatabases into SDE is quite long—500 big diagrams are migrated in about six hours. This time can be doubled—to about 12 hours—in an instance where there are many schematic attributes with a type of Query, Function, Formatted, or Script, and those attributes are evaluated during the migration.

If you have a large number of schematic diagrams to migrate—for example, 10,000 diagrams—you can migrate all of them in one night by doing the following:

• Divide the 10,000 schematic diagrams into 20 subsets—for example, reorganize your schematic diagrams into 20 temporary schematic folders.
• Set up four or five Desktop 10.2 client machines.
• Start four or five parallel migration processes on each Desktop 10.2 client machine, each process working on a particular subset of diagrams, for example, on a specific schematic folder.