Python & arcpy presentations at the 2014 user conference

Once upon a time, there was but a single tech workshop at the Esri international user conference which featured everybody’s favorite scripting language. No longer is the Python offering so paltry. Behold, the 2014 user conference technical workshops & demo theaters returned when you search for “python”.

Consult the online agenda for date and time when these are offered.

Technical workshops

• Python: Getting started
• Python: Beyond the Basics
• Python: Map Automation
• Python Map Automation: An Introduction to arcpy.mapping
• Python Map Automation: Beyond the Basics of arcpy.mapping
• Python: Raster Analysis
• Python: Building Geoprocessing Tools
• Python for Analysis
• Analyzing Multidimensional Scientific Data in ArcGIS
• Automating Geodatabase Creation with Geoprocessing Tools
• Creating Geoprocessing Services
• Customizing ArcPad with JavaScript, Python, and VBScript
• Desktop Mapping: Building Map Books
• ArcGIS Online: Administering your ArcGIS Organization Through Scripting
• ArcGIS for Server Administrative Scripting and Automation
• ArcGIS Network Analyst: Automating Workflows with Geoprocessing
• Integrating Open-Source Statistical Packages with ArcGIS
• Useflul Python Libraries for The GIS Professional

Demo theater presentations

• Administering ArcGIS for Server with Python
• Building Live Data Feeds Using Python
• Consuming Geoprocessing and Hosted Analytic Services in ArcGIS for Desktop
• Debugging Python Scripts
• Esri Production Mapping: Automating Map Production Workflows Using the ArcPyProduction Site Package
• Geoprocessing on Image Services
• Getting Started With Map Algebra Using the Raster Calculator and Python
• Getting Started with Python in ArcGIS
• Interacting with the ArcGIS Server Admin API using Python
• Managing an Online Data Warehouse: Using Python to Publish Data to ArcGIS Online
• Python Add-ins: Tips and Tricks

PyCon 2014 videos are online

The 2014 PyCon US conference ran from April 9 to the 13th in Montreal, Canada.

For those of you looking to expand your knowledge of Python, this can be a good source of tutorial/tech workshops.

All presentations and tutorials are recorded and published here: PyCon 2014 videos

feature class to csv

A few days ago Tony asked for a py script that will write out the OID + x + y for each feature in a polyline feature class / shapefile.  No problem we say, see code example at bottom.

This was quick and fun exercise and we were happy to write this code. But for more timely technical help i’d direct people to the esri forums, or gis stack exchange. 

Cheers and happy coding.

	def export_fc_to_csv(fc, out_csv):
	"""
	Export each vertex in a line or poly fc to a csv with OID
	user requested at https://arcpy.wordpress.com/about/#comment-348
	example
	import geom_snippets
	geom_snippets.export_fc_to_csv(r"c:\proj\fc1.shp", r"c:\proj\fc1.csv")
	output csv looks like this
	1,56019.99998067904,69118.00001450378
	1,56159.99998080942,69026.0000144181
	1,56359.999980995686,68913.00001431286
	2,34985.00002508866,68936.00001433428
	2,35178.000025268404,68805.00001421227
	"""
	import csv
	import json
	with open(out_csv, 'w') as csvfile:
	csvwriter = csv.writer(csvfile, delimiter=',', lineterminator='\n')
	with arcpy.da.SearchCursor(fc,
	field_names=("OID@","SHAPE@JSON")) as cursor:
	for row in cursor:
	geom = json.loads(row[1])
	for path in geom['paths']:
	for pt in path:
	csvwriter.writerow([row[0]] + pt)

view raw export_fc_to_csv.py hosted with ❤ by GitHub

python/arcpy code from the 2014 dev summit

Hey Everyone,

it was great meeting and talking to everybody at the dev summit.

A number of python/arcpy presenters already shared their code to github repos below.

• Mark’s main stage code for Managing ArcGIS Online content with Python
• Kevin’s demo theater code for Administering ArcGIS Server with Python
• Ghis’s demos for the tech workshop on Working with feature data with arcpy
• Ryan’s demos for the tech workshop on Working with Raster data Using Arcpy (hosted on arcgis.com)
• Jason’s tech workshop Accessing C type Libraries with Python

Happy coding.

merge polyline paths

Somebody on the team was having trouble tracing streams due to breaks in the streams introduced when converting raster to feature.

So here is slick little solution which takes n paths in a line features and makes a 1 path (aka 1 part) out of it.
None of the vertices are moved. Where there were two paths separated by a gap you will have one path with no gap.

To work property the paths have to be pointed in the same direction and not be converging. To deal with that would require quite a bit more logic.

As is script modified data in place, so back up your data before using.

	def connect_parts(fc):
	"""
	Takes each part of a line feature. If made of more than one path
	(aka segment, aka part) connects them into a single path.
	Vertices are unchanged.
	Code authored on ArcGIS ver 10.2.2
	"""
	import json
	with arcpy.da.UpdateCursor(fc, "Shape@JSON") as uc:
	for row in uc:
	j = json.loads(row[0])
	paths = j['paths']
	if len(paths) > 1:
	j['paths'] = [list(itertools.chain(*paths))]
	uc.updateRow([json.dumps(j),])

view raw arcpy_merge_paths.py hosted with ❤ by GitHub

Using os.startfile and webbrowser.open in ArcGIS for Desktop

os.startfile and webbrowser.open are two very useful functions in the Python library. However, due to some conflicts in the way the Windows libraries expect to be called, they can fail or crash when called within ArcGIS for Desktop in an add-in script or geoprocessing script tool (see the Remarks section on this MSDN reference page).

import functools
import os
import threading
import webbrowser

# A decorator that will run its wrapped function in a new thread
def run_in_other_thread(function):
    # functool.wraps will copy over the docstring and some other metadata
    # from the original function
    @functools.wraps(function)
    def fn_(*args, **kwargs):
        thread = threading.Thread(target=function, args=args, kwargs=kwargs)
        thread.start()
        thread.join()
    return fn_

# Our new wrapped versions of os.startfile and webbrowser.open
startfile = run_in_other_thread(os.startfile)
openbrowser = run_in_other_thread(webbrowser.open)

The local functions startfile and openbrowser will be made available, which have the same parameters as the versions in the standard library but will run in another thread and therefore work as expected.

Get coded-value descriptions when accessing data with cursors

This question was recently asked on twitter:

“@arcpy Can you point me to an example python script where a SearchCursor returns coded-value descriptions instead of the codes?”

With ArcGIS 10.1, the data access module contains functions for listing subtypes and listing domains. Here are recipes to get descriptions rather than just the codes either for subtypes or domains:

1. Get subtype descriptions

fc = r'c:\data\Portland.gdb\roads'
# Create a dictionary of subtype codes and descriptions.
desc_lu = {key: value['Name'] for (key, value) in arcpy.da.ListSubtypes(fc).iteritems()}
with arcpy.da.SearchCursor(fc, "SUBTYPEFIELD") as cur:
    for row in cur:
        print(desc_lu[row[0]])

2. Get domain descriptions

gdb = r'c:\data\Portland.gdb'
# Get the dictionary of codes and descriptions for the Floodplain_Rules domain.
desc_lu = [d.codedValues for d in arcpy.da.ListDomains(gdb) if d.name == 'Floodplain_Rules'][0]
with arcpy.da.SearchCursor(os.path.join(gdb, "floodplain"), 'RuleID') as cur:
    for row in cur:
        print(desc_lu[row[0]])

* NOTE: If you are running ArcGIS 10.1 with no service packs, just do the following to get a dictionary of subtype codes and descriptions. ListSubtypes was enhanced at ArcGIS 10.1 SP1.

desc_lu = arcpy.da.ListSubtypes(fc)

Listing Database Views

When using ListTables() or ListFeatureClasses() with an enterprise geodatabase, the returned list will include the database views. In many cases, users name views with a unique prefix or suffix for distinction (i.e. VW_PARCELOWNERS or PARCELOWNERS_VW). For views with such a naming convention, you can get a list of them by using a list comprehension:

arcpy.env.workspace = r"c:\data\CityOfRedlands.sde"
views = [v for v in arcpy.ListTables() if v.endswith("_VW")]

If views have no specific naming convention or you want to ensure all the views are returned, you can use the ArcSDESQLExecute class.

For SQL Server:

sql_execute = arcpy.ArcSDESQLExecute(r'Database Connections\SQLSERVERDB.sde')
views = sql_execute.execute("select name from sys.objects where type = 'V'")

For Oracle:

sql_execute = arcpy.ArcSDESQLExecute(r'Database Connections\MYORACLEDB.sde')
# List user views
views = sql_execute.execute("select view_name from user_views")

For other database types, be sure to provide the correct SQL expression.

Ranking field values

At the Esri International User Conference this year, an attendee came to the analysis island to ask “how do I create a rank field?”. They had run the Generate Near Table geoprocessing tool (see illustration of table below) and were looking for a way to further rank the distances. Ideally, the table would be updated to include a RANK field starting at ‘1’ for the smallest distance for each PATIENT and increasing sequentially based on the distance. The rank could then be used to facilitate further review and reporting. We were able to come up with the addRanks function below in a few minutes automating a key missing piece of the user’s workflow.

Original table

Table after running addRanks function.

import arcpy

def addRanks(table, sort_fields, category_field, rank_field='RANK'):
    """Use sort_fields and category_field to apply a ranking to the table.

    Parameters:
        table: string
        sort_fields: list | tuple of strings
            The field(s) on which the table will be sorted.
        category_field: string
            All records with a common value in the category_field
            will be ranked independently.
        rank_field: string
            The new rank field name to be added.
    """

    # add rank field if it does not already exist
    if not arcpy.ListFields(table, rank_field):
        arcpy.AddField_management(table, rank_field, "SHORT")

    sort_sql = ', '.join(['ORDER BY ' + category_field] + sort_fields)
    query_fields = [category_field, rank_field] + sort_fields

    with arcpy.da.UpdateCursor(table, query_fields,
                               sql_clause=(None, sort_sql)) as cur:
        category_field_val = None
        i = 0
        for row in cur:
            if category_field_val == row[0]:
                i += 1
            else:
                category_field_val = row[0]
                i = 1
            row[1] = i
            cur.updateRow(row)

if __name__ == '__main__':
    addRanks(r'C:\Data\f.gdb\gen_near_table_patients2hosp',
             ['distance'], 'patient', 'rank')

Note: dBase (and shapefiles) does not support ORDER BY as used above by arcpy.da.UpdateCursor’s sql_clause argument.

Altering spatial references in Python

With ArcGIS 10.1, a spatial reference object can be created using a name or well-known ID (WKID).

# By name
sr = arcpy.SpatialReference('WGS 1984 UTM Zone 11N')

# By WKID
sr = arcpy.SpatialReference(32611)

However, once a spatial reference is created, many of the properties cannot be altered because they are read-only.

# Not possible
sr.centralMeridian = -110.0

Instead, if you need to change a property, you will need to take advantage of Python’s string manipulation capabilities. Since spatial reference properties can be expressed as well known strings, one solution is to export the spatial reference to a string, modify that string, and then use the altered string to create a new spatial reference.

import arcpy
import re
sr = arcpy.SpatialReference('WGS 1984 UTM Zone 11N')

# Change the central meridian.
sr.loadFromString(re.sub('PARAMETER\[\'Central_Meridian\'\,.+?]',
                         'PARAMETER\[\'Central_Meridian\',-120.0]',
                         sr.exportToString()))

References:
Well-known text representation of spatial reference systems

In addition to the documentation above, storage parameters like the coordinate domains and resolution, as well as tolerances are included in the spatial reference string.

List of Projected Coordinate system well-known IDs
List of Geographic Coordinate System well-known IDs