Conda and ArcGIS Pro

Conda was first included in ArcGIS Pro 1.3 release.  It allows users to easily manage python packages and environments.  For the  1.4 release of Pro we are working on further improving how conda works with ArcGIS Pro.

For those of you looking to get started with conda here are some interesting links.

Happy coding.

-the arcpy team

Mapping NFL Drive Charts

With Superbowl 50 this weekend, we thought it would be fun to use Python to look back at last year’s Superbowl between the Seattle Seahawks and New England Patriots and demonstrate how to create NFL drive charts in ArcGIS. A drive chart is a visual representation of a sequence of plays during a football game. Here, we’ll show how you can use Python and ArcGIS to summarize all of the drives from Superbowl XLIX and also how you can display the plays from individual drives.

For this post we used the following imports:

import os
import nflgame
import arcpy

The nflgame package is an API that is used to read and retrieve NFL Game Center JSON data. It is a convenient package for accessing NFL statistics for multiple games, parsing data for individual games, and working with real-time game data.

The Football Field

The first thing we did was create the football field. We created a football field feature class, assembled polygon features for the field, and inserted them into the feature class with an insert cursor. Here is our function to create the football field:

def build_football_field(output_gdb, output_feature_class):
    print('Creating football field.')
    footbal_field_fields = ('SHAPE@', 'TEAM')
    fc = os.path.join(output_gdb, output_feature_class)
    if not arcpy.Exists(os.path.join(output_gdb, output_feature_class)):
        arcpy.CreateFeatureclass_management(
            output_gdb, output_feature_class, "POLYGON", "#", "DISABLED",
            "DISABLED", arcpy.SpatialReference(3857))
        arcpy.AddField_management(fc, footbal_field_fields[1], "TEXT",
                                  field_length=20)

    cursor = arcpy.da.InsertCursor(fc, footbal_field_fields)

    field = [(0, 533.3),
             (1000, 533.3),
             (1000, 0),
             (0, 0)]
    cursor.insertRow([field, ""])

    home_endzone = [(-100, 533.3),
                    (0, 533.3),
                    (0, 0),
                    (-100, 0)]
    cursor.insertRow([home_endzone, "SEATTLE"])

    away_endzone = [(1000, 533.3),
                    (1100, 533.3),
                    (1100, 0),
                    (1000, 0)]
    cursor.insertRow([away_endzone, "NEW ENGLAND"])

The dimensions of a football field are 100 yards x 53.33 yards with endzones that are 10 yards deep. We scaled the length and width up by a factor or 10. We also created the yardline markers spaced at 10 yard intervals. The ‘MARKER’ field was used to label the yardlines. Here is our function to create the yardlines:

def build_yard_lines(output_gdb, output_feature_class):
    print('Creating yard markers.')
    footbal_line_fields = ('SHAPE@', 'MARKER')
    fc = os.path.join(output_gdb, output_feature_class)
    if not arcpy.Exists(os.path.join(output_gdb, output_feature_class)):
        arcpy.CreateFeatureclass_management(
            output_gdb, output_feature_class, "POLYLINE", "#", "DISABLED",
            "DISABLED", arcpy.SpatialReference(3857))
        arcpy.AddField_management(fc, footbal_line_fields[1], "TEXT",
                                  field_length=10)

    cursor = arcpy.da.InsertCursor(fc, footbal_line_fields)
    markers = [10, 20, 30, 40, 50, 60, 70, 80, 90]
    for marker in markers:
        line_1 = [(marker * 10, 533.3 / 2), (marker * 10, 0)]
        line_2 = [(marker * 10, 533.3 / 2), (marker * 10, 533.3)]
        if marker > 50:
            cursor.insertRow([line_1, str(100 - marker)])
            cursor.insertRow([line_2, str(100 - marker)])
        else:
            cursor.insertRow([line_1, str(marker)])
            cursor.insertRow([line_2, str(marker)])

After creating the field and yardline features, we brought them into ArcMap and designed the field. Here’s what it looks like:
Superbowl_XLIX _Field

The Drive Chart

Next, we created a drive chart showing the results of every drive from Superbowl XLIX. We used nflgame to access the drive data, arcpy to create a drive chart feature class, and an insert cursor to add the geometries and attribution to the feature class. In order to get all of the drive and play data, we created a game object for a single game, i.e. Superbowl XLIX. From the game object, we retrieved a list of all of the drives. Here’s the code:

home = 'SEA'      #Seattle Seahawks
away = 'NE'       #New England Patriots
year = 2014       #2014 Season
week = 5          #Week 5
reg_post = 'POST' #Postseason

print('Getting game data.')
game = nflgame.one(year, week, home, away, reg_post)

print('Getting drive data.')
drives = get_game_drives(game)
drive_count = get_num_drives(drives)

where get_game_drives returns a list of drives

def get_game_drives(game):
    return [drive for drive in game.drives]

and get_num_drives returns the number of drives per game.

def get_num_drives(drives):
    for drive in drives:
        drive_count = drive.drive_num

    return drive_count

We created a feature class to insert the drive data into. This feature class contained the following fields:

drive_fields = ('SHAPE@', 'TEAM', 'DRIVE_NUM', 'START_POS', 'END_POS',
                'DURATION', 'RESULT', 'DESCRIPTION')

where,

  • ‘SHAPE@’ is the shape of the bar representing the drive
  • ‘TEAM’ is the team that had possession of the football
  • ‘DRIVE_NUM’ is the number of the drive
  • ‘START_POS’ is the yardline where the drive started
  • ‘END_POS’ is the yardline where the drive ended
  • ‘DURATION’ is the length of time the drive took
  • ‘RESULT’ is the outcome of the drive
  • ‘DESCRIPTION’ is the NFL Game Center description of the drive

Then, we opened up an insert cursor for the drive chart feature class, and for every drive in the drives object, we created a drive chart polygon and populated the above attribute fields.

print('Opening insert cursor.')
cursor = arcpy.da.InsertCursor(os.path.join(output_gdb, output_fc),
                               drive_fields)

drive_bar_height = 533.3 / drive_count
for drive in drives:
    if drive.field_start:
        start_x, end_x = create_chart_polygon(drive, home, away)
        if start_x == end_x:
            polygon = [
                (start_x, (drive_count - drive.drive_num) * drive_bar_height),
                (end_x + 0.1, (drive_count - drive.drive_num) * drive_bar_height),
                (end_x + 0.1, (drive_count - drive.drive_num) * drive_bar_height + (drive_bar_height - 1)),
                (start_x, (drive_count - drive.drive_num) * drive_bar_height + (drive_bar_height - 1))]
        else:
            polygon = [
                (start_x, (drive_count - drive.drive_num) * drive_bar_height),
                (end_x, (drive_count - drive.drive_num) * drive_bar_height),
                (end_x, (drive_count - drive.drive_num) * drive_bar_height + (drive_bar_height - 1)),
                (start_x, (drive_count - drive.drive_num) * drive_bar_height + (drive_bar_height - 1))]

        cursor.insertRow([polygon, drive.team, drive.drive_num,
                          str(drive.field_start), str(drive.field_end),
                          str(drive.pos_time.__dict__['minutes']) + ' min and ' +
                          str(drive.pos_time.__dict__['seconds']) + ' sec',
                          drive.result, str(drive)])

Here, create_chart_polygon returns the start position (start_x) and end position (end_x) for each drive and looks like:

def create_chart_polygon(drive, home, away):
    scale_by = 10
    if drive.team == home:
        start_x = 50 + drive.field_start.__dict__['offset']
        if drive.result == 'Touchdown':
            end_x = 100
        else:
            end_x = 50 + drive.field_end.__dict__['offset']

    if drive.team == away:
        start_x = 50 - drive.field_start.__dict__['offset']
        if drive.result == 'Touchdown':
            end_x = 0
        else:
            end_x = 50 - drive.field_end.__dict__['offset']

    return scale_by * start_x, scale_by * end_x

After creating the drive chart, we added it to the football field above, symbolized the drives by the ‘TEAM’ field (Seattle is dark blue, New England is red), and added labels corresponding to the ‘RESULT’ field. Here’s what the drive chart looks like.
Superbowl_XLIX _Drives
Drives are ordered from top to bottom, with the top of the chart being the first drive of the game and the bottom being the last.

Individual Drives

We also mapped individual drives. This allows us to look at the plays within the drive and understand how the team moved down the field. If you’re interested in all the details, you can check out our sourcecode on Github. In a nutshell, for every drive we returned the plays and following from the same logic above for the drive chart, we created polygons for each play within the drive. In this case, we created a feature class for every drive and populated the following attribute fields:

play_fields = ('SHAPE@', 'TEAM', 'DRIVE_NUM', 'START_POS', 'END_POS',
               'DURATION', 'PLAY', 'RESULT', 'DESCRIPTION')

The field names are the same as for the drives, except we added a ‘PLAY’ field that indicates the type of play, for example, a “rush”, “pass”, “kick”, etc.

The New England Patriots won Superbowl XLIX 28-24 and on their second to last drive, they took the lead on a touchdown pass from Tom Brady to Julian Edelman. Here’s the play chart.
Superbowl_XLIX _NE_Drive
On the following drive, the Seattle Seahawks had a chance to win the game, but Malcolm Butler intercepted a pass from Russell Wilson to seal the game for the Patriots.
Superbowl_XLIX _SEA_Drive

Drives as Webmaps

We shared the drive charts as webmaps. This allows you to interact with the charts and design popups to view the play results and other attributes. Click on each image to go to the webmap.

ArcGIS_Online_Drive_Chart_NE

ArcGIS_Online_Play_Chart_NE

Feel free to leave any comments or questions in the comment box.

Happy coding,
-the arcpy team

arcpy interoperability with C++ library

ArcGIS geoprocessing tools have rich functionality for messaging and reporting progress within the ArcGIS family of applications. With script tools implemented using python the coding patterns are well established and documented.

One of our users created a script tool which called out to a c++ library (DLL). The processing done within the c++ library was lengthy so we were asked about how to provide progress feedback, as well as how to deal with a user clicking the ‘cancel’ button on the tool within the DLL.

The arcpy-cpp-interop repo on GitHub is a working, simple example of a c++ library called from a python script that demonstrates rich communication and feedback to the ArcGIS application.

Happy coding
-the arcpy team

Get inner-most poly

Nice spatial relationship question today… a co-worker asked ‘how can i get only the inner-most poly from these clusters of polygons’. Looking at the data, none of the polygons overlap, these were clearly generated from contour lines.

SelectPeaksIn

There are likely a few ways to accomplish this, but our approach was as follows: recreate all polygons but fill in their inner holes, then count how many times each filled feature intersects an input feature… those that only intersect once are the inner-most.

The function we created returned the original dataset’s ObjectIDs

Code is as follows

import arcpy

#arcpy.env.overwriteOutput = True

def listOidOfPeaks(fc):
    oids = []

    # Eliminate ALL internal parts (basically fill in all the holes)
    arcpy.management.EliminatePolygonPart(fc , "in_memory\\EPP_tmp",
                        "AREA", "1000000 SquareKilometers", 0, "true")

    arcpy.management.MakeFeatureLayer("in_memory\\EPP_tmp", "EPP_lyr")


    with arcpy.da.SearchCursor(fc, ("OID@", "SHAPE@",)) as c:
        # for each of the original poly feature's labelPoint (which
        #  is a point INSIDE the polygon) get the list of filled
        #  features being overlapped
        for row in c:
            arcpy.management.SelectLayerByLocation("EPP_lyr", "INTERSECT",
                                        arcpy.PointGeometry(row[1].labelPoint))

            # put all the selected filled feature's ORIG_FID into a list
            oids += [i[0] for i in da.SearchCursor("EPP_lyr", ("ORIG_FID"))]

    # every OID which occurs only once are ones we want
    for i in sorted(oids):
        if oids.count(i) == 1:
            yield(i)


# what's returned is a generator
oids = listOidOfPeaks("contour_polys.shp")

# bonus : turn a list of ids into a where_clause (I'm assuming FID is field name)
print("as a query :  FID in ({})".format(",".join(map(str, oids))))


The resulting where_clause which we can use with a number of GP tools such as Select Layer By Attribute
SelectPeaksOut

The New Python Window in ArcGIS Pro

ArcGIS Pro is coming with a totally redesigned UI, and the Python window is no exception.

First off: the Python window is meant to be a utility. It’s not most flashy of designs. You type in code and get feedback, it should help you do that and that is all. Our intention in the redesign wasn’t to get people super excited about typing code into a text box, it was to pleasantly surprise you with some helpful features and get out of your way.

The way it’s set up it a little different in Pro App. Here’s a quick tour of its design and features.

Split window

There are separate places for input and output

pywin_pro_mage1

The biggest obvious change in the Python window is that it’s been divided into two pieces: the input section and the transcript section. We found from a usability point of view, mixing the input and output sections into one control could be confusing. You don’t have to skim the entire window to find where the newest prompt is, you just click in the bottom and start typing. It’s similar to a few other user interfaces we’ve used, old and new:

pywin_pro_mage2

Help Tips

If you’re typing inside of a function call, you’ll get a list of parameters along with the current parameter highlighted:

[highlight-parameter.png]

We’re not blazing any new territory here, it’s just useful.

Autocomplete

This is a carryover from every Python window ever, including the ArcGIS 10.X Python window. The Python window will try to guess what you’re looking for:

pywin_pro_mage3

it works a little harder than most Python auto-completes and will let you even just type the first letter or two of something:

pywin_pro_mage4

that’ll save a few keystrokes. And like the 10.X Python window, it gives extra help for geoprocessing tools:

pywin_pro_mage5

Tools Show Up in History

Any GP tool you run in the Python window will show up in your current project’s Geoprocessing History. Outputs will be added to the current map.

pywin_pro_mage6

Stop!

When the Python interpreter is running, the Python input section is disabled and will show a “running” progressor. You can stop anything running in the Python window by hitting the “X” when it’s visible:

pywin_pro_mage7

History Across Sessions

You can use [Ctrl-Up] and [Ctrl-Down] to go through previous commands you’ve issued in the Python window. It also saves them between sessions, so if you open up a new ArcGIS Pro instance, you can hit [Ctrl-Up] and get what you typed last time.

Drag And Drop

You can drag and drop pretty much anything into the Python window and it’ll try to guess what you want:

pywin_pro_mage8

and Geoprocessing tools:

pywin_pro_mage9

tools you’ve already run in the history show up as full Python snippets:

pywin_pro_mage10

from Windows Explorer:

pywin_pro_mage11

Comfortable And Productive

We can’t show this in screenshots, but a lot of the work we’ve put into the Python window is how it feels. Hopefully you’ll be more productive and the Python window will get out of the way and just let you do your job, speeding you up where it makes sense.

Split into equal length features

Request came in last week for a way to split a line feature into 10 equal length line features.

The input looked like this
stream1

The accepted solution came from Dave on the team who sent this elegant and efficient solution.

in_fc = r'c:\projects\waterway.gdb\stream'
out_fc = r'c:\projects\waterway.gdb\stream10'
out_count = 10 # how many features desired

line = arcpy.da.SearchCursor(in_fc, ("SHAPE@",)).next()[0]
arcpy.CopyFeatures_management([line.segmentAlongLine(i/float(out_count), ((i+1)/float(out_count)), True) for i in range(0, out_count)], out_fc)

Which outpus a new feature class containing 10 line features like so
stream10

How does it work? The first 3 lines are self explanatory. So we will skip.

The following line is also fairly simple, what it does is get the geometry (shape@) of the first record (we only ask for next() once). The [0] is needed to get just the first value in the record (remember cursors return lists of values).


polyline = arcpy.da.SearchCursor(in_fc, ("SHAPE@",)).next()[0]

Next is where the magic happens, python list comprehension is used to turn the polyline object into a list of 10 (as per the out_count variable) equal length segments generated by the segmentAlongLine function. This list of polyline is then used as input to CopyFeatures (as per Using geometry objects with geoprocessing tools) which writes out the 10 polyline as individual features into the output feature class (out_fc).


arcpy.CopyFeatures_management([polyline.segmentAlongLine(i/float(out_count), ((i+1)/float(out_count)), True) for i in range(0, out_count)], out_fc)

EDIT: As was pointed out in the comments, the segmentAlongLine is new at 10.3.

Cheers

pretty print fields

This is a piece of code i’ve written and re-written dozens of times. Typically I code it on a whim for a single use.

Today I went the extra mile and made the output really nice, so here it is in case somebody has use for it.

import arcpy
def pprint_fields(table):
""" pretty print table's fields and their properties """
def _print(l):
print("".join(["{:>12}".format(i) for i in l]))
atts = ['name', 'aliasName', 'type', 'baseName', 'domain',
'editable', 'isNullable', 'length', 'precision',
'required', 'scale',]
_print(atts)
for f in arcpy.ListFields(table):
_print(["{:>12}".format(getattr(f, i)) for i in atts])

And this is what you get as output in the python window*.
pywin_pprint

* this is the nice and new ArcGIS Pro’s python window.

The output is a bit too wide for use in terminal (80 characters) but within the app it’s nice and clear.

Happy coding.

Getting more arcpy help

A lot of people are posting requests asking for a particular snippet in the comments section of this blog.

By all means, continue to do so, we do enjoy tackling the more generic and broadly useful requests… but we just can’t tackle all of them, and not in a timely manner.

So if you need help with code, and you need it fairly soon, please consider also opening a discussion on the Esri geonet site.

Happy coding.