Thursday, October 25, 2007

Stuck in Traffic: Find the Road Segments With the Highest Pollution Levels

Workshop This Afternoon!!

Download Full Workshop Materials
View Zohoshow Presentation
View Handout
View Flyer
View Google Maps Demo

This has been a whirlwind frantic rush to pull together but just about everything is ready to go for this afternoon's GIS workshop. Should be a great one...

Dr. Melanie Sattler, Civil & Environmental Engineering professor, will lead the workshop with a 30 minute discussion of why air pollution is important in the D/FW Metroplex and discuss the various campus research efforts to study and model air quality.

I will then lead everyone though a hands-on GIS exercise that will include the following steps:
  • Calculate total pounds of air emissions within 5 miles of everyone's home address (using spatial join)
  • View distribution of emissions across region (using fishnet polygon)
  • Join various measurement data to air monitoring stations (such as ozone and wind direction)
  • Calculate estimation of ozone concentrations for entire region (using IDW)
  • Calculate average ozone concentrations for highway road segments (using spatial join)
How can I expect to guide novice users through such procedures in a reasonable amount of time? Every procedure is automated using the ModelBuilder and VBA, and is accessible via a custom toolbar.


Data Sources Include:
If you're in the area, drop on by...

Friday, October 05, 2007

Large Table Manipulation: ArcMap vs. Microsoft Access

Helped an engineering student yesterday to display a table with over 500,000 XY coordinates and convert it to a shapefile. The table the student brought was a 2.5 gig CSV file. It gave me some unexpected troubles.

The conclusion is that the manipulation of large tables is often times best done using Microsoft Access. Specifically, converting field types for large tables with hundreds of thousands of records is best done in Microsoft Access, and not in ArcMap.

If you are interested in the particulars, read on below.

Brought the table into ArcMap, but every field in the table was read by ArcMap as a text field. Opened the table in ArcMap, and all the values were numerical coordinates. Opened the table in Excel, and of course the table only partially opened as there were too many records. The portion of the table that did open were all numerical coordinates, but Excel was also reading the fields as text. No problem. Switched back over to ArcMap and attempted to add two new double fields and use the field calculator to copy the text fields into the double fields. ArcMap is unable to add fields to text files, so we had to export the table as a DBF first. This took over 5 minutes as there was so much data. Every time we tried to add a new double field to the large DBF file, the screen would turn white and would hang there indefinitely. We tried this twice, and gave up on each attempt after 5 minutes of inactivity.

Then I had the idea to import the table into a blank Microsoft Access database where I can directly change the field types from text to double. Before changing the field types, I opened the data table and sure enough I saw that every field was surrounded by quotation marks {"} as a text qualifier. THAT was the problem! Switched over the design view and changed the field type and sure enough that did the trick. ArcMap honors Access field specifications and we were able to view the event class in no time. Took a bit more patience waiting while we exported the shapefile, but it was nice to see the completion of what should have been a routine process.

Why were there quotation text qualifiers around each field? Well, the student told me the file was created using Excel 2007, which I have not yet used. Perhaps this is standard procedure for Excel 2007? If not, perhaps there is some setting in Excel that will inadvertently place text qualifiers around fields?

Repeating from above: The conclusion is that the manipulation of large tables is often times best done using Microsoft Access.

Wednesday, October 03, 2007

Interesting GIS/Geography Related Dissertations

[updated 10/05 to include links to free dissertation abstracts and 24-page previews, compliments of UMI Dissertation Publishing at ProQuest. See comments for more info. Thanks Mike!]

Here's the latest installment of GIS-related dissertations that have caught my eye. Previous lists include 11/05, 01/06, 01/06, 04/06, 07/06, 10/06, 04/07. (Of course, this is not a comprehensive list...only those I find interesting.)