ArcMap2GMap Fixes: Thanks for Feedback!

After releasing ArcMap2GMap a couple of weeks ago, two persistent bugs have repeatedly been pointed out by users. I want to document these errors and the fixes, both of them quick, here. The script has been updated on the ESRI ArcScripts page with these fixes, so a fresh download should resolve these issues.

• Download Updated (fixed) Script
• ArcMap2GMap Posts

Bug #1: Process Status Message Not Appearing in ArcGIS 9.2

• For whatever reason the form text specifying the three procedure steps (that alternated while processing to inform the user which step was currently operating) were not appearing in ArcGIS 9.2. This resulted in the appearance that nothing was happening.
• The fix was to add a Me.Repaint command each time the caption of the label is changed. My understanding was that VBA should handle this automatically, but obviously it was not the case here. This creates a bit of unnecessary overhead as the entire form and its contents need to be redrawn jsut to change the label status.

Bug #1: String Length Limitation While Passing to Geoprocessor

• There is a string length limitation when passing text strings as parameters to a Python script. The first Python script is called by the VBA using the following command: gp.ArcMap2GMap apiKey, tbxName.Text, tbxTitle.Text, layerInfo. The layerInfo string (last parameter) contains the names of all included layers, as well as all user specified options, including color, thickness, messagebox text, etc. I do not know exactly what the length limitation is, but if too many layers are selected this string is truncated and of course errors ensue.
• This was brought to my attention by a user needing to create a p0age with 27 layers.
  
• The fix was to have the VBA write the layerInfo contents to a text file and then after the Python script is invoked, read the contents from the text file.

I surely do appreciate all the feedback!

Thanks everyone!

Texas Statewide Historical Maps & Positional Accuracy Pt. 2

Continuation from Pt. 1

Our Geographic Accuracy Measurement Procedures - Overview

• We calculate 5 positional accuracy measurements for each georeferenced map. Texas is divided into 4 quadrants, with each quadrant receiving an independent positional accuracy measurement. The fifth measurement is the average accuracy of the 4 quadrants.
• Each quadrant is further subdivided into 4 sections. At least 4 sample points are taken from each of these sections, with a minimum of 20 points from each quadrant. However, as the NSSDA suggests, 25 points are recommended and is what we aim for. We therefore end up using (ideally) between 80 to 100 sample data points to measure the accuracy of each map.
• The exact points we use for each map of course differ, but here is our priority geographic references. First, clearly identifiable county boundaries. Second, coordinates provided by hash marks labeled on map edges. Third, city locations. City locations have the lowest priority here because we use them primarily to georeference and it is best for the sample points to be different from the points used to georeference. Natural features, such as rivers or lakes, are never used.
  

Our Geographic Accuracy Measurement Procedures - Step by Steps

• Four new feature classes are created for each map to hold the 20 to 25 sample points from each quadrant.
• Point features are created for each sample data point. The coordinates as specified by the map are hand-entered in the attribute table.
  
• The actual X/Y coords (in meters) are generated for each feature in the four quadrant feature classes using ArcMap's Add XY Coordinates tool.
• This data is entered into the horizontal accuracy calculation spreadsheet. Our own customized templates use different NSSDA multipliers, based on the calculated RSME ratio. This is discussed in more detail in the previous blog post.

An Example! Enough Talk, Here is an Example...

• Title: Colton's Texas (1855)
• Original Map Citation: Colton, Joseph H. Colton's Texas. New York: J. H. Colton & Co., 1861.
• Entire map tested 8.982 kilometers horizontal accuracy at 95% confidence level.
  
• Download: Georeferenced Map - No metadata included yet as we are still ironing out taxonomy and copyright issues. You betcha there will be posting about these issues, as well as interface and file format issues as well.
  

• Horizontal Positional Accuracy statement for the Colton's Texas (1855) map is pasted below. As specified above, I am unable to post a complete metadata record at this time as other portions are incomplete. You can see relative high accuracy of NE Texas as opposed to the SE quadrant.
  

---

Horizontal_Positional_Accuracy:

Horizontal_Positional_Accuracy_Report:

Each georeferenced map image file was divided into four quadrants at centerpoint 31.688, -98.634. The horizontal accuracy of each quadrant was calculated independently using the NSSDA (National Standard for Spatial Accuracy) at the 95% confidence level. Where possible, a minimum of 20 sample points from each quadrant were used to measure horizontal accuracy. The horizontal accuracy of of the entire map is the mean average of these four measurements.

Quantitative_Horizontal_Positional_Accuracy_Assessment:

Horizontal_Positional_Accuracy_Value: 8982

Horizontal_Positional_Accuracy_Explanation:

Entire map tested 8.982 kilometers horizontal accuracy at 95% confidence level. Mean average of the four quadrants.

Quantitative_Horizontal_Positional_Accuracy_Assessment:

Horizontal_Positional_Accuracy_Value: 6988

Horizontal_Positional_Accuracy_Explanation:

NW Quadrant tested at 6.988 kilometers at the 95% confidence level.

Twenty-four points were used to test the positional accuracy. The calculated RSME ratio was 0.3344, and the normalized elliptical error table was used to determine error at the 95% confidence level.

Quantitative_Horizontal_Positional_Accuracy_Assessment:

Horizontal_Positional_Accuracy_Value: 8100

Horizontal_Positional_Accuracy_Explanation:

NE quadrant tested at 8.1 kilometers at the 95% confidence level. Twenty-four points were used to test the positional accuracy. The calculated RSME ratio was 0.699, and the normalized circular error table was used to determine error at the 95% confidence level.

Quantitative_Horizontal_Positional_Accuracy_Assessment:

Horizontal_Positional_Accuracy_Value: 12400

Horizontal_Positional_Accuracy_Explanation:

SE quadrant tested at 12.4 kilometers at the 95% confidence level.

Twenty-four points were used to test the positional accuracy. The calculated RSME ratio was 0.609, and the normalized circular error table was used to determine error at the 95% confidence level.

Quantitative_Horizontal_Positional_Accuracy_Assessment:

Horizontal_Positional_Accuracy_Value: 8400

Horizontal_Positional_Accuracy_Explanation:

SW quadrant tested at 8.44 kilometers at the 95% confidence level.

Ten points were used to test the positional accuracy. The calculated RSME ratio was 0.400, and the normalized elliptical error table was used to determine error at the 95% confidence level.

Texas Statewide Historical Maps & Positional Accuracy Pt. 1

We have been undertaking a project, called the Texas Time Machine (TTM), which requires us to georeference statewide historic maps of Texas.

Brief overview of TTM:

Preparing historical materials for use within GIS for geographic analysis requires a large time commitment and high level of expertise. Once prepared, the researcher still needs to understand the fundamentals of operating a large desktop GIS application. TTM resolves this by (1) compiling a collection of prepared historic materials, and (2) enabling interaction with these materials within Google Maps and Google Earth (as well as desktop GIS applications).

TTM provides 4 ways to view geographically referenced maps, statistics, and images. (1) Via Google Map overlays, (2) Via downloadable Google Earth KMZ files, (3) Via downloadable GIS Data, and (4) Via downloadable un-georeferenced images.

I want to focus this post specifically on how we are georeferencing 150 year old maps whose geography encompasses over 250,000 square miles.

Georeference Scanned Map Image

1. The intended coordinate system of the original cartographer needs to be determined. In consultation with our Cartographic Archivist Librarian, we discovered that for the majority of the maps dating int he 19th century a Mercator system was intended.
2. Control points need to be used for georeferencing. For the first order, we use control points from the state outline, namely the Pan Handle, westernmost tip, easternmost tip, and southernmost tip. Then, we overlay a uniform 5x5 grid shapefile (25 standard polygon features) over the ungeoreferenced image within ArcMap. One city per cell is used as a control point. Cells without cities indicated on the scanned map will not contain control points.

NSSDA & Historical Measurements
We are adhering to the National Standard for Spatial Data Accuracy (NSSDA). The two primary resources we followed are:

• ‘Geospatial Positioning Accuracy Standards, Part 3: National Standard for Spatial Data Accuracy’
• ‘Positional Accuracy Handbook: Using the National Standard for Spatial Data Accuracy to measure and report geographic data quality’.

Both resources can be accessed here.

The first work above provides a general overview of the process and specific case studies where one can learn by those examples. There are two cases for measuring horizontal accuracy.

• The first case is on page 3-10. This case demonstrates how to calculate error with 95% confidence when the x-axis error is equal to the y-error. RMSE(x) == RMSE(y). (Root Mean Square Error) I do not anticipate this as applicable as our maps are not consistently drawn to scale.
• The second case is on page 3-11. This case is entitled ‘Approximating Circular Standard Error When RMSE(x) != RSME(y)’. However, the details of the case demonstrate how to calculate error when RMSE(min)/RMSE(max) is between 0.6 and 1.0. This implies an almost consistent error across the x- and y-axis.
• The formula provided is: Accuracy ~ 2.4477 * 0.5 * (RMSE(x) + RMSE(y)). This is in effect the average of the two errors (added and divided by 2) and then multiplied by the full circle confidence of 95% as designated by the ‘Generalized Circular Probable Error’ table. (JSTOR access, page #170).
• This case continues to explain that the circular standard error at 39.35% confidence may be approximated at 0.5 * (RMSE(x) + RMSE(y)).
• The big question for us is how can these numbers be adjusted to accommodate where RMSE(min)/RMSE(max) is less than 0.6.
  
• The second work above is a handbook/workbook that enables the easy use of the first case specified in the first work. Namely, where RMSE(x) == RMSE(y). As stated above, this is not the case with our maps because cartographers could not draw them to scale 150 years ago. However, this second work provides print and downloadable versions of a spreadsheet that modified for our uses, namely adjusting the modifier at the end based on the RMSE ratio.
• Both works provide template language to include in the GIS metadata, as well as specific metadata fields where positional accuracy should be reported.
  

Our Geographic Accuracy Measurement Procedures

• Ha! I am tired of writing at the moment and will lay out out specific in-house procedures tomorrow. I will also include some snippets from one of our metadata records.
  

GIS Librarian (-ish) Positions x2

1.
Geographic Information Systems Specialist (Bucknell University)

"Bucknell University seeks to hire a Geographic Information Systems Specialist for the Library and Information Technology organization... The primary responsibilities of this position are to develop, expand, and support a GIS user community by assisting students and faculty in the selection and use of appropriate GIS technologies; working with faculty and students in designing and executing projects using GIS; and providing instructional support in GIS."

While this position does not have the title Librarian and while a library degree is not required, this sounds quite similar to my responsibilities here. And, of course, this position does indeed reside in Bucknell's Library. With a salary of $40,000 - $60,000 this position sounds pretty fantastic for a GIS professional looking for an academic position.

2.
Data Service Librarian (New York University)

"New York University is seeking an energetic, creative, and knowledgeable librarian to select, acquire, manage, and deliver numeric and geospatial data collections to support campus research and scholarship."

Some snippets:

• The librarian will build numeric and spatial data collections and facilitate access to additional data resources across the sciences
• Reporting to the Data Service Coordinator, the Data Service Librarian works to develop appropriate description for managing research data collections; investigates new sources for metadata; keeps abreast of new and evolving metadata standards such as the Data Documentation Initiative (DDI) and Federal Geographic Data Committee (FGDC) standards.
• The incumbent will develop and maintain awareness of data-centered initiatives across the sciences, attending professional meetings, workshops and conferences for training and continuing professional development.
• Requirements: Basic familiarity with software for statistical and geospatial analysis (e.g. SAS, SPSS, Stata, R, GIS applications).

This really sounds like an exciting position. We are continually increasing the numeric (non-spatial) data services and positions such as this one which straddle both numeric and spatial data services are the thing of the future. My opinion is that GIS technology is becoming more and more commonplace and within 5 years (or so) it might not be so necessary to maintain a professional strictly in GIS or strictly in non-spatial statistics. Great long-term opportunity for a highly experienced librarian.

Specific salary benefits are not mentioned, and considering the cost of NYC especially someplace wiothin an easy commute this perhaps will be a huge factor.

ArcMap to Google Map Polygons

ArcMap2GMap: download
+ (not yet tested for 9.3)
Samples:
+ Presidential Election Data 2004
+ Hodge Podge Sample of Stuff From My Computer
+ Health Resources

Finally completed a major update for the ArcMap2GMap script that exports ArcMap layers to a standalone Google Map webpage.

This latest update now includes support for choropleth polygon layers using the gPolygon object. Previous versions included support for point and line geometries. The choropleth map is hard-coded to generate 4 equal interval classes based on the attribute selected by the user, but we do have plans on providing more flexibility with this in the future. I want to whole-heartedly thank my GRA, Shivkumar Chandrashekhar, for all of his assistance with this project.

This version includes all features of previous versions, including:

• multi-layer support
• geocoding
• proximity searching (top 10 closest visible points displpayed)
• driving directions
  

Students at our university are restricted from registering DLLs, so we could not compile the VBA forms. This means the MXD provided in the download must still be used.

The major issue we needed to resolve to include this polygon support was the complexity of the vertices in a polygon shapefile. Even the simples polygon shapefiles may have thousands of vertices that will timeout any browser on virtually any computer. We have two point reduction methods in place to help resolve this.

1. First, each polygon feature's vertices are filtered through a Douglas Peucker Algorithm. The code for this can be viewed in the DP.py script.
   
2. Second, after the algorithm is run, each polygon feature class is dissolved using ArcMap's geoprocessing dissolve tool. This effectively removes shared boundaries by features with identical color representation.

We are holding an open workshop on campus on September 24. If anyone tries out the script and has any comments, please leave them here or if you do not want them on the permanent blog record, leave them in the IM client to the right.

GIS-Related Dissertations/Theses Late 2007 & 2008

As always, there are oodles of great GIS-related theses and dissertations out there. After taking a long and deep gander at the latest batch, here are the latest ones to have caught my eye. For previous installments, click here.

• An object-oriented programming approach for a GIS data-driven simulation model of traffic on an inland waterway. by Sasso, Daniel, Ph.D., University of Louisville, 2008, 163 pages
• "This research demonstrates how the GIS technology can be integrated with a simulation software as an interface to model the transit of ocean-going vessels through the Panama Canal."


• An assessment of the utility of GIS-based analysis to support the coordination of humanitarian assistance. by Verjee, Firoz, D.Sc., The George Washington University, 2007, 272 pages
• "This dissertation investigated if Geographic Information Systems (GIS)-based analysis can have a decisive influence upon the coordination of international humanitarian assistance. Utility was assessed through a series of case studies demonstrating the application of GIS-based analysis to facilitate coordination of recent or potential humanitarian interventions."


• Artifact management and behavioral discourse in the software development process for a large public participatory geographic information system. by Wu, Jie, Ph.D., University of Washington, 2007, 155 pages
• "The objective of this work is to develop an artifact management framework (AMF) for supporting a large public participation geographic information system (PPGIS) software development effort."


• Assessment of black locust on the Black Kettle National Grasslands. by Farris, Bullit Michael, M.S., Oklahoma State University, 2007, 64 pages
• "Remote sensing combined with GIS analysis has been established as one of the most effective ways to study the conversion of grasslands to forests and has been used in many parts of the world to analyze woody encroachment. The purpose of this study is to analyze the distribution of black locust in the Black Kettle National Grasslands by comparing historical and current stand area using aerial photography and investigate the differences in encroachment rates focusing on the influence of potential soil moisture content using the topographic wetness index."


• Comparative impacts of Web-based GIS on student content knowledge, geography skills, and self-efficacy in introductory human geography. by Songer, Lynn Christine, Ph.D., University of Oregon, 2007, 229 pages
• "This dissertation compares the impacts on student learning outcomes: self-efficacy, content knowledge, and geography skills, when Web-based GIS is used to teach introductory human geography"


• Implementation of Geographic Information Systems in policing: A case study of the Spokane Police Department. by Akgul, Arif, Ph.D., Washington State University, 2007, 228 pages
• "For the last decade Geographic Information Systems (GIS) applications, including crime mapping and crime analysis in police departments, have emerged as one of the most important innovations in the law enforcement. However, very little research has been conducted on how police departments use the system."


• Settlement geography of the Punjab during the early historic and medieval periods: A GIS approach. by Smith, Mark A., Ph.D., New York University, 2007, 476 pages
• "Utilizing Geographic Information Systems (GIS) analysis and quantitative geography techniques, the research provides insights into the meaning of changing settlement patterns in the Pakistani province of Punjab between the 5th century B.C and 18th century A.D."


• Spatial analytical approaches for supporting security monitoring. by Kim, Kamyoung, Ph.D., The Ohio State University, 2007, 144 pages
• "This dissertation addresses the challenges of strategic sensor location in security monitoring and discusses how GIS can be utilized to define the coverage of sensors. Application results demonstrate that sensor placement for supporting security monitoring in 3D urban areas can be approached using coverage-based optimization models combined with visibility analysis."


• Web-based GIS and public participation: An aid to widening female participation in revitalizing outdoor recreational facilities in Saudi Arabia. A case study in Jeddah, Saudi Arabia. by Daghistani, Farouk, Ph.D., Texas A&M University, 2007, 211 pages
• "During the last decade, the Internet and Geographic Information System (GIS) have made changes in the relationship between governments and citizens in many developed countries. Citizens, in such countries, have been given more chances to participate in the decision making process of the spatial issues relevant to them."

Summer 08 Class Final Projects: Highlights

Been working my fingers to the bone teaching 9 credits last Spring, 6 credits this Summer, and getting ready to teach 6 more this Fall (next week, gulp!).

This past Summer was exciting and really stood out, however, as many of the final projects from both courses were exceptionally great. I taught Seminar: Advanced GIS Topics for Real Estate Research and Understanding Geographic Information Systems (an intro to gis course).

Real Estate Project Highlights:

• Automated Foreclosure Selection Model. This project incorporated MLS and foreclosure listings and conducted a failry sophisticated comparable market analysis to whittle down the hundreds of available foreclosure listings to a select few worthy of consideration and further investigating. The analysis would have been sufficient for an excellent grade, but they went the extra mile and automated the process with a custom toolbar, geoprocessing models, and some VBA programming.
• Site Selection (suitability analysis) for a New Mixed-Use Development in Arlington Texas. Of note here is the clever way they included traffic pattern data in their analysis.
• Analyzing the Correlation Between Crime and Property Values. What set the project apart was the student's in depth use of SPSS and linear statistical analysis in conjunction with ArcGIS.

Understanding GIS Project Highlights

• Estimating Surface Runoff Volume. Wow is all I can say about this one. This project used ArcGIS to calculate the Soil Conservation Service Curve Number for a local area in Fort Worth. Landuse, zoning, aerial images, and city-defined drainage areas were used in this analysis.
• Site Selection for an Environmentally Friendly Park in Dallas. Completed by a graduate Landscape Architecture student, this project focused on soil runoff, amount of sunlight, and various other parameters necessary to create a Green park.
• Analyze the Relationship Between Geology and Oil Fields in Texas. This project made extensive use of the Geologic Atlas of Texas to search for common geologic types underlyiing oil fields. This in itself was a great project, but the student went the extra mile and created a Google Maps web page showcasing the results, which was fantastic.

Like I said, this was a most excellent summer in terms of the quality of student final projects. There were of course many other great projects, but these are the six that stick the most in my head at the moment. ;)

Planning Upcoming Workshop: Predicting 2008 Local Voting Results

One of the three GIS workshops we have planned for the Fall 2008 semester is entitled 'Predict 2008 Voting Patterns Across Neighborhoods in Texas'. It is not scheduled until late October, so it is oh so far from being done, but here is how we are planning to go about it.

By Texas neighborhoods, I really mean that each workshop participant will estimate how voters will vote in each Voter Tabulation District (VTD) in Texas. This can be then grouped to form neighborhoods, especially in urban areas.

There will be three parts to the exercise.

1. Participants will first predict voting by using the previous two presidential election results (2000, 2004) and the previous two gubernatorial results (2002, 2006).
1. Data source: Texas Legislative Council: Redistricting FTP Site
2. Participants will build an estimation layer by compiling a weighted average of the four election datasets provided, as well as rates of change between these elections. Everyone will be able to specify which attributes are included in the analysis and what each weight will be.
2. Participants will then explore the bivariate correlation between various demographic attributes and previous election results, such as income or Hispanic population.
1. We will use the Linear Regression (bivariate) ArcMap extension, written by Michael Sawada, of the University of Ottawa.
2. After exploring these tabular relationships, participants can decide whether to use any of these demographics to adjust their estimation layer created in the first step.
   
3. Participants will then have the option to enable random occurrences to adjust their estimation layer for them. This one should be a lot of fun, as a random number generator will specify a last-minute political scandal, natural disaster, or economic crisis that will further adjust the estimates.

Of course, this will all be automated using a combination of Python scripting and the Model Builder. so participants can concentrate on their research and the enormous potential that GIS lends to this type of analysis.

It is scheduled for one week before election day, so we are hoping this hot topic will be a further draw for students and faculty.

Just for kicks, I created a Google Maps webpage showing % votes for Bush in 2004 and total votes for each candidate by VTD for Tarrant County, TX.

Mapping Oak Cliff's Realities, Possibilities

Dallas Morning News Article: Mapping Oak Cliff's Realities, Possibilities (08/16/08)

This recent article spotlights student Charles Jackson's use of GIS in an undergraduate project on the Oak Cliff neighborhood in Dallas, Texas. Charles is a student I assisted with on may GIS projects. For this one in particular, he used GIS to create a Google Map webpage of various resources available in this low-income area. Charles' online project is entitled Oak Cliff Interactive.

Background information about this project and the technology used can be found in a blog entry I wrote last year.

Been a While...

So, it has been a long, long while since my last post here. Why? Ah, oodles of reasons. Most of all, I just kicked a 9-month compulsive WoW habit that impeded and threatened many facets of my work and personal life.

To those of you out there who were negatively affected by my obsessive gaming habits, I sincerely apologize. To everyone, I fully intend at this point to resume writing on this blog and others as it is a great pleasure to me. I have been quite busy working on numerous GIS projects and teaching various classes, so there are loads of good stuff to write about.

Stuck in Traffic: Find the Road Segments With the Highest Pollution Levels

Workshop This Afternoon!!

Download Full Workshop Materials
View Zohoshow Presentation
View Handout
View Flyer
View Google Maps Demo

This has been a whirlwind frantic rush to pull together but just about everything is ready to go for this afternoon's GIS workshop. Should be a great one...

Dr. Melanie Sattler, Civil & Environmental Engineering professor, will lead the workshop with a 30 minute discussion of why air pollution is important in the D/FW Metroplex and discuss the various campus research efforts to study and model air quality.

I will then lead everyone though a hands-on GIS exercise that will include the following steps:

• Calculate total pounds of air emissions within 5 miles of everyone's home address (using spatial join)
• View distribution of emissions across region (using fishnet polygon)
• Join various measurement data to air monitoring stations (such as ozone and wind direction)
• Calculate estimation of ozone concentrations for entire region (using IDW)
• Calculate average ozone concentrations for highway road segments (using spatial join)

How can I expect to guide novice users through such procedures in a reasonable amount of time? Every procedure is automated using the ModelBuilder and VBA, and is accessible via a custom toolbar.


Data Sources Include:

• 2005 Toxic Release Inventory (EPA)
• Air Quality monitoring Stations & Data (TCEQ)
• Various Basemap Layers (NCTCOG)

If you're in the area, drop on by...

Large Table Manipulation: ArcMap vs. Microsoft Access

Helped an engineering student yesterday to display a table with over 500,000 XY coordinates and convert it to a shapefile. The table the student brought was a 2.5 gig CSV file. It gave me some unexpected troubles.

The conclusion is that the manipulation of large tables is often times best done using Microsoft Access. Specifically, converting field types for large tables with hundreds of thousands of records is best done in Microsoft Access, and not in ArcMap.

If you are interested in the particulars, read on below.

Brought the table into ArcMap, but every field in the table was read by ArcMap as a text field. Opened the table in ArcMap, and all the values were numerical coordinates. Opened the table in Excel, and of course the table only partially opened as there were too many records. The portion of the table that did open were all numerical coordinates, but Excel was also reading the fields as text. No problem. Switched back over to ArcMap and attempted to add two new double fields and use the field calculator to copy the text fields into the double fields. ArcMap is unable to add fields to text files, so we had to export the table as a DBF first. This took over 5 minutes as there was so much data. Every time we tried to add a new double field to the large DBF file, the screen would turn white and would hang there indefinitely. We tried this twice, and gave up on each attempt after 5 minutes of inactivity.

Then I had the idea to import the table into a blank Microsoft Access database where I can directly change the field types from text to double. Before changing the field types, I opened the data table and sure enough I saw that every field was surrounded by quotation marks {"} as a text qualifier. THAT was the problem! Switched over the design view and changed the field type and sure enough that did the trick. ArcMap honors Access field specifications and we were able to view the event class in no time. Took a bit more patience waiting while we exported the shapefile, but it was nice to see the completion of what should have been a routine process.

Why were there quotation text qualifiers around each field? Well, the student told me the file was created using Excel 2007, which I have not yet used. Perhaps this is standard procedure for Excel 2007? If not, perhaps there is some setting in Excel that will inadvertently place text qualifiers around fields?

Repeating from above: The conclusion is that the manipulation of large tables is often times best done using Microsoft Access.

Interesting GIS/Geography Related Dissertations

[updated 10/05 to include links to free dissertation abstracts and 24-page previews, compliments of UMI Dissertation Publishing at ProQuest. See comments for more info. Thanks Mike!]

Here's the latest installment of GIS-related dissertations that have caught my eye. Previous lists include 11/05, 01/06, 01/06, 04/06, 07/06, 10/06, 04/07. (Of course, this is not a comprehensive list...only those I find interesting.)

• An investigation of northwest Arkansas High School students' attitudes towards using GIS in learning social studies, by Al-Kamali, Abdulla Ahmed, PhD, University of Arkansas, 2007.
• "The research consisted of three parts: One, the quantitative aspect of the research was completed through a two-part questionnaire in which 204 students attending a high school in Northwest Arkansas responded. The first part contained questions concerning demographic information; the second part contained a 5-point Likert scale to measure their attitudes towards using/learning GIS. Descriptive statistics, Independent t-test, One-Way ANOVA, and tests of between group effects (Two-Way ANOVA) were used to analyze the data collected. The third part contained GIS use information."
  


• A descriptive spatial analysis of the relationship between geology, aeroradioactivity and incidence of cancer with indoor radon measurements for the state of Virginia, by Simoni de Cannon, Fiorella V, PhD, George Mason University, 2007.
• "This dissertation showed that county averages of the incidence of lung cancer, chronic bronchitis, and chronic pneumonia are well correlated. Also, the dissertation showed that aeroradioactivity can be used to locate geological areas with higher soil radioactivity and areas with higher indoor radon."


• Computer-aided qualitative GIS (CAQ-GIS) for critical researchers: An integration of quantitative and qualitative research in the geography of communities, by Jung, Jin-Kyu, PhD, State University of New York at Buffalo, 2007.
• "This project explores a new approach of blending qualitative research with GIS by creating two software-level strategies. The first strategy is to include qualitative data directly into a GIS database from the beginning of research, thereby aiding in the comprehensive analysis of multiple forms of data... The second strategy meshes GIS with CAQDAS (Computer-Aided Qualitative Data Analysis Software), specifically the ATLAS ti ^® program."


• Market competition and nursing home quality of care, by Geng, Yisong, PhD, University of Minnesota, 2007.
• "Findings: Nursing homes in counties with smaller HHI or closer to competitors have more staffing hours, fewer deficiencies, higher quality measure scores and higher total scores. Predicted average occupancy of other nursing homes in the county is not significantly related with quality in general, the reason may be that it does not have sufficient variation."


• Market transition and the spatial dynamics of Internet development in China, by Zhang, Jun, PhD, University of Minnesota, 2007.
• "This thesis examines the spatially uneven evolution of the Internet industry in China's changing socio-economic environment. Using a census data, I demonstrate the evolution of founder structures and business models of China's Internet startups in space and through time. Taking a system view of institutions and a resource-based view of industrial development, I interpret the decade-long history of China's Internet development, characterized by the interplay of multiple factors at multiple geographic scales, through an analytic narrative."


• Spatial query processing using Voronoi diagrams, by Sharifzadeh, Mehdi, PhD, University of Southern California, 2007.
• "In this dissertation, we propose a new index structure, termed VoR-tree , that incorporates Voronoi diagrams into the R-tree index structure for I/O-optimal processing of nearest neighbor queries on point datasets. The neighborhood information encoded in Voronoi diagrams allows a VoR-tree-based algorithm to optimally explore the data space towards the result."


• Service oriented architecture for geographic information systems supporting real time data grids, by Aydin, Galip, PhD, Indiana University, 2007.
• "In this dissertation we propose and evaluate a Service Oriented Architecture for seamless coupling of real-time and non-real time geographic data with scientific geospatial applications. We describe necessary principles and service features for accessing both data types and outline Data Grid architecture to address these requirements. We describe various methods for improving the performance of our Web Services such as adoption of Binary XML Frameworks and integration of a message oriented middleware for streaming the responses. We introduce a novel approach to map Geography Markup Language (GML) based XML queries to relational database queries."


• The effects of war on land-use/land-cover change: An analysis of Landsat imagery for northeast Bosnia, by Witmer, Frank D. W., PhD, University of Colorado at Boulder, 2007.
• "The use of satellite technology by military planners has a relatively long history as a tool of warfare, but little research has used satellite technology to study the effects of war. This research addresses this gap by applying satellite remote sensing imagery to study the effects of war on land-use/land-cover change in northeast Bosnia."


• The Indian Trading Path and colonial settlement development in the North Carolina Piedmont, by Dobbs, Gladys Rebecca, PhD, The University of North Carolina at Chapel Hill, 2007.
• "This is a work of historical geography examining the role of an Indian-origin transportation route in the development of the colonial settlement system that emerged in the North Carolina Piedmont during the mid 18 ^th century. This system has evolved in modern times into an important polycentric urban region, and the area's current inhabitants attribute this pattern to the Indian route, the Indian Trading Path. The relationship between the route and settlement development has not previously been tested, however."

Summer 2008: GIS Librarianship Course

The confirmation is only preliminary at this point, but I am preparing to teach a GIS Librarianship course next summer (2008) for the University of Arizona's School of Information Resources and Library Science. This course will be 100% distance education, so I encourage any library student or librarian out there with an interest in GIS to consider enrolling. I will post more details here as the confirmation becomes more official and as the date gets closer.

This is so, so great. For the past few years, I have been teaching GIS and spatial analysis courses for Earth & Enviro Sciences, Marketing, and Real Estate, but it has always been a dream of mine to work with library science students to attempt to pass on and convey my excitement and enthusiasm for GIS librarianship. As part of this dream, I have posted practicum opportunities at the UNT School of Library and Information Sciences for the last 4 years. My only bite came over the summer, but the night before the practicum was to start, he/she canceled due to an emergency. Ah well...

Anyway, I am typing this and am giving my first concrete thoughts as to how to structure such a course. Here is my first draft/first thoughts outline of the course:

• What is GIS and what contributions can a library provide to this subject area?
• This will be a nice, but brief overview of GIS and GIS librarianship. The final section (below) will consider the role of GIS librarianship in greater detail, but this will serve as a nice introduction.
  
• Basics of GIS
• My thoughts are this will take up the first half the course, and will be a condensed version of an Intro to GIS course, with a focus on data types, data acquisition, and data preparation. In essence, the pre-analysis concerns of GIS.
  
• Marketing GIS services
  
• Managing GIS data
• This will focus on metadata and the use of various types of metadata catalogs.
• Data Acquisition
• This will focus on selecting and processing datasets from data vendors, public data archives, and government organizations.
• Types of GIS Services Offered by Libraries
• I think it best to end the course with this topic. At this point everyone will have a deeper understanding of the potential and can actually contribute to the conversation. This might even make a great final project.

I am sure I am missing something(s) here, but I feel pretty good about getting at least this much down. Of course, if anyone has any comments, suggestions, or criticisms (keep those clean, yeah?) feel free to leave them as comments here or email me at: been [[-at-]] uta [[-dot-]] edu.

Academic Cycle: Projection Time!

GIS projects must be due soon in many of our classes as folks from all over campus are coming into the GIS Lab for assistance with projecting their data. Funny how everyone from different departments and different classes are all focused on the same thing. Wonder if this has any relationship to how women's menstrual cycles sync after living close together? Does this mean the various departments are psychically, biologically, or spiritually moving in unison? Ha!

Anyway, by far the most popular question in the lab recently has been confusion about defining and projecting data. Map of Texas showing up in the Pacific? Saw this a few times in the lab yesterday. Always the same cause. An undefined shapefile, most of the time geographically referenced, was projected to some form of Lambert Conformal Conic without defining the projection first!! ArcMap can not transform a projection without defining the initial projection first. Students from many disciplines are having a tough time understanding when and why it is necessary to define their projection before re-projecting their data.

As stated in the ArcGIS 9.2 Help, the projection definition "records the coordinate system information for the specified input dataset or feature class including any associated projection parameters, datum and spheroid. It creates or modifies the feature class's projection parameters."

So, how can you know when your data's projection needs to be defined? Well, first clue is ArcMap will give you a popup letting you know. These are not ads. Read them. Second clue is that your data will not be in the correct position, such as a map of Texas in the middle of the Pacific. However, if all of your data is consistently un-defined, then you will only notice this if you add another layer that is properly defined. Tough part is that if a student adds 10 layers whose projections are undefined, and then adds another layer that is properly defined, it might seem as if the defined layer is the one in the incorrect position. Like a democratic vote, un-defined wins 10 to 1.

Why is this confusion happening more than it used to? My opinion is the loss of ArcMap's assumed geographic projection definition. ArcMap 9.1 (and previous versions) would detect and assume that un-defined shapefiles were geographically referenced (decimal degrees). If the data is indeed geographically referenced, then ArcMap's assumption removes the need to define the file before projecting it. However, 9.2 makes no such assumptions. Undefined data must be defined, whether it is geographically referenced or not. Which is better? I sure do not know, but I do know that this is forcing our students (and faculty) to give a lot more thought to projections and datums than they needed to in the past.

First GIS Workshop - Standing Room Only!

Yesterday's workshop was a success on every conceivable level.

We had 30 participants, which in a 27 PC lab is the best. Breakdown was 4 faculty, 12 grads, 8 undergrads, 2 staff, and 4 non-affiliates (City of Arlington intern, couple of investors, and an alumnus). By the 5 minute mark every seat was taken and a noticeable number of people dropped by and left when they saw how crowded it was.

Everyone seemed very engaged. Lots of discussions amongst everyone, and the topics invariably lead to how the techniques and/or data sources we used can be incorporated into their own research. Almost everyone was able to complete the entire workshop, which culminated in the creation of a Google Map highlighting their results. These Google Maps were generated using our ArcMap2GMap script.

The university Public Affairs Office sent out a nice press release, that was mentioned in the Ft. Worth Star Telegram. The Shorthorn (university newspaper) printed a fantastic story this morning that featured a number of nice interviews with students.

Let's hope this excitement bleeds over into our next workshop on air pollution in the metroplex, scheduled for October 25. Details here.

Sleepin' at the Bowl: Best Locations for New Hotels in Arlington, TX

Workshop Tomorrow!!

Download Full Workshop Materials (coming soon)
View Zohoshow Presentation
View Handout (1st draft)
View Flyer

Last week, I posted the basic details about this semester's 3 workshops. Well, after a round of late night shifts, the new hotel workshop is just about ready to go.

No doubt about it, there is so much data included in this workshop. Everyone will see that I got carried away... Here are highlights of the details of the data sources used. Complete details in the handout.

• Texas Comptroller's Office
• Hotel locations including units available and gross receipts
• NCTCOG
• Freeways
• City Boundaries
• Land Use 2005
• FEMA Q3
• Features (points of interest)
  
• City of Arlington: GIS
• Zoning
• Tarrant Appraisal District
• Parcel Boundaries
• AAAA Primary Real Estate account data
• LOCA dataset for geographic processing
• Columbia Regional Geospatial Service Center
• 2004 NAIP infrared 1-meter aerial
  
• 2005 NAIP true color 2-meter aerial
• Simply Map (subscription database)
• 2006 Demographics by Census Block Group
  
• education
• income
• vacant units
• population
• various retail sales estimates
• various business counts
• crime index
• various sales potential indices
  

Here is an overview of the how the workshop will go:

1. Explore the data (attributes and symbology)
1. All library GIS workshops are designed to educate, intrigue, and entertain both GIS users and those merely interested. This portion, while slightly slow for the GIS users, makes those new to GIS comfortable with the software.
2. Join hotel data to Tarrant Appraisal District parcels
1. This will allow us calculate ratios such as receipts per sq f, and to calculate the total sq ft of hotel space.
3. Calculate densities
1. We will calculate point densities of:
1. number of units available
2. number of hotels
3. receipts per unit
4. receipts per sq ft
4. Suitability site selection
1. reclassify and combine
5. Narrow suitability raster to identify most suitable locations
6. Generate Google Map displaying results

It should be a lot of fun. If you are in the area, you should definitely stop by... ;)

What did I not have enough time to incorporate? While we do have the zoning and landuse data in the database, it is not incorporated in the site selection process...mainly due to lack of time. It can, of course, be referenced at any time by adding it as a layer.

Texas River Information Management System (TRIMS)

Attended a demonstration of the new Texas River Information Management System (TRIMS) in early September at the Ellison Miles Geotechnology Institute (located in Farmers Branch, TX on the Brookhaven College campus). TRIMS went live in late August.

This ArcGIS Server driven web mapping application is a fantastic resource for us here in the Trinity River Basin (North-Central, Central, and into Eastern Texas).

Features from the demonstration invite:

1. Access to over 30 spatial layers including aerial photography, streams and rivers,reservoirs, 2000 census data, elevation, political districts, USGS topographic maps, groundwater, boundaries, dischargers, water quality sampling sites, and roads
2. Metadata for each layer
3. Ability to measure length and area and create graphics

Here is a list of the layers included.

We discussed two primary uses for the interactive website. First, as a tool for landowners to manage natural resources. Second, as an educational tool to be used in the K-12 and college classroom. There is no doubt that the ease of use and thorough content in TRIMS will serve both of these purposes. I shared this resource with our science & engineering librarians, and we have been pushing this resource to our non-GIS savvy students. As all of the data is derived from public resources, we have access to all of the information available through TRIMS in GIS format, or in formats that can be incorporated into a GIS. We are planning a workshop next Spring semester targeting our education faculty and students, and this is one of the key resources I plan on showcasing to them.

One fantastic feature I want to point out is the live steam gauge data. Using the site ID number, clicking on any gauge will create a link directly to the USGS real time water data. For example the map on the left, the gauge at the Clear Fork of the Trinity River in Fort Worth links out to the following real-time water data. This is a great example of what is most great about this site. The way it brings together numerous public data sources into one user-friendly interface. Yes, as the Map Explorer points out, some folks will complain that it is not as easy to use as Google mapping products, but this is because of the 30 spatial layers and the great markup ability included in ArcGIS Server.

This is a great resource that I look forward to promoting as often as I can.

Some background info about TRIMS:

As part of Governor Perry’s Trinity River Basin Environmental Restoration Initiative, the Trinity River Authority’s Clean River Program in cooperation with TCEQ has funded the Trinity River Information Management System (TRIMS) through Texas A&M University’s Institute of Renewable Natural Resources.

Coast Guard Site Selection & ModelBuilder

I am teaching the spatial analysis course this semester, and the first two projects everyone is working on is based on a suitability site selection exercise. First, they need to identify the best locations for the development of a new Coast Guard facility on the Gulf Coast. Second, they need to develop an ArcMap Model to automate this suitability process.

Here is their exercise scenario (fictitious, of course) :

The U.S. Coast Guard is concerned about the possible damage to the environment and natural habitat along the Texas Gulf Coast if an offshore oil spill were to occur. The Coast Guard has obtained funds to build a new facility whose primary purpose is to protect the environment.

They are required to use at least one Census 2000 attribute by tract, and whichever data sources they deem necessary from the Texas General Land Office. There are some great Gulf Coast environmental and habitat resources at the GLO, including the Environmental Sensitivity Index Shoreline, In-Situ Burn Exclusion Areas, Offshore Oil/Gas Platforms, Priority Protected Habitat Areas, Wildlife Refuges, and much more. For our educational purposes, it is nice that some of the GLO's files, such as the TXDOT Roads/Highways, are undefined shapefiles to give everyone the challenge of defining it.

After the initial suitability site selection project, everyone will create an automated Model using ArcMap's ModelBuilder. Standard suitability analysis such as this is so straight forward that it works well in Models. However, I found a bug in the latest ArcGIS 9.2 SP3 service pack. Seems as if the Classify button is grayed out when the Reclassify tool is incorporated into a Model. Of course this only occurs when the reclassification scheme is a parameter. I never saw this before and can only assume this is the result of this latest service pack, which was released in August. Searched the user forums, but surprisingly could not find anyone else who come across this problem. I found a work-around, however, which has thus far worked well enough for me not to really complain about this. Found that if you also add the reclassification field as a parameter, then the Classify button become activated. Seems as if the Model is not accepting the reclassification field as specified by the developer, and is requiring the user to specify the reclass field. Good enough. Perhaps this is a new safeguard feature and not a bug.

With that said, their presentations for their initial possible site selections are next week. We've got some great students this semester and I am really looking forward to seeing what they came up with.

Web Data Subscriptions: Simply Map & Geolytics On-Line

Our data users are so loving these new online data subscription options available through products such as the Geolytics On-Line Data & Maps and Simply Map. They can hop online, on or off campus, and access the data they need. These products make accessing demographic and business data as accessible as finding journal articles.

I had two faculty this past week alone make an effort to let me know how pleased they are that they and their students can access such data so easily online. Previously, we purchased such datasets via CD or downloadable files. Geolytics products were traditionally distributed as CDs to install on stand-alone PCs. Acquisition departments do not enjoy purchasing electronic monographs such as these, and then users needed to come into the library to access this data when the majority of our electronic holdings are accessible from off-campus. We have also been purchasing Applied Geographic Solutions (AGS) marketing data on the block group level from Spatial Insights, but this data was also distributed on CD for a single user. We worked out a solution where we received permission from Geolytics and AGS to serve the CD data using Citrix (and we continue to do so), but this requires the client to install an application. While the installation is small and free, it imposes numerous barriers that are not present for the majority of the library's other databases. Also, students do not normally have permissions to install software in campus labs.

Now, not only can users access their data so easily via the web, but they have choices to generate and download customized maps, delimited files, and even shapefiles containing their selected attributes. Geolytics CDs have always had this ability, but as of this summer, Simply Map now also provides the ability to generate and download shapefiles.

Seems like every month or so, Geolytics sends an email about another title they now provide via the web, and I discussed over the summer with the folks over at Geographic Research, Inc. about plans they have to build addons to their base Simply Map package. I hope they compete viciously with each other. That's the best!

Bottom line is the users are very happy, and when they actually take the time to thank me then that makes me happy as well...

By the way, I am sitting all alone here by myself during my first couple hours providing GIS Research Assistance in SL. Sure, it gives me a chance to write up this post, but someone come over and chat a bit, yeah?