Some Great AM Hours & Solving a Problem

I tell you, there are few things that can compare with the supreme satisfaction that I experience when I have a single problem, a solvable problem, and of course the time in which to solve it.  Well, time of course is the scarcest commodity, but that’s what the AM hours are for eh?

And, let’s face it, this is what attracted (and still attracts) me to librarianship.  Every day I manage data/information, research data/information, help and educate others to do the same, but am rarely forced to interpret or compile the data into any hard and fast thesis (aka write a paper) and rarely am faced with a project deadline similar to the private sector.  Sure I have to give a little in terms of a smaller salary (slightly…or perhaps not), but it is so worth it.

Why am I writing about this now?  Well, I just spent some great AM hours tackling a problem, solving the problem, and now (best of all) I get to hand it off to the actual researcher who will do the actual work.  Here is the story:

Geosciences professor is having trouble calculating the azimuth between a series of points in a feature class.  After I imagine some headaches trying to figure this out, the professor asks me if I have done this before.  Well no, I say, but surely there must be something built into ArcGIS that can handle this, right?  If there is, the professor says, I sure could not find it.  All right-y then…

So, with a definitive task in mind, I grab myself a nice tall cup of water, sit down at the computer, and pleasantly begin the search.

Easy to use button?

• Nope.

Not even a geoprocessing tool?

• No.

Shoot on over to ArcScripts but no script in sight.

• OK then.

Pull up to the ESRI Forums.

• Wow, there is something here.  Others have had this problem before.


• Found a thread: Measuring azimuth between 2 points.


• This is good.


• The poster, one H Gonzago, posted an ArcObjects sub-procedure.  The post states that he got it (copy & paste) from the ArcObjects Developer Help.


• OK, so I quickly create a button in ArcMap, call the sub-procedure on the onClick event, and presto it works like a charm.


• A for/next loop will be necessary to process all the points, and I sent this to the professor.  Of course I even offered to write the loop for the professor.

Now this was a couple of well spent AM hours that has given me much satisfaction.

Day / Night Population Counts

It was not too long ago that I obtained daytime/nightime population estimates on the block group level for a landscape architect professor.

Well, this must be ahot topic as the US Census Bureau just released estimated daytime population counts for states, counties, and select places.

Via: Directions Magazine

As I predicted, Dallas experiences the second-most percent change from day to night population for large cities. First was our neighbor Houston. Topping the list is Lake Buena Vista city, FL, which experiences a whopping 3076.80 employment to residence ratio. Yes, this boggles my mind.

So, now I have the task of joining these tables to feature classes, generating metadata, and including the files in our metadata catalog. Then students will have easy access to the files.

KU: Overlay Google Maps With Single Image

The Mapdex Blog announced that "Praveen Ponnusamy, a savvy developer in our group" enabled a method to include a single (untiled) image from an ArcIMS service to overlay within a Google Maps API-driven web map. This is very cool.

Via: Mapdex Blog

This advancement is the latest among many web mappers out there dabbling with including ArcIMS layers within Google Maps. Also see this earlier Mapdex Blog post, ka-Map, and discussions surrounding this topic here and here.

Web-Based GIS News Aggregators: Planet Geospatial

James Fee, Spatially Adjusted blogger, has just developed a web-based GIS news aggregator called 'Planet Geospatial.' This is a fantastic resource that I think many folks will take advantage of.

Now, I find this extremely coincidental. Is this a great-minds-think-alike moment?

Well, we have just finished development of a similar concept that we were going to release next week. Our project, un-fantastically called (for the moment) 'GIS News', can be viewed on our test server here. We might play around with the documentation/help a bit this weekend, but I will post here when the project is shifted over to our production server. The purpose for the development of this project is to help the growing GIS community on campus keep up to date with the exciting world of GIS.

Yes, this is the project we were working on when our custom-created aggregator was set to pull feed content in every 5 minutes. Sorry about that. We now have it set to pull fresh content every hour. Please comment here if this is still too unreasonable.

Naticus Cartographica Posts GIS Books He Teaches From

Naticus Cartographica, the author of the All Things Geography blog recently posted the books he teaches from.

For those seeking to learn Remote Sensing, GPS, or GIS programming, give these books a shot: (We actually have all of them in our library.)

Remote Sensing and Image Interpretation - Lillesand, Kiefer, Chipman - 5th ed, 2005
Publisher: Wiley & Sons
ISBN: 0471152277

Introduction to Digital Image Processing, John Jensen, 3rd ed, 2004
Publisher: Prentice Hall
ISBN: 0131453610

GPS Made Easy: Using the Global Positioning System in the Outdoors, Lawrence Letham
Publisher: Mountaineers Books; 4th ed.
ISBN: 0898868238

Build Your Own Database Driven Website Using PHP & MySQL, 3rd edition, 2004, Kevin Yank
Publisher: SitePoint

Getting to Know ArcObjects: Programming ArcGIS with VBA, October 2003, Robert Burke
Publisher: ESRI Press
ISBN: 1-58948-018-X

Elements of Photogrammetery with Applications in GIS, Wolf and DeWitt, 3rd ed, 2000
Publisher: The McGraw-Hill Companies, Inc.
ISBN: 0-07-292454-3

Top 10 Most Common Student-Encountered GIS Problems

I have been working with and teaching undergrad and grad students GIS for 4+ years now and have compiled a list of the 10 most frequent problems that they encounter. In my current position, I spend about 15 hours per week holding office hours in the main GIS Lab on campus, where students, staff, and faculty can visit for GIS assistance, and rarely do I have a free moment. (Well, perhaps during intercession.) I often find myself explaining the same concepts and pointing out the same resources over and over again, so I wanted to pull together this list. ESRI ArcGIS is the main software application used on campus and so many of the examples below refer to this application.

This list is ranked with number one being the most frequent problem.

1. Problems with Spatial References
   
   
   
   • Too often students ignore it entirely. I put some of the blame onto ESRI ArcMap, whose powerful on-the-fly projections lull students into not giving enough consideration to the spatial references of their files. Precision is sacrificed, and of course when the analysis portion of their project begins, errors will start creeping in.
     
   • Students often have trouble understanding the differences between defining a spatial reference and actually applying one..
     
   
   
2. Too task-focused / Not management oriented
   
   
   
   • Projects are often begun without a clear plan or idea of what they are out to accomplish or which data sources they will need. Now, I understand that this is common among college students, but the troubles students will run into are exponentially higher with GIS-involved projects than with papers based solely on print materials. If their research rests upon a particular dataset, it is disastrous if they find this out after many hours of work.
     
   • Files are often named poorly and disorganized within numerous folders among countless older versions. I can never stress how vital it is to properly organize GIS data files as even they will not remember which files are actually being used in their projects.
     
   • Poorly saved .MXD ArcMap project files. I also blame ESRI a bit for this. The default in ArcMap is to save the relative path of the data files in the project file. This can be changed to the full-path, but this is feature is not in an intuitive place for students to find.
     
   
   
3. False Assumptions
   
   
   
   • The number 1 false assumption is the belief that all the datasets they need is inside the GIS software or on the CD included with the book (whichever book is assigned for their class).
     
   • The assumption that there are no GIS products out there beyond ESRI. For example, students who work at non-profit organizations would like to be able to join TIGER shapefiles to Census data to support grant applications, and are not optimistic about ArcMap's pricetag. There are some fantastic easy-to-use free applications out there.
     
   
   
4. Trouble Finding Data
   
   
   
   • Some students cannot find the data they need. This is understandable. GIS data is very dispersed and while there are several good efforts, there is no single 1-stop shopping.
     
   • Now, some students actually purchase data. This frustrates me a bit. Why? Because 9 out of ten times (perhaps even more) the data is freely available. I hate to see freely available data redistributed for a fee without even any added value. The times when the data is not freely available? Well, we might have already have purchased it. If not, then perhaps the library can purchase it and then the entire campus will have access.
     
   
   
5. Poor Understanding of Relational Databases
   
   
   
   • Students who have never dipped their hands into the innards of a relational database before are at a disadvantage when learning and using GIS.
     
   • Cardinality principles are essential
     
   • An understanding of field types and what they can hold
     
   • Microsoft Access is a single-user database system. Ensuring that a personal geodatabase is not being used by another application or set to read only cause students a lot of troubles.
     
   
   
6. Trouble with File Conversions
   
   
   
   • Converting .e00 files. Yet another problem I will lay at the doorstep of ESRI, version 9.0 in particular. The conversion tools involving ArcInfo Interchange files are extremely hidden in ArcCatalog. I do not know whether this tool is more accessible in version 9.1.
     
   • CAD to Feature problems. A great geoprocessing tool exists, but the concept of using a bridge between the CAD and a feature class is a difficult concept for our architects and civil engineers who want to use the capabilities of ESRI software.
     
   • Importing uncommon raster formats
     
   
   
7. Insufficient Understanding of Statistics
   
   
   
   • Students have an idea about the type of analysis they want to do but do not have the background in statistics to do it properly.
     
   • Examples include spatial regression or kriging. Between ESRI and and GeoDa, we have the tools for many complex statistical operations without having to export the data into a statistical package, but a background in statistics is necessary to obtain relevant results.
     
   
   
8. Confidence Problems
   
   
   
   • The nature of being a student (for many students) is that they are not confident enough in their ability to figure it out. I try to impress upon them how nobody knows it all, and that after learning the basics of GIS, most of it is non-stop troubleshooting and figuring things out. Students should not give up when they face a new GIS problem. The need to persevere because, in my opinion, that is what it's all about.
     
     
   
   
9. Problems with Cartographic Principles
   
   
   
   • So focused on the data and the analysis that not enough thought about the symbols used, layout, map audience, placement, font, etc... It is a shame when students develop a beautifully developed GIS project and the maps included in their powerpoint or in their papers are practically unreadable.
     
   
   
10. Technology Reluctances
    
    
    
    • A good example is the reluctance to touch anything that even resembles command-based or scripting languages. Some students have a fear similar to that of needles.

Proquest Will Offer RSS Feeds Soon...

I posted a little ago how Ingenta now provides RSS feeds and therefore you can pick up the table of contents, for free, of many scholarly GIS publications via RSS.

Well, The Shifted Librarian just announced that ProQuest is also set to release RSS feeds, but most likely only for customers. ProQuest databases include journal and newspaper holdings which may be covered by Ingenta, but also include the ProQuest Digital Dissertations database. This means that as soon as a student's dissertation concerning GIS is made available, we can immediately be alerted via RSS. I would like to see this happen.

Cartographic Connections: Improving Teaching through the Use of Historic Maps

The Special Collections/Archives program area within our library has developed a project called Cartographic Connections: Improving Teaching through the Use of Historic Maps, which is designed to provide access to digital historical maps to secondary education teachers.

Well, the origins of this project predate the Library's involvement with GIS, and so only recently has GIS (spatial reference, vectorizing, overlaying layers, etc.) been considered. I was involved with the creation of a new video introduction for Cartographic Connections, but unfortunately at this time the Realplayer version on the website is still the older version (which does not include any mention of GIS). In the new video, we focused on Roemer's Map of Texas (Topographisch-geognostische Karte von Texas) and we demonstrated how this map can be given a spatial reference (rubber sheeting).

Those folks who have seen this video have really enjoyed this portion and want to give the secondary teachers and students an opportunity to duplicate this exercise without being online. This brings us to my current dilemma, which is how to do this? I have never seen ArcIMS or any other web-based GIS application include the capability to give assign a projection and rubber sheet a scanned image remotely. We have access to ArcGIS Server, but I do not know of anyone on campus who has developing with it, especially not me. I asked those out there for assistance, but seems as though noone knew.

So, the plan now is how best to fake or simulate the rubber sheeting experience. All we really need is for the students to get the feel for the process. Have them match up 3 to 5 points and see the map merge and shift to accomodate these points. We considered Flash, but the development time is prohibitive. I have a meeting today where we will try to brainstorm and figure out a solution.

If anyone out there has a suggestion, please feel free to add it as a comment. I would be very thankful.

Geocaching for Freshmen

About a year ago, the library administration asked me about the possibility of setting up a geocaching exercise for incoming freshmen to familiarize themselves with the campus. I spoke with the GPS guru on campus (professor in the Earth & Environmental Sciences department) and his initial response was a no-go. His main reationale for arguing against this use of GPS was the training the students would need (and would not get) and also a fear for the equipment in the hands of 17 to 18 year olds. So the decision was made to shelve the idea.

However, I just came across this press release from the Earth and Mineral Sciences Library at Penn State: Novel treasure hunt explores Earth and Mineral Sciences Library. This is a geocaching exercise that was mostly participated by freshman. You can see now how my interest peaked as I read this press release. From the release:

"The libraries wanted to reduce anxiety and introduce freshmen to the library during orientation, " says Linda Musser, head of the Fletcher L. Byrom Earth and Mineral Sciences Library at Penn State. "This year's overall theme was sports and geocaching is a hot new sport. We wanted to have some fun."

This is something that I will bring up again with the GPS guru. This sounds like a lot of fun and that's what it's all about, right?

But how still to resolve the initial two hurdles that prohibited us in the first place? Seemed to me that the good thing going for Penn State was that this activity was not mandatory so those participating would be the most eager to learn and properly use the hardware. I bet we can teach the students the minimum they need to know within a couple of hours.

Good. My enthusiasm is reinvigorated about setting this up.

Health Geographics Compare G Maps, KML, and MSN VE

The International Journal of Health Geographics, an open access journal, last month published the following editorial: "Web GIS in practice III: creating a simple interactive map of England's Strategic Health Authorities using Google Maps API, Google Earth KML, and MSN Virtual Earth Map Control."

This article is written by Maged N Kamel Boulos, an ESRI software user at the University of Bath. The article is Maged's exploration of online consumer geoinformatics services (the Google Maps API, Google Earth KML, and MSN Virtual Earth) and a discussion of ESRI's response (Maged focuses on the the upgrade of National Geographic MapMachine as ESRI's main response.) From the article:

The planned upgrade aims at bringing satellite imagery, aerial photos, and street-level data to MapMachine users. Users will be able to access the service through a new viewer that is aimed at a mass audience, and appears to be ESRI's direct response to Google Earth and Microsoft Virtual Earth. However, one important difference from those services is that the ArcGIS back end will also allow users of the new service to accomplish much more sophisticated tasks, such as service area analysis. The next generation of MapMachine will also provide a link to GOS data and metadata to help users discover information about their area of interest or study. MapMachine will include capabilities for 3D globe services, allowing GIS users to "pull in" their own map services to overlay onto a globe.

This is a good introduction into a term that is new to me, online consumer geoinformatics services. I will recommend this to GIS students as such.

GIS Feed Aggregator

The Digital Earth Weblog recently noticed that there were an unreasonable amount of hits from UT Arlington. I want to apologize not just to this particular blog owner, but to numerous GIS-related blogs out there. In our spare time, we have been developing a web-based aggregator so that our students and faculty can have a single location for getting the latest GIS news and happenings, and did not realize how many times it was hitting those feeds we are aggregating. The situation should be resolved now.

Virtual Globes List

Alan Glennon at UCSB's Department of Geography has created a listing of virtual globes. As defined in the blog description, "Virtual globes are three-dimensional representations of earth, and arguably, anywhere else in the universe."

Via: Ogle Earth

In our GIS labs we have loaded ArcGlobe, Google Earth (free version), Nasa World Wind, and Erdas Imagine Virtual Delivery for Virtual GIS. I do not have much experience with any of the others products, but I am anxious to try them out (the free ones at least). I will probably not have the time, but it would be great to develop a comparison chart similar to Vector One's ongoing Spatial Software Comparison Charts.

Spatial Murder-Mystery Event Spring 2006 Semester

LordKingSquirrel.com's recent post about the CBS show Numb3rs reminded me about an idea I have been tossing around for a while now about planning a large-scale (well, as large as I can make it) murder-mystery event which solves the crime using GIS and mathematics, similar to Numb3rs. Except, of course, we will be using computers and electronic GIS. The handful of episodes that I have seen showed the math detectives doing spatial analysis on a chalkboard, which I simply would never even contemplate doing...

Well, here is the idea. Circulate in as many university publications (newsletters, newspapers and such) about a mock crime, maybe a mock murder. For the next 4 weeks or so, follow up with more details. This will culminate in a GIS workshop where we use GIS and mathematics to solve the murder-mystery. Now, if I can also create some good signs/posters around campus, and talk this up as much as possible, I think this can get many students excited about math and GIS that would normally scoff at them. (I know as I was one of these during much of my social science undergrad career.) Nothing at work gives me greater pleasure than to see the light in some student's eye when he/she finally and for the first time understands how mathematics and analysis is actually a powerful tool for solving useful real world problems.

Hi-Rez Pakistan Earthquake Satellite Imagery

Katheryn Kramer, developer of the Islamabad Earthquake Map, has posted a list of satellite and aerial photos of the Pakistan earthquake zone that she received by way of Declan Butler, an editor at Nature.

Via: The Map Room

A detailed map of the affected area dated October 10 can be found at the United Nation’s ReliefWeb site.

Via: Cartography

Now that the geoprocessing workshop is over (went well), I will use some of these images to create a map display in the main GIS lab. This was actually requested by a Pakistani student earlier in the week so it's about time I move on this.

2004 Second Edition TIGER/Line Files Available

The 2004 TIGER (Topologically Integrated Geographic Encoding and Referencing) Line Files are available here. Most relevant, numerous street files (tgrXXXXXlka) have been realigned in this new version of TIGER. For a list of counties whose streets received this face-lift, see here.

Via: GISuser.com

These files are in TIGER format. For those using ESRI software, these can be converted easily into shapefile or feature class format by using the Tiger to Coverage geoprocessing tool.

Back in July, I developed a small script to perform batch conversions from TIGER 94 to shp. The script is not set up to be too flexible but a little tweaking of the source will do the trick.

Geoprocessing & GIS Demonstrations

I want to talk a little bit about the ESRI Geoprocessing functionality a bit here and how wonderful they are for creating GIS demonstrations both for students/faculty new to GIS and for more experienced users.

For the last 3 years (or so) I have held 4 open demonstrations each semester, and about 2 each summer. These demos are 2 hours long and it has always been a challenge to see how much sophistication and excitement I can squeeze into two hours without playing the drill sergeant and rushing them through. This of course would defeat the purpose as no one would show up ever again. So, before incorporating models and scripts into my demos, there was a lot of repetition involving numerous conversions, reference transformations, distance calculations, and such. The students and faulty have always gotten pumped up about the analysis, but I would prefer their first experience with GIS or with a new procedure to be as fun and painless as possible. I normally hit them with a bit of reality when they come visit me during my office hours wanting to incorporate what they saw during a demo into their own project with their own datasets.

Anyway, this is where geoprocessing comes in. For the last few demos I have held starting last summer, I have build models and lately Python scripts which enables the students and faculty to engage in complex and time-consuming analysis by entering a few parameters and hitting the OK button. This leaves time for me to expand upon, in length, the sources of the data we are using, the accuracy of the data, the mathematics behind the particular analysis procedure, the importance of reference systems, and I have even been discussing appropriate journals and books in the library that would help with the particular analysis we are conducting. It is wonderful.

I tell you, I have not had one single student or faculty tell me that they miss working through the analysis the long way during the demos. I have to remind myself that the purpose of the demos is not to teach but to demonstrate particular capabilities and resources that hopefully they will use in the future.

These have gotten so popular that a number of students have asked me to hold a demo showing how I develop the models and scripts. Well, tomorrow is the day and I am just about done. Sure it is cutting it a bit close, but I know of no other way to really do things...

The files for the demo are here.

First Google Maps Mashup of Pakistan Earthquakes

The very first mashup that I have yet come across is Islamabad Earthquake Map, developed by Kathryn Cramer using the free CommunityWalk service.

Via: Cartography

I foresee free services that allow users to easily manipulate the Google Maps API, such as CommunityWalk and Google Maps EZ, will play a very dominant role in the near future as more and more folks want to create their own maps.

Creating Metadata: Who Has the Time?

As I reported here and here, we recently finished up the creation of our metadata search called GeoSpat (Geospatial Catalog). Well, initial development is complete, but I have an overwhelming pile of data that needs to be included in GeoSpat. Who will create the FGDC metadata records? Who can fit it in to their busy schedules?

Initially, the Cataloging department donated a library assistant who would devote some hours each week to GIS metadata creation. This did not go as well as planned because the learning curve to understand the spatial data and GIS were too great. I found it easier on my time to just go ahead and create the metadata myself. But this was during the summer. We are now deep in the heart of the fall semester and I am pulling a number of extra hours just to keep up with my non-GeoSpat responsibilities. So, this week I started to turn to my work study student assistant. (I supervise 1 GRA and 1 work study). She has proven extremely capable with everything that I have thrown at her so far and this also has been going well so far.

To be perfectly honest, we are not coming anywhere close to creating full-FGDC compliant records, and are much closer to the absolute minimal. It still takes just so much time...

Measure of Success: Increase GIS Use

How does a GIS Librarian measure success? Well, here is one way...Documented increase in GIS use on campus.

Well, this happened this wek. I held an open demo entitled "Health Care Administration: Where to Open a New Hospital in the Metroplex and What Services to Offer?" a couple of weeks ago and this went well. Last week I repeated the demo for a graduate Nursing Informatics course. The nursing folks are all pumped up about GIS now and the work order is in to install GIS on the computers in the main computer lab in the College of Nursing, as well on the desktops of a handful of professors. I am set to go over and do an additional demo for the nursing faculty next week.

Now I am going to just sit back, put my feet up, smile, and take a breather for a bit this morning. Which of course is what I am doing right now.......

Oh, and the School of Social Work, which has its own complex of buildings on the extreme north side of campus, now also has GIS installed in SWEL, the Social Work Electronic Library.

Friday Night Revamp

Spent the entire day Wednesday revamping the Friday Night Hangouts project. Tuesday evening, I received an email from a student telling me she went to the site (http://gis.uta.edu/fridaynight) and it was all grayed out. I was planning on using this project yesterday (went well) for a new faculty reception the library was sponsoring.

Here is a bit of background on this project. I wanted to develop a GIS application that would:

1. Be fun
   
2. Convey the excitement that I feel about GIS
   
3. Be easy for students and faculty to understand
   
4. Contain enough analysis to counteract the notion that GIS is just for making pretty maps (I actually run into this often).
   
5. Demonstrate how effective it is to use GIS processing with a Google Maps-driven front-end.
   

So, how to pack all this in one punch? The Friday Night Hangout does it all.

A more thorough description and technical details available on the site.

Turned out that the Google Maps Standalone Mode resource that we were using has gone down. Gave me the impetus to finally go ahead and download that Google Maps API and create a more robust Google Maps interface. Never one to do anything the hard way, I made use of the Google Maps EZ javascripts and boy do this free service make it easy. Had the front end up and running in no time at all. Then of course I had to revamp the Python script (download it here) which took the remainder of the day.

Well, the product is so much more robust now and I am very pleased. With Google Maps EZ, I was even able to add the border of Tarrant County using polylines.