Al Rea – Thanks so much for putting on
this workshop. I think we are on our way to making some very significant
progress. It was actually pretty amazing to see how much is available right
now, and it was interesting that almost everyone seemed to learn about resources
they didn't know existed yet. I count myself among those who learned about
several useful web services at the workshop. I expect that soon the list of
such services will be growing at a rate such that the hardest part will simply
be finding what you need out of all that is there.
As I've been thinking over the week's discussions, at least for the U.S., I see a couple of clear directions I think we should be pursuing as a community:
First, we need to make the data and services that currently are available much easier to use. This would include map services to provide graphical views, and to a great extent this is already in place. I'm sure there is room for improvement on these, and the templates and logical framework we began outlining late last week should help a lot in that regard.
Also, a lot of geoprocessing or functional web services already exist to make those map services even more useful. For everyone's reference, here are a couple links to the most useful services I know about. I'm sure there are others:
One near-term need I see is the development of GeoProcessing front-ends to those existing services. Ideally, I would like to be able to load a toolbox into ArcMap that would provide the ability to use many of the above services. As we discussed, it is simply too hard to make use of those services right now. As a member of operational programs like StreamStats, NHD, and NHDPlus, I think one thing that would really help us in the short term would be a GP programming example, preferably in Python, that shows how such services can be integrated as tools on the desktop, using services and data from the "cloud". Given our budget realities and priorities, however, I don't mean to imply that such an example would necessarily be enough to start the ball rolling. In fact, we (in USGS specifically) probably don't have the resources to pursue building GP tools for all these services. Without a code sample, though, it is almost a certainty that we won't be able to provide an easier interface to these services. This is where the community might really be able to help. We could then focus on continuing to serve and enhance the data and services themselves.
The second major direction I think we need to work on is the longer-term refinements to the ArcHydro Data Model. Tuesday night a group of us from the NHD, NHDPlus, and StreamStats teams met and came up with a list of enhancements to the current ArcHydro Data Model and Tools that we think are needed. I'm including those here, so they don't get lost in the shuffle.
Here are some more forward-looking ideas that go beyond incremental improvements:
this second category of more comprehensive changes to the data model, I've been
thinking more about it, and I'm wondering if we really need fully-functioning
networks at different scale levels. Perhaps it is sufficient to build a
complete network at the most detailed scale, which may vary across the
landscape (i.e. it is a "patchwork quilt"). If the more generalized
levels of the network can be tied to the features in that most detailed level,
perhaps that is enough. For example, in the network diagrams on the left side
of Slide 15, from the ArcHydroRiverPlan.pptx (Friday morning's tentative plan),
I count 15 flowlines in the 100K view, represented by
3 flowlines in the "National view". If each
of those 3 generalized flowlines were related to the flowlines in the most detailed view, perhaps we could
simply do the network functions on the detailed level, and relate the results
back to the generalized level? I don't know for sure. We probably need to look
at the use cases and see if they can all be accomplished with such a structure.
Also, in the case of the U.S., it seems to make more sense to try to build from the most detailed level and generalize from that, rather than to start with generalized data and drill down to more detail. The latter is in essence what we've been doing the last couple of decades in the US. However, each step of the way has required rather painful adjustments to the overall framework. (Just ask the NHD folks how difficult it was to conflate from 100K NHD to 24K.) I'm pretty sure it would be much easier to generalize from a more detailed level than it is to go the other way. The NHD group has been working on generalization techniques, and has had some pretty good success. This work could form the basis of such a system. In essence, the Hi-Res NHD currently is morphing into a patchwork quilt as more detailed networks are being added in local areas.
In order to make this work, however, you need to be able to build the network on the whole detailed database. I think this remains a challenge. NHD networks currently are built only at the 4-digit HUC level when being extracted from the national database. This suggests that at least a two-tiered network is needed, unless the geometric network capability of ArcGIS can be dramatically improved. On the raster side of the ArcHydro Tools, we have found that such a structure works pretty well, and in fact have found the need to expand it to at least 3 tiers. If we can design an N-tiered structure that retains network integrity on each tier level, as well as allows inter-tier connectivity, this would give us a lot of flexibility to accommodate disparate data in a true multi-scale network. I'm thinking this is our best hope for getting ahead of the game in that "arms race" I talked about with ever-increasing data resolution.
I guess I've talked myself in a circle. I started thinking maybe one network at the most detailed scale would work, but I think I'm back to the idea that a multi-tier network is probably necessary. Hopefully this exercise is somewhat useful in framing the questions we need to address. Please count me in for the discussions, unless, of course, I've proven myself too clueless with this rambling walk through the hydrographic garden.
Thanks again for hosting our meeting. I hope you got some sleep over the weekend. Those emails from you were coming at all hours...
Alan Rea, P.E., Hydrologist
U.S. Geological Survey
230 Collins Road, Boise ID 83702