Post-Workshop Comments

Al Rea – Thanks so much for putting on this workshop. I think we are on our way to making some very significant progress. It was actually pretty amazing to see how much is available right now, and it was interesting that almost everyone seemed to learn about resources they didn't know existed yet. I count myself among those who learned about several useful web services at the workshop. I expect that soon the list of such services will be growing at a rate such that the hardest part will simply be finding what you need out of all that is there.

As I've been thinking over the week's discussions, at least for the U.S., I see a couple of clear directions I think we should be pursuing as a community:

First, we need to make the data and services that currently are available much easier to use. This would include map services to provide graphical views, and to a great extent this is already in place. I'm sure there is room for improvement on these, and the templates and logical framework we began outlining late last week should help a lot in that regard.

Also, a lot of geoprocessing or functional web services already exist to make those map services even more useful. For everyone's reference, here are a couple links to the most useful services I know about. I'm sure there are others:

http://www.epa.gov/waters/geoservices/index.html
http://streamstatsags.cr.usgs.gov/webservices/wsui.htm

One near-term need I see is the development of GeoProcessing front-ends to those existing services. Ideally, I would like to be able to load a toolbox into ArcMap that would provide the ability to use many of the above services. As we discussed, it is simply too hard to make use of those services right now. As a member of operational programs like StreamStats, NHD, and NHDPlus, I think one thing that would really help us in the short term would be a GP programming example, preferably in Python, that shows how such services can be integrated as tools on the desktop, using services and data from the "cloud". Given our budget realities and priorities, however, I don't mean to imply that such an example would necessarily be enough to start the ball rolling. In fact, we (in USGS specifically) probably don't have the resources to pursue building GP tools for all these services. Without a code sample, though, it is almost a certainty that we won't be able to provide an easier interface to these services. This is where the community might really be able to help. We could then focus on continuing to serve and enhance the data and services themselves.

The second major direction I think we need to work on is the longer-term refinements to the ArcHydro Data Model. Tuesday night a group of us from the NHD, NHDPlus, and StreamStats teams met and came up with a list of enhancements to the current ArcHydro Data Model and Tools that we think are needed. I'm including those here, so they don't get lost in the shuffle.

• More emphasis on the vector network, to take advantage of:
• Better handling of divergent flows
• VAA query-based navigations for  speed
• tree-walking approach to develop main flow paths/channels
• Populate the Z-values on NHD--3D network
• Precompute basin characteristics at the catchment level (instead of HUC)
• More detailed tracking of water use to allow more accurate water budgets

Here are some more forward-looking ideas that go beyond incremental improvements:

• A more scalable "global" system for ArcHydro Tools (multi-scale network, catchments, integrated raster-vector framework)
• Either a common identifier for a "chunk-o-water" across scales – or some way to link features between scales
• Network interpolation/modeling from observation data, e.g. real-time streamflow interpolation network-wide from gage observations
• Ability to specify the "state" of selected network elements without rebuilding the network. For example, some streams may need to be seen as connected for peak-flow modeling, but disconnected for low flows. Likewise for canals in irrigation season vs. non irrigation season. Then there are the bi-direction ones, like tidal streams. It's not clear whether this can be done with the current network model, or if the network would have to be copied and rebuilt using different sets of "enabled" attributes. This could be particularly challenging in a "cloud" environment where different users may want a different network. Is this even possible with the current geometric network?

On this second category of more comprehensive changes to the data model, I've been thinking more about it, and I'm wondering if we really need fully-functioning networks at different scale levels. Perhaps it is sufficient to build a complete network at the most detailed scale, which may vary across the landscape (i.e. it is a "patchwork quilt"). If the more generalized levels of the network can be tied to the features in that most detailed level, perhaps that is enough. For example, in the network diagrams on the left side of Slide 15, from the ArcHydroRiverPlan.pptx (Friday morning's tentative plan), I count 15 flowlines in the 100K view, represented by 3 flowlines in the "National view". If each of those 3 generalized flowlines were related to the flowlines in the most detailed view, perhaps we could simply do the network functions on the detailed level, and relate the results back to the generalized level? I don't know for sure. We probably need to look at the use cases and see if they can all be accomplished with such a structure.

Also, in the case of the U.S., it seems to make more sense to try to build from the most detailed level and generalize from that, rather than to start with generalized data and drill down to more detail. The latter is in essence what we've been doing the last couple of decades in the US. However, each step of the way has required rather painful adjustments to the overall framework. (Just ask the NHD folks how difficult it was to conflate from 100K NHD to 24K.) I'm pretty sure it would be much easier to generalize from a more detailed level than it is to go the other way. The NHD group has been working on generalization techniques, and has had some pretty good success. This work could form the basis of such a system. In essence, the Hi-Res NHD currently is morphing into a patchwork quilt as more detailed networks are being added in local areas.

In order to make this work, however, you need to be able to build the network on the whole detailed database. I think this remains a challenge. NHD networks currently are built only at the 4-digit HUC level when being extracted from the national database. This suggests that at least a two-tiered network is needed, unless the geometric network capability of ArcGIS can be dramatically improved. On the raster side of the ArcHydro Tools, we have found that such a structure works pretty well, and in fact have found the need to expand it to at least 3 tiers. If we can design an N-tiered structure that retains network integrity on each tier level, as well as allows inter-tier connectivity, this would give us a lot of flexibility to accommodate disparate data in a true multi-scale network. I'm thinking this is our best hope for getting ahead of the game in that "arms race" I talked about with ever-increasing data resolution.

I guess I've talked myself in a circle. I started thinking maybe one network at the most detailed scale would work, but I think I'm back to the idea that a multi-tier network is probably necessary. Hopefully this exercise is somewhat useful in framing the questions we need to address. Please count me in for the discussions, unless, of course, I've proven myself too clueless with this rambling walk through the hydrographic garden.

Thanks again for hosting our meeting. I hope you got some sleep over the weekend. Those emails from you were coming at all hours...

Al

Alan Rea, P.E., Hydrologist
U.S. Geological Survey
230 Collins Road, Boise ID 83702
(208)387-1323 ahrea@usgs.gov