Ferret-Thredds Data Server

To create F-TDS we developed a Ferret I/O Service Provider (IOSP) for the Java netCDF library. Ferret is a legacy command-line analysis and graphics package that reads COARDS and CF-1.0 netCDF files. After opening one or more netCDF files, users can define new virtual variables which represent the result of some analysis operation applied to one or more of the data variables. By registering the Ferret IOSP with the THREDDS Data Server (TDS) a Ferret script which reads netCDF data and defines virtual variables then becomes an OPeNDAP data set. All of the variables (the real and virtual variables) defined by the script are visible to OPeNDAP clients through the netCDF API.

One advantage of Ferret is that it only does calculations on the virtual variables when underlying data are required and then it performs the analysis calculations only on the sub-set of the data needed. These two characteristics are carried through to the IOSP such that the OPeNDAP client can "see" the entire virtual data variable, but the IOSP only performs the analysis calculations when the client makes a demand for the data.

The Ferret IOSP and THREDDS Data Server combination can also be used to enable on-the-fly server-side analysis. Data access information for multiple data sources and the Ferret commands to define virtual variable can be embedded into the OPeNDAP URL for this server. This information is parsed on the server; a new script is dropped into the scan area being monitored by TDS and the new data set with its virtual variables is immediately available via the TDS.

This IOSP framework draws heavily from concepts for controlling legacy command-line applications from Java introduced in the GrADS Data Server and therefore could easily be generalized to any command-line application that does analysis on netCDF data. We think it would be a good idea to develop a common syntax for communicating the underlying mathematics of the server-side functions. Having a common syntax would simplify the work for client writers and users who want to take advantage of server-side analysis available from GDS, the Ferret IOSP/TDS and other frameworks and servers. We would appreciate some suggestions, comments and volunteers toward nailing down a server-side analysis framework.