Cropping and Resampling

For steps one and two, each of the 121 raw raster images (39 snapshots of snow depth at 3m, 39 snapshots of snow depth at 50m, 39 snapshots of SWE at 50m, and 4 snowoff rasters) was given assigned to a batch job in which it was first cropped and corrected and then resampled to 3 lower resolutions. 3m snowon data (snow depth and SWE) were resampled to 10, 30, and 50m. 50m snowon data was resampled to 100, 300, and 500m. 50m data was thus obtained both via raw rasters and via resampled 3m rasters to facilitate validation of our resampling techniques. In the case of snowoff rasters, since these were only provided at 3m, they were resampled to 10, 30, 50, 100, 300, and 500m. Node memory constraints in the Yellowstone cluster, where the majority of our core-hour allocation was located, prevented full CPU utilization for cropping and resampling of the 3m images. Thus, our jobs were run on the Geyser cluster, with shared-memory nodes that contained 1TB in memory. Parallel processing was invoked using NCAR's "command file" framework, which allows for flexible Multiple Program Multiple Data (MPMD) parallelism as long as each "command" is independent. To use this framework, we created a file with one command per line and then passed this script as an argument to the batch submission script. The script then called a custom mpirun.lsf executable and passed this "command file" as an argument. We were using a SPMD approach, so did not need the extra MPMD flexibility; nevertheless, this framework provided the simplest way to call a given script independently on multiple datasets within the batch environment.

DEM Transformations

With the exception of the irradiance and MUHA outputs, each of the DEM transformations was essentially a neighborhood analysis performed at each pixel and, as such, could reasonably be parallelized to independent processes via a domain decomposition with no communication necessary. Figure \ref{837925} demonstrates one example of the type of neighborhood analysis performed to generate these transformations. The two exceptions, however, involved horizon angle calculations using a greater extent of the underlying DEM. These transformations were multiple orders of magnitude more expensive, and thus the highest resolution (3m) versions of these transformations were the rate limiting step within our data preprocessing pipeline. We were not able to perform these steps in serial on a single processor due to 24-hour wall-clock time constraints, but a rough estimate of 72 hours for the irradiance calculation seems reasonable.