Accelerating SWAT Simulations Using An In-Memory NoSQL Database

Abstract

Due to its versatility, the Soil and Water Assessment Tool (SWAT) has been widely applied to investigate the effects of management activities and climate change on water availability and quality. However, the use of high spatial resolution data and the advantages of SWAT itself have significantly increased the input/output (I/O) demand and thus the runtime of modeling routines that require a large number of iterative simulations. In this study, we proposed a generic scheme to reduce the SWAT runtime by caching the model inputs using the in-memory NoSQL database Redis. Then the SWAT source codes (rev 488) was modified according to this proposed scheme to develop the MA-SWAT (memory accelerated SWAT) model by incorporating a new subroutine known as Fortran_calls_c to retrieve the cached inputs. We then evaluated MA-SWAT with four synthetic hydrological models and five different parallel schemes in a quad-core commodity laptop. The test results showed that when applied with a parallel simulation program, MA-SWAT could achieve a speedup by a factor of 8.4 ~ 10.9 depending on model complexity. Compared with the original SWAT, MA-SWAT significantly improved the computation speed, indicating that the proposed scheme is a desirable method for solving high computational demand problems such as calibration, sensitivity and uncertainty analysis. Moreover, the proposed concept of linking the SWAT model with Redis via the minimalistic C client driver of Redis is a generic method, and it is possible to apply this method to other Fortran-implemented environmental model to alleviate I/O demands.

Keywords: environmental models, generic method, MA-SWAT, Redis, speedup