Design and development of a web-based EPANET model catalogue and execution environment

ABSTRACT Within the field of environmental and hydrologic modelling, there is a growing recognition of the scientific and educational value of sharing both model programs (i.e. the codes that formulate a model) and model instances (i.e. specific input files and model parameterizations). Indeed, numerous cyberinfrastructure tools have been created in recent years to simplify linking, executing, and sharing models. Multiple challenges hinder the adoption of such systems by modelling communities. These can include technological barriers such as installing and managing software and internet systems as well as social barriers such as a culture of not sharing models due to concerns over private or otherwise proprietary information. In some cases, model-sharing platforms are not easily replicated or implemented to fit the needs of a specific modelling community. This paper presents the design and implementation of a model-sharing repository and a model-viewing application, specifically for the EPANET modelling community – the pattern and structure of which could be easily adopted by any modelling community – using existing open source cyberinfrastructure. We used HydroShare as the backend data store for the EPANET model program, model instances, and metadata, and we used the rapid app development capabilities of Tethys Platform framework to create a web-based front-end for the repository and viewer. Results of this experimental work include a functional model repository based on less than 700 lines of code and a light-weight model viewer application that encompasses nearly 100% of the legacy EPANET desktop GUI’s functionality.


Introduction
Collaboration among modelling experts and stakeholders can increase the rate at which science advances and, in turn, can increase the resulting benefits (Voinov and Bousquet 2010). Sharing model programmes and model instances is an integral part of this collaboration. As such, the means by which models are shared should be efficient and effective Zhang et al. 2019).
Many tools and technologies have been developed to help facilitate sharing of scientific research and engineering application models, model codes, and model instances. Some of these include GitHub (Perkel 2016), FigShare (Singh 2011), CSDMS (Peckham, Hutton, and Norris 2013), OpenGMS , and HydroShare (Tarboton et al. 2014). Following is a brief description of how these tools can be used to share executable models and model codes.
(1) GitHub -'Host and review code, manage projects, and build software alongside 50 million developers ' (github.com). Users can collaboratively work on improving model codes and store model executables.
(2) FigShare -'A home for papers, FAIR data and nontraditional research outputs that is easy to use and ready now' (figshare.com). Researches can store, and discover data in a way that promotes collaboration and reduces duplicated effort.
While there is growing capability to share models and model codes as part of open modelling and simulation for scientific research and engineering, there is a still a sense in the literature that many modellers and modelling communities are not sharing models and model codes (Soranno et al. 2015). Indeed, in light of the recent COVID-19 pandemic, there has been a growing call for openness and transparency of models, model codes, and modelling uncertainty in the medical research field . Indeed, in spite of the many tools and technologies that have been developed to facilitate sharing of model instances and model codes, there is a barrier to entry because of the complexity of using and deploying these tools. These challenges have been noted in terms of sharing web-based decision support tools , reusing heterogeneous geosimulation model resources (Zhang et al. 2019), dealing with social, cultural, and institutional constraints (Nativi, Mazzetti, and Geller 2013), and in the context of using open GIS data in models (Zhu and Yang 2019). Additional research has been done on model sharing in the context of encouraging collaboration and improving research reproducibility (Gan et al. 2020).
We hypothesize that, given a thoughtful and careful integration of existing open source tools and technologies, it is possible to assemble systems for model sharing that lower the barriers to sharing and reusing models. Specifically, for this research, our goal is to investigate the use of free and open source software tools to develop a web-based ecosystem for sharing, viewing, editing, running, and analysing simulation results of model instances of the EPANET water distribution network modelling software. We used HydroShare, described previously, as well as the open source web application system, Tethys Platform , to host a catalogue of EPANET model instances and a live, web-based EPANET model execution tool to test and explore these models in real time. Figure 1 shows the software architecture using integrating HydroShare and Tethys Platform. The steps shown in the figure are explained in Table 1. The Model Repository and Model Viewer form the bulk of this research and are described in the following sections.
This paper presents the development of the architecture in Figure 1 specifically to support the EPANET model and community. We also intend this work to represent and demonstrate a pattern and structure, which could be easily adopted by any modelling community, using existing open source cyberinfrastructure. HydroShare and Tethys Platform development teams have argued that these tools are useful for, 'cleaning your desktop by moving your work to the cloud' and in 'lowering the barriers to web app   Tarboton et al. 2014). This research tests these statements by implementing these tools in an operational modelling community. Our experience developing tools and capabilities for the HydroShare software stack informs our use of this piece of critical cyberinfrastructure (Crawley et al. 2017;Sadler, Ames, and Livingston 2016). Our experience with developing Tethys Platform web applications has proven invaluable to informing the development of the apps presented here (Evans et al. 2020;Kadlec, Miller, and Ames 2016;Perez et al. 2016;Sadler, Ames, and Livingston 2016).
There are a number of limitations and challenges to model sharing within any modelling community. Some of the barriers described in the aforementioned literature include technological barriers such as installing and managing software and internet systems. Other barriers discussed in the literature include social barriers such as a culture of not sharing models due to concerns over private or otherwise proprietary information. The research presented here is intended to direclty address the question of model-sharing platforms not being easily replicated or implemented to fit the needs of a specific modelling community. We recognize that this does not necessarily lower the general technical and social barriers within communities. These challenges certainly provide opportunities for future research.
The remainder of this paper is organized as follows: Section 2 describes our selected cyberinfrastructure, HydroShare and Tethys Platform, in some detail; Section 3 describes our selected simulation model, EPANET, in some detail; Section 4 describes the design and development of an EPANET model repository application for storing and discovering model instances based on Tethys Platform and HydroShare; Section 5 describes the design and development of a model viewing and execution web application for EPANET models based on Tethys Platform; finally, Section 6 includes some discussion and conclusions that summarize this work and look forward to future potential opportunities related to this effort.

EPANET
Developed by the United States Environmental Protection Agency (EPA), EPANET is a powerful opensource software suite used throughout public and private industries to model water distribution networks. In the past 5 years alone, there have been over 7,000 references to EPANET in peer review journals. Many of these research efforts lend to optimizing and extending EPANET's capabilities, making it applicable to a wider range of real-world modelling scenarios. Some highly cited research articles related to EPANET include: (1) Uniformly Distributed Demand EPANET Extension (Menapace et al. 2018).
EPANET models are fairly straightforward and easy to conceptualize. They consist of nodes and links (also known as 'nodes' and 'edges' in computer science terminology) representing the water distribution network to be analysed ( Figure 2). Nodes and links come in various types, each with their own specific set of settings and configurations (Table 2). Besides nodes and links, an EPANET model also includes various global settings. Selected head-loss equation is one of the main settings the model depends on. The EPANET modelling software includes the Hazen-Williams, Darcy-Weisbach, and Chezy-Manning head-loss formulas. Based on the global model settings and hydraulic properties of each component in the network a model simulation can be ascertained and analysed. A sample tutorial model is shown in Figure 3 with only a handlful of nodes and links. Many production level models have 80,000 nodes and links or more As is to be expected, larger models require considerably more time to run, especially if an extended period analysis is desired.
Applications of the EPANET modelling program range from measuring water demands and requirements on Table 1. Descriptions of numbered steps from web ecosystem architecture diagram.
Step Description 1 Repository app sends ajax request to backend python to get the EPANET model list 2 Python sends API request to HydroShare through OAuth to get the EPANET model list, including accompanying metadata 3 HydroShare sends the model list in response to the authenticated request 4 Python sends the model list response to the front end 5 The model list is parsed and rendered 6 When a model is selected to be opened in the viewer app the model ID is sent with the request 7 Viewer app receives the model ID and sends an ajax request to backend python to get the respective EPANET model file 8 Python sends API request to HydroShare through OAuth to get the EPANET model file, including accompanying metadata 9 HydroShare sends the model file in response to the authenticated request 10 Python sends the model file response to the front end 11 The model file is parsed and rendered to be viewed, edited and analysed a system, discovering energy inefficiencies in existing water distribution infrastructure, to identifying security risks of contaminant spread through a system and analysing the after-effects of such an event. When new large construction areas are proposed in a city, an impact study of the new water distributions zones on the existing infrastructure is often required. Hydraulic engineers can run extended period simulations of the existing system with the proposed additions and model how the demands at each point in the network change.
EPANET software includes an interactive model editing GUI for Microsoft Windows. The application allows engineers to more easily construct models and visualize the results of a model simulation. Other editors, such as demand curve, flow curve and pattern editors, are also provided to further streamline the creation of EPANET models. This GUI is what we tried to reproduce and improve upon through the EPANET Model Viewer as a lightweight web application presented in this paper.

Requirements
The design requirements for this project provided a definitive starting point for development of the resulting software and also served as a guide throughout later stages of development. The requirements for the model repository app were very clear from the beginning and did not change significantly throughout development and were as follows: (1) HydroShare integration as backend data store and authentication vehicle (2) A catalogue for discovering and storing EPANET models (3) Filtering and searching tools to make finding relevant models easier (4) The ability to download models to be used locally or to open models in the EPANET Model Viewer for editing and analysing  HydroShare's resource discovery page acted as a basic template for the creation of this application as it meets most of the requirements listed above. As the discovery page is intended to be a catalogue for all resource types that exist on HydroShare, the UI/UX is more complicated than was desired for the scope of our design goals. Having a specialized catalogue in the form of a standalone web application for a specific resource type is a useful and powerful tool. It reduces the amount of time to discover the resources and facilitates added features and tools for using them; such as opening a model type resource in another web application that can read and analyse it (see Figure 4). Clicking the 'Open Model' button launches the model viewer which supports viewing, editing, and running EPANET models in a web-based model viewer application without any added software installed on their machine.
In addition to these features, we wanted the process of installing both applications on any Tethys portal to be extremely easy. The Tethys App Warehouse enabled this capability (Khattar et al. 2018). Any Tethys portal with an  App Warehouse can have the EPANET Model Repository and Model Viewer installed on it with a single click.

Technology
Several open source technologies were used in the creation of the EPANET Model Repository. The key technologies include: (1) Tethys Platform -Rapid web application development and installation on Tethys portals; Python backend and Django front-end; Powerful code scaffolding tools; Built-in social authentication hooks ( Figure 5).
(2) HydroShare -Backend model instance data and metadata store; REST API plugin for retrieving and storing model instances; Social authentication. (3) jQuery -JavaScript library that simplifies verbose DOM interactions (document object model for HTML) and AJAX requests (client-side asynchronous http request technique). (4) CDN DataTables -jQuery plugin for creating reactive, paginated, filterable and interactive HTML tables (See Figure 4 -showing DataTables UI element). (5) Bootstrap -Industry standard front-end styling toolkit.

Implementation
Although a basic knowledge of the standard web development languages (HTML, JavaScript, CSS) and Python were required to build the EPANET Model Repository, many of the technologies used are industry standard tools that make implementation much easier and faster. Enabling HydroShare social authentication was the first step in implementation. Tethys Platform includes a HydroShare REST API Client helper function to facilitate social authentication. Figure 6 illustrates the integration with HydroShare. We first pass the AJAX request object into the Tethys HydroShare authentication helper function and use the returned 'hs' object to loop through all model instance resources that have 'epanet' as their subject. We add each of these resources with pertinent metadata, such as title, owner and availability information, to an array of models and return it to the front-end JavaScript code to generate the catalogue table. Once the model list is returned, we generate a standard HTML table with the pertinent metadata as columns. Creating an interactive table requires registering our table element as a CDN DataTable through its element id.
The table filters were implemented as checkboxes that augment which models in the model list were available to be displayed through the CDN DataTables plugin. The Owner and Subject filters are created 'on the fly' based on the set of owners and subjects of all models in the model list. As there are a set number of availability types (shareable, public, private, discoverable and not shareable) in HydroShare, this filter was hard-coded. The filters' click events only add models to be displayed from the list that meet their criteria.
We also added some click events to the rows in the table; the main event being opening the model in the EPANET Model Viewer. This is done by redirecting the client to the URL of the EPANET Model Viewer with the HydroShare resource ID appended to the URL as a query parameter. Opening models in other web applications that are compatible with EPANET Models could be accomplished the same way as long as the apps are able to implement HydroShare in the backend.

Model repository results
Our primary goal with the EPANET Model Repository was to create a powerful but simple standalone catalogue specifically for EPANET model instances. We intended for the structure to be simple enough that with minimal changes the application could be converted to fulfil the same function for any other modelling community. We believe the EPANET Model Repository meets these goals. With a little over 700 lines of code, most of which being reusable, any data scientist or engineer with minimal web development experience could create a web-based model instance catalogue for the peers in their community to start to share and learn together in one convenient place. Using HydroShare as a backend datastore with its REST Client for storing model instances makes tying into other applications, that can use the model instances, extremely easy. The technology used in the EPANET Model Repository synergizes very well and enables the cohesive web-ecosystem that we sought to create.
In the future, we would like to automate the process of converting the EPANET Model Repository into a repository for a different modelling community. With some careful planning we believe this would not be very difficult.

Requirements
Below, are the key features we identified in the end as most important to realizing our research goal of creating a full web ecosystem for EPANET once the model repository app was complete: (1) HydroShare integration as backend data store (2) Authentication through HydroShare (3) A large graphical window for viewing the visual components of an EPANET model (4) Network editing tools including node creation, edge creation and a search tool (5) Additional model option editing tools including secondary graphical editors, such as demand curve, flow curve and pattern editors (6) Ability to run models and view simulation results including time series animation of results

Technology
Several of the same technologies used in the EPANET Model Repository were also implemented in the EPANET Model Viewer, however, being a more specialized application, it required the addition of more specialized tools. In addition to Tethys Platform, HydroShare, jQuery, and Bootstrap (as described in Section 3.2) key technologies used in the EPANET Model Repository include: (1) EPANETTOOLS -Python wrapper for running EPANET models; Installed with pip; Requires a C compiler.
(2) SigmaJS -JavaScript graphics library for drawing networks of connected nodes. (3) Plotly -High-level JavaScript graphing library; Used to create scientific plots and charts; Includes many tools for data viewing and manipulation.
At the outset of development for this application, we chose PaperJS, as the library for drawing the visual network component of EPANET models. Our reason for choosing this library over others was its ability to draw our water distribution networks, no matter how complicated. After a good deal of proofof-concept development we found that the implementation of PaperJS to meet our needs would require much more time and customization than we felt appropriate for this project. We then went through the same prototyping process as before with SigmaJS. It proved to be much easier to implement and included key features that we needed out of the box. Some of these features were the ability to link nodes and edges together seamlessly, on click event handlers for nodes and edges, and the ability to customize properties of the network on the fly (such as colour of nodes or position of the camera).

Implementation
Before any of the technologies listed about could be implemented, we first needed a way to parse the EPANET model files, which are structured text files, into a format that that could be used in the code.
The EPANETTOOLS documentation does not provide any specification that can be reliably used for this task. As a result, it became necessary to build a custom parsing script. The script closely resembles a tokenizer (Figure 7) that creates a JS object with keys at the root level representing each main section in a .inp file (EPANET models' preferred extension). Although this presented more work than if the Python wrapper had had a function that met our needs, because we wrote a custom script, we could parse the files into a structure that we could pass directly into the initialization of a SigmaJS network graph. This made implementing SigmaJS significantly easier.
Given an EPANET .inp file parser, it then became necessary to implement a .inp file writer. After a model is edited and updated, the user would need to be able to execute the updated version through EPANETTOOLS model simulator. This requires an updated .inp file, thus the need for a .inp file writer. Unlike the parsing script's tokenizer optimization, we could not identify an elegant way to use a JS object representation of an EPANET model and write it back into the structured text file that the EPANET model program expects. The writer follows a brute force approach and consists mostly of structured string concatenations with the right spacing and section separators (colons, semi-colons, etc.). The reader and writer scripts are another aspect of the EPANET Model Repository that is completely specific to EPANET models and therefore would provide little to no reproducible code for other communities looking to create similar web ecosystems. The next step in implementation was creating a viewing window for the pipe network's visual representation ( Figure 8). As mentioned above, we created our model parsing script such that we could pass the objects created for nodes and links (nodes and edges in SigmaJS terminology) directly into the SigmaJS graph constructor. The 'model' object, that we pass into the graph setting of the Sigma object, consists of an array of nodes and an array of pipes. Each node in the array is comprised of EPANET specific settings for its type and an X, Y coordinate. SigmaJS uses the coordinates to plot the points on the canvas (passed in as the container argument below). Each pipe in the array is comprised of EPANET specific settings for its type as well as a target and source node. SigmaJS graphically connects the pipe or edge between the target and source nodes. The last step in implementing SigmaJS was setting up the graph event listeners, including: • 'clickStage' events include events like panning the view and adding new nodes. • 'clickNodes' events include dragging node positions or opening the node settings/results modal • 'clickEdges' events include opening the pipe settings/results modal or selecting the pipe to remove it At this point in development we determined to allow the user to edit any of the model settings in an easy intuitive way. This included recreating the secondary graphical editor windows such as the Pattern Editor and Curve Editor windows. The graphical editor windows are comprised of standard HTML inputs and some manipulation of a PlotlyJS graph. For example, the Pattern Editor (Figure 12), was created with a line plot that take the multipliers given and artificially creates two points on the line, one for the beginning of the timestep and one for the end. This gives the continuous line plot that resembles what is happening over time for each pattern.
After the applications features included editing any setting in the EPANET input file we added features to analyse results of a model simulation including: • A query tool that highlights nodes or pipes that meet the criteria identified in the tool's inputs (Figure 8). • A results page for individual nodes and pipes that opens when a user clicks on one in the viewing window ( Figure 9). • Time series results animation over entire pipe network ( Figure 9). • Combined results plot for nodes and pipes in network ( Figure 11; Can display all nodes or edges at the same time; Legacy GUI only allows for 5 nodes or pipes at one time).
The time series animation was accomplished by first creating a continuous mapping of a colour scale to the range of values of a given result parameter (head, demand, pressure, flow, etc.). At each time step in the animation the colour property of each SigmaJS node and edge was set to its new value. A standard 'setTimout' JS function, changing the timeout based on the indicated animation speed, creates the animation effect. The results graphs were all made with PlotlyJS. The creation of these plots is very simple. A user passes in the datapoints and define certain styling and the library does the rest of the work including zooming, panning and downloading the plot as a PNG image.

Model viewer results
Our goal for the EPANET Model Repository was to create a lightweight web-based GUI that closely mimicked the legacy desktop GUI's functionality while improving on some of its UI/UX. This project proved to us how powerful existing opensource cyberinfrastructure tools are. We were able to create a powerful modelling tool that only requires an internet connection to utilize. We believe the EPANET Model Repository is aesthetically appealing while also providing valuable data analysis tools for EPANET models. The figures compare the EPANET legacy GUI for Windows and the EPANET Model Viewer created for this project. Significant differences are explained for each comparison. The same model will be used as a use case for both applications.

Main viewing window
The main view of each application shows the graphical representation of the pipe network ( Figure  10). Each view below is showing the same timestep animation of simulation results. As shown, the tools on the updated application are larger and more accessible with more vibrant colours to discern values. It is notable that the Legacy GUI's animation colour scale mapping isn't a linear mapping. For example, the head values for this time step range from 125 to 307, not 0 to 100. This causes a slightly misleading initial analysis of the graph. The colour  scale in the web based system is superior for this reason.

Simulation node results graph
The two plots are very similar timeseries representations of the data (Figure 11). The EPANET Model Viewer also provides a stats summary page that provides even more useful context to the analysis of a simulation. The generation of one of these plots from a user's standpoint is more clunky with the legacy GUI as you have to manually find and add elements to the plot, whereas with the app we created, all of the elements that can be added to the graph are listed in one spot and are added to or removed from the plot at the click of a button. The largest difference between the two version however, is the Legacy app only allows for 5 elements to be plotted on the graph. Being able to plot as many of the elements in the pipe network as desired, at one time, provides extra insight into the bigger picture of what is happening in the simulated network.

Pattern editor
Both pattern editors ( Figure 12) are very similar in the way one adds and views the pattern. This is one of the areas where the legacy GUI may be more functional, although more complex. The legacy GUI allows one to load or save existing patterns in the form of a CSV. This is something we may consider to add in the future.

Curve editors
Similarly, the curve editors ( Figure 13) are very similar in form and function, however the Legacy GUI once again allows the user to load and save a curve in the form of a CSV file, functionality not supported by the Model Viewer curve editor.

Control editor
Controls are a more advanced model setting within an EPANET input file. One can control whether pipes are open or closed based on surrounding conditions of the pipe network or based on a time. The same  method of control exists for pump activation and reservoir outflows. In the legacy GUI adding a control is completely up to the user's knowledge of the control syntax. This can lead to input file errors and added learning curves for using EPANET. The Control Editor we created knows the proper syntax and helps the user create controls by offering the options for each control type and parameter in the form of dropdown inputs. Having a more intuitive and simpler UI will hopefully allow more people to take advantage of this powerful setting for EPANET model simulation with the use of our app (Figure 14).

Discussion and conclusions
This experimental system demonstrated using existing open source building blocks to easily create powerful open web modelling applications using a number of web programming libraries and integration with HydroShare and Tethys Platform. Compared to the legacy GUI, our EPANET Model Repository and Model Viewer applications only require a modern browser and an internet connection to run. They are also both primarily comprised of Python, and JavaScript code, some of the most universal languages in use today, thus lowering the barriers for creating plugins and extensions.
Using the tools described above HydroShare and Tethys Platform, creating the EPANET model repository required writing less than 700 lines of code, which for a comprehensive web application is very good. A lot of the code the repository is comprised of is reusable, meaning, to adopt this structure to another modelling community would require far less lines of code, increasing the speed of development and lowering the learning curve even more.
The EPANET model viewer required significantly more lines of code, most of which being non-reusable due to the individual nature of a modelling system. However,  replicating nearly 100% of the legacy EPANET desktop GUI's functionality into a light-weight web application, that anyone with a network connection can use, is a significant and powerful addition to the EPANET community. Incorporating model instances from HydroShare into the viewer app is most likely the only portion of the code that is directly usable by other modelling communities' systems. Some of the graphics and drawing libraries with their respective implementations would most likely also be of use to other modelling systems that include a visual aspect to their editing and analysing.
We believe that the future of modelling and data science lies in the cloud. Giving anyone, anywhere, with an internet connection, access to powerful modelling tools and a vibrant community of modelling experts will provide more of the world with the benefits of these sciences. It will also increase the rate at which these sciences evolve and progress. The web ecosystem of modelling tools that we have created for this project are a step towards the future of modelling in the cloud. The workflow we have developed and presented here is quite simple and can be easily adopted by other modelling communities. After minimal changes, the EPANET Model Repository could be a catalogue for any type of model instance and can be tied into other web-based applications that are compatible with those models. Although the EPANET Model Viewer is a more specialized application, it too has a simple structure that can be adapted to other model types using the repository as its main datastore.
With the advent of the popularization of open source technologies, the barriers to creating web modelling applications using legacy models have been lowered dramatically. This project demonstrates that, with some time spent on creating a solid design plan and finding the right open source technologies, bringing existing modelling tools to the new age of cloud computing is not the daunting task it once was. Much like the impact that these cloud ecosystems will have on their individual scientific communities, as more modelling tools become available on the cloud other communities will take notice and follow suite. We hope that the structure we have presented in the project will play, at least a small roll, in the future of cloud-based open modelling.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Funding
This work was supported by the US Environmental Protection Agency [R835950].