The ARP team is in the process of moving their data from SURF, a collaborative organization for IT in Dutch education and research, to a cloud service.
Background
The Africa Rainfall Project (ARP) aims to simulate rainstorms in sub-Saharan Africa to improve regional weather forecasts. Providing accurate weather predictions is crucial for self-sufficiency of the local farming community. By comparing results obtained through computing rainfall data from various sources using the World Community Grid, scientists can create increasingly more accurate forecasts, improve future simulations and in turn weather predictions. To learn more about the specifics of this project, read our March 2022 research update.
Due to capacity limitations of their storage system, the release of new work units has been paused since December 2022. To read more about the details of this storage limitation, please refer to ARP update from March 2023.
New home for Africa Rainfall Project data
We asked the ARP team for an update on their storage limitation, and this is what they shared with us:
“The ARP project running at the WCG produces a very large dataset. At this moment, there is about 300TB in more than four million files. Storing and moving such enormous amounts of data is not trivial. Until now, all data was stored in the computing centre of SURF, the cooperative association of Dutch educational and research institutions. After about four years of support, the ARP at SURF is about to end. So what to do with the data?
We were very fortunate that, earlier this year, a cloud service provider has agreed to host the data. Presently, the file formats are not very user-friendly but once the move is complete, the files will be transformed to a format that can be used by most researchers. First, this will be organized in individual files that the user has to download. Pretty soon, however, we hope to have a simple map-based interface that would enable users to outline an area of interest and select a period of time as well as the weather variable of interest, after which all chosen data can be downloaded in one file. Stay tuned!”
Thank you to the ARP team for their continued partnership with WCG and for their help in preparing this update. If you have any comments or questions, please leave them in this thread for us to answer.
Thank you for your support!
WCG team