Development of Data Archiving and Distribution System For the Philippines' LiDAR Program Using Object Storage Systems — Ken Abryl Eleazar Salanio, Carlo Santos, Ruby Magturo, Gene Paul Quevedo, Kenan Virtucio, Kenneth Langga, Mark Edwin Tupas, Enrico Paringit
The Philippines' Department of Science and Technology in collaboration with Higher Education Institutions (HEIs), lead by the University of the Philippines, has embarked on a program for producing hazard maps on most major river systems in the Philippines. Realising the utility of LiDAR and its derived datasets, a concurrent program on resource assessment was also initiated. These endeavors aims to produce essential products such as DEMs, Orthophotos and LAS data that can be used for different purposes such as urban planning, resource planning, and other purposes these geospatial data might be able to provide.
The result of both programs are large amounts of data that needs to be distributed and archived at a fast rate. As with other LiDAR operations handling large swaths of spatial data is not an option, hence data sets are organized in contiguous blocks, subdivided by files and grouped by river systems and local government units. Existing spatial content management systems and geoportal solutions were designed and have capabilities for handling rasters and vectors but not for point-cloud data distribution.
This study discusses the development of a simple and straightforward system for storing and delivering LiDAR and LiDAR-derived data using Ceph as object storage system coupled with a spatial content management system derived from GeoNode. This approach hinges on our requirements of being scalable yet robust without much deviations from the current file system based storage structure.
While most operations like data acquisition, preprocessing and quality checking are done centrally, the system aims to address our programs' needs for data exchange between spatially distributed to autonomous partner HEIs who perform data processing, and validation. The system also targets to semi-automate our data distribution process which caters to government institutions and the general public.