Storing and Clustering Large Spatial Datasets Using Big Data Technologies

Abstract

In this paper we present the architecture of a system to store, query and visualize on the web large datasets of geographic information. The architecture includes a component to simulate a large number of drivers that report their position on a regular basis, an ingestion component that is generic and can acommodate three different storage technologies, a query component that aggregates the results in order to reduce the query time and the data transfered, and a web-based map viewer. In addition, we define an evaluation methodology to be used to benchmark and compare different alternatives for some components of the system, and we validate the architecture with experiments using a dataset of 40 million locations of drivers.