Inteligeo is a system that stores a lot of information used by the Brazilian Federal Police Forensics to fight crime, initially in the environmental arena with a later expansion to other types of crime. During the construction of the database a lot of problems appeared for which PostGIS was the key to the solution.
This presentation describes problems encountered by the team while loading 850+ shapefiles into the database, linking with external databases and building 950+ views of the data.
Although the content of the recipes is very technical, the general concepts will be explained in an accessible language and correlated to real world cases.
*Definition of crazy data in our context
- Spike removal
- Invalid geometry detection and fixing
- Filling holes
- Raster image footprints
- Hammering data into correct topologies
- Speeding data visualization with ST_Simplify and PGSQL 9.3's materialized views
- Rough georeferencing using an auxiliary table
- Creating constraints
*How is crazy data generated and our experience in handling each case
- Large datasets
- Lack of validation
- Geometric operations
- Topological errors
- Imprecise definitions
- Legacy databases
- Bad georeferencing
We will also discuss why is handling crazy data important for the Brazilian Federal Police, our efforts in cleaning up data at the source and the implications of geographical data in general for fighting crime.