O'Reilly logo

Geospatial Development By Example with Python by Pablo Carreira

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Importing massive amount of data

Now that our environment is ready, we can begin working with bigger datasets. Let's start by profiling the import process and then optimize it. We will start with our small geocaching dataset and after the code is optimized we will move to bigger sets.

  1. In your geodata_app.py file, edit the if __name__ == '__main__': block to call the profiler.
    if __name__ == '__main__':
        profile = cProfile.Profile()
        profile.enable()
        import_initial_data("../data/geocaching.gpx", 'geocaching')
        profile.disable()
        profile.print_stats(sort='cumulative')
  2. Run the code and see the results. Don't worry about duplicated entries in the database now, we will clean it later. (I removed some information from the following output for space reasons.) ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required