Broken Processing models are nasty and this error is particularly unpleasant:
...
File "/home/agraser/.qgis2/python/plugins/processing/modeler/
ModelerAlgorithm.py", line 110, in algorithm
self._algInstance = ModelerUtils.getAlgorithm(self.consoleName).getCopy()
AttributeError: 'NoneType' object has no attribute 'getCopy'
It shows up if you are trying to open a model in the model editor that contains an algorithm which Processing cannot find.
For example, when I upgraded to Ubuntu 16.04, installing a fresh QGIS version did not automatically install SAGA. Therefore, any model with a dependency on SAGA was broken with the above error message. Installing SAGA and restarting QGIS solves the issue.
Since I’ve started working, transport and movement data have been at the core of many of my projects. The spatial nature of movement data makes it interesting for GIScience but typical GIS tools are not a particularly good match.
Dealing with the temporal dynamics of geographic processes is one of the grand challenges for Geographic Information Science. Geographic Information Systems (GIS) and related spatial analysis methods are quite adept at handling spatial dimensions of patterns and processes, but the temporal and coupled space-time attributes of phenomena are difficult to represent and examine with contemporary GIS. (Dr. Paul M. Torrens, Center for Urban Science + Progress, New York University)
It’s still a hot topic right now, as the variety of related publications and events illustrates. For example, just this month, there is an Animove two-week professional training course (18–30 September 2016, Max-Planck Institute for Ornithology, Lake Konstanz) as well as the GIScience 2016 Workshop on Analysis of Movement Data (27 September 2016, Montreal, Canada).
Space-time cubes and animations are classics when it comes to visualizing movement data in GIS. They can be used for some visual analysis but have their limitations, particularly when it comes to working with and trying to understand lots of data. Visualization and analysis of spatio-temporal data in GIS is further complicated by the fact that the temporal information is not standardized in most GIS data formats. (Some notable exceptions of formats that do support time by design are GPX and NetCDF but those aren’t really first-class citizens in current desktop GIS.)
Most commonly, movement data is modeled as points (x,y, and optionally z) with a timestamp, object or tracker id, and potential additional info, such as speed, status, heading, and so on. With this data model, even simple questions like “Find all tracks that start in area A and end in area B” can become a real pain in “vanilla” desktop GIS. Even if the points come with a sequence number, which makes it easy to identify the start point, getting the end point is tricky without some custom code or queries. That’s why I have been storing the points in databases in order to at least have the powers of SQL to deal with the data. Even so, most queries were still painfully complex and performance unsatisfactory.
ST_IsValidTrajectory — Returns true if the geometry is a valid trajectory.
ST_ClosestPointOfApproach — Returns the measure at which points interpolated along two lines are closest.
ST_DistanceCPA — Returns the distance between closest points of approach in two trajectories.
ST_CPAWithin — Returns true if the trajectories’ closest points of approach are within the specified distance.
Instead of points, these functions expect trajectories that are stored as LinestringM (or LinestringZM) where M is the time dimension. This approach makes many analyses considerably easier to handle. For example, clustering trajectory start and end locations and identifying the most common connections:
Overall, it’s an interesting and promising approach but there are still some open questions I’ll have to look into, such as: Is there an efficient way to store additional info for each location along the trajectory (e.g. instantaneous speed or other status)? How well do desktop GIS play with LinestringM data and what’s the overhead of dealing with it?
In the previous post, Mickael shared a great map design. The download includes a print composer template, that you can use to recreate the design in a few simple steps:
1. Create a new composition based on a template
Open the Composer manager and configure it to use a specific template. Then you can select the .qpt template file and press the Add button to create a new composition based on the template.
2. Update image item paths
If the template uses images, the paths to the images most likely need to be fixed since the .qpt file stores absolute file paths instead of relative ones.
With these steps, you’re now ready to use the design for your own maps. Happy QGISing!
Over the last two weeks, I had the pleasure to attend both the international FOSS4G conference in Bonn, Germany, as well as the regional FOSS4G-NOR in Oslo, Norway. Both events were superbly organized and provided tons of possibilities to share experiences and find new inspiration.
Talks at both conferences have been recorded and can be watched online: Bonn / Oslo
I enjoyed having the opportunity to give two very different talks. In Bonn, I presented work on pedestrian routing and navigation, which was developed within the PERRON project:
It was particularly nice that we had plenty of time for Q&A after this presentation since only two talks were scheduled for this session rather than the usual three. I’d also like to thank everyone for the great feedback – both in person and on Twitter!
In Oslo, I had the honor to give the opening keynote on OpenSource in general and the QGIS project in particular:
Both conferences were packed with great sessions and talks. If I had to pick favorites from last week’s presentations, I would have to opt for Iván Sánchez presenting his latest projects, including what3fucks and geohaiku:
All three projects: OPPTUR, GISTYLE, and the flexible traffic web viewer were great demos of what can be achieved with open source tools. Mathilde’s GISTYLE project is also available on Github.
An inspiring GISummer comes to an end, but with so many videos to watch and workshop materials to explore, I’m convinced that the autumn will be no less exciting.
It’s been a great week in Bonn! I joined the other members of the QGIS project at the pre-FOSS4G code sprint at the Basecamp, the weirdest location we’ve had for a developer meeting so far. We used this opportunity to have a face-to-face meeting of the QGIS PSC with special guests Matthias Kuhn (on QGIS 3.0 and bug tracker discussions) and Lene Fischer (on community team issues) – notes here.
QGIS PSC meeting in action (from left to right: Otto Dassau, Paolo Cavallini, Anita Graser, Andreas Neumann, Jürgen E. Fischer), picture by Tim Sutton
The code sprint was also a great opportunity to present the results of Akbar Gumbira’s Google Summer of Code project: the QGIS Resource Sharing plugin. This plugin makes it possible to easily share resources (such as SVG icons, symbol definitions, QGIS styles, and Processing scripts) with other QGIS users through an interface that closely resembles the well-known QGIS Plugin Manager. Akbar has also prepared great presentation with background info and screencasts showcasing his project.
QGIS Resource Sharing presentation, picture by @foss4g
Thanks to all the organizers who worked hard to make this one of the most well-organized and enjoyable code sprints I’ve ever been to. You are awesome!
Visualizations of mobility data such as taxi or bike sharing trips have become very popular. One of the best most recent examples is cf. city flows developed by Till Nagel and Christopher Pietsch at the FH Potsdam. cf. city flows visualizes the rides in bike sharing systems in New York, Berlin and London at different levels of detail, from overviews of the whole city to detailed comparisons of individual stations:
The visualizations were developed using Unfolding, a library to create interactive maps and geovisualizations in Processing (the other Processing … not the QGIS Processing toolbox) and Java. (I tinkered with the Python port of Processing in 2012, but this is certainly on a completely different level.)
The insights into the design process, which are granted in the methodology section section of the project website are particularly interesting. Various approaches for presenting traffic flows between the stations were tested. Building on initial simple maps, where stations were connected by straight lines, consecutive design decisions are described in detail:
The results are impressive. Particularly the animated trips convey the dynamics of urban mobility very well:
However, a weak point of this (and many similar projects) is the underlying data. This is also addressed directly by the project website:
Lacking actual GPS tracks, the trip trajectories are rendered as smooth paths of the calculated optimal bike routes
This means that the actual route between start and drop off location is not known. The authors therefore estimated the routes using HERE’s routing service. The visualization therefore only shows one of many possible routes. However, cyclists don’t necessarily choose the “best” route as determined by an algorithm – be it the most direct or otherwise preferred. The visualization does not account for this uncertainty in the route selection. Rather, it gives the impression that the cyclist actually traveled on a certain route. It would therefore be undue to use this visualization to derive information about the popularity of certain routes (for example, for urban planning). Moreover, the data only contains information about the fulfilled demand, since only trips that were really performed are recorded. Demand for trips which could not take place due to lack of bicycles or stations, is therefore missing.
As always: exercise some caution when interpreting statistics or visualizations and then sit back and enjoy the animations.
FOSS4G2016 is drawing closer quickly. To get in the mood for a week full of of geogeekery, Locate Press is offering a special FOSS4G discount for QGIS Map Design.
Use the code foss4gbonn to get 25% off your copy.
QGIS Map Design is the reference book to get if you want to bring your mapping skills up to speed. The book comes with a download for all our example map projects:
Wrong navigation instructions can be annoying and sometimes even dangerous, but they happen. No dataset is free of errors. That’s why it’s important to assess the quality of datasets. One specific use case I previously presented at FOSS4G 2013 is the quality assessment of turn restrictions in OSM, which influence vehicle routing results.
The main idea is to compare OSM to another data source. For this example, I used turn restriction data from the City of Toronto. Of the more than 70,000 features in this dataset, I extracted a sample of about 500 turn restrictions around Ryerson University, which I had the pleasure of visiting in 2014.
As you can see from the following screenshot, OSM and the city’s dataset agree on 420 of 504 restrictions (83%), while 36 cases (7%) are in clear disagreement. The remaining cases require further visual inspection.
The following two examples show one case where the turn restriction is modelled in both datasets (on the left) and one case where OSM does not agree with the city data (on the right).
In the first case, the turn restriction (short green arrow) tells us that cars are not allowed to turn right at this location. An OSM-based router (here I used OpenRouteService.org) therefore finds a route (blue dashed arrow) which avoids the forbidden turn. In the second case, the router does not avoid the forbidden turn. We have to conclude that one of the two datasets is wrong.
Interestingly, the disagreement in the second example has been fixed by a recent edit (only 14 hours ago). We can see this in the OSM way history, which reveals that the line direction has been switched, but this change hasn’t made it into the routing databases yet:
now
before
This leads to the funny situation that the oneway is correctly displayed on the map but seemingly ignored by the routers:
To evaluate the results of the automatic analysis, I wrote a QGIS script, which allows me to step through the results and visually compare turn restrictions and routing results. It provides a function called next() which updates a project variable called myvar. This project variable controls which features (i.e. turn restriction and associated route) are rendered. Finally, the script zooms to the route feature:
def next():
f = features.next()
id = f['TURN_ID']
print "Going to %s" % (id)
QgsExpressionContextUtils.setProjectVariable('myvar',id)
iface.mapCanvas().zoomToFeatureExtent(f.geometry().boundingBox())
if iface.mapCanvas().scale() < 500:
iface.mapCanvas().zoomScale(500)
layer = iface.activeLayer()
features = layer.getFeatures()
next()
You can see it in action here:
I’d love to see this as an interactive web map where users can have a look at all results, compare with other routing services – or ideally the real world – and finally fix OSM where necessary.
This work has been in the making for a while. I’d like to thank the team of OpenRouteService.org who’s routing service I used (and who recently added support for North America) as well as my colleagues at Ryerson University in Toronto, who pointed me towards Toronto’s open data.
Reducing the number of “Add layer” buttons in the QGIS GUI is a commonly voiced wish. Multiple approaches have been discussed but no decision has been made so far. One idea is to use the existing browser functionality to replace the “Add layer” dialogs. Others are envisioning completely novel approaches.
Since the topic came up again today on Twitter, I decided to implement a quick & dirty version of a unified Add layer button. This way, I can comfortably reduce my Layer toolbar to three buttons using Settings | Customization …
I pretty much just kept the “Create new layer” button and the “Add delimited text layer” button because, as far as I know, there is no way to call the dialog from the browser. (Instead, CSVs are opened with OGR, which doesn’t have nearly as many nice features.)
And here it is in action:
(I recommend to undock the Browser panel to get the dialog-like behavior that you see in the video.)
To install the plugin: download it and unzip it into your QGIS plugin folder, then activate it in the plugin manager.
I would love to hear what you think about this UX experiment.
As announced in Salzburg a few days ago, I’m happy to present the lastest enhancement to my IDF router for QGIS: travel time routing and catchment computation.
Travel times for pedestrians and cyclists are computed using constant average speeds, while car travel times depend on the speed values provided by the road network data.
Catchment computations return the links that can be traversed completely within the given time (or distance limit). The current implementation does not deal with links at the edge of the catchment area, which can only be traversed partially.
Loading the whole network (2.7GB unzipped IDF) currently requires around 10GB of memory. One of the next plans therefore is to add a way to only load features within a specified bounding box.
Plans to turn this into a full-blown plugin will most likely have to wait for QGIS 3, which will ship with Python 3 and other updated libraries.
Free and Open Source GIS Ramblings 