wetransform, lat/lon and geosparc receive INSPIRE Conference Excellence Award

It came as a very nice surprise to us when Robin Smith (JRC) announced to the teams of wetransform, lat/lon and GeoSparc that we would collectively receive this year’s “Award for recognition of excellence in geo-information technologies“. This award was given in memory of Doug Nebert and Christiner Giger. Doug Nebert, an active member of the OGC Technical Committee and Planning Committee since 1994, received OGC’s highest award in 2005. Christine had been a very active member of the INSPIRE community and Scientific Manager of the HUMBOLDT project, and was thus involved with the HALE team early on.

Alessandro Annoni, Head of the Digital Earth and Reference Data Unit (JRC) explained why the three teams were awarded: “These three teams have been technological enablers of spatial data infrastructures, have ensured the sustainability of research results and have now entered a partnership with great potential“.

Continue reading

All about Harmonisation: Geospatial World Forum and INSPIRE Conference

This year we have a joint Geospatial World Forum and INSPIRE conference. More than 2.000 attendees from all over the world are expected. There are multiple opportunities to learn about our complete INSPIRE solution and interesting projects in Lissabon, Portugal from 25th of May to 29th of May:

Arrange a meeting with our team by sending us an e-mail!

wetransform.to unveiled today!

The data harmonisation panel team at the INSPIRE conference will unveil wetransform.to, the world’s first collaborative Transformation Platform as a Service. The platform helps professionals solve data integration, harmonisation and transformation tasks.

The start-up behind the platform, wetransform gmbh, was recently founded in Darmstadt by Simon Templer to further commercialize the work on Data Harmonisation that was started in 2006 when the HUMBOLDT project began. The company also offers professional support subscriptions for HALE and CST.

Come by our booth or visit the official first presentation at 14:00.

HALE wins SmeSpire challenge

HALE wins the SmeSpire Challenge 2014!

We’re proud to announce that our submission to the SmeSpire challenge won in the “Open Source Software for INSPIRE” category. The SmeSpire team chose HALE “after a long evaluation made by key experts in the fields of INSPIRE Directive and GIS sector” and also chose “Linked Open Data and Spatial Data Infrastructures” by Massimo Zotti (Planetek) and Piergiorgio Cipriano (Sinergis) as Best Practice winners.

We will thus have two additional opportunities to present our work to the participants of the 2014 INSPIRE conference in Aalborg. Accordingly, you have a number of opportunities to meet the HALE team at the INSPIRE Conference:

We look forward to meeting you in Aalborg!

Report: GDI-DE INSPIRE Information Workshop

The Geo Data Infrastructure Germany Coordination Team (GDI-DE) set up a very informative and well-attended one-day workshop on Tuesday, 10th of December. Simon Templer participated and presented a methodology overview and gave a few examples for schema harmonisation using HALE to the approximately 100 participants, who represented German municipalities, state SDIs and other interested parties.

In the morning session, the focus point of the presentations was on motivating the implementation of INSPIRE, but also on showing where organizational pitfalls and other issues are currently visible. Ingo Wanders (“The Usage of Geoinformation in Muncipailities”) and Anja Loddenkemper’s (“Solving Problems we wouldn’t have without INSPIRE?”) presentations made it apparent that especially smaller organizations such as municipalities need better support. Andreas Fritzsche explained what opportunities lie in the INSPIRE implementation for the chamber of commerce by providing an harmonised access interface.

The afternoon session was started by Daniela Hogrebe and Martin Seiler with a state of play presentation on the implementation of INSPIRE within GDI-DE. Followed by this, attention turned to one of the big open issues: How to transform data to achieve harmonisation? Astrid Feichtner opened this topic with her “Data Interoperability for INSPIRE in practice” talk. Simon continued with his presentation titled Semantic Harmonisation – An Experience Report. Interestingly enough, the (Geo)JSON support that HALE now has was a topic that got some response.

Simon Templer's Presentation highlighted challenges and solutions for semantic harmonisation issues such as reclassification.

Simon Templer’s Presentation highlighted challenges and solutions for semantic harmonisation issues such as reclassification.

The afternoon session was concluded with two presentations that dealt with one of the main promises of INSPIRE: Provide a common & reuseable platform through which all kinds of reporting to country, state and EU could be handled in a consistent fashion. The first talk explained the WasserBLICK portal, which is the nation-wide portal for reporting in Germany, and the second showcased the Marine Spatial Data Infrastructure in Germany. Interesting note: According to Dr. Ralf Busskamp, the administrations who receive the reports are lagging behind in implementing INSPIRE compared to those who are reporting.

In the concluding discussion, there was quite some encouragement to have a workshop specifically on schema transformation, to be organized by GDI-DE.

Thanks to Simon for his account of the workshop, from which I wrote this article!

INSPIRE KEN Schema Transformation Workshop in Marne-la-Vallee – Day 2 summary

On 8th and 9th of October, around 50 people gathered for the joint EuroSDR/INSPIRE Knowledge Exchange Network (KEN) Workshop on Schema Transformation. For the first part of my report from this very interesting workshop, continue here.

The second day focused on presentations of commercial, closed-source software and was opened by Ken Bragg of Safe Software, with an in-depth presentation on how to use FME to create INSPIRE-compliant geodata. He explained improvements that were added in the latest versions (2013/2014) that make it simpler to create valid, complex GML structures, dwelling on transformers such as the SchemaMapper Transformer, which can be configured by anyone with basic Excel skills.

Ken Bragg of Safe Software explains FME

Ken Bragg of Safe Software explains FME’s SchemaMapper Transformer

After the presentation of ArcGIS for INSPIRE by Paul Hardy, emphasizing the importance of having an actual GIS in which to fully edit, create and use INSPIRE data, the INTERGRAPH GeoMedia Fusion presentation followed a comparable angle. Next up was one of the presentations I had looked forward to, coming to this workshop: A presentation of how to use Altova MapForce for the transformation of complex XML data. The presentation was given by an end user – Helen Erikson from Lantmateriät. She walked the audience through the process of how to work with the software and highlighted pros such as the large number of built-in functions, the good usability and the fact that a valid GML file could be created easily. One thing she mentioned though is the mapping can become complex very quickly, and consequently hard to document, understand and maintain. This screenshot from her presentation illustrates the challenge and shows the limits of the “connect left to ride side approach” and it’s variants:

Using Altova MapForce to create INSPIRE Data

Using Altova MapForce to create INSPIRE Data

The final presentation in the morning block was given by Robert Chell from 1spatial. They have really come quite far with the radius studio/server products since I last evaluated them in 2010. Their solution helps with the full process, from source data discovery to source data assessment to data reconciliation (that’s where the schema transformation and quality assurance take place) and publishing.

A short afternoon presentation block followed, with Robin Smith of JRC explaining the purpose of the ARE3NA project (A Reusable INSPIRE Reference Platform), and with Sandrine Belley of IGN France exploring the User Experience side of schema transformation. She essentially described Mismatches and other sources of semantic heterogeneity that have been the core of my research (see this paper and this paper) that led to the development of HALE since 2008.

As on the first day, a discussion round completed the programme. This time the main question was “Missing items in Schema Transformation” with the findings again based on Dominique Laurent’s summary (with my favourites being highlighted):

  1. User Experience and Usability are not sufficiently in focus of the application development in this area; one vendor even explicitly stated that “the number of users is too small to spend significant effort to improve the user experience”. This might be true for INSPIRE-specific solutions, but many of the presented tools have value well beyond INSPIRE.
  2. Tools widely lack semantic-level functionality, so that the impact of a schema transformation is hard to assess.
  3. There is need for tools to be able to consume INSPIRE data, especially in desktop and Web GISes.
  4. Not all constraints in the INSPIRE data specifications (in the PDFs and UML Model form) are encoded in the .XSD file and so, can’t be tested during the transformation phase, though it would be useful to check them as soon as possible. Specifically we have to check the data integrity and consistency across multiple themes.
  5. The language to be selected to express and check constraints is still open to discussion; it is unsure that OCL is best option
  6. There is a need of a standardized mapping from the UML models to a relational database that includes measures for acceptable operational performance. This would allow tools to transform this relational data base to INSPIRE GML data, instead of having one-off solutions.
  7. Establish a standard for mapping language: it would enable us to provide the mapping rules to users and not only the transformed data.
  8. We should investigate other formats as optional delivery formats, such as JSON/GeoJSON, because of their importance on the user’s side.
    1. The discussion then shifted to more general issues with the adoption of INSPIRE, such as initiatives to populate missing data or to encourage the use of INSPIRE data. This ended the workshop, which was a really interesting event. Thanks to Dominique and her colleagues for organising it!

INSPIRE KEN Schema Transformation Workshop in Marne-la-Vallee – Day 1 report

On 8th and 9th of October, around 50 people gathered for the joint EuroSDR/INSPIRE Knowledge Exchange Network (KEN) Workshop on Schema Transformation. The workshop gave all participants the opportunity to get an overview of pretty much all approaches that are in market to help complete schema transformation projects.

For the full program, all slides and video recordings of the workshop, please go the the Eurogeographics website. What follows is not a detailed report of every presentation, but rather an account of my personal highlights, including the two discussion sessions that completed each day. Marie-Lise Vautier (IGN France) and myself started with presentations setting the frame by providing definitions of what schema transformation is and what general approaches are available. Morten then continued with experiences highlighting the schema matching methods originally developed by ESDIN and now widely used by Cadastral Agencies and other LMOs. He also mentioned that especially matching tables can get hard to create and maintain.

Marie-Lise Vautier gives the first presentation for the INSPIRE KEN Schema Transformation Workshop

Marie-Lise Vautier gives the first presentation for the INSPIRE KEN Schema Transformation Workshop

After a break, Just van den Broecke opened the block on Open Source Schema Transformation software. He has developed a streaming ETL framework called STETL which is based on GDAL/OGR, XSLT and other libraries and ties everything together using Python. Python support throughout the world of geospatial tools is very good – you can see it becoming a lingua franca in GIS for scripting. Just, like me in my earlier presentation, made it clear that schema transformation projects are essentially programming projects and thus have a certain complexity level. I fully agree, but see it as a disadvantage that for using STETL you have to learn multiple languages. Consequently I see STETL mostly as a tool for programmers who want to use the tools it is based on anyhow and need a rich “boilerplate”. Just also held up the flag for Open Source as a major community enabler, which I see as especially important for INSPIRE.

I was also very interested in the presentations on GeoKettle and Talend Spatial Data Integrator, which on a first glance seem to have close capabilities. Both presentations were given by users who had completed transformation projects using them. About both, I like that they are derived from general-purpose, non-GIS, which proves tool reusability. Talend was showcased by Jean-Loup Delaveau of CERTU. He explained how to create INSPIRE Planned Land Use Data by setting up a workflow in Talend that used components such as XSLT translators. An interesting note from his side was that GML should really be used as a machine-to-machine exchange format, and that providers and users should not see much of it.

Jean-Loup Delaveau of CERTU explains the plan4all workflow to create PLU data

Jean-Loup Delaveau of CERTU explains the plan4all workflow to create PLU data

Edith Vautard of IGN France explained how her group evaluated GeoKettle for INSPIRE Administrative Units generation. One thing that really impresses me is that IGN France is very open and trying out many approaches and tools to collect rich internal knowledge. On GeoKettle, I made note that I’d like to investigate their workspace format a bit. Edith ended with an overall positive assessment of Geokettle, citing from her slides:

  • + It’s intuitive and easy to use
  • + powerful and performant
  • + provides a sufficient diversity of functions
  • + reads the schema from the data
  • – Transformations are only stored in the internal XML format and cannot be exported as executable files (e.g. XSLT)
  • – INSPIRE complex structures are not supported, nor can you create non-simple GML 3.2.1
  • – There is no help in the software, and documentation is light; however, there is good support.

The first day was then completed by an update on the model driven WFS work done by TU Munich, presented by Tatjana Kutzner. She highlighted findings of her recent research, which has been published under the title “Critical Remarks on the Use of Conceptual Schemas in Geospatial Data Modelling — A Schema Translation Perspective” (Kutzer, Donaubauer 2012). The core question they researched was what a core model of all UML profiles being used would look like and how to provide encoding rules for conceptual models in machine-readable formats.

After Tatjana’s presentation, only the discussion round stood between us and dinner – and everybody stayed for an interesting, engaged discussion, with these core findings on the subject “what are the main drivers to choose methods and tools for schema transformation” (citing from Dominique Laurent’s summary):

  • Maintenance and documentation of tools are significant criteria
  • Choice of tools depends on the business models of data providers: some want the best tool for each step (even if using many tools increases complexity), some want only a single supplier (or at least a small number of tool suppliers) and tender accordingly
  • Choice of tools depend also on national policy; there may be order to use open-source tools
  • Skills will also influence the methods and tools: if limited skills, would be better to choose a tool simple to use and/or to envisage training
  • Choice of tools and methods will depend on the existing systems already in place (tools, data, …) and on the organization (e.g. one or several data producers)

Another item of discussion started from my earlier presentation on schema transformation approach classification: “To be able to choose our tools and methods, we need [a framework] to analyse the potential ones, to get an overview”. Meanwhile, I have posted a more extensive description of the framework presented in Paris here. The day then really ended with a very nice dinner :).

INTERGEO 2013 – Fraunhofer IGD represents the DHP

INTERGEO is a central event for the geospatial community in Europe. This year, it takes place in Essen, from Tuesday 8th of October to Thursday 10th of October. The data harmonisation panel is represented by Simon Templer, Eva Klien and Joachim Rix from the Fraunhofer Institute for Computer Graphics IGD. You can find them at the AED SICAD in hall 1 / booth B1.030. You can see the newest release of HALE and CST there in action, as well as see what other things we have up our sleeves. Furthermore, Simon will be presenting HALE to the fair visitors, in German:

Simon Templer: Machen Sie Ihre Daten bereit für INSPIRE mit HALE
Wednesday, 09.10.2013 and Thursday 10.10.2013, at 12:40 – 13:00.
at the OpenSource Park in hall 1 / booth H1.033.

Enjoy your visit to INTERGEO!

INSPIRE KEN & EuroSDR Workshop in Paris

If you’d like to get an excellent overview of available software and approaches for transforming data to INSPIRE formats, a good opportunity is coming up: THE INSPIRE KEN (Knowledge Exchange Network) and EuroSDR are organizing a workshop about schema transformation tools and methods in the premises of the ENSG – Marne-La-Vallée (near Paris) – France.

Citing from the workshop’s information and registration page, “NMCAs, as other data producers, will have to make their data compliant with INSPIRE interoperability Implementing Rules; during [the] next years, this compliance will mainly be achieved through schema and data transformation. The objectives of the workshop is to make a state-of-play about (existing or projected) schema transformation tools, to help NMCAs to assess these tools and to help them to choose the most appropriate and possibly, to provide background to disseminate knowledge about schema transformation at national level.

The workshop is scheduled from Tuesday 8th October 2013 09:00 – Wednesday 9th October 2013 16:00. This is the draft lineup of presentations of the workshop:

Tuesday Morning

Time Topic
09:00 – 09:10 Welcome and introduction
09:10 – 09:35 A Study about schema transformation services
09:35 – 10:00 Approaches & Languages for Schema Transformation: Findings of HUMBOLDT & follow-up Activities
10:00 – 10:25 From production data base to INSPIRE data: potential methods
10:25 – 10:45 Pause
10:45 – 11:30 The ESDIN experience : use of DBMS and WFS, The GeoServer APP schema, Catalogue and mapping generators by Politecnico di Milano
11:30 – 12:10 Deegree and its specific developments for INSPIRE

Tuesday 08th October afternoon : non-commercial tools

Time Topic
13:20 – 13:50 XSLT and its use by Kadaster for ESDIN
13:50 – 14:20 Stetl for INSPIRE transformation
14:20 – 14:50 Talend for INSPIRE Theme Land Use
14:50 – 15:20 Comparison Talend GeoKettle
15:20 – 15:40 Pause
15:40 – 16:10 Humboldt Alignment Editor (HALE) and Conceptual Schema Transformer (CST)
16:10 – 16:40 GeoConverter
16:40 – 17:10 ExoMS for INSPIRE themes Species Distribution – Habitat and Biotopes
17:10 – 17:40 model driven Web Feature Service (mdWFS)
17:40 – 18:30 Discussion : main drivers to choose transformation tool(s) and method(s)

Wednesday 9 October morning : Commercial tools

Time Topic
09:00 – 9:40 Feature Manipulation Engine (FME) and its use for ESDIN
09:40 – 10:20 ArcGIS for INSPIRE – Example of use
10:20 – 10:50 Snowflake GO Publisher
10:50 – 11:10 Pause
11:10 – 11:50 INTERGRAPH GeoMedia Fusion and its use by Gugik
11:50 – 12:20 Use of Altova MapForce by Lantmateriät
12:20 – 12:50 Schema transformation by 1Spatial

Wednesday 9 October afternoon : research and discussions

Time Topic
14:00 – 14:30 Tools to restructure geographic data on the Web
14:30 – 15:00 The ARE3NA project
15:00 – 16:00 Discussion
dhp-blogpost-standardimage

Summer Break is Over: Time to Meet in Nottingham

At FOSS4G 2013 in Nottingham, the HALE community is represented by Silvia Franceschi of Hydrologis. FOSS4G, which starts tomorrow (Tuesday 17th of September) with the preconference workshops, is the worlds largest event on everything GIS && OpenSource.

Sylvia will give the presentation titled “Create and Use INSPIRE Harmonised Geodata with HALE” on 13:00 on Thursday 19th September (in Session 3, starting at noon, EMCC: Room 3). Expect to hear about…

  1. A general introduction to hale
  2. A short update on HALE development
  3. A full example workflow showing how to easily create INSPIRE-compliant geodata
  4. How to contribute to HALE on GitHub and our community platform
  5. Our next releases

See you all in Nottingham!