GeoServer App-Schema Integration: HALE 2.9.4 is out

HALE has been supported well by the community in the past months with contributions, through support subscriptions and projects. We have thus been able to create an additional feature and maintenance release. So, why should you upgrade?

GeoServer App-Schema configuration

Publishing OGC services based on GML Application Schemas and Complex Features with GeoServer is now easier than ever. Use HALE to define the mapping from your data store to an application schema, such as INSPIRE or NAS. Then, upload the resulting configuration for the GeoServer App-Schema plugin directly to your GeoServer instance. This feature was developed together with GeoSolutions S.R.L, Italy. More information on the new feature including future plans is available at the GeoSolutions Blog.

GeoServer App-Schema Integration in HALE for Easy INSPIRE Services

Inline transformations

Declarative Mapping, as implemented in HALE, is about simplification and effective re-use of transformation projects. When you encounter complex structures in your data, the mapping can still get quite complicated. By using the Inline transformation function, you tell HALE that it should transform objects in a complex feature the same way as it would standalone features.

As an example, let’s look at a mapping between two CityGML ADEs. In both source and target schemas, there are a Building and a BuildingPart type. BuildingParts can be features on their own, thus a mapping is defined based on a Retype between the corresponding source and target types. Buildings can also contain BuildingParts, and to transform these internal BuildingParts we use the Inline transformation, with the additional step to include it in the respective target property.

Namespace agnostic loading of XML/GML

HALE relies on qualified names for the transformation and handling of data, which consist of local names and namespaces. In many cases, you still want to transform data where only the local name matches. This can be the case for instance when using data based on a previous version of a schema or for data that is published via an OGC Web Feature Service. We have added an option when loading XML or GML data that instances will be loaded even if the namespaces do not match the schemas exactly.

HL Consulting BVBA funded this work under a Professional Support Subscription for HALE. They use this feature to define their alignments on a schema that allows them to make extensive use of relation inheritance in their alignment, to have one mapping that they can use to transform data from different services.

Tweaks and Bug fixes

We’ve added a lot of smaller improvements and fixed some issues:

  • The command line interface has been extended with the capability to use whole folders as source for the transformation.
  • Deprecated INSPIRE SpatialDataSet export, as INSPIRE now recommends to use a WFS 2.0 FeatureCollection instead, and added support for generating a Dataset Feed file for an ATOM-based INSPIRE predefined Dataset Download Service when exporting to a WFS 2.0 FeatureCollection.
  • Added a Groovy Join transformation that allows for more control on the creation of transformed instances.
  • Introduced helper functions for Groovy scripts for dealing with geometries and performing type conversions.
  • Structural copies with Rename and Retype can now explicitly exclude geometry objects parsed by HALE, for cases where the original geometry encoding should be retained.
  • A default SRS can now be specified on the command line, for reading data from a specific source, in case HALE cannot correctly detect the SRS.
  • Handling foreign key constraints during the export to a database has been improved.

Check out the help for a list of all improvements and more information.

Get HALE 2.9.4

To get HALE 2.9.4, go to the downloads section and enjoy your work with it!

Please use the board to tell us what you think of this new release; don’t hesitate to tell us about problems you’re encountering and features you’re missing, wither there or on Github.

Spatialite and INSPIRE 4.0: HALE 2.9.3 released

In the past weeks, we’ve received feedback from multiple users, projects and workshops where HALE is used, including a substantial code contribution. We’ve collected the feedback and just released the final 2.9.3 build. What’s new?

Support for SQLite and SpatiaLite

You can now work with SQLite and SpatiaLite files using HALE. We even support 2D and 3D geometries. This was made possible by a contribution from GeoSolutions S.R.L, Italy. Special thanks to Stefano Costa and Sandro Salari!

Improved Styling for the Map view

Create individual styles for your feature types with just one click. Using specific styles provides a better overview in the map. To easily identify associated types, the type symbol reflects the style. As an additional option, you can quickly assign a random color to a selected type. For more control specify a custom style with the style editor.

HALE Style Quick Editing

Included INSPIRE version 4.0 schemas

Experienced HALE users know they can load any GML/XML Schema using the corresponding XSD file. However, it’s more convenient to just search and select a schema from the presets defined in HALE. INSPIRE version 4.0 schemas now included.

Performance improvements and bug fixes

A couple of users reported performance issues since release 2.9.0, and we’ve now received reproduction cases and were able to identify the culprits. Other issues fixed:

  • Fixed wrong detection of GML namespace that could happen if certain GML 3.3 schemas were imported in a schema (Thanks to Fabio Vinci for the bug report).
  • Fixed error on XML export that could lead to an invalid file (Thanks to Giuseppe Procino for the bug report).
  • Fixed parsing GML geometries defined through a Ring instead of a LinearRing.
  • Show 3D WKT (instead of 2D WKT) for 3D geometries in the data views.
  • Fixed error that could lead to a Merge relation no longer being editable after a schema upgrade.
  • Variable values for Groovy scripts are no longer converted and passed as a String by default.
  • Fixed deadlock that could occur when disabling the Groovy restriction when a project is loaded.
  • Fixed saving a project as a project archive if resources include a query string (e.g. WFS requests).

Get HALE 2.9.3

To get HALE 2.9.3, go to the downloads page and enjoy your work with it!

Please use the board to tell us what you think of this new release; don’t hesitate to tell us about problems you’re encountering and features you’re missing.

HALE 2.9.2: WFS-T Support and new Database Features

ON the road to HALE 3.0, we’ve release a maintenance release of HALE. In addition to making numerous fixes available to in a released version, 2.9.2 also sports two headline features:

Improved WFS Client + Transactional WFS Support

With this feature, you can publish GML to a transactional Web Feature Service, for instance for publishing INSPIRE data to a deegree server. We currently support are Transactional Web Feature Services following the OGC WFS 1.1.0 or WFS 2.0.0 specifications. There are two different providers for publishing to WFS:

  • Direct upload: Does a single request that starts as soon as the first transformed features are available. Use this option if you are sure the WFS can cope with the amount of data in a single transaction.
  • Partitioned upload: Partitions the transformed data before upload to be able to do multiple requests to the WFS-T for large data sets. The partitioning is done based on the references between features. Features that are connected are kept together so the WFS-T is able to retain those references.

Improved Database Reader/Writer

We’ve had a silently released database writer for PostGreSQL in HALE for a while, and due to popular request we’ve improved it a bit and provided official support and documentation.

PostGreSQL Configuration

Compatibility to FME and other fixes

We’ve also tested whether the FME Plugin for HALE still works with the most current FME versions (2014 SP3 and 2015) and can confirm it’s compatibility. You can thus combine both tools. For example, use HALE to do restructuring of complex data and encoding as GML, but use FME to read formats unsupported by HALE or to employ special transformers.

As usual, there has been a set of other improvements, for example to make the encoding of MultiSurfaces and complex aggregates more flexible.

Get HALE 2.9.2!

To get HALE 2.9.2, go to the downloads page and enjoy your work with it!

Watch the HALE Tutorial video to get started on working with HALE:
http://www.dhpanel.eu/humboldt-framework/hale-tutorial.html

Please tell us what you think of this new release; don’t hesitate to tell us about problems you’re encountering and features you’re missing.
http://www.esdi-community.eu/projects/hale/boards

HALE 2.9.0 – new look, new tools, INSPIRE codelists + Annex II/III

We have just completed testing and documentation for the new HALE 2.9.0 release. There is a lot that has changed under the hood for the 2.9 release. Apart from a lot of bug fixes and an upgrade of the underlying platform from Eclipse Indigo to Eclipse Luna, there also was a new build system introduced that allows us to again provide an installer for Windows and a version of HALE that runs on current Mac OS X operating systems.

It comes with a long list of useful new features, as you would expect after the relatively long break since the last release of HALE. Get your download here and read in detail about the new features here.

Continue reading

dhp-blogpost-standardimage

Final HALE plugin for FME 2014 now available

In the summer, we announced a FME GML/XML Writer plugin that would enable using HALE projects as part of a FME workspace. Using this plug-in can greatly simplify your FME workspace, by letting HALE deal with all the mapping from a simple model to a complex target schema. While the original plan for this plug in was to be delivered with FME 2014, there were some complications from the licensing side, so now the plug-in is made available as a separate download – free for commercial and personal usage. The plug-in is compatible with FME 2014 (including the current beta versions). Apart from FME 2014 you need a Java installation and at least HALE 2.6.0 (newest version recommended) on your machine.

The full description of the plug-in, downloads and documentation is available here. Don’t forget to tell us what you think of this new steps towards interoperability!

HALE 2.8.0 brings PostGreSQL access

Just about a month after the release of HALE 2.7.0, the next intermediate version is ready for you. Despite the short cycle, it brings a lot of improvements, since it includes a few features that were almost but not entirely ready for release with the prior version. So these are the updates you can expect with the 2.8.0 (downloads, documentation):

Database access

Out of the “box”, you can now connect to PostgreSQL/PostGIS databases as a schema and data source for a transformation with the new JDBC based database support in HALE. Each table in the database is loaded as a type; to combine information from different database tables in a mapping, use the Join function, which has also been improved. To support additional database types, HALE’s plug-in mechanisms can be used. For more information, see the Database Import documentation.

Accessing a PostGIS database in HALE

Sample Source Data

Source data is used in the live transformation to directly see the impact of changes to the mapping in the transformed data. Live transformation performance will become sluggish is the source dataset is very big, however. So, you can now configure that only a sub-set of the data is loaded into HALE, which in most cases also speeds up the import.

Groovy Transformation Scripts

While the transformations functions delivered with HALE cover a lot of ground, an occasional request by advanced users was to provide their own functions or customize existing ones. You can now can combine the regular HALE transformation functions with powerful Groovy scripts. HALE provides easy-to-use APIs for accessing and creating complex instances. To author the scripts, a new script editor is included that supports syntax highlighting and script validation. Detailed help and example code is available as well.

Improved classification mapping function

The classification mapping function now uses a tabular representation and has some new features:

  • Fill the lookup table with values encountered in the source data (Occurring values) or enumeration values defined in the schema
  • Load the lookup table from a CSV or Excel file
  • Save the lookup table as a CSV or Excel file

Improved Classification Mapping Function in HALE 2.8.0

Other new functions

As usual, there has been a set of other improvements (about 15). You can go here to learn more about these.

Simon will post separate workflow descriptions for the usage of PostGreSQL databases and on the Groovy Scripting in the next days. Enjoy your work with HALE 2.8.0!

HALE 2.7.0 with focus on Collaboration features

Originally we had planned our Autumn release to be a minor release – 2.6.1 – but this time we actually got far more done than originally expected. In the end, the new HALE version has so much to offer that it also deserves an intermediate version number increase – so we’re now at 2.7.0 (downloads, documentation). These are the things that you can now do with HALE:

Project Templates

To make your start with HALE easier, HALE now offers the possibility to use pre-configured project templates, e.g. for mapping to the INSPIRE Application Schemas. This saves you steps such as loading the schemas and setting up codelists. You can share your own projects as templates, example, or even as reference mappings online, to let others in your community profit from them:

http://hale.igd.fraunhofer.de/templates/

To select a template and to load it in HALE, use the New project from template option in the File menu or use the Project URL copied from the template website and use Open Alignment project and From URL.

HALE Web Templates

The Join Retype operation

Joins were on the backlog for a long time, since they are complex to resolve in our declarative approach. Now however HALE offers attribute based joins of different feature classes – to an arbitrary depth. To create a Join select multiple source types and one target type and choose the Join function.

Configure a Join Retype in HALE

Project and resources view

The new Project view lists all resources like source schemas, target schemas or code lists that are associated to the project. Also, you can edit basic project information in combination with the Properties view. Some types of resources, like code lists and lookup tables, can be removed here from the project using the context menu.

Export to JSON/GeoJSON

Transformed data can now be exported to JSON or GeoJSON, independently of what kind of schema the data is associated to. Objects are generically encoded as JSON/GeoJSON according to their structure.

Improved support for INSPIRE

HALE now supports the new code list XML format introduced recently by the INSPIRE registry. These code lists are relevant for the latest versions of the Annex II and III Application Schemas. In addition, transformed INSPIRE compliant features can now be saved to GML directly as an INSPIRE SpatialDataSet instead of the deprecated GML FeatureCollection.

On some of these features such as the Join functionality and the different types of template projects we will post separate workflow descriptions in the next days. Enjoy your work with HALE 2.7.0!

HALE on GitHub

GitHub has become the most popular platform for sharing open source code. It makes code accessible in a standard fashion – always the same pattern of access URLs and methods, just one account to manage contributions to many projects and it furthermore provides several clients that aim to lower the “geek” threshold of working with Git repositories. We, the HALE team, have been using Git for quite some time now and have furthermore been active on GitHub in various projects. Since our goal is to simplify contributions from the community, you can now access HALE on GitHub:

  1. If you don’t have a GitHub account already, create one. It won’t hurt :).
  2. Go to the hale repository on GitHub
  3. Fork the hale repository, so that it gets copied to your own account
  4. Clone your copy of the repository to your local environment
  5. Set up your local development environment
  6. Perform any changes to your local copy, commit them and push them to your remote repository
  7. Send us a pull request via GitHub to let us know that you have a change you’d like to contribute!
  8. We will then merge, check out whether the change still builds on our jenkins server, and accept the change if it’s good into the main repository.

Of course, you need to be aware that HALE is Open Source (LGPL 3.0), and you have to provide us with a note that you release all contributions you make under this license.

dhp-blogpost-standardimage

A Special Preview: The Next-Generation Schema Transformation Service is coming

The cst-wps project wraps the core transformation engine of HALE 2.1 and provides its functionality through a WPS interface. However, the cst-wps wasn’t in active development for some time, so what has happened?

The cst-wps project in its first incarnation was a collaboration project using the 52°North WPS framework to expose our transformation engine as a OGC Web Processing Service. The 52°N framework handles all requests, parsed and encoded GML and generally, while the cst engine performed the schema transformation according to a previously uploaded configuration. However, due to incompatible licensing at the time (52°North has, since then, changed their licensing rules), we decided to go ahead with a different WPS framework, called PyWPS. Integration proved to be complex, and the resulting project was hard to maintain. Thus, cst-wps became unsupported.

Of course, this wasn’t to be the last word. Based on our all-new transformation engine that is now shipped with HALE 2.5.0, we have also created a new cst web service. It is not yet officially released, but we want to give you a preview of what this entirely new piece of software is capable of. Continue reading