Home :: eea.rdfmarshaller

Metadata

Distro Index Owner:
eea
Home Page:
eea.rdfmarshaller
License
GPL
Version:
12.4
Last updated:
2021-03-10
Keywords:
EEA Add-ons Plone Zope

eea.rdfmarshaller

Files

Indexes

RDF marshaller for Plone

EEA RDF Marshaller

develop master

Export any Archetype content to RDF. It provides a few general adaptors for ATContentTypes and ATVocabularyManager. You can then look in eea.soer to find out how to customize these adaptors for your own RDF schemas and own content types.

Contents

Contents

Install

Updating Semantic Content-Registry

This add-on defines a Plone custom Content rule action called Ping CR. Thus, within Site Setup > Content Rules one can define custom content rules that will trigger the Ping CR action. This is an asynchronous action, thus it can be handled:

Externally via RabbitMQ

Just provide the RABBITMQ_ environment variables:

  • RABBITMQ_HOST - RabbitMQ domain name. e.g.: RABBITMQ_HOST=rabbitmq.apps.eea.europa.eu
  • RABBITMQ_PORT - Connect to RabbitMQ on this port. e.g.: RABBITMQ_PORT=5672
  • RABBITMQ_USER - Username to be used to connect to RabbitMQ. e.g.: RABBITMQ_USER=client
  • RABBITMQ_PASS - Password to be used to connect to RabbitMQ. e.g.: RABBITMQ_PASS=secret

Internally via zc.async

Source code

Funding

EEA - European Environment Agency (EU)

Changelog

12.4 - (2021-03-10)

  • Change: ping_cr updated to use the RabbitMQ instead of plone.app.async [alecghica refs #129707]

12.3 - (2020-01-15)

  • Change: Refs #111827 - Render UID as dcterms:identifier [mgax]

12.2 - (2019-09-11)

  • Bug fix: Fixed broken layout on footer when adding HTML code within Page description [avoinea refs #105522]

12.1 - (2019-08-19)

  • Change: Modified how the get_translation() and get_translated_langauges() methods are obtained for purposes of Plone5 compatibility [davidsilaghi refs #104219]
  • Change: render linkeddata viewlet only if we are on the view template where we really need it [ichim-david refs #107760]

12.0 - (2019-03-20)

  • Upgrade step: Within "Plone > Site setup > Add-ons" click on upgrade button available for eea.rdfmarshaller [avoinea refs #102083]
  • Feature: Ping semantic Content Registry via RabbitMQ message broker instead of zc.async if RabbitMQ is configured via eea.rabbitmq.plone [avoinea refs #102083]

11.6 - (2019-03-15)

  • Feature: configured the PingCR async jobs to retry on failure [alecghica refs #103659]

11.5 - (2019-01-28)

  • Jenkins: Add sonarqube step [avoinea refs #101552]

11.4 - (2018-12-21)

  • Change: use utf-8 for any archetypes field which avoids long dash tranformation when exporting to rdf and field has Windows-1252 encoding [ichim-david refs #100868]

11.3 - (2018-12-19)

  • Bug fix: use utf-8 for any archetypes field which avoids long dash tranformation when exporting to rdf [ichim-david refs #100868]

11.2 - (2018-07-10)

  • Bug fix: fixed linked-data-export viewlet exception [alecghica refs #96997]
  • Bug fix: fixed rdflib.graph in _assertnode error [alecghica refs #96997]

11.1 - (2018-07-03)

  • Change: updated URLs pointing to eea.europa.eu with https:// [alecghica refs #95849]
  • Bug fix: fixed URL for editing /www/SITE LinkedData Homepage [alecghica refs #96375]
  • Bug fix: expose LinkedData Homepage for multilingual content [alecghica refs #96375]

11.0 - (2018-06-05)

  • Bug fix: fixed case in linked data viewlet when title was empty [alecghica refs #95891]

10.9 - (2018-05-10)

  • Feature: added Carousel data export (for Collections and Faceted Navigations) [ichim-david refs #94557]
  • Change: pylint fixes needed after latest rounds of refactoring [ichim-david refs #94557]
  • Feature: added basic tests for json-ld linkeddata functionality [ichim-david refs #94557]

10.8 - (2018-04-05)

  • Feature: implemented Linked Data functionality [tiberich #92045, #91549, #86484]
  • Bug fix: error on editing content rules. [GhitaB]

10.7 - (2018-02-19)

  • Bug fix: Do not re-trigger ObjectMovedEvent on ObjectMovedorRenamed content-rule [avoinea refs #92869]

10.6 - (2018-02-14)

  • Bug fix: error on editing content rules and portal_registry. [GhitaB #82751]

10.5 - (2017-12-20)

  • Bug fix: don't break Plone root pages [tiberich #82751]
  • Refactored the archetypes implementation to split adding field values as a modifier [tiberich #82751]
  • Write a new Plone @@rdf adapter, it was wrongly using the generic Archetypes implementation and yielding unmonitored errors [tiberich #82751]
  • Removed the ATFolderish2Surf adapter, it wasn't doing anything special [tiberich #82751]
  • Refactored the dcterms_hasPart as a modifier in the generic subpackage, it's a generic concept.

10.4 - (2017-12-18)

  • Bug fix: fixed package structure and pinned versions [alecghica]
  • Change: Added superclasses for short field adapters [tiberich refs #90546]
  • Allow defining and assigning licenses per content types [tiberich, ghitab #82751]

10.3 - (2017-12-12)

  • Change: Replace eeacms/zptlint with eeacms/plone-test:4 zptlint [avoinea refs #90415]

10.2 - (2017-09-26)

  • Cleanup: Remove obsolete store.log references [avoinea]

10.1 - (2017-09-22)

  • Change: upgrade rdflib and surf. Change logging to be compatible with new version. Get rid of surf.rdflib dependency, now is included in latest Surf. Many other compatibility changes. [tiberich #88194]
  • Bug fix: sometimes collective.cover is not available, fallback on import error [tiberich]

10.0 - (2017-08-16)

  • Bug fix: For dexterity use relatedItems for backreferences @pingCRSDS_backrel [iulianpetchesi refs #86764]
  • Bug fix: Properly export size for file fields. Don't include file field in rdf output when field has no value [zoltan_andras refs #86696]
  • Change: Added some unit tests for file fields [zoltan_andras refs #86696]

9.9 - (2017-06-26)

  • Change: added skos namespace to eea.rdfmarshaller [ichim-david refs #85617]
  • Bug fix: fixed failing tests due to depiction changes [zoltan_andras refs #81209]

9.8 - (2017-06-06)

  • Change: Save dexterity cover tiles content as unicode [iulianpetchesi refs #85565]

9.7 - (2017-05-23)

  • Bugfix: fix dexterity loading condition [tiberich #83153]

9.6 - (2017-05-22)

  • Change: Add dedicated async quota for RDF jobs (e.g.: ping_CRSDS) [avoinea refs #84915]
  • Change: Remove hard-dependency on plone.app.async [avoinea refs #84915]
  • Change: Fix dexterity test for eea.rdfmarshaller [iulianpetchesi refs #84170]
  • Change: Moved the dexterity image/file modifiers and added zcml condition for plone.app.contenttypes [iulianpetchesi refs #84170]
  • Change: Moved the collective.cover modifier to the extras.py file and added zcml condition for it [iulianpetchesi refs #84170]
  • Feature: Added IValue2Surf implementation for sets and fields to the dc_map [iulianpetchesi refs #84170]
  • Feature: Ported modifiers from archetypes to dexterity, added rdf support for files and images and excluded some dc fields from the rdf schema [iulianpetchesi refs #82861]
  • Feature: added compatiblity with collective.cover [iulianpetchesi refs #79312]
  • Feature: added compatiblity with plone.dexterity content types
  • Change: refactored Archetypes support, to extract bits reused in Dexterity compatibility implementation [tiberich refs #79312]
  • Change: changed how the Distribution is expressed in @@rdf. Now it is a part of a file field representation, which are now exported instead of being hidden. [zoltan_andras refs #83153]
  • Changed: show how to make sparql queries from the playground script [zoltan_andras refs #83153]
  • Change: fixed PyLint warnings and errors [valipod refs #84949]
  • Upgrade step: the "image" and "file" fields should be removed from the site_properties/rdf_marshaller blacklist [alecghica refs #83153]

9.5 - (2017-05-15)

  • Change: fixed PyLint warnings and errors [eduard-fironda refs #84949]

9.4 - (2017-04-24)

  • Change: updated package information [eduard-fironda]

9.3 - (2017-04-14)

  • Bug fix: Improved RDF export sanitize method [avoinea - refs #83543]
  • Change: Registered FOAF namespace
  • Change: Added /scripts for fast experimentation with surf/rdfmarshaller [zoleesan - refs #83153]

9.2 - (2017-03-29)

  • Change: Force ping_cr on HTTP [avoinea - refs #83686]

9.1 - (2017-03-27)

  • Change: rollback changes done in 8.9 where rdf export ignored non ascii characters as we loose multilingual data [ichim-david refs #83543]

9.0 - (2017-01-30)

  • Bug fix: ignore invalid xml given to the lxml parser [ichim-david refs #80209]

8.9 - (2016-12-19)

  • Change: rdf export now ignores non ascii characters [ichim-david refs #77804]

8.8 - (2016-12-05)

  • Change: remove private unicode range characters from rdf output [ichim-david refs #77804]

8.7 - (2016-10-31)

  • Bug fix: fixed error handling [ghicaale refs #75566]

8.6 - (2016-10-14)

  • Bug fix: fixed non ascii characters for rdf export [szabozo0 refs #77766]

8.5 - (2016-10-10)

  • Bug fix: fixed "Can't pickle <class 'urllib2.URLError'>" error from ping CR [ghicaale refs #75566]

8.4 - (2016-08-03)

  • Bug fix: fixed/updated tests [ghicaale refs #74285]

8.3 - (2016-07-28)

  • Change: Include expired content in rdf export [chiridra refs #73776]

8.2 - (2016-05-27)

  • Feature: registered schema.org namespace [alecghica refs #72025]

8.1 - (2016-05-16)

  • Bug fix: Fix pylint warnings [ichim-david refs #71940]
  • Bug fix: Fixed condition for ping_cr with create=true [szabozo0 refs #72273]

7.9 - (2015-12-07)

  • Change: Removed CSRF security quickfix for tests [szabozo0 refs #30858]
  • Bug fix: check if relation is not None before ping action [chiridra refs #30387]

7.8 - (2015-10-05)

  • Bug fix: set socket timeout to prevent hanging out connection on urlopen calls with eventlet [lucas refs #29063]
  • Change: Improved relatedItems rdf export using dublincore [chiridra refs #28697]
  • Upgrade step: we need to re-index all content on SDS
    • in ZMI -> www/0maintenance/ping_sds_content_script/ZPythonScriptHTML_editForm
    • add meta_type as a parameter and run the script
    • [ghicaale refs #28697]

7.7 - (2015-09-10)

  • Bug fix: fixed UnicodeDecodeError: 'utf8' codec can't decode byte for @@rdf [chiridra refs #28293]

7.6 - (2015-03-17)

  • Change: Switched to curl in the jenkins build install script [olimpiurob refs #22402]
  • Change: Changed fetch url for jenkins build install script to the install script from the main EEA CPB repository [olimpiurob refs #22402]

7.5 - (2015-01-22)

  • Feature: Send SDS pings with the url specified in the portal_url environment variable if available [olimpiurob refs #21726]

7.4 - (2014-12-05)

  • Bug fix: fixed ATFileField2Surf marshalling when file or image has no data [ichimdav refs #22047]
  • Bug fix: fixed child pinging when parent object is a comment [ichimdav refs #22047]

7.3 - (2014-11-20)

  • Feature: added SearchableText() to the rdf export as dcterms:abstract [pasoviul refs #21415]
  • Feature: ping backward relations on rename/move and delete event [ghicaale refs #21480]
  • Feature: ping translations [ghicaale refs #21480]
  • Feature: ping all children recursively [ghicaale refs #21480]
  • Change: removed duplicate code from ping action implementation [ghicaale refs #21480]

7.2 - (2014-06-06)

  • Bug fix: fixed the URL for the Ping service [ghicaale refs #18928]

7.1 - (2014-05-30)

  • Bug fix: detect special object used to force acquisition and fix case when there is no Aquisition there is no container. [ghicaale refs #18904]

7.0 - (2014-05-29)

  • Feature: added content rule event for copy&paste. [ghicaale refs #18904]

6.9 - (2014-05-26)

  • Bug fix: fixed the CR PING not to endup in AttributeError: REQUEST when is called from a async task. [ghicaale refs #19830]

6.8 - (2014-05-26)

  • Bug fix: fixed the CR PING not to endup in AttributeError: REQUEST when is called from a async task. [ghicaale refs #19830]

6.7 - (2014-05-19)

  • Bug fix: Event handler for IObjectMovedEvent subscriber now needs two arguments [batradav refs #19725]

6.6 - (2014-05-19)

  • Bug fix: IObjectMovedEvent subscriber only for Archetypes [batradav refs #19725]

6.5 - (2014-05-13)

  • Change: Added a view to allow adding an external method to ping all elements of a known portal type and their aliases [pasoviul refs #18928]
  • Bug fix: Changed permission for @@rdf and @@rdfs to zope2.View so it can be viewed by anonymous users only if the object was published [szabozo0 refs #18904]
  • Feature: Added new content rule for Object Moved Or Renamed event to allow, so it can be configured from @@rules-controlpanel [szabozo0 refs #18904]
  • Bug fix: Monkeypatch for plone.app.contentrules.handlers.DuplicateRuleFilter, so the ping is called also when an object is removed via actions->delete [szabozo0 refs #18904]

6.4 - (2014-03-10)

  • Bug fix: stripped illegal characters out of xml [pasoviul refs #18511]

6.3 - (2014-02-13)

  • Change: Workflow state is linked to a new vocabulary [pasoviul refs #17675]

6.2 - (2014-01-22)

  • Change: RDF marshaller exports file size and download URL [pasoviul refs #14872]
  • Change: no longer hardcode eea website link when exporting the workflow [ichimdav refs #17811]
  • Bug fix: no longer crash rdf export if object doesn't have a workflow enabled [ichimdav refs #17811]
  • Bug fix: fixed encoding issues on rdf export [ghicaale refs #17668]
  • Change: celanup unused code for mimetypes_registry rdf export [ghicaale refs #9191]

6.1 - (2013-12-17)

  • Feature: marshaller exports workflow state [pasoviul refs #17675]

6.0 - (2013-12-10)

  • Bug fix: fixed ReST markup of HISTORY.txt [ghicaale refs #17716]

5.9 - (2013-12-06)

  • Bug fix: find and remove all related brains when creating tmp obj for schema [simiamih refs #17298]
  • Feature: do not ping SDS from .webdev.eea.europa.eu [ghicaale refs #17664]

5.8 - (2013-11-20)

  • Bug fix: Handle wrong ping urls [szabozo0 refs #17485]

5.7 - (2013-11-18)

  • Bug fix: fixed HISTORY.txt to reflect changes [ghicaale refs #17487]
  • Bug fix: fixed encoding issues on rdf export [pasoviul refs #16747]

5.6 - (2013-07-08)

  • Feature: Implemented SDS Ping feature when resource was created or changed [szabozo0 refs #14473]
  • Bug fix: when loading zcml, also include zcml for plone.contentrules and plone.app.contentrules [tiberich]
  • Upgrade Step: - in ZMI -> portal_setup -> upgrades - select the profile: eea.rdfmarshaller:default - upgrade to 5.6
  • Upgrade Step: in https://www.eea.europa.eu/aq_parent/portal_properties/rdfmarshaller_properties remove the "provenances" field from the "blacklist" list.

5.5 - (2013-06-17)

  • Feature: implemented the eea:objectProvides modifier. This add information about provided interfaces to the @@rdf output [tiberich #14718]

5.4 - (2013-01-28)

  • Change: Updated and cleaned up broken doctests. [demarant refs #13589]

5.3 - (2013-01-15)

  • Change: do not export children objects when requesting an RDF export of a folderish content. Keep only hasPart statements. This is to avoid redundancy and speed-up rdf export of large folderish content. [demarant fixes #13589]

5.2 - (2012-11-22)

  • Change: registered OWL namespace globally as it is often needed. [demarant refs #7409]
  • Bug fix: fixed more occurences of UnicodeDecodeError. Do not try to encode already unicode strings. [demarant refs #9293]
  • Bug fix: Map plone location field to dcterms:spatial for geotag information. [demarant refs #3425]
  • Bug fix: also register the geo namespace, for pretty printing [tiberich refs #3425]
  • Refactor: improve the API to make it more clear about the intent of classes and interfaces. Now we no longer assume that all exported objects are archetypes, we have a more pluggable way to override, per field_type + fieldname + context_type the output. Also make the value extracting logic a bit easier to follow by having adapters for each type of value (list, string, datetime, etc). [tiberich refs #3981]

5.1 - (2012-07-17)

  • Bug fix: only provide rdf view for real archetypes content, FTI and portal tools [tiberich refs #5301]

5.0 - (2012-07-16)

  • Bug fix: also handle UnicodeDecodeError in when trying to convert text using chardet.detect [tiberich refs #5308]

4.9 - (2012-07-13)

  • Bug fix: avoid unicode decode errors in @@rdf [tiberich refs #5308]

4.8 - (2012-06-20)

  • Bug fix: hasTranslations must also be an rdf:resource (not a literal value) in coherence with the isTranslationOf and other dcterms relations. [demarant refs #5130]
  • Bug fix: fixed hasTranslation recursive relation to itself. [demarant refs #5130]
  • Bug fix: blacklisted nextPreviousEnabled and excludeFromNav [tiberich refs #5130]

4.7 - (2012-06-12)

  • Bug fix: fixed markup of HISTORY.txt file [ciobabog refs #5231]
  • Change: added eea:isTranslationOf to point to translations [tiberich #5130]
  • Bug fix: use dcterms:issued instead of dcterms:effective for DC [tiberich #5135]
  • Feature: use a mechanism of subscribers to allow modification of the surf resource by package plugins [tiberich #5129]
  • Bug fix: fix @@rdf view for mimetypes_registry [tiberich #5206]

4.6 - (2012-03-30)

  • Empty release

4.5 - (2012-02-15)

  • Bug fix: fixed tests [tiberich]

4.4 - (2012-02-10)

  • Bug fix: fixed a problem with related fields that are empty [tiberich #4942]

4.3 - (2012-02-02)

  • Bug fix: fixed a problem inserting related items in rdf stream when relation field is single value [tiberich #4992]

4.2 - (2011-12-07)

  • Bug fix: Unindex temporary objects created in portal_factory [voineali #4822]

4.1 - (2011-11-18)

  • Rerelease broken egg

4.0 - (2011-11-18)

  • Bug fix: use getFolderContents instead of objectValues() to retrieve children (v0.13 backport) [tiberich #4418]
  • Bug fix: Fix tests for eea.rdfmarshaller [ichimdav #4340]
  • Change: use dcterm namespace for metadata instead of dc [tiberich #4250]
  • Cleaned: pyflakes warnings [ichimdav #4141]
  • Cleaned: pylint violations [ichimdav #4140]
  • Feature: Added upgrade steps [voineali #4392]
  • Feature: Plone 4.0.4 compatible release [ghicaale #4258]

0.8 - (2011-02-09)

  • Bug fix: skip fields for which the accessor cannot be determined [tiberich #3982]

0.7 - (2010-12-16)

  • set surf logging level to CRITICAL to avoid all INFO messages [vincisas #3951]

0.6 - (2010-11-10)

  • typo fix for tests [vincisas]

0.5 - (2010-09-30)

  • dcterms:hasPart added to folderish content [vincisas #3519]
  • dcterms:isPartOf added to contained content [vincisas #3519]

0.4 (2010-09-06)

  • use accessor to get the value of fields [tiberich #3550]
  • 'file' added to blacklist as a temporary fix for encode error [ghicaale]
  • under marshaller.py added temporary fix for: 'tuple' object has no attribute 'replace' [ghicaale]
  • Silent failure with schema or rdf generation, errors are logged [vincisas #3564]
  • Schemageneration contains extended fields too [vincisas #3577]

0.3 (2010-08-03)

  • blacklist is configurable through ZMI in properties_tool/rdfmarshaller_properties [vincisas]
  • lowercased dc elements [vincisas]
  • rdf schema generation from portal type FTIs added [vincisas]
  • rdf export done with autogenerated schemas and portal type specific namespaces [vincisas]
  • folderish export added [vincisas]
  • ATVocabularyManager vocabularies exported as RDF [vincisas]

0.2 (2010-06-15)

  • Cleanup and more generic export moved here from eea.soer [vincisas]
  • Blacklist added [vincisas]

0.1 (2010-05-19)

  • Initial release