Home :: eea.daviz

Metadata

Distro Index Owner:
eea
Home Page:
eea.daviz
License
GPL
Version:
12.4
Last updated:
2020-02-28
Keywords:
EEA Add-ons Plone Zope

eea.daviz

Files

Indexes

EEA DaViz is a plone product which uses Exhibit and Google Charts API to easily create data visualizations based on data from csv/tsv, JSON, SPARQL endpoints and more.

EEA Daviz

DaViz logo

Develop Master

Introduction

EEA Daviz is a web tool developed by the European Environment Agency which helps creating interactive data visualizations easily through the web browser, no extra tools are necessary. It is free and open source.

You can generate attractive and interactive charts and combine them in a dashboard with facets/filters which updates the charts simultaneously. Data can be uploaded as CSV/TSV or you can specify SPARQL to query online Linked open data servers (aka sparql endpoints).

Daviz is the first Semantic web data visualisation tool for Plone CMS, entirely web-based!

At the moment Simile Exhibit and Google Charts visualizations are supported. The architecture allows to extend Daviz with more visualisation libraries (visualisations plugins).

https://eea.github.io/_images/eea.daviz.cover.png

Contents

Main features

Daviz features diagram

  1. No desktop tools needed to make visualizations. all web based.
  2. Data input: easy copy and paste of data table from any webpage or excel sheet
  3. Data input from URL (CSV/TSV/JSON)
  4. Data input: drag-and-drop your data from CSV/TSV files, and we do the rest
  5. Data input advanced: retrieve data from any SPARQL endpoint on the fly
  6. Intuitive visualization editor to create interactive charts.
  7. Large amount of visualizations available: Area, Bar, Bubble, Candlestick, Column, Combo Chart, Gauge, Geo Intensity Maps, Line, Pie, Radar, Scatter, Stepped Area, Table, Tree Map, Motion Charts, Faceted search table, Faceted tiles, Faceted timeline, Faceted map and more...
  8. Dashboard. Group charts into one or more dashboards.
  9. Share any chart. Embeddable visualization in any webpage.
  10. Customizable chart options and colors
  11. Data table manipulation via drag and drop, preparing table for chart
  12. Pivot table (Transpose), transform row values into columns
  13. Modular framework for extending it with more visualizations
  14. Branding: add your own logo to each chart + QR code
  15. And much more...

It is simple to use, needs no desktop application, everything is done through the web by uploading an "excel file", CSV, TSV. You can also query the "web of data" via public available sparql endpoints.

You can easily make visualizations like:

  1. Simile Exhibit
  2. Google Charts

See also initial project wiki page for the reasoning behind this project.

Video Tutorials

Most of DaViz features are presented throughout a series of video tutorials available under the EEA Web Systems Training YouTube channel. The screencasts are divided in two playlists: Basic and Advanced.

Architecture overview

At the moment Simile Exhibit and Google Charts visualizations are supported. The architecture allows to extend Daviz with more visualisation libraries (visualisations plugins).

https://eea.github.io/_images/eea.daviz.layers.svg

Installation

zc.buildout

If you are using zc.buildout and the plone.recipe.zope2instance recipe to manage your project, you can do this:

  • Update your buildout.cfg file:

    • Add eea.daviz to the list of eggs to install
    [instance]
...
eggs =
  ...
  eea.daviz

  • Re-run buildout, e.g. with:

    $ ./bin/buildout

Google Maps API Key

Plone

Within ZMI -> Plone Site -> portal_properties add a plone property sheet called geographical_properties and inside it add a new string property called google_key.

Zope

Within ZMI -> Top Folder -> manage_propertiesForm add a string property called google_key

In this property you have to paste the Google maps API KEY, follow instruction https://developers.google.com/maps/documentation/javascript/tutorial#api_key

The Google account you use to generate the key has to be owner of the site, this is done by verification via Google webmaster tools.

Dependencies

EEA Daviz has the following dependencies:
The following package are optional. Still they can improve the user experience with this tool:
[instance]
...
eggs =
  ...
  eea.daviz [full]

zcml =
  ...
  eea.daviz-overrides
  eea.daviz-full
https://eea.github.io/_images/eea.daviz.dependencies.svg

Source code

Latest source code (Plone 4 compatible):
Plone 2 and 3 compatible (Simile Exhibit visualisations only):
https://github.com/collective/eea.daviz/tree/plone25

Funding and project management

EEA - European Environment Agency (EU)

Changelog

12.4 - (2020-02-28)

  • Change: use prop instead of attr when setting value on input tags for jquery 1.9+ compatibility [ichim-david refs #105112]

12.3 - (2019-12-10)

  • Task: Added a daviz custom adapter for image scales based on plone.app.blob store scale [iulianpetchesi #110463]

12.2 - (2019-09-10)

  • Cleanup: Get rid of getDataOwner possible relation to solve install problem for ENI SEIS. [GhitaB #108504]

12.1 - (2019-01-28)

  • Jenkins: Add sonarqube step [avoinea refs #101552]

12.0 - (2018-09-07)

  • Upgrade step: Within "Plone > Site setup > Add-ons" click on upgrade button available for eea.daviz [avoinea refs #98283]
  • Cleanup: Remove eea.exhibit dependency [avoinea refs #98283]

11.6 - (2018-06-20)

  • Change: updated URLs pointing to eea.europa.eu with https:// [alecghica refs #95849]

11.5 - (2018-05-30)

  • Bug fix: Restrict getBRefs only to related items [avoinea refs #95796]

11.4 - (2018-04-11)

  • Bug fix: retrive url of Assessment Daviz relation from portal_redirection in case the given relation was renamed and the uid no longer points to the correct object [ichim-david refs #94042]
  • Feature: added @@fix_broken_daviz_relations which when called on the Assessment content type will attempt to fix relations that have been renamed [ichim-david refs #94042]
  • Change: clear floats for daviz-settings [ichim-david refs #91590]

11.3 - (2018-03-14)

  • Change: call js code within jQuery dom ready allowing for deferred js loading [ichim-david refs #91577]

11.2 - (2017-12-12)

  • Change: Replace eeacms/zptlint with eeacms/plone-test:4 zptlint [avoinea refs #90415]

11.1 - (2017-11-07)

  • Change: Remove Sphinx generated documentation [petchesi-iulian refs #88212]

11.0 - (2017-05-22)

  • Change: fixed PyLint warnings and errors [valipod refs #84949]

10.9 - (2017-05-15)

  • Change: fixed PyLint warnings and errors [eduard-fironda refs #84949]

10.8 - (2017-05-02)

  • Cleanup: Remove older broken upgrade steps (< 5.0) [avoinea]

10.7 - (2017-04-24)

  • Change: updated package information [eduard-fironda]

10.6 - (2017-01-17)

  • Bug fix: Add extra_require for zope2 only deployments [avoinea]

10.5 - (2016-12-28)

  • Change: Make UnicodeDecodeError more verbose [voineali refs #79895]

10.4 - (2016-10-31)

  • Change: manage_beforeDelete now safely checks for request [ichim-david refs #75296]

10.3 - (2016-05-19)

  • Bug fix: Fix pylint warnings [chiridra refs #71940]

10.2 - (2016-05-04)

  • Feature: html body field for daviz and gis application - added new field "body" - modified the view template [chiridra refs #70146]

10.1 - (2016-03-14)

  • Bug fix: fixed provenances field when using eea.daviz in non eea environment [ichimdav]
  • Bugfix: Fixed order of column labels when labels are numbers [szabozo0 refs #69360]

10.0 - (2016-03-01)

  • Change: Data provenance is no longer required except for publishing [ichimdav refs #68228]

9.9 - (2016-02-15)

  • Bug fix: check for Browser within dataservice.js otherwise Sparql edit fails when plone.quickupload ins't present [ichimdav]

9.8 - (2015-11-16)

  • Feature: Improved chart chooser for inline embed to show only googlecharts: exhibits are currently not supported for embedding [chiridra refs #30417]
  • Feature: relate to object if resolveuid is used in data provenance [lucas refs #30065]

9.7 - (2015-07-28)

  • Bug Fix: unload video over HTTPS [lucas refs #26201]

9.6 - (2015-07-06)

  • Bug Fix: tutorials use now youtube api v.3 [lucas refs #26201]

9.5 - (2015-05-15)

  • Change: Mark data provenance fields as required [szabozo0 refs #25341]

9.4 - (2015-05-05)

  • Change: Made all data provenance fields required: title, link, owner [szabozo0 refs #24412]

9.3 - (2015-04-02)

  • Change: Made data provenance required for published [szabozo0 refs #24412]
  • Upgrade step: Within "Plone > Site setup > Add-ons" click on upgrade button available for eea.daviz
  • Feature: Added IDavizLayer browser layer [olimpiurob refs #24121]
  • Change: Made "MultiDataProvenanceSchemaExtender" browser layer aware [olimpiurob refs #24121]

9.2 - (2015-03-17)

  • Change: Switched to curl in the jenkins build install script [olimpiurob refs #22402]
  • Change: Changed fetch url for jenkins build install script to the install script from the main EEA CPB repository [olimpiurob refs #22402]

9.1 - (2014-11-20)

  • Feature: formated_provenance now contain the acronym information extracted from the title of the data source if found within parenthesis [ichim-david refs #21467]

9.0 - (2014-10-01)

  • Feature: removed "Caching" action. We are now using the more complex "Cache" action registered by eea.cache [ghicaale refs #21143]
  • Feature: added upgrade step to remove absolete "Caching" action [ghicaale refs #21143]

8.9 - (2014-08-29)

  • Feature: On the daviz edit page ask for confirmation before allowing users to modify data sources [szabozo0 refs #20869]
  • Bug fix: Exclude relations where related item has link to current daviz in it's provenance info [szabozo0 refs #20869]

8.8 - (2014-08-27)

  • Change: Also include eea.cache within full.zcml [voineali refs #20678]

8.7 - (2014-06-30)

  • Change: Added documentation for end users [szabozo0 refs #18400]
  • Change: Read the tags for tutorial screencasts from the description of the video [szabozo0 refs #18400]

8.6 - (2014-06-06)

  • Feature: Added tutorials [szabozo0 refs #18400]

8.5 - (2014-05-13)

  • Change: Added support for null value for slickgrid boolean fields [olimpiurob refs #19206]

8.4 - (2014-04-29)

  • Change: Made Daviz Visualization plone_lock_info aware [voineali refs #19172]
  • Bug fix: Disabled rename on spreadsheet when editing an already existing visualization [szabozo0 refs #19149]
  • Feature: Added file type to allowed content types list, required for saving svg versions of the charts [szabozo0 refs #18910]
  • Change: Added tutorials link on edit page [szabozo0 refs #18400]

8.3 - (2014-03-10)

  • Change: Always display the tutorials and example links on edit [szabozo0 refs #18400]
  • Change: Link video tutorials within documentation [avoinea]

8.2 - (2014-02-03)

  • Bug fix: Fixed path handling of internal links [ghicaale refs #16666]

8.1 - (2014-01-22)

  • Bug fix: Fixed the case then provenance URL points to a view or template [ghicaale refs #16666]
  • Feature: adding Sphinx-generated documentation in Page Template format [batradav refs #9502]

8.0 - (2014-01-14)

  • Bug fix: Don't reload edit page after a file was uploaded [szabozo0]
  • Feature: When adding internal link for data provenance add the object as a related item [ghicaale refs #16666]

7.9 - (2013-12-09)

  • Feature: Save chart's customized size in the DAVIZ_CHARTS annotation [szabozo0 refs #17453]

7.8 - (2013-10-04)

  • Change: Updated package dependencies to reference the newly prefered Zope locations that require minimum Plone 4.1 [ichimdav refs #15651]

7.7 - (2013-09-06)

  • No changes

7.6 - (2013-08-30)

  • Bug fix: Display data owner title in provenance info [szabozo0 #16585]

7.5 - (2013-06-17)

  • Feature: Changed data provenance to allow multiple data provenances; and precedence of provenances [szabozo0 #9561]
  • Bug fix: add an override for manage_beforeDelete to trigger a DavizWillBeRemovedEvent; intercept this event to delete the annotation on the backreferenced object that involves the deleted daviz [tiberich #14341]
  • Bug fix: pylint fixes [ghicaale refs #14554]
  • Feature: locales improved [lepri]

7.4 - (2013-04-17)

  • Bug fix: fix migration in eea.relations integration [tiberich #14267]

7.3 - (2013-03-20)

  • Bug fix: make DavizVisualization inherit from Products.ATContentTypes instead of plone.app.folder to avoid problem with LinguaPlone generated mutators [tiberich #14050]

7.2 - (2013-03-18)

  • Bug fix: Fixed loss of data settings customizations if one would have changed any other metadata than 'spreadsheet (Copy and paste a data table from a file/webpage)'within Edit tab [voineali refs #14172]

7.1 - (2013-03-15)

  • Change: make chart selection for relations sortable [tiberich #13993]
  • Upgrade step: go to /www/@@migrate_daviz_relations to migrate to sortable relations for daviz/ims integration [tiberich #13993]
  • Feature: added information for contributors [ciobabog refs #13892]
  • Change: Added description to controlpanel fields [szabozo0]
  • Bug fix: fixed small bug in code that broke get_daviz when daviz was deleted [tiberich #13993]

7.0 - (2013-02-25)

  • Change: removed redundant and unnecessary data_source provenance code, which is already better implemented with @@data.info [demarant refs #9561 and #9571]

6.4 - (2013-02-04)

  • Feature: Support data annotations [voineali refs #9558]

6.3 - (2013-01-15)

  • Upgrade step: Within "Plone > Site setup > Add-ons" run upgrades (if available) for the following packages in the same order as bellow: - EEA Versions (eea.forms) - EEA Forms (eea.forms) - EEA Sparql (eea.sparql) - EEA Visualization API (eea.app.visualisation); - EEA Google Charts (eea.googlecharts); - EEA Exhibit (eea.exhibit); - EEA Daviz (eea.daviz)
  • Feature: Added italian translations [simahawk]

6.2 - (2013-01-10)

  • Upgrade Step: if you have eea.relations installed in your system, in portal_setup, Import tab, select the profile "EEA Daviz: eea.relations integration" and import the Javascripts Registry step.
  • Upgrade step: Within "Plone > Site setup > Add-ons" click on upgrade button available for eea.daviz
  • Feature: Added WYSIWYG table editor (SlickGrid) when one directly copy&paste dataset within free text field [voineali refs #9610]
  • Feature: added integration with eea.relations [tiberich refs #9571]

6.1 - (2012-11-09)

  • Upgrade step: Within "Plone > Site setup > Add-ons" click on upgrade button available for eea.daviz
  • Feature: Added i18n translations [avoinea]
  • Feature: Possibility to add data source from a external URL [voineali refs #9576]

6.0 - (2012-10-08)

  • Upgrade step: ZMI > portal_setup > profile "EEA Daviz" > import Action Providers settings
  • Feature: Added Visualization data source provenance adapter for DavizVisualization objects [voineali refs #5490]
  • Feature: On "Create Visualization" create the visualization and redirect to "Edit visualization" tab [szabozo0 refs #5449]
  • Feature: Use portal_daviz utility for default folder settings [szabozo0 refs #5449]
  • Change: Added daviz.json adapter for IDavizVisualization content-type [voineali refs #5449]
  • Change: Added IVisualizationData adapter for DavizVisualization content-type [voineali refs #5511]

4.9 - (2012-08-10)

  • Upgrade step: Within "Plone > Site setup > Add-ons" click on upgrade button available for eea.daviz [szabozo0 refs #5347]
  • Bugfix: Fixed columns order [szabozo0 refs #5137]
  • Feature: Added new action on data visualization "Create another visualization"; Added action on Sparql "Create visualization with this dataset"; Added action on all object which provides IVisualizationEnabled "Create another visualization" [szabozo0 refs #5347]
  • Feature: use title of data source as default title for visualisation [szabozo0 refs #5138]

4.8 - (2012-07-13)

  • Change: updated readme with info about google map API KEY configuration. [demarant refs #5338]

4.7 - (2012-07-04)

  • Bug fix: Fixed possible_relations.xml when used togheter with eea.relations on a non-EEA Plone site [voineali refs #5330]

4.6 - (2012-06-12)

  • Bug fix: fixed markup of HISTORY.txt file [ciobabog refs #5231]
  • Feature: Warn user about changing data risks [voineali fixes #4973]
  • Feature: Added @@data.info adapter for DavizVisualization content-type in order to easily get data source and data owner metadata [voineali refs #5072]

4.5 - (2012-04-26)

  • Feature: Possibility to add data provenance/source information when one copy&paste data tables or upload them via QuickUpload Widget [voineali refs #5072]

4.4 - (2012-03-30)

  • Feature: added documentation and links on how to examples and how prepare data for visualizations. [demarant #4995]
  • Change: added default workflow mapping for daviz visualization. [demarant #5002] Upgrade step: if you have already been using daviz visualization you would need to migrate the instances to the new workflow states. Plone > Site setup > Types > choose Daviz Visualization and the new worklow to folder_workflow. Define the states mapping as you prefer.
  • Upgrade step: Within "Plone > Site setup > Add-ons" click on upgrade button available for eea.daviz
  • Feature: Updated Daviz Visualization fields help to provide examples and tutorials [voineali #4995]
  • Change: do not show duplicates of sparql and json data when adding existing data. show only last version. Upgrade step (not needed for EEA production): ZMI > portal_setup > profile "EEA Daviz" > import Possible relations settings [demarant #5003]
  • Feature: Refactoring to be able to use eea.googlecharts or eea.exhibit as standalone packages or as part of eea.daviz visualization bundle. [voineali #4996]

4.3 - (2012-03-05)

  • Bug fix: restricted data sources relations to exhibit json or sparql queries. Upgrade step: ZMI > portal_setup > profile "EEA Daviz" > import Possible relations settings [demarant #4919]
  • Change: The new UI expects eea.googlecharts and eea.sparql, therefore they are now an integral part of this product and declared as dependencies. [demarant]
  • Change: Renamed DavizPresentation to DavizVisualization. [demarant]
  • Change: removed label widget for data input, no need for it, we have already many headings. [demarant]
  • Change: Re-group DavizPresentation AT Schema [voineali #4980]
  • Change: Reordered the data input fields. Most common scenarion on top. rephrasing. [demarant #4980]
  • Upgrade step: Within "Plone > Site setup > Add-ons" click on upgrade button available for eea.daviz [voineali]
  • Fixed: Support case insensitive column types in CSV files [voineali #4975]
  • Feature: Also support CSV or any other SV file by auto-detecting dialect [voineali #4977]
  • Feature: Added SpreadSheet field to DavizPresentation Schema in order to easily copy & paste TAB separated text as data source [voineali #4967]
  • Feature: Added QuickUpload field to DavizPresentation Schema in order to easily upload new TAB separated files as data sources [voineali #4965]
  • Change: changed texts for visualizations, changed Data Sources to be mandatory [szabozo0 #4948]
  • Bug fix: properly link images in css files [tiberich #4956]
  • Change: corrected daviz presentation description [demarant]

4.2 - (2012-02-06)

  • no changes

4.1 - (2012-02-06)

  • Upgrade step: Within "Plone > Site setup > Add-ons" click on upgrade button available for eea.daviz [voineali #4866]
  • Feature: Moved facets in global settings tab in order to have more space for views configuration [voineali #4914]
  • Feature: Added a CSS class on section tabs [voineali #4880]
  • Feature: Redirects to daviz-edit if there is no DaViz view/chart configured [voineali #4882]
  • Feature: Added a way to register additional charts as tabs in view mode. [voineali #4873]
  • Feature: Added DavizPresentation content-type in order to easy aggregate internal and external exhibit sources [voineali #4866]

4.0 - (2011-11-07)

  • Feature: Added upgrade steps [voineali #4392]
  • Feature: Plone 4.x compatible [ichimdav, voineali #4234]
  • Feature: Added exhibit 'properties' dict in generated JSON [voineali 4528]

1.5 - (2011-09-21)

  • Feature: Added exhibit 'properties' dict in generated JSON [voineali 4528]

1.4 - (2011-09-07)

  • Feature: Enabled daviz for eea.dataservice EEAFigureFile [voineali #4492]

1.3 - (2011-07-29)

  • Feature: Added babel translator API and a custom translator: rdf-xml to exhibit-json [voineali #4416]
  • Feature: Possibility to add external exhibit data sources like "Google Spreadsheet", "RDF/XML" and "External JSON". [voineali #4416]
  • Feature: Added editable 'Lens' property to Tiles view in order to easily create custom Exhibit Lens Templates [voineali #4414]
  • Feature: Added a way to reorder Exhibit Views [voineali #4177]
  • Feature: Added a way to reorder or hide Exhibit Facets [voineali #4417]
  • Feature: Possibility to add/remove facets [voineali #4416]
  • Feature: Made Exibit edit page more user friendly (WYSIWYG) [voineali #4417]
  • Feature: JSON Converter handles column types if specified in CSV columns header (ex: Title:lable, Year:date, etc) [voineali #4414]
  • Feature: Added 'Details' column for 'Tabular View'. This can be enabled/disabled in 'Tabular View' settings. [voineali #4419]

1.2 - (2011-04-07)

  • Feature: Added edit form for generated JSON [voineali #4199]
  • Feature: If column name contains ":list" exported data in JSON will pe an array [ghicaale #4199]
  • Bug fix: Made exibit available for ATBlob files [voineali #4135]
  • Cleanup: of pyflakes warnings [ichimdav #4141]
  • Cleanup: of sensible pylint violations [ichimdav #4140]
  • Feature: Added live search field as default on top [demarant, #3023]
  • Change: Moved facets to the right [demarant]
  • Bug fix: Fixed Google Map Key alert [voineali #4196]
  • Change: Disabled Thumbnails View as it's bugous on IE [voineali #4196]

1.1 - (2011-03-16)

  • Bug fix: Removed jQuery and jQuery UI from daviz-edit.css, daviz-edit.js as they generate errors if already installed. You'll have to manually install jQuery and jQuery UI in portal_css and portal_javascripts [voineali #4196]

1.0 - (2011-03-16)

  • Initial release [voineali #2773]