{"_id":"@eeacms/volto-migrate-tool","name":"@eeacms/volto-migrate-tool","dist-tags":{"latest":"0.1.0"},"versions":{"0.1.0":{"name":"@eeacms/volto-migrate-tool","version":"0.1.0","description":"(EEA) Provides primitives for data connectivity to volto-plotlycharts and other blocks","main":"src/index.js","author":{"name":"European Environment Agency: IDM2 A-Team"},"license":"MIT","homepage":"https://github.com/eea/volto-migrate-tool","keywords":["volto-addon","volto-block","volto-migrate-tool","volto","plone","react"],"dependencies":{},"devDependencies":{"@cypress/code-coverage":"^3.10.0","@plone/scripts":"*","babel-plugin-transform-class-properties":"^6.24.1","cypress":"13.1.0","cypress-fail-fast":"^5.0.1","dotenv":"^16.3.2","husky":"^8.0.3","lint-staged":"^14.0.1","md5":"^2.3.0"},"repository":{"type":"git","url":"git+https://github.com/eea/volto-migrate-tool.git"},"bugs":{"url":"https://github.com/eea/volto-migrate-tool/issues"},"lint-staged":{"src/**/*.{js,jsx,ts,tsx,json}":["make lint-fix","make prettier-fix"],"src/**/*.{jsx}":["make i18n"],"theme/**/*.{css,less}":["make stylelint-fix"],"src/**/*.{css,less}":["make stylelint-fix"],"theme/**/*.overrides":["make stylelint-fix"],"src/**/*.overrides":["make stylelint-fix"]},"scripts":{"release":"release-it","release-major-beta":"release-it major --preRelease=beta","release-beta":"release-it --preRelease=beta","bootstrap":"npm install -g ejs; npm link ejs; node bootstrap","test":"make test","test:fix":"make test-update","pre-commit":"yarn stylelint:fix && yarn prettier:fix && yarn lint:fix","stylelint":"make stylelint","stylelint:overrides":"make stylelint-overrides","stylelint:fix":"make stylelint-fix","prettier":"make prettier","prettier:fix":"make prettier-fix","lint":"make lint","lint:fix":"make lint-fix","i18n":"make i18n","cypress:run":"make cypress-run","cypress:open":"make cypress-open","prepare":"husky install"},"gitHead":"edc1418645d2152cf44cb8e9613e6dd63d8d1e87","_id":"@eeacms/volto-migrate-tool@0.1.0","_nodeVersion":"16.20.2","_npmVersion":"8.19.4","dist":{"integrity":"sha512-cOUXVtbYH+XfzBfOj/RcU3nKHwe3J0o21pGaeXOViuiJaU2ifI91pXERTMLdm4As9ZvhfrS6pJQFV+qjqcUxyw==","shasum":"6ee46e707a2d57fc3cd08ff3d758e1adc161dd6a","tarball":"https://registry.npmjs.org/@eeacms/volto-migrate-tool/-/volto-migrate-tool-0.1.0.tgz","fileCount":28,"unpackedSize":111448,"signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEYCIQDOIoLgat7Oa85BwCzk3Ye3ht0rcKAwdAE+E7NkWNRnGgIhAIhpH9cq5Lts0gIJpGPKQg2rUxu0+hxlgosBNd9/RT/N"}]},"_npmUser":{"name":"eea-jenkins","email":"eea-jenkins-npm@roles.eea.eionet.europa.eu"},"directories":{},"_npmOperationalInternal":{"host":"s3://npm-registry-packages","tmp":"tmp/volto-migrate-tool_0.1.0_1718952496157_0.09582371645604648"},"_hasShrinkwrap":false}},"time":{"created":"2024-06-21T06:48:16.055Z","0.1.0":"2024-06-21T06:48:16.355Z","modified":"2024-06-21T06:48:16.752Z"},"maintainers":[{"name":"nileshgulia1","email":"nileshgulia@gmail.com"},{"name":"valentinab25","email":"valentina.balan@gmail.com"},{"name":"demarant","email":"eeacms@gmail.com"},{"name":"avoinea","email":"contact@avoinea.com"},{"name":"tiberiuichim","email":"tiberiu.ichim@eaudeweb.ro"},{"name":"zotya","email":"zoltan.szabo@eaudeweb.ro"},{"name":"alecghica","email":"alecghica@gmail.com"},{"name":"eea-jenkins","email":"eea-jenkins-npm@roles.eea.eionet.europa.eu"},{"name":"razvan.miu","email":"miu.razvan28@gmail.com"},{"name":"ichimdav","email":"ichim.david@gmail.com"}],"description":"(EEA) Provides primitives for data connectivity to volto-plotlycharts and other blocks","homepage":"https://github.com/eea/volto-migrate-tool","keywords":["volto-addon","volto-block","volto-migrate-tool","volto","plone","react"],"repository":{"type":"git","url":"git+https://github.com/eea/volto-migrate-tool.git"},"author":{"name":"European Environment Agency: IDM2 A-Team"},"bugs":{"url":"https://github.com/eea/volto-migrate-tool/issues"},"license":"MIT","readme":"# Data-connected Volto components\n\n[![Releases](https://img.shields.io/github/v/release/eea/volto-migrate-tool)](https://github.com/eea/volto-migrate-tool/releases)\n\n[![Pipeline](https://ci.eionet.europa.eu/buildStatus/icon?job=volto-addons%2Fvolto-migrate-tool%2Fmaster&subject=master)](https://ci.eionet.europa.eu/view/Github/job/volto-addons/job/volto-migrate-tool/job/master/display/redirect)\n[![Lines of Code](https://sonarqube.eea.europa.eu/api/project_badges/measure?project=volto-migrate-tool-master&metric=ncloc)](https://sonarqube.eea.europa.eu/dashboard?id=volto-migrate-tool-master)\n[![Coverage](https://sonarqube.eea.europa.eu/api/project_badges/measure?project=volto-migrate-tool-master&metric=coverage)](https://sonarqube.eea.europa.eu/dashboard?id=volto-migrate-tool-master)\n[![Bugs](https://sonarqube.eea.europa.eu/api/project_badges/measure?project=volto-migrate-tool-master&metric=bugs)](https://sonarqube.eea.europa.eu/dashboard?id=volto-migrate-tool-master)\n[![Duplicated Lines (%)](https://sonarqube.eea.europa.eu/api/project_badges/measure?project=volto-migrate-tool-master&metric=duplicated_lines_density)](https://sonarqube.eea.europa.eu/dashboard?id=volto-migrate-tool-master)\n\n[![Pipeline](https://ci.eionet.europa.eu/buildStatus/icon?job=volto-addons%2Fvolto-migrate-tool%2Fdevelop&subject=develop)](https://ci.eionet.europa.eu/view/Github/job/volto-addons/job/volto-migrate-tool/job/develop/display/redirect)\n[![Lines of Code](https://sonarqube.eea.europa.eu/api/project_badges/measure?project=volto-migrate-tool-develop&metric=ncloc)](https://sonarqube.eea.europa.eu/dashboard?id=volto-migrate-tool-develop)\n[![Coverage](https://sonarqube.eea.europa.eu/api/project_badges/measure?project=volto-migrate-tool-develop&metric=coverage)](https://sonarqube.eea.europa.eu/dashboard?id=volto-migrate-tool-develop)\n[![Bugs](https://sonarqube.eea.europa.eu/api/project_badges/measure?project=volto-migrate-tool-develop&metric=bugs)](https://sonarqube.eea.europa.eu/dashboard?id=volto-migrate-tool-develop)\n[![Duplicated Lines (%)](https://sonarqube.eea.europa.eu/api/project_badges/measure?project=volto-migrate-tool-develop&metric=duplicated_lines_density)](https://sonarqube.eea.europa.eu/dashboard?id=volto-migrate-tool-develop)\n\n[Volto](https://github.com/plone/volto) add-on\n\nvolto-migrate-tool is an addon that has various blocks and utilities to provide \"data-connected\" components in volto websites.\n\n**This add-on requires `eea.docker.plonesaas@5.2.4-66`.**\n\n## Concepts and terminology\n\n- data-connected component: is a component or a block that displays information coming from outside sources; For example a table would get its data from a REST api server (discodata.eea.europa.eu) and show it as desired\n- connector / provider: is a dexterity content type characterized by:\n  - title: acts as the name of the connector and it used by plone to compose the short-name or route through which we can query the connector\n  - endpoint_url: the url to the REST api server (ex. discodata.eea.europa.eu)\n  - sql_query: the sql that will be executed on the REST api server\n  - parameters (optional): a list of parameters (keys) through which we can filter the data fetched by the connector\n  - required_parameters (optional): a list of required parameters which if not satisfed the connector will return empty data\n  - collate (optional)\n  - readme (optional)\n- query string parameters or parameters from url; Ex. `https://frontend/home?db_version=latest&p=1&nrOfHits=10`\n- data query are another type of parameters that are composed internally using redux\n- connected data parameters is the global object that stores data queries; we will get more into this later\n\n## Workflow\n\nA data-connected component will use a connector through which it gets its data.\n\n[eea.api.dataconnector](https://github.com/eea/eea.api.dataconnector) is a plone add-on that expose an api through which a connector will run a query and fetch some data.\n\n### How to use eea.api.dataconnector\n\neea.api.dataconnector expose `/@connector-data` through which we can make `POST` requests to get the data of a connector. We can pass some data to the request:\n\n- `form` - parameters from url\n- `data_query` - parameters from connected data parameters\n\nYou can use this api only over connectors.\n\nExample:\n\nLet's say that we have the following connector added into connectors folder:\n\n```\ntitle: Forests per capita\nendpoint_url: https://discodata.eea.europa.eu/sql\nsql_query: SELECT * FROM [FISE].[latest].[v_cnct_forest_per_capita]\nparameters: ['NUTS_CODE']\n```\n\nThe route to the connector will be `https://frontend/connectors/forests-per-capita`\n\nBy making the following request:\n\n```\n> curl --location --request POST 'http://backend/Plone/++api++/connectors/forests-per-capita/@connector-data'\n```\n\nthe sql_query will be executed on specified endpoint_url (discodata) and after data is fetched the response will look like:\n\n```JSON\n{\n   \"@id\": \"https://backend/Plone/connectors/forests-per-capita\",\n   \"data\": {\n      \"metadata\": {...},\n      \"results\": {\n         \"COUNTRY\": [\"Albania\", ...],\n         \"Forest per capita\": [0.39, ...],\n         \"NUTS_CODE\": [\"AL\", ...],\n      }\n   }\n}\n```\n\nWe can filter the data in two ways:\n\n- By updating the sql before it is executed on discodata by adding where statements - this requires parameters property to be specified on the connector\n- By filtering the data after it is fetched from discodata through for loops\n\nObs: both filtering are done on the backend. The response will always contain the filtered data.\n\nIn this case we can filter by `NUTS_CODE`. The request will look like:\n\n```\n> curl --location --request POST 'http://backend/Plone/++api++/connectors/forests-per-capita/@connector-data' \\\n--header 'Content-Type: application/json' \\\n--data '{\n    \"form\": {\n        \"NUTS_CODE\": \"FR\"\n    }\n}'\n```\n\nBecause `NUTS_CODE` is specified in the parameters list of the connector the query will be modified and will look like this:\n\n```sql\nSELECT * FROM FISE.latest.v_cnct_forest_per_capita WHERE NUTS_CODE = 'FR'\n```\n\nSo we get the data from discodata already filtered and the response will look like:\n\n```JSON\n{\n   \"@id\": \"https://backend/Plone/connectors/forests-per-capita\",\n   \"data\": {\n      \"metadata\": {...},\n      \"results\": {\n         \"COUNTRY\": [\"France\"],\n         \"Forest per capita\": [0.23],\n         \"NUTS_CODE\": [\"FR\"],\n      }\n   }\n}\n```\n\nIf we don't have `NUTS_CODE` specified in the parameters list we will still get the data filtered by `NUTS_CODE` but after it is fetched from discodata. So by adding keys to parameters list can dramatically decrease the data usage.\n\nIf we want to set the same parameter but through data_query the request will look like this:\n\n```\n> curl --location --request POST 'http://backend/Plone/++api++/connectors/forests-per-capita/@connector-data' \\\n--header 'Content-Type: application/json' \\\n--data '{\n   \"data_query\": [\n      \"i\": \"NUTS_CODE\"\n      \"o\": \"plone.app.querystring.operation.selection.any\"\n      \"v\": [\"FR\"]\n   ]\n}'\n```\n\nvolto-migrate-tool offers 4 hooks through which a data-connected component can make requests to a connector:\n\n```\nconnectToProviderData\nconnectToProviderDataUnfiltered\nconnectToMultipleProviders\nconnectToMultipleProvidersUnfiltered\n```\n\nObs: a data-connected component needs to specify a provider_url (the path to the connector) to the hook used to fetch the data. We will get more into this later.\n\n## Operators\n\n### Parameters from url - `form`\n\nAvailable operators:\n\n```\neq          - equal\nne          - not equal\nlike\nnot like\nin\nnin         - not in\ngt          - greater than\ngte         - greater than equal\nlt          - lower than\nlte         - lower than equal\n```\n\nTo specify an operator to a parameter from url you need to use this structure:\n\n```\nsome/path?parameter[operation]=value\n```\n\nFor example if on the homepage we have a data-connected component that uses `/connectors/forests-per-capita` as provider and we want to filter it by multiple `NUTS_CODE` we can set a url parameter using the 'in' operator like:\n\n```\nhttps://frontend/?NUTS_CODE[in]=RO,FR\n```\n\n### Parameters from data_query - `connected_data_parameters`\n\nTo be continued...\n\n## Pagination\n\nTo be continued...\n\n## Features\n\nThere are a few data-connected blocks in this add-on:\n\n### SimpleDataTable\n\nA data-connected table which allows pagination and filtering. It can be customized by implementing a template.\n\n`TODO: tutorial on how to customize and demo`\n\n### DataQueryFilter\n\nA dropdown data-connected component that uses a provider to create a filter for it by a parameter selected from block configuration.\n\n`TODO: demo`\n\n### DottedTableChart\n\n### CustomConnectedBlock\n\n## Usage (for developers)\n\n### How to connect a block to a provider?\n\nAs we said we have 4 hooks, 2 that uses filters and 2 that doesn't use filters. They require you to pass a getConfig function that returns an object. That object needs to have some specific data.\n\nHere is the configuration needed to be passed to each hook:\n\n1. `connectToProviderData`\n\n```javascript\n{\n   provider_url: 'path/to/provider', // mandatory\n   pagination: { // optional\n      enabled: true,\n      itemsPerPage: 5\n   }\n}\n// Obs: provider_url is mandatory and pagination is optional. If pagination is not configured then connectToProviderData will run as if pagination is disabled.\n```\n\n2. `connectToProviderDataUnfiltered`\n\n```javascript\n{\n  provider_url: 'path/to/provider'; // mandatory\n}\n```\n\n3. `connectToMultipleProviders`\n\n```javascript\n{\n   provider: [ // mandatory\n      {\n         provider_url: 'path/to/provider', // mandatory\n         name: 'some name', // optional\n         title: 'some title', // optional\n         data_query: [...some_data_query] // optional\n         has_data_query_by_context: true // optional\n         has_data_query_by_provider: true // optional\n      }\n   ]\n}\n```\n\n4. `connectToMultipleProvidersUnfiltered`\n\n```javascript\n{\n  provider: [\n    // mandatory\n    {\n      provider_url: 'path/to/provider', // mandatory\n      name: 'some name', // optional\n      title: 'some title', // optional\n    },\n  ];\n}\n```\n\nConnecting to multiple providers doesn't allow pagination.\n\nHere is an example on how to use `connectToProviderData`:\n\n```javascript\nimport React from 'react';\nimport { compose } from 'redux';\nimport { connectToProviderData } from '@eeacms/volto-migrate-tool/hocs';\n\n...\n\nconst View = props => {\n   ...\n   return <YourComponents />\n}\n\nexport default compose(\n  connectToProviderData((props) => {\n    return {\n      provider_url: props.data?.provider_url,\n    };\n  }),\n)(View);\n\n```\n\n## Getting started\n\n### Try volto-migrate-tool with Docker\n\n      git clone https://github.com/eea/volto-migrate-tool.git\n      cd volto-migrate-tool\n      make\n      make start\n\nGo to http://localhost:3000\n\n### Add volto-migrate-tool to your Volto project\n\n1. Make sure you have a [Plone backend](https://plone.org/download) up-and-running at http://localhost:8080/Plone\n\n   ```Bash\n   docker compose up backend\n   ```\n\n1. Start Volto frontend\n\n- If you already have a volto project, just update `package.json`:\n\n  ```JSON\n  \"addons\": [\n      \"@eeacms/volto-migrate-tool\"\n  ],\n\n  \"dependencies\": {\n      \"@eeacms/volto-migrate-tool\": \"*\"\n  }\n  ```\n\n- If not, create one:\n\n  ```\n  npm install -g yo @plone/generator-volto\n  yo @plone/volto my-volto-project --canary --addon @eeacms/volto-migrate-tool\n  cd my-volto-project\n  ```\n\n1. Install new add-ons and restart Volto:\n\n   ```\n   yarn\n   yarn start\n   ```\n\n1. Go to http://localhost:3000\n\n1. Happy editing!\n\n## Release\n\nSee [RELEASE.md](https://github.com/eea/volto-migrate-tool/blob/master/RELEASE.md).\n\n## How to contribute\n\nSee [DEVELOP.md](https://github.com/eea/volto-migrate-tool/blob/master/DEVELOP.md).\n\n## Copyright and license\n\nThe Initial Owner of the Original Code is European Environment Agency (EEA).\nAll Rights Reserved.\n\nSee [LICENSE.md](https://github.com/eea/volto-migrate-tool/blob/master/LICENSE.md) for details.\n\n## Funding\n\n[European Environment Agency (EU)](http://eea.europa.eu)\n","readmeFilename":"README.md"}