mirror of
https://github.com/opencultureconsulting/openrefine-client.git
synced 2025-04-06 00:01:03 +02:00
Compare commits
No commits in common. "master" and "v0.3.9" have entirely different histories.
4
.gitignore
vendored
4
.gitignore
vendored
@ -4,6 +4,4 @@ dist
|
||||
.*
|
||||
openrefine_client.egg-info
|
||||
refine.spec
|
||||
openrefine-*
|
||||
openrefine-client_*
|
||||
tests-cli.log
|
||||
README.html
|
||||
|
||||
149
README.md
149
README.md
@ -1,20 +1,20 @@
|
||||
# OpenRefine Python Client with extended command line interface (⌨️ for 💎)
|
||||
# OpenRefine Python Client with extended command line interface
|
||||
|
||||
[](https://www.codacy.com/gh/opencultureconsulting/openrefine-client/dashboard) [](https://hub.docker.com/r/felixlohmeier/openrefine-client/) [](https://pypi.org/project/openrefine-client/) [](https://mybinder.org/v2/gh/felixlohmeier/openrefineder/master)
|
||||
[](https://www.codacy.com/app/felixlohmeier/openrefine-client?utm_source=github.com&utm_medium=referral&utm_content=opencultureconsulting/openrefine-client&utm_campaign=Badge_Grade) [](https://hub.docker.com/r/felixlohmeier/openrefine-client/) [](https://pypi.org/project/openrefine-client/) [](https://mybinder.org/v2/gh/felixlohmeier/openrefineder/master)
|
||||
|
||||
The [OpenRefine Python Client from PaulMakepeace](https://github.com/PaulMakepeace/refine-client-py) provides a library for communicating with an [OpenRefine](http://openrefine.org) server.
|
||||
This fork extends the command line interface (CLI) and is distributed as a convenient one-file-executable (Windows, Linux, macOS).
|
||||
It is also available via Docker Hub, PyPI and Binder.
|
||||
|
||||
works with OpenRefine 2.7, 2.8, 3.0, 3.1, 3.2, 3.3, 3.4, 3.4.1, 3.5.0
|
||||
works with OpenRefine 2.7, 2.8, 3.0, 3.1, 3.2
|
||||
|
||||
## Download
|
||||
|
||||
One-file-executables:
|
||||
|
||||
- Windows: [openrefine-client_0-3-10_windows.exe](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.10/openrefine-client_0-3-10_windows.exe) (~5 MB)
|
||||
- macOS: [openrefine-client_0-3-10_macos](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.10/openrefine-client_0-3-10_macos) (~5 MB)
|
||||
- Linux: [openrefine-client_0-3-10_linux](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.10/openrefine-client_0-3-10_linux) (~5 MB)
|
||||
- Windows: [openrefine-client_0-3-9_windows.exe](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.9/openrefine-client_0-3-9_windows.exe) (~5 MB)
|
||||
- macOS: [openrefine-client_0-3-9_macos](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.9/openrefine-client_0-3-9_macos) (~5 MB)
|
||||
- Linux: [openrefine-client_0-3-9_linux](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.9/openrefine-client_0-3-9_linux) (~5 MB)
|
||||
|
||||
For [Docker](#docker) containers, native [Python](#python) installation and free [Binder](#binder) on-demand server see the corresponding chapters below.
|
||||
|
||||
@ -57,13 +57,13 @@ To use the client:
|
||||
- macOS:
|
||||
|
||||
```sh
|
||||
chmod +x openrefine-client_0-3-10_macos
|
||||
chmod +x openrefine-client_0-3-9_macos
|
||||
```
|
||||
|
||||
- Linux:
|
||||
|
||||
```sh
|
||||
chmod +x openrefine-client_0-3-10_linux
|
||||
chmod +x openrefine-client_0-3-9_linux
|
||||
```
|
||||
|
||||
3. Execute the file.
|
||||
@ -71,19 +71,19 @@ To use the client:
|
||||
- Windows:
|
||||
|
||||
```sh
|
||||
.\openrefine-client_0-3-10_windows.exe
|
||||
.\openrefine-client_0-3-9_windows.exe
|
||||
```
|
||||
|
||||
- macOS:
|
||||
|
||||
```sh
|
||||
./openrefine-client_0-3-10_macos
|
||||
./openrefine-client_0-3-9_macos
|
||||
```
|
||||
|
||||
- Linux:
|
||||
|
||||
```sh
|
||||
./openrefine-client_0-3-10_linux
|
||||
./openrefine-client_0-3-9_linux
|
||||
```
|
||||
|
||||
Using tab completion and command history is highly recommended:
|
||||
@ -102,25 +102,25 @@ Download example data (`--download`) and create project from file (`--create`):
|
||||
- Windows:
|
||||
|
||||
```sh
|
||||
.\openrefine-client_0-3-10_windows.exe --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
.\openrefine-client_0-3-10_windows.exe --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
.\openrefine-client_0-3-10_windows.exe --create duplicates.csv
|
||||
.\openrefine-client_0-3-9_windows.exe --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
.\openrefine-client_0-3-9_windows.exe --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
.\openrefine-client_0-3-9_windows.exe --create duplicates.csv
|
||||
```
|
||||
|
||||
- macOS:
|
||||
|
||||
```sh
|
||||
./openrefine-client_0-3-10_macos --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
./openrefine-client_0-3-10_macos --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
./openrefine-client_0-3-10_macos --create duplicates.csv
|
||||
./openrefine-client_0-3-9_macos --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
./openrefine-client_0-3-9_macos --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
./openrefine-client_0-3-9_macos --create duplicates.csv
|
||||
```
|
||||
|
||||
- Linux:
|
||||
|
||||
```sh
|
||||
./openrefine-client_0-3-10_linux --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
./openrefine-client_0-3-10_linux --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
./openrefine-client_0-3-10_linux --create duplicates.csv
|
||||
./openrefine-client_0-3-9_linux --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
./openrefine-client_0-3-9_linux --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
./openrefine-client_0-3-9_linux --create duplicates.csv
|
||||
```
|
||||
|
||||
Other commands:
|
||||
@ -147,7 +147,7 @@ If your OpenRefine server is running somewhere else then you may set hostname an
|
||||
- set host: `-H example.com`
|
||||
- set port: `-P 80`
|
||||
|
||||
### Templating
|
||||
### Advanced Templating
|
||||
|
||||
The OpenRefine [Templating](https://github.com/OpenRefine/OpenRefine/wiki/Export-As-YAML) supports exporting data in any text format (i.e. to construct JSON or XML).
|
||||
The graphical user interface offers four input fields:
|
||||
@ -220,35 +220,10 @@ There is another option to use the value in the first column instead:
|
||||
Because our project "advanced" contains duplicates in the first column "email" this command will overwrite files (e.g. `advanced_melanie.white@example2.edu.json`).
|
||||
When using this option, the first column should contain unique identifiers.
|
||||
|
||||
### Append data to an existing project
|
||||
|
||||
OpenRefine does not support appending rows to an existing project.
|
||||
As long as the [feature request](https://github.com/OpenRefine/OpenRefine/issues/715) is not yet implemented, you can use the openrefine-client to script a workaround:
|
||||
|
||||
1. export existing project as csv
|
||||
2. put old and new data into a zip archive
|
||||
3. create new project by importing the zip archive
|
||||
|
||||
Here is an example that replaces the existing project:
|
||||
|
||||
```
|
||||
openrefine-client --export myproject --output old.csv
|
||||
openrefine-client --delete myproject
|
||||
zip combined.zip old.csv new.csv
|
||||
openrefine-client --create combined.zip --format csv --projectName myproject
|
||||
```
|
||||
|
||||
Note that the project id will change.
|
||||
If you want to distinguish between old and new data, you can use the additional flag includeFileSources:
|
||||
|
||||
```
|
||||
openrefine-client --create combined.zip --format csv --projectName myproject --includeFileSources true
|
||||
```
|
||||
|
||||
### See also
|
||||
|
||||
- Linux Bash script to run OpenRefine in batch mode (import, transform, export): [openrefine-batch](https://github.com/opencultureconsulting/openrefine-batch)
|
||||
- [Jupyter notebook demonstrating usage in Linux Bash](https://nbviewer.jupyter.org/github/felixlohmeier/openrefineder/blob/master/notebooks/openrefine-client-bash.ipynb)
|
||||
- [Jupyter notebook demonstrating usage in Linux Bash](https://nbviewer.jupyter.org/github/felixlohmeier/openrefineder/blob/master/openrefine-client-bash.ipynb)
|
||||
- Use case [HOS-MetadataTransformations](https://github.com/subhh/HOS-MetadataTransformations): Automated workflow for harvesting, transforming and indexing of metadata using metha, OpenRefine and Solr. Part of the Hamburg Open Science "Schaufenster" software stack.
|
||||
- Use case [Data processing of ILS data to facilitate a new discovery layer for the German Literature Archive (DLA)](https://doi.org/10.5281/zenodo.2678113): Custom data processing pipeline based on Pandas (a Python library) and OpenRefine.
|
||||
|
||||
@ -257,7 +232,7 @@ openrefine-client --create combined.zip --format csv --projectName myproject --i
|
||||
[felixlohmeier/openrefine-client](https://hub.docker.com/r/felixlohmeier/openrefine-client/) [](https://hub.docker.com/r/felixlohmeier/openrefine-client/)
|
||||
|
||||
```sh
|
||||
docker pull felixlohmeier/openrefine-client:v0.3.10
|
||||
docker pull felixlohmeier/openrefine-client:v0.3.9
|
||||
```
|
||||
|
||||
### Option 1: Dockerized client
|
||||
@ -265,7 +240,7 @@ docker pull felixlohmeier/openrefine-client:v0.3.10
|
||||
Run client and mount current directory as workspace:
|
||||
|
||||
```sh
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9
|
||||
```
|
||||
|
||||
The docker option `--network=host` allows you to connect to a local or remote OpenRefine via the host network:
|
||||
@ -273,13 +248,13 @@ The docker option `--network=host` allows you to connect to a local or remote Op
|
||||
- list projects on default URL (http://localhost:3333)
|
||||
|
||||
```sh
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 --list
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 --list
|
||||
```
|
||||
|
||||
- list projects on a remote server (http://example.com)
|
||||
|
||||
```sh
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H example.com -P 80 --list
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H example.com -P 80 --list
|
||||
```
|
||||
|
||||
Usage: same commands as explained above (see [Basic Commands](#basic-commands) and [Advanced Templating](#advanced-templating))
|
||||
@ -297,22 +272,22 @@ Run openrefine-client linked to a dockerized OpenRefine ([felixlohmeier/openrefi
|
||||
2. Run server (will be available at http://localhost:3333)
|
||||
|
||||
```sh
|
||||
docker run -d -p 3333:3333 --network=openrefine --name=openrefine-server felixlohmeier/openrefine:3.5.0
|
||||
docker run -d -p 3333:3333 --network=openrefine --name=openrefine-server felixlohmeier/openrefine:3.2
|
||||
```
|
||||
|
||||
3. Run client with some [basic commands](#basic-commands): 1. download example files, 2. create project from file, 3. list projects, 4. show metadata, 5. export to terminal, 6. apply transformation rules (deduplication), 7. export again to terminal, 8. export to xls file and 9. delete project
|
||||
|
||||
```sh
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H openrefine-server --create duplicates.csv
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H openrefine-server --list
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H openrefine-server --info "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H openrefine-server --export "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H openrefine-server --apply duplicates-deletion.json "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H openrefine-server --export "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H openrefine-server --export --output=deduped.xls "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H openrefine-server --delete "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H openrefine-server --create duplicates.csv
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H openrefine-server --list
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H openrefine-server --info "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H openrefine-server --export "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H openrefine-server --apply duplicates-deletion.json "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H openrefine-server --export "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H openrefine-server --export --output=deduped.xls "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H openrefine-server --delete "duplicates"
|
||||
```
|
||||
|
||||
4. Stop and delete server:
|
||||
@ -337,7 +312,7 @@ Customize OpenRefine server:
|
||||
- Example for [allocating more memory](https://github.com/OpenRefine/OpenRefine/wiki/FAQ#out-of-memory-errors---feels-slow---could-not-reserve-enough-space-for-object-heap) to OpenRefine with additional option `-m 4G`
|
||||
|
||||
```sh
|
||||
docker run -d -p 3333:3333 --network=openrefine --name=openrefine-server felixlohmeier/openrefine:3.5.0 -i 0.0.0.0 -d /data -m 4G
|
||||
docker run -d -p 3333:3333 --network=openrefine --name=openrefine-server felixlohmeier/openrefine:3.2 -i 0.0.0.0 -d /data -m 4G
|
||||
```
|
||||
|
||||
- The OpenRefine version is defined by the docker tag.
|
||||
@ -624,8 +599,8 @@ See also:
|
||||
- free to use on-demand server with Jupyter notebook, OpenRefine and Bash
|
||||
- no registration needed, will start within a few minutes
|
||||
- [restricted](https://mybinder.readthedocs.io/en/latest/faq.html#how-much-memory-am-i-given-when-using-binder) to 2 GB RAM and server will be deleted after 10 minutes of inactivity
|
||||
- [bash_kernel demo notebook](https://nbviewer.jupyter.org/github/felixlohmeier/openrefineder/blob/master/notebooks/openrefine-client-bash.ipynb) for using the openrefine-client in a Linux Bash environment [](https://mybinder.org/v2/gh/felixlohmeier/openrefineder/master?urlpath=/tree/notebooks/openrefine-client-bash.ipynb)
|
||||
- [python2 demo notebook](https://nbviewer.jupyter.org/github/felixlohmeier/openrefineder/blob/master/notebooks/openrefine-client-python.ipynb) for using the openrefine-client in a Python 2 environment [](https://mybinder.org/v2/gh/felixlohmeier/openrefineder/master?urlpath=/tree/notebooks/openrefine-client-python.ipynb)
|
||||
- [bash_kernel demo notebook](https://nbviewer.jupyter.org/github/felixlohmeier/openrefineder/blob/master/openrefine-client-bash.ipynb) for using the openrefine-client in a Linux Bash environment [](https://mybinder.org/v2/gh/felixlohmeier/openrefineder/master?urlpath=/tree/openrefine-client-bash.ipynb)
|
||||
- [python2 demo notebook](https://nbviewer.jupyter.org/github/felixlohmeier/openrefineder/blob/master/openrefine-client-python.ipynb) for using the openrefine-client in a Python 2 environment [](https://mybinder.org/v2/gh/felixlohmeier/openrefineder/master?urlpath=/tree/openrefine-client-python.ipynb)
|
||||
|
||||
## Development
|
||||
|
||||
@ -651,54 +626,39 @@ The Python client library includes several unit tests.
|
||||
|
||||
There is also a script that uses docker images to run the unit tests with different versions of OpenRefine.
|
||||
|
||||
- run tests on all OpenRefine versions (from 2.0 up to 3.5.0)
|
||||
- run tests on all OpenRefine versions (from 2.0 up to 3.2)
|
||||
|
||||
```sh
|
||||
./tests.sh -a
|
||||
```
|
||||
|
||||
- run tests on tag 3.5.0
|
||||
- run tests on tag 3.2
|
||||
|
||||
```sh
|
||||
./tests.sh -t 3.5.0
|
||||
./tests.sh -t 3.2
|
||||
```
|
||||
|
||||
- run tests on tag 3.5.0 interactively (pause before and after tests)
|
||||
- run tests on tag 3.2 interactively (pause before and after tests)
|
||||
|
||||
```sh
|
||||
./tests.sh -t 3.5.0 -i
|
||||
./tests.sh -t 3.2 -i
|
||||
```
|
||||
|
||||
- run tests on tags 3.5.0 and 2.7
|
||||
- run tests on tags 3.2 and 2.7
|
||||
|
||||
```sh
|
||||
./tests.sh -t 3.5.0 -t 2.7
|
||||
```
|
||||
|
||||
For Linux there are also functional tests for all command line options.
|
||||
|
||||
- run all functional tests on OpenRefine 3.5.0
|
||||
|
||||
```sh
|
||||
./tests-cli.sh 3.5.0
|
||||
```
|
||||
|
||||
- run all functional tests on OpenRefine 3.5.0 with one-file-executable
|
||||
|
||||
```sh
|
||||
./tests-cli.sh 3.5.0 openrefine-client_0-3-7_linux
|
||||
./tests.sh -t 3.2 -t 2.7
|
||||
```
|
||||
|
||||
### Distributing
|
||||
|
||||
Note to myself: When releasing a new version...
|
||||
|
||||
1. Run functional tests
|
||||
1. Run tests
|
||||
|
||||
```sh
|
||||
for v in 2.7 2.8 3.0 3.1 3.2 3.3 3.4 3.4.1 3.5.0; do
|
||||
./tests-cli.sh $v
|
||||
done
|
||||
./tests.sh -a
|
||||
jupyter notebook tests/cli_python2.ipynb
|
||||
```
|
||||
|
||||
2. Make final changes in Git
|
||||
@ -725,12 +685,11 @@ Note to myself: When releasing a new version...
|
||||
python2 -m PyInstaller --onefile refine.py --hidden-import google.refine.__main__
|
||||
```
|
||||
|
||||
4. Run functional tests with Linux executable
|
||||
4. Run test with Linux executable
|
||||
|
||||
```sh
|
||||
for v in 2.7 2.8 3.0 3.1 3.2 3.3 3.4 3.4.1 3.5.0; do
|
||||
./tests-cli.sh $v openrefine-client_0-3-7_linux
|
||||
done
|
||||
./tests.sh -a
|
||||
jupyter notebook tests/cli_bash.ipynb
|
||||
```
|
||||
|
||||
5. Create release in GitHub
|
||||
@ -752,7 +711,7 @@ Note to myself: When releasing a new version...
|
||||
8. Bump openrefine-client version in related projects
|
||||
|
||||
- openrefine-batch: [openrefine-batch.sh](https://github.com/opencultureconsulting/openrefine-batch/blob/master/openrefine-batch.sh#L7) and [openrefine-batch-docker.sh](https://github.com/opencultureconsulting/openrefine-batch/blob/master/openrefine-batch-docker.sh)
|
||||
- openrefineder: [postBuild](https://github.com/felixlohmeier/openrefineder/blob/master/binder/postBuild)
|
||||
- openrefineder: [postBuild](https://github.com/felixlohmeier/openrefineder/blob/master/postBuild)
|
||||
|
||||
## Credits
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
FROM alpine:3.11
|
||||
FROM alpine:latest
|
||||
LABEL maintainer="felixlohmeier@opencultureconsulting.com"
|
||||
# The OpenRefine Python Client Library from PaulMakepeace provides an interface to communicating with an OpenRefine server. This fork extends the command line interface (CLI) and supports communication between docker containers.
|
||||
# Source: https://github.com/opencultureconsulting/openrefine-client
|
||||
|
||||
@ -41,6 +41,7 @@ def apply(project_id, history_file):
|
||||
print('File %s has been successfully applied to project %s' %
|
||||
(history_file, project_id))
|
||||
|
||||
|
||||
def create(project_file,
|
||||
project_format=None,
|
||||
columnWidths=None,
|
||||
@ -164,19 +165,17 @@ def download(url, output_file=None):
|
||||
def export(project_id, encoding=None, output_file=None, export_format=None):
|
||||
"""Dump a project to stdout or file."""
|
||||
project = refine.RefineProject(project_id)
|
||||
if not export_format:
|
||||
export_format = 'tsv'
|
||||
if not output_file:
|
||||
if not export_format:
|
||||
export_format = 'tsv'
|
||||
if export_format in ['csv', 'tsv', 'txt']:
|
||||
encoding = 'UTF-8'
|
||||
sys.stdout.write(project.export(
|
||||
export_format=export_format, encoding=encoding).read())
|
||||
else:
|
||||
ext = os.path.splitext(output_file)[1][1:]
|
||||
if ext and not export_format:
|
||||
if ext:
|
||||
export_format = ext.lower()
|
||||
if not export_format:
|
||||
export_format = 'tsv'
|
||||
if export_format in ['csv', 'tsv', 'txt']:
|
||||
encoding = 'UTF-8'
|
||||
with open(output_file, 'wb') as f:
|
||||
@ -200,7 +199,7 @@ def info(project_id):
|
||||
project_model = refine.RefineProject(project_id).get_models()
|
||||
columns = [c['name'] for c in project_model['columnModel']['columns']]
|
||||
for (i, v) in enumerate(columns, start=1):
|
||||
print(u'{0:>20}: {1}'.format(u'column ' + str(i).zfill(3), v).encode('utf-8'))
|
||||
print(u'{0:>20}: {1}'.format(u'column ' + str(i).zfill(3), v))
|
||||
else:
|
||||
print('Error: No project found with id %s.\n'
|
||||
'Check existing projects with command --list' % (project_id))
|
||||
@ -216,7 +215,7 @@ def ls():
|
||||
projects.sort(key=lambda v: date_to_epoch(v[1]['modified']), reverse=True)
|
||||
if projects:
|
||||
for project_id, project_info in projects:
|
||||
print(u'{0:>14}: {1}'.format(project_id, project_info['name']).encode('utf-8'))
|
||||
print(u'{0:>14}: {1}'.format(project_id, project_info['name']))
|
||||
else:
|
||||
print('Error: No projects found')
|
||||
|
||||
|
||||
@ -53,18 +53,6 @@ class RefineServer(object):
|
||||
server = self.url()
|
||||
self.server = server[:-1] if server.endswith('/') else server
|
||||
self.__version = None # see version @property below
|
||||
self.token = None # CSRF token introduced in OpenRefine 3.3
|
||||
self.get_csrf_token()
|
||||
|
||||
def get_csrf_token(self):
|
||||
"""Return csrf token."""
|
||||
try:
|
||||
url = self.server + '/command/core/get-csrf-token'
|
||||
response = json.loads(urllib2.urlopen(url).read())
|
||||
self.token = response['token']
|
||||
return self.token
|
||||
except:
|
||||
pass # fail silently to not disturb usage of OpenRefine <3.3
|
||||
|
||||
def urlopen(self, command, data=None, params=None, project_id=None):
|
||||
"""Open a Refine URL and with optional query params and POST data.
|
||||
@ -85,9 +73,6 @@ class RefineServer(object):
|
||||
data['project'] = project_id
|
||||
else:
|
||||
params['project'] = project_id
|
||||
# be lazy and send the token for each API call (even when not needed)
|
||||
if self.token:
|
||||
params['csrf_token'] = self.token
|
||||
if params:
|
||||
url += '?' + urllib.urlencode(params)
|
||||
req = urllib2.Request(url)
|
||||
@ -97,8 +82,7 @@ class RefineServer(object):
|
||||
try:
|
||||
response = urllib2.urlopen(req)
|
||||
except urllib2.HTTPError as e:
|
||||
raise Exception('HTTP %d "%s" for %s\n\t%s' %
|
||||
(e.code, e.msg, e.geturl(), data))
|
||||
raise Exception('HTTP %d "%s" for %s\n\t%s' % (e.code, e.msg, e.geturl(), data))
|
||||
except urllib2.URLError as e:
|
||||
raise urllib2.URLError(
|
||||
'%s for %s. No Refine server reachable/running; ENV set?' %
|
||||
@ -114,10 +98,6 @@ class RefineServer(object):
|
||||
"""Open a Refine URL, optionally POST data, and return parsed JSON."""
|
||||
response = json.loads(self.urlopen(*args, **kwargs).read())
|
||||
if 'code' in response and response['code'] not in ('ok', 'pending'):
|
||||
if 'Missing or invalid csrf_token parameter' == response['message']:
|
||||
self.get_csrf_token()
|
||||
response = json.loads(self.urlopen(*args, **kwargs).read())
|
||||
return response
|
||||
error_message = ('server ' + response['code'] + ': ' +
|
||||
response.get('message', response.get('stack', response)))
|
||||
raise Exception(error_message)
|
||||
@ -418,10 +398,7 @@ class RefineProject:
|
||||
for i, column in enumerate(column_model['columns']):
|
||||
name = column['name']
|
||||
self.column_order[name] = i
|
||||
try:
|
||||
column_index[name] = column['cellIndex']
|
||||
except KeyError:
|
||||
column_index[name] = i
|
||||
column_index[name] = column['cellIndex']
|
||||
self.key_column = column_model['keyColumnName']
|
||||
self.has_records = response['recordModel'].get('hasRecords', False)
|
||||
self.rows_response_factory = RowsResponseFactory(column_index)
|
||||
|
||||
2
setup.py
2
setup.py
@ -25,7 +25,7 @@ def read(filename):
|
||||
return open(os.path.join(os.path.dirname(__file__), filename)).read()
|
||||
|
||||
setup(name='openrefine-client',
|
||||
version='0.3.10',
|
||||
version='0.3.9',
|
||||
description=('The OpenRefine Python Client Library provides an '
|
||||
'interface to communicating with an OpenRefine server. '
|
||||
'This fork extends the command line interface (CLI).'),
|
||||
|
||||
123
tests-cli.sh
123
tests-cli.sh
@ -1,123 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Script for running functional tests against the CLI
|
||||
|
||||
# Copyright (c) 2011 Paul Makepeace, Real Programmers. All rights reserved.
|
||||
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU General Public License as published by
|
||||
# the Free Software Foundation, either version 3 of the License, or
|
||||
# (at your option) any later version.
|
||||
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU General Public License for more details.
|
||||
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with this program. If not, see <http://www.gnu.org/licenses/>
|
||||
|
||||
# ================================== CONFIG ================================== #
|
||||
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
|
||||
port=3334
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
version="${1}"
|
||||
else
|
||||
version="3.2"
|
||||
fi
|
||||
refine="openrefine-${version}/refine"
|
||||
|
||||
if [[ ${2} ]]; then
|
||||
client="$(readlink -e "${2}")"
|
||||
else
|
||||
client="python2 $(readlink -e refine.py)"
|
||||
fi
|
||||
cmd="${client} -H localhost -P ${port}"
|
||||
|
||||
if [[ ${3} ]]; then
|
||||
filename="${3%%.*}"
|
||||
else
|
||||
filename=""
|
||||
fi
|
||||
cmd="${client} -H localhost -P ${port}"
|
||||
|
||||
# =============================== REQUIREMENTS =============================== #
|
||||
|
||||
# check existence of java and cURL
|
||||
if [[ -z "$(command -v java 2> /dev/null)" ]] ; then
|
||||
echo 1>&2 "ERROR: OpenRefine requires JAVA runtime environment (jre)" \
|
||||
"https://openjdk.java.net/install/"
|
||||
exit 1
|
||||
fi
|
||||
if [[ -z "$(command -v curl 2> /dev/null)" ]] ; then
|
||||
echo 1>&2 "ERROR: This shell script requires cURL" \
|
||||
"https://curl.haxx.se/download.html"
|
||||
exit 1
|
||||
fi
|
||||
# download OpenRefine
|
||||
if [[ -z "$(readlink -e "${refine}")" ]]; then
|
||||
echo "Download OpenRefine ${version}..."
|
||||
mkdir -p "$(dirname "${refine}")"
|
||||
curl -L --output openrefine.tar.gz \
|
||||
"https://github.com/OpenRefine/OpenRefine/releases/download/${version}/openrefine-linux-${version}.tar.gz"
|
||||
echo "Install OpenRefine ${version} in subdirectory $(dirname "${refine}")..."
|
||||
tar -xzf openrefine.tar.gz -C "$(dirname "${refine}")" --strip 1 --totals
|
||||
rm -f openrefine.tar.gz
|
||||
# do not try to open OpenRefine in browser
|
||||
sed -i '$ a JAVA_OPTIONS=-Drefine.headless=true' \
|
||||
"$(dirname "${refine}")"/refine.ini
|
||||
# set autosave period from 5 minutes to 25 hours
|
||||
sed -i 's/#REFINE_AUTOSAVE_PERIOD=60/REFINE_AUTOSAVE_PERIOD=1500/' \
|
||||
"$(dirname "${refine}")"/refine.ini
|
||||
echo
|
||||
fi
|
||||
|
||||
# ================================== SETUP =================================== #
|
||||
|
||||
dir="$(readlink -f "tests/tmp")"
|
||||
mkdir -p "${dir}"
|
||||
rm -f tests-cli.log
|
||||
|
||||
echo "start OpenRefine ${version}..."
|
||||
${refine} -v warn -p ${port} -d "${dir}" &>> tests-cli.log &
|
||||
pid_server=${!}
|
||||
timeout 30s bash -c "until curl -s 'http://localhost:3334' \
|
||||
| cat | grep -q -o 'OpenRefine' ; do sleep 1; done" \
|
||||
|| error "starting OpenRefine server failed!"
|
||||
echo
|
||||
|
||||
# ================================== TESTS =================================== #
|
||||
|
||||
echo "running tests, please wait..."
|
||||
tests=()
|
||||
results=()
|
||||
for t in tests/*${filename}*.sh; do
|
||||
tests+=("${t}")
|
||||
echo "======================= ${t} =======================" &>> tests-cli.log
|
||||
bash "${t}" "${cmd}" "${version}" &>> tests-cli.log
|
||||
results+=(${?})
|
||||
done
|
||||
echo
|
||||
|
||||
# ================================= TEARDOWN ================================= #
|
||||
|
||||
echo "cleanup..."
|
||||
{ kill -9 "${pid_server}" && wait "${pid_server}"; } 2>/dev/null
|
||||
rm -rf "${dir}"
|
||||
echo
|
||||
|
||||
# ================================= SUMMARY ================================== #
|
||||
|
||||
printf "%s\t%s\n" "code" "test"
|
||||
printf "%s\t%s\n" "----" "----------------"
|
||||
for i in "${!tests[@]}"; do
|
||||
printf "%s\t%s\n" "${results[$i]}" "${tests[$i]}"
|
||||
done
|
||||
echo
|
||||
if [[ " ${results[*]} " =~ [1-9] ]]; then
|
||||
echo "failed tests! check tests-cli.log for debugging"; echo
|
||||
else
|
||||
echo "all tests passed!"; echo
|
||||
fi
|
||||
12
tests.sh
12
tests.sh
@ -17,8 +17,8 @@
|
||||
# along with this program. If not, see <http://www.gnu.org/licenses/>
|
||||
|
||||
# defaults:
|
||||
all=(3.5.0 3.4.1 3.4 3.3 3.2-java12 3.2-java11 3.2-java10 3.2-java9 3.2 3.1-java9 3.1 3.0-java9 3.0 2.8-java9 2.8 2.8-java7 2.7 2.7-java7 2.5-java7 2.5-java6 2.1-java6 2.0-java6)
|
||||
main=(3.5.0 3.4.1 3.4 3.3 3.2 3.1 3.0 2.8 2.7 2.5-java6 2.1-java6 2.0-java6)
|
||||
all=(3.2-java12 3.2-java11 3.2-java10 3.2-java9 3.2 3.1-java9 3.1 3.0-java9 3.0 2.8-java9 2.8 2.8-java7 2.7 2.7-java7 2.5-java7 2.5-java6 2.1-java6 2.0-java6)
|
||||
main=(3.2 3.1 3.0 2.8 2.7 2.5-java6 2.1-java6 2.0-java6)
|
||||
interactively=false
|
||||
port="3333"
|
||||
|
||||
@ -31,10 +31,10 @@ Script for running tests with different OpenRefine and Java versions.
|
||||
It uses docker images from https://hub.docker.com/r/felixlohmeier/openrefine.
|
||||
|
||||
Examples:
|
||||
./tests.sh -a # run tests on all OpenRefine versions (from 2.0 up to 3.5.0)
|
||||
./tests.sh -t 3.5.0 # run tests on tag 3.5.0
|
||||
./tests.sh -t 3.5.0 -i # run tests on tag 3.5.0 interactively (pause before and after tests)
|
||||
./tests.sh -t 3.5.0 -t 2.7 # run tests on tags 3.5.0 and 2.7
|
||||
./tests.sh -a # run tests on all OpenRefine versions (from 2.0 up to 3.2)
|
||||
./tests.sh -t 3.2 # run tests on tag 3.2
|
||||
./tests.sh -t 3.2 -i # run tests on tag 3.2 interactively (pause before and after tests)
|
||||
./tests.sh -t 3.2 -t 2.7 # run tests on tags 3.2 and 2.7
|
||||
|
||||
Advanced:
|
||||
./tests.sh -j # run tests on all OpenRefine versions and each with all supported Java versions (requires a lot of docker images to be downloaded!)
|
||||
|
||||
@ -1,57 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/column-addition",
|
||||
"engineConfig": {
|
||||
"mode": "row-based"
|
||||
},
|
||||
"newColumnName": "apply",
|
||||
"columnInsertIndex": 2,
|
||||
"baseColumnName": "b",
|
||||
"expression": "grel:value.replace('2','⛲')",
|
||||
"onError": "set-to-blank"
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b apply c
|
||||
1 2 ⛲ 3
|
||||
0 0 0 0
|
||||
$ \ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,57 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/column-addition",
|
||||
"engineConfig": {
|
||||
"mode": "row-based"
|
||||
},
|
||||
"newColumnName": "apply",
|
||||
"columnInsertIndex": 2,
|
||||
"baseColumnName": "b",
|
||||
"expression": "grel:value.replace('2','TEST')",
|
||||
"onError": "set-to-blank"
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b apply c
|
||||
1 2 TEST 3
|
||||
0 0 0 0
|
||||
$ \ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
819
tests/cli_bash.ipynb
Normal file
819
tests/cli_bash.ipynb
Normal file
@ -0,0 +1,819 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Test executable in a Linux Bash environment"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Install\n",
|
||||
"\n",
|
||||
"This notebook requires a [Bash kernel](https://github.com/takluyver/bash_kernel) environment and an OpenRefine server running at http://127.0.0.1:3333."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 49,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"/tmp/20190822_013937\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"workspace=$(date +%Y%m%d_%H%M%S)\n",
|
||||
"mkdir -p /tmp/$workspace\n",
|
||||
"cp -r data /tmp/$workspace\n",
|
||||
"cd /tmp/$workspace && pwd"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 50,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.8/openrefine-client_0-3-8_linux:\n",
|
||||
"2019-08-22 01:39:40 ERROR 404: Not Found.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"wget -nv https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.8/openrefine-client_0-3-8_linux -O openrefine-client\n",
|
||||
"chmod +x openrefine-client"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## README.MD"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Download"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 51,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Download to file duplicates.csv complete\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --download \"https://git.io/fj5hF\" --output=duplicates.csv"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Create"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 52,
|
||||
"metadata": {
|
||||
"scrolled": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"id: 2334935475634\n",
|
||||
"rows: 10\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --create duplicates.csv"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### List"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 53,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 2334935475634: duplicates\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Info"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 54,
|
||||
"metadata": {
|
||||
"scrolled": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" id: 2334935475634\n",
|
||||
" url: http://127.0.0.1:3333/project?project=2334935475634\n",
|
||||
" name: duplicates\n",
|
||||
" modified: 2019-08-21T23:40:30Z\n",
|
||||
" created: 2019-08-21T23:40:30Z\n",
|
||||
" rowCount: 10\n",
|
||||
"importOptionMetadata: [{u'storeEmptyStrings': True, u'fileSource': u'duplicates.csv', u'storeBlankRows': True, u'encoding': u'', u'projectName': u'duplicates', u'processQuotes': True, u'separator': u',', u'trimStrings': False, u'limit': -1, u'storeBlankCellsAsNulls': True, u'guessCellValueTypes': False, u'includeFileSources': False}]\n",
|
||||
" column 001: email\n",
|
||||
" column 002: name\n",
|
||||
" column 003: state\n",
|
||||
" column 004: gender\n",
|
||||
" column 005: purchase\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --info \"duplicates\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Export"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 55,
|
||||
"metadata": {
|
||||
"scrolled": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"email\tname\tstate\tgender\tpurchase\n",
|
||||
"danny.baron@example1.com\tDanny Baron\tCA\tM\tTV\n",
|
||||
"melanie.white@example2.edu\tMelanie White\tNC\tF\tiPhone\n",
|
||||
"danny.baron@example1.com\tD. Baron\tCA\tM\tWinter jacket\n",
|
||||
"ben.tyler@example3.org\tBen Tyler\tNV\tM\tFlashlight\n",
|
||||
"arthur.duff@example4.com\tArthur Duff\tOR\tM\tDining table\n",
|
||||
"danny.baron@example1.com\tDaniel Baron\tCA\tM\tBike\n",
|
||||
"jean.griffith@example5.org\tJean Griffith\tWA\tF\tPower drill\n",
|
||||
"melanie.white@example2.edu\tMelanie White\tNC\tF\tiPad\n",
|
||||
"ben.morisson@example6.org\tBen Morisson\tFL\tM\tAmplifier\n",
|
||||
"arthur.duff@example4.com\tArthur Duff\tOR\tM\tNight table\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --export \"duplicates\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Apply"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 56,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Download to file duplicates-deletion.json complete\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --download \"https://git.io/fj5ju\" --output=duplicates-deletion.json"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 57,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"File duplicates-deletion.json has been successfully applied to project 2334935475634\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --apply duplicates-deletion.json \"duplicates\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 58,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"email\tcount\tname\tstate\tgender\tpurchase\n",
|
||||
"arthur.duff@example4.com\t2\tArthur Duff\tOR\tM\tDining table\n",
|
||||
"ben.morisson@example6.org\t1\tBen Morisson\tFL\tM\tAmplifier\n",
|
||||
"ben.tyler@example3.org\t1\tBen Tyler\tNV\tM\tFlashlight\n",
|
||||
"danny.baron@example1.com\t3\tDanny Baron\tCA\tM\tTV\n",
|
||||
"jean.griffith@example5.org\t1\tJean Griffith\tWA\tF\tPower drill\n",
|
||||
"melanie.white@example2.edu\t2\tMelanie White\tNC\tF\tiPhone\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --export \"duplicates\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Export XLS"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 59,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Export to file deduped.xls complete\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --export \"duplicates\" --output deduped.xls"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Delete"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 60,
|
||||
"metadata": {
|
||||
"scrolled": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Project 2334935475634 has been successfully deleted\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --delete \"duplicates\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Templating"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 61,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"id: 1633409429491\n",
|
||||
"rows: 10\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --create duplicates.csv --projectName=advanced"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 62,
|
||||
"metadata": {
|
||||
"scrolled": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"{ \"events\" : [\n",
|
||||
" { \"name\" : \"Melanie White\", \"purchase\" : \"iPhone\" },\n",
|
||||
" { \"name\" : \"Jean Griffith\", \"purchase\" : \"Power drill\" },\n",
|
||||
" { \"name\" : \"Melanie White\", \"purchase\" : \"iPad\" }\n",
|
||||
"] }"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --export \"advanced\" \\\n",
|
||||
"--prefix='{ \"events\" : [\n",
|
||||
"' \\\n",
|
||||
"--template=' { \"name\" : {{jsonize(cells[\"name\"].value)}}, \"purchase\" : {{jsonize(cells[\"purchase\"].value)}} }' \\\n",
|
||||
"--rowSeparator=',\n",
|
||||
"' \\\n",
|
||||
"--suffix='\n",
|
||||
"] }' \\\n",
|
||||
"--filterQuery='^F$' \\\n",
|
||||
"--filterColumn='gender'"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 63,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Export to files complete. Last file: advanced_3.json\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --export \"advanced\" \\\n",
|
||||
"--prefix='{ \"events\" : [\n",
|
||||
"' \\\n",
|
||||
"--template=' { \"name\" : {{jsonize(cells[\"name\"].value)}}, \"purchase\" : {{jsonize(cells[\"purchase\"].value)}} }' \\\n",
|
||||
"--rowSeparator=',\n",
|
||||
"' \\\n",
|
||||
"--suffix='\n",
|
||||
"] }' \\\n",
|
||||
"--filterQuery='^F$' \\\n",
|
||||
"--filterColumn='gender' \\\n",
|
||||
"--output=advanced.json \\\n",
|
||||
"--splitToFiles=true"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 64,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Export to files complete. Last file: advanced_melanie.white@example2.edu.json\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --export \"advanced\" \\\n",
|
||||
"--prefix='{ \"events\" : [\n",
|
||||
"' \\\n",
|
||||
"--template=' { \"name\" : {{jsonize(cells[\"name\"].value)}}, \"purchase\" : {{jsonize(cells[\"purchase\"].value)}} }' \\\n",
|
||||
"--rowSeparator=',\n",
|
||||
"' \\\n",
|
||||
"--suffix='\n",
|
||||
"] }' \\\n",
|
||||
"--filterQuery='^F$' \\\n",
|
||||
"--filterColumn='gender' \\\n",
|
||||
"--output=advanced.json \\\n",
|
||||
"--splitToFiles=true \\\n",
|
||||
"--suffixById=true"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 65,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"advanced_1.json \u001b[0m\u001b[38;5;33mdata\u001b[0m\n",
|
||||
"advanced_2.json deduped.xls\n",
|
||||
"advanced_3.json duplicates.csv\n",
|
||||
"advanced_jean.griffith@example5.org.json duplicates-deletion.json\n",
|
||||
"advanced_melanie.white@example2.edu.json \u001b[38;5;40mopenrefine-client\u001b[0m\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"ls"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Delete"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 66,
|
||||
"metadata": {
|
||||
"scrolled": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Project 1633409429491 has been successfully deleted\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --delete \"advanced\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Unicode"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### fruits"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 67,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"id: 2280962953279\n",
|
||||
"rows: 5\n",
|
||||
" id: 2280962953279\n",
|
||||
" url: http://127.0.0.1:3333/project?project=2280962953279\n",
|
||||
" name: evil-fruits\n",
|
||||
" modified: 2019-08-21T23:40:43Z\n",
|
||||
" created: 2019-08-21T23:40:43Z\n",
|
||||
" rowCount: 5\n",
|
||||
"importOptionMetadata: [{u'storeEmptyStrings': True, u'fileSource': u'data/cli/evil-fruits.tsv', u'storeBlankRows': True, u'encoding': u'', u'projectName': u'evil-fruits', u'processQuotes': True, u'limit': -1, u'trimStrings': False, u'storeBlankCellsAsNulls': True, u'guessCellValueTypes': False, u'includeFileSources': False}]\n",
|
||||
" column 001: 🔣\n",
|
||||
" column 002: code\n",
|
||||
" column 003: meaning\n",
|
||||
"🔣\tcode\tmeaning\n",
|
||||
"🍇\t1F347\tGRAPES\n",
|
||||
"🍉\t1F349\tWATERMELON\n",
|
||||
"🍒\t1F352\tCHERRIES\n",
|
||||
"🍓\t1F353\tSTRAWBERRY\n",
|
||||
"🍍\t1F34D\tPINEAPPLE\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --create data/cli/evil-fruits.tsv\n",
|
||||
"./openrefine-client --info \"evil-fruits\"\n",
|
||||
"./openrefine-client --export \"evil-fruits\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 68,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Export to file emojis.csv complete\n",
|
||||
"🔣,code,meaning\n",
|
||||
"🍇,1F347,GRAPES\n",
|
||||
"🍉,1F349,WATERMELON\n",
|
||||
"🍒,1F352,CHERRIES\n",
|
||||
"🍓,1F353,STRAWBERRY\n",
|
||||
"🍍,1F34D,PINEAPPLE\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --export \"evil-fruits\" --output emojis.csv\n",
|
||||
"cat emojis.csv"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 69,
|
||||
"metadata": {
|
||||
"scrolled": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"{ \"emojis\" : [\n",
|
||||
" { \"symbol\" : \"🍇\", \"meaning\" : \"GRAPES\" },\n",
|
||||
" { \"symbol\" : \"🍉\", \"meaning\" : \"WATERMELON\" },\n",
|
||||
" { \"symbol\" : \"🍍\", \"meaning\" : \"PINEAPPLE\" }\n",
|
||||
"] }"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --export \"evil-fruits\" \\\n",
|
||||
"--prefix='{ \"emojis\" : [\n",
|
||||
"' \\\n",
|
||||
"--template=' { \"symbol\" : {{jsonize(with(row.columnNames[0],cn,cells[cn].value))}}, \"meaning\" : {{jsonize(cells[\"meaning\"].value)}} }' \\\n",
|
||||
"--rowSeparator=',\n",
|
||||
"' \\\n",
|
||||
"--suffix='\n",
|
||||
"] }' \\\n",
|
||||
"--filterQuery='^1F34' \\\n",
|
||||
"--filterColumn='code'"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 70,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Export to files complete. Last file: trái cây_3.json\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --export \"evil-fruits\" \\\n",
|
||||
"--prefix='{ \"emojis\" : [\n",
|
||||
"' \\\n",
|
||||
"--template=' { \"symbol\" : {{jsonize(with(row.columnNames[0],cn,cells[cn].value))}}, \"meaning\" : {{jsonize(cells[\"meaning\"].value)}} }' \\\n",
|
||||
"--rowSeparator=',\n",
|
||||
"' \\\n",
|
||||
"--suffix='\n",
|
||||
"] }' \\\n",
|
||||
"--filterQuery='^1F34' \\\n",
|
||||
"--filterColumn='code' \\\n",
|
||||
"--output='trái cây.json' \\\n",
|
||||
"--splitToFiles=true"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 71,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Export to files complete. Last file: trái cây_🍍.json\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --export \"evil-fruits\" \\\n",
|
||||
"--prefix='{ \"emojis\" : [\n",
|
||||
"' \\\n",
|
||||
"--template=' { \"symbol\" : {{jsonize(with(row.columnNames[0],cn,cells[cn].value))}}, \"meaning\" : {{jsonize(cells[\"meaning\"].value)}} }' \\\n",
|
||||
"--rowSeparator=',\n",
|
||||
"' \\\n",
|
||||
"--suffix='\n",
|
||||
"] }' \\\n",
|
||||
"--filterQuery='^1F34' \\\n",
|
||||
"--filterColumn='code' \\\n",
|
||||
"--output='trái cây.json' \\\n",
|
||||
"--splitToFiles=true \\\n",
|
||||
"--suffixById=true"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 72,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" advanced_1.json emojis.csv\n",
|
||||
" advanced_2.json \u001b[0m\u001b[38;5;40mopenrefine-client\u001b[0m\n",
|
||||
" advanced_3.json 'trái cây_1.json'\n",
|
||||
" advanced_jean.griffith@example5.org.json 'trái cây_2.json'\n",
|
||||
" advanced_melanie.white@example2.edu.json 'trái cây_3.json'\n",
|
||||
" \u001b[38;5;33mdata\u001b[0m 'trái cây_🍇.json'\n",
|
||||
" deduped.xls 'trái cây_🍉.json'\n",
|
||||
" duplicates.csv 'trái cây_🍍.json'\n",
|
||||
" duplicates-deletion.json\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"ls"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 73,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Project 2280962953279 has been successfully deleted\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --delete \"evil-fruits\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### emoji-data"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 74,
|
||||
"metadata": {
|
||||
"scrolled": true
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"id: 2019865211741\n",
|
||||
"rows: 20\n",
|
||||
" id: 2019865211741\n",
|
||||
" url: http://127.0.0.1:3333/project?project=2019865211741\n",
|
||||
" name: dữ liệu biểu tượng cảm xúc\n",
|
||||
" modified: 2019-08-21T23:41:06Z\n",
|
||||
" created: 2019-08-21T23:41:06Z\n",
|
||||
" rowCount: 20\n",
|
||||
"importOptionMetadata: [{u'storeEmptyStrings': True, u'fileSource': u'data/cli/d\\u1eef li\\u1ec7u bi\\u1ec3u t\\u01b0\\u1ee3ng c\\u1ea3m x\\xfac.txt', u'storeBlankRows': True, u'encoding': u'', u'projectName': u'd\\u1eef li\\u1ec7u bi\\u1ec3u t\\u01b0\\u1ee3ng c\\u1ea3m x\\xfac', u'processQuotes': True, u'skipDataLines': 34, u'limit': 20, u'trimStrings': False, u'storeBlankCellsAsNulls': True, u'guessCellValueTypes': False, u'includeFileSources': False, u'headerLines': 0}]\n",
|
||||
" column 001: Column 1\n",
|
||||
" column 002: Column 2\n",
|
||||
" column 003: Column 3\n",
|
||||
" column 004: Column 4\n",
|
||||
" column 005: Column 5\n",
|
||||
" column 006: Column 6\n",
|
||||
"Column 1\tColumn 2\tColumn 3\tColumn 4\tColumn 5\tColumn 6\n",
|
||||
"00A9 ;\ttext ;\tL1 ;\tnone ;\tj\t# V1.1 (©) COPYRIGHT SIGN\n",
|
||||
"00AE ;\ttext ;\tL1 ;\tnone ;\tj\t# V1.1 (®) REGISTERED SIGN\n",
|
||||
"203C ;\ttext ;\tL1 ;\tnone ;\ta j\t# V1.1 (‼) DOUBLE EXCLAMATION MARK\n",
|
||||
"2049 ;\ttext ;\tL1 ;\tnone ;\ta j\t# V3.0 (⁉) EXCLAMATION QUESTION MARK\n",
|
||||
"2122 ;\ttext ;\tL1 ;\tnone ;\tj\t# V1.1 (™) TRADE MARK SIGN\n",
|
||||
"2139 ;\ttext ;\tL1 ;\tnone ;\tj\t# V3.0 (ℹ) INFORMATION SOURCE\n",
|
||||
"2194 ;\ttext ;\tL1 ;\tnone ;\tz j\t# V1.1 (↔) LEFT RIGHT ARROW\n",
|
||||
"2195 ;\ttext ;\tL1 ;\tnone ;\tz j\t# V1.1 (↕) UP DOWN ARROW\n",
|
||||
"2196 ;\ttext ;\tL1 ;\tnone ;\tj\t# V1.1 (↖) NORTH WEST ARROW\n",
|
||||
"2197 ;\ttext ;\tL1 ;\tnone ;\tj\t# V1.1 (↗) NORTH EAST ARROW\n",
|
||||
"2198 ;\ttext ;\tL1 ;\tnone ;\tj\t# V1.1 (↘) SOUTH EAST ARROW\n",
|
||||
"2199 ;\ttext ;\tL1 ;\tnone ;\tj\t# V1.1 (↙) SOUTH WEST ARROW\n",
|
||||
"21A9 ;\ttext ;\tL1 ;\tnone ;\tj\t# V1.1 (↩) LEFTWARDS ARROW WITH HOOK\n",
|
||||
"21AA ;\ttext ;\tL1 ;\tnone ;\tj\t# V1.1 (↪) RIGHTWARDS ARROW WITH HOOK\n",
|
||||
"231A ;\temoji ;\tL1 ;\tnone ;\tj\t# V1.1 (⌚) WATCH\n",
|
||||
"231B ;\temoji ;\tL1 ;\tnone ;\tj\t# V1.1 (⌛) HOURGLASS\n",
|
||||
"2328 ;\ttext ;\tL2 ;\tnone ;\tx\t# V1.1 (⌨) KEYBOARD\n",
|
||||
"23CF ;\ttext ;\tL2 ;\tnone ;\tx\t# V4.0 (⏏) EJECT SYMBOL\n",
|
||||
"23E9 ;\temoji ;\tL1 ;\tnone ;\tj w\t# V6.0 (⏩) BLACK RIGHT-POINTING DOUBLE TRIANGLE\n",
|
||||
"23EA ;\temoji ;\tL1 ;\tnone ;\tj w\t# V6.0 (⏪) BLACK LEFT-POINTING DOUBLE TRIANGLE\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --create \"data/cli/dữ liệu biểu tượng cảm xúc.txt\" \\\n",
|
||||
"--format=tsv \\\n",
|
||||
"--headerLines=0 \\\n",
|
||||
"--skipDataLines=34 \\\n",
|
||||
"--limit=20\n",
|
||||
"./openrefine-client --info \"dữ liệu biểu tượng cảm xúc\"\n",
|
||||
"./openrefine-client --export \"dữ liệu biểu tượng cảm xúc\""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 75,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
" 2019865211741: dữ liệu biểu tượng cảm xúc\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --list"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Delete"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 76,
|
||||
"metadata": {
|
||||
"scrolled": false
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Project 2019865211741 has been successfully deleted\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"./openrefine-client --delete \"dữ liệu biểu tượng cảm xúc\""
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Bash",
|
||||
"language": "bash",
|
||||
"name": "bash"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": "shell",
|
||||
"file_extension": ".sh",
|
||||
"mimetype": "text/x-sh",
|
||||
"name": "bash"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
2824
tests/cli_python2.ipynb
Normal file
2824
tests/cli_python2.ipynb
Normal file
File diff suppressed because it is too large
Load Diff
@ -1,41 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}-utf8.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
ä,é,ß
|
||||
$,\,'
|
||||
DATA
|
||||
iconv -f UTF-8 -t ISO-8859-1 "tmp/${t}/${t}-utf8.csv" > "tmp/${t}/${t}.csv"
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
ä é ß
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --encoding "ISO-8859-1"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
01,02,03
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
1 2 3
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --guessCellValueTypes "true"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,41 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
Column 1 Column 2 Column 3
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --headerLines "0"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,39 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --ignoreLines "1"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,39 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --limit "2"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,41 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,"2,0",3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c Column 4
|
||||
1 2 0 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
# OpenRefine 4.x fails without manually set headerLines
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --processQuotes "false" --headerLines 1
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,45 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
version="${2}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "${version:0:1}" = "2" ]]; then
|
||||
echo "projectTags were introduced in OpenRefine 3.0"
|
||||
exit 200
|
||||
else
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
tags: [u'beta', u'client1']
|
||||
DATA
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --projectTags "beta" --projectTags "client1"
|
||||
${cmd} --info "${t}" | grep ' tags: ' > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a;b;c
|
||||
1;2;3
|
||||
0;0;0
|
||||
$;\;'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --separator ";"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,39 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --skipDataLines "1"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,58 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "b",
|
||||
"expression": "grel:isNull(value)",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 false 3
|
||||
0 false 0
|
||||
$ false '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --storeBlankCellsAsNulls "false"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,39 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
,,
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --storeBlankRows "false"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,FLAG IN HOLE
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
⌨ code meaning
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --projectName "${t} biểu tượng cảm xúc ⛲"
|
||||
${cmd} --export "${t} biểu tượng cảm xúc ⛲" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,55 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.json"
|
||||
{
|
||||
"rows":[
|
||||
{
|
||||
"a":1,
|
||||
"b":2,
|
||||
"c":3
|
||||
},
|
||||
{
|
||||
"a":0,
|
||||
"b":0,
|
||||
"c":0
|
||||
},
|
||||
{
|
||||
"a":"$",
|
||||
"b":"\\",
|
||||
"c":"\""
|
||||
}
|
||||
]
|
||||
}
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
_ - a _ - b _ - c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ """"
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.json" --recordPath "_" --recordPath "rows" --recordPath "_"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,52 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.json"
|
||||
[
|
||||
{
|
||||
"a": 1,
|
||||
"b": 2,
|
||||
"c": 3
|
||||
},
|
||||
{
|
||||
"a": "",
|
||||
"b": "",
|
||||
"c": ""
|
||||
},
|
||||
{
|
||||
"a": "$",
|
||||
"b": "\\",
|
||||
"c": "\""
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
_ - a _ - b _ - c
|
||||
1 2 3
|
||||
$ \ """"
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.json" --storeEmptyStrings "false"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,62 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
version="${2}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.json"
|
||||
[
|
||||
{
|
||||
"a": 1,
|
||||
"b": 2,
|
||||
"c": 3
|
||||
},
|
||||
{
|
||||
"a": "0",
|
||||
"b": " 0",
|
||||
"c": "0 "
|
||||
},
|
||||
{
|
||||
"a": "$",
|
||||
"b": "\\",
|
||||
"c": "\""
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "${version:0:1}" = "2" || "${version}" = "3.0" || "${version}" = "3.1" || "${version}" = "3.2" || "${version}" = "3.3" ]]; then
|
||||
echo "trimStrings option does not work in OpenRefine <=3.3"
|
||||
echo "https://github.com/OpenRefine/OpenRefine/issues/2409"
|
||||
exit 200
|
||||
else
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
_ - a _ - b _ - c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ """"
|
||||
DATA
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.json" --trimStrings "true"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,53 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.json"
|
||||
[
|
||||
{
|
||||
"⌨": "⛲",
|
||||
"code": "1F347",
|
||||
"meaning": "FOUNTAIN"
|
||||
},
|
||||
{
|
||||
"⌨": "⛳",
|
||||
"code": "1F349",
|
||||
"meaning": "FLAG IN HOLE"
|
||||
},
|
||||
{
|
||||
"⌨": "⛵",
|
||||
"code": "1F352",
|
||||
"meaning": "SAILBOAT"
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
_ - ⌨ _ - code _ - meaning
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.json"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,53 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.json"
|
||||
[
|
||||
{
|
||||
"a": 1,
|
||||
"b": 2,
|
||||
"c": 3
|
||||
},
|
||||
{
|
||||
"a": 0,
|
||||
"b": 0,
|
||||
"c": 0
|
||||
},
|
||||
{
|
||||
"a": "$",
|
||||
"b": "\\",
|
||||
"c": "\""
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
_ - a _ - b _ - c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ """"
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.json"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
version="${2}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cp "data/example.ods" "tmp/${t}/${t}.ods"
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "${version}" = "2.7" ]]; then
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
⌨ code meaning Column Column 5 Column 6 Column 7 Column 8
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
else
|
||||
#TODO
|
||||
echo "https://github.com/opencultureconsulting/openrefine-client/issues/4"
|
||||
exit 200
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.ods" --sheets 1
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,48 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
version="${2}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cp "data/example.ods" "tmp/${t}/${t}.ods"
|
||||
#a b c
|
||||
#1 2 3
|
||||
#0 0 0
|
||||
#$ \ '
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "${version}" = "2.7" ]]; then
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c Column Column 5 Column 6 Column 7 Column 8
|
||||
1.0 2.0 3.0
|
||||
0.0 0.0 0.0
|
||||
$ \ '
|
||||
DATA
|
||||
else
|
||||
#TODO
|
||||
echo "https://github.com/opencultureconsulting/openrefine-client/issues/4"
|
||||
exit 200
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.ods"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.tsv"
|
||||
⌨ code meaning
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,FLAG IN HOLE
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.tsv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.csv"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.csv"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.tsv"
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.tsv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.csv"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.csv"
|
||||
@ -1,80 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.txt"
|
||||
1 2 3
|
||||
mon tue wed
|
||||
$2 $300 $1
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "1",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
},
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "2",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
},
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "3",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
1 2 3
|
||||
mon tue wed
|
||||
$2 $300 $1
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.txt" --columnWidths "6" --columnWidths "6" --columnWidths "6" --headerLines "1"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,81 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.txt"
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "Column 1",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
},
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "Column 2",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
},
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "Column 3",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
Column 1 Column 2 Column 3
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.txt" --columnWidths "6" --columnWidths "6" --columnWidths "60"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,81 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.txt"
|
||||
1 2 3
|
||||
mon tue wed
|
||||
$2 $300 $1
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "Column 1",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
},
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "Column 2",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
},
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "Column 3",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
Column 1 Column 2 Column 3
|
||||
1 2 3
|
||||
mon tue wed
|
||||
$2 $300 $1
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.txt" --columnWidths "6" --columnWidths "6" --columnWidths "6"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,39 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.txt"
|
||||
mon tue wed
|
||||
$2 $300 $1
|
||||
thu fri sat
|
||||
$70 $20 $50
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
Column 1 Column 2
|
||||
mon tue wed $2 $300 $1
|
||||
thu fri sat $70 $20 $50
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.txt" --linesPerRow "2"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,39 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.txt"
|
||||
1 2 3
|
||||
mon tue wed
|
||||
$2 $300 $1
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
Column 1
|
||||
1 2 3
|
||||
mon tue wed
|
||||
$2 $300 $1
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.txt"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
version="${2}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cp "data/example.xls" "tmp/${t}/${t}.xls"
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "${version}" = "2.7" ]]; then
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
⌨ code meaning
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
else
|
||||
#TODO
|
||||
echo "https://github.com/opencultureconsulting/openrefine-client/issues/4"
|
||||
exit 200
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.xls" --sheets 1
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,48 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
version="${2}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cp "data/example.xls" "tmp/${t}/${t}.xls"
|
||||
#a b c
|
||||
#1 2 3
|
||||
#0 0 0
|
||||
#$ \ '
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "${version}" = "2.7" ]]; then
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1.0 2.0 3.0
|
||||
0.0 0.0 0.0
|
||||
$ \ '
|
||||
DATA
|
||||
else
|
||||
#TODO
|
||||
echo "https://github.com/opencultureconsulting/openrefine-client/issues/4"
|
||||
exit 200
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.xls"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
version="${2}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cp "data/example.xlsx" "tmp/${t}/${t}.xlsx"
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "${version}" = "2.7" ]]; then
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
⌨ code meaning
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
else
|
||||
#TODO
|
||||
echo "https://github.com/opencultureconsulting/openrefine-client/issues/4"
|
||||
exit 200
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.xlsx" --sheets 1
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,48 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
version="${2}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cp "data/example.xlsx" "tmp/${t}/${t}.xlsx"
|
||||
#a b c
|
||||
#1 2 3
|
||||
#0 0 0
|
||||
#$ \ '
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "${version}" = "2.7" ]]; then
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1.0 2.0 3.0
|
||||
0.0 0.0 0.0
|
||||
$ \ '
|
||||
DATA
|
||||
else
|
||||
#TODO
|
||||
echo "https://github.com/opencultureconsulting/openrefine-client/issues/4"
|
||||
exit 200
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.xlsx"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,96 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.xml"
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<root>
|
||||
<record>
|
||||
<a>1</a>
|
||||
<b>2</b>
|
||||
<c>3</c>
|
||||
</record>
|
||||
<record>
|
||||
<a>0</a>
|
||||
<b>0</b>
|
||||
<c>0</c>
|
||||
</record>
|
||||
<record>
|
||||
<a>$</a>
|
||||
<b>\</b>
|
||||
<c>'</c>
|
||||
</record>
|
||||
</root>
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/column-reorder",
|
||||
"columnNames": [
|
||||
"record - a",
|
||||
"record - b",
|
||||
"record - c"
|
||||
],
|
||||
"description": "Reorder columns"
|
||||
},
|
||||
{
|
||||
"op": "core/row-removal",
|
||||
"engineConfig": {
|
||||
"facets": [
|
||||
{
|
||||
"type": "list",
|
||||
"name": "Blank Rows",
|
||||
"expression": "(filter(row.columnNames,cn,isNonBlank(cells[cn].value)).length()==0).toString()",
|
||||
"columnName": "",
|
||||
"invert": false,
|
||||
"omitBlank": false,
|
||||
"omitError": false,
|
||||
"selection": [
|
||||
{
|
||||
"v": {
|
||||
"v": "true",
|
||||
"l": "true"
|
||||
}
|
||||
}
|
||||
],
|
||||
"selectBlank": false,
|
||||
"selectError": false
|
||||
}
|
||||
],
|
||||
"mode": "record-based"
|
||||
}
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
record - a record - b record - c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.xml" --recordPath "root" --recordPath "record"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,96 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.xml"
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<root>
|
||||
<record>
|
||||
<icon>⛲</icon>
|
||||
<code>1F347</code>
|
||||
<meaning>FOUNTAIN</meaning>
|
||||
</record>
|
||||
<record>
|
||||
<icon>⛳</icon>
|
||||
<code>1F349</code>
|
||||
<meaning>FLAG IN HOLE</meaning>
|
||||
</record>
|
||||
<record>
|
||||
<icon>⛵</icon>
|
||||
<code>1F352</code>
|
||||
<meaning>SAILBOAT</meaning>
|
||||
</record>
|
||||
</root>
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/column-reorder",
|
||||
"columnNames": [
|
||||
"root - record - icon",
|
||||
"root - record - code",
|
||||
"root - record - meaning"
|
||||
],
|
||||
"description": "Reorder columns"
|
||||
},
|
||||
{
|
||||
"op": "core/row-removal",
|
||||
"engineConfig": {
|
||||
"facets": [
|
||||
{
|
||||
"type": "list",
|
||||
"name": "Blank Rows",
|
||||
"expression": "(filter(row.columnNames,cn,isNonBlank(cells[cn].value)).length()==0).toString()",
|
||||
"columnName": "",
|
||||
"invert": false,
|
||||
"omitBlank": false,
|
||||
"omitError": false,
|
||||
"selection": [
|
||||
{
|
||||
"v": {
|
||||
"v": "true",
|
||||
"l": "true"
|
||||
}
|
||||
}
|
||||
],
|
||||
"selectBlank": false,
|
||||
"selectError": false
|
||||
}
|
||||
],
|
||||
"mode": "record-based"
|
||||
}
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
root - record - icon root - record - code root - record - meaning
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.xml"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,96 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.xml"
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<root>
|
||||
<record>
|
||||
<a>1</a>
|
||||
<b>2</b>
|
||||
<c>3</c>
|
||||
</record>
|
||||
<record>
|
||||
<a>0</a>
|
||||
<b>0</b>
|
||||
<c>0</c>
|
||||
</record>
|
||||
<record>
|
||||
<a>$</a>
|
||||
<b>\</b>
|
||||
<c>'</c>
|
||||
</record>
|
||||
</root>
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/column-reorder",
|
||||
"columnNames": [
|
||||
"root - record - a",
|
||||
"root - record - b",
|
||||
"root - record - c"
|
||||
],
|
||||
"description": "Reorder columns"
|
||||
},
|
||||
{
|
||||
"op": "core/row-removal",
|
||||
"engineConfig": {
|
||||
"facets": [
|
||||
{
|
||||
"type": "list",
|
||||
"name": "Blank Rows",
|
||||
"expression": "(filter(row.columnNames,cn,isNonBlank(cells[cn].value)).length()==0).toString()",
|
||||
"columnName": "",
|
||||
"invert": false,
|
||||
"omitBlank": false,
|
||||
"omitError": false,
|
||||
"selection": [
|
||||
{
|
||||
"v": {
|
||||
"v": "true",
|
||||
"l": "true"
|
||||
}
|
||||
}
|
||||
],
|
||||
"selectBlank": false,
|
||||
"selectError": false
|
||||
}
|
||||
],
|
||||
"mode": "record-based"
|
||||
}
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
root - record - a root - record - b root - record - c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.xml"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}-1.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}-2.csv"
|
||||
a,b,c
|
||||
4,5,6
|
||||
DATA
|
||||
|
||||
zip "tmp/${t}/${t}.zip" "tmp/${t}/${t}-1.csv" "tmp/${t}/${t}-2.csv"
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << DATA > "tmp/${t}/${t}.assert"
|
||||
File a b c
|
||||
tmp/${t}/${t}-1.csv 1 2 3
|
||||
tmp/${t}/${t}-2.csv 4 5 6
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.zip" --includeFileSources "true"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}-1.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}-2.csv"
|
||||
a,b,c
|
||||
4,5,6
|
||||
DATA
|
||||
|
||||
zip "tmp/${t}/${t}.zip" "tmp/${t}/${t}-1.csv" "tmp/${t}/${t}-2.csv"
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
4 5 6
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.zip"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
@ -1,36 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << DATA > "tmp/${t}/${t}.assert"
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --projectName "${t} biểu tượng cảm xúc ⛲"
|
||||
${cmd} --list | grep "${t}" || exit 1
|
||||
${cmd} --delete "${t} biểu tượng cảm xúc ⛲"
|
||||
${cmd} --list | grep "${t}" | cut -d ':' -f 2 > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,36 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << DATA > "tmp/${t}/${t}.assert"
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --list | grep "${t}" || exit 1
|
||||
${cmd} --delete "${t}"
|
||||
${cmd} --list | grep "${t}" | cut -d ':' -f 2 > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,21 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --download "https://git.io/fj5ju" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "data/duplicates-deletion.json" "tmp/${t}/${t}.output"
|
||||
@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.tsv"
|
||||
🔣 code meaning
|
||||
🍇 1F347 GRAPES
|
||||
🍉 1F349 WATERMELON
|
||||
🍒 1F352 CHERRIES
|
||||
🍓 1F353 STRAWBERRY
|
||||
🍍 1F34D PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
🔣,code,meaning
|
||||
🍇,1F347,GRAPES
|
||||
🍉,1F349,WATERMELON
|
||||
🍒,1F352,CHERRIES
|
||||
🍓,1F353,STRAWBERRY
|
||||
🍍,1F34D,PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.tsv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t} biểu tượng cảm xúc 🍉.csv"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t} biểu tượng cảm xúc 🍉.csv"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.tsv"
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.tsv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.csv"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.csv"
|
||||
@ -1,72 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
majorversion="${2%%.*}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,FLAG IN HOLE
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "$majorversion" = 2 ]]; then
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
<html>
|
||||
<head>
|
||||
<title>export-html-utf8</title>
|
||||
<meta charset="utf-8" />
|
||||
</head>
|
||||
<body>
|
||||
<table>
|
||||
<tr><th>⌨</th><th>code</th><th>meaning</th></tr>
|
||||
<tr><td>⛲</td><td>1F347</td><td>FOUNTAIN</td></tr>
|
||||
<tr><td>⛳</td><td>1F349</td><td>FLAG IN HOLE</td></tr>
|
||||
<tr><td>⛵</td><td>1F352</td><td>SAILBOAT</td></tr>
|
||||
</table>
|
||||
</body>
|
||||
</html>
|
||||
DATA
|
||||
else
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
<html>
|
||||
<head>
|
||||
<title>export-html-utf8</title>
|
||||
<meta charset="utf-8" />
|
||||
</head>
|
||||
<body>
|
||||
<table>
|
||||
<tr><th>⌨</th><th>code</th><th>meaning</th></tr>
|
||||
<tr><td>⛲</td><td>1F347</td><td>FOUNTAIN</td></tr>
|
||||
<tr><td>⛳</td><td>1F349</td><td>FLAG IN HOLE</td></tr>
|
||||
<tr><td>⛵</td><td>1F352</td><td>SAILBOAT</td></tr>
|
||||
</table>
|
||||
</body>
|
||||
</html>
|
||||
DATA
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.html"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.html"
|
||||
@ -1,50 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
<html>
|
||||
<head>
|
||||
<title>export-html</title>
|
||||
<meta charset="utf-8" />
|
||||
</head>
|
||||
<body>
|
||||
<table>
|
||||
<tr><th>a</th><th>b</th><th>c</th></tr>
|
||||
<tr><td>1</td><td>2</td><td>3</td></tr>
|
||||
<tr><td>0</td><td>0</td><td>0</td></tr>
|
||||
<tr><td>$</td><td>\</td><td>'</td></tr>
|
||||
</table>
|
||||
</body>
|
||||
</html>
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.html"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.html"
|
||||
@ -1,43 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,FLAG IN HOLE
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,"FLAG IN HOLE"
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.ods"
|
||||
(cd tmp/"${t}" &&
|
||||
ssconvert -S "${t}.ods" "${t}.csv" &&
|
||||
mv "${t}.csv.1" "${t}.output")
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,47 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ -z "$(command -v ssconvert 2> /dev/null)" ]] ; then
|
||||
echo 1>&2 "ERROR: This test requires ssconvert (gnumeric)"
|
||||
exit 127
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.ods"
|
||||
(cd tmp/"${t}" &&
|
||||
ssconvert -S "${t}.ods" "${t}.csv" &&
|
||||
mv "${t}.csv.1" "${t}.output")
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
🔣,code,meaning
|
||||
🍇,1F347,GRAPES
|
||||
🍉,1F349,WATERMELON
|
||||
🍒,1F352,CHERRIES
|
||||
🍓,1F353,STRAWBERRY
|
||||
🍍,1F34D,PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
🔣 code meaning
|
||||
🍇 1F347 GRAPES
|
||||
🍉 1F349 WATERMELON
|
||||
🍒 1F352 CHERRIES
|
||||
🍓 1F353 STRAWBERRY
|
||||
🍍 1F34D PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.tsv"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.tsv"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.tsv"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.tsv"
|
||||
@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
🔣,code,meaning
|
||||
🍇,1F347,GRAPES
|
||||
🍉,1F349,WATERMELON
|
||||
🍒,1F352,CHERRIES
|
||||
🍓,1F353,STRAWBERRY
|
||||
🍍,1F34D,PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
🔣 code meaning
|
||||
🍇 1F347 GRAPES
|
||||
🍉 1F349 WATERMELON
|
||||
🍒 1F352 CHERRIES
|
||||
🍓 1F353 STRAWBERRY
|
||||
🍍 1F34D PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,43 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,FLAG IN HOLE
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,FLAG IN HOLE
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.xls"
|
||||
(cd tmp/"${t}" &&
|
||||
ssconvert -S "${t}.xls" "${t}.csv" &&
|
||||
mv "${t}.csv" "${t}.output")
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,43 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.xls"
|
||||
(cd tmp/"${t}" &&
|
||||
ssconvert -S "${t}.xls" "${t}.csv" &&
|
||||
mv "${t}.csv" "${t}.output")
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,45 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,FLAG IN HOLE
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,FLAG IN HOLE
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.xlsx"
|
||||
(cd tmp/"${t}" &&
|
||||
ssconvert -S "${t}.xlsx" "${t}.csv" &&
|
||||
mv "${t}.csv" "${t}.output")
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
|
||||
|
||||
@ -1,43 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.xlsx"
|
||||
(cd tmp/"${t}" &&
|
||||
ssconvert -S "${t}.xlsx" "${t}.csv" &&
|
||||
mv "${t}.csv" "${t}.output")
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.txt"
|
||||
a;b;c
|
||||
1;2;3
|
||||
0;0;0
|
||||
$;\;'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.txt" --format "csv" --separator ";"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,41 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
Column 1
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --format "line-based"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --format "csv" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --format "csv" > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,27 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
Script to provide a command line interface to an OpenRefine server.
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --help | sed '3q;d' > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,39 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.tsv"
|
||||
🔣 code meaning
|
||||
🍇 1F347 GRAPES
|
||||
🍉 1F349 WATERMELON
|
||||
🍒 1F352 CHERRIES
|
||||
🍓 1F353 STRAWBERRY
|
||||
🍍 1F34D PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << DATA > "tmp/${t}/${t}.assert"
|
||||
column 001: 🔣
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.tsv"
|
||||
${cmd} --info "${t}" | grep 'column 001' > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,35 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << DATA > "tmp/${t}/${t}.assert"
|
||||
column 002: b
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --info "${t}" | grep 'column 002' > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,35 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << DATA > "tmp/${t}/${t}.assert"
|
||||
${t} biểu tượng cảm xúc ⛲
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --projectName "${t} biểu tượng cảm xúc ⛲"
|
||||
${cmd} --list | grep "${t}" | cut -d ':' -f 2 > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,35 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << DATA > "tmp/${t}/${t}.assert"
|
||||
${t}
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --list | grep "${t}" | cut -d ':' -f 2 > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,58 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
email,name,state,gender,purchase
|
||||
danny.baron@example1.com,Danny Baron,CA,M,TV
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPhone
|
||||
danny.baron@example1.com,D. Baron,CA,M,Winter jacket
|
||||
ben.tyler@example3.org,Ben Tyler,NV,M,Flashlight
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Dining table
|
||||
danny.baron@example1.com,Daniel Baron,CA,M,Bike
|
||||
jean.griffith@example5.org,Jean Griffith,WA,F,Power drill
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPad
|
||||
ben.morisson@example6.org,Ben Morisson,FL,M,Amplifier
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Night table
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "events" : [
|
||||
{ "name" : "Melanie White", "purchase" : "iPhone" },
|
||||
{ "name" : "Jean Griffith", "purchase" : "Power drill" },
|
||||
{ "name" : "Melanie White", "purchase" : "iPad" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "events" : [
|
||||
' \
|
||||
--template ' { "name" : {{jsonize(cells["name"].value)}}, "purchase" : {{jsonize(cells["purchase"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--facets '{"type":"list","name":"gender","columnName":"gender","expression":"value","omitBlank":false,"omitError":false,"selection":[{"v":{"v":"F","l":"F"}}],"selectBlank":false,"selectError":false,"invert":false}' \
|
||||
--output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,54 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
🔣,code,meaning
|
||||
🍇,1F347,GRAPES
|
||||
🍉,1F349,WATERMELON
|
||||
🍒,1F352,CHERRIES
|
||||
🍓,1F353,STRAWBERRY
|
||||
🍍,1F34D,PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "emojis" : [
|
||||
{ "symbol" : "🍇", "meaning" : "GRAPES" },
|
||||
{ "symbol" : "🍉", "meaning" : "WATERMELON" },
|
||||
{ "symbol" : "🍍", "meaning" : "PINEAPPLE" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "emojis" : [
|
||||
' \
|
||||
--template ' { "symbol" : {{jsonize(with(row.columnNames[0],cn,cells[cn].value))}}, "meaning" : {{jsonize(cells["meaning"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--filterQuery '^1F34' \
|
||||
--filterColumn 'code' \
|
||||
--output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,59 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
email,name,state,gender,purchase
|
||||
danny.baron@example1.com,Danny Baron,CA,M,TV
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPhone
|
||||
danny.baron@example1.com,D. Baron,CA,M,Winter jacket
|
||||
ben.tyler@example3.org,Ben Tyler,NV,M,Flashlight
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Dining table
|
||||
danny.baron@example1.com,Daniel Baron,CA,M,Bike
|
||||
jean.griffith@example5.org,Jean Griffith,WA,F,Power drill
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPad
|
||||
ben.morisson@example6.org,Ben Morisson,FL,M,Amplifier
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Night table
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "events" : [
|
||||
{ "name" : "Melanie White", "purchase" : "iPhone" },
|
||||
{ "name" : "Jean Griffith", "purchase" : "Power drill" },
|
||||
{ "name" : "Melanie White", "purchase" : "iPad" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "events" : [
|
||||
' \
|
||||
--template ' { "name" : {{jsonize(cells["name"].value)}}, "purchase" : {{jsonize(cells["purchase"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--filterQuery '^F$' \
|
||||
--filterColumn 'gender' \
|
||||
--output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,58 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
email,name,state,gender,purchase
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Dining table
|
||||
,Arthur Duff,OR,M,Night table
|
||||
ben.morisson@example6.org,Ben Morisson,FL,M,Amplifier
|
||||
ben.tyler@example3.org,Ben Tyler,NV,M,Flashlight
|
||||
danny.baron@example1.com,Daniel Baron,CA,M,Bike
|
||||
,Danny Baron,CA,M,TV
|
||||
,D. Baron,CA,M,Winter jacket
|
||||
jean.griffith@example5.org,Jean Griffith,WA,F,Power drill
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPad
|
||||
,Melanie White,NC,F,iPhone
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "events" : [
|
||||
{ "name" : "Melanie White", "purchase" : "iPad" } { "name" : "Melanie White", "purchase" : "iPhone" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "events" : [
|
||||
' \
|
||||
--template ' { "name" : {{jsonize(cells["name"].value)}}, "purchase" : {{jsonize(cells["purchase"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--mode "record-based" \
|
||||
--splitToFiles true \
|
||||
--output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
ls "tmp/${t}"
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}_6.output"
|
||||
@ -1,52 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
🔣,code,meaning
|
||||
🍇,1F347,GRAPES
|
||||
🍉,1F349,WATERMELON
|
||||
🍒,1F352,CHERRIES
|
||||
🍓,1F353,STRAWBERRY
|
||||
🍍,1F34D,PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "emojis" : [
|
||||
{ "symbol" : "🍍", "meaning" : "PINEAPPLE" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "emojis" : [
|
||||
' \
|
||||
--template ' { "symbol" : {{jsonize(with(row.columnNames[0],cn,cells[cn].value))}}, "meaning" : {{jsonize(cells["meaning"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--splitToFiles true \
|
||||
--suffixById true \
|
||||
--output "tmp/${t}/trái cây.json"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/trái cây_🍍.json"
|
||||
@ -1,58 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
email,name,state,gender,purchase
|
||||
danny.baron@example1.com,Danny Baron,CA,M,TV
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPhone
|
||||
danny.baron@example1.com,D. Baron,CA,M,Winter jacket
|
||||
ben.tyler@example3.org,Ben Tyler,NV,M,Flashlight
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Dining table
|
||||
danny.baron@example1.com,Daniel Baron,CA,M,Bike
|
||||
jean.griffith@example5.org,Jean Griffith,WA,F,Power drill
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPad
|
||||
ben.morisson@example6.org,Ben Morisson,FL,M,Amplifier
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Night table
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "events" : [
|
||||
{ "name" : "Arthur Duff", "purchase" : "Night table" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "events" : [
|
||||
' \
|
||||
--template ' { "name" : {{jsonize(cells["name"].value)}}, "purchase" : {{jsonize(cells["purchase"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--splitToFiles true \
|
||||
--suffixById true \
|
||||
--output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
ls "tmp/${t}"
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}_arthur.duff@example4.com.output"
|
||||
@ -1,51 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
🔣,code,meaning
|
||||
🍇,1F347,GRAPES
|
||||
🍉,1F349,WATERMELON
|
||||
🍒,1F352,CHERRIES
|
||||
🍓,1F353,STRAWBERRY
|
||||
🍍,1F34D,PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "emojis" : [
|
||||
{ "symbol" : "🍍", "meaning" : "PINEAPPLE" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "emojis" : [
|
||||
' \
|
||||
--template ' { "symbol" : {{jsonize(with(row.columnNames[0],cn,cells[cn].value))}}, "meaning" : {{jsonize(cells["meaning"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--splitToFiles true \
|
||||
--output "tmp/${t}/trái cây.json"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/trái cây_5.json"
|
||||
@ -1,57 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
email,name,state,gender,purchase
|
||||
danny.baron@example1.com,Danny Baron,CA,M,TV
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPhone
|
||||
danny.baron@example1.com,D. Baron,CA,M,Winter jacket
|
||||
ben.tyler@example3.org,Ben Tyler,NV,M,Flashlight
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Dining table
|
||||
danny.baron@example1.com,Daniel Baron,CA,M,Bike
|
||||
jean.griffith@example5.org,Jean Griffith,WA,F,Power drill
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPad
|
||||
ben.morisson@example6.org,Ben Morisson,FL,M,Amplifier
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Night table
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "events" : [
|
||||
{ "name" : "Arthur Duff", "purchase" : "Night table" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "events" : [
|
||||
' \
|
||||
--template ' { "name" : {{jsonize(cells["name"].value)}}, "purchase" : {{jsonize(cells["purchase"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--splitToFiles true \
|
||||
--output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
ls "tmp/${t}"
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}_10.output"
|
||||
@ -1,54 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
🔣,code,meaning
|
||||
🍇,1F347,GRAPES
|
||||
🍉,1F349,WATERMELON
|
||||
🍒,1F352,CHERRIES
|
||||
🍓,1F353,STRAWBERRY
|
||||
🍍,1F34D,PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "emojis" : [
|
||||
{ "symbol" : "🍇", "meaning" : "GRAPES" },
|
||||
{ "symbol" : "🍉", "meaning" : "WATERMELON" },
|
||||
{ "symbol" : "🍒", "meaning" : "CHERRIES" },
|
||||
{ "symbol" : "🍓", "meaning" : "STRAWBERRY" },
|
||||
{ "symbol" : "🍍", "meaning" : "PINEAPPLE" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "emojis" : [
|
||||
' \
|
||||
--template ' { "symbol" : {{jsonize(with(row.columnNames[0],cn,cells[cn].value))}}, "meaning" : {{jsonize(cells["meaning"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,64 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
email,name,state,gender,purchase
|
||||
danny.baron@example1.com,Danny Baron,CA,M,TV
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPhone
|
||||
danny.baron@example1.com,D. Baron,CA,M,Winter jacket
|
||||
ben.tyler@example3.org,Ben Tyler,NV,M,Flashlight
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Dining table
|
||||
danny.baron@example1.com,Daniel Baron,CA,M,Bike
|
||||
jean.griffith@example5.org,Jean Griffith,WA,F,Power drill
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPad
|
||||
ben.morisson@example6.org,Ben Morisson,FL,M,Amplifier
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Night table
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "events" : [
|
||||
{ "name" : "Danny Baron", "purchase" : "TV" },
|
||||
{ "name" : "Melanie White", "purchase" : "iPhone" },
|
||||
{ "name" : "D. Baron", "purchase" : "Winter jacket" },
|
||||
{ "name" : "Ben Tyler", "purchase" : "Flashlight" },
|
||||
{ "name" : "Arthur Duff", "purchase" : "Dining table" },
|
||||
{ "name" : "Daniel Baron", "purchase" : "Bike" },
|
||||
{ "name" : "Jean Griffith", "purchase" : "Power drill" },
|
||||
{ "name" : "Melanie White", "purchase" : "iPad" },
|
||||
{ "name" : "Ben Morisson", "purchase" : "Amplifier" },
|
||||
{ "name" : "Arthur Duff", "purchase" : "Night table" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "events" : [
|
||||
' \
|
||||
--template ' { "name" : {{jsonize(cells["name"].value)}}, "purchase" : {{jsonize(cells["purchase"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,27 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
Usage:
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} | head -n 1 | cut -c 1-6 > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
Loading…
x
Reference in New Issue
Block a user