mirror of
https://github.com/opencultureconsulting/openrefine-client.git
synced 2025-04-06 00:01:03 +02:00
Compare commits
No commits in common. "master" and "v0.3.7" have entirely different histories.
4
.gitignore
vendored
4
.gitignore
vendored
@ -4,6 +4,4 @@ dist
|
||||
.*
|
||||
openrefine_client.egg-info
|
||||
refine.spec
|
||||
openrefine-*
|
||||
openrefine-client_*
|
||||
tests-cli.log
|
||||
README.html
|
||||
|
||||
359
README.md
359
README.md
@ -1,20 +1,18 @@
|
||||
# OpenRefine Python Client with extended command line interface (⌨️ for 💎)
|
||||
# OpenRefine Python Client with extended command line interface
|
||||
|
||||
[](https://www.codacy.com/gh/opencultureconsulting/openrefine-client/dashboard) [](https://hub.docker.com/r/felixlohmeier/openrefine-client/) [](https://pypi.org/project/openrefine-client/) [](https://mybinder.org/v2/gh/felixlohmeier/openrefineder/master)
|
||||
[](https://www.codacy.com/app/felixlohmeier/openrefine-client?utm_source=github.com&utm_medium=referral&utm_content=opencultureconsulting/openrefine-client&utm_campaign=Badge_Grade) [](https://hub.docker.com/r/felixlohmeier/openrefine-client/) [](https://pypi.org/project/openrefine-client/) [](https://mybinder.org/v2/gh/betatim/openrefineder/22fbb07?filepath=openrefine-client.ipynb)
|
||||
|
||||
The [OpenRefine Python Client from PaulMakepeace](https://github.com/PaulMakepeace/refine-client-py) provides a library for communicating with an [OpenRefine](http://openrefine.org) server.
|
||||
This fork extends the command line interface (CLI) and is distributed as a convenient one-file-executable (Windows, Linux, macOS).
|
||||
It is also available via Docker Hub, PyPI and Binder.
|
||||
|
||||
works with OpenRefine 2.7, 2.8, 3.0, 3.1, 3.2, 3.3, 3.4, 3.4.1, 3.5.0
|
||||
|
||||
## Download
|
||||
|
||||
One-file-executables:
|
||||
|
||||
- Windows: [openrefine-client_0-3-10_windows.exe](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.10/openrefine-client_0-3-10_windows.exe) (~5 MB)
|
||||
- macOS: [openrefine-client_0-3-10_macos](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.10/openrefine-client_0-3-10_macos) (~5 MB)
|
||||
- Linux: [openrefine-client_0-3-10_linux](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.10/openrefine-client_0-3-10_linux) (~5 MB)
|
||||
- Windows: [openrefine-client_0-3-7_windows.exe](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.7/openrefine-client_0-3-7_windows.exe) (~5 MB)
|
||||
- macOS: [openrefine-client_0-3-7_macos](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.7/openrefine-client_0-3-7_macos) (~5 MB)
|
||||
- Linux: [openrefine-client_0-3-7_linux](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.7/openrefine-client_0-3-7_linux) (~5 MB)
|
||||
|
||||
For [Docker](#docker) containers, native [Python](#python) installation and free [Binder](#binder) on-demand server see the corresponding chapters below.
|
||||
|
||||
@ -34,19 +32,19 @@ To use the client:
|
||||
|
||||
- Windows: Open PowerShell and enter following command
|
||||
|
||||
```sh
|
||||
```
|
||||
cd ~\Downloads
|
||||
```
|
||||
|
||||
- macOS: Open Terminal (Finder > Applications > Utilities > Terminal) and enter following command
|
||||
|
||||
```sh
|
||||
```
|
||||
cd ~/Downloads
|
||||
```
|
||||
|
||||
- Linux: Open terminal app (Terminal, Konsole, xterm, ...) and enter following command
|
||||
|
||||
```sh
|
||||
```
|
||||
cd ~/Downloads
|
||||
```
|
||||
|
||||
@ -56,34 +54,34 @@ To use the client:
|
||||
|
||||
- macOS:
|
||||
|
||||
```sh
|
||||
chmod +x openrefine-client_0-3-10_macos
|
||||
```
|
||||
chmod +x openrefine-client_0-3-7_macos
|
||||
```
|
||||
|
||||
- Linux:
|
||||
|
||||
```sh
|
||||
chmod +x openrefine-client_0-3-10_linux
|
||||
```
|
||||
chmod +x openrefine-client_0-3-7_linux
|
||||
```
|
||||
|
||||
3. Execute the file.
|
||||
|
||||
- Windows:
|
||||
|
||||
```sh
|
||||
.\openrefine-client_0-3-10_windows.exe
|
||||
```
|
||||
.\openrefine-client_0-3-7_windows.exe
|
||||
```
|
||||
|
||||
- macOS:
|
||||
|
||||
```sh
|
||||
./openrefine-client_0-3-10_macos
|
||||
```
|
||||
./openrefine-client_0-3-7_macos
|
||||
```
|
||||
|
||||
- Linux:
|
||||
|
||||
```sh
|
||||
./openrefine-client_0-3-10_linux
|
||||
```
|
||||
./openrefine-client_0-3-7_linux
|
||||
```
|
||||
|
||||
Using tab completion and command history is highly recommended:
|
||||
@ -101,26 +99,26 @@ Download example data (`--download`) and create project from file (`--create`):
|
||||
|
||||
- Windows:
|
||||
|
||||
```sh
|
||||
.\openrefine-client_0-3-10_windows.exe --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
.\openrefine-client_0-3-10_windows.exe --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
.\openrefine-client_0-3-10_windows.exe --create duplicates.csv
|
||||
```
|
||||
.\openrefine-client_0-3-7_windows.exe --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
.\openrefine-client_0-3-7_windows.exe --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
.\openrefine-client_0-3-7_windows.exe --create duplicates.csv
|
||||
```
|
||||
|
||||
- macOS:
|
||||
|
||||
```sh
|
||||
./openrefine-client_0-3-10_macos --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
./openrefine-client_0-3-10_macos --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
./openrefine-client_0-3-10_macos --create duplicates.csv
|
||||
```
|
||||
./openrefine-client_0-3-7_macos --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
./openrefine-client_0-3-7_macos --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
./openrefine-client_0-3-7_macos --create duplicates.csv
|
||||
```
|
||||
|
||||
- Linux:
|
||||
|
||||
```sh
|
||||
./openrefine-client_0-3-10_linux --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
./openrefine-client_0-3-10_linux --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
./openrefine-client_0-3-10_linux --create duplicates.csv
|
||||
```
|
||||
./openrefine-client_0-3-7_linux --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
./openrefine-client_0-3-7_linux --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
./openrefine-client_0-3-7_linux --create duplicates.csv
|
||||
```
|
||||
|
||||
Other commands:
|
||||
@ -147,7 +145,7 @@ If your OpenRefine server is running somewhere else then you may set hostname an
|
||||
- set host: `-H example.com`
|
||||
- set port: `-P 80`
|
||||
|
||||
### Templating
|
||||
### Advanced Templating
|
||||
|
||||
The OpenRefine [Templating](https://github.com/OpenRefine/OpenRefine/wiki/Export-As-YAML) supports exporting data in any text format (i.e. to construct JSON or XML).
|
||||
The graphical user interface offers four input fields:
|
||||
@ -163,20 +161,20 @@ It even provides an additional feature for splitting results into multiple files
|
||||
|
||||
To try out the functionality create another project from the example file above.
|
||||
|
||||
```sh
|
||||
```
|
||||
--create duplicates.csv --projectName=advanced
|
||||
```
|
||||
|
||||
The following example code will export...
|
||||
|
||||
- the columns "name" and "purchase" in JSON format
|
||||
- from the project "advanced"
|
||||
- for rows matching the regex text filter `^F$` in column "gender"
|
||||
- from the project "duplicates"
|
||||
- for rows matching the regex text filter `^CA$` in column "state"
|
||||
|
||||
macOS/Linux Terminal (multi-line input with `\` ):
|
||||
|
||||
```sh
|
||||
"advanced" \
|
||||
```
|
||||
--export "advanced" \
|
||||
--prefix='{ "events" : [
|
||||
' \
|
||||
--template=' { "name" : {{jsonize(cells["name"].value)}}, "purchase" : {{jsonize(cells["purchase"].value)}} }' \
|
||||
@ -184,14 +182,14 @@ macOS/Linux Terminal (multi-line input with `\` ):
|
||||
' \
|
||||
--suffix='
|
||||
] }' \
|
||||
--filterQuery='^F$' \
|
||||
--filterColumn='gender'
|
||||
--filterQuery='^CA$' \
|
||||
--filterColumn='state'
|
||||
```
|
||||
|
||||
Windows PowerShell (multi-line input with `` ` ``; quotes needs to be doubled):
|
||||
|
||||
```sh
|
||||
"advanced" `
|
||||
```
|
||||
--export "advanced" `
|
||||
--prefix='{ ""events"" : [
|
||||
' `
|
||||
--template=' { ""name"" : {{jsonize(cells[""name""].value)}}, ""purchase"" : {{jsonize(cells[""purchase""].value)}} }' `
|
||||
@ -199,56 +197,31 @@ Windows PowerShell (multi-line input with `` ` ``; quotes needs to be doubled):
|
||||
' `
|
||||
--suffix='
|
||||
] }' `
|
||||
--filterQuery='^F$' `
|
||||
--filterColumn='gender'
|
||||
--filterQuery='^CA$' `
|
||||
--filterColumn='state'
|
||||
```
|
||||
|
||||
Add the following options to the last command (recall with `↑`) to store the results in multiple files.
|
||||
Each file will contain the prefix, an processed row, and the suffix.
|
||||
|
||||
```sh
|
||||
```
|
||||
--output=advanced.json --splitToFiles=true
|
||||
```
|
||||
|
||||
Filenames are suffixed with the row number by default (e.g. `advanced_1.json`, `advanced_2.json` etc.).
|
||||
There is another option to use the value in the first column instead:
|
||||
|
||||
```sh
|
||||
```
|
||||
--output=advanced.json --splitToFiles=true --suffixById=true
|
||||
```
|
||||
|
||||
Because our project "advanced" contains duplicates in the first column "email" this command will overwrite files (e.g. `advanced_melanie.white@example2.edu.json`).
|
||||
Because our project "advanced" contains duplicates in the first column "email" this command will store only one file `advanced_danny.baron@example1.com.json`.
|
||||
When using this option, the first column should contain unique identifiers.
|
||||
|
||||
### Append data to an existing project
|
||||
|
||||
OpenRefine does not support appending rows to an existing project.
|
||||
As long as the [feature request](https://github.com/OpenRefine/OpenRefine/issues/715) is not yet implemented, you can use the openrefine-client to script a workaround:
|
||||
|
||||
1. export existing project as csv
|
||||
2. put old and new data into a zip archive
|
||||
3. create new project by importing the zip archive
|
||||
|
||||
Here is an example that replaces the existing project:
|
||||
|
||||
```
|
||||
openrefine-client --export myproject --output old.csv
|
||||
openrefine-client --delete myproject
|
||||
zip combined.zip old.csv new.csv
|
||||
openrefine-client --create combined.zip --format csv --projectName myproject
|
||||
```
|
||||
|
||||
Note that the project id will change.
|
||||
If you want to distinguish between old and new data, you can use the additional flag includeFileSources:
|
||||
|
||||
```
|
||||
openrefine-client --create combined.zip --format csv --projectName myproject --includeFileSources true
|
||||
```
|
||||
|
||||
### See also
|
||||
|
||||
- Linux Bash script to run OpenRefine in batch mode (import, transform, export): [openrefine-batch](https://github.com/opencultureconsulting/openrefine-batch)
|
||||
- [Jupyter notebook demonstrating usage in Linux Bash](https://nbviewer.jupyter.org/github/felixlohmeier/openrefineder/blob/master/notebooks/openrefine-client-bash.ipynb)
|
||||
- [Jupyter notebook demonstrating usage in Linux Bash](https://nbviewer.jupyter.org/github/betatim/openrefineder/blob/master/openrefine-client.ipynb)
|
||||
- Use case [HOS-MetadataTransformations](https://github.com/subhh/HOS-MetadataTransformations): Automated workflow for harvesting, transforming and indexing of metadata using metha, OpenRefine and Solr. Part of the Hamburg Open Science "Schaufenster" software stack.
|
||||
- Use case [Data processing of ILS data to facilitate a new discovery layer for the German Literature Archive (DLA)](https://doi.org/10.5281/zenodo.2678113): Custom data processing pipeline based on Pandas (a Python library) and OpenRefine.
|
||||
|
||||
@ -256,30 +229,30 @@ openrefine-client --create combined.zip --format csv --projectName myproject --i
|
||||
|
||||
[felixlohmeier/openrefine-client](https://hub.docker.com/r/felixlohmeier/openrefine-client/) [](https://hub.docker.com/r/felixlohmeier/openrefine-client/)
|
||||
|
||||
```sh
|
||||
docker pull felixlohmeier/openrefine-client:v0.3.10
|
||||
```
|
||||
docker pull felixlohmeier/openrefine-client:v0.3.7
|
||||
```
|
||||
|
||||
### Option 1: Dockerized client
|
||||
|
||||
Run client and mount current directory as workspace:
|
||||
|
||||
```sh
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10
|
||||
```
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.7
|
||||
```
|
||||
|
||||
The docker option `--network=host` allows you to connect to a local or remote OpenRefine via the host network:
|
||||
|
||||
- list projects on default URL (http://localhost:3333)
|
||||
|
||||
```sh
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 --list
|
||||
```
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.7 --list
|
||||
```
|
||||
|
||||
- list projects on a remote server (http://example.com)
|
||||
|
||||
```sh
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H example.com -P 80 --list
|
||||
```
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.7 -H example.com -P 80 --list
|
||||
```
|
||||
|
||||
Usage: same commands as explained above (see [Basic Commands](#basic-commands) and [Advanced Templating](#advanced-templating))
|
||||
@ -290,41 +263,41 @@ Run openrefine-client linked to a dockerized OpenRefine ([felixlohmeier/openrefi
|
||||
|
||||
1. Create docker network
|
||||
|
||||
```sh
|
||||
```
|
||||
docker network create openrefine
|
||||
```
|
||||
|
||||
2. Run server (will be available at http://localhost:3333)
|
||||
|
||||
```sh
|
||||
docker run -d -p 3333:3333 --network=openrefine --name=openrefine-server felixlohmeier/openrefine:3.5.0
|
||||
```
|
||||
docker run -d -p 3333:3333 --network=openrefine --name=openrefine-server felixlohmeier/openrefine:3.2
|
||||
```
|
||||
|
||||
3. Run client with some [basic commands](#basic-commands): 1. download example files, 2. create project from file, 3. list projects, 4. show metadata, 5. export to terminal, 6. apply transformation rules (deduplication), 7. export again to terminal, 8. export to xls file and 9. delete project
|
||||
|
||||
```sh
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H openrefine-server --create duplicates.csv
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H openrefine-server --list
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H openrefine-server --info "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H openrefine-server --export "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H openrefine-server --apply duplicates-deletion.json "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H openrefine-server --export "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H openrefine-server --export --output=deduped.xls "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.10 -H openrefine-server --delete "duplicates"
|
||||
```
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.7 --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.7 --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.7 -H openrefine-server --create duplicates.csv
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.7 -H openrefine-server --list
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.7 -H openrefine-server --info "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.7 -H openrefine-server --export "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.7 -H openrefine-server --apply duplicates-deletion.json "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.7 -H openrefine-server --export "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.7 -H openrefine-server --export --output=deduped.xls "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.7 -H openrefine-server --delete "duplicates"
|
||||
```
|
||||
|
||||
4. Stop and delete server:
|
||||
|
||||
```sh
|
||||
```
|
||||
docker stop openrefine-server
|
||||
docker rm openrefine-server
|
||||
```
|
||||
|
||||
5. Delete docker network:
|
||||
|
||||
```sh
|
||||
```
|
||||
docker network rm openrefine
|
||||
```
|
||||
|
||||
@ -336,21 +309,21 @@ Customize OpenRefine server:
|
||||
|
||||
- Example for [allocating more memory](https://github.com/OpenRefine/OpenRefine/wiki/FAQ#out-of-memory-errors---feels-slow---could-not-reserve-enough-space-for-object-heap) to OpenRefine with additional option `-m 4G`
|
||||
|
||||
```sh
|
||||
docker run -d -p 3333:3333 --network=openrefine --name=openrefine-server felixlohmeier/openrefine:3.5.0 -i 0.0.0.0 -d /data -m 4G
|
||||
```
|
||||
docker run -d -p 3333:3333 --network=openrefine --name=openrefine-server felixlohmeier/openrefine:3.2 -i 0.0.0.0 -d /data -m 4G
|
||||
```
|
||||
|
||||
- The OpenRefine version is defined by the docker tag.
|
||||
Check the [DockerHub repository](https://hub.docker.com/r/felixlohmeier/openrefine) for available tags.
|
||||
Example for OpenRefine `2.8` with same options as above:
|
||||
|
||||
```sh
|
||||
```
|
||||
docker run -d -p 3333:3333 --network=openrefine --name=openrefine-server felixlohmeier/openrefine:2.8 -i 0.0.0.0 -d /data -m 4G
|
||||
```
|
||||
|
||||
- If you want OpenRefine to read and write persistent data in host directory (i.e. store projects) you can mount the container path `/data`. Example for host directory `/home/felix/refine`:
|
||||
|
||||
```sh
|
||||
```
|
||||
docker run -d -p 3333:3333 -v /home/felix/refine:/data:z --network=openrefine name=openrefine-server felixlohmeier/openrefine:2.8 -i 0.0.0.0 -d /data -m 4G
|
||||
```
|
||||
|
||||
@ -363,8 +336,8 @@ See also:
|
||||
|
||||
[openrefine-client](https://pypi.org/project/openrefine-client/) [](https://pypi.org/project/openrefine-client/) (requires Python 2.x)
|
||||
|
||||
```sh
|
||||
python2 -m pip install openrefine-client --user
|
||||
```
|
||||
pip install openrefine-client
|
||||
```
|
||||
|
||||
This will install the package `openrefine-client` containing modules in `google.refine`.
|
||||
@ -373,7 +346,7 @@ A command line script `openrefine-client` will also be installed.
|
||||
|
||||
### Option 1: command line script
|
||||
|
||||
```sh
|
||||
```
|
||||
openrefine-client --help
|
||||
```
|
||||
|
||||
@ -383,20 +356,20 @@ Usage: same commands as explained above (see [Basic Commands](#basic-commands) a
|
||||
|
||||
Import module cli:
|
||||
|
||||
```python
|
||||
```
|
||||
from google.refine import cli
|
||||
```
|
||||
|
||||
Change URL (if necessary):
|
||||
|
||||
```python
|
||||
cli.refine.REFINE_HOST = 'localhost'
|
||||
cli.refine.REFINE_PORT = '3333'
|
||||
```
|
||||
refine.REFINE_HOST = 'localhost'
|
||||
refine.REFINE_PORT = 3333
|
||||
```
|
||||
|
||||
Help screen:
|
||||
|
||||
```python
|
||||
```
|
||||
help(cli)
|
||||
```
|
||||
|
||||
@ -404,62 +377,59 @@ Commands:
|
||||
|
||||
* download (e.g. example data):
|
||||
|
||||
```python
|
||||
```
|
||||
cli.download('https://git.io/fj5hF','duplicates.csv')
|
||||
cli.download('https://git.io/fj5ju','duplicates-deletion.json')
|
||||
```
|
||||
|
||||
* list projects:
|
||||
|
||||
```python
|
||||
```
|
||||
cli.ls()
|
||||
```
|
||||
|
||||
* create project:
|
||||
|
||||
```python
|
||||
```
|
||||
p1 = cli.create('duplicates.csv')
|
||||
```
|
||||
|
||||
* show metadata:
|
||||
|
||||
```python
|
||||
```
|
||||
cli.info(p1.project_id)
|
||||
```
|
||||
|
||||
* apply rules from file to project:
|
||||
|
||||
```python
|
||||
```
|
||||
cli.apply(p1.project_id, 'duplicates-deletion.json')
|
||||
```
|
||||
|
||||
* export project to terminal:
|
||||
|
||||
```python
|
||||
```
|
||||
cli.export(p1.project_id)
|
||||
```
|
||||
|
||||
* export project to file in xls format:
|
||||
|
||||
```python
|
||||
```
|
||||
cli.export(p1.project_id, 'deduped.xls')
|
||||
```
|
||||
|
||||
* export templating (see [Advanced Templating](#advanced-templating) above):
|
||||
|
||||
```python
|
||||
cli.templating(
|
||||
p1.project_id,
|
||||
prefix='''{ "events" : [
|
||||
''',template=''' { "name" : {{jsonize(cells["name"].value)}}, "purchase" : {{jsonize(cells["purchase"].value)}} }''',
|
||||
rowSeparator=''',
|
||||
''',suffix='''
|
||||
```
|
||||
cli.templating(p1.project_id, prefix='''{ "events" : [
|
||||
''', template=''' { "name" : {{jsonize(cells["name"].value)}}, "purchase" : {{jsonize(cells["purchase"].value)}} }''', rowSeparator=''',
|
||||
''', suffix='''
|
||||
] }''')
|
||||
```
|
||||
|
||||
* delete project:
|
||||
|
||||
```python
|
||||
```
|
||||
cli.delete(p1.project_id)
|
||||
```
|
||||
|
||||
@ -471,7 +441,7 @@ Some functions in the python client library are not yet compatible with OpenRefi
|
||||
|
||||
Import module refine:
|
||||
|
||||
```python
|
||||
```
|
||||
from google.refine import refine
|
||||
```
|
||||
|
||||
@ -479,39 +449,39 @@ Server Commands:
|
||||
|
||||
* set up connection:
|
||||
|
||||
```python
|
||||
```
|
||||
server1 = refine.Refine('http://localhost:3333')
|
||||
```
|
||||
|
||||
- show version:
|
||||
|
||||
```python
|
||||
```
|
||||
server1.server.get_version()
|
||||
server1.server.version
|
||||
```
|
||||
|
||||
- list projects:
|
||||
|
||||
```python
|
||||
```
|
||||
server1.list_projects()
|
||||
```
|
||||
|
||||
- pretty print the returned dict with json.dumps:
|
||||
|
||||
```python
|
||||
```
|
||||
import json
|
||||
print(json.dumps(server1.list_projects(), indent=1))
|
||||
```
|
||||
|
||||
- create project:
|
||||
- create project (**function was edited in this fork**):
|
||||
|
||||
```python
|
||||
```
|
||||
server1.new_project(project_file='duplicates.csv')
|
||||
```
|
||||
|
||||
* create and open the returned project in one step:
|
||||
|
||||
```python
|
||||
```
|
||||
project1 = server1.new_project(project_file='duplicates.csv')
|
||||
```
|
||||
|
||||
@ -519,31 +489,31 @@ Project commands:
|
||||
|
||||
* open project:
|
||||
|
||||
```python
|
||||
```
|
||||
project1 = server1.open_project('1234567890123')
|
||||
```
|
||||
|
||||
* print full URL to project:
|
||||
|
||||
```python
|
||||
```
|
||||
project1.project_url()
|
||||
```
|
||||
|
||||
* list columns:
|
||||
|
||||
```python
|
||||
```
|
||||
project1.columns
|
||||
```
|
||||
|
||||
* compute text facet on first column (**fails with OpenRefine >=3.2**):
|
||||
|
||||
```python
|
||||
```
|
||||
project1.compute_facets(facet.TextFacet(project1.columns[0]))
|
||||
```
|
||||
|
||||
* print returned object
|
||||
|
||||
```python
|
||||
```
|
||||
facets = project1.compute_facets(facet.TextFacet(project1.columns[0])).facets[0]
|
||||
for k in sorted(facets.choices, key=lambda k: facets.choices[k].count, reverse=True):
|
||||
print(facets.choices[k].count, k)
|
||||
@ -551,62 +521,60 @@ Project commands:
|
||||
|
||||
* compute clusters on first column:
|
||||
|
||||
```python
|
||||
```
|
||||
project1.compute_clusters(project1.columns[0])
|
||||
```
|
||||
|
||||
* apply rules from file to project:
|
||||
|
||||
```python
|
||||
```
|
||||
project1.apply_operations('duplicates-deletion.json')
|
||||
```
|
||||
|
||||
* export project:
|
||||
|
||||
```python
|
||||
```
|
||||
project1.export(export_format='tsv')
|
||||
```
|
||||
|
||||
* print the returned fileobject:
|
||||
|
||||
```python
|
||||
```
|
||||
print(project1.export(export_format='tsv').read())
|
||||
```
|
||||
|
||||
* save the returned fileobject to file:
|
||||
|
||||
```python
|
||||
```
|
||||
with open('export.tsv', 'wb') as f:
|
||||
f.write(project1.export(export_format='tsv').read())
|
||||
```
|
||||
|
||||
* templating export (**function was added in this fork**, see [Advanced Templating](#advanced-templating) above):
|
||||
|
||||
```python
|
||||
data = project1.export_templating(
|
||||
prefix='''{ "events" : [
|
||||
''',template=''' { "name" : {{jsonize(cells["name"].value)}}, "purchase" : {{jsonize(cells["purchase"].value)}} }''',
|
||||
rowSeparator=''',
|
||||
''',suffix='''
|
||||
```
|
||||
data = project1.export_templating(prefix='''{ "events" : [
|
||||
''', template=''' { "name" : {{jsonize(cells["name"].value)}}, "purchase" : {{jsonize(cells["purchase"].value)}} }''', rowSeparator=''',
|
||||
''', suffix='''
|
||||
] }''')
|
||||
print(data.read())
|
||||
```
|
||||
|
||||
* print help screen with available commands (many more!):
|
||||
|
||||
```python
|
||||
```
|
||||
help(project1)
|
||||
```
|
||||
|
||||
* example for custom commands:
|
||||
|
||||
```python
|
||||
```
|
||||
project1.do_json('get-rows')['total']
|
||||
```
|
||||
|
||||
* delete project:
|
||||
|
||||
```python
|
||||
```
|
||||
project1.delete()
|
||||
```
|
||||
|
||||
@ -619,13 +587,12 @@ See also:
|
||||
|
||||
## Binder
|
||||
|
||||
[](https://mybinder.org/v2/gh/felixlohmeier/openrefineder/master)
|
||||
[openrefineder](https://github.com/betatim/openrefineder) [](https://mybinder.org/v2/gh/betatim/openrefineder/22fbb07?filepath=openrefine-client.ipynb)
|
||||
|
||||
- free to use on-demand server with Jupyter notebook, OpenRefine and Bash
|
||||
- no registration needed, will start within a few minutes
|
||||
- [restricted](https://mybinder.readthedocs.io/en/latest/faq.html#how-much-memory-am-i-given-when-using-binder) to 2 GB RAM and server will be deleted after 10 minutes of inactivity
|
||||
- [bash_kernel demo notebook](https://nbviewer.jupyter.org/github/felixlohmeier/openrefineder/blob/master/notebooks/openrefine-client-bash.ipynb) for using the openrefine-client in a Linux Bash environment [](https://mybinder.org/v2/gh/felixlohmeier/openrefineder/master?urlpath=/tree/notebooks/openrefine-client-bash.ipynb)
|
||||
- [python2 demo notebook](https://nbviewer.jupyter.org/github/felixlohmeier/openrefineder/blob/master/notebooks/openrefine-client-python.ipynb) for using the openrefine-client in a Python 2 environment [](https://mybinder.org/v2/gh/felixlohmeier/openrefineder/master?urlpath=/tree/notebooks/openrefine-client-python.ipynb)
|
||||
- includes [demo notebook](https://nbviewer.jupyter.org/github/betatim/openrefineder/blob/master/openrefine-client.ipynb) for using openrefine-client with Linux Bash
|
||||
|
||||
## Development
|
||||
|
||||
@ -639,72 +606,55 @@ The Python client library includes several unit tests.
|
||||
|
||||
- run all tests
|
||||
|
||||
```sh
|
||||
```
|
||||
python setup.py test
|
||||
```
|
||||
|
||||
- run subset test_facet
|
||||
|
||||
```sh
|
||||
```
|
||||
python setup.py --test-suite tests.test_facet
|
||||
```
|
||||
|
||||
There is also a script that uses docker images to run the unit tests with different versions of OpenRefine.
|
||||
|
||||
- run tests on all OpenRefine versions (from 2.0 up to 3.5.0)
|
||||
- run tests on all OpenRefine versions (from 2.0 up to 3.2)
|
||||
|
||||
```sh
|
||||
```
|
||||
./tests.sh -a
|
||||
```
|
||||
|
||||
- run tests on tag 3.5.0
|
||||
- run tests on tag 3.2
|
||||
|
||||
```sh
|
||||
./tests.sh -t 3.5.0
|
||||
```
|
||||
./tests.sh -t 3.2
|
||||
```
|
||||
|
||||
- run tests on tag 3.5.0 interactively (pause before and after tests)
|
||||
- run tests on tag 3.2 interactively (pause before and after tests)
|
||||
|
||||
```sh
|
||||
./tests.sh -t 3.5.0 -i
|
||||
```
|
||||
./tests.sh -t 3.2 -i
|
||||
```
|
||||
|
||||
- run tests on tags 3.5.0 and 2.7
|
||||
- run tests on tags 3.2 and 2.7
|
||||
|
||||
```sh
|
||||
./tests.sh -t 3.5.0 -t 2.7
|
||||
```
|
||||
|
||||
For Linux there are also functional tests for all command line options.
|
||||
|
||||
- run all functional tests on OpenRefine 3.5.0
|
||||
|
||||
```sh
|
||||
./tests-cli.sh 3.5.0
|
||||
```
|
||||
|
||||
- run all functional tests on OpenRefine 3.5.0 with one-file-executable
|
||||
|
||||
```sh
|
||||
./tests-cli.sh 3.5.0 openrefine-client_0-3-7_linux
|
||||
./tests.sh -t 3.2 -t 2.7
|
||||
```
|
||||
|
||||
### Distributing
|
||||
|
||||
Note to myself: When releasing a new version...
|
||||
|
||||
1. Run functional tests
|
||||
1. Run tests
|
||||
|
||||
```sh
|
||||
for v in 2.7 2.8 3.0 3.1 3.2 3.3 3.4 3.4.1 3.5.0; do
|
||||
./tests-cli.sh $v
|
||||
done
|
||||
```
|
||||
./tests.sh -a
|
||||
```
|
||||
|
||||
2. Make final changes in Git
|
||||
2. Make final changes in GitHub
|
||||
|
||||
- update versions (e.g. 0.3.7 und 0-3-7) in [README.md](https://github.com/opencultureconsulting/openrefine-client/blob/master/README.md#download)
|
||||
- update version in [setup.py](https://github.com/opencultureconsulting/openrefine-client/blob/master/setup.py)
|
||||
- update versions and download links (guess in advance) in [README.md](https://github.com/opencultureconsulting/openrefine-client/blob/master/README.md#download)
|
||||
- check if [Dockerfile](https://github.com/opencultureconsulting/openrefine-client/blob/master/docker/Dockerfile) needs to be changed
|
||||
|
||||
3. Build executables with PyInstaller
|
||||
@ -712,47 +662,40 @@ Note to myself: When releasing a new version...
|
||||
- Run PyInstaller in Python 2 environments on native Windows, macOS and Linux. Should be "the oldest version of the OS you need to support"! Current release is built with:
|
||||
|
||||
- Ubuntu 16.04 LTS (64-bit)
|
||||
- macOS Sierra 10.12 (64-bit)
|
||||
- Windows 7 (32-bit)
|
||||
- macOS Sierra 10.12
|
||||
- Windows 10
|
||||
|
||||
- One-file-executables will be available in `dist/`.
|
||||
|
||||
```sh
|
||||
```
|
||||
git clone https://github.com/opencultureconsulting/openrefine-client.git
|
||||
cd openrefine-client
|
||||
python2 -m pip install pyinstaller --user
|
||||
python2 -m pip install urllib2_file --user
|
||||
python2 -m PyInstaller --onefile refine.py --hidden-import google.refine.__main__
|
||||
python -m pip install . --user
|
||||
python -m pip install pyinstaller --user
|
||||
pyinstaller --onefile refine.py --hidden-import google.refine.__main__
|
||||
```
|
||||
|
||||
4. Run functional tests with Linux executable
|
||||
|
||||
```sh
|
||||
for v in 2.7 2.8 3.0 3.1 3.2 3.3 3.4 3.4.1 3.5.0; do
|
||||
./tests-cli.sh $v openrefine-client_0-3-7_linux
|
||||
done
|
||||
```
|
||||
|
||||
5. Create release in GitHub
|
||||
4. Create release in GitHub
|
||||
|
||||
- draft [release notes](https://github.com/opencultureconsulting/openrefine-client/releases) and attach one-file-executables
|
||||
|
||||
6. Build package and upload to PyPI
|
||||
5. Build package and upload to PyPI
|
||||
|
||||
```sh
|
||||
```
|
||||
python3 setup.py sdist bdist_wheel
|
||||
python3 -m twine upload dist/*
|
||||
```
|
||||
|
||||
7. Update Docker container
|
||||
6. Update Docker container
|
||||
|
||||
- add new autobuild for release version
|
||||
- trigger latest build
|
||||
|
||||
8. Bump openrefine-client version in related projects
|
||||
7. Bump openrefine-client version in related projects
|
||||
|
||||
- openrefine-batch: [openrefine-batch.sh](https://github.com/opencultureconsulting/openrefine-batch/blob/master/openrefine-batch.sh#L7) and [openrefine-batch-docker.sh](https://github.com/opencultureconsulting/openrefine-batch/blob/master/openrefine-batch-docker.sh)
|
||||
- openrefineder: [postBuild](https://github.com/felixlohmeier/openrefineder/blob/master/binder/postBuild)
|
||||
|
||||
- openrefineder: [postBuild](https://github.com/betatim/openrefineder/blob/master/postBuild)
|
||||
|
||||
## Credits
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
FROM alpine:3.11
|
||||
FROM alpine:latest
|
||||
LABEL maintainer="felixlohmeier@opencultureconsulting.com"
|
||||
# The OpenRefine Python Client Library from PaulMakepeace provides an interface to communicating with an OpenRefine server. This fork extends the command line interface (CLI) and supports communication between docker containers.
|
||||
# Source: https://github.com/opencultureconsulting/openrefine-client
|
||||
|
||||
@ -138,14 +138,14 @@ group5.add_option('--linesPerRow', dest='linesPerRow',
|
||||
group5.add_option('--processQuotes', dest='processQuotes',
|
||||
metavar='true/false', choices=('true', 'false'),
|
||||
help='(csv,tsv), default: true')
|
||||
group5.add_option('--projectName', dest='projectName',
|
||||
group5.add_option('--projectName', dest='project_name',
|
||||
help='(all formats), default: filename')
|
||||
group5.add_option('--projectTags', dest='projectTags',
|
||||
action='append',
|
||||
help='(all formats), please provide tags in multiple arguments, e.g. --projectTags=beta --projectTags=client1')
|
||||
group5.add_option('--recordPath', dest='recordPath',
|
||||
action='append',
|
||||
help='(xml,json), please provide path in multiple arguments, e.g. /collection/record/ should be entered: --recordPath=collection --recordPath=record, default xml: root element, default json: _ _')
|
||||
help='(xml,json), please provide path in multiple arguments without slashes, e.g. /collection/record/ should be entered like this: --recordPath=collection --recordPath=record, default xml: record, default json: _ _')
|
||||
group5.add_option('--separator', dest='separator',
|
||||
help='(csv,tsv), default csv: , default tsv: \\t')
|
||||
group5.add_option('--sheets', dest='sheets',
|
||||
@ -213,7 +213,7 @@ def main():
|
||||
projects = refine.Refine(refine.RefineServer()).list_projects().items()
|
||||
idlist = []
|
||||
for project_id, project_info in projects:
|
||||
if args[0].decode('UTF-8') == project_info['name']:
|
||||
if args[0] == project_info['name']:
|
||||
idlist.append(str(project_id))
|
||||
if len(idlist) > 1:
|
||||
print('Error: Found %s projects with name %s.\n'
|
||||
|
||||
@ -25,7 +25,6 @@ import ssl
|
||||
import sys
|
||||
import time
|
||||
import urllib
|
||||
from xml.etree import ElementTree
|
||||
|
||||
from google.refine import refine
|
||||
|
||||
@ -41,8 +40,10 @@ def apply(project_id, history_file):
|
||||
print('File %s has been successfully applied to project %s' %
|
||||
(history_file, project_id))
|
||||
|
||||
|
||||
def create(project_file,
|
||||
project_format=None,
|
||||
project_name=None,
|
||||
columnWidths=None,
|
||||
encoding=None,
|
||||
guessCellValueTypes=False,
|
||||
@ -53,7 +54,6 @@ def create(project_file,
|
||||
linesPerRow=None,
|
||||
processQuotes=True,
|
||||
projectName=None,
|
||||
projectTags=None,
|
||||
recordPath=None,
|
||||
separator=None,
|
||||
sheets=None,
|
||||
@ -69,15 +69,15 @@ def create(project_file,
|
||||
project_format = os.path.splitext(project_file)[1][1:].lower()
|
||||
if project_format == 'txt':
|
||||
try:
|
||||
columnWidths[0]
|
||||
columnWidths
|
||||
project_format = 'fixed-width'
|
||||
except TypeError:
|
||||
except NameError:
|
||||
project_format = 'line-based'
|
||||
# defaults for each file type
|
||||
if project_format == 'xml':
|
||||
project_format = 'text/xml'
|
||||
if not recordPath:
|
||||
recordPath = [ElementTree.parse(project_file).getroot().tag]
|
||||
recordPath = 'record'
|
||||
elif project_format == 'csv':
|
||||
project_format = 'text/line-based/*sv'
|
||||
elif project_format == 'tsv':
|
||||
@ -95,35 +95,22 @@ def create(project_file,
|
||||
elif project_format == 'json':
|
||||
project_format = 'text/json'
|
||||
if not recordPath:
|
||||
recordPath = ['_', '_']
|
||||
recordPath = ('_', '_')
|
||||
elif project_format == 'xls':
|
||||
project_format = 'binary/text/xml/xls/xlsx'
|
||||
if not sheets:
|
||||
sheets = [0]
|
||||
# TODO: new format for sheets option introduced in OpenRefine 2.8
|
||||
sheets = 0
|
||||
elif project_format == 'xlsx':
|
||||
project_format = 'binary/text/xml/xls/xlsx'
|
||||
if not sheets:
|
||||
sheets = [0]
|
||||
# TODO: new format for sheets option introduced in OpenRefine 2.8
|
||||
sheets = 0
|
||||
elif project_format == 'ods':
|
||||
project_format = 'text/xml/ods'
|
||||
if not sheets:
|
||||
sheets = [0]
|
||||
# TODO: new format for sheets option introduced in OpenRefine 2.8
|
||||
sheets = 0
|
||||
# execute
|
||||
kwargs = {k: v for k, v in vars().items() if v is not None}
|
||||
project = refine.Refine(refine.RefineServer()).new_project(
|
||||
guess_cell_value_types=guessCellValueTypes,
|
||||
ignore_lines=ignoreLines,
|
||||
header_lines=headerLines,
|
||||
skip_data_lines=skipDataLines,
|
||||
store_blank_rows=storeBlankRows,
|
||||
process_quotes=processQuotes,
|
||||
project_name=projectName,
|
||||
store_blank_cells_as_nulls=storeBlankCellsAsNulls,
|
||||
include_file_sources=includeFileSources,
|
||||
**kwargs)
|
||||
project = refine.Refine(refine.RefineServer()).new_project(**kwargs)
|
||||
rows = project.do_json('get-rows')['total']
|
||||
if rows > 0:
|
||||
print('{0}: {1}'.format('id', project.project_id))
|
||||
@ -161,46 +148,43 @@ def download(url, output_file=None):
|
||||
print('Download to file %s complete' % output_file)
|
||||
|
||||
|
||||
def export(project_id, encoding=None, output_file=None, export_format=None):
|
||||
def export(project_id, output_file=None, export_format=None):
|
||||
"""Dump a project to stdout or file."""
|
||||
project = refine.RefineProject(project_id)
|
||||
if not export_format:
|
||||
export_format = 'tsv'
|
||||
if not output_file:
|
||||
if not export_format:
|
||||
export_format = 'tsv'
|
||||
if export_format in ['csv', 'tsv', 'txt']:
|
||||
encoding = 'UTF-8'
|
||||
sys.stdout.write(project.export(
|
||||
export_format=export_format, encoding=encoding).read())
|
||||
sys.stdout.write(project.export(export_format=export_format).read())
|
||||
else:
|
||||
ext = os.path.splitext(output_file)[1][1:]
|
||||
if ext and not export_format:
|
||||
if ext:
|
||||
export_format = ext.lower()
|
||||
if not export_format:
|
||||
export_format = 'tsv'
|
||||
if export_format in ['csv', 'tsv', 'txt']:
|
||||
encoding = 'UTF-8'
|
||||
with open(output_file, 'wb') as f:
|
||||
f.write(project.export(
|
||||
export_format=export_format, encoding=encoding).read())
|
||||
f.write(project.export(export_format).read())
|
||||
print('Export to file %s complete' % output_file)
|
||||
|
||||
|
||||
def info(project_id):
|
||||
"""Show project metadata"""
|
||||
projects = refine.Refine(refine.RefineServer()).list_projects()
|
||||
if project_id in projects.keys():
|
||||
print('{0:>20}: {1}'.format('id', project_id))
|
||||
print('{0:>20}: {1}'.format('url', 'http://' +
|
||||
refine.REFINE_HOST + ':' +
|
||||
refine.REFINE_PORT +
|
||||
'/project?project=' + project_id))
|
||||
for k, v in projects[project_id].items():
|
||||
if v:
|
||||
print(u'{0:>20}: {1}'.format(k, v))
|
||||
project_model = refine.RefineProject(project_id).get_models()
|
||||
columns = [c['name'] for c in project_model['columnModel']['columns']]
|
||||
projects = refine.Refine(refine.RefineServer()).list_projects().items()
|
||||
if project_id in [item[0] for item in projects]:
|
||||
project = refine.RefineProject(project_id)
|
||||
projects = refine.Refine(refine.RefineServer()).list_projects().items()
|
||||
for projects_id, projects_info in projects:
|
||||
if project_id == projects_id:
|
||||
print('{0:>20}: {1}'.format('id', project_id))
|
||||
print('{0:>20}: {1}'.format('url', 'http://' +
|
||||
refine.REFINE_HOST + ':' +
|
||||
refine.REFINE_PORT +
|
||||
'/project?project=' + project_id))
|
||||
for k, v in projects_info.items():
|
||||
if v:
|
||||
print('{0:>20}: {1}'.format(k, v))
|
||||
project_model = project.get_models()
|
||||
column_model = project_model['columnModel']
|
||||
columns = [column['name'] for column in column_model['columns']]
|
||||
for (i, v) in enumerate(columns, start=1):
|
||||
print(u'{0:>20}: {1}'.format(u'column ' + str(i).zfill(3), v).encode('utf-8'))
|
||||
print('{0:>20}: {1}'.format('column ' + str(i).zfill(3), v))
|
||||
else:
|
||||
print('Error: No project found with id %s.\n'
|
||||
'Check existing projects with command --list' % (project_id))
|
||||
@ -216,14 +200,13 @@ def ls():
|
||||
projects.sort(key=lambda v: date_to_epoch(v[1]['modified']), reverse=True)
|
||||
if projects:
|
||||
for project_id, project_info in projects:
|
||||
print(u'{0:>14}: {1}'.format(project_id, project_info['name']).encode('utf-8'))
|
||||
print('{0:>14}: {1}'.format(project_id, project_info['name']))
|
||||
else:
|
||||
print('Error: No projects found')
|
||||
|
||||
|
||||
def templating(project_id,
|
||||
template,
|
||||
encoding='UTF-8',
|
||||
output_file=None,
|
||||
mode=None,
|
||||
prefix='',
|
||||
@ -242,8 +225,7 @@ def templating(project_id,
|
||||
templateconfig = {'prefix': prefix,
|
||||
'suffix': suffix,
|
||||
'template': template,
|
||||
'rowSeparator': rowSeparator,
|
||||
'encoding': encoding}
|
||||
'rowSeparator': rowSeparator}
|
||||
|
||||
# construct the engine config
|
||||
if mode == 'record-based':
|
||||
@ -264,8 +246,8 @@ def templating(project_id,
|
||||
engine['facets'].append(textFilter)
|
||||
templateconfig.update({'engine': json.dumps(engine)})
|
||||
|
||||
# normal output or some refinable magic for splitToFiles functionality
|
||||
if not splitToFiles:
|
||||
# normal output
|
||||
if not output_file:
|
||||
sys.stdout.write(project.export_templating(
|
||||
**templateconfig).read())
|
||||
@ -274,10 +256,11 @@ def templating(project_id,
|
||||
f.write(project.export_templating(**templateconfig).read())
|
||||
print('Export to file %s complete' % output_file)
|
||||
else:
|
||||
# splitToFiles functionality
|
||||
# common config for row-based and record-based
|
||||
prefix = templateconfig['prefix']
|
||||
suffix = templateconfig['suffix']
|
||||
split = '===|||THISISTHEBEGINNINGOFANEWRECORD|||==='
|
||||
keyColumn = project.get_models()['columnModel']['keyColumnName']
|
||||
if not output_file:
|
||||
output_file = time.strftime('%Y%m%d')
|
||||
else:
|
||||
@ -285,24 +268,23 @@ def templating(project_id,
|
||||
ext = os.path.splitext(output_file)[1][1:]
|
||||
if not ext:
|
||||
ext = 'txt'
|
||||
# generate config for subfeature suffixById
|
||||
if suffixById:
|
||||
ids_template = ('{{forNonBlank(' +
|
||||
'with(row.columnNames[0],cn,cells[cn].value),' +
|
||||
'v,v,"")}}')
|
||||
ids_template = ('{{forNonBlank(cells["' +
|
||||
keyColumn +
|
||||
'"].value, v, v, "")}}')
|
||||
ids_templateconfig = {'engine': json.dumps(engine),
|
||||
'template': ids_template,
|
||||
'rowSeparator': '\n',
|
||||
'encoding': encoding}
|
||||
'rowSeparator': '\n'}
|
||||
ids = [line.rstrip('\n') for line in project.export_templating(
|
||||
**ids_templateconfig) if line.rstrip('\n')]
|
||||
# generate common config
|
||||
if mode == 'record-based':
|
||||
# record-based: split-character into template
|
||||
# if key column is not blank (=record)
|
||||
template = ('{{forNonBlank(' +
|
||||
'with(row.columnNames[0],cn,cells[cn].value),' +
|
||||
'v,"' + split + '", "")}}' +
|
||||
template = ('{{forNonBlank(cells["' +
|
||||
keyColumn +
|
||||
'"].value, v, "' +
|
||||
split +
|
||||
'", "")}}' +
|
||||
templateconfig['template'])
|
||||
templateconfig.update({'prefix': '',
|
||||
'suffix': '',
|
||||
@ -315,7 +297,6 @@ def templating(project_id,
|
||||
'suffix': '',
|
||||
'template': template,
|
||||
'rowSeparator': ''})
|
||||
# execute
|
||||
records = project.export_templating(
|
||||
**templateconfig).read().split(split)
|
||||
del records[0] # skip first blank entry
|
||||
|
||||
@ -53,18 +53,6 @@ class RefineServer(object):
|
||||
server = self.url()
|
||||
self.server = server[:-1] if server.endswith('/') else server
|
||||
self.__version = None # see version @property below
|
||||
self.token = None # CSRF token introduced in OpenRefine 3.3
|
||||
self.get_csrf_token()
|
||||
|
||||
def get_csrf_token(self):
|
||||
"""Return csrf token."""
|
||||
try:
|
||||
url = self.server + '/command/core/get-csrf-token'
|
||||
response = json.loads(urllib2.urlopen(url).read())
|
||||
self.token = response['token']
|
||||
return self.token
|
||||
except:
|
||||
pass # fail silently to not disturb usage of OpenRefine <3.3
|
||||
|
||||
def urlopen(self, command, data=None, params=None, project_id=None):
|
||||
"""Open a Refine URL and with optional query params and POST data.
|
||||
@ -85,9 +73,6 @@ class RefineServer(object):
|
||||
data['project'] = project_id
|
||||
else:
|
||||
params['project'] = project_id
|
||||
# be lazy and send the token for each API call (even when not needed)
|
||||
if self.token:
|
||||
params['csrf_token'] = self.token
|
||||
if params:
|
||||
url += '?' + urllib.urlencode(params)
|
||||
req = urllib2.Request(url)
|
||||
@ -97,8 +82,7 @@ class RefineServer(object):
|
||||
try:
|
||||
response = urllib2.urlopen(req)
|
||||
except urllib2.HTTPError as e:
|
||||
raise Exception('HTTP %d "%s" for %s\n\t%s' %
|
||||
(e.code, e.msg, e.geturl(), data))
|
||||
raise Exception('HTTP %d "%s" for %s\n\t%s' % (e.code, e.msg, e.geturl(), data))
|
||||
except urllib2.URLError as e:
|
||||
raise urllib2.URLError(
|
||||
'%s for %s. No Refine server reachable/running; ENV set?' %
|
||||
@ -114,10 +98,6 @@ class RefineServer(object):
|
||||
"""Open a Refine URL, optionally POST data, and return parsed JSON."""
|
||||
response = json.loads(self.urlopen(*args, **kwargs).read())
|
||||
if 'code' in response and response['code'] not in ('ok', 'pending'):
|
||||
if 'Missing or invalid csrf_token parameter' == response['message']:
|
||||
self.get_csrf_token()
|
||||
response = json.loads(self.urlopen(*args, **kwargs).read())
|
||||
return response
|
||||
error_message = ('server ' + response['code'] + ': ' +
|
||||
response.get('message', response.get('stack', response)))
|
||||
raise Exception(error_message)
|
||||
@ -167,127 +147,41 @@ class Refine:
|
||||
"""Open a Refine project."""
|
||||
return RefineProject(self.server, project_id)
|
||||
|
||||
# These aren't used yet but are included for reference
|
||||
new_project_defaults = {
|
||||
'text/line-based/*sv': {
|
||||
'encoding': '',
|
||||
'separator': ',',
|
||||
'ignore_lines': -1,
|
||||
'header_lines': 1,
|
||||
'skip_data_lines': 0,
|
||||
'limit': -1,
|
||||
'store_blank_rows': True,
|
||||
'guess_cell_value_types': True,
|
||||
'process_quotes': True,
|
||||
'store_blank_cells_as_nulls': True,
|
||||
'include_file_sources': False},
|
||||
'text/line-based': {
|
||||
'encoding': '',
|
||||
'lines_per_row': 1,
|
||||
'ignore_lines': -1,
|
||||
'limit': -1,
|
||||
'skip_data_lines': -1,
|
||||
'store_blank_rows': True,
|
||||
'store_blank_cells_as_nulls': True,
|
||||
'include_file_sources': False},
|
||||
'text/line-based/fixed-width': {
|
||||
'encoding': '',
|
||||
'column_widths': [20],
|
||||
'ignore_lines': -1,
|
||||
'header_lines': 0,
|
||||
'skip_data_lines': 0,
|
||||
'limit': -1,
|
||||
'guess_cell_value_types': False,
|
||||
'store_blank_rows': True,
|
||||
'store_blank_cells_as_nulls': True,
|
||||
'include_file_sources': False},
|
||||
'text/line-based/pc-axis': {
|
||||
'encoding': '',
|
||||
'limit': -1,
|
||||
'skip_data_lines': -1,
|
||||
'include_file_sources': False},
|
||||
'text/rdf+n3': {'encoding': ''},
|
||||
'text/xml/ods': {
|
||||
'sheets': [],
|
||||
'ignore_lines': -1,
|
||||
'header_lines': 1,
|
||||
'skip_data_lines': 0,
|
||||
'limit': -1,
|
||||
'store_blank_rows': True,
|
||||
'store_blank_cells_as_nulls': True,
|
||||
'include_file_sources': False},
|
||||
'binary/xls': {
|
||||
'xml_based': False,
|
||||
'sheets': [],
|
||||
'ignore_lines': -1,
|
||||
'header_lines': 1,
|
||||
'skip_data_lines': 0,
|
||||
'limit': -1,
|
||||
'store_blank_rows': True,
|
||||
'store_blank_cells_as_nulls': True,
|
||||
'include_file_sources': False}
|
||||
}
|
||||
def new_project(self, project_file=None, project_name=None,
|
||||
project_format='text/line-based/*sv', **kwargs):
|
||||
"""Create a Refine project."""
|
||||
|
||||
def new_project(self, project_file=None, project_url=None, project_name=None, project_format='text/line-based/*sv',
|
||||
encoding='',
|
||||
separator=',',
|
||||
ignore_lines=-1,
|
||||
header_lines=1,
|
||||
skip_data_lines=0,
|
||||
limit=-1,
|
||||
store_blank_rows=True,
|
||||
guess_cell_value_types=False,
|
||||
process_quotes=True,
|
||||
store_blank_cells_as_nulls=True,
|
||||
include_file_sources=False,
|
||||
**opts):
|
||||
defaults = {'guessCellValueTypes': False,
|
||||
'headerLines': 1,
|
||||
'ignoreLines': -1,
|
||||
'includeFileSources': False,
|
||||
'limit': -1,
|
||||
'linesPerRow': 1,
|
||||
'processQuotes': True,
|
||||
'separator': ',',
|
||||
'skipDataLines': 0,
|
||||
'storeBlankCellsAsNulls': True,
|
||||
'storeBlankRows': True,
|
||||
'storeEmptyStrings': True,
|
||||
'trimStrings': False}
|
||||
|
||||
if (project_file and project_url) or (not project_file and not project_url):
|
||||
raise ValueError('One (only) of project_file and project_url must be set')
|
||||
|
||||
def s(opt):
|
||||
if isinstance(opt, bool):
|
||||
return 'true' if opt else 'false'
|
||||
if opt is None:
|
||||
return ''
|
||||
return str(opt)
|
||||
|
||||
# the new APIs requires a json in the 'option' POST or GET argument
|
||||
# POST is broken at the moment, so we send it in the URL
|
||||
new_style_options = dict(opts, **{
|
||||
'encoding': s(encoding),
|
||||
'separator': s(separator)
|
||||
})
|
||||
params = {
|
||||
'options': json.dumps(new_style_options),
|
||||
}
|
||||
|
||||
# old style options
|
||||
options = {
|
||||
'format': project_format,
|
||||
'ignore-lines': s(ignore_lines),
|
||||
'header-lines': s(header_lines),
|
||||
'skip-data-lines': s(skip_data_lines),
|
||||
'limit': s(limit),
|
||||
'guess-value-type': s(guess_cell_value_types),
|
||||
'process-quotes': s(process_quotes),
|
||||
'store-blank-rows': s(store_blank_rows),
|
||||
'store-blank-cells-as-nulls': s(store_blank_cells_as_nulls),
|
||||
'include-file-sources': s(include_file_sources)
|
||||
}
|
||||
|
||||
if project_url is not None:
|
||||
options['url'] = project_url
|
||||
elif project_file is not None:
|
||||
options['project-file'] = {
|
||||
'fd': open(project_file),
|
||||
'filename': project_file,
|
||||
}
|
||||
# options
|
||||
options = {'format': project_format}
|
||||
if project_file is not None:
|
||||
options['project-file'] = {'fd': open(project_file),
|
||||
'filename': project_file}
|
||||
if project_name is None:
|
||||
# make a name for itself by stripping extension and directories
|
||||
project_name = (project_file or 'New project').rsplit('.', 1)[0]
|
||||
project_name = os.path.basename(project_name)
|
||||
options['project-name'] = project_name
|
||||
|
||||
# params
|
||||
params_dict = dict(defaults)
|
||||
params_dict.update(kwargs)
|
||||
params = {'options': json.dumps(params_dict)}
|
||||
|
||||
# submit
|
||||
response = self.server.urlopen(
|
||||
'create-project-from-upload', options, params
|
||||
)
|
||||
@ -418,10 +312,7 @@ class RefineProject:
|
||||
for i, column in enumerate(column_model['columns']):
|
||||
name = column['name']
|
||||
self.column_order[name] = i
|
||||
try:
|
||||
column_index[name] = column['cellIndex']
|
||||
except KeyError:
|
||||
column_index[name] = i
|
||||
column_index[name] = column['cellIndex']
|
||||
self.key_column = column_model['keyColumnName']
|
||||
self.has_records = response['recordModel'].get('hasRecords', False)
|
||||
self.rows_response_factory = RowsResponseFactory(column_index)
|
||||
@ -450,31 +341,23 @@ class RefineProject:
|
||||
return 'ok'
|
||||
return response_json['code'] # can be 'ok' or 'pending'
|
||||
|
||||
def export(self, encoding=None, export_format='tsv'):
|
||||
def export(self, export_format='tsv'):
|
||||
"""Return a fileobject of a project's data."""
|
||||
url = ('export-rows/' +
|
||||
urllib.quote(self.project_name().encode('utf8')) +
|
||||
'.' + export_format)
|
||||
data = {'format': export_format}
|
||||
if encoding:
|
||||
data['encoding'] = encoding
|
||||
return self.do_raw(url, data)
|
||||
url = ('export-rows/' + urllib.quote(self.project_name()) + '.' +
|
||||
export_format)
|
||||
return self.do_raw(url, data={'format': export_format})
|
||||
|
||||
def export_templating(self, encoding=None, engine='', prefix='',
|
||||
def export_templating(self, engine='', prefix='',
|
||||
template='', rowSeparator='\n', suffix=''):
|
||||
"""Return a fileobject of a project's data in templating mode."""
|
||||
url = ('export-rows/' +
|
||||
urllib.quote(self.project_name().encode('utf8')) +
|
||||
'.' + 'txt')
|
||||
data = {'format': 'template',
|
||||
'template': template,
|
||||
'engine': engine,
|
||||
'prefix': prefix,
|
||||
'suffix': suffix,
|
||||
'separator': rowSeparator}
|
||||
if encoding:
|
||||
data['encoding'] = encoding
|
||||
return self.do_raw(url, data)
|
||||
url = ('export-rows/' + urllib.quote(self.project_name()) + '.' +
|
||||
'txt')
|
||||
return self.do_raw(url, data={'format': 'template',
|
||||
'template': template,
|
||||
'engine': engine,
|
||||
'prefix': prefix,
|
||||
'suffix': suffix,
|
||||
'separator': rowSeparator})
|
||||
|
||||
def export_rows(self, **kwargs):
|
||||
"""Return an iterable of parsed rows of a project's data."""
|
||||
|
||||
10
refine.py
10
refine.py
@ -18,18 +18,8 @@ Script to provide a command line interface to a Refine server.
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with this program. If not, see <http://www.gnu.org/licenses/>
|
||||
|
||||
import sys
|
||||
|
||||
from google.refine import __main__, cli, refine
|
||||
|
||||
# workaround for pyinstaller
|
||||
if getattr(sys, 'frozen', False) and hasattr(sys, '_MEIPASS'):
|
||||
reload(sys)
|
||||
sys.setdefaultencoding('utf-8')
|
||||
if sys.platform == "win32":
|
||||
import codecs
|
||||
codecs.register(lambda name: codecs.lookup(
|
||||
'utf-8') if name == 'cp65001' else None)
|
||||
|
||||
if __name__ == '__main__':
|
||||
__main__.main()
|
||||
|
||||
2
setup.py
2
setup.py
@ -25,7 +25,7 @@ def read(filename):
|
||||
return open(os.path.join(os.path.dirname(__file__), filename)).read()
|
||||
|
||||
setup(name='openrefine-client',
|
||||
version='0.3.10',
|
||||
version='0.3.7',
|
||||
description=('The OpenRefine Python Client Library provides an '
|
||||
'interface to communicating with an OpenRefine server. '
|
||||
'This fork extends the command line interface (CLI).'),
|
||||
|
||||
123
tests-cli.sh
123
tests-cli.sh
@ -1,123 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Script for running functional tests against the CLI
|
||||
|
||||
# Copyright (c) 2011 Paul Makepeace, Real Programmers. All rights reserved.
|
||||
|
||||
# This program is free software: you can redistribute it and/or modify
|
||||
# it under the terms of the GNU General Public License as published by
|
||||
# the Free Software Foundation, either version 3 of the License, or
|
||||
# (at your option) any later version.
|
||||
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU General Public License for more details.
|
||||
|
||||
# You should have received a copy of the GNU General Public License
|
||||
# along with this program. If not, see <http://www.gnu.org/licenses/>
|
||||
|
||||
# ================================== CONFIG ================================== #
|
||||
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
|
||||
port=3334
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
version="${1}"
|
||||
else
|
||||
version="3.2"
|
||||
fi
|
||||
refine="openrefine-${version}/refine"
|
||||
|
||||
if [[ ${2} ]]; then
|
||||
client="$(readlink -e "${2}")"
|
||||
else
|
||||
client="python2 $(readlink -e refine.py)"
|
||||
fi
|
||||
cmd="${client} -H localhost -P ${port}"
|
||||
|
||||
if [[ ${3} ]]; then
|
||||
filename="${3%%.*}"
|
||||
else
|
||||
filename=""
|
||||
fi
|
||||
cmd="${client} -H localhost -P ${port}"
|
||||
|
||||
# =============================== REQUIREMENTS =============================== #
|
||||
|
||||
# check existence of java and cURL
|
||||
if [[ -z "$(command -v java 2> /dev/null)" ]] ; then
|
||||
echo 1>&2 "ERROR: OpenRefine requires JAVA runtime environment (jre)" \
|
||||
"https://openjdk.java.net/install/"
|
||||
exit 1
|
||||
fi
|
||||
if [[ -z "$(command -v curl 2> /dev/null)" ]] ; then
|
||||
echo 1>&2 "ERROR: This shell script requires cURL" \
|
||||
"https://curl.haxx.se/download.html"
|
||||
exit 1
|
||||
fi
|
||||
# download OpenRefine
|
||||
if [[ -z "$(readlink -e "${refine}")" ]]; then
|
||||
echo "Download OpenRefine ${version}..."
|
||||
mkdir -p "$(dirname "${refine}")"
|
||||
curl -L --output openrefine.tar.gz \
|
||||
"https://github.com/OpenRefine/OpenRefine/releases/download/${version}/openrefine-linux-${version}.tar.gz"
|
||||
echo "Install OpenRefine ${version} in subdirectory $(dirname "${refine}")..."
|
||||
tar -xzf openrefine.tar.gz -C "$(dirname "${refine}")" --strip 1 --totals
|
||||
rm -f openrefine.tar.gz
|
||||
# do not try to open OpenRefine in browser
|
||||
sed -i '$ a JAVA_OPTIONS=-Drefine.headless=true' \
|
||||
"$(dirname "${refine}")"/refine.ini
|
||||
# set autosave period from 5 minutes to 25 hours
|
||||
sed -i 's/#REFINE_AUTOSAVE_PERIOD=60/REFINE_AUTOSAVE_PERIOD=1500/' \
|
||||
"$(dirname "${refine}")"/refine.ini
|
||||
echo
|
||||
fi
|
||||
|
||||
# ================================== SETUP =================================== #
|
||||
|
||||
dir="$(readlink -f "tests/tmp")"
|
||||
mkdir -p "${dir}"
|
||||
rm -f tests-cli.log
|
||||
|
||||
echo "start OpenRefine ${version}..."
|
||||
${refine} -v warn -p ${port} -d "${dir}" &>> tests-cli.log &
|
||||
pid_server=${!}
|
||||
timeout 30s bash -c "until curl -s 'http://localhost:3334' \
|
||||
| cat | grep -q -o 'OpenRefine' ; do sleep 1; done" \
|
||||
|| error "starting OpenRefine server failed!"
|
||||
echo
|
||||
|
||||
# ================================== TESTS =================================== #
|
||||
|
||||
echo "running tests, please wait..."
|
||||
tests=()
|
||||
results=()
|
||||
for t in tests/*${filename}*.sh; do
|
||||
tests+=("${t}")
|
||||
echo "======================= ${t} =======================" &>> tests-cli.log
|
||||
bash "${t}" "${cmd}" "${version}" &>> tests-cli.log
|
||||
results+=(${?})
|
||||
done
|
||||
echo
|
||||
|
||||
# ================================= TEARDOWN ================================= #
|
||||
|
||||
echo "cleanup..."
|
||||
{ kill -9 "${pid_server}" && wait "${pid_server}"; } 2>/dev/null
|
||||
rm -rf "${dir}"
|
||||
echo
|
||||
|
||||
# ================================= SUMMARY ================================== #
|
||||
|
||||
printf "%s\t%s\n" "code" "test"
|
||||
printf "%s\t%s\n" "----" "----------------"
|
||||
for i in "${!tests[@]}"; do
|
||||
printf "%s\t%s\n" "${results[$i]}" "${tests[$i]}"
|
||||
done
|
||||
echo
|
||||
if [[ " ${results[*]} " =~ [1-9] ]]; then
|
||||
echo "failed tests! check tests-cli.log for debugging"; echo
|
||||
else
|
||||
echo "all tests passed!"; echo
|
||||
fi
|
||||
14
tests.sh
14
tests.sh
@ -17,8 +17,8 @@
|
||||
# along with this program. If not, see <http://www.gnu.org/licenses/>
|
||||
|
||||
# defaults:
|
||||
all=(3.5.0 3.4.1 3.4 3.3 3.2-java12 3.2-java11 3.2-java10 3.2-java9 3.2 3.1-java9 3.1 3.0-java9 3.0 2.8-java9 2.8 2.8-java7 2.7 2.7-java7 2.5-java7 2.5-java6 2.1-java6 2.0-java6)
|
||||
main=(3.5.0 3.4.1 3.4 3.3 3.2 3.1 3.0 2.8 2.7 2.5-java6 2.1-java6 2.0-java6)
|
||||
all=(3.2-java12 3.2-java11 3.2-java10 3.2-java9 3.2 3.1-java9 3.1 3.0-java9 3.0 2.8-java9 2.8 2.8-java7 2.7 2.7-java7 2.5-java7 2.5-java6 2.1-java6 2.0-java6)
|
||||
main=(3.2 3.1 3.0 2.8 2.7 2.5-java6 2.1-java6 2.0-java6)
|
||||
interactively=false
|
||||
port="3333"
|
||||
|
||||
@ -31,10 +31,10 @@ Script for running tests with different OpenRefine and Java versions.
|
||||
It uses docker images from https://hub.docker.com/r/felixlohmeier/openrefine.
|
||||
|
||||
Examples:
|
||||
./tests.sh -a # run tests on all OpenRefine versions (from 2.0 up to 3.5.0)
|
||||
./tests.sh -t 3.5.0 # run tests on tag 3.5.0
|
||||
./tests.sh -t 3.5.0 -i # run tests on tag 3.5.0 interactively (pause before and after tests)
|
||||
./tests.sh -t 3.5.0 -t 2.7 # run tests on tags 3.5.0 and 2.7
|
||||
./tests.sh -a # run tests on all OpenRefine versions (from 2.0 up to 3.2)
|
||||
./tests.sh -t 3.2 # run tests on tag 3.2
|
||||
./tests.sh -t 3.2 -i # run tests on tag 3.2 interactively (pause before and after tests)
|
||||
./tests.sh -t 3.2 -t 2.7 # run tests on tags 3.2 and 2.7
|
||||
|
||||
Advanced:
|
||||
./tests.sh -j # run tests on all OpenRefine versions and each with all supported Java versions (requires a lot of docker images to be downloaded!)
|
||||
@ -122,7 +122,7 @@ for t in ${tags[*]} ; do
|
||||
until curl --silent -N http://localhost:"$port" | cat | grep -q -o "Refine" ; do sleep 1; done
|
||||
echo "Refine running at http://localhost:${port}"
|
||||
if [ $interactively = true ]; then read -r -p "Press [Enter] key to start tests..."; fi
|
||||
python2 setup.py test
|
||||
python setup.py test
|
||||
if [ $interactively = true ]; then read -r -p "Press [Enter] key to stop OpenRefine..."; fi
|
||||
${docker[*]} stop "$t"
|
||||
echo "End: $(date)"
|
||||
|
||||
@ -1,57 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/column-addition",
|
||||
"engineConfig": {
|
||||
"mode": "row-based"
|
||||
},
|
||||
"newColumnName": "apply",
|
||||
"columnInsertIndex": 2,
|
||||
"baseColumnName": "b",
|
||||
"expression": "grel:value.replace('2','⛲')",
|
||||
"onError": "set-to-blank"
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b apply c
|
||||
1 2 ⛲ 3
|
||||
0 0 0 0
|
||||
$ \ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,57 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/column-addition",
|
||||
"engineConfig": {
|
||||
"mode": "row-based"
|
||||
},
|
||||
"newColumnName": "apply",
|
||||
"columnInsertIndex": 2,
|
||||
"baseColumnName": "b",
|
||||
"expression": "grel:value.replace('2','TEST')",
|
||||
"onError": "set-to-blank"
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b apply c
|
||||
1 2 TEST 3
|
||||
0 0 0 0
|
||||
$ \ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,41 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}-utf8.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
ä,é,ß
|
||||
$,\,'
|
||||
DATA
|
||||
iconv -f UTF-8 -t ISO-8859-1 "tmp/${t}/${t}-utf8.csv" > "tmp/${t}/${t}.csv"
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
ä é ß
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --encoding "ISO-8859-1"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
01,02,03
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
1 2 3
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --guessCellValueTypes "true"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,41 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
Column 1 Column 2 Column 3
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --headerLines "0"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,39 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --ignoreLines "1"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,39 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --limit "2"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,41 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,"2,0",3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c Column 4
|
||||
1 2 0 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
# OpenRefine 4.x fails without manually set headerLines
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --processQuotes "false" --headerLines 1
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,45 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
version="${2}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "${version:0:1}" = "2" ]]; then
|
||||
echo "projectTags were introduced in OpenRefine 3.0"
|
||||
exit 200
|
||||
else
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
tags: [u'beta', u'client1']
|
||||
DATA
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --projectTags "beta" --projectTags "client1"
|
||||
${cmd} --info "${t}" | grep ' tags: ' > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a;b;c
|
||||
1;2;3
|
||||
0;0;0
|
||||
$;\;'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --separator ";"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,39 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --skipDataLines "1"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,58 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "b",
|
||||
"expression": "grel:isNull(value)",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 false 3
|
||||
0 false 0
|
||||
$ false '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --storeBlankCellsAsNulls "false"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,39 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
,,
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --storeBlankRows "false"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,FLAG IN HOLE
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
⌨ code meaning
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --projectName "${t} biểu tượng cảm xúc ⛲"
|
||||
${cmd} --export "${t} biểu tượng cảm xúc ⛲" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,55 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.json"
|
||||
{
|
||||
"rows":[
|
||||
{
|
||||
"a":1,
|
||||
"b":2,
|
||||
"c":3
|
||||
},
|
||||
{
|
||||
"a":0,
|
||||
"b":0,
|
||||
"c":0
|
||||
},
|
||||
{
|
||||
"a":"$",
|
||||
"b":"\\",
|
||||
"c":"\""
|
||||
}
|
||||
]
|
||||
}
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
_ - a _ - b _ - c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ """"
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.json" --recordPath "_" --recordPath "rows" --recordPath "_"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,52 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.json"
|
||||
[
|
||||
{
|
||||
"a": 1,
|
||||
"b": 2,
|
||||
"c": 3
|
||||
},
|
||||
{
|
||||
"a": "",
|
||||
"b": "",
|
||||
"c": ""
|
||||
},
|
||||
{
|
||||
"a": "$",
|
||||
"b": "\\",
|
||||
"c": "\""
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
_ - a _ - b _ - c
|
||||
1 2 3
|
||||
$ \ """"
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.json" --storeEmptyStrings "false"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,62 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
version="${2}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.json"
|
||||
[
|
||||
{
|
||||
"a": 1,
|
||||
"b": 2,
|
||||
"c": 3
|
||||
},
|
||||
{
|
||||
"a": "0",
|
||||
"b": " 0",
|
||||
"c": "0 "
|
||||
},
|
||||
{
|
||||
"a": "$",
|
||||
"b": "\\",
|
||||
"c": "\""
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "${version:0:1}" = "2" || "${version}" = "3.0" || "${version}" = "3.1" || "${version}" = "3.2" || "${version}" = "3.3" ]]; then
|
||||
echo "trimStrings option does not work in OpenRefine <=3.3"
|
||||
echo "https://github.com/OpenRefine/OpenRefine/issues/2409"
|
||||
exit 200
|
||||
else
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
_ - a _ - b _ - c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ """"
|
||||
DATA
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.json" --trimStrings "true"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,53 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.json"
|
||||
[
|
||||
{
|
||||
"⌨": "⛲",
|
||||
"code": "1F347",
|
||||
"meaning": "FOUNTAIN"
|
||||
},
|
||||
{
|
||||
"⌨": "⛳",
|
||||
"code": "1F349",
|
||||
"meaning": "FLAG IN HOLE"
|
||||
},
|
||||
{
|
||||
"⌨": "⛵",
|
||||
"code": "1F352",
|
||||
"meaning": "SAILBOAT"
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
_ - ⌨ _ - code _ - meaning
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.json"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,53 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.json"
|
||||
[
|
||||
{
|
||||
"a": 1,
|
||||
"b": 2,
|
||||
"c": 3
|
||||
},
|
||||
{
|
||||
"a": 0,
|
||||
"b": 0,
|
||||
"c": 0
|
||||
},
|
||||
{
|
||||
"a": "$",
|
||||
"b": "\\",
|
||||
"c": "\""
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
_ - a _ - b _ - c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ """"
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.json"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
version="${2}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cp "data/example.ods" "tmp/${t}/${t}.ods"
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "${version}" = "2.7" ]]; then
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
⌨ code meaning Column Column 5 Column 6 Column 7 Column 8
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
else
|
||||
#TODO
|
||||
echo "https://github.com/opencultureconsulting/openrefine-client/issues/4"
|
||||
exit 200
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.ods" --sheets 1
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,48 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
version="${2}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cp "data/example.ods" "tmp/${t}/${t}.ods"
|
||||
#a b c
|
||||
#1 2 3
|
||||
#0 0 0
|
||||
#$ \ '
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "${version}" = "2.7" ]]; then
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c Column Column 5 Column 6 Column 7 Column 8
|
||||
1.0 2.0 3.0
|
||||
0.0 0.0 0.0
|
||||
$ \ '
|
||||
DATA
|
||||
else
|
||||
#TODO
|
||||
echo "https://github.com/opencultureconsulting/openrefine-client/issues/4"
|
||||
exit 200
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.ods"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.tsv"
|
||||
⌨ code meaning
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,FLAG IN HOLE
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.tsv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.csv"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.csv"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.tsv"
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.tsv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.csv"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.csv"
|
||||
@ -1,80 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.txt"
|
||||
1 2 3
|
||||
mon tue wed
|
||||
$2 $300 $1
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "1",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
},
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "2",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
},
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "3",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
1 2 3
|
||||
mon tue wed
|
||||
$2 $300 $1
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.txt" --columnWidths "6" --columnWidths "6" --columnWidths "6" --headerLines "1"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,81 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.txt"
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "Column 1",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
},
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "Column 2",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
},
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "Column 3",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
Column 1 Column 2 Column 3
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.txt" --columnWidths "6" --columnWidths "6" --columnWidths "60"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,81 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.txt"
|
||||
1 2 3
|
||||
mon tue wed
|
||||
$2 $300 $1
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "Column 1",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
},
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "Column 2",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
},
|
||||
{
|
||||
"op": "core/text-transform",
|
||||
"engineConfig": {
|
||||
"facets": [],
|
||||
"mode": "row-based"
|
||||
},
|
||||
"columnName": "Column 3",
|
||||
"expression": "grel:value.trim()",
|
||||
"onError": "keep-original",
|
||||
"repeat": false,
|
||||
"repeatCount": 10
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
Column 1 Column 2 Column 3
|
||||
1 2 3
|
||||
mon tue wed
|
||||
$2 $300 $1
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.txt" --columnWidths "6" --columnWidths "6" --columnWidths "6"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,39 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.txt"
|
||||
mon tue wed
|
||||
$2 $300 $1
|
||||
thu fri sat
|
||||
$70 $20 $50
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
Column 1 Column 2
|
||||
mon tue wed $2 $300 $1
|
||||
thu fri sat $70 $20 $50
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.txt" --linesPerRow "2"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,39 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.txt"
|
||||
1 2 3
|
||||
mon tue wed
|
||||
$2 $300 $1
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
Column 1
|
||||
1 2 3
|
||||
mon tue wed
|
||||
$2 $300 $1
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.txt"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
version="${2}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cp "data/example.xls" "tmp/${t}/${t}.xls"
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "${version}" = "2.7" ]]; then
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
⌨ code meaning
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
else
|
||||
#TODO
|
||||
echo "https://github.com/opencultureconsulting/openrefine-client/issues/4"
|
||||
exit 200
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.xls" --sheets 1
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,48 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
version="${2}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cp "data/example.xls" "tmp/${t}/${t}.xls"
|
||||
#a b c
|
||||
#1 2 3
|
||||
#0 0 0
|
||||
#$ \ '
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "${version}" = "2.7" ]]; then
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1.0 2.0 3.0
|
||||
0.0 0.0 0.0
|
||||
$ \ '
|
||||
DATA
|
||||
else
|
||||
#TODO
|
||||
echo "https://github.com/opencultureconsulting/openrefine-client/issues/4"
|
||||
exit 200
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.xls"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
version="${2}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cp "data/example.xlsx" "tmp/${t}/${t}.xlsx"
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "${version}" = "2.7" ]]; then
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
⌨ code meaning
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
else
|
||||
#TODO
|
||||
echo "https://github.com/opencultureconsulting/openrefine-client/issues/4"
|
||||
exit 200
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.xlsx" --sheets 1
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,48 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
version="${2}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cp "data/example.xlsx" "tmp/${t}/${t}.xlsx"
|
||||
#a b c
|
||||
#1 2 3
|
||||
#0 0 0
|
||||
#$ \ '
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "${version}" = "2.7" ]]; then
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1.0 2.0 3.0
|
||||
0.0 0.0 0.0
|
||||
$ \ '
|
||||
DATA
|
||||
else
|
||||
#TODO
|
||||
echo "https://github.com/opencultureconsulting/openrefine-client/issues/4"
|
||||
exit 200
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.xlsx"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,96 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.xml"
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<root>
|
||||
<record>
|
||||
<a>1</a>
|
||||
<b>2</b>
|
||||
<c>3</c>
|
||||
</record>
|
||||
<record>
|
||||
<a>0</a>
|
||||
<b>0</b>
|
||||
<c>0</c>
|
||||
</record>
|
||||
<record>
|
||||
<a>$</a>
|
||||
<b>\</b>
|
||||
<c>'</c>
|
||||
</record>
|
||||
</root>
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/column-reorder",
|
||||
"columnNames": [
|
||||
"record - a",
|
||||
"record - b",
|
||||
"record - c"
|
||||
],
|
||||
"description": "Reorder columns"
|
||||
},
|
||||
{
|
||||
"op": "core/row-removal",
|
||||
"engineConfig": {
|
||||
"facets": [
|
||||
{
|
||||
"type": "list",
|
||||
"name": "Blank Rows",
|
||||
"expression": "(filter(row.columnNames,cn,isNonBlank(cells[cn].value)).length()==0).toString()",
|
||||
"columnName": "",
|
||||
"invert": false,
|
||||
"omitBlank": false,
|
||||
"omitError": false,
|
||||
"selection": [
|
||||
{
|
||||
"v": {
|
||||
"v": "true",
|
||||
"l": "true"
|
||||
}
|
||||
}
|
||||
],
|
||||
"selectBlank": false,
|
||||
"selectError": false
|
||||
}
|
||||
],
|
||||
"mode": "record-based"
|
||||
}
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
record - a record - b record - c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.xml" --recordPath "root" --recordPath "record"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,96 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.xml"
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<root>
|
||||
<record>
|
||||
<icon>⛲</icon>
|
||||
<code>1F347</code>
|
||||
<meaning>FOUNTAIN</meaning>
|
||||
</record>
|
||||
<record>
|
||||
<icon>⛳</icon>
|
||||
<code>1F349</code>
|
||||
<meaning>FLAG IN HOLE</meaning>
|
||||
</record>
|
||||
<record>
|
||||
<icon>⛵</icon>
|
||||
<code>1F352</code>
|
||||
<meaning>SAILBOAT</meaning>
|
||||
</record>
|
||||
</root>
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/column-reorder",
|
||||
"columnNames": [
|
||||
"root - record - icon",
|
||||
"root - record - code",
|
||||
"root - record - meaning"
|
||||
],
|
||||
"description": "Reorder columns"
|
||||
},
|
||||
{
|
||||
"op": "core/row-removal",
|
||||
"engineConfig": {
|
||||
"facets": [
|
||||
{
|
||||
"type": "list",
|
||||
"name": "Blank Rows",
|
||||
"expression": "(filter(row.columnNames,cn,isNonBlank(cells[cn].value)).length()==0).toString()",
|
||||
"columnName": "",
|
||||
"invert": false,
|
||||
"omitBlank": false,
|
||||
"omitError": false,
|
||||
"selection": [
|
||||
{
|
||||
"v": {
|
||||
"v": "true",
|
||||
"l": "true"
|
||||
}
|
||||
}
|
||||
],
|
||||
"selectBlank": false,
|
||||
"selectError": false
|
||||
}
|
||||
],
|
||||
"mode": "record-based"
|
||||
}
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
root - record - icon root - record - code root - record - meaning
|
||||
⛲ 1F347 FOUNTAIN
|
||||
⛳ 1F349 FLAG IN HOLE
|
||||
⛵ 1F352 SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.xml"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,96 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.xml"
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<root>
|
||||
<record>
|
||||
<a>1</a>
|
||||
<b>2</b>
|
||||
<c>3</c>
|
||||
</record>
|
||||
<record>
|
||||
<a>0</a>
|
||||
<b>0</b>
|
||||
<c>0</c>
|
||||
</record>
|
||||
<record>
|
||||
<a>$</a>
|
||||
<b>\</b>
|
||||
<c>'</c>
|
||||
</record>
|
||||
</root>
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.transform"
|
||||
[
|
||||
{
|
||||
"op": "core/column-reorder",
|
||||
"columnNames": [
|
||||
"root - record - a",
|
||||
"root - record - b",
|
||||
"root - record - c"
|
||||
],
|
||||
"description": "Reorder columns"
|
||||
},
|
||||
{
|
||||
"op": "core/row-removal",
|
||||
"engineConfig": {
|
||||
"facets": [
|
||||
{
|
||||
"type": "list",
|
||||
"name": "Blank Rows",
|
||||
"expression": "(filter(row.columnNames,cn,isNonBlank(cells[cn].value)).length()==0).toString()",
|
||||
"columnName": "",
|
||||
"invert": false,
|
||||
"omitBlank": false,
|
||||
"omitError": false,
|
||||
"selection": [
|
||||
{
|
||||
"v": {
|
||||
"v": "true",
|
||||
"l": "true"
|
||||
}
|
||||
}
|
||||
],
|
||||
"selectBlank": false,
|
||||
"selectError": false
|
||||
}
|
||||
],
|
||||
"mode": "record-based"
|
||||
}
|
||||
}
|
||||
]
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
root - record - a root - record - b root - record - c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.xml"
|
||||
${cmd} --apply "tmp/${t}/${t}.transform" "${t}"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}-1.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}-2.csv"
|
||||
a,b,c
|
||||
4,5,6
|
||||
DATA
|
||||
|
||||
zip "tmp/${t}/${t}.zip" "tmp/${t}/${t}-1.csv" "tmp/${t}/${t}-2.csv"
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << DATA > "tmp/${t}/${t}.assert"
|
||||
File a b c
|
||||
tmp/${t}/${t}-1.csv 1 2 3
|
||||
tmp/${t}/${t}-2.csv 4 5 6
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.zip" --includeFileSources "true"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}-1.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
DATA
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}-2.csv"
|
||||
a,b,c
|
||||
4,5,6
|
||||
DATA
|
||||
|
||||
zip "tmp/${t}/${t}.zip" "tmp/${t}/${t}-1.csv" "tmp/${t}/${t}-2.csv"
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
4 5 6
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.zip"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,11 +0,0 @@
|
||||
email,name,state,gender,purchase,count,date
|
||||
danny.baron@example1.com,Danny Baron,CA,M,TV (UTF-8: 📺),1,"Wed, 4 Jul 2001"
|
||||
melanie.white@example2.edu,Melanie White,NC,F,<iPhone>,1,2001-07-04T12:08:56
|
||||
danny.baron@example1.com, D. ("Tab") Baron,CA,M,Winter jacket,1,2001-07-04
|
||||
ben.tyler@example3.org,Ben Tyler,NV,M,Flashlight,1,2001/07/04
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Dining table,1,2001-07
|
||||
danny.baron@example1.com,Daniel Baron,,,Bike,1,2001
|
||||
jean.griffith@example5.org,Jean Griffith,WA,F,Power drill,1,2000
|
||||
melanie.white@example2.edu,Melanie White,NC,F,'iPad',1,1999
|
||||
ben.morisson@example6.org,Ben Morisson,FL,M,Amplifier,1,1998
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Night table,1,1997
|
||||
|
Can't render this file because it contains an unexpected character in line 4 and column 33.
|
@ -1,92 +0,0 @@
|
||||
[
|
||||
{
|
||||
"email": "danny.baron@example1.com",
|
||||
"name": "Danny Baron",
|
||||
"state": "CA",
|
||||
"gender": "M",
|
||||
"purchase": "TV (UTF-8: 📺)",
|
||||
"count": 1,
|
||||
"date": "Wed, 4 Jul 2001"
|
||||
},
|
||||
{
|
||||
"email": "melanie.white@example2.edu",
|
||||
"name": "Melanie White",
|
||||
"state": "NC",
|
||||
"gender": "F",
|
||||
"purchase": "<iPhone>",
|
||||
"count": 1,
|
||||
"date": "2001-07-04T12:08:56"
|
||||
},
|
||||
{
|
||||
"email": "danny.baron@example1.com",
|
||||
"name": " D.\t(\"Tab\") Baron",
|
||||
"state": "CA",
|
||||
"gender": "M",
|
||||
"purchase": "Winter jacket",
|
||||
"count": 1,
|
||||
"date": "2001-07-04"
|
||||
},
|
||||
{
|
||||
"email": "ben.tyler@example3.org",
|
||||
"name": "Ben Tyler",
|
||||
"state": "NV",
|
||||
"gender": "M",
|
||||
"purchase": "Flashlight",
|
||||
"count": 1,
|
||||
"date": "2001/07/04"
|
||||
},
|
||||
{
|
||||
"email": "arthur.duff@example4.com",
|
||||
"name": "Arthur Duff",
|
||||
"state": "OR",
|
||||
"gender": "M",
|
||||
"purchase": "Dining table",
|
||||
"count": 1,
|
||||
"date": "2001-07"
|
||||
},
|
||||
{
|
||||
"email": "danny.baron@example1.com",
|
||||
"name": "Daniel Baron",
|
||||
"state": "",
|
||||
"gender": "",
|
||||
"purchase": "Bike",
|
||||
"count": 1,
|
||||
"date": 2001
|
||||
},
|
||||
{
|
||||
"email": "jean.griffith@example5.org",
|
||||
"name": "Jean Griffith",
|
||||
"state": "WA",
|
||||
"gender": "F",
|
||||
"purchase": "Power drill",
|
||||
"count": 1,
|
||||
"date": 2000
|
||||
},
|
||||
{
|
||||
"email": "melanie.white@example2.edu",
|
||||
"name": "Melanie White",
|
||||
"state": "NC",
|
||||
"gender": "F",
|
||||
"purchase": "'iPad'",
|
||||
"count": 1,
|
||||
"date": 1999
|
||||
},
|
||||
{
|
||||
"email": "ben.morisson@example6.org",
|
||||
"name": "Ben Morisson",
|
||||
"state": "FL",
|
||||
"gender": "M",
|
||||
"purchase": "Amplifier",
|
||||
"count": 1,
|
||||
"date": 1998
|
||||
},
|
||||
{
|
||||
"email": "arthur.duff@example4.com",
|
||||
"name": "Arthur Duff",
|
||||
"state": "OR",
|
||||
"gender": "M",
|
||||
"purchase": "Night table",
|
||||
"count": 1,
|
||||
"date": 1997
|
||||
}
|
||||
]
|
||||
Binary file not shown.
@ -1,11 +0,0 @@
|
||||
email name state gender purchase count date
|
||||
danny.baron@example1.com Danny Baron CA M TV (UTF-8: 📺) 1 Wed, 4 Jul 2001
|
||||
melanie.white@example2.edu Melanie White NC F <iPhone> 1 2001-07-04T12:08:56
|
||||
danny.baron@example1.com "D. (""Tab"") Baron" CA M Winter jacket 1 2001-07-04
|
||||
ben.tyler@example3.org Ben Tyler NV M Flashlight 1 2001/07/04
|
||||
arthur.duff@example4.com Arthur Duff OR M Dining table 1 2001-07
|
||||
danny.baron@example1.com Daniel Baron Bike 1 2001
|
||||
jean.griffith@example5.org Jean Griffith WA F Power drill 1 2000
|
||||
melanie.white@example2.edu Melanie White NC F 'iPad' 1 1999
|
||||
ben.morisson@example6.org Ben Morisson FL M Amplifier 1 1998
|
||||
arthur.duff@example4.com Arthur Duff OR M Night table 1 1997
|
||||
|
@ -1,11 +0,0 @@
|
||||
email name state gender purchase count date
|
||||
danny.baron@example1.com Danny Baron CA M TV (UTF-8: 📺) 1 Wed, 4 Jul 2001
|
||||
melanie.white@example2.edu Melanie White NC F <iPhone> 1 2001-07-04T12:08:5
|
||||
danny.baron@example1.com D. ("Tab") Baron CA M Winter jacket 1 2001-07-04
|
||||
ben.tyler@example3.org Ben Tyler NV M Flashlight 1 2001/07/04
|
||||
arthur.duff@example4.com Arthur Duff OR M Dining table 1 2001-07
|
||||
danny.baron@example1.com Daniel Baron Bike 1 2001
|
||||
jean.griffith@example5.org Jean Griffith WA F Power drill 1 2000
|
||||
melanie.white@example2.edu Melanie White NC F 'iPad' 1 1999
|
||||
ben.morisson@example6.org Ben Morisson FL M Amplifier 1 1998
|
||||
arthur.duff@example4.com Arthur Duff OR M Night table 1 1997
|
||||
Binary file not shown.
Binary file not shown.
@ -1,93 +0,0 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<root>
|
||||
<record>
|
||||
<email>danny.baron@example1.com</email>
|
||||
<name>Danny Baron</name>
|
||||
<state>CA</state>
|
||||
<gender>M</gender>
|
||||
<purchase>TV (UTF-8: 📺)</purchase>
|
||||
<count>1</count>
|
||||
<date>Wed, 4 Jul 2001</date>
|
||||
</record>
|
||||
<record>
|
||||
<email>melanie.white@example2.edu</email>
|
||||
<name>Melanie White</name>
|
||||
<state>NC</state>
|
||||
<gender>F</gender>
|
||||
<purchase><iPhone></purchase>
|
||||
<count>1</count>
|
||||
<date>2001-07-04T12:08:56</date>
|
||||
</record>
|
||||
<record>
|
||||
<email>danny.baron@example1.com</email>
|
||||
<name> D. ("Tab") Baron</name>
|
||||
<state>CA</state>
|
||||
<gender>M</gender>
|
||||
<purchase>Winter jacket</purchase>
|
||||
<count>1</count>
|
||||
<date>2001-07-04</date>
|
||||
</record>
|
||||
<record>
|
||||
<email>ben.tyler@example3.org</email>
|
||||
<name>Ben Tyler</name>
|
||||
<state>NV</state>
|
||||
<gender>M</gender>
|
||||
<purchase>Flashlight</purchase>
|
||||
<count>1</count>
|
||||
<date>2001/07/04</date>
|
||||
</record>
|
||||
<record>
|
||||
<email>arthur.duff@example4.com</email>
|
||||
<name>Arthur Duff</name>
|
||||
<state>OR</state>
|
||||
<gender>M</gender>
|
||||
<purchase>Dining table</purchase>
|
||||
<count>1</count>
|
||||
<date>2001-07</date>
|
||||
</record>
|
||||
<record>
|
||||
<email>danny.baron@example1.com</email>
|
||||
<name>Daniel Baron</name>
|
||||
<state></state>
|
||||
<gender></gender>
|
||||
<purchase>Bike</purchase>
|
||||
<count>1</count>
|
||||
<date>2001</date>
|
||||
</record>
|
||||
<record>
|
||||
<email>jean.griffith@example5.org</email>
|
||||
<name>Jean Griffith</name>
|
||||
<state>WA</state>
|
||||
<gender>F</gender>
|
||||
<purchase>Power drill</purchase>
|
||||
<count>1</count>
|
||||
<date>2000</date>
|
||||
</record>
|
||||
<record>
|
||||
<email>melanie.white@example2.edu</email>
|
||||
<name>Melanie White</name>
|
||||
<state>NC</state>
|
||||
<gender>F</gender>
|
||||
<purchase>'iPad'</purchase>
|
||||
<count>1</count>
|
||||
<date>1999</date>
|
||||
</record>
|
||||
<record>
|
||||
<email>ben.morisson@example6.org</email>
|
||||
<name>Ben Morisson</name>
|
||||
<state>FL</state>
|
||||
<gender>M</gender>
|
||||
<purchase>Amplifier</purchase>
|
||||
<count>1</count>
|
||||
<date>1998</date>
|
||||
</record>
|
||||
<record>
|
||||
<email>arthur.duff@example4.com</email>
|
||||
<name>Arthur Duff</name>
|
||||
<state>OR</state>
|
||||
<gender>M</gender>
|
||||
<purchase>Night table</purchase>
|
||||
<count>1</count>
|
||||
<date>1997</date>
|
||||
</record>
|
||||
</root>
|
||||
Binary file not shown.
@ -1,10 +0,0 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<record>
|
||||
<email>danny.baron@example1.com</email>
|
||||
<name>Danny Baron</name>
|
||||
<state>CA</state>
|
||||
<gender>M</gender>
|
||||
<purchase>TV (UTF-8: 📺)</purchase>
|
||||
<count>1</count>
|
||||
<date>Wed, 4 Jul 2001</date>
|
||||
</record>
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
File diff suppressed because it is too large
Load Diff
@ -1,6 +0,0 @@
|
||||
🔣 code meaning
|
||||
🍇 1F347 GRAPES
|
||||
🍉 1F349 WATERMELON
|
||||
🍒 1F352 CHERRIES
|
||||
🍓 1F353 STRAWBERRY
|
||||
🍍 1F34D PINEAPPLE
|
||||
|
Binary file not shown.
Binary file not shown.
Binary file not shown.
@ -1,36 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << DATA > "tmp/${t}/${t}.assert"
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --projectName "${t} biểu tượng cảm xúc ⛲"
|
||||
${cmd} --list | grep "${t}" || exit 1
|
||||
${cmd} --delete "${t} biểu tượng cảm xúc ⛲"
|
||||
${cmd} --list | grep "${t}" | cut -d ':' -f 2 > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,36 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << DATA > "tmp/${t}/${t}.assert"
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --list | grep "${t}" || exit 1
|
||||
${cmd} --delete "${t}"
|
||||
${cmd} --list | grep "${t}" | cut -d ':' -f 2 > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,21 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --download "https://git.io/fj5ju" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "data/duplicates-deletion.json" "tmp/${t}/${t}.output"
|
||||
@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.tsv"
|
||||
🔣 code meaning
|
||||
🍇 1F347 GRAPES
|
||||
🍉 1F349 WATERMELON
|
||||
🍒 1F352 CHERRIES
|
||||
🍓 1F353 STRAWBERRY
|
||||
🍍 1F34D PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
🔣,code,meaning
|
||||
🍇,1F347,GRAPES
|
||||
🍉,1F349,WATERMELON
|
||||
🍒,1F352,CHERRIES
|
||||
🍓,1F353,STRAWBERRY
|
||||
🍍,1F34D,PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.tsv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t} biểu tượng cảm xúc 🍉.csv"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t} biểu tượng cảm xúc 🍉.csv"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.tsv"
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.tsv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.csv"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.csv"
|
||||
@ -1,72 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ ${2} ]]; then
|
||||
majorversion="${2%%.*}"
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,FLAG IN HOLE
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
if [[ "$majorversion" = 2 ]]; then
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
<html>
|
||||
<head>
|
||||
<title>export-html-utf8</title>
|
||||
<meta charset="utf-8" />
|
||||
</head>
|
||||
<body>
|
||||
<table>
|
||||
<tr><th>⌨</th><th>code</th><th>meaning</th></tr>
|
||||
<tr><td>⛲</td><td>1F347</td><td>FOUNTAIN</td></tr>
|
||||
<tr><td>⛳</td><td>1F349</td><td>FLAG IN HOLE</td></tr>
|
||||
<tr><td>⛵</td><td>1F352</td><td>SAILBOAT</td></tr>
|
||||
</table>
|
||||
</body>
|
||||
</html>
|
||||
DATA
|
||||
else
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
<html>
|
||||
<head>
|
||||
<title>export-html-utf8</title>
|
||||
<meta charset="utf-8" />
|
||||
</head>
|
||||
<body>
|
||||
<table>
|
||||
<tr><th>⌨</th><th>code</th><th>meaning</th></tr>
|
||||
<tr><td>⛲</td><td>1F347</td><td>FOUNTAIN</td></tr>
|
||||
<tr><td>⛳</td><td>1F349</td><td>FLAG IN HOLE</td></tr>
|
||||
<tr><td>⛵</td><td>1F352</td><td>SAILBOAT</td></tr>
|
||||
</table>
|
||||
</body>
|
||||
</html>
|
||||
DATA
|
||||
fi
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.html"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.html"
|
||||
@ -1,50 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
<html>
|
||||
<head>
|
||||
<title>export-html</title>
|
||||
<meta charset="utf-8" />
|
||||
</head>
|
||||
<body>
|
||||
<table>
|
||||
<tr><th>a</th><th>b</th><th>c</th></tr>
|
||||
<tr><td>1</td><td>2</td><td>3</td></tr>
|
||||
<tr><td>0</td><td>0</td><td>0</td></tr>
|
||||
<tr><td>$</td><td>\</td><td>'</td></tr>
|
||||
</table>
|
||||
</body>
|
||||
</html>
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.html"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.html"
|
||||
@ -1,43 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,FLAG IN HOLE
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,"FLAG IN HOLE"
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.ods"
|
||||
(cd tmp/"${t}" &&
|
||||
ssconvert -S "${t}.ods" "${t}.csv" &&
|
||||
mv "${t}.csv.1" "${t}.output")
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,47 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
if [[ -z "$(command -v ssconvert 2> /dev/null)" ]] ; then
|
||||
echo 1>&2 "ERROR: This test requires ssconvert (gnumeric)"
|
||||
exit 127
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.ods"
|
||||
(cd tmp/"${t}" &&
|
||||
ssconvert -S "${t}.ods" "${t}.csv" &&
|
||||
mv "${t}.csv.1" "${t}.output")
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
🔣,code,meaning
|
||||
🍇,1F347,GRAPES
|
||||
🍉,1F349,WATERMELON
|
||||
🍒,1F352,CHERRIES
|
||||
🍓,1F353,STRAWBERRY
|
||||
🍍,1F34D,PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
🔣 code meaning
|
||||
🍇 1F347 GRAPES
|
||||
🍉 1F349 WATERMELON
|
||||
🍒 1F352 CHERRIES
|
||||
🍓 1F353 STRAWBERRY
|
||||
🍍 1F34D PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.tsv"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.tsv"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.tsv"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.tsv"
|
||||
@ -1,44 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
🔣,code,meaning
|
||||
🍇,1F347,GRAPES
|
||||
🍉,1F349,WATERMELON
|
||||
🍒,1F352,CHERRIES
|
||||
🍓,1F353,STRAWBERRY
|
||||
🍍,1F34D,PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
🔣 code meaning
|
||||
🍇 1F347 GRAPES
|
||||
🍉 1F349 WATERMELON
|
||||
🍒 1F352 CHERRIES
|
||||
🍓 1F353 STRAWBERRY
|
||||
🍍 1F34D PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,43 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,FLAG IN HOLE
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,FLAG IN HOLE
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.xls"
|
||||
(cd tmp/"${t}" &&
|
||||
ssconvert -S "${t}.xls" "${t}.csv" &&
|
||||
mv "${t}.csv" "${t}.output")
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,43 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.xls"
|
||||
(cd tmp/"${t}" &&
|
||||
ssconvert -S "${t}.xls" "${t}.csv" &&
|
||||
mv "${t}.csv" "${t}.output")
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,45 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,FLAG IN HOLE
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
⌨,code,meaning
|
||||
⛲,1F347,FOUNTAIN
|
||||
⛳,1F349,FLAG IN HOLE
|
||||
⛵,1F352,SAILBOAT
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.xlsx"
|
||||
(cd tmp/"${t}" &&
|
||||
ssconvert -S "${t}.xlsx" "${t}.csv" &&
|
||||
mv "${t}.csv" "${t}.output")
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
|
||||
|
||||
@ -1,43 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.xlsx"
|
||||
(cd tmp/"${t}" &&
|
||||
ssconvert -S "${t}.xlsx" "${t}.csv" &&
|
||||
mv "${t}.csv" "${t}.output")
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.txt"
|
||||
a;b;c
|
||||
1;2;3
|
||||
0;0;0
|
||||
$;\;'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a b c
|
||||
1 2 3
|
||||
0 0 0
|
||||
$ \ '
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.txt" --format "csv" --separator ";"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,41 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
Column 1
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --format "line-based"
|
||||
${cmd} --export "${t}" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --format "csv" --output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,40 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
a,b,c
|
||||
1,2,3
|
||||
0,0,0
|
||||
$,\,'
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" --format "csv" > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,27 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
Script to provide a command line interface to an OpenRefine server.
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --help | sed '3q;d' > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,39 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.tsv"
|
||||
🔣 code meaning
|
||||
🍇 1F347 GRAPES
|
||||
🍉 1F349 WATERMELON
|
||||
🍒 1F352 CHERRIES
|
||||
🍓 1F353 STRAWBERRY
|
||||
🍍 1F34D PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << DATA > "tmp/${t}/${t}.assert"
|
||||
column 001: 🔣
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.tsv"
|
||||
${cmd} --info "${t}" | grep 'column 001' > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,35 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << DATA > "tmp/${t}/${t}.assert"
|
||||
column 002: b
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --info "${t}" | grep 'column 002' > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,35 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << DATA > "tmp/${t}/${t}.assert"
|
||||
${t} biểu tượng cảm xúc ⛲
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv" --projectName "${t} biểu tượng cảm xúc ⛲"
|
||||
${cmd} --list | grep "${t}" | cut -d ':' -f 2 > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,35 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
a,b,c
|
||||
1,2,3
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << DATA > "tmp/${t}/${t}.assert"
|
||||
${t}
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --list | grep "${t}" | cut -d ':' -f 2 > "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,58 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
email,name,state,gender,purchase
|
||||
danny.baron@example1.com,Danny Baron,CA,M,TV
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPhone
|
||||
danny.baron@example1.com,D. Baron,CA,M,Winter jacket
|
||||
ben.tyler@example3.org,Ben Tyler,NV,M,Flashlight
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Dining table
|
||||
danny.baron@example1.com,Daniel Baron,CA,M,Bike
|
||||
jean.griffith@example5.org,Jean Griffith,WA,F,Power drill
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPad
|
||||
ben.morisson@example6.org,Ben Morisson,FL,M,Amplifier
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Night table
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "events" : [
|
||||
{ "name" : "Melanie White", "purchase" : "iPhone" },
|
||||
{ "name" : "Jean Griffith", "purchase" : "Power drill" },
|
||||
{ "name" : "Melanie White", "purchase" : "iPad" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "events" : [
|
||||
' \
|
||||
--template ' { "name" : {{jsonize(cells["name"].value)}}, "purchase" : {{jsonize(cells["purchase"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--facets '{"type":"list","name":"gender","columnName":"gender","expression":"value","omitBlank":false,"omitError":false,"selection":[{"v":{"v":"F","l":"F"}}],"selectBlank":false,"selectError":false,"invert":false}' \
|
||||
--output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,54 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
🔣,code,meaning
|
||||
🍇,1F347,GRAPES
|
||||
🍉,1F349,WATERMELON
|
||||
🍒,1F352,CHERRIES
|
||||
🍓,1F353,STRAWBERRY
|
||||
🍍,1F34D,PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "emojis" : [
|
||||
{ "symbol" : "🍇", "meaning" : "GRAPES" },
|
||||
{ "symbol" : "🍉", "meaning" : "WATERMELON" },
|
||||
{ "symbol" : "🍍", "meaning" : "PINEAPPLE" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "emojis" : [
|
||||
' \
|
||||
--template ' { "symbol" : {{jsonize(with(row.columnNames[0],cn,cells[cn].value))}}, "meaning" : {{jsonize(cells["meaning"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--filterQuery '^1F34' \
|
||||
--filterColumn 'code' \
|
||||
--output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,59 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
email,name,state,gender,purchase
|
||||
danny.baron@example1.com,Danny Baron,CA,M,TV
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPhone
|
||||
danny.baron@example1.com,D. Baron,CA,M,Winter jacket
|
||||
ben.tyler@example3.org,Ben Tyler,NV,M,Flashlight
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Dining table
|
||||
danny.baron@example1.com,Daniel Baron,CA,M,Bike
|
||||
jean.griffith@example5.org,Jean Griffith,WA,F,Power drill
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPad
|
||||
ben.morisson@example6.org,Ben Morisson,FL,M,Amplifier
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Night table
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "events" : [
|
||||
{ "name" : "Melanie White", "purchase" : "iPhone" },
|
||||
{ "name" : "Jean Griffith", "purchase" : "Power drill" },
|
||||
{ "name" : "Melanie White", "purchase" : "iPad" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "events" : [
|
||||
' \
|
||||
--template ' { "name" : {{jsonize(cells["name"].value)}}, "purchase" : {{jsonize(cells["purchase"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--filterQuery '^F$' \
|
||||
--filterColumn 'gender' \
|
||||
--output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}.output"
|
||||
@ -1,58 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
email,name,state,gender,purchase
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Dining table
|
||||
,Arthur Duff,OR,M,Night table
|
||||
ben.morisson@example6.org,Ben Morisson,FL,M,Amplifier
|
||||
ben.tyler@example3.org,Ben Tyler,NV,M,Flashlight
|
||||
danny.baron@example1.com,Daniel Baron,CA,M,Bike
|
||||
,Danny Baron,CA,M,TV
|
||||
,D. Baron,CA,M,Winter jacket
|
||||
jean.griffith@example5.org,Jean Griffith,WA,F,Power drill
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPad
|
||||
,Melanie White,NC,F,iPhone
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "events" : [
|
||||
{ "name" : "Melanie White", "purchase" : "iPad" } { "name" : "Melanie White", "purchase" : "iPhone" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "events" : [
|
||||
' \
|
||||
--template ' { "name" : {{jsonize(cells["name"].value)}}, "purchase" : {{jsonize(cells["purchase"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--mode "record-based" \
|
||||
--splitToFiles true \
|
||||
--output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
ls "tmp/${t}"
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}_6.output"
|
||||
@ -1,52 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
🔣,code,meaning
|
||||
🍇,1F347,GRAPES
|
||||
🍉,1F349,WATERMELON
|
||||
🍒,1F352,CHERRIES
|
||||
🍓,1F353,STRAWBERRY
|
||||
🍍,1F34D,PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "emojis" : [
|
||||
{ "symbol" : "🍍", "meaning" : "PINEAPPLE" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "emojis" : [
|
||||
' \
|
||||
--template ' { "symbol" : {{jsonize(with(row.columnNames[0],cn,cells[cn].value))}}, "meaning" : {{jsonize(cells["meaning"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--splitToFiles true \
|
||||
--suffixById true \
|
||||
--output "tmp/${t}/trái cây.json"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/trái cây_🍍.json"
|
||||
@ -1,58 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
email,name,state,gender,purchase
|
||||
danny.baron@example1.com,Danny Baron,CA,M,TV
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPhone
|
||||
danny.baron@example1.com,D. Baron,CA,M,Winter jacket
|
||||
ben.tyler@example3.org,Ben Tyler,NV,M,Flashlight
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Dining table
|
||||
danny.baron@example1.com,Daniel Baron,CA,M,Bike
|
||||
jean.griffith@example5.org,Jean Griffith,WA,F,Power drill
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPad
|
||||
ben.morisson@example6.org,Ben Morisson,FL,M,Amplifier
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Night table
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "events" : [
|
||||
{ "name" : "Arthur Duff", "purchase" : "Night table" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "events" : [
|
||||
' \
|
||||
--template ' { "name" : {{jsonize(cells["name"].value)}}, "purchase" : {{jsonize(cells["purchase"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--splitToFiles true \
|
||||
--suffixById true \
|
||||
--output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
ls "tmp/${t}"
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}_arthur.duff@example4.com.output"
|
||||
@ -1,51 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
🔣,code,meaning
|
||||
🍇,1F347,GRAPES
|
||||
🍉,1F349,WATERMELON
|
||||
🍒,1F352,CHERRIES
|
||||
🍓,1F353,STRAWBERRY
|
||||
🍍,1F34D,PINEAPPLE
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "emojis" : [
|
||||
{ "symbol" : "🍍", "meaning" : "PINEAPPLE" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "emojis" : [
|
||||
' \
|
||||
--template ' { "symbol" : {{jsonize(with(row.columnNames[0],cn,cells[cn].value))}}, "meaning" : {{jsonize(cells["meaning"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--splitToFiles true \
|
||||
--output "tmp/${t}/trái cây.json"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/trái cây_5.json"
|
||||
@ -1,57 +0,0 @@
|
||||
#!/bin/bash
|
||||
|
||||
# =============================== ENVIRONMENT ================================ #
|
||||
|
||||
if [[ ${1} ]]; then
|
||||
cmd="${1}"
|
||||
else
|
||||
echo 1>&2 "execute tests-cli.sh to run all tests"; exit 1
|
||||
fi
|
||||
|
||||
t="$(basename "${BASH_SOURCE[0]}" .sh)"
|
||||
cd "${BASH_SOURCE%/*}/" || exit 1
|
||||
mkdir -p "tmp/${t}"
|
||||
|
||||
# =================================== DATA =================================== #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.csv"
|
||||
email,name,state,gender,purchase
|
||||
danny.baron@example1.com,Danny Baron,CA,M,TV
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPhone
|
||||
danny.baron@example1.com,D. Baron,CA,M,Winter jacket
|
||||
ben.tyler@example3.org,Ben Tyler,NV,M,Flashlight
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Dining table
|
||||
danny.baron@example1.com,Daniel Baron,CA,M,Bike
|
||||
jean.griffith@example5.org,Jean Griffith,WA,F,Power drill
|
||||
melanie.white@example2.edu,Melanie White,NC,F,iPad
|
||||
ben.morisson@example6.org,Ben Morisson,FL,M,Amplifier
|
||||
arthur.duff@example4.com,Arthur Duff,OR,M,Night table
|
||||
DATA
|
||||
|
||||
# ================================= ASSERTION ================================ #
|
||||
|
||||
cat << "DATA" > "tmp/${t}/${t}.assert"
|
||||
{ "events" : [
|
||||
{ "name" : "Arthur Duff", "purchase" : "Night table" }
|
||||
] }
|
||||
DATA
|
||||
|
||||
# ================================== ACTION ================================== #
|
||||
|
||||
${cmd} --create "tmp/${t}/${t}.csv"
|
||||
${cmd} --export "${t}" \
|
||||
--prefix '{ "events" : [
|
||||
' \
|
||||
--template ' { "name" : {{jsonize(cells["name"].value)}}, "purchase" : {{jsonize(cells["purchase"].value)}} }' \
|
||||
--rowSeparator ',
|
||||
' \
|
||||
--suffix '
|
||||
] }
|
||||
' \
|
||||
--splitToFiles true \
|
||||
--output "tmp/${t}/${t}.output"
|
||||
|
||||
# =================================== TEST =================================== #
|
||||
|
||||
ls "tmp/${t}"
|
||||
diff -u "tmp/${t}/${t}.assert" "tmp/${t}/${t}_10.output"
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
x
Reference in New Issue
Block a user