OpenRefine command-line interface written in Bash. Supports batch processing (import, transform, export).

Go to file

Felix Lohmeier 9ba1c7f067 Merge pull request #112 from opencultureconsulting/45-run-server-log-in-error-msg run: print 50 lines of server log		2023-10-26 07:16:36 +02:00
.devcontainer	codespaces portforwarding	2023-10-20 18:35:59 +00:00
help	generate markdown help	2023-10-22 22:09:14 +00:00
src	run: print 50 lines of server log	2023-10-26 05:16:15 +00:00
tests	remove trailing newline from input	2023-10-25 23:25:40 +00:00
.gitignore	setup dev environment	2022-03-25 11:55:57 +01:00
LICENSE	Initial commit	2022-03-25 10:34:28 +01:00
README.md	add link to help README	2023-10-22 22:15:42 +00:00
demo.gif	compress asciinema gif	2023-10-23 00:31:52 +02:00
help.sh	generate markdown help	2023-10-22 22:09:14 +00:00
orcli	run: print 50 lines of server log	2023-10-26 05:16:15 +00:00

orcli (💎+🤖)

Bash script to control OpenRefine via its HTTP API.

Features

works with latest OpenRefine version (currently 3.7)
run batch processes (import, transform, export)
- orcli takes care of starting and stopping OpenRefine with temporary workspaces
- allows execution of arbitrary bash scripts
- interactive mode for playing around and debugging
- your existing OpenRefine data will not be touched
import CSV, TSV, ~~line-based TXT, fixed-width TXT, JSON or XML~~ (and specify input options)
- supports stdin, multiple files and URLs
transform data by providing an undo/redo JSON file
- orcli calls specific endpoints for each operation to provide improved error handling and logging
- supports stdin, multiple files and URLs
export to TSV, ~~CSV, HTML, XLS, XLSX, ODS~~
templating export to additional formats like JSON or XML

wget https://github.com/opencultureconsulting/orcli/raw/main/orcli
chmod +x orcli

Optional:

Create a symlink in your $PATH (e.g. to ~/.local/bin)
```
ln -s "${PWD}/orcli" ~/.local/bin/
```

Install Bash tab completion

permanently

mkdir -p ~/.bashrc.d
orcli completions > ~/.bashrc.d/orcli

./orcli run --interactive

orcli import csv "https://git.io/fj5hF" --projectName "duplicates"

orcli transform "duplicates" "https://git.io/fj5ju"

Export data from OpenRefine project to tab-separated-values (TSV) file duplicates.tsv

orcli export tsv "duplicates" --output "duplicates.tsv"

Write out your session history to file example.sh (and delete the last line to remove the history command)

history -a "example.sh"
sed -i '$ d' example.sh

exit

./orcli run example.sh

Use help screens for available options and examples for each command.
```
orcli --help
```
If your OpenRefine is running on a different port or host, then use the environment variable OPENREFINE_URL.
```
OPENREFINE_URL="http://localhost:3333" orcli list
```
If OpenRefine does not have enough memory to process the data, it becomes slow and may even crash. Check the message after the run command finishes to see how much memory was used and adjust the memory allocated to OpenRefine accordingly with the --memory flag (default: 2048M).

orcli uses bashly for generating the one-file script from files in the src directory

gem install bashly

bashly generate --upgrade

./orcli test

./help.sh