OpenRefine command-line interface written in Bash. Supports batch processing (import, transform, export).
Go to file
felixlohmeier dc6b325dd2 update README 2022-10-06 11:35:34 +00:00
src update README 2022-10-06 11:35:34 +00:00
.gitignore setup dev environment 2022-03-25 11:55:57 +01:00
.gitpod.yml first draft completions cmd 2022-10-04 21:19:18 +00:00
LICENSE Initial commit 2022-03-25 10:34:28 +01:00
README.md update README 2022-10-06 11:35:34 +00:00
orcli update README 2022-10-06 11:35:34 +00:00

README.md

orcli (💎+🤖)

Bash script to control OpenRefine via its HTTP API.

Features

  • works with latest OpenRefine version (currently 3.5)
  • batch processing (import, transform, export)
    • orcli takes care of starting and stopping OpenRefine with temporary workspaces
    • your existing OpenRefine data will not be touched
  • import CSV, TSV, line-based TXT, fixed-width TXT, JSON or XML (and specify input options)
    • supports stdin, multiple files and URLs
  • transform data by providing an undo/redo JSON file
    • orcli calls specific endpoints for each operation to provide improved error handling and logging
    • supports stdin, multiple files and URLs
  • export to TSV, CSV, HTML, XLS, XLSX, ODS
  • templating export to additional formats like JSON or XML

Requirements

Install

  1. Navigate to the OpenRefine program directory

  2. Download bash script there and make it executable

wget https://github.com/opencultureconsulting/orcli/raw/main/orcli
chmod +x orcli
  1. Create a symlink in your $PATH (e.g. to ~/.local/bin)
ln -s "${PWD}/orcli" ~/.local/bin/

Usage

Ensure you have OpenRefine running (i.e. available at http://localhost:3333 or another URL) or use the integrated start command first.

Use integrated help screens for available options and examples for each command.

$ orcli --help
orcli - OpenRefine command-line interface written in Bash

Usage:
  orcli COMMAND
  orcli [COMMAND] --help | -h
  orcli --version | -v

Commands:
  completions   Generate bash completions
  batch         run tmp OpenRefine workspace and execute shell script
  import        import commands
  list          list projects on OpenRefine server
  info          show project metadata
  export        export commands

Options:
  --help, -h
    Show this help

  --version, -v
    Show version number

Environment Variables:
  OPENREFINE_URL
    URL to OpenRefine server
    Default: http://localhost:3333

Examples:
  orcli import csv "https://git.io/fj5hF" --projectName "duplicates"
  orcli list
  orcli info "duplicates"
  orcli export tsv "duplicates"
  orcli export tsv "duplicates" --output "duplicates.tsv"
  orcli batch << EOF
    orcli import csv "https://git.io/fj5hF" --projectName "duplicates"
    orcli info "duplicates"
    orcli export tsv "duplicates"
  EOF

https://github.com/opencultureconsulting/orcli

Development

orcli uses bashly for generating the one-file script from files in the src directory

  1. Install bashly (requires ruby)
gem install bashly
  1. Edit code in src directory

  2. Generate script

bashly generate --upgrade