Merge branch 'master' into py3_port
This commit is contained in:
Felix Lohmeier
GitHub
commit
f06fc54e12
72
README.md
72
README.md
|
|
@ -6,13 +6,15 @@ The [OpenRefine Python Client from PaulMakepeace](https://github.com/PaulMakepea
|
|||
This fork extends the command line interface (CLI) and is distributed as a convenient one-file-executable (Windows, Linux, macOS).
|
||||
It is also available via Docker Hub, PyPI and Binder.
|
||||
|
||||
works with OpenRefine 2.7, 2.8, 3.0, 3.1, 3.2
|
||||
|
||||
## Download
|
||||
|
||||
One-file-executables:
|
||||
|
||||
- Windows: [openrefine-client_0-3-8_windows.exe](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.8/openrefine-client_0-3-8_windows.exe) (~5 MB)
|
||||
- macOS: [openrefine-client_0-3-8_macos](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.8/openrefine-client_0-3-8_macos) (~5 MB)
|
||||
- Linux: [openrefine-client_0-3-8_linux](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.8/openrefine-client_0-3-8_linux) (~5 MB)
|
||||
- Windows: [openrefine-client_0-3-9_windows.exe](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.9/openrefine-client_0-3-9_windows.exe) (~5 MB)
|
||||
- macOS: [openrefine-client_0-3-9_macos](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.9/openrefine-client_0-3-9_macos) (~5 MB)
|
||||
- Linux: [openrefine-client_0-3-9_linux](https://github.com/opencultureconsulting/openrefine-client/releases/download/v0.3.9/openrefine-client_0-3-9_linux) (~5 MB)
|
||||
|
||||
For [Docker](#docker) containers, native [Python](#python) installation and free [Binder](#binder) on-demand server see the corresponding chapters below.
|
||||
|
||||
|
|
@ -55,13 +57,13 @@ To use the client:
|
|||
- macOS:
|
||||
|
||||
```sh
|
||||
chmod +x openrefine-client_0-3-8_macos
|
||||
chmod +x openrefine-client_0-3-9_macos
|
||||
```
|
||||
|
||||
- Linux:
|
||||
|
||||
```sh
|
||||
chmod +x openrefine-client_0-3-8_linux
|
||||
chmod +x openrefine-client_0-3-9_linux
|
||||
```
|
||||
|
||||
3. Execute the file.
|
||||
|
|
@ -69,19 +71,19 @@ To use the client:
|
|||
- Windows:
|
||||
|
||||
```sh
|
||||
.\openrefine-client_0-3-8_windows.exe
|
||||
.\openrefine-client_0-3-9_windows.exe
|
||||
```
|
||||
|
||||
- macOS:
|
||||
|
||||
```sh
|
||||
./openrefine-client_0-3-8_macos
|
||||
./openrefine-client_0-3-9_macos
|
||||
```
|
||||
|
||||
- Linux:
|
||||
|
||||
```sh
|
||||
./openrefine-client_0-3-8_linux
|
||||
./openrefine-client_0-3-9_linux
|
||||
```
|
||||
|
||||
Using tab completion and command history is highly recommended:
|
||||
|
|
@ -100,25 +102,25 @@ Download example data (`--download`) and create project from file (`--create`):
|
|||
- Windows:
|
||||
|
||||
```sh
|
||||
.\openrefine-client_0-3-8_windows.exe --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
.\openrefine-client_0-3-8_windows.exe --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
.\openrefine-client_0-3-8_windows.exe --create duplicates.csv
|
||||
.\openrefine-client_0-3-9_windows.exe --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
.\openrefine-client_0-3-9_windows.exe --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
.\openrefine-client_0-3-9_windows.exe --create duplicates.csv
|
||||
```
|
||||
|
||||
- macOS:
|
||||
|
||||
```sh
|
||||
./openrefine-client_0-3-8_macos --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
./openrefine-client_0-3-8_macos --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
./openrefine-client_0-3-8_macos --create duplicates.csv
|
||||
./openrefine-client_0-3-9_macos --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
./openrefine-client_0-3-9_macos --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
./openrefine-client_0-3-9_macos --create duplicates.csv
|
||||
```
|
||||
|
||||
- Linux:
|
||||
|
||||
```sh
|
||||
./openrefine-client_0-3-8_linux --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
./openrefine-client_0-3-8_linux --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
./openrefine-client_0-3-8_linux --create duplicates.csv
|
||||
./openrefine-client_0-3-9_linux --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
./openrefine-client_0-3-9_linux --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
./openrefine-client_0-3-9_linux --create duplicates.csv
|
||||
```
|
||||
|
||||
Other commands:
|
||||
|
|
@ -230,7 +232,7 @@ When using this option, the first column should contain unique identifiers.
|
|||
[felixlohmeier/openrefine-client](https://hub.docker.com/r/felixlohmeier/openrefine-client/) [](https://hub.docker.com/r/felixlohmeier/openrefine-client/)
|
||||
|
||||
```sh
|
||||
docker pull felixlohmeier/openrefine-client:v0.3.8
|
||||
docker pull felixlohmeier/openrefine-client:v0.3.9
|
||||
```
|
||||
|
||||
### Option 1: Dockerized client
|
||||
|
|
@ -238,7 +240,7 @@ docker pull felixlohmeier/openrefine-client:v0.3.8
|
|||
Run client and mount current directory as workspace:
|
||||
|
||||
```sh
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9
|
||||
```
|
||||
|
||||
The docker option `--network=host` allows you to connect to a local or remote OpenRefine via the host network:
|
||||
|
|
@ -246,13 +248,13 @@ The docker option `--network=host` allows you to connect to a local or remote Op
|
|||
- list projects on default URL (http://localhost:3333)
|
||||
|
||||
```sh
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 --list
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 --list
|
||||
```
|
||||
|
||||
- list projects on a remote server (http://example.com)
|
||||
|
||||
```sh
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H example.com -P 80 --list
|
||||
docker run --rm --network=host -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H example.com -P 80 --list
|
||||
```
|
||||
|
||||
Usage: same commands as explained above (see [Basic Commands](#basic-commands) and [Advanced Templating](#advanced-templating))
|
||||
|
|
@ -276,16 +278,16 @@ Run openrefine-client linked to a dockerized OpenRefine ([felixlohmeier/openrefi
|
|||
3. Run client with some [basic commands](#basic-commands): 1. download example files, 2. create project from file, 3. list projects, 4. show metadata, 5. export to terminal, 6. apply transformation rules (deduplication), 7. export again to terminal, 8. export to xls file and 9. delete project
|
||||
|
||||
```sh
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H openrefine-server --create duplicates.csv
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H openrefine-server --list
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H openrefine-server --info "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H openrefine-server --export "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H openrefine-server --apply duplicates-deletion.json "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H openrefine-server --export "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H openrefine-server --export --output=deduped.xls "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.8 -H openrefine-server --delete "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 --download "https://git.io/fj5hF" --output=duplicates.csv
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 --download "https://git.io/fj5ju" --output=duplicates-deletion.json
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H openrefine-server --create duplicates.csv
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H openrefine-server --list
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H openrefine-server --info "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H openrefine-server --export "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H openrefine-server --apply duplicates-deletion.json "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H openrefine-server --export "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H openrefine-server --export --output=deduped.xls "duplicates"
|
||||
docker run --rm --network=openrefine -v ${PWD}:/data:z felixlohmeier/openrefine-client:v0.3.9 -H openrefine-server --delete "duplicates"
|
||||
```
|
||||
|
||||
4. Stop and delete server:
|
||||
|
|
@ -503,8 +505,8 @@ Note to myself: When releasing a new version...
|
|||
- Run PyInstaller in Python 3 environments on native Windows, macOS and Linux. Should be "the oldest version of the OS you need to support"! Current release is built with:
|
||||
|
||||
- Ubuntu 16.04 LTS (64-bit)
|
||||
- macOS Sierra 10.12
|
||||
- Windows 10
|
||||
- macOS Sierra 10.12 (64-bit)
|
||||
- Windows 7 (32-bit)
|
||||
|
||||
- One-file-executables will be available in `dist/`.
|
||||
|
||||
|
|
@ -513,7 +515,7 @@ Note to myself: When releasing a new version...
|
|||
cd openrefine-client
|
||||
python3 -m pip install . --user
|
||||
python3 -m pip install pyinstaller --user
|
||||
pyinstaller --onefile refine.py --hidden-import google.refine.__main__
|
||||
python3 -m pip PyInstaller --onefile refine.py --hidden-import google.refine.__main__
|
||||
```
|
||||
|
||||
4. Run test with Linux executable
|
||||
|
|
@ -542,8 +544,8 @@ Note to myself: When releasing a new version...
|
|||
8. Bump openrefine-client version in related projects
|
||||
|
||||
- openrefine-batch: [openrefine-batch.sh](https://github.com/opencultureconsulting/openrefine-batch/blob/master/openrefine-batch.sh#L7) and [openrefine-batch-docker.sh](https://github.com/opencultureconsulting/openrefine-batch/blob/master/openrefine-batch-docker.sh)
|
||||
|
||||
- openrefineder: [postBuild](https://github.com/felixlohmeier/openrefineder/blob/master/postBuild) and [openrefine-client-bash.ipynb](https://github.com/felixlohmeier/openrefineder/blob/master/openrefine-client-python.ipynb)
|
||||
- openrefineder: [postBuild](https://github.com/felixlohmeier/openrefineder/blob/master/postBuild)
|
||||
|
||||
## Credits
|
||||
|
||||
|
|
|
|||
|
|
@ -248,6 +248,7 @@ class Refine:
|
|||
# POST is broken at the moment, so we send it in the URL
|
||||
new_style_options = dict(opts, **{
|
||||
'encoding': s(encoding),
|
||||
'separator': s(separator)
|
||||
})
|
||||
params = {
|
||||
'options': json.dumps(new_style_options),
|
||||
|
|
@ -256,7 +257,6 @@ class Refine:
|
|||
# old style options
|
||||
options = {
|
||||
'format': project_format,
|
||||
'separator': s(separator),
|
||||
'ignore-lines': s(ignore_lines),
|
||||
'header-lines': s(header_lines),
|
||||
'skip-data-lines': s(skip_data_lines),
|
||||
|
|
@ -265,7 +265,7 @@ class Refine:
|
|||
'process-quotes': s(process_quotes),
|
||||
'store-blank-rows': s(store_blank_rows),
|
||||
'store-blank-cells-as-nulls': s(store_blank_cells_as_nulls),
|
||||
'include-file-sources': s(include_file_sources),
|
||||
'include-file-sources': s(include_file_sources)
|
||||
}
|
||||
|
||||
files = None
|
||||
|
|
|
|||
2
setup.py
2
setup.py
|
|
@ -25,7 +25,7 @@ def read(filename):
|
|||
return open(os.path.join(os.path.dirname(__file__), filename)).read()
|
||||
|
||||
setup(name='openrefine-client',
|
||||
version='0.3.8',
|
||||
version='0.3.9',
|
||||
description=('The OpenRefine Python Client Library provides an '
|
||||
'interface to communicating with an OpenRefine server. '
|
||||
'This fork extends the command line interface (CLI).'),
|
||||
|
|
|
|||
Loading…
Reference in New Issue