opencultureconsulting
/
openrefine-client
mirror of https://github.com/opencultureconsulting/openrefine-client.git
2
0
Fork 0
Code Issues Releases Activity

removed urllib2 dependencies

This commit is contained in:
Felix Lohmeier 2017-03-14 22:16:36 +01:00
parent bf91e918df
commit 44fd7c4611
2 changed files with 14 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
docker/Dockerfile
View File

@ -3,25 +3,18 @@ MAINTAINER felixlohmeier <felixlohmeier@opencultureconsulting.com>
# The OpenRefine Python Client Library from PaulMakepeace provides an interface to communicating with an OpenRefine server. This fork extends the CLI with some options to create new OpenRefine projects from files. # The OpenRefine Python Client Library from PaulMakepeace provides an interface to communicating with an OpenRefine server. This fork extends the CLI with some options to create new OpenRefine projects from files.
# Source: https://github.com/felixlohmeier/openrefine-client # Source: https://github.com/felixlohmeier/openrefine-client
# Install python, pip, wget, unzip and bash # Install python, wget, unzip, curl and grep
RUN apk add --no-cache \ RUN apk add --no-cache \
bash \
curl \
grep \
python \ python \
py-pip \
wget \ wget \
unzip unzip
curl \
# Install dependency urllib2_file grep \
RUN pip install urllib2_file==0.2.1
# Download and build openrefine-client-master # Download and build openrefine-client-master
WORKDIR /app WORKDIR /app
RUN wget --no-check-certificate https://github.com/felixlohmeier/openrefine-client/archive/master.zip RUN wget --no-check-certificate https://github.com/felixlohmeier/openrefine-client/archive/master.zip
RUN unzip master.zip && rm master.zip RUN unzip master.zip && rm master.zip
RUN python openrefine-client-master/setup.py build
RUN python openrefine-client-master/setup.py install
# Change docker WORKDIR (shall be mounted) # Change docker WORKDIR (shall be mounted)
WORKDIR /data WORKDIR /data

16
docker/README.md
View File

@ -2,6 +2,8 @@
There are some client libraries for OpenRefine that communicate with the [OpenRefine API](https://github.com/OpenRefine/OpenRefine/wiki/OpenRefine-API). I have prepared a docker container on top of the [Python Library from PaulMakepeace](https://github.com/PaulMakepeace/refine-client-py/) and extended the CLI with some options to create new OpenRefine projects from files. There are some client libraries for OpenRefine that communicate with the [OpenRefine API](https://github.com/OpenRefine/OpenRefine/wiki/OpenRefine-API). I have prepared a docker container on top of the [Python Library from PaulMakepeace](https://github.com/PaulMakepeace/refine-client-py/) and extended the CLI with some options to create new OpenRefine projects from files.
If you are looking for a ready to use command line interface to OpenRefine for batch processing then you might be interested in the following bash shell script: [felixlohmeier/openrefine-batch](https://github.com/felixlohmeier/openrefine-batch)
### basic usage ### basic usage
**1) start server:** **1) start server:**
@ -62,22 +64,26 @@ check help screen for more options:
```docker run -d --name=openrefine-server -v ${workingdir}:/data:z felixlohmeier/openrefine -i 0.0.0.0 -m 4G -d /data``` ```docker run -d --name=openrefine-server -v ${workingdir}:/data:z felixlohmeier/openrefine -i 0.0.0.0 -m 4G -d /data```
**3) create project (import file)** **3) wait until server is ready**
```until docker run --rm --link openrefine-server --entrypoint /usr/bin/curl felixlohmeier/openrefine-client --silent -N http://openrefine-server:3333 | cat | grep -q -o "OpenRefine" ; do sleep 1; done```
**4) create project (import file)**
```docker run --rm --link openrefine-server -v ${workingdir}:/data:z felixlohmeier/openrefine-client --create $inputfile``` ```docker run --rm --link openrefine-server -v ${workingdir}:/data:z felixlohmeier/openrefine-client --create $inputfile```
**4) get project id** **5) get project id**
```project=($(docker run --rm --link openrefine-server -v ${workingdir}:/data felixlohmeier/openrefine-client --list | cut -c 2-14))``` ```project=($(docker run --rm --link openrefine-server -v ${workingdir}:/data felixlohmeier/openrefine-client --list | cut -c 2-14))```
**5) apply transformations from json file** **6) apply transformations from json file**
```docker run --rm --link openrefine-server -v ${workingdir}:/data felixlohmeier/openrefine-client --apply ${jsonfile} ${project}``` ```docker run --rm --link openrefine-server -v ${workingdir}:/data felixlohmeier/openrefine-client --apply ${jsonfile} ${project}```
**6) export project to file** **7) export project to file**
```docker run --rm --link openrefine-server -v ${workingdir}:/data felixlohmeier/openrefine-client --export --output=${project}.tsv ${project}``` ```docker run --rm --link openrefine-server -v ${workingdir}:/data felixlohmeier/openrefine-client --export --output=${project}.tsv ${project}```
**7) cleanup** **8) cleanup**
```docker stop -t=500 openrefine-server && docker rm openrefine-server``` ```docker stop -t=500 openrefine-server && docker rm openrefine-server```