From 44fd7c4611765579595cd908d66ceac3cc10cf98 Mon Sep 17 00:00:00 2001 From: Felix Lohmeier Date: Tue, 14 Mar 2017 22:16:36 +0100 Subject: [PATCH] removed urllib2 dependencies --- docker/Dockerfile | 13 +++---------- docker/README.md | 16 +++++++++++----- 2 files changed, 14 insertions(+), 15 deletions(-) diff --git a/docker/Dockerfile b/docker/Dockerfile index bb4ed3f..8178a3d 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -3,25 +3,18 @@ MAINTAINER felixlohmeier # The OpenRefine Python Client Library from PaulMakepeace provides an interface to communicating with an OpenRefine server. This fork extends the CLI with some options to create new OpenRefine projects from files. # Source: https://github.com/felixlohmeier/openrefine-client -# Install python, pip, wget, unzip and bash +# Install python, wget, unzip, curl and grep RUN apk add --no-cache \ - bash \ - curl \ - grep \ python \ - py-pip \ wget \ unzip - -# Install dependency urllib2_file -RUN pip install urllib2_file==0.2.1 + curl \ + grep \ # Download and build openrefine-client-master WORKDIR /app RUN wget --no-check-certificate https://github.com/felixlohmeier/openrefine-client/archive/master.zip RUN unzip master.zip && rm master.zip -RUN python openrefine-client-master/setup.py build -RUN python openrefine-client-master/setup.py install # Change docker WORKDIR (shall be mounted) WORKDIR /data diff --git a/docker/README.md b/docker/README.md index 205fc8f..9601113 100644 --- a/docker/README.md +++ b/docker/README.md @@ -2,6 +2,8 @@ There are some client libraries for OpenRefine that communicate with the [OpenRefine API](https://github.com/OpenRefine/OpenRefine/wiki/OpenRefine-API). I have prepared a docker container on top of the [Python Library from PaulMakepeace](https://github.com/PaulMakepeace/refine-client-py/) and extended the CLI with some options to create new OpenRefine projects from files. +If you are looking for a ready to use command line interface to OpenRefine for batch processing then you might be interested in the following bash shell script: [felixlohmeier/openrefine-batch](https://github.com/felixlohmeier/openrefine-batch) + ### basic usage **1) start server:** @@ -62,22 +64,26 @@ check help screen for more options: ```docker run -d --name=openrefine-server -v ${workingdir}:/data:z felixlohmeier/openrefine -i 0.0.0.0 -m 4G -d /data``` -**3) create project (import file)** +**3) wait until server is ready** + +```until docker run --rm --link openrefine-server --entrypoint /usr/bin/curl felixlohmeier/openrefine-client --silent -N http://openrefine-server:3333 | cat | grep -q -o "OpenRefine" ; do sleep 1; done``` + +**4) create project (import file)** ```docker run --rm --link openrefine-server -v ${workingdir}:/data:z felixlohmeier/openrefine-client --create $inputfile``` -**4) get project id** +**5) get project id** ```project=($(docker run --rm --link openrefine-server -v ${workingdir}:/data felixlohmeier/openrefine-client --list | cut -c 2-14))``` -**5) apply transformations from json file** +**6) apply transformations from json file** ```docker run --rm --link openrefine-server -v ${workingdir}:/data felixlohmeier/openrefine-client --apply ${jsonfile} ${project}``` -**6) export project to file** +**7) export project to file** ```docker run --rm --link openrefine-server -v ${workingdir}:/data felixlohmeier/openrefine-client --export --output=${project}.tsv ${project}``` -**7) cleanup** +**8) cleanup** ```docker stop -t=500 openrefine-server && docker rm openrefine-server```