This page describes the setup of a Google Cloud based OCR setup for use with the Extended DMS environment. DMS will issue commands so that documents to be OCR'ed are uploaded to Google Cloud, OCR is performed there, and the result is returned. The advantage is that the DMS appliance's resource (CPU, RAM) is not consumed by the OCR processing and that OCR takes advantage of the high accuracy and potential speed of the Google Cloud Vision technology. Note that the Google Cloud service is not free of charge an requires an active Google Cloud account. See: https://cloud.google.com/vision/pricing

The below instructions describe the setup of the DMS host machine. Using this OCR method will not significantly affect performance of the DMS service; the scripts involved are only making https calls to the google API and will be relaying documents but do not perform any processing of the documents.

Components and setup required:

  1. Configure Google Cloud account: enable Google Vision API there and create Google Cloud Storage bucket for temp files created while processing
  2. Configure Google Cloud SDK on DMS host machine
  3. Prepare service account key in JSON format for Google Cloud here: https://console.cloud.google.com/apis/credentials/serviceaccountkey

  4. Create 2 permanent (can be done with adding lines to /etc/environment) environment variables (replace $gcloud_storage_bucket_name and $path_to_json with actual values):

    1. GCLOUD_OCR_BUCKET=$gcloud_storage_bucket_name
    2. GOOGLE_APPLICATION_CREDENTIALS=$path_to_json

  5. Download pi-gcloud-ocr.jar (Link will be provided after successful testing) to /opt/pi-gcloud-ocr.jar

  6. DMS host needs /<storagepath>/nuxeo/data linked to /var/lib/nuxeo/data and /<storagepath>/nuxeo/tmp linked to /opt/nuxeo/server/tmp

  7. Place the below script with name "pi-google-ocr" to /usr/bin/ and make it executable (chmod +x). This script contains the commands to drive the OCR process for each file.
  8. Place the below script with name "piocr" in /<storagepath>/nuxeo/scripts/ and make it executable (chmod +x). This script contains the commands to access pi-google-ocr script from inside of nuxeo container.
  9. Create ssh key in DMS appliance using "ssh-keygen" command, copy public key (id_rsa.pub) and paste into <<adminuserhome>>/.ssh/authorized_keys file of DMS host vm
  10. Login from DMS appliance to itself using ssh. After successful login, move the DMS appliance's key files "id_rsa", "id_rsa.pub" and "known_hosts" from ~/.ssh/ to /<storagepath>/nuxeo/ssh/ 
  11. Set key "ocr.engine.name" in PAT_DMS_SETTINGS table of Patricia db to "piocr"
  12. Add below change commands to the "commands.conf" file of auto-deploy client specific repository (also found in ~/deploy/config/)
  13. Start re-deploy of DMS using ~/deploy/deploy_script/deploy.sh command as outlined here: Auto-deploy script

Scripts:

#!/usr/bin/env bash
gsutil cp "$2" gs://${GCLOUD_OCR_BUCKET}
input_filename=$(basename $2)
output_filename=$(basename $1)
json=$(java -jar /opt/pi-google-ocr.jar gs://${GCLOUD_OCR_BUCKET}/${input_filename} gs://${GCLOUD_OCR_BUCKET}/${output_filename})
echo "$json"  > "$1"
gsutil rm "gs://${GCLOUD_OCR_BUCKET}/${input_filename}"
gsutil rm "gs://${GCLOUD_OCR_BUCKET}/${output_filename}output-1-to-1.json"



#!/bin/bash
echo "variables $1 $2"
ssh <<adminuser>>@<<dms.host.name>> "pi-google-ocr $1 $2"

$1 is the path and filename of the output file, $2 is the path and filename of the input file that are handed over when the DMS calls the piocr script. The command in piocr script must be such that the OCR engine reads the input file (pointed to by $2) and writes to the output file (pointed to by $1).

Make sure you replace <<adminuser>> with correct user name of an administrative user in OCR appliance, and <<dms.host.name>> with proper FQDN or IP address.


  add_volume "/<storage_path>/nuxeo/ssh" "/home/nuxeo/.ssh"
  add_volume "/<storage_path>/nuxeo/scripts/piocr" "/usr/local/bin/piocr"