Beginning on 9/18 you will be able to change your enterprise password from within Jira!

Skip to end of metadata
Go to start of metadata

A guide to deploying SimplyE using Docker containers.

Contents:

Purpose

So you're deploying your library's circulation manager. Awesome! If you'd like to get up and running quickly, we recommend using our Docker image.
If you're already familiar with Docker and/or would like to contribute to our Docker builds, you can find our build files at NYPL-Simplified/circulation-docker.

Prep Work

  1. Install Docker. Docker has step-by-step instructions to grab its most up-to-date version. Depending on your package manager, you could also install a slightly older version with: sudo apt-get install docker-ce or sudo yum install docker-ce.
  2. Create any dependent, temporary containers (optional) for integrations like Elasticsearch and Postgres. We don't recommend using containers in the long-term for holding or maintaining data. However, if you just want to get a sense of how your Circulation Manager will work, containers are a quick option. Instructions for integrating Elasticsearch and Postgres via Docker can be found below.
  3. Get the Docker images for the Library Simplified Circulation Manager. Run:
   $ sudo docker pull nypl/circ-webapp && sudo docker pull nypl/circ-scripts

Running Circulation Manager containers

Running scripts

To deploy an app filled with your library's books, you'll need to run a number of scripts. Read the environment variable details below before running this script; you will likely need to alter it to meet your needs.

Example docker run script

   $ sudo docker run -d --name circ-scripts \ -e TZ="US/Central" \ -e SIMPLIFIED_PRODUCTION_DATABASE='postgres://[username]:[password]@[host]:[port]/[database_name]' \ nypl/circ-scripts

What It Does

The example above runs this resulting container in detached mode (-d), linked to the (-e) SIMPLIFIED_PRODUCTION_DATABASE and calling it "circ-scripts". With the (-e) optional argument TZ, you can pass a Debian-system timezone representing your local time zone, which will cause timed scripts to run according to your local time. If the database you've connected in your configuration has never been used before, use -e to set the optional argument SIMPLIFIED_DB_TASK to 'init'. This will keep track of the state of the database you've created and create an alias on your Elasticsearch cluster, allowing database updates to be easily managed with scripts.

Running Scripts

Once you've given your scripts some time to run (~30 minutes should be enough time to start having works move through the import process), you'll want to refresh your cached materialized views so they show up in your deployed app.

   $ sudo docker exec circ-scripts /var/www/circulation/core/bin/run refresh_materialized_views

Troubleshooting

You'll want to check the logs of your container. For example:
# check logs of the database task and running supervisor processes$ sudo docker logs circ-scripts # check logs of cron and scripts$ sudo docker exec circ-scripts cat /var/log/cron.log | less$ sudo docker exec circ-scripts ls /var/log/simplified$ sudo docker exec circ-scripts cat /var/log/simplified/overdrive_monitor_full | less # The log directory can also be found on the production server. # Its location can be found using this command.$ sudo docker inspect circ-scripts \ --format='range $mount := .Mountsif eq $mount.Destination "/var/log"$mount.Sourceendend'
You can hop into a running container at any time with the command: $ sudo docker exec -it circ /bin/bash
Docker has fantastic documentation to get more familiar with its command line tools, like docker exec and docker inspect. We recommend you check them out.

Deploying the App

Using an nypl/circ-deploy container deploys the OPDS feeds expected by the SimplyE client applications. Read the environment variable details below before running the following script; you will likely need to alter it to meet your needs.

Example docker run script

   $ sudo docker run --name circ-webapp \ -d -p 80:80 \ -e SIMPLIFIED_PRODUCTION_DATABASE='postgres://[username]:[password]@[host]:[port]/[database_name]' \ -e SIMPLIFIED_DB_TASK="auto" \ nypl/circ-webapp

What It Does

The script above runs the container in detached mode (-d), binding its port 80 to your server's port 80 (-p), connecting it to your PostgreSQL database -e SIMPLIFIED_PRODUCTION_DATABASE) and calling it "circ-deploy". Unless you've been running a scripts container for while, when you visit your server through a browser, you'll see a very sparse OPDS feed. If the database you've connected in your configuration has never been used before, use -e to set the optional argument SIMPLIFIED_DB_TASK to 'init'. This will keep track of the state of the database you've created and create an alias on your Elasticsearch cluster, allowing database updates to be easily managed with scripts.

Troubleshooting

You'll want to check the logs of your container (/var/log/nginx/error.log and /var/log/libsimple/uwsgi.log) to troubleshoot:
# check logs of the database task and running supervisor processes$ sudo docker logs circ-deploy # check logs inside the container$ sudo docker exec circ-deploy cat /var/log/nginx/error.log | less$ sudo docker exec circ-deploy cat /var/log/libsimple/uwsgi.log | less # restart the application$ sudo docker exec circ-deploy touch uwsgi.ini
You can hop into a running container at any time with the command: $ sudo docker exec -it circ /bin/bash
Docker has fantastic documentation to get more familiar with its command line tools, like docker exec and docker inspect. We recommend you check them out.

Environment Variables

SIMPLIFIED_CONFIGURATION_FILE

Optional The full path to configuration file in the container. Configuration is now held in the database and accessed via an administrative interface at /admin, so you probably don't need this.
If you do, use this documentation to create the JSON file for your particular library's configuration. If you're unfamiliar with JSON, you can use this JSON Formatter & Validator to validate your configuration file.

SIMPLIFIED_DB_TASK

Optional. Performs a task against the database at container runtime. Options are:

  • auto : Either initializes or migrates the database, depending on if it is new or not. This is the default value.
  • ignore : Does nothing.
  • init : Initializes the app against a brand new database. If you are running a circulation manager for the first time every, use this value to set up an Elasticsearch alias and account for the database schema for future migrations.
  • migrate : Migrates an existing database against a new release. Use this value when switching from one stable version to another.

SIMPLIFIED_PRODUCTION_DATABASE

Required. The URL of the production PostgreSQL database for the application.

SIMPLIFIED_TEST_DATABASE

Optional. The URL of a PostgreSQL database for tests. This optional variable allows unit tests to be run in the container.

TZ

Optional. Applies to circ-scripts only. The time zone that cron should use to run scheduled scripts–usually the timezone of the library or libraries on the circulation manager instance. This value should be selected according to Debian-system timezone options. This value allows scripts to run at ideal times.

Evaluating Success

If your Docker containers are running successfully, you should have a /var/log/simplifieddirectory full of logfiles in your circ-scripts container, and you should be able to visit your server's domain and see an OPDS feed from circ-deploy. If either of these things aren't occurring, use the troubleshooting details above to check var/log/cron.log or the logfiles in /var/log/simplified for circ-scripts and/or /var/log/libsimple/uwsgi.log or /var/log/nginx/error.log.

Support Containers (for use in development or testing)

Elasticsearch

While we do not recommend you run Elasticsearch from a Docker container permanently, you may want to get up and running with a throwaway search index. Elasticsearch isn't installed via the Dockerfile, so the fastest way to connect to it will be through another container. Here's how:

  1. Get the Docker image for Elasticsearch v1.x:
  2. $ sudo docker pull elasticsearch:1
  3. Create an Elasticsearch container, and grab its IP Address, running the following commands:

$ sudo docker run -d --name es elasticsearch:1   # create an elasticsearch container
$ sudo docker ps  # confirm that it's running; note its IP address
$ sudo docker inspect es --format="range .NetworkSettings.Networks.IPAddressend"

 

  1. Record the Elasticsearch URL for use when configuring the Search integration in the Circulation Manager Admin interface. When you run sudo docker ps, you'll see a single running container called es. Use the IP that comes from running inspect to update your your config.json file with the proper Elasticsearch location. You should end up with something like http://172.17.0.2:9200.

Postgres

While we do not recommend you run Postgres from a Docker container permanently, you may want to get up and running with a throwaway database. Postgres isn't installed via the Dockerfile, so the best way to connect to Postgres will be through another container. Here's how:

  1. Get the Docker image for Postgres 9.4 or 9.5:
  2. $ sudo docker pull postgres:9.5
  3. Create a Postgres container, and grab its IP Address, running the following commands:

$ sudo docker run -d --name pg postgres:9.5   # create a postgres container
$ sudo docker ps  # confirm that it's running; note its IP address
$ sudo docker inspect pg --format="range .NetworkSettings.Networks.IPAddressend"

  1. Create a Postgres database, running the following commands:

$ docker exec -u postgres pg psql -c "create user simplified with password 'test';"  # create a user and password
$ docker exec -u postgres pg psql -c "create database simplified_circ_db;"  # create database
$ docker exec -u postgres pg psql -c "grant all privileges on database simplified_circ_db to simplified;"
$ docker exec -u postgres pg psql -d simplified_circ_db -c "create extension pgcrypto;"

  1. Add the Postgres URL to database environment variable as shown above. It will be of the form "postgres://simplified:test@172.17.0.3:5432/simplified_circ_db", following the "postgres://<USERNAME>:<PASSWORD>@<HOST>:<PORT>/<DATABASE_NAME>" format.