Is there a way to delay container startup to support dependant services with a longer startup time

I have a MySQL container that takes a little time to start up as it needs to import data.

I have an Alfresco container that depends upon the MySQL container.

At the moment, when I use fig, the Alfresco service inside the Alfresco container fails when it attempts to connect to the MySQL container... ostensibly because the MySQL service is not yet listening.

Is there a way to handle this kind of issue in Fig?

At work we wrap our dependent services in a script that check if the link is up yet. I know one of my colleagues would be interested in this too! Personally I feel it's a container-level concern to wait for services to be available, but I may be wrong :)

We do the same thing with wrapping. You can see an example here: https://github.com/dominionenterprises/tol-api-php/blob/master/tests/provisioning/set-env.sh

It'd be handy to have an entrypoint script that loops over all of the links and waits until they're working before starting the command passed to it.

This should be built in to Docker itself, but the solution is a way off. A container shouldn't be considered started until the link it exposes has opened.

@bfirsh

@bfirsh that's more than I was imagining, but would be excellent.

A container shouldn't be considered started until the link it exposes has opened.

I think that's exactly what people need.

For now, I'll be using a variation on https://github.com/aanand/docker-wait

Yeah, I'd be interested in something like this - meant to post about it earlier.

The smallest impact pattern I can think of that would fix this usecase for us would be to be the following:

Add "wait" as a new key in fig.yml, with similar value semantics as link. Docker would treat this as a pre-requisite and wait until this container has exited prior to carrying on.

So, my docker file would look something like:

db:
  image: tutum/mysql:5.6

initdb:
  build: /path/to/db
  link:
    - db:db
  command: /usr/local/bin/init_db

app:
  link:
    - db:db
  wait:
    - initdb

On running app, it will start up all the link containers, then run the wait container and only progress to the actual app container once the wait container (initdb) has exited. initdb would run a script that waits for the database to be available, then runs any initialisations/migrations/whatever, then exits.

That's my thoughts, anyway.

(revised, see below)

+1 here too. It's not very appealing to have to do this in the commands themselves.

+1 as well. Just ran into this issue. Great tool btw, makes my life so much easier!

+1 would be great to have this.

+1 also. Recently run into the same set of problems

+1 also. any statement from dockerguys?

I am writing wrapper scripts as entrypoints to synchronise at the moment, not sure if having a mechanism in fig is wise if you have other targets for your containers that perform orchestration a different way. Seems very application specific to me, as such the responsibility of the containers doing the work.

After some thought and experimentation I do kind of agree with this.

As such an application I'm building basically has a synchronous
waitfor(host, port) function that lets me waits for services the application
is depending on (either detected via environment or explicitly
configuration via cli options).

cheers
James

James Mills / prologic

E: [email protected]
W: prologic.shortcircuit.net.au

On Fri, Aug 22, 2014 at 6:34 PM, Mark Stuart [email protected]
wrote:

I am writing wrapper scripts as entrypoints to synchronise at the moment,
not sure if having a mechanism in fig is wise if you have other targets for
your containers that perform orchestration a different way. Seems very
application specific to me as such the responsibility of the containers
doing the work.

—
Reply to this email directly or view it on GitHub
#374 (comment).

Yes some basic "depend's on" neeeded here...
so if you have 20 container, you just wan't to run fig up and everything starts with correct order...
However it also have some timeout option or other failure catching mechanisms

Another +1 here. I have Postgres taking longer than Django to start so the DB isn't there for the migration command without hackery.

Unfortunatelly, condition is not supported anymore in v3. Here is workaround, that I've found:

website:
    depends_on:
      - 'postgres'
    build: .
    ports:
      - '3000'
    volumes:
      - '.:/news_app'
      - 'bundle_data:/bundle'
    entrypoint: ./wait-for-postgres.sh postgres 5432

  postgres:
    image: 'postgres:9.6.2'
    ports:
      - '5432'

wait-for-postgres.sh:

#!/bin/sh

postgres_host=$1
postgres_port=$2
shift 2
cmd="$@"

# wait for the postgres docker to be running
while ! pg_isready -h $postgres_host -p $postgres_port -q -U postgres; do
  >&2 echo "Postgres is unavailable - sleeping"
  sleep 1
done

>&2 echo "Postgres is up - executing command"

# run the command
exec $cmd

@slava-nikulin

@slava-nikulin custom entrypoint is a common practice, it is almost the only (docker native) way how you can define and check all conditions you need before staring your app in a container.

Truth is there was a lot of debate and I think the 2.x support for the conditional support to natively integrate with health checks and order the startup was a much needed support. Docker does not support local pod of containers natively and when it does it will have to support something similar again just like kubernetes for example provides the semantics. Docker 3.x is a series to bring swarm support into compose and hence bunch of options has been dropped keeping the distributed nature in mind. 2.x series preserves the original compose/local topology features. Docker has to figure out how to merge these 2 versions because forcing swarm onto compose by reducing feature set of compose is not a welcome direction.

…

On May 10, 2017, at 8:15 PM, Slava Nikulin ***@***.***> wrote: Unfortunatelly, condition is not supported anymore in v3. Here is workaround, that I've found: website: depends_on: - 'postgres' build: . ports: - '3000' volumes: - '.:/news_app' - 'bundle_data:/bundle' entrypoint: ./wait-for-postgres.sh postgres 5432 postgres: image: 'postgres:9.6.2' ports: - '5432' wait-for-postgres.sh: #!/bin/sh postgres_host=$1 postgres_port=$2 cmd="$@" # wait for the postgres docker to be running while ! pg_isready -h $postgres_host -p $postgres_port -q -U postgres; do >&2 echo "Postgres is unavailable - sleeping" sleep 1 done >&2 echo "Postgres is up - executing command" # run the command exec $cmd — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

I was able to do something like this
// start.sh

#!/bin/sh
set -eu

docker volume create --name=gql-sync
echo "Building docker containers"
docker-compose build
echo "Running tests inside docker container"
docker-compose up -d pubsub
docker-compose up -d mongo
docker-compose up -d botms
docker-compose up -d events
docker-compose up -d identity
docker-compose up -d importer
docker-compose run status
docker-compose run testing

exit $?

// status.sh

#!/bin/sh

set -eu pipefail

echo "Attempting to connect to bots"
until $(nc -zv botms 3000); do
    printf '.'
    sleep 5
done
echo "Attempting to connect to events"
until $(nc -zv events 3000); do
    printf '.'
    sleep 5
done
echo "Attempting to connect to identity"
until $(nc -zv identity 3000); do
    printf '.'
    sleep 5
done
echo "Attempting to connect to importer"
until $(nc -zv importer 8080); do
    printf '.'
    sleep 5
done
echo "Was able to connect to all"

exit 0

// in my docker compose file

  status:
    image: yikaus/alpine-bash
    volumes:
      - "./internals/scripts:/scripts"
    command: "sh /scripts/status.sh"
    depends_on:
      - "mongo"
      - "importer"
      - "events"
      - "identity"
      - "botms"

I have a similar problem but a bit different. I have to wait for MongoDB to start and initialize a replica set.
Im doing all of the procedure in docker. i.e. creating and authentication replica set. But I have another python script in which I have to connect to the primary node of the replica set. I'm getting an error there.

docker-compose.txt
Dockerfile.txt
and in the python script im trying to do something like this
for x in range(1, 4): client = MongoClient(host='node' + str(x), port=27017, username='admin', password='password') if client.is_primary: print('the client.address is: ' + str(client.address)) print(dbName) print(collectionName) break

Am having difficulty in doing so, anyone has any idea?

@patrickml

@patrickml If I don't use docker compose, How I you do it with Dockerfile?
I need 'cqlsh' to execute my build_all.cql. However, 'cqlsh' is not ready...have to wait for 60 seconds to be ready.

cat Dockerfile

FROM store/datastax/dse-server:5.1.8

USER root

RUN apt-get update
RUN apt-get install -y vim

ADD db-scripts-2.1.33.2-RFT-01.tar /docker/cms/
COPY entrypoint.sh /entrypoint.sh

WORKDIR /docker/cms/db-scripts-2.1.33.2/
RUN cqlsh -f build_all.cql

USER dse

=============

Step 8/9 : RUN cqlsh -f build_all.cql
---> Running in 08c8a854ebf4
Connection error: ('Unable to connect to any servers', {'127.0.0.1': error(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: Connection refused")})
The command '/bin/sh -c cqlsh -f build_all.cql' returned a non-zero code: 1

use fix for docker/compose#374

Remove db_clean service from docker-compose.yml

Wait 3 seconds to let server to start

Requires= var-lib-libvirt.mount var-lib-libvirt-images-ram.mount

In case anyone comes back to this years later, read this wonderful page from the Docker Compose docs for updated info!

There are also health checks built in now, which solve the problem better.

Wait 3 seconds to let server to start

Navigation Menu

Repository navigation

Is there a way to delay container startup to support dependant services with a longer startup time #374

302 remaining items

cat Dockerfile

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Is there a way to delay container startup to support dependant services with a longer startup time #374

Description

Activity

d11wtq commented on Aug 4, 2014

nubs commented on Aug 4, 2014

bfirsh commented on Aug 4, 2014

dancrumb commented on Aug 4, 2014

silarsis commented on Aug 4, 2014

dnephin commented on Aug 5, 2014

dsyer commented on Aug 14, 2014

jcalazan commented on Aug 15, 2014

arruda commented on Aug 16, 2014

prologic commented on Aug 19, 2014

chymian commented on Aug 19, 2014

codeitagile commented on Aug 22, 2014

prologic commented on Aug 22, 2014

shuron commented on Aug 31, 2014

ahknight commented on Oct 23, 2014

302 remaining items

slava-nikulin commented on May 11, 2017

riuvshin commented on May 11, 2017

mbdas commented on May 11, 2017

patrickml commented on Jun 22, 2017

usamaB commented on Oct 30, 2017

chaicode88 commented on Apr 26, 2018

cat Dockerfile

henroFall commented on Oct 27, 2020

Lunar2kPS commented on May 24, 2022

skorokithakis commented on May 24, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions