Multiple hera instances appear to conflict with each other

[pirate]

I've noticed when using hera with multiple projects on the same host, the moment I try to run two instances of Hera they seem to fight with eachother.
Errors about "cant find container with id " show up in both hera instance's logs when run together, but when run individually there are no errors.

Steps to reproduce

1. Create projectA/docker-compose.yml:

version: "3"
                                                                                                                               
services:
  hera:
    image: aschzero/hera:latest
    container_name: projectA_hera
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./data/hera:/certs
    links:
      - web
    networks:
      - projectA
                                                                                                                               
  web:
    image: nginx:latest
    expose:
      - 8080
    volumes:
      - ./etc/nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./html:/var/www/html
    networks:
      - projectA
    labels:
      hera.hostname: projectA.example.com
      hera.port: 8080
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080"]

networks:
  projectA:

2. And then create projectB/docker-compose.yml with almost the same content, just a few lines changed:

version: "3"
                                                                                                                               
services:
  hera:
    image: aschzero/hera:latest
    container_name: projectB_hera
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./data/hera:/certs
    links:
      - web
    networks:
      - projectB
                                                                                                                               
  web:
    image: nginx:latest
    expose:
      - 8080
    volumes:
      - ./etc/nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./html:/var/www/html
    networks:
      - projectB
    labels:
      hera.hostname: projectB.example.com
      hera.port: 8080
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080"]

networks:
  projectB:

3. Start both projects cd projectA; docker-compose up -d; cd ../projectB; docker-compose up -d

4. Errors like this appear in both separate Hera instance's docker logs:

projectA_hera  | [INFO] Unable to connect, retrying... (4/5)
projectA_hera  | [INFO] Unable to connect, retrying... (5/5)
projectA_hera  | [ERROR] Unable to connect to e8f8bfe62cb8
projectA_hera  | [INFO] Container found, connecting to db3985d686ee...
projectA_hera  | [INFO] Unable to connect, retrying... (1/5)
projectA_hera  | [INFO] Unable to connect, retrying... (2/5)
projectA_hera  | [INFO] Unable to connect, retrying... (3/5)
projectA_hera  | [INFO] Unable to connect, retrying... (4/5)
projectA_hera  | [INFO] Unable to connect, retrying... (5/5)
projectA_hera  | [ERROR] Unable to connect to db3985d686ee
projectA_hera  | [INFO] Container found, connecting to 2754df6f4c5b...
projectA_hera  | [INFO] Unable to connect, retrying... (1/5)
projectA_hera  | [INFO] Unable to connect, retrying... (2/5)

[JEMeyer]

Are there any known workarounds for this? The whole point of using hera (for me) is to have tunnels to each of my containers and remove unnecessary reverse-proxies.

[pirate]

To be clear this only happens if you run multiple instances of the hera management container, not multiple tunnels to different containers controlled by a single hera container. The recommended setup is to have only one hera management instance per host, which sets up all the tunnels to each docker container automatically based on labels.

[JEMeyer]

I'll have to keep going through my logs then because I have one management container up, the first container I registered with it went flawlessly, but now I cannot get other containers to fully register. I either get the retry mentioned here or it loops on a connection refused on the IP of the container inside the hera network.

I believe one of my containers simply takes too long to startup and the 1/5 -> 5/5 retry kills it before it can accept, but that's just my working hypothesis.

The other container that was failing to register I'll chalk up to a bad image, as a different container was able to register. So the only behavior I'm troubleshooting now is the error message you get, but for a large container that takes a while to start up.