[GH-ISSUE #327] dependency failed to start: container pangolin is unhealthy #8155

Closed
opened 2026-04-30 03:35:44 -05:00 by GiteaMirror · 26 comments
Owner

Originally created by @jayphizzle on GitHub (Mar 11, 2025).
Original GitHub issue: https://github.com/fosrl/pangolin/issues/327

Hi!

Tried latest pangolin today on a fresh installed Debian VPS.

Ran the installer, but the pangolin container does not start.

Image

Log of the container:

Image

If you need more information please let me know

Originally created by @jayphizzle on GitHub (Mar 11, 2025). Original GitHub issue: https://github.com/fosrl/pangolin/issues/327 Hi! Tried latest pangolin today on a fresh installed Debian VPS. Ran the installer, but the pangolin container does not start. <img width="848" alt="Image" src="https://github.com/user-attachments/assets/02eef152-8d6b-4943-98dc-f56aa11fe712" /> Log of the container: ![Image](https://github.com/user-attachments/assets/72eeee41-57d0-4d6b-97ce-d84d59bacee9) If you need more information please let me know
Author
Owner

@miloschwartz commented on GitHub (Mar 11, 2025):

Hey, looks like based on your screenshot of the logs that the Pangolin container is starting. Can you try to shut everything down and do a fresh restart? Are there anymore errors that show up in the logs potentially?

<!-- gh-comment-id:2715813739 --> @miloschwartz commented on GitHub (Mar 11, 2025): Hey, looks like based on your screenshot of the logs that the Pangolin container is starting. Can you try to shut everything down and do a fresh restart? Are there anymore errors that show up in the logs potentially?
Author
Owner

@jayphizzle commented on GitHub (Mar 12, 2025):

Image

what I noticed during installation:

stuck long time on the last pulls

Image

<!-- gh-comment-id:2718485811 --> @jayphizzle commented on GitHub (Mar 12, 2025): ![Image](https://github.com/user-attachments/assets/a29132d3-cb09-40a5-aef8-d76a67f4e422) what I noticed during installation: stuck long time on the last pulls ![Image](https://github.com/user-attachments/assets/cbaff191-1344-43d3-9543-e6fe35177766)
Author
Owner

@miloschwartz commented on GitHub (Mar 12, 2025):

Can you do sudo docker compose logs -f and look for any errors and send them our way so we can take a look?

<!-- gh-comment-id:2718782011 --> @miloschwartz commented on GitHub (Mar 12, 2025): Can you do `sudo docker compose logs -f` and look for any errors and send them our way so we can take a look?
Author
Owner

@jayphizzle commented on GitHub (Mar 12, 2025):

sudo docker compose logs -f

https://notebin.de/?b4d1d500bb15e1e3#BSXqBeTYH6ZAXagvukF4mnxkXxXDt7zwTR28JtiPPwiE

<!-- gh-comment-id:2718980414 --> @jayphizzle commented on GitHub (Mar 12, 2025): > `sudo docker compose logs -f` https://notebin.de/?b4d1d500bb15e1e3#BSXqBeTYH6ZAXagvukF4mnxkXxXDt7zwTR28JtiPPwiE
Author
Owner

@wedge-kc commented on GitHub (Mar 13, 2025):

I had a similar issue. After first docker compose up and unhealthy dependency try docker compose up again (without docker compose down) and see if it comes up. If so, you may need to increase the number of retries for the health check on the pangolin container. After doing that, a single docker compose up got everything up for me.

<!-- gh-comment-id:2719618780 --> @wedge-kc commented on GitHub (Mar 13, 2025): I had a similar issue. After first docker compose up and unhealthy dependency try docker compose up again (without docker compose down) and see if it comes up. If so, you may need to increase the number of retries for the health check on the pangolin container. After doing that, a single docker compose up got everything up for me.
Author
Owner

@miloschwartz commented on GitHub (Mar 13, 2025):

Yeah I am honestly really confused here because the logs seems to indicate Pangolin is healthy and the API started up where the health check occur.

pangolin  | Starting migrations from version 1.0.1
pangolin  | Migrations to run:
pangolin  | All migrations completed successfully
pangolin  | 2025-03-12T20:03:04.125Z [warn]: Email SMTP configuration is missing. Emails will not be sent.
pangolin  | 2025-03-12T20:03:09.171Z [info]: Server admin (****) already exists
pangolin  | 2025-03-12T20:03:15.253Z [info]: API server is running on http://localhost:3000/
pangolin  | 2025-03-12T20:03:15.261Z [info]: Internal server is running on http://localhost:3001/
pangolin  | 2025-03-12T20:03:19.405Z [info]: Next.js server is running on http://localhost:3002/
<!-- gh-comment-id:2719680020 --> @miloschwartz commented on GitHub (Mar 13, 2025): Yeah I am honestly really confused here because the logs seems to indicate Pangolin is healthy and the API started up where the health check occur. ```log pangolin | Starting migrations from version 1.0.1 pangolin | Migrations to run: pangolin | All migrations completed successfully pangolin | 2025-03-12T20:03:04.125Z [warn]: Email SMTP configuration is missing. Emails will not be sent. pangolin | 2025-03-12T20:03:09.171Z [info]: Server admin (****) already exists pangolin | 2025-03-12T20:03:15.253Z [info]: API server is running on http://localhost:3000/ pangolin | 2025-03-12T20:03:15.261Z [info]: Internal server is running on http://localhost:3001/ pangolin | 2025-03-12T20:03:19.405Z [info]: Next.js server is running on http://localhost:3002/ ```
Author
Owner

@oschwartz10612 commented on GitHub (Mar 13, 2025):

Could you maybe post your docker compose file? You could also try to docker exec -it pangolin curl -f http://localhost:3001/api/v1/ and see if that returns an error. Thats all the health check does

<!-- gh-comment-id:2719784035 --> @oschwartz10612 commented on GitHub (Mar 13, 2025): Could you maybe post your docker compose file? You could also try to `docker exec -it pangolin curl -f http://localhost:3001/api/v1/` and see if that returns an error. Thats all the health check does
Author
Owner

@jayphizzle commented on GitHub (Mar 13, 2025):

I had a similar issue. After first docker compose up and unhealthy dependency try docker compose up again (without docker compose down) and see if it comes up. If so, you may need to increase the number of retries for the health check on the pangolin container. After doing that, a single docker compose up got everything up for me.

changed retries to 10 - no changes

Could you maybe post your docker compose file? You could also try to docker exec -it pangolin curl -f http://localhost:3001/api/v1/ and see if that returns an error. Thats all the health check does

docker exec -it pangolin curl -f http://localhost:3001/api/v1/ -> {"message":"Healthy"}

i reinstalled my vps (netcup) with latest ubuntu - also same result.

docker-compose.yml https://pastebin.com/7Hp4cJNT (should be default one as I used the installer)

<!-- gh-comment-id:2722675715 --> @jayphizzle commented on GitHub (Mar 13, 2025): > I had a similar issue. After first docker compose up and unhealthy dependency try docker compose up again (without docker compose down) and see if it comes up. If so, you may need to increase the number of retries for the health check on the pangolin container. After doing that, a single docker compose up got everything up for me. changed retries to 10 - no changes > Could you maybe post your docker compose file? You could also try to `docker exec -it pangolin curl -f http://localhost:3001/api/v1/` and see if that returns an error. Thats all the health check does `docker exec -it pangolin curl -f http://localhost:3001/api/v1/ `-> `{"message":"Healthy"}` i reinstalled my vps (netcup) with latest ubuntu - also same result. docker-compose.yml https://pastebin.com/7Hp4cJNT (should be default one as I used the installer)
Author
Owner

@aszurnasirpal commented on GitHub (Mar 14, 2025):

I have the same problem.

Results of the docker exec -it pangolin curl -f http://localhost:3001/api/v1/ also return to me {"message":"Healthy"}

<!-- gh-comment-id:2723874199 --> @aszurnasirpal commented on GitHub (Mar 14, 2025): I have the same problem. Results of the `docker exec -it pangolin curl -f http://localhost:3001/api/v1/` also return to me `{"message":"Healthy"} `
Author
Owner

@EliasMasche commented on GitHub (Mar 14, 2025):

I have the same issue too but my Pangolin instance returns this error, I have SMTP and email address for Let's Encrypt

Image

<!-- gh-comment-id:2725847607 --> @EliasMasche commented on GitHub (Mar 14, 2025): I have the same issue too but my Pangolin instance returns this error, I have SMTP and email address for Let's Encrypt ![Image](https://github.com/user-attachments/assets/69fbaa66-4722-4e70-846c-7bf6d982b821)
Author
Owner

@miloschwartz commented on GitHub (Mar 14, 2025):

@EliasMasche Check your configuration file for the no-reply email and make sure it's valid. Does it have any weird formatting or special characters that might cause this invalid email error?

<!-- gh-comment-id:2725909397 --> @miloschwartz commented on GitHub (Mar 14, 2025): @EliasMasche Check your configuration file for the no-reply email and make sure it's valid. Does it have any weird formatting or special characters that might cause this invalid email error?
Author
Owner

@EliasMasche commented on GitHub (Mar 14, 2025):

@miloschwartz where is located the configuration file to change or I can run the installer to modify it?

<!-- gh-comment-id:2725921429 --> @EliasMasche commented on GitHub (Mar 14, 2025): @miloschwartz where is located the configuration file to change or I can run the installer to modify it?
Author
Owner

@miloschwartz commented on GitHub (Mar 14, 2025):

@EliasMasche The installer places the config files in the working directly used when you executed it. You would need to find that path, then view the config.yml file to check the no-reply email. No need to run the installer again since it already created the needed files to run everything.

<!-- gh-comment-id:2725925881 --> @miloschwartz commented on GitHub (Mar 14, 2025): @EliasMasche The installer places the config files in the working directly used when you executed it. You would need to find that path, then view the config.yml file to check the no-reply email. No need to run the installer again since it already created the needed files to run everything.
Author
Owner

@EliasMasche commented on GitHub (Mar 14, 2025):

Thanks for the detail on my no-reply email had <> on the field. Now is working

<!-- gh-comment-id:2725966490 --> @EliasMasche commented on GitHub (Mar 14, 2025): Thanks for the detail on my no-reply email had `<>` on the field. Now is working
Author
Owner

@syl779 commented on GitHub (Mar 15, 2025):

Same problem I had a few weeks ago. [https://github.com/fosrl/pangolin/issues/281](https://github.com/fosrl/pangolin/issues/281)

Still happens on most installs (I have installed about a dozen times now) - but only if you say "yes" to crowdsec.

<!-- gh-comment-id:2726606219 --> @syl779 commented on GitHub (Mar 15, 2025): Same problem I had a few weeks ago. [[https://github.com/fosrl/pangolin/issues/281](url)](https://github.com/fosrl/pangolin/issues/281) Still happens on most installs (I have installed about a dozen times now) - but only if you say "yes" to crowdsec.
Author
Owner

@jayphizzle commented on GitHub (Mar 15, 2025):

Same problem I had a few weeks ago. [https://github.com/fosrl/pangolin/issues/281](#281)

Still happens on most installs (I have installed about a dozen times now) - but only if you say "yes" to crowdsec.

Without crowdsec it is working for me too.

<!-- gh-comment-id:2726943514 --> @jayphizzle commented on GitHub (Mar 15, 2025): > Same problem I had a few weeks ago. [[https://github.com/fosrl/pangolin/issues/281](url)]([#281](https://github.com/fosrl/pangolin/issues/281)) > > Still happens on most installs (I have installed about a dozen times now) - but only if you say "yes" to crowdsec. Without crowdsec it is working for me too.
Author
Owner

@FunDeckHermit commented on GitHub (Mar 15, 2025):

Had the same problem during a couple (re)-installations. Happened after exactly 30 seconds on weak hardware.

<!-- gh-comment-id:2727017381 --> @FunDeckHermit commented on GitHub (Mar 15, 2025): Had the same problem during a couple (re)-installations. Happened after exactly 30 seconds on weak hardware.
Author
Owner

@jayphizzle commented on GitHub (Mar 16, 2025):

Had the same problem during a couple (re)-installations. Happened after exactly 30 seconds on weak hardware.

weak hardware fits also in my scenario: 1 vCore & 1 GB RAM VPS.

<!-- gh-comment-id:2727603689 --> @jayphizzle commented on GitHub (Mar 16, 2025): > Had the same problem during a couple (re)-installations. Happened after exactly 30 seconds on weak hardware. weak hardware fits also in my scenario: 1 vCore & 1 GB RAM VPS.
Author
Owner

@oschwartz10612 commented on GitHub (Mar 16, 2025):

Maybe we need more sensible timeouts and retries for the compose. You could increase the timeout and retries and see if that helps?

<!-- gh-comment-id:2727604152 --> @oschwartz10612 commented on GitHub (Mar 16, 2025): Maybe we need more sensible timeouts and retries for the compose. You could increase the timeout and retries and see if that helps?
Author
Owner

@FunDeckHermit commented on GitHub (Mar 16, 2025):

I think that would be the easiest method to fix the issue: increasing the timeout. Maybe add some automated test to measure the "Time to Healthy" in your CI pipeline.

<!-- gh-comment-id:2727617877 --> @FunDeckHermit commented on GitHub (Mar 16, 2025): I think that would be the easiest method to fix the issue: increasing the timeout. Maybe add some automated test to measure the "Time to Healthy" in your CI pipeline.
Author
Owner

@mrsreenair commented on GitHub (Mar 29, 2025):

I encountered a similar issue, which was resolved by increasing the virtual private server’s CPU and RAM.

<!-- gh-comment-id:2763784655 --> @mrsreenair commented on GitHub (Mar 29, 2025): I encountered a similar issue, which was resolved by increasing the virtual private server’s CPU and RAM.
Author
Owner

@odoncm commented on GitHub (Mar 31, 2025):

Or increase the RAM or CPU. That machine is very weak. It happened to me the other day while testing on an Oracle Free Tier VPS (1 CPU, 1 GB). In the end, I got it up and running like this:

healthcheck:
   test:
      - CMD
      - curl
      - -f
      - http://localhost:3001/api/v1/
   interval: 30s
   timeout: 10s
   retries: 4
   start_period: 30s

You'd have to adjust the timings, but it should start with that.


I'm also saying that I'm currently testing with another VPS (not Oracle) with 0.5 VCore and 1 GB, and it works without making any changes to the original ./installer.sh configuration.

I'm also more inclined to believe that there may be some slow processing on the machines on those VPSs.

<!-- gh-comment-id:2766236238 --> @odoncm commented on GitHub (Mar 31, 2025): Or increase the RAM or CPU. That machine is very weak. It happened to me the other day while testing on an Oracle Free Tier VPS (1 CPU, 1 GB). In the end, I got it up and running like this: ``` healthcheck: test: - CMD - curl - -f - http://localhost:3001/api/v1/ interval: 30s timeout: 10s retries: 4 start_period: 30s ``` You'd have to adjust the timings, but it should start with that. --- I'm also saying that I'm currently testing with another VPS (not Oracle) with 0.5 VCore and 1 GB, and it works without making any changes to the original ./installer.sh configuration. I'm also more inclined to believe that there may be some slow processing on the machines on those VPSs.
Author
Owner

@markmsv7 commented on GitHub (Apr 2, 2025):

Adding my similar experience. I'm a huge newb when it comes to selfhosting, so thank you for making this easy. It's almost as easy as those Tteck (RIP) Proxmox helper scripts.

That said, I started a Digital ocean droplet (1vCPU, 512MB, 10GB SSD w Ubuntu) and had same issue when installing crowdsec. Could not figure it out after installing it about 10 times and the difference was crowdsec vs not. I wanted crowdsec installed since again I'm a selfhosting newbie and this is my first proxy service I've attempted since I've been scared to try caddy, traefik, Nginx, etc for fear of improperly securing it.

Eventually I found through forums/docs that Ubuntu likes 1GB better.

So increasing to 1vCPU, 1GB, 25GB SSD tier fixed it to where crowdsec doesn't break things.

<!-- gh-comment-id:2772372495 --> @markmsv7 commented on GitHub (Apr 2, 2025): Adding my similar experience. I'm a huge newb when it comes to selfhosting, so thank you for making this easy. It's almost as easy as those Tteck (RIP) Proxmox helper scripts. That said, I started a Digital ocean droplet (1vCPU, 512MB, 10GB SSD w Ubuntu) and had same issue when installing crowdsec. Could not figure it out after installing it about 10 times and the difference was crowdsec vs not. I wanted crowdsec installed since again I'm a selfhosting newbie and this is my first proxy service I've attempted since I've been scared to try caddy, traefik, Nginx, etc for fear of improperly securing it. Eventually I found through forums/docs that Ubuntu likes 1GB better. So increasing to 1vCPU, 1GB, 25GB SSD tier fixed it to where crowdsec doesn't break things.
Author
Owner

@Maxklos commented on GitHub (Apr 4, 2025):

I had this once after an update, docker compose down then purging the whole image from the server and a new docker compose up -d fixed it for me

<!-- gh-comment-id:2779831682 --> @Maxklos commented on GitHub (Apr 4, 2025): I had this once after an update, `docker compose down` then purging the whole image from the server and a new `docker compose up -d` fixed it for me
Author
Owner

@Ard3ny commented on GitHub (Apr 7, 2025):

From what I've noticed, the problem only occur when crowdsec is enabled and VPS is somewhat "resource low" like 1vCPU, 1GB Ram etc...

Changing healthcheck timings in docker-compose file according to @odoncm helped.

<!-- gh-comment-id:2783406782 --> @Ard3ny commented on GitHub (Apr 7, 2025): From what I've noticed, the problem only occur when crowdsec is enabled and VPS is somewhat "resource low" like 1vCPU, 1GB Ram etc... Changing healthcheck timings in docker-compose file according to @odoncm helped.
Author
Owner

@oschwartz10612 commented on GitHub (Apr 22, 2025):

Closing for now. Please let us know if this issue continues but the first step to troubleshooting is playing with the retries and timeouts in the docker-compose.yml file as shown: 4d814b3714

<!-- gh-comment-id:2819892634 --> @oschwartz10612 commented on GitHub (Apr 22, 2025): Closing for now. Please let us know if this issue continues but the first step to troubleshooting is playing with the retries and timeouts in the docker-compose.yml file as shown: https://github.com/fosrl/docs/commit/4d814b37146e0e8ac18bb0761f526dfc17bf0501
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/pangolin#8155