[GH-ISSUE #2968] Health is Unknown but healthcheck is healthy! #17265

Closed
opened 2026-05-18 17:44:50 -05:00 by GiteaMirror · 15 comments
Owner

Originally created by @itsbhanusharma on GitHub (May 2, 2026).
Original GitHub issue: https://github.com/fosrl/pangolin/issues/2968

Describe the Bug

I am trying to make some sense of what's exactly happening here or if I am missing something here, My healthchecks are passing for public sites but health stays at unknown.

Image Image

Environment

  • OS Type & Version: (e.g., Ubuntu 22.04) Debian 13
  • Pangolin Version: 1.18.1 (CE)
  • Gerbil Version: 1.3.1
  • Traefik Version: 3.6
  • Newt Version: 1.12.3
  • Olm Version: (if applicable)

To Reproduce

Visit Resources > Public page.

Expected Behavior

Health Column should reflect the health status, uptime bars should show up as expected.

Originally created by @itsbhanusharma on GitHub (May 2, 2026). Original GitHub issue: https://github.com/fosrl/pangolin/issues/2968 ### Describe the Bug I am trying to make some sense of what's exactly happening here or if I am missing something here, My healthchecks are passing for public sites but health stays at unknown. <img width="1017" height="478" alt="Image" src="https://github.com/user-attachments/assets/a03367ea-6b41-4d17-8b50-859a8ef2143d" /> <img width="1025" height="558" alt="Image" src="https://github.com/user-attachments/assets/e6da4d28-0884-45a7-a143-350db160f7fa" /> ### Environment - OS Type & Version: (e.g., Ubuntu 22.04) Debian 13 - Pangolin Version: 1.18.1 (CE) - Gerbil Version: 1.3.1 - Traefik Version: 3.6 - Newt Version: 1.12.3 - Olm Version: (if applicable) ### To Reproduce Visit Resources > Public page. ### Expected Behavior Health Column should reflect the health status, uptime bars should show up as expected.
Author
Owner

@svillar commented on GitHub (May 2, 2026):

I think the problem is that in the resource listing the healthcheck status is not properly updated. In my case I stopped newt in one of my resources, the healthcheck changed to unhealthy at some point, but when newt reconnected, the status didn't change. What's more, if you open the dropdown from healthcheck status it shows the site is healthy but the listing shows unhealthy.

<!-- gh-comment-id:4364048194 --> @svillar commented on GitHub (May 2, 2026): I think the problem is that in the resource listing the healthcheck status is not properly updated. In my case I stopped newt in one of my resources, the healthcheck changed to unhealthy at some point, but when newt reconnected, the status didn't change. What's more, if you open the dropdown from healthcheck status it shows the site is healthy but the listing shows unhealthy.
Author
Owner

@AstralDestiny commented on GitHub (May 2, 2026):

If you remove the target, save and re-add the target does the issue persist?

<!-- gh-comment-id:4364175191 --> @AstralDestiny commented on GitHub (May 2, 2026): If you remove the target, save and re-add the target does the issue persist?
Author
Owner

@itsbhanusharma commented on GitHub (May 2, 2026):

If you remove the target, save and re-add the target does the issue persist?

@AstralDestiny I deleted the target, Clicked save, then added the target again, enabled health check, saved again, the situation remains the same, Healthcheck works but health stays unknown, not sure if this has something to do with CE vs EE? (I am on CE and a lot of features are unable there)

<!-- gh-comment-id:4364397242 --> @itsbhanusharma commented on GitHub (May 2, 2026): > If you remove the target, save and re-add the target does the issue persist? @AstralDestiny I deleted the target, Clicked save, then added the target again, enabled health check, saved again, the situation remains the same, Healthcheck works but health stays unknown, not sure if this has something to do with CE vs EE? (I am on CE and a lot of features are unable there)
Author
Owner

@itsbhanusharma commented on GitHub (May 2, 2026):

I think the problem is that in the resource listing the healthcheck status is not properly updated. In my case I stopped newt in one of my resources, the healthcheck changed to unhealthy at some point, but when newt reconnected, the status didn't change. What's more, if you open the dropdown from healthcheck status it shows the site is healthy but the listing shows unhealthy.

@svillar I had thought of that possibility but I restarted everything just to be sure, mine stayed stuck at unknown even after everything was rebooted.

<!-- gh-comment-id:4364399745 --> @itsbhanusharma commented on GitHub (May 2, 2026): > I think the problem is that in the resource listing the healthcheck status is not properly updated. In my case I stopped newt in one of my resources, the healthcheck changed to unhealthy at some point, but when newt reconnected, the status didn't change. What's more, if you open the dropdown from healthcheck status it shows the site is healthy but the listing shows unhealthy. @svillar I had thought of that possibility but I restarted everything just to be sure, mine stayed stuck at unknown even after everything was rebooted.
Author
Owner

@oschwartz10612 commented on GitHub (May 2, 2026):

This should work just fine on the CE. I would expect deleting and recreating the target to work - but perhaps for some reason it is not and there is a bug there.

If you create a new resources does it work?

<!-- gh-comment-id:4364516007 --> @oschwartz10612 commented on GitHub (May 2, 2026): This should work just fine on the CE. I would expect deleting and recreating the target to work - but perhaps for some reason it is not and there is a bug there. If you create a new resources does it work?
Author
Owner

@itsbhanusharma commented on GitHub (May 2, 2026):

@oschwartz10612 I tried creating a new resource and it is still stuck on the same "unknown" status. if I click the drop down next to unknown it shows the resource as healthy. Check the attached screenshot.

Image

Next I can try setting up a fresh newt and trying to spin up a new site using that. Just curiously asking, would the install method for newt make a difference? all my newt instances are dockerized and I haven't tested with one installed on the host os.

<!-- gh-comment-id:4364561926 --> @itsbhanusharma commented on GitHub (May 2, 2026): @oschwartz10612 I tried creating a new resource and it is still stuck on the same "unknown" status. if I click the drop down next to unknown it shows the resource as healthy. Check the attached screenshot. <img width="559" height="81" alt="Image" src="https://github.com/user-attachments/assets/a97b1e2f-5945-4333-8c7d-ba4dce48607b" /> Next I can try setting up a fresh newt and trying to spin up a new site using that. Just curiously asking, would the install method for newt make a difference? all my newt instances are dockerized and I haven't tested with one installed on the host os.
Author
Owner

@svillar commented on GitHub (May 2, 2026):

@oschwartz10612 I tried creating a new resource and it is still stuck on the same "unknown" status. if I click the drop down next to unknown it shows the resource as healthy. Check the attached screenshot.

Image Next I can try setting up a fresh newt and trying to spin up a new site using that. Just curiously asking, would the install method for newt make a difference? all my newt instances are dockerized and I haven't tested with one installed on the host os.

That's exactly what I see and I use the binary distributed newt, so it does not depend on that.

<!-- gh-comment-id:4364596999 --> @svillar commented on GitHub (May 2, 2026): > [@oschwartz10612](https://github.com/oschwartz10612) I tried creating a new resource and it is still stuck on the same "unknown" status. if I click the drop down next to unknown it shows the resource as healthy. Check the attached screenshot. > > <img alt="Image" width="559" height="81" src="https://private-user-images.githubusercontent.com/9054382/586775436-a97b1e2f-5945-4333-8c7d-ba4dce48607b.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Nzc3NTE3NjIsIm5iZiI6MTc3Nzc1MTQ2MiwicGF0aCI6Ii85MDU0MzgyLzU4Njc3NTQzNi1hOTdiMWUyZi01OTQ1LTQzMzMtOGM3ZC1iYTRkY2U0ODYwN2IucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI2MDUwMiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNjA1MDJUMTk1MTAyWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ZDYxYTcyOTYxZDkyMDAyMmUxMDM2ZmFmMzc5OTMzZmJkMjY1Y2VmOGRmM2M2NGJiOWRiMTFiYmMyMDZiZjJiYiZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmcmVzcG9uc2UtY29udGVudC10eXBlPWltYWdlJTJGcG5nIn0.80zBQJYsKJtyTyFbg4cYo-HxJWaX3Ht7-ltI0xCyqHI"> > Next I can try setting up a fresh newt and trying to spin up a new site using that. Just curiously asking, would the install method for newt make a difference? all my newt instances are dockerized and I haven't tested with one installed on the host os. That's exactly what I see and I use the binary distributed newt, so it does not depend on that.
Author
Owner

@AstralDestiny commented on GitHub (May 2, 2026):

Image Image

Like on EE,

vs on CE,

Image

Even after many refreshes on the CE part, Yes CE is behind the EE but newt can connect just fine directly to the CE edition.

Healthcheck despite being unknown shows up in docker exec pangolin curl http://localhost:3001/api/v1/traefik-config | jq that it's in the routing config for traefik.
@oschwartz10612

<!-- gh-comment-id:4364600991 --> @AstralDestiny commented on GitHub (May 2, 2026): <img width="1715" height="345" alt="Image" src="https://github.com/user-attachments/assets/5380bfba-4ce6-4a96-b2e0-04a943a95d06" /> <img width="1892" height="277" alt="Image" src="https://github.com/user-attachments/assets/1c3a4330-cfa7-4845-81c3-792770f960f5" /> Like on EE, vs on CE, <img width="1501" height="196" alt="Image" src="https://github.com/user-attachments/assets/d0a445a9-6e1b-499c-9114-a8f25d7629ac" /> Even after many refreshes on the CE part, Yes CE is behind the EE but newt can connect just fine directly to the CE edition. Healthcheck despite being unknown shows up in `docker exec pangolin curl http://localhost:3001/api/v1/traefik-config | jq` that it's in the routing config for traefik. @oschwartz10612
Author
Owner

@oschwartz10612 commented on GitHub (May 2, 2026):

Ahh okay yes after looking into the code this is a CE issue. Will get a fix out in this next patch release today

<!-- gh-comment-id:4364654051 --> @oschwartz10612 commented on GitHub (May 2, 2026): Ahh okay yes after looking into the code this is a CE issue. Will get a fix out in this next patch release today
Author
Owner

@itsbhanusharma commented on GitHub (May 3, 2026):

Thanks @oschwartz10612

After updating to 1.18.2 I had to remove and re-add all the resources but once that was done, statuses seem to be reflecting correctly now.

Image
<!-- gh-comment-id:4365468564 --> @itsbhanusharma commented on GitHub (May 3, 2026): Thanks @oschwartz10612 After updating to 1.18.2 I had to remove and re-add all the resources but once that was done, statuses seem to be reflecting correctly now. <img width="449" height="426" alt="Image" src="https://github.com/user-attachments/assets/da81297c-1526-43d6-b780-7e546bac79b6" />
Author
Owner

@benello commented on GitHub (May 3, 2026):

I also had the same problem. Disabling and enabling the health check did the trick as well!

<!-- gh-comment-id:4365835338 --> @benello commented on GitHub (May 3, 2026): I also had the same problem. Disabling and enabling the health check did the trick as well!
Author
Owner

@Nanianmichaels commented on GitHub (May 4, 2026):

Thanks @oschwartz10612

After updating to 1.18.2 I had to remove and re-add all the resources but once that was done, statuses seem to be reflecting correctly now.

Had to do the same thing. Which, with over 20 targets, was a bit of a slog. It would be nice not to have to do that.

Weirdly enough, I had a couple of targets that showed proper health status without me needing to touch them, so I wonder if there's something else going on in the code that causes this kind of behavior...

<!-- gh-comment-id:4370257405 --> @Nanianmichaels commented on GitHub (May 4, 2026): > Thanks [@oschwartz10612](https://github.com/oschwartz10612) > > After updating to 1.18.2 I had to remove and re-add all the resources but once that was done, statuses seem to be reflecting correctly now. Had to do the same thing. Which, with over 20 targets, was a bit of a slog. It would be nice not to have to do that. Weirdly enough, I had a couple of targets that showed proper health status without me needing to touch them, so I wonder if there's something else going on in the code that causes this kind of behavior...
Author
Owner

@oschwartz10612 commented on GitHub (May 4, 2026):

Unfortunately this effects people who used the original 1.18.0 migration before it was patched in the 1.18.1 version. I added a migration to the 1.18.3 version that will come out earlier this week to help fix this and the mismatch between the health check status and resource status issue.

<!-- gh-comment-id:4373511506 --> @oschwartz10612 commented on GitHub (May 4, 2026): Unfortunately this effects people who used the original 1.18.0 migration before it was patched in the 1.18.1 version. I added a migration to the 1.18.3 version that will come out earlier this week to help fix this and the mismatch between the health check status and resource status issue.
Author
Owner

@fpillajo commented on GitHub (May 8, 2026):

I upgraded from version 1.16.2 CE to version 1.18.3 CE and no information appears on the sites or in the resources.

Screenshot Site

Image

Screenshot Resources

Image
<!-- gh-comment-id:4409978886 --> @fpillajo commented on GitHub (May 8, 2026): I upgraded from version 1.16.2 CE to version 1.18.3 CE and no information appears on the sites or in the resources. Screenshot Site <img width="381" height="228" alt="Image" src="https://github.com/user-attachments/assets/f4dd5668-ad08-4152-b3c6-74eec1e1a49a" /> Screenshot Resources <img width="456" height="276" alt="Image" src="https://github.com/user-attachments/assets/f87ef759-1271-43d5-b9a5-2450252a3ffb" />
Author
Owner

@oschwartz10612 commented on GitHub (May 8, 2026):

Yeah this is a known bug will be fixed in the next patch

It is excluding data for "today"

<!-- gh-comment-id:4410116407 --> @oschwartz10612 commented on GitHub (May 8, 2026): Yeah this is a known bug will be fixed in the next patch It is excluding data for "today"
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/pangolin#17265