[GH-ISSUE #330] Update to 1.12.0 Healtcheck broken #2074

Closed
opened 2026-05-03 05:48:30 -05:00 by GiteaMirror · 17 comments
Owner

Originally created by @Serph91P on GitHub (Apr 28, 2026).
Original GitHub issue: https://github.com/fosrl/newt/issues/330

Describe the Bug

After the update to 1.12.0 all healtchecks are broken as soon as I disable them I can access the sites again. Else I get the Maintenance Screen I have enabled. Maybe related to: https://github.com/fosrl/newt/issues/329

Image Image

Environment

  • OS Type & Version: Ubuntu 26.04 and Ubuntu 24.04
  • Pangolin Version: ee-1.17.1
  • Gerbil Version: 1.3.1
  • Traefik Version: 3.6.14
  • Newt Version: 1.12.0
  • Olm Version: (if applicable)

To Reproduce

Install the setup with the versions already state and use docker socket to connect to services and try to use docker DNS and docker ip address.
Health checks don't work.

Expected Behavior

Health checks report a successful connection.

Originally created by @Serph91P on GitHub (Apr 28, 2026). Original GitHub issue: https://github.com/fosrl/newt/issues/330 ### Describe the Bug After the update to 1.12.0 all healtchecks are broken as soon as I disable them I can access the sites again. Else I get the Maintenance Screen I have enabled. Maybe related to: https://github.com/fosrl/newt/issues/329 <img width="653" height="794" alt="Image" src="https://github.com/user-attachments/assets/b9296107-82de-4791-b0d7-072a68752634" /> <img width="1246" height="115" alt="Image" src="https://github.com/user-attachments/assets/d84cd4fa-4bda-4a88-a3c0-32a625aea7c5" /> ### Environment - OS Type & Version: Ubuntu 26.04 and Ubuntu 24.04 - Pangolin Version: ee-1.17.1 - Gerbil Version: 1.3.1 - Traefik Version: 3.6.14 - Newt Version: 1.12.0 - Olm Version: (if applicable) ### To Reproduce Install the setup with the versions already state and use docker socket to connect to services and try to use docker DNS and docker ip address. Health checks don't work. ### Expected Behavior Health checks report a successful connection.
Author
Owner

@AstralDestiny commented on GitHub (Apr 28, 2026):

What are you trying to hit healtcheck wise? you can't just hit stuff that might serve a redirect can you show the service and healthcheck config?

<!-- gh-comment-id:4333519564 --> @AstralDestiny commented on GitHub (Apr 28, 2026): What are you trying to hit healtcheck wise? you can't just hit stuff that might serve a redirect can you show the service and healthcheck config?
Author
Owner

@Serph91P commented on GitHub (Apr 28, 2026):

What are you trying to hit healtcheck wise? you can't just hit stuff that might serve a redirect can you show the service and healthcheck config?

All of my healthchecks stopped working after updating to 1.12.0, while everything worked perfectly fine on 1.11.0. I did include a healthcheck config along with screenshots in my initial post. Bookstack was just one example, but the issue also affects Webmin, Nextcloud, and essentially all of my 80+ services.

I completely understand that more details are helpful for debugging, but I hope you can appreciate that providing individual configs for all 80+ services isn't really feasible. 😊
I'd be happy to share additional specific examples if that would help narrow things down!

<!-- gh-comment-id:4333715528 --> @Serph91P commented on GitHub (Apr 28, 2026): > What are you trying to hit healtcheck wise? you can't just hit stuff that might serve a redirect can you show the service and healthcheck config? All of my healthchecks stopped working after updating to 1.12.0, while everything worked perfectly fine on 1.11.0. I did include a healthcheck config along with screenshots in my initial post. Bookstack was just one example, but the issue also affects Webmin, Nextcloud, and essentially all of my 80+ services. I completely understand that more details are helpful for debugging, but I hope you can appreciate that providing individual configs for all 80+ services isn't really feasible. 😊 I'd be happy to share additional specific examples if that would help narrow things down!
Author
Owner

@maieredv-manuel commented on GitHub (Apr 28, 2026):

Same here!

Newt runs as Docker Container.

Some hosts were working, some not, all healthchecks were configured the same way - looking for "/"

Ive checked, one host, that isnt working redirects to "/login" - when i change the healthcheck to look for "/login" it works again ...

<!-- gh-comment-id:4334168045 --> @maieredv-manuel commented on GitHub (Apr 28, 2026): Same here! Newt runs as Docker Container. Some hosts were working, some not, all healthchecks were configured the same way - looking for "/" Ive checked, one host, that isnt working redirects to "/login" - when i change the healthcheck to look for "/login" it works again ...
Author
Owner

@LaurenceJJones commented on GitHub (Apr 28, 2026):

Ive checked, one host, that isnt working redirects to "/login" - when i change the healthcheck to look for "/login" it works again ...

This usually happens when your health check is targeting /, but the application responds with a redirect (for example to /login). Since the health check is expecting a success status, the redirect is treated as a failure.

By default, if you don’t specify valid status codes, it only considers responses in the 200–299 range as healthy. Redirects like 301 or 302 will fail unless you explicitly allow them.

@Serph91P, can you confirm whether BookStack on port 80 returns a 200 status for /? If it redirects or return any status outside of 200 status you defined.

I don’t remember if we have tools like curl or wget inside the Newt container, you can spin up a temporary container on the same network to test it:

docker run --rm -it --network <your_network_name> alpine sh

Then inside the container:

apk add --no-cache curl
curl -i http://bookstack:80/

This will let you verify the actual response code and confirm whether it’s returning a redirect.

<!-- gh-comment-id:4334422489 --> @LaurenceJJones commented on GitHub (Apr 28, 2026): > Ive checked, one host, that isnt working redirects to "/login" - when i change the healthcheck to look for "/login" it works again ... This usually happens when your health check is targeting `/`, but the application responds with a redirect (for example to `/login`). Since the health check is expecting a success status, the redirect is treated as a failure. By default, if you don’t specify valid status codes, it only considers responses in the `200–299` range as healthy. Redirects like `301` or `302` will fail unless you explicitly allow them. @Serph91P, can you confirm whether BookStack on port 80 returns a `200` status for `/`? If it redirects or return any status outside of 200 status you defined. I don’t remember if we have tools like `curl` or `wget` inside the Newt container, you can spin up a temporary container on the same network to test it: ```bash docker run --rm -it --network <your_network_name> alpine sh ``` Then inside the container: ```sh apk add --no-cache curl curl -i http://bookstack:80/ ``` This will let you verify the actual response code and confirm whether it’s returning a redirect.
Author
Owner

@maieredv-manuel commented on GitHub (Apr 28, 2026):

Strange that this starts with the latest 1.12.0 Version - with 1.11.0 all is working fine one "/" with Statuscode 200-299.

<!-- gh-comment-id:4334439802 --> @maieredv-manuel commented on GitHub (Apr 28, 2026): Strange that this starts with the latest 1.12.0 Version - with 1.11.0 all is working fine one "/" with Statuscode 200-299.
Author
Owner

@ElYoM commented on GitHub (Apr 28, 2026):

Exactly same for me as maierdev-manuel describe. All healthchecks cease to work with "/".

<!-- gh-comment-id:4334445747 --> @ElYoM commented on GitHub (Apr 28, 2026): Exactly same for me as maierdev-manuel describe. All healthchecks cease to work with "/".
Author
Owner

@LaurenceJJones commented on GitHub (Apr 28, 2026):

Just note that we have added failure thresholds in 1.12 which defaults to 1 so if you dont want to do the above, another option is to define the LOG_LEVEL=DEBUG inside the environment and it will print all healthchecks to stdout that may help us debug further as its quite verbose and will literally print the reason for the failure or success.

<!-- gh-comment-id:4334449118 --> @LaurenceJJones commented on GitHub (Apr 28, 2026): Just note that we have added failure thresholds in `1.12` which defaults to `1` so if you dont want to do the above, another option is to define the `LOG_LEVEL=DEBUG` inside the environment and it will print all healthchecks to stdout that may help us debug further as its quite verbose and will literally print the reason for the failure or success.
Author
Owner

@LaurenceJJones commented on GitHub (Apr 28, 2026):

Okay debugged a little further I think the main issue was that the default golang http client follow redirects by default, however, we added a configuration which will be exposed within 1.18 pangolin release which allows you to enable "follow redirect". The configuration within 1.12 defaults to false (since you cannot alter this configuration for now) which means now redirects are not blindly followed which was not the behavior of 1.11 which I expect most are now seeing.

I understand that this could be seen as a regression but a configured healthcheck which was actually hitting a redirect falls outside of the actual response code configured.

So for now either disable healthchecks until 1.18 release then you can configure "follow redirects" or configure the healthcheck to actually point towards a valid http path that returns the status code (primarily an actual health endpoint that most application have nowadays) within the default range or configure / with a 301 or 302 depending on the response code.

@oschwartz10612

<!-- gh-comment-id:4334555229 --> @LaurenceJJones commented on GitHub (Apr 28, 2026): Okay debugged a little further I think the main issue was that the default golang http client follow redirects by default, however, we added a configuration which will be exposed within `1.18` pangolin release which allows you to enable "follow redirect". The configuration within `1.12` defaults to false (since you cannot alter this configuration for now) which means now redirects are not blindly followed which was not the behavior of `1.11` which I expect most are now seeing. I understand that this could be seen as a regression but a configured healthcheck which was actually hitting a redirect falls outside of the actual response code configured. So for now either disable healthchecks until `1.18` release then you can configure "follow redirects" or configure the healthcheck to actually point towards a valid http path that returns the status code (primarily an actual health endpoint that most application have nowadays) within the default range or configure `/` with a `301` or `302` depending on the response code. @oschwartz10612
Author
Owner

@AstralDestiny commented on GitHub (Apr 28, 2026):

Following a redirect for a healthcheck is not the best unless you specifically asked for it, Though newt 1.12 is built for 1.18+ so it has a lot of the features that 1.18 will have in the end.

<!-- gh-comment-id:4334596834 --> @AstralDestiny commented on GitHub (Apr 28, 2026): Following a redirect for a healthcheck is not the best unless you specifically asked for it, Though newt ``1.12`` is built for ``1.18+`` so it has a lot of the features that ``1.18`` will have in the end.
Author
Owner

@Serph91P commented on GitHub (Apr 28, 2026):

Following a redirect for a healthcheck is not the best unless you specifically asked for it, Though newt 1.12 is built for 1.18+ so it has a lot of the features that 1.18 will have in the end.

That's a good point about redirect following it definitely makes sense to only allow that when explicitly configured!

Out of curiosity though: since Newt 1.12 seems to be built around features that are coming in Pangolin 1.18, which isn't released yet, was it also tested against the current stable Pangolin version? I'm just wondering about the reasoning behind releasing Newt 1.12 ahead of the Pangolin version it's designed for, no criticism at all, I'd just love to understand the intended workflow a bit better! 😊

<!-- gh-comment-id:4334671565 --> @Serph91P commented on GitHub (Apr 28, 2026): > Following a redirect for a healthcheck is not the best unless you specifically asked for it, Though newt `1.12` is built for `1.18+` so it has a lot of the features that `1.18` will have in the end. That's a good point about redirect following it definitely makes sense to only allow that when explicitly configured! Out of curiosity though: since Newt 1.12 seems to be built around features that are coming in Pangolin 1.18, which isn't released yet, was it also tested against the current stable Pangolin version? I'm just wondering about the reasoning behind releasing Newt 1.12 ahead of the Pangolin version it's designed for, no criticism at all, I'd just love to understand the intended workflow a bit better! 😊
Author
Owner

@Serph91P commented on GitHub (Apr 28, 2026):

Ive checked, one host, that isnt working redirects to "/login" - when i change the healthcheck to look for "/login" it works again ...

This usually happens when your health check is targeting /, but the application responds with a redirect (for example to /login). Since the health check is expecting a success status, the redirect is treated as a failure.

By default, if you don’t specify valid status codes, it only considers responses in the 200–299 range as healthy. Redirects like 301 or 302 will fail unless you explicitly allow them.

@Serph91P, can you confirm whether BookStack on port 80 returns a 200 status for /? If it redirects or return any status outside of 200 status you defined.

I don’t remember if we have tools like curl or wget inside the Newt container, you can spin up a temporary container on the same network to test it:

docker run --rm -it --network <your_network_name> alpine sh

Then inside the container:

apk add --no-cache curl
curl -i http://bookstack:80/

This will let you verify the actual response code and confirm whether it’s returning a redirect.

@LaurenceJJones Bookstack issues an 302 to /login.
But I think a lot of docker container do and the previous version worked fine with this. It has been written here that a lot of features in newt 1.12 are for pangolin 1.18+ so maybe the defaults should be adapted to fit the current latest version?

<!-- gh-comment-id:4334689835 --> @Serph91P commented on GitHub (Apr 28, 2026): > > Ive checked, one host, that isnt working redirects to "/login" - when i change the healthcheck to look for "/login" it works again ... > > This usually happens when your health check is targeting `/`, but the application responds with a redirect (for example to `/login`). Since the health check is expecting a success status, the redirect is treated as a failure. > > By default, if you don’t specify valid status codes, it only considers responses in the `200–299` range as healthy. Redirects like `301` or `302` will fail unless you explicitly allow them. > > [@Serph91P](https://github.com/Serph91P), can you confirm whether BookStack on port 80 returns a `200` status for `/`? If it redirects or return any status outside of 200 status you defined. > > I don’t remember if we have tools like `curl` or `wget` inside the Newt container, you can spin up a temporary container on the same network to test it: > > docker run --rm -it --network <your_network_name> alpine sh > > Then inside the container: > > apk add --no-cache curl > curl -i http://bookstack:80/ > > This will let you verify the actual response code and confirm whether it’s returning a redirect. @LaurenceJJones Bookstack issues an 302 to /login. But I think a lot of docker container do and the previous version worked fine with this. It has been written here that a lot of features in newt 1.12 are for pangolin 1.18+ so maybe the defaults should be adapted to fit the current latest version?
Author
Owner

@LaurenceJJones commented on GitHub (Apr 28, 2026):

Out of curiosity though: since Newt 1.12 seems to be built around features that are coming in Pangolin 1.18, which isn't released yet, was it also tested against the current stable Pangolin version? I'm just wondering about the reasoning behind releasing Newt 1.12 ahead of the Pangolin version it's designed for, no criticism at all, I'd just love to understand the intended workflow a bit better! 😊

The aim was to give users with larger deployments enough time to upgrade Newt first, so they could take advantage of the new features once the Pangolin release was available (which as far as im aware should be imminent).

Even though the new features are only available in 1.18 ultimately this still would of meant those healthchecks would fail on upgrade no matter if the configuration was available. Cause we always went in with the mindset that healthcheck is pointing towards a first response path, we didnt think to make it an opt out instead of a opt in. (EG: default to following redirects but a user can configure to disable it)

I've fed this back to the team, cause depending on the blow back we might want to reevaluate the opting path.

But I guess the problem was that since it did follow redirects by default, user configured a healthcheck it just worked and never actually spent time to configure an alternative path such as a health endpoint or the actual path that is returned from the redirect.

<!-- gh-comment-id:4334771378 --> @LaurenceJJones commented on GitHub (Apr 28, 2026): > Out of curiosity though: since Newt 1.12 seems to be built around features that are coming in Pangolin 1.18, which isn't released yet, was it also tested against the current stable Pangolin version? I'm just wondering about the reasoning behind releasing Newt 1.12 ahead of the Pangolin version it's designed for, no criticism at all, I'd just love to understand the intended workflow a bit better! 😊 The aim was to give users with larger deployments enough time to upgrade Newt first, so they could take advantage of the new features once the Pangolin release was available (which as far as im aware should be imminent). Even though the new features are only available in `1.18` ultimately this still would of meant those healthchecks would fail on upgrade no matter if the configuration was available. Cause we always went in with the mindset that healthcheck is pointing towards a first response path, we didnt think to make it an opt out instead of a opt in. (EG: default to following redirects but a user can configure to disable it) I've fed this back to the team, cause depending on the blow back we might want to reevaluate the opting path. But I guess the problem was that since it did follow redirects by default, user configured a healthcheck it just worked and never actually spent time to configure an alternative path such as a health endpoint or the actual path that is returned from the redirect.
Author
Owner

@Serph91P commented on GitHub (Apr 28, 2026):

Out of curiosity though: since Newt 1.12 seems to be built around features that are coming in Pangolin 1.18, which isn't released yet, was it also tested against the current stable Pangolin version? I'm just wondering about the reasoning behind releasing Newt 1.12 ahead of the Pangolin version it's designed for, no criticism at all, I'd just love to understand the intended workflow a bit better! 😊

The aim was to give users with larger deployments enough time to upgrade Newt first, so they could take advantage of the new features once the Pangolin release was available (which as far as im aware should be imminent).

Even though the new features are only available in 1.18 ultimately this still would of meant those healthchecks would fail on upgrade no matter if the configuration was available. Cause we always went in with the mindset that healthcheck is pointing towards a first response path, we didnt think to make it an opt out instead of a opt in. (EG: default to following redirects but a user can configure to disable it)

I've fed this back to the team, cause depending on the blow back we might want to reevaluate the opting path.

Got it thank you for the fast responses. Then I maybe have to check all my healthchecks and point them to a url which reports with an 200. I just pointed them all to / because it worked and never really bothered to use an "actual" path which responds with an 200.

<!-- gh-comment-id:4334791617 --> @Serph91P commented on GitHub (Apr 28, 2026): > > Out of curiosity though: since Newt 1.12 seems to be built around features that are coming in Pangolin 1.18, which isn't released yet, was it also tested against the current stable Pangolin version? I'm just wondering about the reasoning behind releasing Newt 1.12 ahead of the Pangolin version it's designed for, no criticism at all, I'd just love to understand the intended workflow a bit better! 😊 > > The aim was to give users with larger deployments enough time to upgrade Newt first, so they could take advantage of the new features once the Pangolin release was available (which as far as im aware should be imminent). > > Even though the new features are only available in `1.18` ultimately this still would of meant those healthchecks would fail on upgrade no matter if the configuration was available. Cause we always went in with the mindset that healthcheck is pointing towards a first response path, we didnt think to make it an opt out instead of a opt in. (EG: default to following redirects but a user can configure to disable it) > > I've fed this back to the team, cause depending on the blow back we might want to reevaluate the opting path. Got it thank you for the fast responses. Then I maybe have to check all my healthchecks and point them to a url which reports with an 200. I just pointed them all to / because it worked and never really bothered to use an "actual" path which responds with an 200.
Author
Owner

@cpuks commented on GitHub (Apr 28, 2026):

@LaurenceJJones the bug I submitted (#329) is duplicate of healthcheck issue, so I closed mine

<!-- gh-comment-id:4336241983 --> @cpuks commented on GitHub (Apr 28, 2026): @LaurenceJJones the bug I submitted (#329) is duplicate of healthcheck issue, so I closed mine
Author
Owner

@oschwartz10612 commented on GitHub (Apr 28, 2026):

Will do a patch update shortly to allow it to follow redirects when not configured from the new server version

<!-- gh-comment-id:4337195016 --> @oschwartz10612 commented on GitHub (Apr 28, 2026): Will do a patch update shortly to allow it to follow redirects when not configured from the new server version
Author
Owner

@oschwartz10612 commented on GitHub (Apr 28, 2026):

Try to update to https://github.com/fosrl/newt/releases/tag/1.12.1 and
LMK if fully resolved

<!-- gh-comment-id:4337743436 --> @oschwartz10612 commented on GitHub (Apr 28, 2026): Try to update to https://github.com/fosrl/newt/releases/tag/1.12.1 and LMK if fully resolved
Author
Owner

@Serph91P commented on GitHub (Apr 28, 2026):

Try to update to https://github.com/fosrl/newt/releases/tag/1.12.1 and
LMK if fully resolved

@oschwartz10612 yes works like a charm, thank you.

<!-- gh-comment-id:4337762824 --> @Serph91P commented on GitHub (Apr 28, 2026): > Try to update to https://github.com/fosrl/newt/releases/tag/1.12.1 and > LMK if fully resolved @oschwartz10612 yes works like a charm, thank you.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/newt#2074