Availability to Configure Komodo Health Check Connection Timeout to Servers #142

New Issue

GiteaMirror · 2025-10-31T15:03:11-05:00

GiteaMirror commented

2025-10-31 15:03:11 -05:00

Originally created by @carl-ac on GitHub (Nov 29, 2024).

Hi,

We’ve encountered an issue where some of our team’s servers are consistently entering a "not ok" status in Komodo.
However, upon investigation, the servers themselves appear to be functioning normally, and there are no indications of network issues (e.g., firewall rules blocking traffic).

Observations:
After further debugging, we suspect the issue might be related to network latency between the Komodo server and the target server. Specifically, the network latency appears to exceed Komodo's default connection timeout for health checks.

Here are some latency measurements we observed:

64 bytes from 10.*.*.* (10.*.*.*): icmp_seq=1 ttl=61 time=255 ms  
64 bytes from 10.*.*.* (10.*.*.*): icmp_seq=2 ttl=61 time=254 ms  
64 bytes from 10.*.*.* (10.*.*.*): icmp_seq=3 ttl=61 time=255 ms

Additionally, we found that the request time from the Komodo server to the target server can occasionally exceed 1 second.

Feature Request:
It would be highly beneficial to have the ability to configure the health check connection timeout for servers in Komodo. This feature would allow us to adjust the timeout to accommodate servers with higher network latency.

Originally created by @carl-ac on GitHub (Nov 29, 2024). Hi, We’ve encountered an issue where some of our team’s servers are consistently entering a "not ok" status in Komodo. However, upon investigation, the servers themselves appear to be functioning normally, and there are no indications of network issues (e.g., firewall rules blocking traffic). Observations: After further debugging, we suspect the issue might be related to network latency between the Komodo server and the target server. Specifically, the network latency appears to exceed Komodo's default connection timeout for health checks. Here are some latency measurements we observed: ``` 64 bytes from 10.*.*.* (10.*.*.*): icmp_seq=1 ttl=61 time=255 ms 64 bytes from 10.*.*.* (10.*.*.*): icmp_seq=2 ttl=61 time=254 ms 64 bytes from 10.*.*.* (10.*.*.*): icmp_seq=3 ttl=61 time=255 ms ``` Additionally, we found that the request time from the Komodo server to the target server can occasionally exceed 1 second. Feature Request: It would be highly beneficial to have the ability to configure the health check connection timeout for servers in Komodo. This feature would allow us to adjust the timeout to accommodate servers with higher network latency.

GiteaMirror added the done label 2025-10-31 15:03:11 -05:00

GiteaMirror closed this issue

2025-10-31 15:03:11 -05:00

GiteaMirror commented

2025-10-31 15:03:12 -05:00

@mbecker20 commented on GitHub (Nov 29, 2024):

Thanks for the report, the timeout currently default to only 1 second and it is not configurable. This is a good reason to make the timeout configurable per-server, and just keep the default 1 second.

@mbecker20 commented on GitHub (Nov 29, 2024): Thanks for the report, the timeout currently default to only 1 second and it is not configurable. This is a good reason to make the timeout configurable per-server, and just keep the default 1 second.

GiteaMirror commented

2025-10-31 15:03:12 -05:00

@carl-ac commented on GitHub (Nov 29, 2024):

Yes it's good idea and thanks for the feedback!

@carl-ac commented on GitHub (Nov 29, 2024): Yes it's good idea and thanks for the feedback!

GiteaMirror commented

2025-10-31 15:03:13 -05:00

@gitcatpeter commented on GitHub (Nov 29, 2024):

+1
I was just to report the same. I'm getting random "Server Unreachable" which are immediately followed by cancellations.
This only happens to last two servers (out of 6) I added to Komodo - if being the last has any meaning for polling....

Very lightly loaded servers in the same subnet as Komodo-core, even same hypervisor.
No issues observed in logs.

@gitcatpeter commented on GitHub (Nov 29, 2024): +1 I was just to report the same. I'm getting random "Server Unreachable" which are immediately followed by cancellations. This only happens to last two servers (out of 6) I added to Komodo - if being the last has any meaning for polling.... Very lightly loaded servers in the same subnet as Komodo-core, even same hypervisor. No issues observed in logs.

GiteaMirror commented

2025-10-31 15:03:13 -05:00

@gitcatpeter commented on GitHub (Nov 30, 2024):

Just to give you some idea, Zabbix uses 3s by default, configurable in the range of 1-30s
https://www.zabbix.com/documentation/current/en/manual/appendix/config/zabbix_server#timeout

@gitcatpeter commented on GitHub (Nov 30, 2024): Just to give you some idea, Zabbix uses 3s by default, configurable in the range of 1-30s https://www.zabbix.com/documentation/current/en/manual/appendix/config/zabbix_server#timeout

GiteaMirror commented

2025-10-31 15:03:13 -05:00

@gitcatpeter commented on GitHub (Dec 3, 2024):

3s timeout solved the problem for me.

@gitcatpeter commented on GitHub (Dec 3, 2024): 3s timeout solved the problem for me.

GiteaMirror commented

2025-10-31 15:03:13 -05:00

@carl-ac commented on GitHub (Dec 4, 2024):

Thanks so much for handling this!

@carl-ac commented on GitHub (Dec 4, 2024): Thanks so much for handling this!