Enabling ElasticSearch causes 502 Bad Gateway at random #11727

Closed
opened 2025-11-02 09:45:58 -06:00 by GiteaMirror · 4 comments
Owner

Originally created by @Aetherinox on GitHub (Sep 26, 2023).

Description

Sort of looking for some direction here if possible.

Today I switched ISSUE_INDEXER_TYPE to elastic search, and I get random 502 Bad Gateway errors. I can refresh the page once, and the site works fine, however, if I refresh it again, I get the 502 Bad Gateway. Then another refresh makes the site come back. Sometimes I have to refresh 2-3 times and then it works for a few refreshes. Completely random.

As soon as I set ISSUE_INDEXER_TYPE to bleve the site runs fine again.

Could this be some type of memory issue? ElasticSearch comes with Gitea, so I'm not sure if it's a resource issue and I need to increase the allocation. I'm used to standalone installs of ElasticSearch (currently not installed).

I also noticed if I go to the Site Administration section, the Server Uptime resets back to 0 and starts counting up again like it's crashing.

If I go to http://localhost:9200/ I get

{
  "name" : "localhost",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "xxxxxx",
  "version" : {
    "number" : "7.17.13",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "2b211dbb8bfdecaf7f5b44d356bdfe54b1050c13",
    "build_date" : "2023-08-31T17:33:19.958690787Z",
    "build_snapshot" : false,
    "lucene_version" : "8.11.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

App.ini [indexer]

[indexer]
REPO_INDEXER_ENABLED = true
ISSUE_INDEXER_TYPE = elasticsearch
REPO_INDEXER_PATH = indexers/repos.elasticsearch
ISSUE_INDEXER_CONN_STR = http://elastic:elastic@localhost:9200
MAX_FILE_SIZE = 1048576
REPO_INDEXER_INCLUDE =
REPO_INDEXER_EXCLUDE = resources/bin/**

Update

So I noticed after writing my ticket that my version of Elastic was rather outdated (v7), I installed v8, and receive

{
  "name" : "localhost",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "xxxxxx",
  "version" : {
    "number" : "8.10.2",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "6d20dd8ce62365be9b1aca96427de4622e970e9e",
    "build_date" : "2023-09-19T08:16:24.564900370Z",
    "build_snapshot" : false,
    "lucene_version" : "9.7.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

It appears with this version, I am able to browse the site more. But it does still give the Bad Gateway error about 1 out of every 10 - 15 refreshes. Whereas with v7 it was about 1 out of every 2 or 3.

Still looking into it.

I'm not sure if this is a Gitea issue, or Elastic itself. I tested elastic using some forum software I run, and search queries / connection seem to be fine there.


Gitea Version

gitea-1.20.4-linux-amd64

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

https://gist.github.com/Aetherinox/56f9428f9152147cfa5c569cee5c85dc

Screenshots

No response

Git Version

2.25.1

Operating System

Ubuntu 22 LTS

How are you running Gitea?

Manual download via

sudo wget -O /usr/local/bin/gitea https://dl.gitea.com/gitea/1.20.4/gitea-1.20.4-linux-amd64
systemctl start gitea

Database

MySQL/MariaDB

Originally created by @Aetherinox on GitHub (Sep 26, 2023). ### Description Sort of looking for some direction here if possible. Today I switched `ISSUE_INDEXER_TYPE` to elastic search, and I get random 502 Bad Gateway errors. I can refresh the page once, and the site works fine, however, if I refresh it again, I get the 502 Bad Gateway. Then another refresh makes the site come back. Sometimes I have to refresh 2-3 times and then it works for a few refreshes. Completely random. As soon as I set `ISSUE_INDEXER_TYPE` to `bleve` the site runs fine again. Could this be some type of memory issue? ElasticSearch comes with Gitea, so I'm not sure if it's a resource issue and I need to increase the allocation. I'm used to standalone installs of ElasticSearch (currently not installed). I also noticed if I go to the Site Administration section, the `Server Uptime` resets back to 0 and starts counting up again like it's crashing. If I go to `http://localhost:9200/` I get ```json { "name" : "localhost", "cluster_name" : "elasticsearch", "cluster_uuid" : "xxxxxx", "version" : { "number" : "7.17.13", "build_flavor" : "default", "build_type" : "deb", "build_hash" : "2b211dbb8bfdecaf7f5b44d356bdfe54b1050c13", "build_date" : "2023-08-31T17:33:19.958690787Z", "build_snapshot" : false, "lucene_version" : "8.11.1", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" } ``` ### App.ini [indexer] ```ini [indexer] REPO_INDEXER_ENABLED = true ISSUE_INDEXER_TYPE = elasticsearch REPO_INDEXER_PATH = indexers/repos.elasticsearch ISSUE_INDEXER_CONN_STR = http://elastic:elastic@localhost:9200 MAX_FILE_SIZE = 1048576 REPO_INDEXER_INCLUDE = REPO_INDEXER_EXCLUDE = resources/bin/** ``` --- ## Update So I noticed after writing my ticket that my version of Elastic was rather outdated (v7), I installed v8, and receive ```json { "name" : "localhost", "cluster_name" : "elasticsearch", "cluster_uuid" : "xxxxxx", "version" : { "number" : "8.10.2", "build_flavor" : "default", "build_type" : "deb", "build_hash" : "6d20dd8ce62365be9b1aca96427de4622e970e9e", "build_date" : "2023-09-19T08:16:24.564900370Z", "build_snapshot" : false, "lucene_version" : "9.7.0", "minimum_wire_compatibility_version" : "7.17.0", "minimum_index_compatibility_version" : "7.0.0" }, "tagline" : "You Know, for Search" } ``` It appears with this version, I am able to browse the site more. But it does still give the Bad Gateway error about 1 out of every 10 - 15 refreshes. Whereas with v7 it was about 1 out of every 2 or 3. Still looking into it. I'm not sure if this is a Gitea issue, or Elastic itself. I tested elastic using some forum software I run, and search queries / connection seem to be fine there. --- ### Gitea Version gitea-1.20.4-linux-amd64 ### Can you reproduce the bug on the Gitea demo site? No ### Log Gist https://gist.github.com/Aetherinox/56f9428f9152147cfa5c569cee5c85dc ### Screenshots _No response_ ### Git Version 2.25.1 ### Operating System Ubuntu 22 LTS ### How are you running Gitea? Manual download via ``` sudo wget -O /usr/local/bin/gitea https://dl.gitea.com/gitea/1.20.4/gitea-1.20.4-linux-amd64 systemctl start gitea ``` ### Database MySQL/MariaDB
GiteaMirror added the issue/needs-feedback label 2025-11-02 09:45:58 -06:00
Author
Owner

@lunny commented on GitHub (Sep 26, 2023):

According to the log

2023/09/26 00:56:08 ...er/issues/indexer.go:189:func2() [F] Unable to initialize Elastic Search Issue Indexer at connection:  Error: health check timeout: no Elasticsearch node available

Your configuration [indexer].IssueConnStr is not right.

@lunny commented on GitHub (Sep 26, 2023): According to the log ```log 2023/09/26 00:56:08 ...er/issues/indexer.go:189:func2() [F] Unable to initialize Elastic Search Issue Indexer at connection: Error: health check timeout: no Elasticsearch node available ``` Your configuration `[indexer].IssueConnStr` is not right.
Author
Owner

@Aetherinox commented on GitHub (Sep 26, 2023):

So, the ISSUE_INDEXER_CONN_STR was "sort of" right.

The password I pasted above wasn't the password I used for the elastic user. It was more complex. One of the characters in the password is what caused it to freak out.

Changing the password to remove the character# made the full password begin to work. In the config, I had tried with and without single / double quotes but neither worked. Once I went into elastic search, changed the elastic user password, and then updated that in the gitea config, it started running fine.

@Aetherinox commented on GitHub (Sep 26, 2023): So, the `ISSUE_INDEXER_CONN_STR ` was "sort of" right. The password I pasted above wasn't the password I used for the `elastic` user. It was more complex. One of the characters in the password is what caused it to freak out. Changing the password to remove the character`#` made the full password begin to work. In the config, I had tried with and without single / double quotes but neither worked. Once I went into elastic search, changed the `elastic` user password, and then updated that in the gitea config, it started running fine.
Author
Owner

@techknowlogick commented on GitHub (Sep 26, 2023):

@Aetherinox could you try wrapping it in ` single ticks? I'll close this now, but I'd be interested in if you do wrap it and it starts working (as single ticks do escape ini formatting)

@techknowlogick commented on GitHub (Sep 26, 2023): @Aetherinox could you try wrapping it in \` single ticks? I'll close this now, but I'd be interested in if you do wrap it and it starts working (as single ticks do escape ini formatting)
Author
Owner

@Aetherinox commented on GitHub (Sep 27, 2023):

@techknowlogick Interesting. Didn't know about the ticks.

Yes it appears that a complex password does work if you wrap it in ticks. I tried it a few ways this evening and ticks are the only ones that work.

That's partially why I opted to upgrade Elastic last night, because I went to test changing the user password on elastic v7 and the docs kept mentioning a file called ./elasticsearch-reset-password which v7 has nowhere. So I went searching and noticed that it mentioned v8 was the version with this file, which is how I came to realize I had a way outdated version.

Sorry for the trouble. Elastic in Gitea works great now.

As a note, I usually have pretty complex passwords that use numerous special characters, I don't know why, but the pound symbol is what it had issues with. It cut off everything after # in the password the first time.

@Aetherinox commented on GitHub (Sep 27, 2023): @techknowlogick Interesting. Didn't know about the ticks. Yes it appears that a complex password does work if you wrap it in ticks. I tried it a few ways this evening and ticks are the only ones that work. That's partially why I opted to upgrade Elastic last night, because I went to test changing the user password on elastic v7 and the docs kept mentioning a file called `./elasticsearch-reset-password ` which v7 has nowhere. So I went searching and noticed that it mentioned v8 was the version with this file, which is how I came to realize I had a way outdated version. Sorry for the trouble. Elastic in Gitea works great now. As a note, I usually have pretty complex passwords that use numerous special characters, I don't know why, but the pound symbol is what it had issues with. It cut off everything after `#` in the password the first time.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#11727