How to search for file and folder name inside repo? #4323

@ili101 commented on GitHub (May 6, 2020):

@ili101 we gladly welcome PRs. 😁

That's an awesome thing! unforgivably Go is not one of my expertise (yet) 😁

@ili101 commented on GitHub (May 6, 2020): > > > @ili101 we gladly welcome PRs. 😁 That's an awesome thing! unforgivably Go is not one of my expertise (yet) 😁

GiteaMirror commented

@love1900905 commented on GitHub (Sep 17, 2021):

+1, for many time, filename/pathname is even more meaningful than file content.

@love1900905 commented on GitHub (Sep 17, 2021): +1, for many time, filename/pathname is even more meaningful than file content.

GiteaMirror commented

@rengui commented on GitHub (Dec 17, 2021):

First of all, thanks great gitea! I'm using gitea heavyly and it works lightweightly and smoothly for years!

Back to this topic, I need this feature as well recently. After kinds of struggling,

gitea code search, kinds of config => no supported yet.
Kooder (a gitee open source code search tool for gitlab/gitee/gitea), can search filename itself, but not pathname yet (https://gitee.com/koode/kooder/issues/I4MWJA?from=project-issue).

It was resolved today, by change 1 line of code (gitea branch 1.15.7, for bleve search only):

// change modules\indexer\code\elastic_search.go if you are using ES.
modules\indexer\code\bleve.go

func (b *BleveIndexer) addUpdate(batchWriter ...
	......
	return batch.Index(id, &RepoIndexerData{
	RepoID:   repo.ID,
		CommitID: commitSha,
		Content:   string(charset.ToUTF8DropErrors(fileContents)),    // <--- this line

==>

	// Content:   string("Pathname: ") + string(update.Filename) + string(" \n") + string(charset.ToUTF8DropErrors(fileContents)),
	// replace '/' with ' / ', so that bleve think  '/foo/bar.txt' as  '/ foo / bar.txt'
	//   (then can be searched by both 'foo' and 'bar'; otherwise bleve only like '/foo/bar.txt' ???)
	Content:   string("Pathname: ") + strings.ReplaceAll(string(update.Filename), "/", " / ") + string(" \n") + string(charset.ToUTF8DropErrors(fileContents)),

then build your own gitea.exe and enjoy it :)

After replace this gitea.exe, you may need to delete all the code search index files (app.ini: REPO_INDEXER_PATH = ), restart gitea to make a full indexing (with pathname search enabled)

@rengui commented on GitHub (Dec 17, 2021): First of all, thanks great gitea! I'm using gitea heavyly and it works lightweightly and smoothly for years! Back to this topic, I need this feature as well recently. After kinds of struggling, - gitea code search, kinds of config => no supported yet. - Kooder (a gitee open source code search tool for gitlab/gitee/gitea), can search filename itself, but not pathname yet (https://gitee.com/koode/kooder/issues/I4MWJA?from=project-issue). It was resolved today, by change 1 line of code (gitea branch 1.15.7, for bleve search only): // change modules\indexer\code\elastic_search.go if you are using ES. modules\indexer\code\bleve.go ``` func (b *BleveIndexer) addUpdate(batchWriter ... ...... return batch.Index(id, &RepoIndexerData{ RepoID: repo.ID, CommitID: commitSha, Content: string(charset.ToUTF8DropErrors(fileContents)), // <--- this line ``` ==> ``` // Content: string("Pathname: ") + string(update.Filename) + string(" \n") + string(charset.ToUTF8DropErrors(fileContents)), // replace '/' with ' / ', so that bleve think '/foo/bar.txt' as '/ foo / bar.txt' // (then can be searched by both 'foo' and 'bar'; otherwise bleve only like '/foo/bar.txt' ???) Content: string("Pathname: ") + strings.ReplaceAll(string(update.Filename), "/", " / ") + string(" \n") + string(charset.ToUTF8DropErrors(fileContents)), ``` then build your own gitea.exe and enjoy it :) After replace this gitea.exe, you may need to delete all the code search index files (app.ini: REPO_INDEXER_PATH = ), restart gitea to make a full indexing (with pathname search enabled)

GiteaMirror commented

@lunny commented on GitHub (Dec 17, 2021):

First of all, thanks great gitea! I'm using gitea heavyly and it works lightweightly and smoothly for years!

Back to this topic, I need this feature as well recently. After kinds of struggling,

gitea code search, kinds of config => no supported yet.

Kooder (a gitee open source code search tool for gitlab/gitee/gitea), can search filename itself, but not pathname yet (https://gitee.com/koode/kooder/issues/I4MWJA?from=project-issue).

It was resolved today, by change 1 line of code (gitea branch 1.15.7, for bleve search only):

// change modules\indexer\code\elastic_search.go if you are using ES. modules\indexer\code\bleve.go
func (b *BleveIndexer) addUpdate(batchWriter ...
	......
	return batch.Index(id, &RepoIndexerData{
	RepoID:   repo.ID,
		CommitID: commitSha,
		Content:   string(charset.ToUTF8DropErrors(fileContents)),    // <--- this line
==>
	// Content:   string("Pathname: ") + string(update.Filename) + string(" \n") + string(charset.ToUTF8DropErrors(fileContents)),
	// replace '/' with ' / ', so that bleve think  '/foo/bar.txt' as  '/ foo / bar.txt'
	//   (then can be searched by both 'foo' and 'bar'; otherwise bleve only like '/foo/bar.txt' ???)
	Content:   string("Pathname: ") + strings.ReplaceAll(string(update.Filename), "/", " / ") + string(" \n") + string(charset.ToUTF8DropErrors(fileContents)),
then build your own gitea.exe and enjoy it :)

After replace this gitea.exe, you may need to delete all the code search index files (app.ini: REPO_INDEXER_PATH = ), restart gitea to make a full indexing (with pathname search enabled)

A general method is to add a new field Filename in RepoIndexerData.

@lunny commented on GitHub (Dec 17, 2021): > First of all, thanks great gitea! I'm using gitea heavyly and it works lightweightly and smoothly for years! > > Back to this topic, I need this feature as well recently. After kinds of struggling, > > * gitea code search, kinds of config => no supported yet. > * Kooder (a gitee open source code search tool for gitlab/gitee/gitea), can search filename itself, but not pathname yet (https://gitee.com/koode/kooder/issues/I4MWJA?from=project-issue). > > It was resolved today, by change 1 line of code (gitea branch 1.15.7, for bleve search only): > > // change modules\indexer\code\elastic_search.go if you are using ES. modules\indexer\code\bleve.go > > ``` > func (b *BleveIndexer) addUpdate(batchWriter ... > ...... > return batch.Index(id, &RepoIndexerData{ > RepoID: repo.ID, > CommitID: commitSha, > Content: string(charset.ToUTF8DropErrors(fileContents)), // <--- this line > ``` > > ==> > > ``` > // Content: string("Pathname: ") + string(update.Filename) + string(" \n") + string(charset.ToUTF8DropErrors(fileContents)), > // replace '/' with ' / ', so that bleve think '/foo/bar.txt' as '/ foo / bar.txt' > // (then can be searched by both 'foo' and 'bar'; otherwise bleve only like '/foo/bar.txt' ???) > Content: string("Pathname: ") + strings.ReplaceAll(string(update.Filename), "/", " / ") + string(" \n") + string(charset.ToUTF8DropErrors(fileContents)), > ``` > > then build your own gitea.exe and enjoy it :) > > After replace this gitea.exe, you may need to delete all the code search index files (app.ini: REPO_INDEXER_PATH = ), restart gitea to make a full indexing (with pathname search enabled) A general method is to add a new field `Filename` in `RepoIndexerData`.

GiteaMirror commented

@rengui commented on GitHub (Dec 17, 2021):

A general method is to add a new field Filename in RepoIndexerData.

Yes you are right, from gitea deve pov, it shall go in a general way, ensure code quality and extensibility.
What I mentioned was just from a gitea user pov, try to hack and get a working WA asap. :)

@rengui commented on GitHub (Dec 17, 2021): > > A general method is to add a new field `Filename` in `RepoIndexerData`. Yes you are right, from gitea deve pov, it shall go in a general way, ensure code quality and extensibility. What I mentioned was just from a gitea user pov, try to hack and get a working WA asap. :)

GiteaMirror commented

@delanym commented on GitHub (Jun 30, 2022):

@lunny why cant your workaround be a PR?

@delanym commented on GitHub (Jun 30, 2022): @lunny why cant your workaround be a PR?

GiteaMirror commented

@wxiaoguang commented on GitHub (Jul 4, 2022):

FYI, there is a new feature: Go to file

https://github.com/go-gitea/gitea/pull/15028

It should be more convenient if you know you are searching for file names:

@wxiaoguang commented on GitHub (Jul 4, 2022): FYI, there is a new feature: Go to file * https://github.com/go-gitea/gitea/pull/15028 It should be more convenient if you know you are searching for file names: <details> ![image](https://user-images.githubusercontent.com/2114189/177109015-9d32527a-6df3-46e3-af06-57d81bcf10e3.png) ![image](https://user-images.githubusercontent.com/2114189/177109055-3f0474c5-8de7-49c7-9048-d3dea33c6366.png) </details>

GiteaMirror commented

@delanym commented on GitHub (Jul 4, 2022):

@wxiaoguang thanks this does the expected.
It's a little slow on large repos the each time I bring up the page to do a search. What configuration options are there?

@delanym commented on GitHub (Jul 4, 2022): @wxiaoguang thanks this does the expected. It's a little slow on large repos the each time I bring up the page to do a search. What configuration options are there?

GiteaMirror commented

@wxiaoguang commented on GitHub (Jul 4, 2022):

It's a little slow on large repos the each time I bring up the page to do a search.

Yup, it's not optimized yet. How large is your repo? For linux kernel repo (4G, 80k files) is about ten seconds on my side.

What configuration options are there?

What's the configuration do you mean? This feature doesn't have a config option yet, it's in 1.17 release.

@wxiaoguang commented on GitHub (Jul 4, 2022): > It's a little slow on large repos the each time I bring up the page to do a search. Yup, it's not optimized yet. How large is your repo? For linux kernel repo (4G, 80k files) is about ten seconds on my side. > What configuration options are there? What's the `configuration` do you mean? This feature doesn't have a config option yet, it's in 1.17 release.

GiteaMirror commented

@delanym commented on GitHub (Jul 4, 2022):

@wxiaoguang config to enable a cache. I can cope with 10sec the first time the repo is indexed, but not every time I search for a file.
Im testing on a repo/branch with 86000 files - it's more like 15sec.
The equivalent find takes 0.18s: find . -iname "*IdleState*"

Also it should treat the search query as a sequence of strings, not a sequence of chars. Currently a search for "IdleState" returns a file like
IssuingSystem/Modules/UserControls/Views/ShortcutBarPresenter.cs

And actually - if I know anything about grep - its probably slower because its searching those individual characters.

@delanym commented on GitHub (Jul 4, 2022): @wxiaoguang config to enable a cache. I can cope with 10sec the first time the repo is indexed, but not every time I search for a file. Im testing on a repo/branch with 86000 files - it's more like 15sec. The equivalent find takes 0.18s: `find . -iname "*IdleState*"` Also it should treat the search query as a sequence of strings, not a sequence of chars. Currently a search for "IdleState" returns a file like **I**ssuingSystem/Mo**d**u**le**s/UserControls/Views/**S**hor**t**cutB**a**rPresen**te**r.cs And actually - if I know anything about grep - its probably slower because its searching those individual characters.

GiteaMirror commented

@oetiker commented on GitHub (Jul 4, 2022):

In our setup, we would love to be able to search for a filename across all repos ...

@oetiker commented on GitHub (Jul 4, 2022): In our setup, we would love to be able to search for a filename across all repos ...

GiteaMirror commented

@wxiaoguang commented on GitHub (Jul 4, 2022):

@delanym Since the Go to file feature is not the same as this issue, I opened a new issue for its discussion and optimization.

#20231

@wxiaoguang commented on GitHub (Jul 4, 2022): @delanym Since the `Go to file` feature is not the same as this issue, I opened a new issue for its discussion and optimization. * #20231

GiteaMirror commented

@michaelfresco commented on GitHub (Jan 28, 2023):

@delanym
I was looking at the code from Lunny but, this function looks a bit different at the moment. Does anyone know how to implement the fix right now?
e81ccc406b/modules/indexer/code/bleve.go (L183)

@michaelfresco commented on GitHub (Jan 28, 2023): @delanym I was looking at the code from Lunny but, this function looks a bit different at the moment. Does anyone know how to implement the fix right now? https://github.com/go-gitea/gitea/blob/e81ccc406bf723a5a58d685e7782f281736affd4/modules/indexer/code/bleve.go#L183

GiteaMirror commented

@delvh commented on GitHub (Apr 29, 2023):

This has already been implemented for some time (I think since 1.18?):

with the following dialogue

@delvh commented on GitHub (Apr 29, 2023): This has already been implemented for some time (I think since 1.18?): ![image](https://user-images.githubusercontent.com/51889757/235327310-38f1be4b-515f-4a02-b6de-7b8f1b0614d4.png) with the following dialogue ![image](https://user-images.githubusercontent.com/51889757/235327337-aa551adc-e099-477a-8e1c-dd48e96fbc3a.png)

GiteaMirror commented

@wxiaoguang commented on GitHub (Apr 30, 2023):

Hmm, IIRC this issue is asking about "repo indexer" , the "goto file" could help in some cases but not 100% resolves the issue.

@wxiaoguang commented on GitHub (Apr 30, 2023): Hmm, IIRC this issue is asking about "repo indexer" , the "goto file" could help in some cases but not 100% resolves the issue.

GiteaMirror commented