[GH-ISSUE #5588] [Docs] llms.txt is incorrect format, unusable by Claude Code and many other agentic coding tools #27614

Closed
opened 2026-04-17 18:42:54 -05:00 by GiteaMirror · 5 comments
Owner

Originally created by @pattobrien on GitHub (Oct 26, 2025).
Original GitHub issue: https://github.com/better-auth/better-auth/issues/5588

Is this suited for github?

  • Yes, this is suited for github

To Reproduce

Navigate to better-auth.com/llms.txt

Current vs. Expected behavior

According to tokenizer, the llm.txt file for better-auth.com is ~230k tokens, rendering it completely unusable by agentic coding tools like Claude Code.

In many implementations, the llms.txt file is meant to be similar to a sitemap, with links to documentation pages within the main site.

For a good reference, see the oRPC docs:

And each markdown link points to another markdown file, for example:

What version of Better Auth are you using?

0.0.0

System info

n/a

Which area(s) are affected? (Select all that apply)

Documentation

Auth config (if applicable)

import { betterAuth } from "better-auth"
export const auth = betterAuth({
  emailAndPassword: {  
    enabled: true
  },
});

Additional context

No response

Originally created by @pattobrien on GitHub (Oct 26, 2025). Original GitHub issue: https://github.com/better-auth/better-auth/issues/5588 ### Is this suited for github? - [x] Yes, this is suited for github ### To Reproduce Navigate to `better-auth.com/llms.txt` ### Current vs. Expected behavior According to [tokenizer](https://platform.openai.com/tokenizer), the llm.txt file for better-auth.com is ~230k tokens, rendering it completely unusable by agentic coding tools like Claude Code. In many implementations, the `llms.txt` file is meant to be similar to a sitemap, with links to documentation pages within the main site. For a good reference, see the `oRPC` docs: - https://orpc.unnoq.com/llms.txt And each markdown link points to another markdown file, for example: - https://orpc.unnoq.com/docs/adapters/http.md (which is a markdown version of https://orpc.unnoq.com/docs/adapters/http) ### What version of Better Auth are you using? 0.0.0 ### System info ```bash n/a ``` ### Which area(s) are affected? (Select all that apply) Documentation ### Auth config (if applicable) ```typescript import { betterAuth } from "better-auth" export const auth = betterAuth({ emailAndPassword: { enabled: true }, }); ``` ### Additional context _No response_
GiteaMirror added the locked label 2026-04-17 18:42:54 -05:00
Author
Owner

@dosubot[bot] commented on GitHub (Oct 26, 2025):

Hi @pattobrien! I'm Dosu and I’m helping the better-auth team.

The current llms.txt file on better-auth.com is very large (~230k tokens) because it includes the full processed markdown content of all documentation pages, not just links. This makes it unusable for agentic coding tools like Claude Code, which expect a lightweight, sitemap-style file with markdown links to documentation pages—similar to what oRPC provides.

To resolve this, llms.txt should be updated to only include a list of markdown links to documentation pages, rather than the full content. This will dramatically reduce the file size and make it compatible with agentic coding tools. The current approach is a result of how the export logic is implemented, and changing it to output only links (not full docs) will align with industry best practices and user expectations.

For reference, see the oRPC llms.txt format:
https://orpc.unnoq.com/llms.txt

Recent PRs have improved markdown delivery for AI tools, but the file content itself still needs to be trimmed down to just links for optimal compatibility and usability source source.

To reply, just mention @dosu.


How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X

<!-- gh-comment-id:3448246072 --> @dosubot[bot] commented on GitHub (Oct 26, 2025): <!-- Greeting --> Hi @pattobrien! I'm [Dosu](https://go.dosu.dev/dosubot) and I’m helping the better-auth team. <!-- Answer --> The current llms.txt file on better-auth.com is very large (~230k tokens) because it includes the full processed markdown content of all documentation pages, not just links. This makes it unusable for agentic coding tools like Claude Code, which expect a lightweight, sitemap-style file with markdown links to documentation pages—similar to what oRPC provides. To resolve this, llms.txt should be updated to only include a list of markdown links to documentation pages, rather than the full content. This will dramatically reduce the file size and make it compatible with agentic coding tools. The current approach is a result of how the export logic is implemented, and changing it to output only links (not full docs) will align with industry best practices and user expectations. For reference, see the oRPC llms.txt format: https://orpc.unnoq.com/llms.txt Recent PRs have improved markdown delivery for AI tools, but the file content itself still needs to be trimmed down to just links for optimal compatibility and usability [source](https://github.com/better-auth/better-auth/pull/3423) [source](https://github.com/better-auth/better-auth/pull/5105). <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/472ea025-9e82-4ceb-ba0a-fff3fa8604f8?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/472ea025-9e82-4ceb-ba0a-fff3fa8604f8?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/472ea025-9e82-4ceb-ba0a-fff3fa8604f8?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/472ea025-9e82-4ceb-ba0a-fff3fa8604f8?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/472ea025-9e82-4ceb-ba0a-fff3fa8604f8?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/472ea025-9e82-4ceb-ba0a-fff3fa8604f8?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/472ea025-9e82-4ceb-ba0a-fff3fa8604f8?feedback_type=other)</sup>&nbsp;&nbsp;[![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/cdda13d9-dd27-4d31-b09a-5d8bec92de21/ask?utm_source=github)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/better-auth/better-auth/issues/5588)
Author
Owner

@pattobrien commented on GitHub (Oct 28, 2025):

@bytaesu thanks for the PR! but your PR has several bugs, and did not close this issue.

  • all of the relative links start with "llms.txt" (e.g. - [API](/llms.txt/docs/concepts/api.md): Better Auth API.); this is not a valid HTTP URI
  • none of the generated links seem to exist, e.g API or API without "llms.txt"

CC @himself65 , I believe this should be reopened, but can also open a new issue.

<!-- gh-comment-id:3453963608 --> @pattobrien commented on GitHub (Oct 28, 2025): @bytaesu thanks for the PR! but your PR has **_several_** bugs, and did not close this issue. - all of the relative links start with "llms.txt" (e.g. `- [API](/llms.txt/docs/concepts/api.md): Better Auth API.`); this is not a valid HTTP URI - none of the generated links seem to exist, e.g [API](https://better-auth.com/llms.txt/docs/concepts/api.md) or [API without "llms.txt"](https://better-auth.com/docs/concepts/api.md) CC @himself65 , I believe this should be reopened, but can also open a new issue.
Author
Owner

@himself65 commented on GitHub (Oct 28, 2025):

all of the relative links start with "llms.txt" (e.g. - API: Better Auth API.); this is not a valid HTTP URI

I just fixed the 500 issue. But the path is not correct. I will leave this issue open. Please fix it

/cc @bytaesu thx

<!-- gh-comment-id:3454916436 --> @himself65 commented on GitHub (Oct 28, 2025): > all of the relative links start with "llms.txt" (e.g. - [API](/llms.txt/docs/concepts/api.md): Better Auth API.); this is not a valid HTTP URI I just fixed the 500 issue. But the path is not correct. I will leave this issue open. Please fix it /cc @bytaesu thx
Author
Owner

@bytaesu commented on GitHub (Oct 28, 2025):

Hello @pattobrien,

It turned out to be a Vercel Edge Functions runtime issue!

I’m aware of the previous problem with llms.txt, so feel free to open an issue anytime if you run into anything else.

I’ll also check if there are any other related issues and get them fixed 🙂

<!-- gh-comment-id:3455225469 --> @bytaesu commented on GitHub (Oct 28, 2025): Hello @pattobrien, It turned out to be a Vercel Edge Functions runtime issue! I’m aware of the previous problem with llms.txt, so feel free to open an issue anytime if you run into anything else. I’ll also check if there are any other related issues and get them fixed 🙂
Author
Owner

@pattobrien commented on GitHub (Oct 28, 2025):

Thanks a lot @bytaesu ! There are still problems with the generated pages, but I put it in a new ticket:

https://github.com/better-auth/better-auth/issues/5640

CC @himself65

<!-- gh-comment-id:3456196722 --> @pattobrien commented on GitHub (Oct 28, 2025): Thanks a lot @bytaesu ! There are still problems with the generated pages, but I put it in a new ticket: https://github.com/better-auth/better-auth/issues/5640 CC @himself65
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/better-auth#27614