github-starred/gitea
2
0
Fork 0
mirror of https://github.com/go-gitea/gitea.git synced 2026-03-12 10:39:38 -05:00
Code Issues 2.6k Packages Projects Releases 100 Wiki Activity

gitea database dump doesn't escape LF character #3316

New Issue
Closed
opened 2025-11-02 05:07:54 -06:00 by GiteaMirror · 6 comments
GiteaMirror commented 2025-11-02 05:07:54 -06:00
Owner
Copy Link

Originally created by @samangh on GitHub (May 11, 2019).

  • Gitea version (or commit ref): 1.8.0
  • Git version: 2.21.0
  • Operating system: Linux (Debian Buster)
  • Database (use [x]):
    • MySQL
    • SQLite
  • Can you reproduce the bug at https://try.gitea.io:
    • Not relevant
  • Log gist:

When exporting via gitea dump,the \n at then end of commit message title (in column content of table action) is not properly escaped.

I've checked that this happens when exporting both from MySQL and NoSQL databases. As an example,

$ ./gitea dump
$ unzip -j gitea-dump-1557579865.zip gitea-db.sql `pwd`/gitea-db.sql
$ grep "66fd8c3d224c1f358f57d242374563ad55543761" gitea-db.sql

INSERT INTO `action` (`id`, `user_id`, `op_type`, `act_user_id`, `repo_id`, `comment_id`, `is_deleted`, `ref_name`, `is_private`, `content`, `created_unix`) VALUES (894, 1, 5, 1, 1, 0, false, 'master', true, '{"Len":1,"Commits":[{"Sha1":"66fd8c3d224c1f358f57d242374563ad55543761","Message":"test \\n \\r \\t\n","AuthorEmail":"saman@saman-gh.co.uk","AuthorName":"Saman Ghannadzadeh","CommitterEmail":"saman@saman-gh.co.uk","CommitterName":"Saman Ghannadzadeh","Timestamp":"2019-05-11T13:31:40+01:00"}],"CompareURL":"saman/dotfiles/compare/5f81ccf3c05313658d4a9edd817526046e6a8cef...66fd8c3d224c1f358f57d242374563ad55543761"}', 1557577915);

Notice the Message":"test \\n \\r \\t\n". The special characters are escpaed, except the last \n.

This can cause mashalling errors when the dump is later imported into a database, see #6793.

Originally created by @samangh on GitHub (May 11, 2019). - Gitea version (or commit ref): 1.8.0 - Git version: 2.21.0 - Operating system: Linux (Debian Buster) - Database (use `[x]`): - [x] MySQL - [x] SQLite - Can you reproduce the bug at https://try.gitea.io: - [x] Not relevant - Log gist: When exporting via `gitea dump`,the `\n` at then end of commit message title (in column `content` of table `action`) is not properly escaped. I've checked that this happens when exporting both from MySQL and NoSQL databases. As an example, ``` $ ./gitea dump $ unzip -j gitea-dump-1557579865.zip gitea-db.sql `pwd`/gitea-db.sql $ grep "66fd8c3d224c1f358f57d242374563ad55543761" gitea-db.sql INSERT INTO `action` (`id`, `user_id`, `op_type`, `act_user_id`, `repo_id`, `comment_id`, `is_deleted`, `ref_name`, `is_private`, `content`, `created_unix`) VALUES (894, 1, 5, 1, 1, 0, false, 'master', true, '{"Len":1,"Commits":[{"Sha1":"66fd8c3d224c1f358f57d242374563ad55543761","Message":"test \\n \\r \\t\n","AuthorEmail":"saman@saman-gh.co.uk","AuthorName":"Saman Ghannadzadeh","CommitterEmail":"saman@saman-gh.co.uk","CommitterName":"Saman Ghannadzadeh","Timestamp":"2019-05-11T13:31:40+01:00"}],"CompareURL":"saman/dotfiles/compare/5f81ccf3c05313658d4a9edd817526046e6a8cef...66fd8c3d224c1f358f57d242374563ad55543761"}', 1557577915); ``` Notice the `Message":"test \\n \\r \\t\n"`. The special characters are escpaed, except the last `\n`. This can cause mashalling errors when the dump is later imported into a database, see #6793.
GiteaMirror added the issue/confirmedtype/bug labels 2025-11-02 05:07:54 -06:00
GiteaMirror commented 2025-11-02 05:07:55 -06:00
Author
Owner
Copy Link

@stale[bot] commented on GitHub (Jul 11, 2019):

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 weeks. Thank you for your contributions.

@stale[bot] commented on GitHub (Jul 11, 2019): This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs during the next 2 weeks. Thank you for your contributions.
GiteaMirror commented 2025-11-02 05:07:55 -06:00
Author
Owner
Copy Link

@PerfectSlayer commented on GitHub (Aug 23, 2019):

Hello,

I encounter the same issue with 1.8.2 version after dumping (gitea dump) and re-importing data (mysql -uUSER -pPASSWD DATABASE < gitea-db.sql).

I think I will try to write a script to patch my backup to convert \n to \\n in Commits[].Message from content field for op_type = 5||16 in action database. Any device before implementing this?

Thanks!

@PerfectSlayer commented on GitHub (Aug 23, 2019): Hello, I encounter the same issue with `1.8.2` version after dumping (`gitea dump`) and re-importing data (`mysql -uUSER -pPASSWD DATABASE < gitea-db.sql`). I think I will try to write a script to patch my backup to convert `\n` to `\\n` in `Commits[].Message` from `content` field for `op_type = 5||16` in `action` database. Any device before implementing this? Thanks!
GiteaMirror commented 2025-11-02 05:07:55 -06:00
Author
Owner
Copy Link

@nodiscc commented on GitHub (Feb 10, 2020):

I also encounter this problem when trying to restore a backup from Gitea 1.8.0 on a recent release. This workaround worked for me:

sed -i 's/\\n/\\\\n/g' gitea-dump-1581252886/gitea-db.sql
sed -i 's/\\r/\\\\e/g' gitea-dump-1581252886/gitea-db.sql

However considering the numerous issues I noted at https://github.com/go-gitea/gitea/issues/9100#issuecomment-583864450 you may want to remove the dump command altogether from gitea, and simply provide backup instructions (dump the database using standard tools, backup the dump and data directory)

@nodiscc commented on GitHub (Feb 10, 2020): I also encounter this problem when trying to restore a backup from Gitea 1.8.0 on a recent release. This workaround worked for me: ``` sed -i 's/\\n/\\\\n/g' gitea-dump-1581252886/gitea-db.sql sed -i 's/\\r/\\\\e/g' gitea-dump-1581252886/gitea-db.sql ``` However considering the numerous issues I noted at https://github.com/go-gitea/gitea/issues/9100#issuecomment-583864450 you may want to remove the dump command altogether from gitea, and simply provide backup instructions (dump the database using standard tools, backup the dump and data directory)
GiteaMirror commented 2025-11-02 05:07:55 -06:00
Author
Owner
Copy Link

@hefengxian commented on GitHub (Jul 26, 2021):

For temp solution, we can use database dump tool, for example on MySQL

mysqldump -h192.168.1.xxx -uxxx -pxxx -P3306 --single-transaction --set-gtid-purged=OFF --column-statistics=0 --default-character-set=utf8mb4 --databases gitea > gitea.sql

then import to MySQL

mysql --default-character-set=utf8mb4 -uxxx -pxxx gitea < gitea.sql
@hefengxian commented on GitHub (Jul 26, 2021): For temp solution, we can use database dump tool, for example on MySQL ```bash mysqldump -h192.168.1.xxx -uxxx -pxxx -P3306 --single-transaction --set-gtid-purged=OFF --column-statistics=0 --default-character-set=utf8mb4 --databases gitea > gitea.sql ``` then import to MySQL ```bash mysql --default-character-set=utf8mb4 -uxxx -pxxx gitea < gitea.sql ```
GiteaMirror commented 2025-11-02 05:07:56 -06:00
Author
Owner
Copy Link

@nodiscc commented on GitHub (Jul 26, 2021):

I also use standard filesystem/database tools to manage gitea backups. It has been more reliable so far:

@nodiscc commented on GitHub (Jul 26, 2021): I also use standard filesystem/database tools to manage gitea backups. It has been more reliable so far: - https://github.com/nodiscc/xsrv/blob/master/roles/gitea/templates/etc_rsnapshot.d_gitea.conf.j2 - https://github.com/nodiscc/xsrv/blob/master/roles/postgresql/templates/etc_rsnapshot.d_postgresql.conf.j2 - https://github.com/nodiscc/xsrv/blob/master/roles/postgresql/templates/usr_local_bin_postgres-dump-all-databases.sh.j2
GiteaMirror commented 2025-11-02 05:07:56 -06:00
Author
Owner
Copy Link

@zeripath commented on GitHub (Jul 26, 2021):

It would be helpful to report which versions of Gitea have this problem and if they are fixed on the newer versions. (including 1.14 - where it should be fixed)

@zeripath commented on GitHub (Jul 26, 2021): It would be helpful to report which versions of Gitea have this problem and if they are fixed on the newer versions. (including 1.14 - where it should be fixed)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
$20
$250
$50
$500
backport/done
💎 Bounty
docs-update-needed
good first issue
hacktoberfest
issue/bounty
issue/confirmed
issue/critical
issue/duplicate
issue/needs-feedback
issue/not-a-bug
issue/regression
issue/stale
issue/workaround
lgtm/need 2
modifies/api
modifies/translation
outdated/backport/v1.18
outdated/theme/markdown
outdated/theme/timetracker
performance/bigrepo
performance/cpu
performance/memory
performance/speed
pr/breaking
proposal/accepted
proposal/rejected
pr/wip
pull-request
Mirrored from GitHub Pull Request
 reviewed/wontfix
💰 Rewarded
skip-changelog
status/blocked
topic/accessibility
topic/api
topic/authentication
topic/build
topic/code-linting
topic/commit-signing
topic/content-rendering
topic/deployment
topic/distribution
topic/federation
topic/gitea-actions
topic/issues
topic/lfs
topic/mobile
topic/moderation
topic/packages
topic/pr
topic/projects
topic/repo
topic/repo-migration
topic/security
topic/theme
topic/ui
topic/ui-interaction
topic/ux
topic/webhooks
topic/wiki
type/bug
type/deprecation
type/docs
type/enhancement
type/feature
type/miscellaneous
type/proposal
type/question
type/refactoring
type/summary
type/testing
type/upstream
No Label issue/confirmed type/bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/gitea#3316
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?