- Backport of https://codeberg.org/forgejo/forgejo/pulls/1284
- Databases are one of the most important parts of Forgejo, every
interaction with Forgejo uses the database in one way or another.
Therefore, it is important to maintain the database and recognize when
Forgejo is not doing well with the database. Forgejo already has the
option to log *every* SQL query along with its execution time, but
monitoring becomes impractical for larger instances and takes up
unnecessary storage in the logs.
- Add a QoL enhancement that allows instance administrators to specify a
threshold value beyond which query execution time is logged as a warning
in the xorm logger. The default value is a conservative five seconds to
avoid this becoming a source of spam in the logs.
- The use case for this patch is that with an instance the size of Codeberg, monitoring SQL logs is not very fruitful and most of them are uninteresting. Recently, in the context of persistent deadlock issues (https://codeberg.org/forgejo/forgejo/issues/220), I have noticed that certain queries hold locks on tables like comment and issue for several seconds. This patch helps to identify which queries these are and when they happen.
- Added unit test.
Backport #26494 by @wxiaoguang
"ogg" is just a "container" format for audio and video.
Golang's `DetectContentType` only reports "application/ogg" for
potential ogg files.
Actually it could do more "guess" to see whether it is a audio file or a
video file.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 4bdb8dd9cc)
Backport #26441 by @lunny
This PR rewrites the function `getStorage` and make it more clear.
Include tests from #26435, thanks @earl-warren
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Earl Warren <contact@earl-warren.org>
(cherry picked from commit f1c5d33d3e)
Backport #26470 by @wxiaoguang
Close stdout correctly for "git blame", otherwise the failed "git blame"
would cause the request hanging forever.
And "os.Stderr" should never (seldom) be used as git command's stderr
(there seems some similar problems in code, they could be fixed later).
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit fe1b11b639)
Backport #26468 by @wxiaoguang
When users put the secrets into a file (GITEA__sec__KEY__FILE), the
newline sometimes is different to avoid (eg: echo/vim/...)
So the last newline could be removed when reading, it makes the users
easier to maintain the secret files.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 80d7288ea4)
Backport #26420 by @lunny
For some reason, the permission of the client_id and secret may cannot
create bucket, so now we will check whether bucket does exist first and
then try to create a bucket if it doesn't exist.
Try to fix#25984
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
(cherry picked from commit 2d1202b32c)
Backport #26412 by @nekrondev
The MinIO client isn't redirecting to the correct AWS endpoint if a
non-default data center is used.
In my use case I created an AWS bucket at `eu-central-1` region. Because
of the missing region initialization of the client the default
`us-east-1` API endpoint is used returning a `301 Moved Permanently`
response that's not handled properly by MinIO client. This in return
aborts using S3 storage on AWS as the `BucketExists()` call will fail
with the http moved error.
MinIO client trace shows the issue:
```text
---------START-HTTP---------
HEAD / HTTP/1.1
Host: xxxxxxxxxxx-prod-gitea-data.s3.dualstack.us-east-1.amazonaws.com
User-Agent: MinIO (windows; amd64) minio-go/v7.0.61
Authorization: AWS4-HMAC-SHA256 Credential=**REDACTED**/20230809/accesspoint.eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230809T141143Z
HTTP/1.1 301 Moved Permanently
Connection: close
Content-Type: application/xml
Date: Wed, 09 Aug 2023 14:11:43 GMT
Server: AmazonS3
X-Amz-Bucket-Region: eu-central-1
X-Amz-Id-2: UK7wfeYi0HcTcytNvQ3wTAZ5ZP1mOSMnvRZ9Fz4xXzeNsS47NB/KfFx2unFxo3L7XckHpMNPPVo=
X-Amz-Request-Id: S1V2MJV8SZ11GEVN
---------END-HTTP---------
```
Co-authored-by: nekrondev <heiko@noordsee.de>
Co-authored-by: Heiko Besemann <heiko.besemann@qbeyond.de>
(cherry picked from commit 981ab48503)
Backport #26392 by @wxiaoguang
Fix#26389
And complete an old TODO: `ctx.Params does un-escaping,..., which is
incorrect.`
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 2d1a7e1cd4)
Backport #26325 by @wxiaoguang
Fix#26064
Some git commands should use parent context, otherwise it would exit too
early (by the default timeout, 10m), and the "cmd.Wait" waits till the
pipes are closed.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 9451781ebe)
Backport #26271 by @lunny
This PR will fix#26264, caused by #23911.
The package configuration derive is totally wrong when storage type is
local in that PR.
This PR fixed the inherit logic when storage type is local with some
unit tests.
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 88f6f7579c)
Backport #26290 by @Zettat123
Fixes#26270.
Co-Author: @wxiaoguang
Thanks @lunny for providing this solution
As
https://github.com/go-gitea/gitea/issues/26270#issuecomment-1661695151
said, at present we cannot get the names of changed files correctly when
the `OldCommitID` is `EmptySHA`. In this PR, the `GetCommitFilesChanged`
method is added and will be used to get the changed files by commit ID.
References:
- https://stackoverflow.com/a/424142
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit a57568bad7)
Backport #26267 by @wxiaoguang
1. Fix the wrong document (add the missing `MODE=`)
2. Add a more friendly log message to tell users to add `MODE=` in their
config
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit a758337046)
Backport #26107 by @wxiaoguang
Just like others (oauth2 secret, internal token, etc), do not generate
if no install lock
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit e2596b0a99)
(cherry picked from commit 78722734fe)
- Backport of #1224
- In org mode you can specify an description for media via the following
syntax `[[description][media link]]`. The description is then used as
title or alt.
- This patch fixes the rendering of the description by seperating the
description and non-description cases and using `org.String()`.
- Added unit tests.
- Inspired by 6eb20dbda9/org/html_writer.go (L406-L427)
- Resolves https://codeberg.org/Codeberg/Community/issues/848
This reverts commit 78722734fe.
It does not create `LFS_JWT_SECRET` if `INSTALL_LOCK` is true and the
value of `LFS_JWT_SECRET` found in `app.ini` is incorrect. As a result
LFS_JWT_SECRET will not be set at all and the Forgejo admin will not
be notified that the value in the `app.ini` was ignored.
Backport #26137 by @wxiaoguang
A low-level mistake:
* `log.Info` is global `Info` function, which calls "default" logger
* `logger.Info` is the for router's logger
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 782b137682)
Backport #26107 by @wxiaoguang
Just like others (oauth2 secret, internal token, etc), do not generate
if no install lock
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit e2596b0a99)
Backport #26087 by @KN4CK3R
Fixes#26022
- Fix handling of files with trailing slash
- Fix handling of duplicate package file errors
- Added test for both
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
(cherry picked from commit a424f6d4f8)
Backport #26074 by @wxiaoguang
If the log file can't be opened, what should it do? panic/exit? ignore
logs? fallback to stderr?
It seems that "fallback to stderr" is slightly better than others ....
(cherry picked from commit 8d9193680d)
Backport #26001 by @wxiaoguang
Regression of #24832
Fix the bug and add a test for it
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 4d5e3b9372)
Backport #25417 by @wxiaoguang
Update WorkPath/WORK_PATH related documents, remove out-dated
information.
Remove "StaticRootPath" on the admin config display page, because few
end user really need it, it only causes misconfiguration.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit ee47face12)
Backport #25944
The "creating dir/file during load config" is a longstanding and complex
problem.
This PR only does a quick patch, it still needs more refactorings in the
future.
Fix#25938
(cherry picked from commit 5c3662b902)
Regression: https://github.com/go-gitea/gitea/pull/24805Closes: #25945
- Disallow `javascript`, `vbscript` and `data` (data uri images still
work) url schemes even if all other schemes are allowed
- Fixed older `cbthunderlink` tests
---------
Co-authored-by: delvh <dev.lh@web.de>
(cherry picked from commit ab54310731)
Backport #25919 by @wxiaoguang
Before, in dev mode, there might be some error logs like:
```
2023/07/17 13:54:51 ...s/assetfs/layered.go:221:WatchLocalChanges() [E] Unable to watch directory .: lstat /data/work/gitea/custom/templates: no such file or directory
```
Because there is no "custom/templates" directory.
After: ignore such error, no such error message anymore.
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit 9369b38315)
It breaks existing instances that would otherwise work perfectly
fine. Failing to start an instance should only happen when there is a
compelling reason to do so, for instance if the `app.ini` could not be
modified in a way that is backward compatible. If the only motivation
is to remove the setting for cosmetic reason, it must not be fatal.
(cherry picked from commit c09ef82e9a)
(cherry picked from commit e80aaa11d9)
Backport #25920 by @KN4CK3R
Got the same problem as #25915 when updating an instance. The
`log.Fatal` should have been marked as breaking in #23911.
This PR adds a notice that the system is shutting down because of the
deprecated setting.
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
(cherry picked from commit 6e82d0bb7c)
It breaks existing instances that would otherwise work perfectly
fine. Failing to start an instance should only happen when there is a
compelling reason to do so, for instance if the `app.ini` could not be
modified in a way that is backward compatible. If the only motivation
is to remove the setting for cosmetic reason, it must not be fatal.
(cherry picked from commit c09ef82e9a)
- The library that's being used for org-mode, [doesn't render the status
of list items](https://github.com/niklasfasching/go-org/issues/63).
- Add a modified version of the proposed CSS snippet to still display
the status for the list items. The alternative was parsing HTML and
transforming it, which is too complicated for this small task.
- Resolves https://codeberg.org/Codeberg/Community/issues/1099
- Follow up for: #540, #802
- Add API routes for user blocking from user and organization
perspective.
- The new routes have integration testing.
- The new model functions have unit tests.
- Actually quite boring to write and to read this pull request.
(cherry picked from commit f3afaf15c7)
(cherry picked from commit 6d754db3e5)
(cherry picked from commit d0fc8bc9d3)
(cherry picked from commit 9a53b0d1a0)
(cherry picked from commit 44a2a4fd48)
(cherry picked from commit 182025db9c)
(cherry picked from commit 558a35963e)
FORGEJO_* environment variables are set to the corresponding GITEA_*
variable when the cli starts. This approach is intended to minimize
the conflicts on rebase. All occurences of GITEA_* are left untouched
in the codebase and they are only changed to FORGEJO_* if exposed to
the user.
(cherry picked from commit e466f9d10e)
(cherry picked from commit e33e95931b)
(cherry picked from commit 2cfc6519b7)
(cherry picked from commit af8864373a)
(cherry picked from commit a0550ff339)
(cherry picked from commit 24dc0a5191)
(cherry picked from commit e255eea2b4)
(cherry picked from commit 0c4f5afa7a)
(cherry picked from commit 42fce708d0)
(cherry picked from commit e7278c3c22)
(cherry picked from commit 0fb9ed7e0e)
(cherry picked from commit a98308aa4d)
(cherry picked from commit b8695fcbe0)
(cherry picked from commit 4aee8719f5)
(cherry picked from commit 1c503c1ba7)
(cherry picked from commit cf1ed8551e)
(cherry picked from commit c52459b088)
(cherry picked from commit 92cac277b2)
(cherry picked from commit 2c744f1118)
(cherry picked from commit 1482cfabe5)
