0
0
Fork 0
mirror of https://github.com/go-gitea/gitea synced 2024-11-09 11:31:34 +01:00
Commit graph

4353 commits

Author SHA1 Message Date
Lunny Xiao
f064d716c3
Move feed notification service layer (#26908)
Extract from #22266
2023-09-05 13:00:52 +00:00
Lunny Xiao
6c73c0da53
Move ui notification to service layer (#26907)
Extract from #22266
2023-09-05 11:15:42 +00:00
Lunny Xiao
aa678b8098
Move indexer notification to service layer (#26906)
Extract from #22266
2023-09-05 10:07:57 +00:00
Lunny Xiao
a66d883a18
Move mail notification logic to service layer (#26905)
Extract from #22266
2023-09-05 17:26:59 +08:00
wxiaoguang
fc039167d2
Use Go 1.21 and update dependencies (#26878)
To make sure Gitea's next release's lifecycle could have active Golang
support.

And min/max are builtin now.
2023-09-03 10:34:57 +00:00
wxiaoguang
c802c46a9b
Refactor og:description to limit the max length (#26876)
1. The `og:description` should be "a one to two sentence description of
your object"
* It shouldn't output all the user inputted content -- it would be
pretty huge.
    * Maybe it only needs at most 300 bytes.
2. Do not render commit message as HTML
2023-09-03 01:43:29 +00:00
Earl Warren
4ab8e56c91
restrict certificate type for builtin SSH server (#26789)
- While doing some sanity checks over OpenSSH's code for how they handle
certificates authentication. I stumbled on an condition that checks the
certificate type is really an user certificate on the server-side
authentication. This checks seems to be a formality and just for the
sake of good domain seperation, because an user and host certificate
don't differ in their generation, verification or flags that can be
included.
- Add this check to the builtin SSH server to stay close to the
unwritten SSH specification.
- This is an breaking change for setups where the builtin SSH server is
being used and for some reason host certificates were being used for
authentication.
- 
(cherry picked from commit de35b141b79a3d6efe2127ed2c73fd481515e481)

Refs: https://codeberg.org/forgejo/forgejo/pulls/1172

## ⚠️ BREAKING ⚠️

Like OpenSSH, the built-in SSH server will now only accept SSH user
certificates, not server certificates.

Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-09-01 13:45:22 +00:00
wxiaoguang
f01bed2443
Avoid double-unescaping of form value (#26853)
1. The old `prepareQueryArg` did double-unescaping of form value.
2. By the way, remove the unnecessary `ctx.Flash = ...` in
`MockContext`.

Co-authored-by: Giteabot <teabot@gitea.io>
2023-09-01 12:01:36 +00:00
wxiaoguang
e8aae43f56
Move web/api context related testing function into a separate package (#26859)
Just like `models/unittest`, the testing helper functions should be in a
separate package: `contexttest`

And complete the TODO:

> // TODO: move this function to other packages, because it depends on
"models" package
2023-09-01 11:26:07 +00:00
silverwind
3d109861dd
Render code blocks in repo description (#26830)
Backtick syntax now works in repo description too. Also, I replaced the
CSS for this was a new single class, making it more flexible and not
dependent on a parent. Also, very slightly reduced font size from 16.8px
to 16px.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-08-31 05:01:01 +00:00
techknowlogick
45976a1bde
Check blocklist for emails when adding them to account (#26812) 2023-08-30 10:46:49 -05:00
KN4CK3R
5315153059
Use Set[Type] instead of map[Type]bool/struct{}. (#26804) 2023-08-30 06:55:25 +00:00
Chongyi Zheng
ad43486cd3
Fix some slice append usages (#26778)
Co-authored-by: delvh <dev.lh@web.de>
2023-08-29 15:47:26 +00:00
Eng Zer Jun
ad3cbbc3b1
Remove redundant nil check in WalkGitLog (#26773)
From the Go specification:

> "1. For a nil slice, the number of iterations is 0."
https://go.dev/ref/spec#For_range

Therefore, an additional nil check for before the loop is unnecessary.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-08-29 11:03:43 +00:00
yp05327
60ca3d04a4
Add fix incorrect can_create_org_repo for org owner team (#26683)
Related to: #8312 #26491

In migration v109, we only added a new column `CanCreateOrgRepo` in Team
table, but not initial the value of it.
This may cause bug like #26491.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-08-29 01:11:23 +00:00
Chongyi Zheng
ac2f8c9ac6
Reduce some allocations in type conversion (#26772) 2023-08-29 00:43:16 +08:00
Bo-Yi Wu
8cd46024fd
refactor(API): refactor secret creation and update functionality (#26751)
According to the GitHub API Spec:
https://docs.github.com/en/rest/actions/secrets?apiVersion=2022-11-28#create-or-update-an-organization-secret

Merge the Create and Update secret into a single API.

- Remove the `CreateSecretOption` struct and replace it with
`CreateOrUpdateSecretOption` in `modules/structs/secret.go`
- Update the `CreateOrUpdateOrgSecret` function in
`routers/api/v1/org/action.go` to use `CreateOrUpdateSecretOption`
instead of `UpdateSecretOption`
- Remove the `CreateOrgSecret` function in
`routers/api/v1/org/action.go` and replace it with
`CreateOrUpdateOrgSecret`
- Update the Swagger documentation in
`routers/api/v1/swagger/options.go` and `templates/swagger/v1_json.tmpl`
to reflect the changes in the struct names and function names

Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
2023-08-28 13:08:19 +08:00
Lunny Xiao
6945918d34
Fix bug for ctx usage (#26762)
Regression from #26158

Fix #26684
2023-08-28 04:06:47 +00:00
mainboarder
c533991519
Expanded minimum RSA Keylength to 3072 (#26604)
German Federal Office for Information Security requests in its technical
guideline BSI TR-02102-1 RSA Keylength not shorter than 3000bits
starting 2024, in the year 2023 3000bits as a recommendation. Gitea
should request longer RSA Keys by default in favor of security and drop
old clients which do not support longer keys.


https://www.bsi.bund.de/SharedDocs/Downloads/DE/BSI/Publikationen/TechnischeRichtlinien/TR02102/BSI-TR-02102.pdf?__blob=publicationFile&v=9
- Page 19, Table 1.2

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-08-28 00:53:16 +00:00
Lunny Xiao
476b9d1589
Use docs.gitea.com instead of docs.gitea.io (#26739) 2023-08-27 11:59:12 +00:00
Chongyi Zheng
43652746f2
Move modules/mirror to services (#26737)
To solve the cyclic imports in a better way

Closes #20261
2023-08-27 10:24:45 +08:00
wxiaoguang
412e5c0946
Make web context initialize correctly for different cases (#26726)
The web context (modules/context.Context) is quite complex, it's
difficult for the callers to initialize correctly.

This PR introduces a `NewWebContext` function, to make sure the web
context have the same behavior for different cases.
2023-08-25 19:07:42 +08:00
Earl Warren
a9ce570298
add Upload URL to release API (#26663)
- Resolves https://codeberg.org/forgejo/forgejo/issues/580
- Return a `upload_field` to any release API response, which points to
the API URL for uploading new assets.
- Adds unit test.
- Adds integration testing to verify URL is returned correctly and that
upload endpoint actually works

---------

Co-authored-by: Gusted <postmaster@gusted.xyz>
2023-08-24 10:36:10 +00:00
Laurent Cahour
b21b63c61a
Add merge files files to GetCommitFileStatus (#20515)
Hi,

We'd like to add merge files files to GetCommitFileStatus fucntions so
API returns the list of all the files associated to a merged pull
request commit, like GitHub API does.
The list of affectedFiles for an API commit is fetched from toCommit()
function in routers/api/v1/repo/commits.go, and API was returning no
file in case of a pull request with no conflict, or just files
associated to the confict resolution, but NOT the full list of merged
files.

This would lead to situations where a CI polling a repo for changes
could miss some file changes due to API returning an empty / partial
list in case of such merged pull requests. (Hope this makes sense :) )

NOTE: I'd like to add a unittest in
integrations/api_repo_git_commits_test.go but failed to understand how
to add my own test bare repo so I can make a test on a merged pull
request commit to check for affectedFiles.
Is there a merged pull request in there that I could use maybe?
Could someone please direct me to the relevant ressources with
informations on how to do that please?

Thanks for your time,
Laurent.

---------

Co-authored-by: Thomas Desveaux <desveaux.thomas@gmail.com>
2023-08-24 10:03:49 +00:00
Lunny Xiao
0d55f64e6c
chore(actions): support cron schedule task (#26655)
Replace #22751 

1. only support the default branch in the repository setting.
2. autoload schedule data from the schedule table after starting the
service.
3. support specific syntax like `@yearly`, `@monthly`, `@weekly`,
`@daily`, `@hourly`

## How to use

See the [GitHub Actions
document](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#schedule)
for getting more detailed information.

```yaml
on:
  schedule:
    - cron: '30 5 * * 1,3'
    - cron: '30 5 * * 2,4'

jobs:
  test_schedule:
    runs-on: ubuntu-latest
    steps:
      - name: Not on Monday or Wednesday
        if: github.event.schedule != '30 5 * * 1,3'
        run: echo "This step will be skipped on Monday and Wednesday"
      - name: Every time
        run: echo "This step will always run"
```

Signed-off-by: Bo-Yi.Wu <appleboy.tw@gmail.com>

---------


Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-08-24 03:06:51 +00:00
Bo-Yi Wu
b62c8e7765
feat(API): update and delete secret for managing organization secrets (#26660)
- Add `UpdateSecret` function to modify org or user repo secret
- Add `DeleteSecret` function to delete secret from an organization
- Add `UpdateSecretOption` struct for updating secret options
- Add `UpdateOrgSecret` function to update a secret in an organization
- Add `DeleteOrgSecret` function to delete a secret in an organization

GitHub API

1. Update Org Secret:
https://docs.github.com/en/rest/actions/secrets?apiVersion=2022-11-28#create-or-update-an-organization-secret
2. Delete Org Secret:
https://docs.github.com/en/rest/actions/secrets?apiVersion=2022-11-28#delete-an-organization-secret

---------

Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
2023-08-24 02:07:00 +00:00
Jason Song
5db21ce7e1
Fix counting and filtering on the dashboard page for issues (#26657)
This PR has multiple parts, and I didn't split them because
it's not easy to test them separately since they are all about the
dashboard page for issues.

1. Support counting issues via indexer to fix #26361
2. Fix repo selection so it also fixes #26653
3. Keep keywords in filter links.

The first two are regressions of #26012.

After:

https://github.com/go-gitea/gitea/assets/9418365/71dfea7e-d9e2-42b6-851a-cc081435c946

Thanks to @CaiCandong  for helping with some tests.
2023-08-23 02:29:17 +00:00
Bo-Yi Wu
23addde28e
feat: implement organization secret creation API (#26566)
- Add a new `CreateSecretOption` struct for creating secrets
- Implement a `CreateOrgSecret` function to create a secret in an
organization
- Add a new route in `api.go` to handle the creation of organization
secrets
- Update the Swagger template to include the new `CreateOrgSecret` API
endpoint

---------

Signed-off-by: appleboy <appleboy.tw@gmail.com>
2023-08-22 11:20:34 +08:00
techknowlogick
b3f7137174
Update minimum password length requirements (#25946) 2023-08-21 19:27:50 +00:00
delvh
3d80308b36
Use correct minio error (#26634)
Previously, `err` was defined above, checked for `err == nil` and used
nowhere else.
Hence, the result of `convertMinioErr` would always be `nil`.
This leads to a NPE further down the line.
That is not intentional, it should convert the error of the most recent
operation, not one of its predecessors.

Found through
https://discord.com/channels/322538954119184384/322538954119184384/1143185780206993550.
2023-08-21 16:20:11 +00:00
wxiaoguang
0731abc444
Remove avatarHTML from template helpers (#26598)
The HTML code is more readable and more correct (it needs `"ui avatar"`
class)

Co-authored-by: Giteabot <teabot@gitea.io>
2023-08-21 15:00:14 +00:00
Yarden Shoham
16dee4f1b2
Add branch_filter to hooks API endpoints (#26599)
We now include the branch filler in the response.

- Closes #26591 

# Before

![image](https://github.com/go-gitea/gitea/assets/20454870/73933940-c1a7-4573-abae-f340b63028b2)

# After

![image](https://github.com/go-gitea/gitea/assets/20454870/3b3c4a85-0f7c-48c7-8617-def7a66c671d)

---------

Signed-off-by: Yarden Shoham <git@yardenshoham.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-08-21 11:43:52 +00:00
yp05327
f6e7798405
Add link to job details and tooltip to commit status in repo list in dashboard (#26326)
Tooltip:

![image](https://github.com/go-gitea/gitea/assets/18380374/237cb545-7844-424b-b995-1008eaaaedec)

Link to the target job:

![image](https://github.com/go-gitea/gitea/assets/18380374/0c11a97f-6517-47f2-8773-f381488c084e)
2023-08-21 15:26:10 +08:00
silverwind
3db3f5daae
Update tool dependencies (#26607)
- Updated all tool dependencies to latest versions
- Add Makefile to `swagger` files because it specifies `go-swagger`
version
- Fix lint
2023-08-20 22:59:19 +00:00
TimberBro
84d05516dc
Fix NPM packages name validation (#26595)
- Added new tests to cover corner cases
- Replace existing regex with new one
Closes #26551 

---
As @silverwind suggested, I started from
[validate-npm-package-name](https://github.com/npm/validate-npm-package-name),
but found this solution too complicated.
Then I tried to fix existing regex myself, but thought, that exclude all
restricted symbols is harder, than set only allowed symbols.
Then I search a bit more and found
[package-name-regex](https://github.com/dword-design/package-name-regex)
and regex from it works for all new test cases.

Let me know, if more information or help with this PR is needed.
2023-08-20 15:06:42 +00:00
Jason Song
47fddaadc8
Support rebuilding issue indexer manually (#26546)
Provide a way to rebuild issue indexer manually.

So if the indexer get outdated because of some bugs like #26539, we can
rebuild it.

<img width="1104" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/ac242e29-6f04-47ca-b3d0-801a796448d3">

Co-authored-by: Giteabot <teabot@gitea.io>
2023-08-17 14:05:17 +00:00
wxiaoguang
c6b92c84fe
Sync repo's IsEmpty status correctly (#26517)
Close #26509
2023-08-17 04:43:39 +00:00
Jason Song
3b129aaa80
Explain SearchOptions and fix ToSearchOptions (#26542)
Follow #26012 #26490.

A detailed description has been added to the comment.
2023-08-16 15:40:13 +00:00
Jason Song
7e75e58ae4
Add missing triggers to update issue indexer (#26539)
Fix #26536

Follow #26012

Whatever the comment type is, always update the issue indexer. So the
issue indexer will be updated when there is a change in Status,
Assignee, Label, and so on.

I added the logic for `NotifyUpdateComment`, but missed it for
`NotifyCreateIssueComment` and `NotifyDeleteComment`.
2023-08-16 17:36:05 +08:00
CaiCandong
0e74fc4a84
Fix project filter bugs (#26490)
related: #26012

### Bugs
1. missing project filter on the issue page.

1e76a824bc/modules/indexer/issues/dboptions.go (L11-L15)
3. incorrect SQL condition: some issue does not belong to a project but
exists on the project_issue table.

f5dbac9d36/models/issues/issue_search.go (L233)

### Before:

![before](https://github.com/go-gitea/gitea/assets/50507092/1dcde39e-3e2f-4151-b2c6-4d67bf493c2f)

### After:

![after](https://github.com/go-gitea/gitea/assets/50507092/badfb81f-056d-4a2f-9838-1cba9c15768d)

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-08-15 14:50:12 +00:00
yp05327
7f8028e5a1
Fix display problems of members and teams unit (#26363)
Fix:
- display member count and team count in the menu bar

![image](https://github.com/go-gitea/gitea/assets/18380374/7f03ced4-67e2-41ce-b19f-a992823726bb)
- Also display member unit in the menu bar if there are no hidden
members in public org

![image](https://github.com/go-gitea/gitea/assets/18380374/31422ad6-7190-438d-8e99-8a4af9cce908)
- hidden member board when there's no seeable members.
In this org, we only have hidden members: 

![image](https://github.com/go-gitea/gitea/assets/18380374/d749420b-554a-4483-8cd2-221df61b5ca7)
We will hidden the member board when doer is not the member of this org

![image](https://github.com/go-gitea/gitea/assets/18380374/93bb782e-7d4d-4ad3-a096-133afbc51f8a)
Before:

![image](https://github.com/go-gitea/gitea/assets/18380374/eafc0b3e-6218-42ab-a892-39645d08a5eb)
If you click the number in the members board, you will access the
members page, which is not expected.

![image](https://github.com/go-gitea/gitea/assets/18380374/73d6dadc-0ef2-4ca9-8485-c5f4211bffb2)

---------

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-08-15 16:00:35 +02:00
Bo-Yi Wu
79d74d208f
Add API route to list org secrets (#26485)
- Add a new function `CountOrgSecrets` in the file
`models/secret/secret.go`
- Add a new file `modules/structs/secret.go`
- Add a new function `ListActionsSecrets` in the file
`routers/api/v1/api.go`
- Add a new file `routers/api/v1/org/action.go`
- Add a new function `listActionsSecrets` in the file
`routers/api/v1/org/action.go`

go-sdk: https://gitea.com/gitea/go-sdk/pulls/629

---------

Signed-off-by: Bo-Yi Wu <appleboy.tw@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-08-15 14:32:43 +02:00
wxiaoguang
ced34bab1a
Detect ogg mime-type as audio or video (#26494)
"ogg" is just a "container" format for audio and video.

Golang's `DetectContentType` only reports "application/ogg" for
potential ogg files.

Actually it could do more "guess" to see whether it is a audio file or a
video file.
2023-08-15 10:31:25 +08:00
wxiaoguang
ed1be4ca68
Handle base64 decoding correctly to avoid panic (#26483)
Fix the panic if the "base64 secret" is too long.
2023-08-14 10:30:16 +00:00
puni9869
cafce3b4b5
Allow to archive labels (#26478)
## Archived labels 

This adds the structure to allow for archived labels.
Archived labels are, just like closed milestones or projects, a medium to hide information without deleting it.
It is especially useful if there are outdated labels that should no longer be used without deleting the label entirely.

## Changes

1. UI and API have been equipped with the support to mark a label as archived
2. The time when a label has been archived will be stored in the DB

## Outsourced for the future

There's no special handling for archived labels at the moment.
This will be done in the future.

## Screenshots

![image](https://github.com/go-gitea/gitea/assets/80308335/208f95cd-42e4-4ed7-9a1f-cd2050a645d4)

![image](https://github.com/go-gitea/gitea/assets/80308335/746428e0-40bb-45b3-b992-85602feb371d)

Part of https://github.com/go-gitea/gitea/issues/25237

---------

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-08-14 11:56:14 +02:00
Lunny Xiao
389fb754a2
Fix storage path logic especially for relative paths (#26441)
This PR rewrites the function `getStorage` and make it more clear.

Include tests from #26435, thanks @earl-warren

---------

Co-authored-by: Earl Warren <contact@earl-warren.org>
2023-08-13 22:09:25 +02:00
wxiaoguang
7018659a1d
Close stdout correctly for "git blame" (#26470)
Close stdout correctly for "git blame", otherwise the failed "git blame"
would case the request hanging forever.

And "os.Stderr" should never (seldom) be used as git command's stderr
2023-08-13 02:11:20 +00:00
wxiaoguang
bcccf4c0d6
Remove last newline from config file (#26468)
When users put the secrets into a file (GITEA__sec__KEY__FILE), the
newline sometimes is different to avoid (eg: echo/vim/...)

So the last newline could be removed when reading, it makes the users
easier to maintain the secret files.

Co-authored-by: Giteabot <teabot@gitea.io>
2023-08-12 15:28:35 +00:00
Lunny Xiao
a321a4c2fc
Adjust minio new sequence, now it will check whether bucket exist first and then create one if it doesn't exist (#26420)
For some reason, the permission of the client_id and secret may cannot
create bucket, so now we will check whether bucket does exist first and
then try to create a bucket if it doesn't exist.

Try to fix #25984

Co-authored-by: silverwind <me@silverwind.io>
2023-08-12 08:03:54 +00:00
Jack Hay
c5888eb985
Set commit id when ref used explicitly (#26447)
In the `RepoRefForAPI()` context function `CommitID` is not set if `ref`
is used. It is set correctly for other if/else branches where `Commit`
is set. It doesn't appear that any routes that use `RepoRefForAPI()`
also use `CommitID` but that may be the case in the future.

## Changes
- Sets `ctx.Repo.CommitID` when `ref` is explicitly used for api routes
that use `RepoRefForAPI()`
2023-08-12 07:33:12 +00:00
nekrondev
3c91b09c19
minio: add missing region on client initialization (#26412)
The MinIO client isn't redirecting to the correct AWS endpoint if a
non-default data center is used.

In my use case I created an AWS bucket at `eu-central-1` region. Because
of the missing region initialization of the client the default
`us-east-1` API endpoint is used returning a `301 Moved Permanently`
response that's not handled properly by MinIO client. This in return
aborts using S3 storage on AWS as the `BucketExists()` call will fail
with the http moved error.

MinIO client trace shows the issue:

```text
---------START-HTTP---------
HEAD / HTTP/1.1
Host: xxxxxxxxxxx-prod-gitea-data.s3.dualstack.us-east-1.amazonaws.com
User-Agent: MinIO (windows; amd64) minio-go/v7.0.61
Authorization: AWS4-HMAC-SHA256 Credential=**REDACTED**/20230809/accesspoint.eu-central-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=**REDACTED**
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20230809T141143Z

HTTP/1.1 301 Moved Permanently
Connection: close
Content-Type: application/xml
Date: Wed, 09 Aug 2023 14:11:43 GMT
Server: AmazonS3
X-Amz-Bucket-Region: eu-central-1
X-Amz-Id-2: UK7wfeYi0HcTcytNvQ3wTAZ5ZP1mOSMnvRZ9Fz4xXzeNsS47NB/KfFx2unFxo3L7XckHpMNPPVo=
X-Amz-Request-Id: S1V2MJV8SZ11GEVN
---------END-HTTP---------
```

Co-authored-by: Heiko Besemann <heiko.besemann@qbeyond.de>
2023-08-10 11:21:09 +00:00
wxiaoguang
a370efc13f
Use template context function for avatar rendering (#26385)
Introduce `AvatarUtils`, no need to pass `$.Context` to every
sub-template, and simplify the template helper functions.
2023-08-10 11:19:39 +08:00
wxiaoguang
a85a862804
Fix admin queue page title (#26409)
Fix #26408

```
# locale_en-US.ini
[admin]
monitor.queues = Queues
```

---------

Co-authored-by: silverwind <me@silverwind.io>
2023-08-10 02:05:37 +00:00
Denys Konovalov
63ab92d797
Pre-register OAuth2 applications for git credential helpers (#26291)
This PR is an extended implementation of #25189 and builds upon the
proposal by @hickford in #25653, utilizing some ideas proposed
internally by @wxiaoguang.

Mainly, this PR consists of a mechanism to pre-register OAuth2
applications on startup, which can be enabled or disabled by modifying
the `[oauth2].DEFAULT_APPLICATIONS` parameter in app.ini. The OAuth2
applications registered this way are being marked as "locked" and
neither be deleted nor edited over UI to prevent confusing/unexpected
behavior. Instead, they're being removed if no longer enabled in config.


![grafik](https://github.com/go-gitea/gitea/assets/47871822/81a78b1c-4b68-40a7-9e99-c272ebb8f62e)

The implemented mechanism can also be used to pre-register other OAuth2
applications in the future, if wanted.

Co-authored-by: hickford <mirth.hickford@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>

---------

Co-authored-by: M Hickford <mirth.hickford@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-08-09 14:24:07 +02:00
CaiCandong
d41aee1d1e
Make user-content-* consistent with github (#26388)
Fix #26367
Related #19745

Thanks @lazyky for providing  test cases
2023-08-09 09:30:31 +00:00
wxiaoguang
c2e0143bfe
Introduce ctx.PathParamRaw to avoid incorrect unescaping (#26392)
Fix #26389

And complete an old TODO: `ctx.Params does un-escaping,..., which is
incorrect.`
2023-08-09 14:57:45 +08:00
wxiaoguang
4fc4f6e634
Refactor "editorconfig" (#26391)
There are 2 kinds of ".Editorconfig" in code, one is `JSON string` for
the web edtior, another is `*editorconfig.Editorconfig` for the file
rendering (used by `TabSizeClass`)

This PR distinguish them with different names.

And by the way, change the default tab size from 8 to 4, I think few
people would like to use 8-size tabs nowadays.
2023-08-08 10:44:19 +00:00
wxiaoguang
71d253f42e
Remove unnecessary template helper DisableGravatar (#26386)
And one "AllowedUserVisibilityModes" was missing, add it.

Co-authored-by: Giteabot <teabot@gitea.io>
2023-08-08 08:29:14 +00:00
wxiaoguang
78b2a1cc36
Remove unnecessary template helper repoAvatar (#26387)
And simplify the "repo/icon" code
2023-08-08 15:29:35 +08:00
wxiaoguang
6913053223
Start using template context function (#26254)
Before:

* `{{.locale.Tr ...}}`
* `{{$.locale.Tr ...}}`
* `{{$.root.locale.Tr ...}}`
* `{{template "sub" .}}`
* `{{template "sub" (dict "locale" $.locale)}}`
* `{{template "sub" (dict "root" $)}}`
* .....

With context function: only need to `{{ctx.Locale.Tr ...}}`

The "ctx" could be considered as a super-global variable for all
templates including sub-templates.


To avoid potential risks (any bug in the template context function
package), this PR only starts using "ctx" in "head.tmpl" and
"footer.tmpl" and it has a "DataRaceCheck". If there is anything wrong,
the code can be fixed or reverted easily.
2023-08-08 01:22:47 +00:00
cassiozareck
ab0eb1c47b
Rename code_langauge.go to code_language.go (#26377) 2023-08-07 15:00:53 -04:00
Earl Warren
87f70979cf
Do not highlight #number in documents (#26365)
- Currently the post processing will transform all issue indexes (such as `#6`) into a clickable link.
- This makes sense in an situation like issues or PRs,
where referencing to other issues is quite common
and only referencing their issue index is an handy and efficient way to do it.
- Currently this is also run for documents
(which is the user profile and viewing rendered files),
but in those situations it's less common to reference issues by their index and instead could mean something else.
- This patch disables this post processing for issue index for documents. Matches Github's behavior.
- Added unit tests.
- Resolves https://codeberg.org/Codeberg/Community/issues/1120

Co-authored-by: Gusted <postmaster@gusted.xyz>
2023-08-07 15:11:25 +02:00
wxiaoguang
d92b4cd093
Fix incorrect CLI exit code and duplicate error message (#26346)
Follow the CLI refactoring, and add tests.
2023-08-05 23:36:45 +08:00
KN4CK3R
2d3924d0e7
Prevent newline errors with Debian packages (#26332)
Fixes #26313
2023-08-05 10:59:52 +02:00
Zettat123
9a8af92577
Fix the bug when getting files changed for pull_request_target event (#26320)
Follow #25229

Copy from
https://github.com/go-gitea/gitea/pull/26290#issuecomment-1663135186

The bug is that we cannot get changed files for the
`pull_request_target` event. This event runs in the context of the base
branch, so we won't get any changes if we call
`GetFilesChangedSinceCommit` with `PullRequest.Base.Ref`.
2023-08-05 14:26:06 +08:00
wxiaoguang
5db4c8db93
Refactor backend SVG package and add tests (#26335)
Introduce a well-tested `svg.Normalize` function.
Make `RenderHTML` faster and more stable.
2023-08-05 04:34:59 +00:00
wxiaoguang
2de0752be7
Make git batch operations use parent context timeout instead of default timeout (#26325)
Fix #26064

Some git commands should use parent context, otherwise it would exit too
early (by the default timeout, 10m), and the "cmd.Wait" waits till the
pipes are closed.
2023-08-04 14:50:41 +02:00
Lunny Xiao
96f151392f
Fix the wrong derive path (#26271)
This PR will fix #26264, caused by #23911.

The package configuration derive is totally wrong when storage type is
local in that PR.

This PR fixed the inherit logic when storage type is local with some
unit tests.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-08-04 03:41:16 +00:00
Zettat123
8a2f019d69
Support getting changed files when commit ID is EmptySHA (#26290)
Fixes #26270.

Co-Author: @wxiaoguang 

Thanks @lunny for providing this solution

As
https://github.com/go-gitea/gitea/issues/26270#issuecomment-1661695151
said, at present we cannot get the names of changed files correctly when
the `OldCommitID` is `EmptySHA`. In this PR, the `GetCommitFilesChanged`
method is added and will be used to get the changed files by commit ID.

References:
- https://stackoverflow.com/a/424142

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-08-04 02:53:15 +00:00
wxiaoguang
54c28fddd8
Clarify the logger's MODE config option (#26267)
1. Fix the wrong document (add the missing `MODE=`)
2. Add a more friendly log message to tell users to add `MODE=` in their
config

Co-authored-by: Giteabot <teabot@gitea.io>
2023-08-01 18:28:23 +00:00
Panagiotis "Ivory" Vasilopoulos
d58c542579
Add 'Show on a map' button to Location in profile, fix layout (#26214)
Not too important, but I think that it'd be a pretty neat touch.

Also fixes some layout bugs introduced by a previous PR.

---------

Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-authored-by: Caesar Schinas <caesar@caesarschinas.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-07-31 08:44:45 +00:00
Jason Song
1e76a824bc
Refactor and enhance issue indexer to support both searching, filtering and paging (#26012)
Fix #24662.

Replace #24822 and #25708 (although it has been merged)


## Background

In the past, Gitea supported issue searching with a keyword and
conditions in a less efficient way. It worked by searching for issues
with the keyword and obtaining limited IDs (as it is heavy to get all)
on the indexer (bleve/elasticsearch/meilisearch), and then querying with
conditions on the database to find a subset of the found IDs. This is
why the results could be incomplete.

To solve this issue, we need to store all fields that could be used as
conditions in the indexer and support both keyword and additional
conditions when searching with the indexer.

## Major changes

- Redefine `IndexerData` to include all fields that could be used as
filter conditions.
- Refactor `Search(ctx context.Context, kw string, repoIDs []int64,
limit, start int, state string)` to `Search(ctx context.Context, options
*SearchOptions)`, so it supports more conditions now.
- Change the data type stored in `issueIndexerQueue`. Use
`IndexerMetadata` instead of `IndexerData` in case the data has been
updated while it is in the queue. This also reduces the storage size of
the queue.
- Enhance searching with Bleve/Elasticsearch/Meilisearch, make them
fully support `SearchOptions`. Also, update the data versions.
- Keep most logic of database indexer, but remove
`issues.SearchIssueIDsByKeyword` in `models` to avoid confusion where is
the entry point to search issues.
- Start a Meilisearch instance to test it in unit tests.
- Add unit tests with almost full coverage to test
Bleve/Elasticsearch/Meilisearch indexer.

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-07-31 06:28:53 +00:00
KN4CK3R
d08ab7be9c
Fix typo in metadata (#26207)
https://github.com/go-gitea/gitea/pull/26194#discussion_r1277094525

There is no need to backport because these names are just used internal.
2023-07-30 23:54:22 +00:00
Lunny Xiao
ce27de4d48
Fix allowed user types setting problem (#26200)
Fix #25951
2023-07-28 12:15:39 -04:00
Earl Warren
494ee56de2
improve unit test for caching (#26185)
- Configure `setting.CacheService.TTL` which will force the code to go
trough the caching mechanism.
- Remove the TODO and uncomment the test code.

(cherry picked from commit a201f2f1896c1a086f3e37f86b1306d44c07f2e4)

Refs: https://codeberg.org/forgejo/forgejo/pulls/974

---------

Co-authored-by: Gusted <postmaster@gusted.xyz>
2023-07-27 22:24:40 +02:00
Earl Warren
eaea530d6b
Render plaintext task list items for markdown files (#26186)
- The library that's being used for org-mode, [doesn't render the status
of list items](https://github.com/niklasfasching/go-org/issues/63).
- Add a modified version of the proposed CSS snippet to still display
the status for the list items. The alternative was parsing HTML and
transforming it, which is too complicated for this small task.
- Resolves https://codeberg.org/Codeberg/Community/issues/1099

(cherry picked from commit 9753c7e4b8490b8f1e3d19cb06187503b88afb88)

Refs: https://codeberg.org/forgejo/forgejo/pulls/1071

Co-authored-by: Gusted <postmaster@gusted.xyz>
2023-07-27 14:15:31 +00:00
delvh
bd6ef71854
Show branches and tags that contain a commit (#25180)
Now, you can see for a commit which existing branches and tags contain it.
You first have to click on the `load branches and tags` button, they are not preloaded by default.
All branches and tags are ordered descending by creation date.
You can even see without much hassle if the given commit is already part of the default branch.

Closes #25152 

## Screenshots

### Initial

![image](https://github.com/go-gitea/gitea/assets/51889757/84db2c0b-aaef-4f69-ab92-0b812793d2ad)

### Loaded

![image](https://github.com/go-gitea/gitea/assets/51889757/a9b84e66-8e44-4c55-b017-c37f4a45f41b)

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-07-27 12:47:41 +02:00
wxiaoguang
8baa42c8d7
Calculate MAX_WORKERS default value by CPU number (#26177)
To avoid consuming user's 100% CPU, limit the default value of
MAX_WORKERS

Fix #26063 (the CPU 100% problem mentioned in it)
2023-07-27 16:40:35 +08:00
caicandong
f2cc4daf60
Doc update swagger doc for POST /orgs/{org}/teams (#26155)
close #26111
2023-07-26 10:22:56 +00:00
caicandong
13359581df
refactor improve NoBetterThan (#26126)
- The `NoBetterThan` function can only handle comparisons between
"pending," "success," "error," and "failure." For any other comparison,
we directly return false. This prevents logic errors like the one in
#26121.
- The callers of the `NoBetterThan` function should also avoid making
incomparable calls.

---------

Co-authored-by: yp05327 <576951401@qq.com>
Co-authored-by: puni9869 <80308335+puni9869@users.noreply.github.com>
2023-07-26 08:52:07 +00:00
wxiaoguang
dcd3a63128
Move web JSON functions to web context and simplify code (#26132)
The JSONRedirect/JSONOK/JSONError functions were put into "Base" context
incorrectly, it would cause abuse.

Actually, they are for "web context" only, so, move them to the correct
place.

And by the way, use them to simplify old code: +75 -196
2023-07-26 06:04:01 +00:00
Lunny Xiao
5dc37ef97a
Display deprecated warning in admin panel pages as well as in the log file (#26094)
This PR includes #26007 's changes but have a UI to prompt administrator
about the deprecated settings as well as the log or console warning.
Then users will have enough time to notice the problem and don't have
surprise like before.

<img width="1293" alt="图片"
src="https://github.com/go-gitea/gitea/assets/81045/c33355f0-1ea7-4fb3-ad43-cd23cd15391d">

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-07-26 03:53:37 +00:00
wxiaoguang
3e4a4f9c67
Fix incorrect router logger (#26137)
A low-level mistake:

* `log.Info` is global `Info` function, which calls "default" logger
* `logger.Info` is the for router's logger
2023-07-25 22:21:09 +00:00
JakobDev
6598d0291c
Allow Organisations to have a E-Mail (#25082)
Resolves #25057

This adds a E-Mail field to Organisations. The E-Mail is just shown on
the Profile when it is visited by a logged in User. The E-mail is not
used for something else.

**Screenshots:**

![grafik](https://github.com/go-gitea/gitea/assets/15185051/a8d622b3-7278-4c08-984b-9c5ebfdb5471)

![grafik](https://github.com/go-gitea/gitea/assets/15185051/6dcb1dd7-d04b-49eb-bc96-6582cfe9757b)

---------

Co-authored-by: Denys Konovalov <kontakt@denyskon.de>
Co-authored-by: Denys Konovalov <privat@denyskon.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-07-25 08:26:27 +00:00
Jason Song
862f748b2b
Fix loading LFS_JWT_SECRET from wrong section (#26109)
Regression of #25408.

`LFS_JWT_SECRET_URI` and `LFS_JWT_SECRET` are under the `server`
section, not `lfs`.

So each time gitea restarts, it resets `LFS_JWT_SECRET`.

<img width="1378" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/527ec9b9-eea2-4334-a274-6d1471456edd">
2023-07-25 05:06:31 +00:00
wxiaoguang
d0bdfbc545
Avoid writing config file if not installed (#26107)
Just like others (oauth2 secret, internal token, etc), do not generate
if no install lock
2023-07-25 04:03:41 +00:00
KN4CK3R
6aa30af724
Fix handling of Debian files with trailing slash (#26087)
Fixes #26022

- Fix handling of files with trailing slash
- Fix handling of duplicate package file errors
- Added test for both
2023-07-24 16:19:44 +02:00
idk
cdd3d4b8d8
Allow the use of alternative net.Listener implementations by downstreams (#25855)
This is a simple PR which moves the `GetListener` function to a
`DefaultGetListener` function, and changes `GetListener` to be a
variable which by default points to the `DefaultGetListener` function.
This allows people who may exist quasi-downstream of Gitea to create
alternate "GetListener" functions, with identical signatures, which
return different implementations of the `net.Listener` interface. This
approach is expressly intended to be non-invasive and have the least
possible impact on the gitea codebase. A previous version of this idea
was rejected before: https://github.com/go-gitea/gitea/issues/15544 but
because of issues like: https://github.com/go-gitea/gitea/issues/22335 I
**really** think that recommending people configure proxies by hand is
exactly the wrong way to do things(This is why there is a Tor Browser.).
This tiny change lets me put proper hidden service configuration into
single `i2p.go` file which lives in `modules/graceful/` and which never
has to be checked in to your codebase or affect your dependencies or
bloat your project in any way, it can live on a branch in my fork and
I'll fast-forward every release and never the twain shall meet.

The main use-case for this is to listen on Peer-to-Peer networks and
Hidden Services directly without error-prone and cumbersome
port-forwarding configuration. For instance, I might implement an
"I2PGetListener" as follows:

```Go
// adapted from i2p.go which is unchecked-in in my modules/graceful/ directory
import "github.com/eyedeekay/onramp"

var garlic = &onramp.Garlic{}

func I2PGetListener(network, address string) (net.Listener, error) {
	// Add a deferral to say that we've tried to grab a listener
	defer GetManager().InformCleanup()
	switch network {
	case "tcp", "tcp4", "tcp6", "i2p", "i2pt":
		return garlic.Listen()
	case "unix", "unixpacket":
// I2P isn't really a replacement for the stuff you use Unix sockets for and it's also not an anonymity risk, so treat them normally
		unixAddr, err := net.ResolveUnixAddr(network, address)
		if err != nil {
			return nil, err
		}
		return GetListenerUnix(network, unixAddr)
	default:
		return nil, net.UnknownNetworkError(network)
	}
}
```

I could then substitute that GetListener function and be 50% of the way
to having a fully-functioning gitea-over-hidden-services instance
without any additional configuration(The other 50% doesn't require any
code-changes on gitea's part).

There are 2 advantages here, one being convenience, first this turns
hidden services into a zero-configuration option for self-hosting gitea,
and second safety, these Go libraries are passing around
hidden-service-only versions of the net.Addr struct, they're using
hidden-service-only versions of the sockets, which are both expressly
designed to never require access to any information outside the hidden
service network, manipulating the application so it reveals information
about the host becomes much more difficult, and some attacks become
nearly impossible. It also opens up TLS-over-Hidden Services support
which is niche right now, of course, but in a future where gitea
instances federate if hidden services want to be part of the federation
they're probably going to need TLS certificates. They don't need to be
painful to set up.

This doesn't fix an open issue, but it might affect:
- https://github.com/go-gitea/gitea/issues/22335 - my `i2p.go` file
actually has a mod that fixes this but it requires adding a handful of
new dependencies to gitea and isn't compatible with the normal way you
guys recommend using a proxy so I don't think it's ready to send to you
as a PR, but if I can find a non-invasive way to fix it I will.
 - https://github.com/go-gitea/gitea/issues/18240

I hereby agree to the Code of Conduct published here:
8b89563bf1/CODE_OF_CONDUCT.md
I have read and understood the recommendations published here:
8b89563bf1/CONTRIBUTING.md

Thank you for your consideration.

---------

Co-authored-by: eyedeekay <idk@mulder>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-07-24 07:18:17 +00:00
wxiaoguang
674df05b16
Use stderr as fallback if the log file can't be opened (#26074)
If the log file can't be opened, what should it do? panic/exit? ignore
logs? fallback to stderr?

It seems that "fallback to stderr" is slightly better than others ....
2023-07-24 04:57:21 +00:00
Lunny Xiao
b167f35113
Add context parameter to some database functions (#26055)
To avoid deadlock problem, almost database related functions should be
have ctx as the first parameter.
This PR do a refactor for some of these functions.
2023-07-22 22:14:27 +08:00
wxiaoguang
52fb936773
Serve pre-defined files in "public", add "security.txt", add CORS header for ".well-known" (#25974)
Replace #25892

Close  #21942
Close  #25464

Major changes:

1. Serve "robots.txt" and ".well-known/security.txt" in the "public"
custom path
* All files in "public/.well-known" can be served, just like
"public/assets"
3. Add a test for ".well-known/security.txt"
4. Simplify the "FileHandlerFunc" logic, now the paths are consistent so
the code can be simpler
5. Add CORS header for ".well-known" endpoints
6. Add logs to tell users they should move some of their legacy custom
public files

```
2023/07/19 13:00:37 cmd/web.go:178:serveInstalled() [E] Found legacy public asset "img" in CustomPath. Please move it to /work/gitea/custom/public/assets/img
2023/07/19 13:00:37 cmd/web.go:182:serveInstalled() [E] Found legacy public asset "robots.txt" in CustomPath. Please move it to /work/gitea/custom/public/robots.txt
```
This PR is not breaking.

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-07-21 12:14:20 +00:00
HesterG
2f0e79e639
Use frontend fetch for branch dropdown component (#25719)
- Send request to get branch/tag list, use loading icon when waiting for
response.
- Only fetch when the first time branch/tag list shows.
- For backend, removed assignment to `ctx.Data["Branches"]` and
`ctx.Data["Tags"]` from `context/repo.go` and passed these data wherever
needed.
- Changed some `v-if` to `v-show` and used native `svg` as mentioned in
https://github.com/go-gitea/gitea/pull/25719#issuecomment-1631712757 to
improve perfomance when there are a lot of branches.
- Places Used the dropdown component:

     Repo Home Page
    
<img width="1429" alt="Screen Shot 2023-07-06 at 12 17 51"
src="https://github.com/go-gitea/gitea/assets/17645053/6accc7b6-8d37-4e88-ae1a-bd2b3b927ea0">

    Commits Page

<img width="1431" alt="Screen Shot 2023-07-06 at 12 18 34"
src="https://github.com/go-gitea/gitea/assets/17645053/2d0bf306-d1e2-45a8-a784-bc424879f537">

    Specific commit -> operations -> cherry-pick
    
<img width="758" alt="Screen Shot 2023-07-06 at 12 23 28"
src="https://github.com/go-gitea/gitea/assets/17645053/1e557948-3881-4e45-a625-8ef36d45ae2d">

    Release Page
    
<img width="1433" alt="Screen Shot 2023-07-06 at 12 25 05"
src="https://github.com/go-gitea/gitea/assets/17645053/3ec82af1-15a4-4162-a50b-04a9502161bb">

- Demo


https://github.com/go-gitea/gitea/assets/17645053/d45d266b-3eb0-465a-82f9-57f78dc5f9f3

- Note:

UI of dropdown menu could be improved in another PR as it should apply
to more dropdown menus.

Fix #14180

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-07-21 11:20:04 +00:00
caicandong
840830b655
Remove commit status running and warning to align GitHub (#25839)
Fix #25776. Close #25826.

In the discussion of #25776, @wolfogre's suggestion was to remove the
commit status of `running` and `warning` to keep it consistent with
github.

references:
-
https://docs.github.com/en/rest/commits/statuses?apiVersion=2022-11-28#about-commit-statuses


## ⚠️ BREAKING ⚠️

So the commit status of Gitea will be consistent with GitHub, only
`pending`, `success`, `error` and `failure`, while `warning` and
`running` are not supported anymore.

---------

Co-authored-by: Jason Song <i@wolfogre.com>
2023-07-21 16:24:36 +08:00
Lunny Xiao
037c9895a7
Support copy protected branch from template repository (#25889)
Fix #14303
2023-07-21 12:32:47 +08:00
wxiaoguang
9b25bfa8f4
Remove redundant "RouteMethods" method (#26024)
The `RouteMethods` is mainly an alias for `Methods` with different
argument order.

Remove it to keep the "route.go" code clear
2023-07-21 00:43:49 +02:00
puni9869
d12ba978a7
Adding remaining enum for migration repo model type. (#26021) 2023-07-20 12:28:38 -05:00
harryzcy
ec227d6682
Remove nfnt/resize and oliamb/cutter (#25999)
The package `github.com/nfnt/resize` is deprecated and archived by the
author. `github.com/oliamb/cutter` is not maintained since 2018. We
could use `golang.org/x/image/draw` instead.
2023-07-20 19:52:42 +08:00
wxiaoguang
28ecac8024
Fix env config parsing for "GITEA____APP_NAME" (#26001)
Regression of #24832 

Fix the bug and add a test for it

Co-authored-by: Giteabot <teabot@gitea.io>
2023-07-20 09:16:29 +00:00
jeremiepozzigithub
d7a8d09da0
Add file status for API "Get a single commit from a repository" (#16205) (#25831)
#16205 To obtain a closer behavior to the api from github, the status
(added, modified, removed) of a file should be available in addition to
the filename.
See github doc :

https://docs.github.com/fr/rest/commits/commits?apiVersion=2022-11-28#get-a-commit
2023-07-20 16:35:47 +08:00
harryzcy
0f9f6567bb
Bump github.com/golang-jwt/jwt to v5 (#25975)
Bumping `github.com/golang-jwt/jwt` from v4 to v5.

`github.com/golang-jwt/jwt` v5 is bringing some breaking changes:

- standard `Valid()` method on claims is removed. It's replaced by
`ClaimsValidator` interface implementing `Validator()` method instead,
which is called after standard validation. Gitea doesn't seem to be
using this logic.
- `jwt.Token` has a field `Valid`, so it's checked in `ParseToken`
function in `services/auth/source/oauth2/token.go`

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-07-19 09:57:10 +00:00
wxiaoguang
50e14699d3
Update path related documents (#25417)
Update WorkPath/WORK_PATH related documents, remove out-dated
information.

Remove "StaticRootPath" on the admin config display page, because few
end user really need it, it only causes misconfiguration.


![image](https://github.com/go-gitea/gitea/assets/2114189/8095afa4-da76-436b-9e89-2a92c229c01d)

Co-authored-by: Giteabot <teabot@gitea.io>
2023-07-19 11:22:57 +02:00
wxiaoguang
236c645bf1
Refactor "Content" for file uploading (#25851)
Before: the concept "Content string" is used everywhere. It has some
problems:

1. Sometimes it means "base64 encoded content", sometimes it means "raw
binary content"
2. It doesn't work with large files, eg: uploading a 1G LFS file would
make Gitea process OOM

This PR does the refactoring: use "ContentReader" / "ContentBase64"
instead of "Content"

This PR is not breaking because the key in API JSON is still "content":
`` ContentBase64 string `json:"content"` ``
2023-07-18 18:14:47 +00:00
wxiaoguang
faa28b5a44
Move public asset files to the proper directory (#25907)
Move `public/*` to `public/assets/*`

Some old PRs (like #15219) introduced inconsistent directory system.

For example: why the local directory "public" is accessed by
`http://site/assets`? How to serve the ".well-known" files properly in
the public directory?

For convention rules, the "public" directory is widely used for the
website's root directory. It shouldn't be an exception for Gitea.

So, this PR makes the things consistent:

* `http://site/assets/foo` means `{CustomPath}/public/assets/foo`.
* `{CustomPath}/public/.well-known` and `{CustomPath}/public/robots.txt`
can be used in the future.

This PR is also a prerequisite for a clear solution for:
* #21942
* #25892 
* discourse.gitea.io: [.well-known path serving custom files behind
proxy?](https://discourse.gitea.io/t/well-known-path-serving-custom-files-behind-proxy/5445/1)

This PR is breaking for users who have custom "public" files (CSS/JS).
After getting approvals, I will update the documents.

----

## ⚠️ BREAKING ⚠️

If you have files in your "custom/public/" folder, please move them to
"custom/public/assets/".

---------

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-07-18 18:06:43 +02:00
KN4CK3R
8af96f585f
Disallow dangerous url schemes (#25960)
Regression: https://github.com/go-gitea/gitea/pull/24805
Closes: #25945

- Disallow `javascript`, `vbscript` and `data` (data uri images still
work) url schemes even if all other schemes are allowed
- Fixed older `cbthunderlink` tests

---------

Co-authored-by: delvh <dev.lh@web.de>
2023-07-18 15:18:37 +00:00
wxiaoguang
cc73e84fa3
Avoid creating directories when loading config (#25944)
The "creating dir/file during load config" is a longstanding and complex
problem.

This PR only does a quick patch, it still needs more refactorings in the
future.

Fix #25938
2023-07-18 07:32:36 -05:00
wxiaoguang
ff9a7afe99
Skip unuseful error message in dev mode when watching local filesystem (#25919)
Before, in dev mode, there might be some error logs like:

```
2023/07/17 13:54:51 ...s/assetfs/layered.go:221:WatchLocalChanges() [E] Unable to watch directory .: lstat /data/work/gitea/custom/templates: no such file or directory

```

Because there is no "custom/templates" directory.

After: ignore such error, no such error message anymore.
2023-07-17 09:47:55 +00:00
KN4CK3R
81a8120bc3
Add shutting down notice (#25920)
Got the same problem as #25915 when updating an instance. The
`log.Fatal` should have been marked as breaking in #23911.

This PR adds a notice that the system is shutting down because of the
deprecated setting.
2023-07-17 10:58:47 +02:00
wxiaoguang
9672085d94
Fix "Flash" message usage (#25895)
Resolve https://github.com/go-gitea/gitea/pull/25820/files#r1264309059
2023-07-15 11:52:03 +03:00
KN4CK3R
bd82d8974e
Add support for different Maven POM encoding (#25873)
Fixes #25853

- Maven POM files aren't always UTF-8 encoded.
- Reject the upload of unparsable POM files
2023-07-14 09:39:15 +00:00
harryzcy
c5e187c389
Upgrade go dependencies (#25819) 2023-07-14 11:00:31 +08:00
Jason Song
aee14b9c0b
Remove git.FileBlame (#25841)
The `FileBlame` function looks strange, it has `revision` as argument
but doesn't use it.

Since the function never be used, I think we could just remove it.

If anyone thinks it should be kept, please help fix `revision`.

Co-authored-by: Giteabot <teabot@gitea.io>
2023-07-12 19:07:29 +02:00
wxiaoguang
fa0b5b14c2
Make "install page" respect environment config (#25648)
Replace #25580

Fix #19453

The problem was: when users set "GITEA__XXX__YYY" , the "install page"
doesn't respect it.

So, to make the result consistent and avoid surprising end users, now
the "install page" also writes the environment variables to the config
file.

And, to make things clear, there are enough messages on the UI to tell
users what will happen.

There are some necessary/related changes to `environment-to-ini.go`:

* The "--clear" flag is removed and it was incorrectly written there.
The "clear" operation should be done if INSTALL_LOCK=true
* The "--prefix" flag is removed because it's never used, never
documented and it only causes inconsistent behavior.


![image](https://github.com/go-gitea/gitea/assets/2114189/12778ee4-3fb5-4664-a73a-41ebbd77cd5b)
2023-07-09 22:43:37 +00:00
KN4CK3R
321383db22
Revert package access change from #23879 (#25707)
Fixes (?) #25538
Fixes https://codeberg.org/forgejo/forgejo/issues/972

Regression #23879

#23879 introduced a change which prevents read access to packages if a
user is not a member of an organization.

That PR also contained a change which disallows package access if the
team unit is configured with "no access" for packages. I don't think
this change makes sense (at the moment). It may be relevant for private
orgs. But for public or limited orgs that's useless because an
unauthorized user would have more access rights than the team member.
This PR restores the old behaviour "If a user has read access for an
owner, they can read packages".

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-07-09 13:00:07 +00:00
wxiaoguang
819aed35bf
Make route middleware/handler mockable (#25766)
To mock a handler:

```go
web.RouteMock(web.MockAfterMiddlewares, func(ctx *context.Context) {
	// ...
})
defer web.RouteMockReset()
```


It helps:

* Test the middleware's behavior (assert the ctx.Data, etc)
* Mock the middleware's behavior (prepare some context data for handler)
* Mock the handler's response for some test cases, especially for some
integration tests and e2e tests.
2023-07-09 12:25:53 +00:00
silverwind
887a683af9
Update tool dependencies, lock govulncheck and actionlint (#25655)
- Update all tool dependencies
- Lock `govulncheck` and `actionlint` to their latest tags

---------

Co-authored-by: 6543 <m.huber@kithara.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-07-09 11:58:06 +00:00
Zettat123
a42a838843
Fix ref for workflows triggered by pull_request_target (#25743)
Follow #25229

At present, when the trigger event is `pull_request_target`, the `ref`
and `sha` of `ActionRun` are set according to the base branch of the
pull request. This makes it impossible for us to find the head branch of
the `ActionRun` directly. In this PR, the `ref` and `sha` will always be
set to the head branch and they will be changed to the base branch when
generating the task context.
2023-07-07 19:22:03 +00:00
techknowlogick
cb01b8691d
Add open/closed field support for issue index (#25708)
A couple of notes:
* Future changes should refactor arguments into a struct
* This filtering only is supported by meilisearch right now
* Issue index number is bumped which will cause a re-index
2023-07-07 17:10:13 +00:00
Zettat123
7586b5815a
Fix activity type match in matchPullRequestEvent (#25746)
Fix #25736 
Caused by #24048

Right now we only check the activity type for `pull_request` event when
`types` is specified or there are no `types` and filter. If a workflow
only specifies filters but no `types` like this:
```
on:
  pull_request:
    branches: [main]
```
the workflow will be triggered even if the activity type is not one of
`[opened, reopened, sync]`. We need to check the activity type in this
case.

Co-authored-by: Giteabot <teabot@gitea.io>
2023-07-07 16:30:07 +00:00
6543
8995046110
Less naked returns (#25713)
just a step towards  #25655

and some related refactoring
2023-07-07 05:31:56 +00:00
puni9869
2af30f715e
Fix inconsistent user profile layout across tabs (#25625)
Fix ::User Profile Page  Project Tab Have Inconsistent Layout and Style
Added the big_avator for consistency in the all header_items tabs.
Fixes: #24871
> ### Description
> in the user profile page the `Packages` and `Projects` tab have small
icons for user but other tabs have bigger profile picture with user
info:
> 
> ### Screenshots
> ### **For Packages And Projects:**
>
![image](https://user-images.githubusercontent.com/25511175/240148601-2420d77b-ba25-4718-9ccb-c5d0d95e3079.png)
> 
> ### **For Other Tabs:**
>
![image](https://user-images.githubusercontent.com/25511175/240148461-ce9636b3-fe11-4c46-a230-30d83eee5947.png)
> 

## Before

![image](https://github.com/go-gitea/gitea/assets/80308335/975ad038-07ca-4b10-b75d-ccf259be7b9d)


 ## After changes
Project View 
<img width="1394" alt="image"
src="https://github.com/go-gitea/gitea/assets/80308335/95d181d7-8e61-496d-9899-7b825c91ad56">


Packages View
<img width="1378" alt="image"
src="https://github.com/go-gitea/gitea/assets/80308335/7f5fd60f-6b18-4fa8-8c56-7b0d45d1a610">


## Org view for projects page
<img width="1385" alt="image"
src="https://github.com/go-gitea/gitea/assets/80308335/6400dc89-a5ae-4f0a-831b-5b6efa020d89">

## Org view for packages page
<img width="1387" alt="image"
src="https://github.com/go-gitea/gitea/assets/80308335/4e1e9ffe-1e4b-4334-8657-de11b5fd31d0">

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: silverwind <me@silverwind.io>
2023-07-06 18:59:24 +00:00
wxiaoguang
f0bde0e4f9
Simplify the LFS GC logger usage (#25717)
Remove unnecessary `if opts.Logger != nil` checks.

* For "CLI doctor" mode, output to the console's "logger.Info".
* For "Web Task" mode, output to the default "logger.Debug", to avoid
flooding the server's log in a busy production instance.

Co-authored-by: Giteabot <teabot@gitea.io>
2023-07-06 16:52:41 +00:00
Lunny Xiao
45ac90eb54
Sync branches when mirroring (#25722)
Caused by #22743

---------

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-07-06 13:18:37 +02:00
silverwind
88f835192d
Replace interface{} with any (#25686)
Result of running `perl -p -i -e 's#interface\{\}#any#g' **/*` and `make fmt`.

Basically the same [as golang did](2580d0e08d).
2023-07-04 18:36:08 +00:00
Lunny Xiao
f35ea2b09a
Add elapsed time on debug for slow git commands (#25642)
To record which command is slow, this PR adds a debug log for slow git
operations.

---------

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: delvh <dev.lh@web.de>
2023-07-04 16:22:37 +00:00
Earl Warren
43c9a84ae5
Add unit test for HashAvatar (#25662)
- Add 100% unit test for this function.

---------

Co-authored-by: Gusted <postmaster@gusted.xyz>
2023-07-04 12:43:38 +00:00
Jason Song
9958642502
Fix issues indexer document mapping (#25619)
Fix regression of #5363 (so long ago).

The old code definded a document mapping for `issueIndexerDocType`, and
assigned it to `BleveIndexerData` as its type. (`BleveIndexerData` has
been renamed to `IndexerData` in #25174, but nothing more.) But the old
code never used `BleveIndexerData`, it wrote the index with an anonymous
struct type. Nonetheless, bleve would use the default auto-mapping for
struct it didn't know, so the indexer still worked. This means the
custom document mapping was always dead code.

The custom document mapping is not useless, it can reduce index storage,
this PR brings it back and disable default mapping to prevent it from
happening again. Since `IndexerData`(`BleveIndexerData`) has JSON tags,
and bleve uses them first, so we should use `repo_id` as the field name
instead of `RepoID`.

I did a test to compare the storage size before and after this, with
about 3k real comments that were migrated from some public repos.

Before:

```text
[ 160]  .
├── [  42]  index_meta.json
├── [  13]  rupture_meta.json
└── [ 128]  store
    ├── [6.9M]  00000000005d.zap
    └── [256K]  root.bolt
```

After:

```text
[ 160]  .
├── [  42]  index_meta.json
├── [  13]  rupture_meta.json
└── [ 128]  store
    ├── [3.5M]  000000000065.zap
    └── [256K]  root.bolt
```

It saves about half the storage space.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-07-04 09:05:28 +00:00
KN4CK3R
c890454769
Add direct serving of package content (#25543)
Fixes #24723

Direct serving of content aka HTTP redirect is not mentioned in any of
the package registry specs but lots of official registries do that so it
should be supported by the usual clients.
2023-07-03 15:33:28 +02:00
techknowlogick
469d89b95a
Set SSH_AUTHORIZED_KEYS_BACKUP to false (#25412)
This prevents the disk from overflowing with auth keys file

Fixes #17117

## ⚠️ BREAKING

This changes the default option for creating a backup of the authorized
key file when an update is made to default to false.
2023-07-01 04:07:48 +00:00
Lunny Xiao
65acd1e9ef
Fix branch commit message too long problem (#25588)
When branch's commit CommitMessage is too long, the column maybe too
short.(TEXT 16K for mysql).
This PR will fix it to only store the summary because these message will
only show on branch list or possible future search?
2023-06-30 17:03:05 +08:00
Jason Song
67bd9d4f1e
Restrict [actions].DEFAULT_ACTIONS_URL to only github or self (#25581)
Resolve #24789

## ⚠️ BREAKING ⚠️

Before this, `DEFAULT_ACTIONS_URL` cound be set to any custom URLs like
`https://gitea.com` or `http://your-git-server,https://gitea.com`, and
the default value was `https://gitea.com`.

But now, `DEFAULT_ACTIONS_URL` supports only
`github`(`https://github.com`) or `self`(the root url of current Gitea
instance), and the default value is `github`.

If it has configured with a URL, an error log will be displayed and it
will fallback to `github`.

Actually, what we really want to do is always make it
`https://github.com`, however, this may not be acceptable for some
instances of internal use, so there's extra support for `self`, but no
more, even `https://gitea.com`.

Please note that `uses: https://xxx/yyy/zzz` always works and it does
exactly what it is supposed to do.

Although it's breaking, I belive it should be backported to `v1.20` due
to some security issues.

Follow-up on the runner side:

- https://gitea.com/gitea/act_runner/pulls/262
- https://gitea.com/gitea/act/pulls/70
2023-06-30 07:26:36 +00:00
JakobDev
254a82842a
Add API for changing Avatars (#25369)
This adds an API for uploading and Deleting Avatars for of Users, Repos
and Organisations. I'm not sure, if this should also be added to the
Admin API.

Resolves #25344

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-06-29 23:22:55 +00:00
isla w
e882398c5a
Update emoji set to Unicode 15 (#25595)
Update emoji set to Unicode 15 which was added upstream here:
cb5c514d47

<img width="854" alt="Screenshot 2023-06-29 at 11 02 56 AM"
src="https://github.com/go-gitea/gitea/assets/1669571/7bfb663d-0804-4d23-a62d-f585a6783ca6">

---------

Co-authored-by: silverwind <me@silverwind.io>
2023-06-29 16:29:48 +00:00
Lunny Xiao
6e19484f4d
Sync branches into databases (#22743)
Related #14180
Related #25233 
Related #22639
Close #19786
Related #12763 

This PR will change all the branches retrieve method from reading git
data to read database to reduce git read operations.

- [x] Sync git branches information into database when push git data
- [x] Create a new table `Branch`, merge some columns of `DeletedBranch`
into `Branch` table and drop the table `DeletedBranch`.
- [x] Read `Branch` table when visit `code` -> `branch` page
- [x] Read `Branch` table when list branch names in `code` page dropdown
- [x] Read `Branch` table when list git ref compare page
- [x] Provide a button in admin page to manually sync all branches.
- [x] Sync branches if repository is not empty but database branches are
empty when visiting pages with branches list
- [x] Use `commit_time desc` as the default FindBranch order by to keep
consistent as before and deleted branches will be always at the end.

---------

Co-authored-by: Jason Song <i@wolfogre.com>
2023-06-29 10:03:20 +00:00
Vitaliy Filippov
f0b773e0ce
Support downloading raw task logs (#24451)
Hi!
This pull request adds support for downloading raw task logs for Gitea
Actions, similar to Github Actions
It looks like the following:

![image](https://user-images.githubusercontent.com/945339/235376746-405d5019-710b-468b-8113-9e82eab8e752.png)
2023-06-29 10:58:56 +08:00
a1012112796
4aba8a6a5f
Split lfs size from repository size (#22900)
releated to #21820

- Split `Size` in repository table as two new colunms, one is `GitSize`
for git size, the other is `LFSSize` for lfs data. still store full size
in `Size` colunm.
- Show full size on ui, but show each of them by a `title`;  example:

![image](https://user-images.githubusercontent.com/25342410/218636251-e200f085-d7e7-4a25-9ff1-b586a63e07a9.png)

- Return full size in api response.

---------

Signed-off-by: a1012112796 <1012112796@qq.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: DmitryFrolovTri <23313323+DmitryFrolovTri@users.noreply.github.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-06-28 22:41:02 +00:00
wxiaoguang
b4d0036fc3
Do not prepare oauth2 config if it is not enabled, do not write config in some sub-commands (#25567)
Ref:

* https://github.com/go-gitea/gitea/issues/25377#issuecomment-1609757289

And some sub-commands like "generate" / "docs", they do not need to use
the ini config
2023-06-28 23:30:06 +02:00
Jason Song
6daf21c9b7
Fix content holes in Actions task logs file (#25560)
Fix #25451.

Bugfixes:
- When stopping the zombie or endless tasks, set `LogInStorage` to true
after transferring the file to storage. It was missing, it could write
to a nonexistent file in DBFS because `LogInStorage` was false.
- Always update `ActionTask.Updated` when there's a new state reported
by the runner, even if there's no change. This is to avoid the task
being judged as a zombie task.

Enhancement:
- Support `Stat()` for DBFS file.
- `WriteLogs` refuses to write if it could result in content holes.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-06-28 19:07:29 +02:00
wxiaoguang
65d3e1161b
Fix sub-command log level (#25537)
More fix for #24981

* #24981


Close #22361

* #22361

There were many patches for Gitea's sub-commands to satisfy the facts:

* Some sub-commands shouldn't output any log, otherwise the git protocol
would be broken
* Sometimes the users want to see "verbose" or "quiet" outputs

That's a longstanding problem, and very fragile. This PR is only a quick
patch for the problem.

In the future, the sub-command system should be refactored to a clear
solution.

----

Other changes:

* Use `ReplaceAllWriters` to replace
`RemoveAllWriters().AddWriters(writer)`, then it's an atomic operation.
* Remove unnecessary `syncLevelInternal` calls, because
`AddWriters/addWritersInternal` already calls it.

Co-authored-by: Giteabot <teabot@gitea.io>
2023-06-28 08:02:06 +02:00
Lunny Xiao
d44a415bf0
Add Adopt repository event and handler (#25497)
Fix #14304

---------

Co-authored-by: delvh <dev.lh@web.de>
2023-06-26 06:59:15 +00:00
Zettat123
48e5a74f21
Support pull_request_target event (#25229)
Fix #25088

This PR adds the support for
[`pull_request_target`](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request_target)
workflow trigger. `pull_request_target` is similar to `pull_request`,
but the workflow triggered by the `pull_request_target` event runs in
the context of the base branch of the pull request rather than the head
branch. Since the workflow from the base is considered trusted, it can
access the secrets and doesn't need approvals to run.
2023-06-26 14:33:18 +08:00
wxiaoguang
cc1d61f1f5
Use InitWorkPathAndCfgProvider for environment-to-ini to avoid unnecessary checks (#25480)
Fix #25481

The `InitWorkPathAndCommonConfig` calls `LoadCommonSettings` which does
many checks like "current user is root or not".

Some commands like "environment-to-ini" shouldn't do such check, because
it might be run with "root" user at the moment (eg: the docker's setup
script)

ps: in the future, the docker's setup script should be improved to avoid
Gitea's command running with "root"
2023-06-24 09:13:35 +00:00
Jason Song
3c7f5ed7b5
Fix Init of db indexer (#25470)
Fix regression of #25174.

The `Init` of the db indexer should return true to indicate that the
index was opened/existed, or the indexer will try to populate the index
(not really populate, just fill the queue, `Index` method of the db
indexer is a dummy).
2023-06-23 14:39:52 +00:00
Jason Song
375fd15fbf
Refactor indexer (#25174)
Refactor `modules/indexer` to make it more maintainable. And it can be
easier to support more features. I'm trying to solve some of issue
searching, this is a precursor to making functional changes.

Current supported engines and the index versions:

| engines | issues | code |
| - | - | - |
| db | Just a wrapper for database queries, doesn't need version | - |
| bleve | The version of index is **2** | The version of index is **6**
|
| elasticsearch | The old index has no version, will be treated as
version **0** in this PR | The version of index is **1** |
| meilisearch | The old index has no version, will be treated as version
**0** in this PR | - |


## Changes

### Split

Splited it into mutiple packages

```text
indexer
├── internal
│   ├── bleve
│   ├── db
│   ├── elasticsearch
│   └── meilisearch
├── code
│   ├── bleve
│   ├── elasticsearch
│   └── internal
└── issues
    ├── bleve
    ├── db
    ├── elasticsearch
    ├── internal
    └── meilisearch
```

- `indexer/interanal`: Internal shared package for indexer.
- `indexer/interanal/[engine]`: Internal shared package for each engine
(bleve/db/elasticsearch/meilisearch).
- `indexer/code`: Implementations for code indexer.
- `indexer/code/internal`: Internal shared package for code indexer.
- `indexer/code/[engine]`: Implementation via each engine for code
indexer.
- `indexer/issues`: Implementations for issues indexer.

### Deduplication

- Combine `Init/Ping/Close` for code indexer and issues indexer.
- ~Combine `issues.indexerHolder` and `code.wrappedIndexer` to
`internal.IndexHolder`.~ Remove it, use dummy indexer instead when the
indexer is not ready.
- Duplicate two copies of creating ES clients.
- Duplicate two copies of `indexerID()`.


### Enhancement

- [x] Support index version for elasticsearch issues indexer, the old
index without version will be treated as version 0.
- [x] Fix spell of `elastic_search/ElasticSearch`, it should be
`Elasticsearch`.
- [x] Improve versioning of ES index. We don't need `Aliases`:
- Gitea does't need aliases for "Zero Downtime" because it never delete
old indexes.
- The old code of issues indexer uses the orignal name to create issue
index, so it's tricky to convert it to an alias.
- [x] Support index version for meilisearch issues indexer, the old
index without version will be treated as version 0.
- [x] Do "ping" only when `Ping` has been called, don't ping
periodically and cache the status.
- [x] Support the context parameter whenever possible.
- [x] Fix outdated example config.
- [x] Give up the requeue logic of issues indexer: When indexing fails,
call Ping to check if it was caused by the engine being unavailable, and
only requeue the task if the engine is unavailable.
- It is fragile and tricky, could cause data losing (It did happen when
I was doing some tests for this PR). And it works for ES only.
- Just always requeue the failed task, if it caused by bad data, it's a
bug of Gitea which should be fixed.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-06-23 12:37:56 +00:00
6543
b0215c40cd
Store and use seconds for timeline time comments (#25392)
this will allow us to fully localize it later

PS: we can not migrate back as the old value was a one-way conversion


prepare for  #25213

---
*Sponsored by Kithara Software GmbH*
2023-06-23 12:12:39 +00:00
techknowlogick
c0fc53e226
Import additional secrets via file uri (#25408) 2023-06-23 00:16:12 +00:00
silverwind
d2142ba3c3
Update octicons and use octicon-file-directory-symlink (#25453)
Make use of the [new
octicon](https://github.com/primer/octicons/issues/945) that indicates a
symlink to a directory:

<img width="189" alt="Screenshot 2023-06-22 at 22 50 57"
src="https://github.com/go-gitea/gitea/assets/115237/a70690ea-ebfc-48fe-af23-cdc33bcb2098">
2023-06-22 22:05:52 +00:00
Jason Song
174213530d
Fix Permission in API returned repository struct (#25388)
The old code generates `structs.Repository.Permissions` with only
`access.Permission.AccessMode`, however, it should check the units too,
or the value could be incorrect. For example,
`structs.Repository.Permissions.Push` could be false even the doer has
write access to code unit.

Should fix
https://github.com/renovatebot/renovate/issues/14059#issuecomment-1047961128
(Not reported by it, I just found it when I was looking into this bug)

---

Review tips:

The major changes are
- `modules/structs/repo.go`
https://github.com/go-gitea/gitea/pull/25388/files#diff-870406f6857117f8b03611c43fca0ab9ed6d6e76a2d0069a7c1f17e8fa9092f7
- `services/convert/repository.go`
https://github.com/go-gitea/gitea/pull/25388/files#diff-7736f6d2ae894c9edb7729a80ab89aa183b888a26a811a0c1fdebd18726a7101

And other changes are passive.
2023-06-22 13:08:08 +00:00
John Olheiser
8afc1b1cb5
Move some regexp out of functions (#25430)
/cc @KN4CK3R
https://github.com/go-gitea/gitea/pull/25294#discussion_r1237425343

I also searched the codebase and found a few more.

---------

Signed-off-by: jolheiser <john.olheiser@gmail.com>
2023-06-21 19:57:18 +00:00
wxiaoguang
ce46834b93
Remove "CHARSET" config option for MySQL, always use "utf8mb4" (#25413)
In modern days, there is no reason to make users set "charset" anymore.

Close #25378

## ⚠️ BREAKING

The key `[database].CHARSET` was removed completely as every newer
(>10years) MySQL database supports `utf8mb4` already.
There is a (deliberately) undocumented new fallback option if anyone
still needs to use it, but we don't recommend using it as it simply
causes problems.
2023-06-21 10:49:25 +00:00
Jason Song
59d2189875
Fix missing commit message body when the message has leading newlines (#25418)
Commit with `echo "\nmessage after a blank line\nsecond line of the
message" | git commit --cleanup=verbatim -F -` and push.

<img width="1139" alt="image"
src="https://github.com/go-gitea/gitea/assets/9418365/f9a2c28c-e307-4c78-9e31-3d3ace7b9274">
2023-06-21 09:14:34 +00:00
wxiaoguang
2cdf260f42
Refactor path & config system (#25330)
# The problem

There were many "path tricks":

* By default, Gitea uses its program directory as its work path
* Gitea tries to use the "work path" to guess its "custom path" and
"custom conf (app.ini)"
* Users might want to use other directories as work path
* The non-default work path should be passed to Gitea by GITEA_WORK_DIR
or "--work-path"
* But some Gitea processes are started without these values
    * The "serv" process started by OpenSSH server
    * The CLI sub-commands started by site admin
* The paths are guessed by SetCustomPathAndConf again and again
* The default values of "work path / custom path / custom conf" can be
changed when compiling

# The solution

* Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use
test code to cover its behaviors.
* When Gitea's web server runs, write the WORK_PATH to "app.ini", this
value must be the most correct one, because if this value is not right,
users would find that the web UI doesn't work and then they should be
able to fix it.
* Then all other sub-commands can use the WORK_PATH in app.ini to
initialize their paths.
* By the way, when Gitea starts for git protocol, it shouldn't output
any log, otherwise the git protocol gets broken and client blocks
forever.

The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path
> env var GITEA_WORK_DIR > builtin default

The "app.ini" searching order is: cmd arg --config > cmd arg "work path
/ custom path" > env var "work path / custom path" > builtin default


## ⚠️ BREAKING

If your instance's "work path / custom path / custom conf" doesn't meet
the requirements (eg: work path must be absolute), Gitea will report a
fatal error and exit. You need to set these values according to the
error log.



----

Close #24818
Close #24222
Close #21606
Close #21498
Close #25107
Close #24981
Maybe close #24503

Replace #23301
Replace #22754

And maybe more
2023-06-21 13:50:26 +08:00
wxiaoguang
df5cf5ddbd
Avoid polluting config file when "save" (#25395)
That's a longstanding INI package problem: the "MustXxx" calls change
the option values, and the following "Save" will save a lot of garbage
options into the user's config file.

Ideally we should refactor the INI package to a clear solution, but it's
a huge work.

A clear workaround is what this PR does: when "Save", load a clear INI
instance and save it.

Partially fix #25377, the "install" page needs more fine tunes.
2023-06-21 10:31:40 +08:00
Kyle D
8220e50b56
Substitute variables in path names of template repos too (#25294)
### Summary

Extend the template variable substitution to replace file paths. This
can be helpful for setting up log files & directories that should match
the repository name.

### PR Changes

 - Move files matching glob pattern when setting up repos from template
- For security, added ~escaping~ sanitization for cross-platform support
and to prevent directory traversal (thanks @silverwind for the
reference)
 - Added unit testing for escaping function 
- Fixed the integration tests for repo template generation by passing
the repo_template_id
- Updated the integration testfiles to add some variable substitution &
assert the outputs

I had to fix the existing repo template integration test and extend it
to add a check for variable substitutions.

Example:

![image](https://github.com/go-gitea/gitea/assets/12700993/621feb09-0ef3-460e-afa8-da74cd84fa4e)
2023-06-20 21:14:47 +00:00
wxiaoguang
a1c5057fe8
Batch delete issue and improve tippy opts (#25253)
1. Add "batch delete" button for selected issues, close #22273
2. Address the review in
https://github.com/go-gitea/gitea/pull/25219#discussion_r1229266083
2023-06-19 15:46:50 +08:00
wxiaoguang
a5d0246fff
Avoid polluting the config (#25345)
Caught by #25330

Co-authored-by: Giteabot <teabot@gitea.io>
2023-06-18 16:10:44 +00:00
wxiaoguang
4e2f1ee58d
Refactor web package and context package (#25298)
1. The "web" package shouldn't depends on "modules/context" package,
instead, let each "web context" register themselves to the "web"
package.
2. The old Init/Free doesn't make sense, so simplify it
* The ctx in "Init(ctx)" is never used, and shouldn't be used that way
* The "Free" is never called and shouldn't be called because the SSPI
instance is shared

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-06-18 09:59:09 +02:00
wxiaoguang
b71cb7acdc
Use fetch to send requests to create issues/comments (#25258)
Follow #23290

Network error won't make content lost. And this is a much better
approach than "loading-button".

The UI is not perfect and there are still some TODOs, they can be done
in following PRs, not a must in this PR's scope.

<details>


![image](https://github.com/go-gitea/gitea/assets/2114189/c94ba958-aa46-4747-8ddf-6584deeed25c)

</details>
2023-06-16 06:32:43 +00:00
Lunny Xiao
cc3910d8c3
Fix index generation parallelly failure (#25235)
Fix #22109

---------

Co-authored-by: silverwind <me@silverwind.io>
2023-06-15 02:14:43 +02:00
HesterG
a43ea22479
Change form actions to fetch for submit review box (#25219)
Co-author: @wxiaoguang 

Close #25096 

The way to fix it in this PR is to change form submit to fetch using
formData, and add flags to avoid post repeatedly.
Should be able to apply to more forms that have the same issue after
this PR.

In the demo below, 'approve' is clicked several times, and then
'comment' is clicked several time after 'request changes' clicked.

After:


https://github.com/go-gitea/gitea/assets/17645053/beabeb1d-fe66-4b76-b048-4f022b4e83a0


Update: screenshots from /devtest

>
![image](https://user-images.githubusercontent.com/2114189/245680011-ee4231e0-a53d-4c2a-a9c2-71ccd98005cc.png)
> 
>
![image](https://user-images.githubusercontent.com/2114189/245680057-9215d348-63d8-406d-8828-17e171163aaa.png)
> 
>
![image](https://user-images.githubusercontent.com/2114189/245680148-89d7b3d1-d7b6-442f-b69e-eadaee112482.png)

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-06-14 16:01:37 +08:00
Lunny Xiao
d6dd6d641b
Fix all possible setting error related storages and added some tests (#23911)
Follow up #22405

Fix #20703 

This PR rewrites storage configuration read sequences with some breaks
and tests. It becomes more strict than before and also fixed some
inherit problems.

- Move storage's MinioConfig struct into setting, so after the
configuration loading, the values will be stored into the struct but not
still on some section.
- All storages configurations should be stored on one section,
configuration items cannot be overrided by multiple sections. The
prioioty of configuration is `[attachment]` > `[storage.attachments]` |
`[storage.customized]` > `[storage]` > `default`
- For extra override configuration items, currently are `SERVE_DIRECT`,
`MINIO_BASE_PATH`, `MINIO_BUCKET`, which could be configured in another
section. The prioioty of the override configuration is `[attachment]` >
`[storage.attachments]` > `default`.
- Add more tests for storages configurations.
- Update the storage documentations.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-06-14 11:42:38 +08:00
silverwind
a51b115b0a
Use inline SVG for built-in OAuth providers (#25171)
The plan is that all built-in auth providers use inline SVG for more
flexibility in styling and to get the GitHub icon to follow
`currentcolor`. This only removes the `public/img/auth` directory and
adds the missing svgs to our svg build.

It should map the built-in providers to these SVGs and render them. If
the user has set a Icon URL, it should render that as an `img` tag
instead.

```
gitea-azure-ad
gitea-bitbucket
gitea-discord
gitea-dropbox
gitea-facebook
gitea-gitea
gitea-gitlab
gitea-google
gitea-mastodon
gitea-microsoftonline
gitea-nextcloud
gitea-twitter
gitea-yandex
octicon-mark-github
```

GitHub logo is now white again on dark theme:

<img width="431" alt="Screenshot 2023-06-12 at 21 45 34"
src="https://github.com/go-gitea/gitea/assets/115237/27a43504-d60a-4132-a502-336b25883e4d">

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-06-13 10:51:02 +00:00
yp05327
22a39bb961
Fix profile render when the README.md size is larger than 1024 bytes (#25131)
Fixes https://github.com/go-gitea/gitea/issues/25094

`GetBlobContent` will only get the first 1024 bytes, if the README.md
size is larger than 1024 bytes,
We can not render the rest of them.
After this fix, we should provide the limited size to read when call
`GetBlobContent`.

After:

![image](https://github.com/go-gitea/gitea/assets/18380374/22a42936-4cf8-40b4-a5c7-e384082beb0d)

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-06-13 09:02:25 +00:00
Jonathan Tran
f62cd2f473
Fix task list checkbox toggle to work with YAML front matter (#25184)
Fixes #25160.

`data-source-position` of checkboxes in a task list was incorrect
whenever there was YAML front matter. This would result in issue content
or PR descriptions getting corrupted with random `x` or space characters
when a user checked or unchecked a task.
2023-06-13 14:44:47 +08:00
Lunny Xiao
419804fd4d
Fix compatible for webhook ref type (#25195)
Fix #25185 
Caused by #24634
2023-06-13 06:05:28 +00:00
Lauris BH
9aaaf980f0
Fix open redirect check for more cases (#25143)
If redirect_to parameter has set value starting with `\\example.com`
redirect will be created with header `Location: /\\example.com` that
will redirect to example.com domain.
2023-06-08 14:08:14 +00:00
Denys Konovalov
eac1bddb8d
fix swagger documentation for multiple files API endpoint (#25110)
Fixes some issues with the swagger documentation for the new multiple
files API endpoint (#24887) which were overlooked when submitting the
original PR:

1. add some missing parameter descriptions
2. set correct `required` option for required parameters
3. change endpoint description to match it full functionality (every
kind of file modification is supported, not just creating and updating)
2023-06-07 23:49:58 +08:00
Wim
62ac3251fa
Remove stars when repo goes private (#19904)
Fixes #18600
2023-06-05 13:25:43 +00:00
Lunny Xiao
3d1fda737b
Use git command instead of the ini package to remove the origin remote (#25066) 2023-06-05 18:05:31 +08:00
Denys Konovalov
ca35dec18b
Add ability to set multiple redirect URIs in OAuth application UI (#25072)
OAuth applications can already have multiple redirect URIs if
created/edited over API.

This change allows for setting multiple redirect URIs through the UI as
a comma-separated list (e. g.
`https://example.org/redirect,https://redirect.example.org`)

<details>
<summary>Screenshots</summary>

![Bildschirmfoto vom 2023-06-04
17-14-40](https://github.com/go-gitea/gitea/assets/47871822/2206dc32-e7e4-4953-9ecb-e098890b3f54)
![Bildschirmfoto vom 2023-06-04
17-14-50](https://github.com/go-gitea/gitea/assets/47871822/cd97c73c-9310-44ee-a83a-b927a1ef94da)

</details>

Closes #25068
2023-06-05 15:00:12 +08:00
Jack Hay
18de83b2a3
Redesign Scoped Access Tokens (#24767)
## Changes
- Adds the following high level access scopes, each with `read` and
`write` levels:
    - `activitypub`
    - `admin` (hidden if user is not a site admin)
    - `misc`
    - `notification`
    - `organization`
    - `package`
    - `issue`
    - `repository`
    - `user`
- Adds new middleware function `tokenRequiresScopes()` in addition to
`reqToken()`
  -  `tokenRequiresScopes()` is used for each high-level api section
- _if_ a scoped token is present, checks that the required scope is
included based on the section and HTTP method
  - `reqToken()` is used for individual routes
- checks that required authentication is present (but does not check
scope levels as this will already have been handled by
`tokenRequiresScopes()`
- Adds migration to convert old scoped access tokens to the new set of
scopes
- Updates the user interface for scope selection

### User interface example
<img width="903" alt="Screen Shot 2023-05-31 at 1 56 55 PM"
src="https://github.com/go-gitea/gitea/assets/23248839/654766ec-2143-4f59-9037-3b51600e32f3">
<img width="917" alt="Screen Shot 2023-05-31 at 1 56 43 PM"
src="https://github.com/go-gitea/gitea/assets/23248839/1ad64081-012c-4a73-b393-66b30352654c">

## tokenRequiresScopes  Design Decision
- `tokenRequiresScopes()` was added to more reliably cover api routes.
For an incoming request, this function uses the given scope category
(say `AccessTokenScopeCategoryOrganization`) and the HTTP method (say
`DELETE`) and verifies that any scoped tokens in use include
`delete:organization`.
- `reqToken()` is used to enforce auth for individual routes that
require it. If a scoped token is not present for a request,
`tokenRequiresScopes()` will not return an error

## TODO
- [x] Alphabetize scope categories
- [x] Change 'public repos only' to a radio button (private vs public).
Also expand this to organizations
- [X] Disable token creation if no scopes selected. Alternatively, show
warning
- [x] `reqToken()` is missing from many `POST/DELETE` routes in the api.
`tokenRequiresScopes()` only checks that a given token has the correct
scope, `reqToken()` must be used to check that a token (or some other
auth) is present.
   -  _This should be addressed in this PR_
- [x] The migration should be reviewed very carefully in order to
minimize access changes to existing user tokens.
   - _This should be addressed in this PR_
- [x] Link to api to swagger documentation, clarify what
read/write/delete levels correspond to
- [x] Review cases where more than one scope is needed as this directly
deviates from the api definition.
   - _This should be addressed in this PR_
   - For example: 
   ```go
	m.Group("/users/{username}/orgs", func() {
		m.Get("", reqToken(), org.ListUserOrgs)
		m.Get("/{org}/permissions", reqToken(), org.GetUserOrgsPermissions)
}, tokenRequiresScopes(auth_model.AccessTokenScopeCategoryUser,
auth_model.AccessTokenScopeCategoryOrganization),
context_service.UserAssignmentAPI())
   ```

## Future improvements
- [ ] Add required scopes to swagger documentation
- [ ] Redesign `reqToken()` to be opt-out rather than opt-in
- [ ] Subdivide scopes like `repository`
- [ ] Once a token is created, if it has no scopes, we should display
text instead of an empty bullet point
- [ ] If the 'public repos only' option is selected, should read
categories be selected by default

Closes #24501
Closes #24799

Co-authored-by: Jonathan Tran <jon@allspice.io>
Co-authored-by: Kyle D <kdumontnu@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
2023-06-04 20:57:16 +02:00
wxiaoguang
520eb57d76
Use a separate admin page to show global stats, remove actions stat (#25062)
Before, Gitea shows the database table stats on the `admin dashboard`
page.

It has some problems:

* `count(*)` is quite heavy. If tables have many records, this blocks
loading the admin page blocks for a long time
* Some users had even reported issues that they can't visit their admin
page because this page causes blocking or `50x error (reverse proxy
timeout)`
* The `actions` stat is not useful. The table is simply too large. Does
it really matter if it contains 1,000,000 rows or 9,999,999 rows?
* The translation `admin.dashboard.statistic_info` is difficult to
maintain.

So, this PR uses a separate page to show the stats and removes the
`actions` stat.


![image](https://github.com/go-gitea/gitea/assets/2114189/babf7c61-b93b-4a62-bfaa-22983636427e)

## ⚠️ BREAKING

The `actions` Prometheus metrics collector has been removed for the
reasons mentioned beforehand.
Please do not rely on its output anymore.
2023-06-03 22:03:41 +08:00
wxiaoguang
de4a21fcb4
Refactor INI package (first step) (#25024)
The INI package has many bugs and quirks, and in fact it is
unmaintained.

This PR is the first step for the INI package refactoring: 

* Use Gitea's "config_provider" to provide INI access
* Deprecate the INI package by golangci.yml rule
2023-06-02 17:27:30 +08:00
谈笑风生间
a5acec329e
Rename NotifyPullReviewRequest to NotifyPullRequestReviewRequest (#24988)
after #24481

To keep consistent, maybe pull_request_review_request is better.

## what my pull request does

Rename NotifyPullReviewRequest to NotifyPullRequestReviewRequest.

<img width="1087" alt="image"
src="https://github.com/go-gitea/gitea/assets/75628309/2cf91f02-74f2-42fe-a998-0a039b821d8d">
2023-05-31 15:54:46 +08:00
silverwind
50bd7d0b24
Remove the service worker (#25010)
It's been disabled by default since 1.17
(https://github.com/go-gitea/gitea/pull/18914), and it never really
delivered any benefit except being another cache layer that has its own
unsolved invalidation issues. HTTP cache works, we don't need two cache
layers at the browser for assets.

## ⚠️ BREAKING

You can remove the config `[ui].USE_SERVICE_WORKER` from your `app.ini`
now.
2023-05-31 02:07:04 +00:00
Lunny Xiao
faae819f5d
revert the removed method to fix tmpl break on graph page (#25005)
Fix #24996 
Caused by #24634
2023-05-30 21:36:58 +08:00
wxiaoguang
ca5f302876
Fix admin config page error, use tests to cover the admin config and 500 error page (#24965)
The admin config page has been broken for many many times, a little
refactoring would make this page panic.

So, add a test for it, and add another test to cover the 500 error page.

Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-29 15:00:21 +00:00
Denys Konovalov
275d4b7e3f
API endpoint for changing/creating/deleting multiple files (#24887)
This PR creates an API endpoint for creating/updating/deleting multiple
files in one API call similar to the solution provided by
[GitLab](https://docs.gitlab.com/ee/api/commits.html#create-a-commit-with-multiple-files-and-actions).

To archive this, the CreateOrUpdateRepoFile and DeleteRepoFIle functions
in files service are unified into one function supporting multiple files
and actions.

Resolves #14619
2023-05-29 17:41:35 +08:00
wxiaoguang
84c8ab9fd1
Help to recover from corrupted levelqueue (#24912)
gitea.com experienced the corrupted LevelQueue bug again.

I think the problem is clear now: if the keys in LevelDB went
out-of-sync, the LevelQueue itself doesn't have the ability to recover,
eg:

* LevelQueue.Len() reports 100
* LevelQueue.LPop() reports ErrNotFound = errors.New("no key found")

So it needs to dive into the LevelDB to remove all keys to recover the
corrupted LevelQueue.

More comments are in TestCorruptedLevelQueue.
2023-05-29 10:52:32 +08:00
silverwind
c7612d178c
Remove meta tags theme-color and default-theme (#24960)
As discussed in
https://github.com/go-gitea/gitea/pull/24953#issuecomment-1565630156.

## ⚠️ BREAKING ⚠️

1. The `ui.THEME_COLOR_META_TAG` setting has been removed. If you still
need to set the `theme-color` meta tag, add it via
`$GITEA_CUSTOM/templates/custom/header.tmpl` instead.

2. The non-standard `default-theme` meta-tag added in
https://github.com/go-gitea/gitea/pull/13809 has been removed. Third
party code that needs to obtain the currently loaded theme should use
the `theme-<name>` class on the `<html>` node instead, which reflect the
currently active theme.
2023-05-28 22:33:17 +00:00
wxiaoguang
2f149c5c9d
Use [git.config] for reflog cleaning up (#24958)
Follow
https://github.com/go-gitea/gitea/pull/24860#discussion_r1200589651

Use `[git.config]` for reflog cleaning up, the new options are more
flexible.

*
https://git-scm.com/docs/git-config#Documentation/git-config.txt-corelogAllRefUpdates
*
https://git-scm.com/docs/git-config#Documentation/git-config.txt-gcreflogExpire

## ⚠️ BREAKING

The section `[git.reflog]` is now obsolete and its keys have been moved
to the following replacements:
- `[git.reflog].ENABLED` → `[git.config].core.logAllRefUpdates`
- `[git.reflog].EXPIRATION` → `[git.config].gc.reflogExpire`
2023-05-28 01:07:14 +00:00
wxiaoguang
0d54395fb5
Improve logger Pause handling (#24946)
The old EventWriter's Run does: 

```go
for {
    handlePause()
    select {
    case event <- Queue:
         write the log event ...
    }
}
```

So, if an event writer is started before the logger is paused, there is
a chance that the logger isn't paused for the first message.

The new logic is:

```go
for {
    select {
    case event <- Queue:
         handlePause()
         write the log event ...
    }
}
```

Then the event writer can be correctly paused
2023-05-27 22:35:44 +02:00
wxiaoguang
7314726bab
Do not output "Trace" level logs from process manager by default (#24952)
The old process manager's `Trace` function by default calls `log.Printf`
to output "trace" level logs. That's not ideal because by default the
trace level logs should not be outputted. In history it didn't cause
problems because there was no other call to the process manager before
the logger system's initialization.

But if there is any package using the process manager before the "Trace"
function gets assigned to the logger system's trace function, the
process manager will outputs unexpected verbose messages, this behavior
is not expected in most cases.

Now, the logger system also uses process manager to manage its goroutine
contexts, so it's the time to fix the old "trace" behavior: by default,
do not output the trace level messages. Fix #24951
2023-05-27 10:55:24 +00:00
wxiaoguang
18f26cfbf7
Improve queue and logger context (#24924)
Before there was a "graceful function": RunWithShutdownFns, it's mainly
for some modules which doesn't support context.

The old queue system doesn't work well with context, so the old queues
need it.

After the queue refactoring, the new queue works with context well, so,
use Golang context as much as possible, the `RunWithShutdownFns` could
be removed (replaced by RunWithCancel for context cancel mechanism), the
related code could be simplified.

This PR also fixes some legacy queue-init problems, eg:

* typo : archiver: "unable to create codes indexer queue" => "unable to
create repo-archive queue"
* no nil check for failed queues, which causes unfriendly panic

After this PR, many goroutines could have better display name:

![image](https://github.com/go-gitea/gitea/assets/2114189/701b2a9b-8065-4137-aeaa-0bda2b34604a)

![image](https://github.com/go-gitea/gitea/assets/2114189/f1d5f50f-0534-40f0-b0be-f2c9daa5fe92)
2023-05-26 07:31:55 +00:00
Lunny Xiao
f9cfd6ce5b
Use the type RefName for all the needed places and fix pull mirror sync bugs (#24634)
This PR replaces all string refName as a type `git.RefName` to make the
code more maintainable.

Fix #15367
Replaces #23070 
It also fixed a bug that tags are not sync because `git remote --prune
origin` will not remove local tags if remote removed.

We in fact should use `git fetch --prune --tags origin` but not `git
remote update origin` to do the sync.

Some answer from ChatGPT as ref.

> If the git fetch --prune --tags command is not working as expected,
there could be a few reasons why. Here are a few things to check:
> 
>Make sure that you have the latest version of Git installed on your
system. You can check the version by running git --version in your
terminal. If you have an outdated version, try updating Git and see if
that resolves the issue.
> 
>Check that your Git repository is properly configured to track the
remote repository's tags. You can check this by running git config
--get-all remote.origin.fetch and verifying that it includes
+refs/tags/*:refs/tags/*. If it does not, you can add it by running git
config --add remote.origin.fetch "+refs/tags/*:refs/tags/*".
> 
>Verify that the tags you are trying to prune actually exist on the
remote repository. You can do this by running git ls-remote --tags
origin to list all the tags on the remote repository.
> 
>Check if any local tags have been created that match the names of tags
on the remote repository. If so, these local tags may be preventing the
git fetch --prune --tags command from working properly. You can delete
local tags using the git tag -d command.

---------

Co-authored-by: delvh <dev.lh@web.de>
2023-05-26 01:04:48 +00:00
JakobDev
aaa1094663
Add the ability to pin Issues (#24406)
This adds the ability to pin important Issues and Pull Requests. You can
also move pinned Issues around to change their Position. Resolves #2175.

## Screenshots

![grafik](https://user-images.githubusercontent.com/15185051/235123207-0aa39869-bb48-45c3-abe2-ba1e836046ec.png)

![grafik](https://user-images.githubusercontent.com/15185051/235123297-152a16ea-a857-451d-9a42-61f2cd54dd75.png)

![grafik](https://user-images.githubusercontent.com/15185051/235640782-cbfe25ec-6254-479a-a3de-133e585d7a2d.png)

The Design was mostly copied from the Projects Board.

## Implementation
This uses a new `pin_order` Column in the `issue` table. If the value is
set to 0, the Issue is not pinned. If it's set to a bigger value, the
value is the Position. 1 means it's the first pinned Issue, 2 means it's
the second one etc. This is dived into Issues and Pull requests for each
Repo.

## TODO
- [x] You can currently pin as many Issues as you want. Maybe we should
add a Limit, which is configurable. GitHub uses 3, but I prefer 6, as
this is better for bigger Projects, but I'm open for suggestions.
- [x] Pin and Unpin events need to be added to the Issue history.
- [x] Tests
- [x] Migration

**The feature itself is currently fully working, so tester who may find
weird edge cases are very welcome!**

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-25 15:17:19 +02:00
techknowlogick
033d92997f
Allow skipping forks and mirrors from being indexed (#23187)
This PR adds two new options to disable repo/code search indexing of
both forks and mirrors.

Related: #22842
2023-05-25 16:13:47 +08:00
wxiaoguang
5f39285d6d
Improve RunMode / dev mode (#24886)
1. non-dev mode is treated as prod mode, to protect users from
accidentally running in dev mode if there is a typo in this value.
2. in dev mode, do not need to really exit if there are template errors,
because the template errors could be fixed by developer soon and the
templates get reloaded, help:
* https://github.com/go-gitea/gitea/issues/24845#issuecomment-1557615382
3. Fine tune the mail template loading message.
2023-05-25 03:47:30 +00:00
谈笑风生间
309354c70e
New webhook trigger for receiving Pull Request review requests (#24481)
close https://github.com/go-gitea/gitea/issues/16321

Provided a webhook trigger for requesting someone to review the Pull
Request.

Some modifications have been made to the returned `PullRequestPayload`
based on the GitHub webhook settings, including:
- add a description of the current reviewer object as
`RequestedReviewer` .
- setting the action to either **review_requested** or
**review_request_removed** based on the operation.
- adding the `RequestedReviewers` field to the issues_model.PullRequest.
This field will be loaded into the PullRequest through
`LoadRequestedReviewers()` when `ToAPIPullRequest` is called.

After the Pull Request is merged, I will supplement the relevant
documentation.
2023-05-24 22:06:27 -04:00
wxiaoguang
395bb33e4c
Merge different languages for language stats (#24900)
Fix #24896

If users set different languages by `linguist-language`, the `stats` map
could be: `java: 100, Java: 200`.

Language stats are stored as case-insensitive in database and there is a
unique key.

So, the different language names should be merged to one unique name:
`Java: 300`
2023-05-24 19:37:36 +00:00
wxiaoguang
c21605951b
Make environment-to-ini support loading key value from file (#24832)
Replace #19857

Close #19856
Close #10311
Close #10123

Major changes:

1. Move a lot of code from `environment-to-ini.go` to `config_env.go` to
make them testable.
2. Add `__FILE` support
3. Update documents
4. Add tests
2023-05-24 11:37:22 +08:00
wxiaoguang
8080ace6fc
Support changing git config through app.ini, use diff.algorithm=histogram by default (#24860)
Close #13454 , Close #23255, Close #14697 (and maybe more related
issues)

Many users have the requirement to customize the git config. This PR
introduces an easy way: put the options in Gitea's app.ini
`[git.config]`, then the config options will be applied to git config.

And it can support more flexible default config values, eg: now
`diff.algorithm=histogram` by default. According to:
https://stackoverflow.com/a/32367597/4754037 , `histogram diff` is
efficient and doesn't like to cause server-side problems.

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-23 16:30:19 +00:00
wxiaoguang
910bf31546
Fix flakey test in logger test (#24883)
Fix #24882

The goroutines are all asynchronized. So it needs a little "sleep" to
make sure the writer's goroutine has been paused before sending messages
to it.

Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-23 16:01:20 +00:00
wxiaoguang
16a766cba1
Do not call nil handler for a dummy queue (#24880)
A dummy queue doesn't really have a handler (see line 211), so the
`safeHandler` can safely drop all items
2023-05-23 13:40:55 +02:00
JakobDev
25dc1556cd
Add API for Label templates (#24602)
This adds API that allows getting the Label templates of the Gitea
Instance
2023-05-23 18:10:23 +08:00
wxiaoguang
abcf5a7b5e
Fix install page context, make the install page tests really test (#24858)
Fix #24856

Rename "context.contextKey" to "context.WebContextKey", this context is
for web context only. But the Context itself is not renamed, otherwise
it would cause a lot of changes (if we really want to rename it, there
could be a separate PR).

The old test code doesn't really test, the "install page" gets broken
not only one time, so use new test code to make sure the "install page"
could work.
2023-05-23 09:29:15 +08:00
Brecht Van Lommel
3588edbb08
Add gitea manager reload-templates command (#24843)
This can be useful to update custom templates in production mode, when
they are updated frequently and a full Gitea restart each time is
disruptive.
2023-05-22 23:51:40 +08:00
KN4CK3R
cdb088cec2
Add CRAN package registry (#22331)
This PR adds a [CRAN](https://cran.r-project.org/) package registry.

![grafik](https://user-images.githubusercontent.com/1666336/210450039-d6fa6f77-20cd-4741-89a8-1624def267f7.png)
2023-05-22 10:57:49 +08:00
wxiaoguang
ec2a01d1e2
Fix regression: access log template, gitea manager cli command (#24838)
Close #24836

![image](https://github.com/go-gitea/gitea/assets/2114189/95b025d2-f25f-4246-a08a-fe44ecb787a9)

![image](https://github.com/go-gitea/gitea/assets/2114189/c3afe1fa-2a23-420d-a016-3b67dcd04cd5)
2023-05-22 09:38:38 +08:00
wxiaoguang
2cb66fff60
Support wildcard in email domain allow/block list (#24831)
Replace #20257 (which is stale and incomplete)

Close #20255

Major changes:

* Deprecate the "WHITELIST", use "ALLOWLIST"
* Add wildcard support for EMAIL_DOMAIN_ALLOWLIST/EMAIL_DOMAIN_BLOCKLIST
* Update example config file and document
* Improve tests
2023-05-22 00:05:44 +00:00
wxiaoguang
4647660776
Rewrite logger system (#24726)
## ⚠️ Breaking

The `log.<mode>.<logger>` style config has been dropped. If you used it,
please check the new config manual & app.example.ini to make your
instance output logs as expected.

Although many legacy options still work, it's encouraged to upgrade to
the new options.

The SMTP logger is deleted because SMTP is not suitable to collect logs.

If you have manually configured Gitea log options, please confirm the
logger system works as expected after upgrading.

## Description

Close #12082 and maybe more log-related issues, resolve some related
FIXMEs in old code (which seems unfixable before)

Just like rewriting queue #24505 : make code maintainable, clear legacy
bugs, and add the ability to support more writers (eg: JSON, structured
log)

There is a new document (with examples): `logging-config.en-us.md`

This PR is safer than the queue rewriting, because it's just for
logging, it won't break other logic.

## The old problems

The logging system is quite old and difficult to maintain:
* Unclear concepts: Logger, NamedLogger, MultiChannelledLogger,
SubLogger, EventLogger, WriterLogger etc
* Some code is diffuclt to konw whether it is right:
`log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs
`log.DelLogger("console")`
* The old system heavily depends on ini config system, it's difficult to
create new logger for different purpose, and it's very fragile.
* The "color" trick is difficult to use and read, many colors are
unnecessary, and in the future structured log could help
* It's difficult to add other log formats, eg: JSON format
* The log outputer doesn't have full control of its goroutine, it's
difficult to make outputer have advanced behaviors
* The logs could be lost in some cases: eg: no Fatal error when using
CLI.
* Config options are passed by JSON, which is quite fragile.
* INI package makes the KEY in `[log]` section visible in `[log.sub1]`
and `[log.sub1.subA]`, this behavior is quite fragile and would cause
more unclear problems, and there is no strong requirement to support
`log.<mode>.<logger>` syntax.


## The new design

See `logger.go` for documents.


## Screenshot

<details>


![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff)


![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9)


![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee)

</details>

## TODO

* [x] add some new tests
* [x] fix some tests
* [x] test some sub-commands (manually ....)

---------

Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-21 22:35:11 +00:00
Brecht Van Lommel
268d121f4b
Fix video width overflow in markdown, and other changes to match img (#24834)
This change makes the CSS for `<video>` in markup match that of `<img>`,
and also allows additional attributes to be used. This way the width,
padding, alignment should work equally well for both.
2023-05-21 21:19:37 +00:00
wxiaoguang
6b33152b7d
Decouple the different contexts from each other (#24786)
Replace #16455

Close #21803

Mixing different Gitea contexts together causes some problems:

1. Unable to respond proper content when error occurs, eg: Web should
respond HTML while API should respond JSON
2. Unclear dependency, eg: it's unclear when Context is used in
APIContext, which fields should be initialized, which methods are
necessary.


To make things clear, this PR introduces a Base context, it only
provides basic Req/Resp/Data features.

This PR mainly moves code. There are still many legacy problems and
TODOs in code, leave unrelated changes to future PRs.
2023-05-21 09:50:53 +08:00
silverwind
32d9c47ec7
Add RTL rendering support to Markdown (#24816)
Support RTL content in Markdown:


![image](https://github.com/go-gitea/gitea/assets/115237/dedb1b0c-2f05-40dc-931a-0d9dc81f7c97)

Example document:
https://try.gitea.io/silverwind/symlink-test/src/branch/master/bidi-text.md
Same on GitHub:
https://github.com/silverwind/symlink-test/blob/master/bidi-text.md

`dir=auto` enables a browser heuristic that sets the text direction
automatically. It is the only way to get automatic text direction.

Ref: https://codeberg.org/Codeberg/Community/issues/1021

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-05-20 23:02:52 +02:00
Yarden Shoham
f5ce2ed292
Allow all URL schemes in Markdown links by default (#24805)
- Closes #21146
- Closes #16721

## ⚠️ BREAKING ⚠️
This changes the default behavior to now create links for any URL scheme
when the user uses the markdown form for links (`[label](URL)`), this
doesn't affect the rendering of inline links. To opt-out set the
`markdown.CUSTOM_URL_SCHEMES` setting to a list of allowed schemes, all
other schemes (except `http` and `https`) won't be allowed.

# Before

![image](https://github.com/go-gitea/gitea/assets/20454870/35fa18ce-7dda-4995-b5b3-3f360f38296d)

# After

![image](https://github.com/go-gitea/gitea/assets/20454870/0922216b-0b35-4b77-9919-21a5c21dd5d0)

---------

Signed-off-by: Yarden Shoham <git@yardenshoham.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-19 17:17:07 +02:00
Lunny Xiao
38cf43d060
Some refactors for issues stats (#24793)
This PR

- [x] Move some functions from `issues.go` to `issue_stats.go` and
`issue_label.go`
- [x] Remove duplicated issue options `UserIssueStatsOption` to keep
only one `IssuesOptions`
2023-05-19 22:17:48 +08:00
FuXiaoHei
c757765a9e
Implement actions artifacts (#22738)
Implement action artifacts server api.

This change is used for supporting
https://github.com/actions/upload-artifact and
https://github.com/actions/download-artifact in gitea actions. It can
run sample workflow from doc
https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts.
The api design is inspired by
https://github.com/nektos/act/blob/master/pkg/artifacts/server.go and
includes some changes from gitea internal structs and methods.

Actions artifacts contains two parts:

- Gitea server api and storage (this pr implement basic design without
some complex cases supports)
- Runner communicate with gitea server api (in comming)

Old pr https://github.com/go-gitea/gitea/pull/22345 is outdated after
actions merged. I create new pr from main branch.


![897f7694-3e0f-4f7c-bb4b-9936624ead45](https://user-images.githubusercontent.com/2142787/219382371-eb3cf810-e4e0-456b-a8ff-aecc2b1a1032.jpeg)

Add artifacts list in actions workflow page.
2023-05-19 21:37:57 +08:00
Jason Song
7985cde84d
Fix Actions being enabled accidentally (#24802)
Regression of #24536. If the user doesn't explicitly disable Actions, it
will be enabled.

1. Gitea will call `loadRepositoryFrom` before `loadActionsFrom`.

25d4f95df2/modules/setting/setting.go (L234-L237)
2. In `loadRepositoryFrom`,
`rootCfg.Section("actions").Key("ENABLED").MustBool(true)` will set
`actions.ENABLED` with `true`.

25d4f95df2/modules/setting/repository.go (L313-L315)
3. In `loadActionsFrom`, `rootCfg.Section("actions")` will get a section
with Actions enabled.

25d4f95df2/modules/setting/actions.go (L23-L26)

Although the cause of the problem was using `true` by copy-paste
mistake, it also surprised me that
**`rootCfg.Section("actions").Key("ENABLED").MustBool(true)` doesn't
only read, but also write.**
2023-05-19 19:35:12 +08:00
silverwind
655c890cbd
Fix TestMinioStorageIterator skip message (#24765)
Followup to https://github.com/go-gitea/gitea/pull/24762, fix this
message.
2023-05-17 16:22:17 +00:00
Lunny Xiao
6c8235dc98
Skip TestMinioStorageIterator on CI (#24762)
Fix https://github.com/go-gitea/gitea/pull/24691#issuecomment-1550987681

---------

Co-authored-by: silverwind <me@silverwind.io>
2023-05-17 10:00:58 +00:00
wxiaoguang
584c0789fa
Make mailer SMTP check have timed context (#24751)
Make mailer SMTP check have timed context

Otherwise Gitea may block for long time if the DNS request blocks.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-16 22:55:51 +02:00
zeripath
7565e5c3de
Implement systemd-notify protocol (#21151)
This PR adds support for the systemd notify protocol. Several status
messagess are provided. We should likely add a common notify/status
message for graceful.

Replaces #21140

Signed-off-by: Andrew Thornton <art27@cantab.net>

---------

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: ltdk <usr@ltdk.xyz>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-15 22:20:30 +00:00
wxiaoguang
99283415bc
Refactor Pull Mirror and fix out-of-sync bugs (#24732)
The "mirror" table and "repository" table might be out-of-sync in some
cases.

It means that "IsMirror=true" but "Mirror=nil"

This PR removes unnecessary "Mirror" field, rename "Mirror" to
"PullMirror" and fix nil panic bug.


Screenshot of changed templates:


![image](https://github.com/go-gitea/gitea/assets/2114189/c0f2bdfc-5911-43ea-b989-b19619de4235)



![image](https://github.com/go-gitea/gitea/assets/2114189/1078b41a-484f-4c06-8c2f-edb9e658275d)
2023-05-15 19:02:10 +00:00
KN4CK3R
5968c63a11
Add Go package registry (#24687)
Fixes #7608

This PR adds a Go package registry usable with the Go proxy protocol.

![grafik](https://github.com/go-gitea/gitea/assets/1666336/328feb5c-3df2-4f9d-8eae-fe3126d14c37)
2023-05-14 23:38:40 +08:00
FuXiaoHei
61ad4c607b
fix minio storage iterator path (#24691)
minio storage iterator shows different behavior with local fs iterator.

in local fs storage:

``` go
s.IterateObjects("prefix", func(path,obj)
     println(path) // show "prefix/xxx.file"
})
```

in minio storage:

```go
s.IterateObjects("prefix", func(path,obj)
     println(path) // show "xxx.file"
})
```

I think local fs is correct, minio use wrong `basePath` to trim storage
path prefix.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-13 22:33:25 +00:00
Yarden Shoham
4810fe55e3
Add status indicator on main home screen for each repo (#24638)
It will show the calculated commit status state of the latest commit on
the default branch for each repository in the dashboard repo list

- Closes #15620

# Before

![image](https://github.com/go-gitea/gitea/assets/20454870/aa1326c7-43c0-458a-a798-3102c766bcf9)

# After

![image](https://github.com/go-gitea/gitea/assets/20454870/8658cc03-2224-442a-b1c8-bf64126e4575)

---------

Signed-off-by: Yarden Shoham <git@yardenshoham.com>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-13 21:59:01 +00:00
wxiaoguang
82224c54e0
Improve avatar uploading / resizing / compressing, remove Fomantic card module (#24653)
Fixes: #8972
Fixes: #24263

And I think it also (partially) fix #24263 (no need to convert) ,
because users could upload any supported image format if it isn't larger
than AVATAR_MAX_ORIGIN_SIZE


The main idea: 

* if the uploaded file size is not larger than AVATAR_MAX_ORIGIN_SIZE,
use the origin
* if the resized size is larger than the origin, use the origin

Screenshots:

JPG:

<details>


![image](https://github.com/go-gitea/gitea/assets/2114189/70e98bb0-ecb9-4c4e-a89f-4a37d4e37f8e)

</details>

APNG:

<details>


![image](https://github.com/go-gitea/gitea/assets/2114189/9055135b-5e2d-4152-bd72-596fcb7c6671)


![image](https://github.com/go-gitea/gitea/assets/2114189/50364caf-f7f6-4241-a289-e485fe4cd582)

</details>

WebP (animated)

<details>


![image](https://github.com/go-gitea/gitea/assets/2114189/f642eb85-498a-49a5-86bf-0a7b04089ae0)

</details>

The only exception: if a WebP image is larger than MaxOriginSize and it
is animated, then current `webp` package can't decode it, so only in
this case it isn't supported. IMO no need to support such case: why a
user would upload a 1MB animated webp as avatar? crazy .....

---------

Co-authored-by: silverwind <me@silverwind.io>
2023-05-13 20:59:11 +02:00
wxiaoguang
a94a8d0ab1
Use standard HTTP library to serve files (#24693)
`http.ServeFile/ServeContent` handles `If-xxx`, `Content-Length`,
`Range` and `Etag` correctly

After this PR, storage files (eg: avatar) could be responded with
correct Content-Length.
2023-05-13 16:04:57 +02:00
KN4CK3R
9173e079ae
Add Alpine package registry (#23714)
This PR adds an Alpine package registry. You can follow [this
tutorial](https://wiki.alpinelinux.org/wiki/Creating_an_Alpine_package)
to build a *.apk package for testing.

This functionality is similar to the Debian registry (#22854) and
therefore shares some methods. I marked this PR as blocked because it
should be merged after #22854.


![grafik](https://user-images.githubusercontent.com/1666336/227779595-b76163aa-eea1-4a79-9583-775c24ad74e8.png)

---------

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-12 17:27:50 +00:00
wxiaoguang
bfa0fc2c98
Make testlogger output "TestLogger" prefix (#24683)
Make testlogger output "TestLogger" prefix instead of "Unknown Test"
when there is no running test case.
2023-05-12 14:20:29 +00:00
rune
4b80813341
Support SSH for go get (#24664)
fix #12192 Support SSH for go get

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: mfk <mfk@hengwei.com.cn>
Co-authored-by: silverwind <me@silverwind.io>
2023-05-12 09:44:37 +00:00
wxiaoguang
f6e029e6c7
Make repo migration cancelable and fix various bugs (#24605)
Replace #12917

Close #24601
Close #12845


![image](https://github.com/go-gitea/gitea/assets/2114189/39378118-064d-40fb-8396-4579ebf33917)


![image](https://github.com/go-gitea/gitea/assets/2114189/faf37418-191c-46a6-90a8-353141e00e2d)


![image](https://github.com/go-gitea/gitea/assets/2114189/fdc8ee4d-125f-4737-9990-89bcdf9eb388)


![image](https://github.com/go-gitea/gitea/assets/2114189/9a3bd2c2-fe20-4011-81f0-990ed869d139)

---------

Co-authored-by: Yarden Shoham <git@yardenshoham.com>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-11 08:25:46 +00:00
wxiaoguang
58dfaf3a75
Improve queue & process & stacktrace (#24636)
Although some features are mixed together in this PR, this PR is not
that large, and these features are all related.

Actually there are more than 70 lines are for a toy "test queue", so
this PR is quite simple.

Major features:

1. Allow site admin to clear a queue (remove all items in a queue)
* Because there is no transaction, the "unique queue" could be corrupted
in rare cases, that's unfixable.
* eg: the item is in the "set" but not in the "list", so the item would
never be able to be pushed into the queue.
* Now site admin could simply clear the queue, then everything becomes
correct, the lost items could be re-pushed into queue by future
operations.
3. Split the "admin/monitor" to separate pages
4. Allow to download diagnosis report
* In history, there were many users reporting that Gitea queue gets
stuck, or Gitea's CPU is 100%
    * With diagnosis report, maintainers could know what happens clearly

The diagnosis report sample:
[gitea-diagnosis-20230510-192913.zip](https://github.com/go-gitea/gitea/files/11441346/gitea-diagnosis-20230510-192913.zip)
, use "go tool pprof profile.dat" to view the report.


Screenshots:


![image](https://github.com/go-gitea/gitea/assets/2114189/320659b4-2eda-4def-8dc0-5ea08d578063)


![image](https://github.com/go-gitea/gitea/assets/2114189/c5c46fae-9dc0-44ca-8cd3-57beedc5035e)


![image](https://github.com/go-gitea/gitea/assets/2114189/6168a811-42a1-4e64-a263-0617a6c8c4fe)

---------

Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-11 07:45:47 +00:00
Kyle D
a54c8b4ce9
Kd/fix redis unit test (#24650)
Replaces #24641

Currently, unit tests fail when run locally (unless users have minio
instance running). This PR only requires redis unit tests if in CI.

- Only run redis unit tests when `CI` env variable is set
- Add minio as a service in unit tests actions
2023-05-11 10:09:37 +08:00
wxiaoguang
54f399c4df
Increase default LFS auth timeout from 20m to 24h (#24628)
According to the discussion with DanielGibson, the default "20m" seems
too short. It would make LFS fail if the file is large / network is
slow.

I think relaxing this timeout doesn't have side affect. So change the
default value to 24h, IMO that should be long enough.

## ⚠️ BREAKING

If admins want the previous timeout, they should set the setting
`[server].LFS_HTTP_AUTH_EXPIRY`.
2023-05-10 22:23:47 +08:00
Hester Gong
ea7954f069
Modify luminance calculation and extract related functions into single files (#24586)
Close #24508

Main changes:
As discussed in the issue

1. Change luminance calculation function to use [Relative
Luminance](https://www.w3.org/WAI/GL/wiki/Relative_luminance)
2. Move the luminance related functions into color.go/color.js
3. Add tests for both the files (Not sure if test cases are too many
now)

Before (tests included by `UseLightTextOnBackground` are labels started
with `##`):
https://try.gitea.io/HesterG/testrepo/labels

After:
<img width="1307" alt="Screen Shot 2023-05-08 at 13 37 55"
src="https://user-images.githubusercontent.com/17645053/236742562-fdfc3a4d-2fab-466b-9613-96f2bf96b4bc.png">
<img width="1289" alt="Screen Shot 2023-05-08 at 13 38 06"
src="https://user-images.githubusercontent.com/17645053/236742570-022db68e-cec0-43bb-888a-fc54f5332cc3.png">
<img width="1299" alt="Screen Shot 2023-05-08 at 13 38 20"
src="https://user-images.githubusercontent.com/17645053/236742572-9af1de45-fb7f-460b-828d-ba25fae20f51.png">

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-10 11:19:03 +00:00
Lunny Xiao
c355728a6f
Remove unnecessary code (#24610)
As title, remove unnecessary code.
2023-05-10 04:57:06 +00:00
来自村里的小螃蟹
cd9a13ebb4
Create a branch directly from commit on the create branch API (#22956)
#### Added
- API: Create a branch directly from commit on the create branch API
- Added `old_ref_name` parameter to allow creating a new branch from a
specific commit, tag, or branch.
- Deprecated `old_branch_name` parameter in favor of the new
`old_ref_name` parameter.

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-05-09 18:22:32 +08:00
wxiaoguang
023a048f52
Make repository response support HTTP range request (#24592)
Replace #20480
Replace #18448

Close #16414
2023-05-09 15:34:36 +08:00
wxiaoguang
def4956122
Improve Gitea's web context, decouple "issue template" code into service package (#24590)
1. Remove unused fields/methods in web context.
2. Make callers call target function directly instead of the light
wrapper like "IsUserRepoReaderSpecific"
3. The "issue template" code shouldn't be put in the "modules/context"
package, so move them to the service package.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-09 01:30:14 +02:00
wxiaoguang
6f9c278559
Rewrite queue (#24505)
# ⚠️ Breaking

Many deprecated queue config options are removed (actually, they should
have been removed in 1.18/1.19).

If you see the fatal message when starting Gitea: "Please update your
app.ini to remove deprecated config options", please follow the error
messages to remove these options from your app.ini.

Example:

```
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options
```

Many options in `[queue]` are are dropped, including:
`WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`,
`BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed
from app.ini.

# The problem

The old queue package has some legacy problems:

* complexity: I doubt few people could tell how it works.
* maintainability: Too many channels and mutex/cond are mixed together,
too many different structs/interfaces depends each other.
* stability: due to the complexity & maintainability, sometimes there
are strange bugs and difficult to debug, and some code doesn't have test
(indeed some code is difficult to test because a lot of things are mixed
together).
* general applicability: although it is called "queue", its behavior is
not a well-known queue.
* scalability: it doesn't seem easy to make it work with a cluster
without breaking its behaviors.

It came from some very old code to "avoid breaking", however, its
technical debt is too heavy now. It's a good time to introduce a better
"queue" package.

# The new queue package

It keeps using old config and concept as much as possible.

* It only contains two major kinds of concepts:
    * The "base queue": channel, levelqueue, redis
* They have the same abstraction, the same interface, and they are
tested by the same testing code.
* The "WokerPoolQueue", it uses the "base queue" to provide "worker
pool" function, calls the "handler" to process the data in the base
queue.
* The new code doesn't do "PushBack"
* Think about a queue with many workers, the "PushBack" can't guarantee
the order for re-queued unhandled items, so in new code it just does
"normal push"
* The new code doesn't do "pause/resume"
* The "pause/resume" was designed to handle some handler's failure: eg:
document indexer (elasticsearch) is down
* If a queue is paused for long time, either the producers blocks or the
new items are dropped.
* The new code doesn't do such "pause/resume" trick, it's not a common
queue's behavior and it doesn't help much.
* If there are unhandled items, the "push" function just blocks for a
few seconds and then re-queue them and retry.
* The new code doesn't do "worker booster"
* Gitea's queue's handlers are light functions, the cost is only the
go-routine, so it doesn't make sense to "boost" them.
* The new code only use "max worker number" to limit the concurrent
workers.
* The new "Push" never blocks forever
* Instead of creating more and more blocking goroutines, return an error
is more friendly to the server and to the end user.

There are more details in code comments: eg: the "Flush" problem, the
strange "code.index" hanging problem, the "immediate" queue problem.

Almost ready for review.

TODO:

* [x] add some necessary comments during review
* [x] add some more tests if necessary
* [x] update documents and config options
* [x] test max worker / active worker
* [x] re-run the CI tasks to see whether any test is flaky
* [x] improve the `handleOldLengthConfiguration` to provide more
friendly messages
* [x] fine tune default config values (eg: length?)

## Code coverage:

![image](https://user-images.githubusercontent.com/2114189/236620635-55576955-f95d-4810-b12f-879026a3afdf.png)
2023-05-08 19:49:59 +08:00
wxiaoguang
cb700aedd1
Split "modules/context.go" to separate files (#24569)
The "modules/context.go" is too large to maintain.

This PR splits it to separate files, eg: context_request.go,
context_response.go, context_serve.go

This PR will help:

1. The future refactoring for Gitea's web context (eg: simplify the context)
2. Introduce proper "range request" support
3. Introduce context function

This PR only moves code, doesn't change any logic.
2023-05-08 17:36:54 +08:00
Matthew Walowski
ff5629268c
Pass 'not' to commit count (#24473)
Due to #24409 , we can now specify '--not' when getting all commits from
a repo to exclude commits from a different branch.

When I wrote that PR, I forgot to also update the code that counts the
number of commits in the repo. So now, if the --not option is used, it
may return too many commits, which can indicate that another page of
data is available when it is not.

This PR passes --not to the commands that count the number of commits in
a repo
2023-05-08 07:10:53 +00:00
wxiaoguang
0bb52883eb
Improve decryption failure message (#24573)
Help some users like #16832 #1851

There are many users reporting similar problem: if the SECRET_KEY
mismatches, some operations (like 2FA login) only reports unclear 500
error and unclear "base64 decode error" log (some maintainers ever spent
a lot of time on debugging such problem)

The SECRET_KEY was not well-designed and it is also a kind of technical
debt. Since it couldn't be fixed easily, it's good to add clearer error
messages, then at least users could know what the real problem is.

---------

Co-authored-by: delvh <dev.lh@web.de>
2023-05-07 19:29:43 +08:00
wxiaoguang
56ae853ca0
Simplify template helper functions (#24570)
To avoid bloating the template helper functions, some functions could be
provided by type methods.

And the new code `data-line-type="{{.GetHTMLDiffLineType}}"` reads
better than `data-line-type="{{DiffLineTypeToStr .GetType}}"`


After the fix, screenshots (the same as before):

<details>


![image](https://user-images.githubusercontent.com/2114189/236657918-20ce01e0-1192-443e-aeb4-6b3fe1aa2102.png)


![image](https://user-images.githubusercontent.com/2114189/236657950-ee19727f-a1fc-4133-afc7-e5d1a8c1783f.png)

</details>
2023-05-07 09:49:46 +00:00
techknowlogick
4daf40505a
Sort users and orgs on explore by recency by default (#24279)
This gives more "freshness" to the explore page. So it's not just the
same X users on the explore page by default, now it matches the same
sort as the repos on the explore page.

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-05-06 22:04:55 +08:00
KN4CK3R
05209f0d1d
Add RPM registry (#23380)
Fixes #20751

This PR adds a RPM package registry. You can follow [this
tutorial](https://opensource.com/article/18/9/how-build-rpm-packages) to
build a *.rpm package for testing.

This functionality is similar to the Debian registry (#22854) and
therefore shares some methods. I marked this PR as blocked because it
should be merged after #22854.


![grafik](https://user-images.githubusercontent.com/1666336/223806549-d8784fd9-9d79-46a2-9ae2-f038594f636a.png)
2023-05-05 20:33:37 +00:00
Jason Song
ea1afb945d
Replace placeholders in licenses (#24354)
Replace #22117. Implement it in a more maintainable way.

Some licenses have placeholders e.g. the BSD licenses start with this
line:
```
Copyright (c) <year> <owner>. 
```
This PR replaces the placeholders with the correct value when initialize
a new repo.

### FAQ

- Why not use a regex?
It will be a pretty complicated regex which could be hard to maintain.

- There're still missing placeholders.
There are over 500 licenses, it's impossible for anyone to inspect all
of them alone. Please help to add them if you find any, and it is also
OK to leave them for the future.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-05 13:46:17 +00:00
sillyguodong
a866cb0cb9
Fix the permission of team's Actions unit issue (#24536)
close #24449

The unit of `Actions` should be contorlled not only by
`repository.DISABLED_REPO_UNITS` but also by `actions.ENABLED`
in the `app.ini`.
Previously, the permission of the team's `Actions` unit was not
controlled by `actions.Enabled`. So, even if the user sets
`actions.Enabled` to false, he can still select the permission of the
`Actions` unit for the team.

This PR makes the permissions of the team's `Actions` unit also
controlled by `actions.Enabled`. Just append`TypeActions` into
`DisabledRepoUnits` slice when initializing if `actions.Enabled` is
false.

### Changes:

If `Actions` is set disbaled in `app.ini`, like below:
```yaml
[actions]
ENABLED = false
```

1. If user try to create/edit a team, will prompt user that `Actions` is disabled.

![image](https://user-images.githubusercontent.com/33891828/236370415-961082b2-82d2-4d9e-8025-83872ad08cbb.png)

2. `actions` is not displayed in the sidebar on the team details page

![image](https://user-images.githubusercontent.com/33891828/236371817-f39f9bc9-5926-4b88-b5e6-d93617fcfb07.png)
2023-05-05 20:02:30 +08:00
wxiaoguang
5d77691d42
Improve template system and panic recovery (#24461)
Partially for #24457

Major changes:

1. The old `signedUserNameStringPointerKey` is quite hacky, use
`ctx.Data[SignedUser]` instead
2. Move duplicate code from `Contexter` to `CommonTemplateContextData`
3. Remove incorrect copying&pasting code `ctx.Data["Err_Password"] =
true` in API handlers
4. Use one unique `RenderPanicErrorPage` for panic error page rendering
5. Move `stripSlashesMiddleware` to be the first middleware
6. Install global panic recovery handler, it works for both `install`
and `web`
7. Make `500.tmpl` only depend minimal template functions/variables,
avoid triggering new panics

Screenshot:

<details>

![image](https://user-images.githubusercontent.com/2114189/235444895-cecbabb8-e7dc-4360-a31c-b982d11946a7.png)

</details>
2023-05-04 14:36:34 +08:00
oliverpool
75ea0d5dba
Faster git.GetDivergingCommits (#24482)
Using `git rev-list --left-right` is almost 2x faster than calling `git
rev-list` twice.

Co-authored-by: silverwind <me@silverwind.io>
2023-05-04 05:08:41 +00:00
Lunny Xiao
377a0a20f0
Merge setting.InitXXX into one function with options (#24389)
This PR will merge 3 Init functions on setting packages as 1 and
introduce an options struct.
2023-05-04 11:55:35 +08:00
zeripath
ad8631c069
Fix intermittent CI failure in EmptyQueue (#23753)
The ordering of the final token causing a close of the queue in this
test may be out of sync due to concurrency. Instead just use ensure that
the queue is closed when everything expected is done.

Fixes: https://github.com/go-gitea/gitea/issues/23608
Fixes: https://github.com/go-gitea/gitea/issues/23977

---------

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-05-03 21:37:30 -04:00
yp05327
dbb3736785
Fix incorrect webhook time and use relative-time to display it (#24477)
Fixes #24414
After click replay this webhook, it will display `now`

![image](https://user-images.githubusercontent.com/18380374/235559399-05a23927-13f5-442d-8f10-2c7cd24022a0.png)

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-03 19:53:43 -04:00
KN4CK3R
bf999e4069
Add Debian package registry (#24426)
Co-authored-by: @awkwardbunny

This PR adds a Debian package registry.
You can follow [this
tutorial](https://www.baeldung.com/linux/create-debian-package) to build
a *.deb package for testing.
Source packages are not supported at the moment and I did not find
documentation of the architecture "all" and how these packages should be
treated.


![grafik](https://user-images.githubusercontent.com/1666336/218126879-eb80a866-775c-4c8e-8529-5797203a64e6.png)

Part of #20751.

Revised copy of #22854.

---------

Co-authored-by: Brian Hong <brian@hongs.me>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-02 12:31:35 -04:00
silverwind
d6f8238492
Replace N/A with - everywhere (#24474)
Followup to https://github.com/go-gitea/gitea/pull/24427.

Reasoning is that `N/A` is specific to english while `-` is
language-neutral and does not need translation.

Before:
<img width="891" alt="Screenshot 2023-05-01 at 20 58 20"
src="https://user-images.githubusercontent.com/115237/235511592-8a36d0f2-34ff-4dbe-b642-67c0ade644fe.png">

After:
<img width="901" alt="Screenshot 2023-05-01 at 20 59 59"
src="https://user-images.githubusercontent.com/115237/235511594-d49f6d09-92e8-4e99-be7b-2a37f5d24129.png">
2023-05-02 05:54:29 -04:00
wxiaoguang
2a56666fd2
Fix incorrect CurrentUser check for docker rootless (#24441)
The IsRunUserMatchCurrentUser logic is fragile, the "SSH" config is not
ready when it executes.
2023-04-30 14:14:57 -04:00
wxiaoguang
e3750370df
Use globally shared HTMLRender (#24436)
The old `HTMLRender` is not ideal.

1. It shouldn't be initialized multiple times, it consumes a lot of
memory and is slow.
2. It shouldn't depend on short-lived requests, the `WatchLocalChanges`
needs a long-running context.
3. It doesn't make sense to use FuncsMap slice.


HTMLRender was designed to only work for GItea's specialized 400+
templates, so it's good to make it a global shared instance.
2023-04-30 08:22:23 -04:00
Yarden Shoham
cc84c58aff
Remove unused setting time.FORMAT (#24430)
It's loaded and then never used.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-04-29 22:51:43 +02:00
Matthew Walowski
f766b00293
Add ability to specify '--not' from GetAllCommits (#24409)
For my specific use case, I'd like to get all commits that are on one
branch but NOT on the other branch.

For instance, I'd like to get all the commits on `Branch1` that are not
also on `master` (I.e. all commits that were made after `Branch1` was
created).

This PR adds a `not` query param that gets passed down to the `git log`
command to allow the user to exclude items from `GetAllCommits`.

See [git
documentation](https://git-scm.com/docs/git-log#Documentation/git-log.txt---not)

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-04-29 08:34:14 -04:00
wxiaoguang
241b74f6c5
Improve template helper (#24417)
It seems that we really need the "context function" soon. So we should
clean up the helper functions first.

Major changes:

* Improve StringUtils and add JsonUtils
* Remove one-time-use helper functions like CompareLink
* Move other code (no change) to util_avatar/util_render/util_misc (no
need to propose changes for them)

I have tested the changed templates:


![image](https://user-images.githubusercontent.com/2114189/235283862-608dbf6b-2da3-4d06-8157-b523ca93edb4.png)


![image](https://user-images.githubusercontent.com/2114189/235283888-1dfc0471-e622-4d64-9d76-7859819580d3.png)


![image](https://user-images.githubusercontent.com/2114189/235283903-d559f14d-4abb-4a50-915f-2b9cbc381a7a.png)


![image](https://user-images.githubusercontent.com/2114189/235283955-b7b5adea-aca3-4758-b38a-3aae3f7c6048.png)

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-04-29 08:02:29 -04:00