## Overview
Hi all, I'm a first-time contributor to Forgejo. I was looking for something interesting to contribute and the first thing that caught my attention was https://codeberg.org/forgejo/forgejo/issues/6043, a request for an enhancement to include "issue previews" when publishing links to social media platforms. As a bit of background, the way these platforms work is that they search for meta tags in the posted link's content, and if they find a meta `og:image` (along with other meta tags) they'll pull the image to include in the social media post. Forgejo currently provides an `og:image` tag but it just renders the repository or repository-owner's avatar.
This PR will render `og:image` for an issue or PR into a link to `{...}/summary-card`, which is a dynamically generated image that contains a summary of the issue.
## Design Notes
### Rendering / Rasterization
The tricky part of solving this problem is rendering an image that combines some text, some images, and some layout elements. To address this, I've created a `card` module which allows for a handful of operations:
- Create a new rendered image (a "Card")
- Add a margin to a card
- Split the card, horizontally or vertically, into two pieces with a proportional layout (eg. 70%/30%, as desired), each of which are "Cards" that render into the same root image
- Render text into a card, with line-wrapping and text-alignment capabilities
- Render an image onto a card
- Fetches an external image as safely as possible (for server-side fetch of Gravatar, etc.)
The card module can be reused to create `og:image` summary cards for any object in the future, although obviously it's capabilities are limited. The current implementation is on issues/PRs.
I considered a few alternative approaches before taking this approach, and here's why I rejected those options:
- Provide the summary card as an SVG object which could be rendered much more easily with a template file -- however, support for SVG isn't defined as positive for OpenGraph, and a quick look through some existing implementations suggest that it is not widely supported, if at all
- Rendering as HTML/CSS, or SVG, and then using an external tool to convert into a PNG (or other static) image -- this would be much nicer and easier to implement, but would require tying in some very heavy-weight dependencies
- Rendering using a more sophisticated graphics library, eg. cairo -- also would be nicer and easier to implement, but again a heavy dependency for a small functionality
As a result of the limited capabilities of the new card module, summary cards don't have icons on them (which would require SVG rasterization) or pretty status badges with colors and rounded rects. In the future if better drawing capabilities were added, the graphics could be improved, but it doesn't seem too important.
### External Avatars
In order to rasterize a user's avatar onto the summary card, it might have to be retrieved by the server from the external source (eg. Gravatar). A `fetchExternalImage` routine attempts to do this in the safest way possible to protect the server from any possible security exposure from this; (a) verifying that the content-types are acceptable, (b) ensuring that the file-size and image-size are within the safe bounds that are used for custom avatars, (c) using a very-short timeout to avoid stalling the server if an external dependency is offline.
### Caching
Summary cards are cached after rendered. This has the downside of causing updates to statuses, avatars, titles, etc. being stale on the summary card for the cache TTL. However, during testing I found that some social media engines like Mastodon will cause the summary card to be accessed a significant number of times after being referenced by a post, causing a mini-tornado of requests. The cache compensates for this to avoid server load in this situation.
### Scope
I'm considering out-of-scope:
- Summary cards on other objects (eg. repos, users) can be left for future implementation
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [x] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- ~~I added test coverage for JavaScript changes...~~ n/a, no JS changes
- [x] ~~in `web_src/js/*.test.js` if it can be unit tested.~~
- [x] ~~in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).~~
- Manual testing
- [x] Access & attach screenshots of both an issue and a pull-request's summary card; see below
- [x] Ensure reasonable (non-crash) behavior of rendering text with glyphs outside the font -- correctly rendered as replacement unicode chars
- [x] Using a public test instance, verify that og:image behavior looks good on platforms like Mastodon and BlueSky
- [x] Bluesky: ✅
- [x] Mastodon: ✅ (Note that the summary card will be requested many times as the post is federated; either each server, or each client, will fetch it itself)
- [x] OpenGraph test site (https://www.opengraph.xyz/): ✅
- [x] Discord: Looks OK ✅; needs "twitter:card" to be set to "summary_large_image" to display the large-scale image, but (a) that's probably annoying to use, (b) probably wrong because it doesn't match Twitter Card's spec for a "photographic image", and (c) don't want to encourage/continue use of vendor-specific tag
- [x] Verify cases with user avatar missing (or autogen), and repo avatar missing (falls back to repo owner avatar)
Pull request summary card:
Issue summary card:
(images to the right are the custom repo avatar, w/ fallback to the repo owner avatar)
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
- OpenGraph capabilities are expected to work in the background without user awareness, and so there is no need for documentation to explain the capabilities for users.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/6053
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Mathieu Fenniak <mathieu@fenniak.net>
Co-committed-by: Mathieu Fenniak <mathieu@fenniak.net>
- In the case that Enry correctly recognized the language, but returns
the language name in a way that isn't recognized by enry. Although
overkill I've added a map such that new entries should be easier to add.
- Resolves#6077
- Added unit test
- Currently the TOTP secrets are stored using the `secrets` module with
as key the MD5 hash of the Secretkey, the `secrets` module uses general
bad practices. This patch migrates the secrets to use the `keying`
module (#5041) which is easier to use and use better practices to store
secrets in databases.
- Migration test added.
- Remove the Forgejo migration databases, and let the gitea migration
databases also run forgejo migration databases. This is required as the
Forgejo migration is now also touching tables that the forgejo migration
didn't create itself.
When running e2e tests on flaky networks, gravatar can cause a timeout
and test failures. Turn off, and populate avatars on e2e test suite run
to make them reliable.
(cherry picked from commit 9ac74a1a408136455a9e0586fb8e65163048597b)
Conflicts:
models/fixtures/user.yml
services/repository/contributors_graph_test.go
trivial context conflicts
(cherry picked from commit 33850a83fe4ebd23a762a7aac81614c42e303bfa)
This really is just the cherry pick of 407b6e6dfc7ee9ebb8a16c7f1a786e4c24d0516e
which is the first commit of the pull request, the one with the
change. The rest of the changes is a refactor that is unrelated to the
bug fix.
Conflicts:
modules/git/commit_test.go
trivial context conflict
Extend API MarkupOptions to contain branch path.
The `api.MarkupOptions{}` to have `BranchPath` which contains the
current branch. The `RenderMarkup` function utilizes a struct since there
are too many variables passed as arguments and that is not a good sign
for readability.
And `repo-editor.js` will contain a new form data which is `branch-path`
which will then be utilized by `edit.tmpl` as `data-branch-path`.
Closes: #4510
## Checklist
The [contributor guide](https://forgejo.org/docs/next/contributor/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [ ] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [x] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [ ] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/5781
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Nirmal Kumar R <tildezero@gmail.com>
Co-committed-by: Nirmal Kumar R <tildezero@gmail.com>
- Add a new setting `EMAIL_DOMAIN_BLOCK_DISPOSABLE` that will append a list of
domains that are known for being used by temporary or disposable email
services.
- Add a utility to automatically download and format the list of domains from
the disposable-email-domains project on github.
(https://github.com/disposable-email-domains/disposable-email-domains)
license: CC0 1.0 Universal (CC0 1.0) [Public Domain]
from README:
"""
This repo contains a list of disposable and temporary email address domains often used to register dummy users in order to spam or abuse some services.
We cannot guarantee all of these can still be considered disposable but we do basic checking so chances are they were disposable at one point in time.
"""
It is possible to set a Email for a Organization. This Email is optional and only used to be displayed on the profile page. However, once you set an EMail, you can no longer remove it. This PR fixes that.
While working on the tests, I found out, that the API returns a 500 when trying to set an invalid EMail. I fixed that too. It returns a 422 now.
Fixes#4567
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/5517
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Reviewed-by: Otto <otto@codeberg.org>
Co-authored-by: JakobDev <jakobdev@gmx.de>
Co-committed-by: JakobDev <jakobdev@gmx.de>
Git has a cool feature called git notes. It allows adding a text to a commit without changing the commit itself. Forgejo already displays git notes. With this PR you can also now change git notes.
<details>
<summary>Screenshots</summary>
</details>
## Checklist
The [developer guide](https://forgejo.org/docs/next/developer/) contains information that will be helpful to first time contributors. There also are a few [conditions for merging Pull Requests in Forgejo repositories](https://codeberg.org/forgejo/governance/src/branch/main/PullRequestsAgreement.md). You are also welcome to join the [Forgejo development chatroom](https://matrix.to/#/#forgejo-development:matrix.org).
### Tests
- I added test coverage for Go changes...
- [ ] in their respective `*_test.go` for unit tests.
- [x] in the `tests/integration` directory if it involves interactions with a live Forgejo server.
- I added test coverage for JavaScript changes...
- [ ] in `web_src/js/*.test.js` if it can be unit tested.
- [ ] in `tests/e2e/*.test.e2e.js` if it requires interactions with a live Forgejo server (see also the [developer guide for JavaScript testing](https://codeberg.org/forgejo/forgejo/src/branch/forgejo/tests/e2e/README.md#end-to-end-tests)).
### Documentation
- [ ] I created a pull request [to the documentation](https://codeberg.org/forgejo/docs) to explain to Forgejo users how to use this change.
- [x] I did not document these changes and I do not expect someone else to do it.
### Release notes
- [ ] I do not want this change to show in the release notes.
- [x] I want the title to show in the release notes with a link to this pull request.
- [ ] I want the content of the `release-notes/<pull request number>.md` to be be used for the release notes instead of the title.
<!--start release-notes-assistant-->
## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
- [PR](https://codeberg.org/forgejo/forgejo/pulls/4753): <!--number 4753 --><!--line 0 --><!--description QWxsb3cgY2hhbmdpbmcgZ2l0IG5vdGVz-->Allow changing git notes<!--description-->
<!--end release-notes-assistant-->
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4753
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: JakobDev <jakobdev@gmx.de>
Co-committed-by: JakobDev <jakobdev@gmx.de>
- Instead of having code that relied on the result being sorted (which
wasn't specified in the query and therefore not safe to assume so). Use
a map where it doesn't care if the result that we get from the database
is sorted or not.
- Added unit test.
- _Simply_ add `^$` to regexp that didn't had it yet, this avoids any
content being allowed that simply had the allowed content as a
substring.
- Fix file-preview regex to have `$` instead of `*`.
- Add a `purpose` column, this allows the `forgejo_auth_token` table to
be used by other parts of Forgejo, while still enjoying the
no-compromise architecture.
- Remove the 'roll your own crypto' time limited code functions and
migrate them to the `forgejo_auth_token` table. This migration ensures
generated codes can only be used for their purpose and ensure they are
invalidated after their usage by deleting it from the database, this
also should help making auditing of the security code easier, as we're
no longer trying to stuff a lot of data into a HMAC construction.
-Helper functions are rewritten to ensure a safe-by-design approach to
these tokens.
- Add the `forgejo_auth_token` to dbconsistency doctor and add it to the
`deleteUser` function.
- TODO: Add cron job to delete expired authorization tokens.
- Unit and integration tests added.
- When a dependency is renamed, specified via `package="actual-name"` in
Cargo.toml, this should become the name of the depedency when the
package is retrieved from the registery by cargo and the old name should
be available in the `package` field.
- The reference implementation also does this: 490e66a9d6/src/controllers/krate/publish.rs (L702-L705)
- Resolves#5936
- Unit test added.
Some preparations are only used by a few tests, so to make the tests fast, they should only be prepared when they are used.
By the way, this PR splits PrepareTestEnv into small functions to make it simple.
---
Conflict resolution: Mostly magical and just re-pasting the code into
the right places.
Done differently: use `require.NoError` instead of `assert.NoError`.
(cherry picked from commit ec2d1593c269e06655525deb96f74b8094221b6f)
- Use the forked [binding](https://code.forgejo.org/go-chi/binding)
library. This library has two benefits, it removes the usage of
`github.com/goccy/go-json` (has no benefit as the minimo library is also
using it). It adds the `TrimSpace` feature, which will during the
binding part trim the spaces around the value it got from the form, this
is done before validation.
Closes https://github.com/go-gitea/gitea/issues/30296
- Adds a DB fixture for actions artifacts
- Adds artifacts test files
- Clears artifacts test files between each run
- Note: I initially initialized the artifacts only for artifacts tests,
but because the files are small it only takes ~8ms, so I changed it to
always run in test setup for simplicity
- Fix some otherwise flaky tests by making them not depend on previous
tests
(cherry picked from commit 66971e591e5dddd5b6dc1572ac48f4e4ab29b8e0)
Conflicts:
- tests/integration/api_actions_artifact_test.go
Conflict resolved by manually changing the tested artifact
name from "artifact" to "artifact-download"
- tests/integration/api_actions_artifact_v4_test.go
Conflict resolved by manually updating the tested artifact
names, and adjusting the test case only present in our tree.
- tests/test_utils.go
Resolved by manually copying the added function.
Fix#28121
I did some tests and found that the `missing signature key` error is
caused by an incorrect `Content-Type` header. Gitea correctly sets the
`Content-Type` header when serving files.
348d1d0f32/routers/api/packages/container/container.go (L712-L717)
However, when `SERVE_DIRECT` is enabled, the `Content-Type` header may
be set to an incorrect value by the storage service. To fix this issue,
we can use query parameters to override response header values.
https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html
<img width="600px"
src="https://github.com/user-attachments/assets/f2ff90f0-f1df-46f9-9680-b8120222c555"
/>
In this PR, I introduced a new parameter to the `URL` method to support
additional parameters.
```
URL(path, name string, reqParams url.Values) (*url.URL, error)
```
---
Most S3-like services support specifying the content type when storing
objects. However, Gitea always use `application/octet-stream`.
Therefore, I believe we also need to improve the `Save` method to
support storing objects with the correct content type.
b7fb20e73e/modules/storage/minio.go (L214-L221)
(cherry picked from commit 0690cb076bf63f71988a709f62a9c04660b51a4f)
Conflicts:
- modules/storage/azureblob.go
Dropped the change, as we do not support Azure blob storage.
- modules/storage/helper.go
Resolved by adjusting their `discardStorage` to our
`DiscardStorage`
- routers/api/actions/artifacts.go
routers/api/actions/artifactsv4.go
routers/web/repo/actions/view.go
routers/web/repo/download.go
Resolved the conflicts by manually adding the new `nil`
parameter to the `storage.Attachments.URL()` calls.
Originally conflicted due to differences in the if expression
above these calls.
(cherry picked from commit f4d3aaeeb9e1b11c5495e4608a3f52f316c35758)
Conflicts:
- modules/charset/charset_test.go
Resolved by manually changing a `=` to `:=`, as per the
original patch. Conflict was due to `require.NoError`.
This contains two backwards-compatible changes:
* in the lfs http_client, the number of lfs oids requested per batch is
loaded from lfs_client#BATCH_SIZE and defaulted to the previous value of
20
* in the lfs server/service, the max number of lfs oids allowed in a
batch api request is loaded from server#LFS_MAX_BATCH_SIZE and defaults
to 'nil' which equates to the previous behavior of 'infinite'
This fixes#32306
---------
Signed-off-by: Royce Remer <royceremer@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
(cherry picked from commit c60e4dc1095ef90a790582cacfad27c972637bb2)
Conflicts:
- services/lfs/server.go
Conflict due to our Quota implementation. Resolved by manually
adding the change after the quota check.
When a workflow has
on:
pull_request:
types:
- labeled
- unlabeled
The payload misses the label field describing the added or removed
label.
The unlabeled event type was also incorrectly mapped to the labeled
event type.
Follow https://github.com/go-gitea/gitea/pull/32383
This PR cleans up the "Deadline" usages in templates, make them call
`ParseLegacy` first to get a `Time` struct then display by `DateUtils`.
Now it should be pretty clear how "deadline string" works, it makes it
possible to do further refactoring and correcting.
(cherry picked from commit 259811617ba15c77ddd89360178a59251d611af2)
---
Conflict resolution: Trivial, for `repo_attributes.go` move where the
`IsErrCanceledOrKilled` needs to happen because of other changes that
happened in this file.
To add some words to this change: It seems to be mostly simplifying the
error handling of git operations.
(cherry picked from commit e524f63d58900557d7d57fc3bcd19d9facc8b8ee)
