Commit graph

20538 commits

Author SHA1 Message Date
Jason Song
a627b885c7
Support compression for Actions logs (#31761)
Support compression for Actions logs to save storage space and
bandwidth. Inspired by
https://github.com/go-gitea/gitea/issues/24256#issuecomment-1521153015

The biggest challenge is that the compression format should support
[seekable](https://github.com/facebook/zstd/blob/dev/contrib/seekable_format/zstd_seekable_compression_format.md).
So when users are viewing a part of the log lines, Gitea doesn't need to
download the whole compressed file and decompress it.

That means gzip cannot help here. And I did research, there aren't too
many choices, like bgzip and xz, but I think zstd is the most popular
one. It has an implementation in Golang with
[zstd](https://github.com/klauspost/compress/tree/master/zstd) and
[zstd-seekable-format-go](https://github.com/SaveTheRbtz/zstd-seekable-format-go),
and what is better is that it has good compatibility: a seekable format
zstd file can be read by a regular zstd reader.

This PR introduces a new package `zstd` to combine and wrap the two
packages, to provide a unified and easy-to-use API.

And a new setting `LOG_COMPRESSION` is added to the config, although I
don't see any reason why not to use compression, I think's it's a good
idea to keep the default with `none` to be consistent with old versions.

`LOG_COMPRESSION` takes effect for only new log files, it adds `.zst` as
an extension to the file name, so Gitea can determine if it needs
decompression according to the file name when reading. Old files will
keep the format since it's not worth converting them, as they will be
cleared after #31735.

<img width="541" alt="image"
src="https://github.com/user-attachments/assets/e9598764-a4e0-4b68-8c2b-f769265183c9">

(cherry picked from commit 33cc5837a655ad544b936d4d040ca36d74092588)

Conflicts:
	assets/go-licenses.json
	go.mod
	go.sum
  resolved with make tidy
2024-08-13 06:51:49 +02:00
Edip Emre Bodur
8b6747173a
Fix null requested_reviewer from API (#31773)
If the assign the pull request review to a team, it did not show the
members of the team in the "requested_reviewers" field, so the field was
null. As a solution, I added the team members to the array.

fix #31764

(cherry picked from commit 94cca8846e7d62c8a295d70c8199d706dfa60e5c)
2024-08-13 06:51:49 +02:00
Earl Warren
439c8fb246 Merge pull request '[CHORE] Drop go-git support' (#4941) from gusted/drop-gogit into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4941
Reviewed-by: Otto <otto@codeberg.org>
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-08-13 04:47:54 +00:00
Radosław Piliszek
f4d86b4ab0 git-grep: fix for initial dashes in expressions
There is no reason to reject initial dashes in git-grep
expressions... other than the code not supporting it previously.
A new method is introduced to relax the security checks.
2024-08-12 22:12:47 +02:00
Otto Richter
e330c88411 fix(api): Correct descriptions for quota calls 2024-08-12 22:05:56 +02:00
Gusted
9c5c08859d
[BUG] Make logout event non-blocking
- When people click on the logout button, a event is sent to all
browser tabs (actually to a shared worker) to notify them of this
logout. This is done in a blocking fashion, to ensure every registered
channel (which realistically should be one for every user because of the
shared worker) for a user receives this message. While doing this, it
locks the mutex for the eventsource module.
- Codeberg is currently observing a deadlock that's caused by this
blocking behavior, a channel isn't receiving the logout event. We
currently don't have a good theory of why this is being caused. This in
turn is causing that the logout functionality is no longer working and
people no longer receive notifications, unless they refresh the page.
- This patchs makes this message non-blocking and thus making it
consistent with the other messages. We don't see a good reason why this
specific event needs to be blocking and the commit introducing it
doesn't offer a rationale either.
2024-08-12 19:13:23 +02:00
Gusted
a21128a734
[CHORE] Drop go-git support
See https://codeberg.org/forgejo/discussions/issues/164 for the
rationale and discussion of this change.

Everything related to the `go-git` dependency is dropped (Only a single
instance is left in a test file to test for an XSS, it requires crafting
an commit that Git itself refuses to craft). `_gogit` files have
been removed entirely, `go:build: !gogit` is removed, `XXX_nogogit.go` files
either have been renamed or had their code being merged into the
`XXX.go` file.
2024-08-12 19:11:09 +02:00
Otto
4132b18e59 Merge pull request 'git-grep: ensure bounded default for MatchesPerFile' (#4928) from yoctozepto/forgejo:git-grep-limit-matches-per-file into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4928
Reviewed-by: Shiny Nematoda <snematoda@noreply.codeberg.org>
Reviewed-by: Otto <otto@codeberg.org>
2024-08-12 14:37:43 +00:00
Otto
5e6bc07610 Merge branch 'forgejo' into revert-no-closed-pr-review 2024-08-12 14:30:03 +00:00
Otto
b7fcc5e29d Merge pull request 'chore(ci): Use forgejo mirror for common docker images' (#4939) from fnetx/mirror-ci-images into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4939
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
2024-08-12 14:28:15 +00:00
Otto Richter
395c10596e chore(ci): use postgres image from mirror 2024-08-12 16:19:20 +02:00
Otto Richter
c3f2b60b3b chore(renovate): Add mirrored node image to renovate.json 2024-08-12 16:19:20 +02:00
Otto Richter
53ca3d4e6c chore(ci): Use mirrored container images for node 2024-08-12 16:11:35 +02:00
Gusted
fdabcf5d44 Merge pull request 'Lock file maintenance (forgejo)' (#4936) from renovate/forgejo-lock-file-maintenance into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4936
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-08-12 12:26:19 +00:00
Caesar Schinas
65c2595f26
Revert "Prevent allow/reject reviews on merged/closed PRs"
This reverts commit 4ed372af13.
This change from Gitea was not considered by the Forgejo UI team and there is a consensus that it feels like a regression.

The test which was added in that commit is kept and modified to test that reviews can successfully be submitted on closed and merged PRs.

Closes forgejo/design#11
2024-08-12 12:24:52 +01:00
Michael Kriese
2d05e922a2
fix(agit): run full pr checks on force-push 2024-08-12 09:00:41 +02:00
forgejo-renovate-action
0d2efa2c4a Merge pull request 'Update renovate to v38.25.0 (forgejo)' (#4933) from renovate/forgejo-renovate into forgejo 2024-08-12 04:50:22 +00:00
Renovate Bot
88169e74ed Lock file maintenance 2024-08-12 02:06:15 +00:00
Renovate Bot
eb6afae1c0 Update renovate to v38.25.0 2024-08-12 00:04:22 +00:00
Earl Warren
8359b26d3f Merge pull request '[PORT] Add warning message in merge instructions when AutodetectManualMerge was not enabled (gitea#31805)' (#4930) from gusted/forgejo-gt-bp-31805 into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4930
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-08-11 22:15:40 +00:00
Gusted
d2184dd931 Merge pull request 'git-grep: skip binary files' (#4927) from yoctozepto/forgejo:git-grep-skip-binary into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4927
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Reviewed-by: Shiny Nematoda <snematoda@noreply.codeberg.org>
2024-08-11 18:58:48 +00:00
a1012112796
e5f8d144f2
[PORT] Add warning message in merge instructions when AutodetectManualMerge was not enabled (gitea#31805)
---

Conflict resolution: trivial
Things done differently: Improve localization message, use the paragraph
element instead of the div element, fix passing this variable to the
template and add a integration test

(cherry picked from commit 9633f336c87947dc7d2a5e76077a10699ba5e50d)
2024-08-11 19:15:37 +02:00
Radosław Piliszek
f4a7bf6d2a git-grep: skip binary files
It is a waste of resources to scan them looking for matches
because they are never returned back - they appear as empty
lines in the current format.

Notably, even if they were returned, it is unlikely that matching
in binary files makes sense when the goal is "code search".
2024-08-11 19:10:23 +02:00
Earl Warren
44002a6399 Merge pull request 'chore(refactor): split repo_service.ForkRepository in two' (#4879) from earl-warren/forgejo:wip-fork-split into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4879
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-08-11 15:50:52 +00:00
Radosław Piliszek
2fbb51ceb2 git-grep: ensure bounded default for MatchesPerFile
Analogously to how it happens for MaxResultLimit.

The default of 20 is inspired by a well-known, commercial code
hosting platform.

Unbounded limits are risky because they expose Forgejo to a class
of DoS attacks where queries are crafted to take advantage of
missing bounds.
2024-08-11 14:59:46 +02:00
Earl Warren
cfefe2b6c9
chore(refactor): split repo_service.ForkRepository in two
ForkRepository performs two different functions:

* The fork itself, if it does not already exist
* Updates and notifications after the fork is performed

The function is split to reflect that and otherwise unmodified.

The two function are given different names to:

* clarify which integration tests provides coverage
* distinguish it from the notification method by the same name
2024-08-11 12:40:34 +02:00
Exploding Dragon
87d50eca87 feat: support grouping by any path for arch package (#4903)
Previous arch package grouping was not well-suited for complex or multi-architecture environments. It now supports the following content:

- Support grouping by any path.
- New support for packages in `xz` format.
- Fix clean up rules

<!--start release-notes-assistant-->

## Draft release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Features
  - [PR](https://codeberg.org/forgejo/forgejo/pulls/4903): <!--number 4903 --><!--line 0 --><!--description c3VwcG9ydCBncm91cGluZyBieSBhbnkgcGF0aCBmb3IgYXJjaCBwYWNrYWdl-->support grouping by any path for arch package<!--description-->
<!--end release-notes-assistant-->

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4903
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Co-authored-by: Exploding Dragon <explodingfkl@gmail.com>
Co-committed-by: Exploding Dragon <explodingfkl@gmail.com>
2024-08-11 10:35:11 +00:00
Earl Warren
a4da672134 Merge pull request 'git-grep: update comment' (#4921) from yoctozepto/forgejo:git-grep-update-comment into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4921
Reviewed-by: Shiny Nematoda <snematoda@noreply.codeberg.org>
2024-08-11 09:47:58 +00:00
Earl Warren
1b24180327 Merge pull request 'chore(ci): do not remove tags from forgejo-integration' (#4923) from earl-warren/forgejo:wip-integration-cleanup into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4923
Reviewed-by: Michael Kriese <michael.kriese@gmx.de>
2024-08-11 09:46:25 +00:00
Earl Warren
f250f89491
chore(ci): do not remove tags from forgejo-integration
If the tag of a stable release is removed from integration, it won't
be properly described when building the test release. It will be:

8.0.0-dev-1648-7b31a541c0+gitea-1.22.0

instead of:

8.0.1-5-7b31a541c0+gitea-1.22.0
2024-08-11 07:22:21 +02:00
Radosław Piliszek
7dd7cc7ebc git-grep: update comment
It was outdated and missing detail.
2024-08-10 16:41:12 +02:00
Earl Warren
a83f5cd0f0 Merge pull request 'chore(ci): remove old releases from forgejo-integration' (#4920) from earl-warren/forgejo:wip-integration-cleanup into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4920
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-08-10 13:18:08 +00:00
Earl Warren
6e94be527a
chore(ci): remove old releases from forgejo-integration
The releases are created when:

* a tag is pushed to the integration repository it will create a
  vX.Y.Z release
* a new commit is pushed to a branch and mirrored to the integration
  repository, it will create a vX.Y-test release named after the branch

When both vX.Y.Z and vX.Y-test release are present, the end-to-end
tests will use vX.Y.Z because it comes first in release sort
order. This ensures that a last round of end-to-end tests is run from
the release built in the integration repository, exactly as it will be
published and signed.

In between stable releases, the vX.Y-test releases are built daily and
must be used instead for end-to-end testing so that problems can be
detected as soon as possible. For that to happen, the stable release
must be removed from the integration repository and this is done 24h
after they were published.

The vX.Y-test releases are removed if they have not been updated in 18
months. As of August 2024 it is possible for a LTS to still be needed
in tests over a year after it was last updated, although it is
unlikely that such a lack of activity happens, there is no reason to
remove the test release before that.
2024-08-10 15:16:00 +02:00
Gusted
6102f48c7d Merge pull request '[CHORE] Fix swagger deprecation message' (#4916) from gusted/swagger-deprecated into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4916
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-08-10 12:50:40 +00:00
Earl Warren
3b82a634c5 Merge pull request 'feat(i18n): make the test string more fun :D' (#4904) from n0toose/i18n-fun-test-string into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4904
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-08-10 06:03:51 +00:00
Earl Warren
f8728ad881 Merge pull request '[BUG] Return blocking errors as JSON errors' (#4914) from gusted/forgejo-block-json into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4914
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
2024-08-10 05:52:27 +00:00
Earl Warren
40e51e4ca7 Merge pull request 'fix(ui): allow unreacting from comment popover' (#4798) from solomonv/forgejo:issue-reaction-fixes into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4798
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-08-10 05:45:55 +00:00
Gusted
851d567776
[CHORE] Fix swagger deprecation message
- Fix "WARNING: item list for enum is not a valid JSON array, using the
old deprecated format" messages from
https://github.com/go-swagger/go-swagger in the CI.
2024-08-10 01:21:13 +02:00
Gusted
784173f7e9 Merge pull request 'Update dependency @stylistic/eslint-plugin-js to v2 (forgejo)' (#4910) from renovate/forgejo-major-eslint-stylistic-monorepo into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4910
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-08-09 23:17:38 +00:00
Renovate Bot
ca00643416 Update dependency @stylistic/eslint-plugin-js to v2 2024-08-09 22:03:02 +00:00
Gusted
6ba4fb5cf6 Merge pull request 'Update vitest monorepo to v2 (forgejo) (major)' (#4913) from renovate/forgejo-major-vitest-monorepo into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4913
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-08-09 20:38:56 +00:00
Gusted
9cc2fdffde Merge pull request 'Update dependency minimatch to v10 (forgejo)' (#4912) from renovate/forgejo-minimatch-10.x into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4912
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-08-09 20:20:27 +00:00
Gusted
967153ba45 Merge pull request 'Update dependency @stylistic/stylelint-plugin to v3 (forgejo)' (#4911) from renovate/forgejo-stylistic-stylelint-plugin-3.x into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4911
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-08-09 20:19:46 +00:00
Gusted
437b84a5f9 Merge pull request 'Update module github.com/editorconfig-checker/editorconfig-checker/v2/cmd/editorconfig-checker to v3 (forgejo)' (#4909) from renovate/forgejo-github.com-editorconfig-checker-editorconfig-checker-v2-cmd-editorconfig-checker-3.x into forgejo
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/4909
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
2024-08-09 20:19:32 +00:00
Panagiotis "Ivory" Vasilopoulos
57a2b99b3c feat(i18n): make the test string more fun :D 2024-08-09 21:51:07 +02:00
Renovate Bot
8039240c26
Update module github.com/editorconfig-checker/editorconfig-checker/v2/cmd/editorconfig-checker to v3 2024-08-09 21:03:37 +02:00
Gusted
d97cf0e854
[BUG] Return blocking errors as JSON errors
- These endspoints are since b71cb7acdc
JSON-based and should therefore return JSON errors.
- Integration tests adjusted.
2024-08-09 20:34:38 +02:00
Renovate Bot
f70d50a8dc Update vitest monorepo to v2 2024-08-09 18:13:31 +00:00
Renovate Bot
ade201095a Update dependency minimatch to v10 2024-08-09 18:13:13 +00:00
Renovate Bot
c541431773 Update dependency @stylistic/stylelint-plugin to v3 2024-08-09 18:12:59 +00:00