Commit graph

54 commits

Author SHA1 Message Date
wxiaoguang
8a2c4e9ff2
Fix debian InRelease Acquire-By-Hash newline (#29204) (#29299)
Backport #29204

Co-authored-by: Robin Schoonover <robin@cornhooves.org>
(cherry picked from commit f634982d237b38e0634c5997612f50230898247e)
2024-02-22 11:07:28 +01:00
Gusted
7afbc62057
[GITEA] Generate install if condition for Alpine
- If the APKINFO contains an install if condition, write it in the APKINDEX.
- No integration testing, as I don't have the files to regenerate the
hardcoded compressed(?) APKINFO in the test.
- Resolves #2174

(cherry picked from commit 11da776bef)
(cherry picked from commit 2824ae4cf2)
2024-02-03 19:30:48 +01:00
Mihir Joshi
54a6dc73ef
Fix reverting a merge commit failing (#28794) (#28825)
Backport https://github.com/go-gitea/gitea/pull/28794

Fixes #22236

---
Error occurring currently while trying to revert commit using read-tree
-m approach:
> 2022/12/26 16:04:43 ...rvices/pull/patch.go:240:AttemptThreeWayMerge()
[E] [63a9c61a] Unable to run read-tree -m! Error: exit status 128 -
fatal: this operation must be run in a work tree
> 	 - fatal: this operation must be run in a work tree

We need to clone a non-bare repository for `git read-tree -m` to work.

bb371aee6e
adds support to create a non-bare cloned temporary upload repository.

After cloning a non-bare temporary upload repository, we [set default
index](https://github.com/go-gitea/gitea/blob/main/services/repository/files/cherry_pick.go#L37)
(`git read-tree HEAD`).
This operation ends up resetting the git index file (see investigation
details below), due to which, we need to call `git update-index
--refresh` afterward.

Here's the diff of the index file before and after we execute
SetDefaultIndex: https://www.diffchecker.com/hyOP3eJy/

Notice the **ctime**, **mtime** are set to 0 after SetDefaultIndex.

You can reproduce the same behavior using these steps:
```bash
$ git clone https://try.gitea.io/me-heer/test.git -s -b main
$ cd test
$ git read-tree HEAD
$ git read-tree -m 1f085d7ed8 1f085d7ed8 9933caed00
error: Entry '1' not uptodate. Cannot merge.
```

After which, we can fix like this:
```bash
$ git update-index --refresh
$ git read-tree -m 1f085d7ed8 1f085d7ed8 9933caed00
```

(cherry picked from commit 62f2d717b7)
2024-01-31 14:18:25 +01:00
Giteabot
881159f593
Fix alpine package files are not rebuilt (#28638) (#28665)
Backport #28638 by @lng2020

I noticed the `BuildAllRepositoryFiles` function under the Alpine folder
is unused and I thought it was a bug.
But I'm not sure about this. Was it on purpose?

Co-authored-by: Nanguan Lin <70063547+lng2020@users.noreply.github.com>
(cherry picked from commit 3e1bd61000)
2024-01-16 14:16:55 +00:00
Giteabot
160ef74363
Fix possible nil pointer access (#28428) (#28440)
Backport #28428 by @KN4CK3R

There could be a nil pointer exception if the file is not found because
that specific error is suppressed but not handled.

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
(cherry picked from commit 97a0bf151a)
2023-12-22 12:05:11 +01:00
Giteabot
b1b0d9a3fe
Fix RPM/Debian signature key creation (#28352) (#28353)
Backport #28352 by @KN4CK3R

Fixes #28324

The name parameter can't contain some characters
(https://github.com/keybase/go-crypto/blob/master/openpgp/keys.go#L680)
but is optional. Therefore just use an empty string.

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
(cherry picked from commit b71d4c3ec0)
2023-12-08 13:41:16 +01:00
Giteabot
25acbfed36
Close all hashed buffers (#27787) (#27790)
Backport #27787 by @KN4CK3R

Add missing `.Close()` calls. The current code does not delete the
temporary files if the data grows over 32mb.

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-10-25 22:24:25 +02:00
Giteabot
77bd3acb65
Do not force creation of _cargo-index repo on publish (#27266) (#27765)
Backport #27266 by @merlleu

Hello there,
Cargo Index over HTTP is now prefered over git for package updates: we
should not force users who do not need the GIT repo to have the repo
created/updated on each publish (it can still be created in the packages
settings).

The current behavior when publishing is to check if the repo exist and
create it on the fly if not, then update it's content.
Cargo HTTP Index does not rely on the repo itself so this will be
useless for everyone not using the git protocol for cargo registry.

This PR only disable the creation on the fly of the repo when publishing
a crate.

This is linked to #26844 (error 500 when trying to publish a crate if
user is missing write access to the repo) because it's now optional.

Co-authored-by: merlleu <r.langdorph@gmail.com>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-10-24 15:40:02 +08:00
Giteabot
fc7d3f7315
Another round of db.DefaultContext refactor (#27103) (#27262)
Backport #27103 by @JakobDev

Part of #27065

Co-authored-by: JakobDev <jakobdev@gmx.de>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-09-25 19:24:35 +02:00
JakobDev
c548dde205
More refactoring of db.DefaultContext (#27083)
Next step of #27065
2023-09-15 06:13:19 +00:00
Lunny Xiao
4f32abaf94
move repository deletion to service layer (#26948)
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-09-08 04:51:15 +00:00
Lunny Xiao
b9df9fa2e2
Move createrepository from module to service layer (#26927)
Repository creation depends on many models, so moving it to service
layer is better.
2023-09-06 12:08:51 +00:00
Lunny Xiao
540bf9fa6d
Move notification interface to services layer (#26915)
Extract from #22266
2023-09-05 18:37:47 +00:00
Chongyi Zheng
ac2f8c9ac6
Reduce some allocations in type conversion (#26772) 2023-08-29 00:43:16 +08:00
merlleu
a587d25261
Add auth-required to config.json for Cargo http registry (#26729)
Cargo registry-auth feature requires config.json to have a property
auth-required set to true in order to send token to all registry
requests.
This is ok for git index because you can manually edit the config.json
file to add the auth-required, but when using sparse
(setting index url to
"sparse+https://git.example.com/api/packages/{owner}/cargo/"), the
config.json is dynamically rendered, and does not reflect changes to the
config.json file in the repo.

I see two approaches:
- Serve the real config.json file when fetching the config.json on the
cargo service.
- Automatically detect if the registry requires authorization. (This is
what I implemented in this PR).

What the PR does:
- When a cargo index repository is created, on the config.json, set
auth-required to wether or not the repository is private.
- When the cargo/config.json endpoint is called, set auth-required to
wether or not the request was authorized using an API token.
2023-08-28 07:05:39 +00:00
KN4CK3R
0c6ae61229
Allow package cleanup from admin page (#25307)
Until now expired package data gets deleted daily by a cronjob. The
admin page shows the size of all packages and the size of unreferenced
data. The users (#25035, #20631) expect the deletion of this data if
they run the cronjob from the admin page but the job only deletes data
older than 24h.

This PR adds a new button which deletes all expired data.

![grafik](https://github.com/go-gitea/gitea/assets/1666336/b3e35d73-9496-4fa7-a20c-e5d30b1f6850)

---------

Co-authored-by: silverwind <me@silverwind.io>
2023-08-08 00:46:10 +00:00
KN4CK3R
2d3924d0e7
Prevent newline errors with Debian packages (#26332)
Fixes #26313
2023-08-05 10:59:52 +02:00
Peter Verraedt
48cd82b759
Fix version in rpm repodata/primary.xml.gz (#26009)
The version listed in rpm repodata should only contain the rpm version
(1.0.0) and not the combination of version and release (1.0.0-2). We
correct this behaviour in primary.xml.gz, filelists.xml.gz and
others.xml.gz.

Signed-off-by: Peter Verraedt <peter@verraedt.be>
2023-07-21 19:41:38 +00:00
harryzcy
0f9f6567bb
Bump github.com/golang-jwt/jwt to v5 (#25975)
Bumping `github.com/golang-jwt/jwt` from v4 to v5.

`github.com/golang-jwt/jwt` v5 is bringing some breaking changes:

- standard `Valid()` method on claims is removed. It's replaced by
`ClaimsValidator` interface implementing `Validator()` method instead,
which is called after standard validation. Gitea doesn't seem to be
using this logic.
- `jwt.Token` has a field `Valid`, so it's checked in `ParseToken`
function in `services/auth/source/oauth2/token.go`

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-07-19 09:57:10 +00:00
silverwind
88f835192d
Replace interface{} with any (#25686)
Result of running `perl -p -i -e 's#interface\{\}#any#g' **/*` and `make fmt`.

Basically the same [as golang did](2580d0e08d).
2023-07-04 18:36:08 +00:00
KN4CK3R
c890454769
Add direct serving of package content (#25543)
Fixes #24723

Direct serving of content aka HTTP redirect is not mentioned in any of
the package registry specs but lots of official registries do that so it
should be supported by the usual clients.
2023-07-03 15:33:28 +02:00
KN4CK3R
cdb088cec2
Add CRAN package registry (#22331)
This PR adds a [CRAN](https://cran.r-project.org/) package registry.

![grafik](https://user-images.githubusercontent.com/1666336/210450039-d6fa6f77-20cd-4741-89a8-1624def267f7.png)
2023-05-22 10:57:49 +08:00
KN4CK3R
5968c63a11
Add Go package registry (#24687)
Fixes #7608

This PR adds a Go package registry usable with the Go proxy protocol.

![grafik](https://github.com/go-gitea/gitea/assets/1666336/328feb5c-3df2-4f9d-8eae-fe3126d14c37)
2023-05-14 23:38:40 +08:00
KN4CK3R
9173e079ae
Add Alpine package registry (#23714)
This PR adds an Alpine package registry. You can follow [this
tutorial](https://wiki.alpinelinux.org/wiki/Creating_an_Alpine_package)
to build a *.apk package for testing.

This functionality is similar to the Debian registry (#22854) and
therefore shares some methods. I marked this PR as blocked because it
should be merged after #22854.


![grafik](https://user-images.githubusercontent.com/1666336/227779595-b76163aa-eea1-4a79-9583-775c24ad74e8.png)

---------

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-12 17:27:50 +00:00
KN4CK3R
05209f0d1d
Add RPM registry (#23380)
Fixes #20751

This PR adds a RPM package registry. You can follow [this
tutorial](https://opensource.com/article/18/9/how-build-rpm-packages) to
build a *.rpm package for testing.

This functionality is similar to the Debian registry (#22854) and
therefore shares some methods. I marked this PR as blocked because it
should be merged after #22854.


![grafik](https://user-images.githubusercontent.com/1666336/223806549-d8784fd9-9d79-46a2-9ae2-f038594f636a.png)
2023-05-05 20:33:37 +00:00
KN4CK3R
723598b803
Implement Cargo HTTP index (#24452)
This implements the HTTP index
[RFC](https://rust-lang.github.io/rfcs/2789-sparse-index.html) for Cargo
registries.

Currently this is a preview feature and you need to use the nightly of
`cargo`:

`cargo +nightly -Z sparse-registry update`

See https://github.com/rust-lang/cargo/issues/9069 for more information.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-03 16:58:43 -04:00
KN4CK3R
bf999e4069
Add Debian package registry (#24426)
Co-authored-by: @awkwardbunny

This PR adds a Debian package registry.
You can follow [this
tutorial](https://www.baeldung.com/linux/create-debian-package) to build
a *.deb package for testing.
Source packages are not supported at the moment and I did not find
documentation of the architecture "all" and how these packages should be
treated.


![grafik](https://user-images.githubusercontent.com/1666336/218126879-eb80a866-775c-4c8e-8529-5797203a64e6.png)

Part of #20751.

Revised copy of #22854.

---------

Co-authored-by: Brian Hong <brian@hongs.me>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-02 12:31:35 -04:00
Yarden Shoham
c0ddec8a2a
Revert "Add Debian package registry" (#24412)
Reverts go-gitea/gitea#22854
2023-04-28 18:06:41 -04:00
KN4CK3R
bf77e2163b
Add Debian package registry (#22854)
Co-authored-by: @awkwardbunny

This PR adds a Debian package registry. You can follow [this
tutorial](https://www.baeldung.com/linux/create-debian-package) to build
a *.deb package for testing. Source packages are not supported at the
moment and I did not find documentation of the architecture "all" and
how these packages should be treated.

---------

Co-authored-by: Brian Hong <brian@hongs.me>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2023-04-28 17:51:36 -04:00
wxiaoguang
ce9dee5a1e
Introduce path Clean/Join helper functions (#23495)
Since #23493 has conflicts with latest commits, this PR is my proposal
for fixing #23371

Details are in the comments

And refactor the `modules/options` module, to make it always use
"filepath" to access local files.

Benefits:

* No need to do `util.CleanPath(strings.ReplaceAll(p, "\\", "/"))),
"/")` any more (not only one before)
* The function behaviors are clearly defined
2023-03-21 16:02:49 -04:00
KN4CK3R
c709fa17a7
Add Swift package registry (#22404)
This PR adds a [Swift](https://www.swift.org/) package registry.


![grafik](https://user-images.githubusercontent.com/1666336/211842523-07521cbd-8fb6-400f-820c-ee8048b05ae8.png)
2023-03-13 15:28:39 -05:00
Lunny Xiao
b116418f05
Use CleanPath instead of path.Clean (#23371)
As title.
2023-03-08 20:17:39 +08:00
KN4CK3R
f8c1e14a13
Use import of OCI structs (#22765)
Fixes #22758

Otherwise we would need to rewrite the structs in `oci.go`.
2023-02-06 10:07:09 +00:00
KN4CK3R
d987ac6bf1
Add Chef package registry (#22554)
This PR implements a [Chef registry](https://chef.io/) to manage
cookbooks. This package type was a bit complicated because Chef uses RSA
signed requests as authentication with the registry.


![grafik](https://user-images.githubusercontent.com/1666336/213747995-46819fd8-c3d6-45a2-afd4-a4c3c8505a4a.png)


![grafik](https://user-images.githubusercontent.com/1666336/213748145-d01c9e81-d4dd-41e3-a3cc-8241862c3166.png)

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-02-06 09:49:21 +08:00
KN4CK3R
df789d962b
Add Cargo package registry (#21888)
This PR implements a [Cargo registry](https://doc.rust-lang.org/cargo/)
to manage Rust packages. This package type was a little bit more
complicated because Cargo needs an additional Git repository to store
its package index.

Screenshots:

![grafik](https://user-images.githubusercontent.com/1666336/203102004-08d812ac-c066-4969-9bda-2fed818554eb.png)

![grafik](https://user-images.githubusercontent.com/1666336/203102141-d9970f14-dca6-4174-b17a-50ba1bd79087.png)

![grafik](https://user-images.githubusercontent.com/1666336/203102244-dc05743b-78b6-4d97-998e-ef76341a978f.png)

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-02-05 18:12:31 +08:00
KN4CK3R
6ba9ff7b48
Add Conda package registry (#22262)
This PR adds a [Conda](https://conda.io/) package registry.
2023-02-01 12:30:39 -06:00
KN4CK3R
d283a31f03
Check quota limits for container uploads (#22450)
The test coverage has revealed that container packages were not checked
against the quota limits.
2023-01-29 11:34:29 -06:00
Lunny Xiao
f59ce77772
Display unreferenced packages total size in package admin panel (#22498) 2023-01-18 09:52:04 -06:00
Lunny Xiao
ca67c5a8a7
refactor auth interface to return error when verify failure (#22119)
This PR changed the Auth interface signature from 
`Verify(http *http.Request, w http.ResponseWriter, store DataStore, sess
SessionStore) *user_model.User`
to 
`Verify(http *http.Request, w http.ResponseWriter, store DataStore, sess
SessionStore) (*user_model.User, error)`.

There is a new return argument `error` which means the verification
condition matched but verify process failed, we should stop the auth
process.

Before this PR, when return a `nil` user, we don't know the reason why
it returned `nil`. If the match condition is not satisfied or it
verified failure? For these two different results, we should have
different handler. If the match condition is not satisfied, we should
try next auth method and if there is no more auth method, it's an
anonymous user. If the condition matched but verify failed, the auth
process should be stop and return immediately.

This will fix #20563

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Jason Song <i@wolfogre.com>
2022-12-28 13:53:28 +08:00
Jason Song
f59a74852b
Update gitea-vet to check FSFE REUSE (#22004)
Related to:
- #21840
- https://gitea.com/gitea/gitea-vet/pulls/21

What it looks like when it's working:
https://drone.gitea.io/go-gitea/gitea/64040/1/5

All available SPDX license identifiers: [SPDX License
List](https://spdx.org/licenses/).

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-12-02 22:14:57 +08:00
Jason Song
9607750b5e
Replace fmt.Sprintf with hex.EncodeToString (#21960)
`hex.EncodeToString` has better performance than `fmt.Sprintf("%x",
[]byte)`, we should use it as much as possible.

I'm not an extreme fan of performance, so I think there are some
exceptions:

- `fmt.Sprintf("%x", func(...)[N]byte())`
- We can't slice the function return value directly, and it's not worth
adding lines.
    ```diff
    func A()[20]byte { ... }
    - a := fmt.Sprintf("%x", A())
    - a := hex.EncodeToString(A()[:]) // invalid
    + tmp := A()
    + a := hex.EncodeToString(tmp[:])
    ```
- `fmt.Sprintf("%X", []byte)`
- `strings.ToUpper(hex.EncodeToString(bytes))` has even worse
performance.
2022-11-28 11:19:18 +00:00
flynnnnnnnnnn
e81ccc406b
Implement FSFE REUSE for golang files (#21840)
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2022-11-27 18:20:29 +00:00
KN4CK3R
32db62515f
Add package registry cleanup rules (#21658)
Fixes #20514
Fixes #20766
Fixes #20631

This PR adds Cleanup Rules for the package registry. This allows to
delete unneeded packages automatically. Cleanup rules can be set up from
the user or org settings.
Please have a look at the documentation because I'm not a native english
speaker.

Rule Form

![grafik](https://user-images.githubusercontent.com/1666336/199330792-c13918a6-e196-4e71-9f53-18554515edca.png)

Rule List

![grafik](https://user-images.githubusercontent.com/1666336/199331261-5f6878e8-a80c-4985-800d-ebb3524b1a8d.png)

Rule Preview

![grafik](https://user-images.githubusercontent.com/1666336/199330917-c95e4017-cf64-4142-a3e4-af18c4f127c3.png)

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-20 16:08:38 +02:00
KN4CK3R
044c754ea5
Add context.Context to more methods (#21546)
This PR adds a context parameter to a bunch of methods. Some helper
`xxxCtx()` methods got replaced with the normal name now.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-19 16:12:33 +08:00
Lunny Xiao
34283a74e8
Allow detect whether it's in a database transaction for a context.Context (#21756)
Fix #19513

This PR introduce a new db method `InTransaction(context.Context)`,
and also builtin check on `db.TxContext` and `db.WithTx`.
There is also a new method `db.AutoTx` has been introduced but could be used by other PRs.

`WithTx` will always open a new transaction, if a transaction exist in context, return an error.
`AutoTx` will try to open a new transaction if no transaction exist in context.
That means it will always enter a transaction if there is no error.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
2022-11-12 21:18:50 +01:00
KN4CK3R
20674dd05d
Add package registry quota limits (#21584)
Related #20471

This PR adds global quota limits for the package registry. Settings for
individual users/orgs can be added in a seperate PR using the settings
table.

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-11-09 14:34:27 +08:00
KN4CK3R
dc0253b063
Replace ServeStream with ServeContent (#20903)
* Replace ServeStream with ServeContent.

* Update modules/timeutil/timestamp.go

Co-authored-by: delvh <dev.lh@web.de>

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2022-08-25 12:05:21 -04:00
KN4CK3R
b678d1adae
Use correct context to get package content (#20673) 2022-08-04 16:14:47 -05:00
KN4CK3R
86e5268c39
Add Docker /v2/_catalog endpoint (#20469)
* Added properties for packages.
* Fixed authenticate header format.
* Added _catalog endpoint.
* Check owner visibility.
* Extracted condition.
* Added test for _catalog.

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-07-28 11:59:39 +08:00
KN4CK3R
a3d55ac523
Hide internal package versions (#20492)
* Hide internal versions from most searches.

* Added test.
2022-07-27 09:59:10 +08:00