Commit graph

3533 commits

Author SHA1 Message Date
Lunny Xiao 6fe3c8b398
Support org/user level projects (#22235)
Fix #13405

<img width="1151" alt="image"
src="https://user-images.githubusercontent.com/81045/209442911-7baa3924-c389-47b6-b63b-a740803e640e.png">

Co-authored-by: 6543 <6543@obermui.de>
2023-01-20 19:42:33 +08:00
Sybren 151b1a9508
Support importing comment types (#22510)
This commit adds support for specifying comment types when importing
with `gitea restore-repo`. It makes it possible to import issue changes,
such as "title changed" or "assigned user changed".

An earlier version of this pull request was made by Matti Ranta, in
https://future.projects.blender.org/blender-migration/gitea-bf/pulls/3

There are two changes with regard to Matti's original code:

1. The comment type was an `int64` in Matti's code, and is now using a
string. This makes it possible to use `comment_type: title`, which is
more reliable and future-proof than an index into an internal list in
the Gitea Go code.

2. Matti's code also had support for including labels, but in a way that
would require knowing the database ID of the labels before the import
even starts, which is impossible. This can be solved by using label
names instead of IDs; for simplicity I I left that out of this PR.
2023-01-18 21:14:56 -05:00
Sybren 326d29dce0
Reliable selection of admin user (#22509)
When importing a repository via `gitea restore-repo`, external users
will get remapped to an admin user. This admin user is obtained via
`users.GetAdminUser()`, which unfortunately picks a more-or-less random
admin to return.

This makes it hard to predict which admin user will get assigned. This
patch orders the admin by ascending ID before choosing the first one,
i.e. it picks the admin with the lowest ID.

Even though it would be nicer to have full control over which user is
chosen, this at least gives us a predictable result.
2023-01-18 10:57:16 -06:00
Lunny Xiao f59ce77772
Display unreferenced packages total size in package admin panel (#22498) 2023-01-18 09:52:04 -06:00
Chongyi Zheng de484e86bc
Support scoped access tokens (#20908)
This PR adds the support for scopes of access tokens, mimicking the
design of GitHub OAuth scopes.

The changes of the core logic are in `models/auth` that `AccessToken`
struct will have a `Scope` field. The normalized (no duplication of
scope), comma-separated scope string will be stored in `access_token`
table in the database.
In `services/auth`, the scope will be stored in context, which will be
used by `reqToken` middleware in API calls. Only OAuth2 tokens will have
granular token scopes, while others like BasicAuth will default to scope
`all`.
A large amount of work happens in `routers/api/v1/api.go` and the
corresponding `tests/integration` tests, that is adding necessary scopes
to each of the API calls as they fit.


- [x] Add `Scope` field to `AccessToken`
- [x] Add access control to all API endpoints
- [x] Update frontend & backend for when creating tokens
- [x] Add a database migration for `scope` column (enable 'all' access
to past tokens)

I'm aiming to complete it before Gitea 1.19 release.

Fixes #4300
2023-01-17 15:46:03 -06:00
Lunny Xiao db2286bbb6
some refactor about code comments (#20821) 2023-01-17 15:03:44 -06:00
Jason Song 21c91b7dff
Set disable_gravatar/enable_federated_avatar when offline mode is true (#22479)
When offline mode is true, we should set `disable_gravatar` to `true`
and `enable_federated_avatar` to `false` in system settings.
2023-01-17 17:00:19 +02:00
KN4CK3R 3510d7e33a
Fix container blob mount (#22226) 2023-01-16 17:35:48 -05:00
zeripath 2cc3a6381c
Add cron method to gc LFS MetaObjects (#22385)
This PR adds a task to the cron service to allow garbage collection of
LFS meta objects. As repositories may have a large number of
LFSMetaObjects, an updated column is added to this table and it is used
to perform a generational GC to attempt to reduce the amount of work.
(There may need to be a bit more work here but this is probably enough
for the moment.)

Fix #7045

Signed-off-by: Andrew Thornton <art27@cantab.net>
2023-01-16 13:50:53 -06:00
zeripath 16e9dec827
Fix Operator does not exist bug on explore page with ONLY_SHOW_RELEVANT_REPOS (#22454)
There is a mistake in the code for SearchRepositoryCondition where it
tests topics as a string. This is incorrect for postgres where topics is
cast and stored as json. topics needs to be cast to text for this to
work. (For some reason JSON_ARRAY_LENGTH does not work, so I have taken
the simplest solution of casting to text and doing a string comparison.)

Ref https://github.com/go-gitea/gitea/pull/21962#issuecomment-1379584057

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
2023-01-16 11:25:22 +00:00
Lunny Xiao 2782c14396
Supports wildcard protected branch (#20825)
This PR introduce glob match for protected branch name. The separator is
`/` and you can use `*` matching non-separator chars and use `**` across
separator.

It also supports input an exist or non-exist branch name as matching
condition and branch name condition has high priority than glob rule.

Should fix #2529 and #15705

screenshots

<img width="1160" alt="image"
src="https://user-images.githubusercontent.com/81045/205651179-ebb5492a-4ade-4bb4-a13c-965e8c927063.png">

Co-authored-by: zeripath <art27@cantab.net>
2023-01-16 16:00:22 +08:00
Jimmy Praet fe519d8633
Restore previous official review when an official review is deleted (#22449)
Fix #22406

Co-authored-by: Lauris BH <lauris@nix.lv>
2023-01-15 07:00:09 +02:00
KN4CK3R fc037b4b82
Add support for incoming emails (#22056)
closes #13585
fixes #9067
fixes #2386
ref #6226
ref #6219
fixes #745

This PR adds support to process incoming emails to perform actions.
Currently I added handling of replies and unsubscribing from
issues/pulls. In contrast to #13585 the IMAP IDLE command is used
instead of polling which results (in my opinion 😉) in cleaner code.

Procedure:
- When sending an issue/pull reply email, a token is generated which is
present in the Reply-To and References header.
- IMAP IDLE waits until a new email arrives
- The token tells which action should be performed

A possible signature and/or reply gets stripped from the content.

I added a new service to the drone pipeline to test the receiving of
incoming mails. If we keep this in, we may test our outgoing emails too
in future.

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-01-14 23:57:10 +08:00
Lunny Xiao a3ab82e592
Fix error when calculate the repository size (#22392)
Fix #22386 

`GetDirectorySize` moved as `getDirectorySize` because it becomes a
special function which should not be put in `util`.

Co-authored-by: Jason Song <i@wolfogre.com>
2023-01-13 18:54:02 +00:00
techknowlogick 6f231a7980
Replace deprecated Webauthn library (#22400)
Fix #22052

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-01-11 21:51:00 -05:00
Jason Song 477a1cc40e
Improve utils of slices (#22379)
- Move the file `compare.go` and `slice.go` to `slice.go`.
- Fix `ExistsInSlice`, it's buggy
  - It uses `sort.Search`, so it assumes that the input slice is sorted.
- It passes `func(i int) bool { return slice[i] == target })` to
`sort.Search`, that's incorrect, check the doc of `sort.Search`.
- Conbine `IsInt64InSlice(int64, []int64)` and `ExistsInSlice(string,
[]string)` to `SliceContains[T]([]T, T)`.
- Conbine `IsSliceInt64Eq([]int64, []int64)` and `IsEqualSlice([]string,
[]string)` to `SliceSortedEqual[T]([]T, T)`.
- Add `SliceEqual[T]([]T, T)` as a distinction from
`SliceSortedEqual[T]([]T, T)`.
- Redesign `RemoveIDFromList([]int64, int64) ([]int64, bool)` to
`SliceRemoveAll[T]([]T, T) []T`.
- Add `SliceContainsFunc[T]([]T, func(T) bool)` and
`SliceRemoveAllFunc[T]([]T, func(T) bool)` for general use.
- Add comments to explain why not `golang.org/x/exp/slices`.
- Add unit tests.
2023-01-11 13:31:16 +08:00
Jason Song a35714372d
Fix halfCommitter and WithTx (#22366)
Related to #22362.

I overlooked that there's always `committer.Close()`, like:

```go
		ctx, committer, err := db.TxContext(db.DefaultContext)
		if err != nil {
			return nil
		}
		defer committer.Close()

		// ...

		if err != nil {
			return nil
		}

		// ...

		return committer.Commit()
```

So the `Close` of `halfCommitter` should ignore `commit and close`, it's
not a rollback.

See: [Why `halfCommitter` and `WithTx` should rollback IMMEDIATELY or
commit
LATER](https://github.com/go-gitea/gitea/pull/22366#issuecomment-1374778612).

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2023-01-09 12:19:19 -05:00
Jason Song 7adc2de464
Use context parameter in models/git (#22367)
After #22362, we can feel free to use transactions without
`db.DefaultContext`.

And there are still lots of models using `db.DefaultContext`, I think we
should refactor them carefully and one by one.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-01-09 11:50:54 +08:00
Lunny Xiao dfd2db5695
Fix set system setting failure once it cached (#22333)
Unfortunately, #22295 introduced a bug that when set a cached system
setting, it will not affect.
This PR make sure to remove the cache key when updating a system
setting.

Fix #22332
2023-01-08 21:22:41 +08:00
Jason Song 6135359a04
Always reuse transaction (#22362) 2023-01-08 09:34:58 +08:00
Jason Song 3dbd2d942b
Remove old HookEventType (#22358)
Supplement to #22256.
2023-01-06 19:49:14 +08:00
Lunny Xiao fdbb0b4762
fix gravatar disable bug (#22336) 2023-01-04 16:19:56 +08:00
Lunny Xiao a1c30740bb
Fix get system setting bug when enabled redis cache (#22295)
Fix #22281

In #21621 , `Get[V]` and `Set[V]` has been introduced, so that cache
value will be `*Setting`. For memory cache it's OK. But for redis cache,
it can only store `string` for the current implementation. This PR
revert some of changes of that and just store or return a `string` for
system setting.
2023-01-02 00:06:52 +08:00
delvh 0f4e1b9ac6
Restructure webhook module (#22256)
Previously, there was an `import services/webhooks` inside
`modules/notification/webhook`.
This import was removed (after fighting against many import cycles).
Additionally, `modules/notification/webhook` was moved to
`modules/webhook`,
and a few structs/constants were extracted from `models/webhooks` to
`modules/webhook`.

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-01-01 23:23:15 +08:00
Lunny Xiao 9c8fc7f677
Fix bug of DisableGravatar default value (#22296)
#18058 made a mistake. The disableGravatar's default value depends on
`OfflineMode`. If it's `true`, then `disableGravatar` is true, otherwise
it's `false`. But not opposite.

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-01-01 20:19:23 +08:00
KN4CK3R 3fef47b41c
Use ErrInvalidArgument in packages (#22268)
Related to
https://github.com/go-gitea/gitea/pull/22262#discussion_r1059010774

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-12-31 12:49:37 +01:00
Jason Song e5deeda0aa
Support estimated count with multiple schemas (#22276)
The `EstimateCount` could be incorrect when the table lives in multiple
schemas. Related to #19775.
2022-12-30 12:25:58 -06:00
Gusted b76970f2e4
Fix key signature error page (#22229)
- When the GPG key contains an error, such as an invalid signature or an
email address that does not match the user.A page will be shown that
says you must provide a signature for the token.
- This page had two errors: one had the wrong translation key and the
other tried to use an undefined variable
[`.PaddedKeyID`](e81ccc406b/models/asymkey/gpg_key.go (L65-L72)),
which is a function implemented on the `GPGKey` struct, given that we
don't have that, we use
[`KeyID`](e81ccc406b/routers/web/user/setting/keys.go (L102))
which is [the fingerprint of the
publickey](https://pkg.go.dev/golang.org/x/crypto/openpgp/packet#PublicKey.KeyIdString)
and is a valid way for opengpg to refer to a key.

Before:

![image](https://user-images.githubusercontent.com/25481501/209404800-0e7c39ce-861a-455b-b234-62498d750aa8.png)

After:

![image](https://user-images.githubusercontent.com/25481501/209404821-c70f81c6-fd10-4197-ab58-61cb9fc873d8.png)

Co-authored-by: zeripath <art27@cantab.net>
2022-12-30 12:53:05 +08:00
Xinyu Zhou 7cc7db73b9
Add option to prohibit fork if user reached maximum limit of repositories (#21848)
If user has reached the maximum limit of repositories:

- Before
  - disallow create
  - allow fork without limit
- This patch:
  - disallow create
  - disallow fork
- Add option `ALLOW_FORK_WITHOUT_MAXIMUM_LIMIT` (Default **true**) :
enable this allow user fork repositories without maximum number limit

fixed https://github.com/go-gitea/gitea/issues/21847

Signed-off-by: Xinyu Zhou <i@sourcehut.net>
2022-12-27 15:21:14 -06:00
Jason Song 6cf09ccab4
Use complete SHA to create and query commit status (#22244)
Fix #13485.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-12-27 21:12:49 +08:00
Jason Song 71ca3067bc
Check primary keys for all tables and drop ForeignReference (#21721)
Some dbs require that all tables have primary keys, see
- #16802
- #21086

We can add a test to keep it from being broken again.

Edit:

~Added missing primary key for `ForeignReference`~ Dropped the
`ForeignReference` table to satisfy the check, so it closes #21086.

More context can be found in comments.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: zeripath <art27@cantab.net>
2022-12-23 19:35:43 +08:00
Nick a2779def36 Test views of LFS files (#22196) 2022-12-23 07:41:56 +08:00
Jason Song 659055138b
Secrets storage with SecretKey encrypted (#22142)
Fork of #14483, but [gave up
MasterKey](https://github.com/go-gitea/gitea/pull/14483#issuecomment-1350728557),
and fixed some problems.

Close #12065.
Needed by #13539.

Featrues:
- Secrets for repo and org, not user yet.
- Use SecretKey to encrypte/encrypt secrets.
- Trim spaces of secret value.
- Add a new locale ini block, to make it easy to support secrets for
user.

Snapshots:

Repo level secrets:

![image](https://user-images.githubusercontent.com/9418365/207823319-b8a4903f-38ca-4af7-9d05-336a5af906f3.png)

Rrg level secrets

![image](https://user-images.githubusercontent.com/9418365/207823371-8bd02e93-1928-40d1-8c76-f48b255ace36.png)

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2022-12-20 17:07:13 +08:00
zeripath 651fe4bb7d
Add doctor command for full GC of LFS (#21978)
The recent PR adding orphaned checks to the LFS storage is not
sufficient to completely GC LFS, as it is possible for LFSMetaObjects to
remain associated with repos but still need to be garbage collected.

Imagine a situation where a branch is uploaded containing LFS files but
that branch is later completely deleted. The LFSMetaObjects will remain
associated with the Repository but the Repository will no longer contain
any pointers to the object.

This PR adds a second doctor command to perform a full GC.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-12-15 20:44:16 +00:00
KN4CK3R 07461e18d3
Fix condition for is_internal (#22095)
depends on #22094

Fixes https://codeberg.org/forgejo/forgejo/issues/77

The old logic did not consider `is_internal`.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2022-12-14 17:16:01 +08:00
Lunny Xiao 36a2d2f919
Add a simple test for external renderer (#20033)
Fix #16402
2022-12-12 20:45:21 +08:00
KN4CK3R 0e2d04601a
Update xorm (#22094) 2022-12-10 18:53:32 -05:00
Lunny Xiao 68704532c2
Rename almost all Ctx functions (#22071) 2022-12-10 10:46:31 +08:00
KN4CK3R 3c59d31bc6
Add API management for issue/pull and comment attachments (#21783)
Close #14601
Fix #3690

Revive of #14601.
Updated to current code, cleanup and added more read/write checks.

Signed-off-by: Andrew Thornton <art27@cantab.net>
Signed-off-by: Andre Bruch <ab@andrebruch.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Norwin <git@nroo.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-12-09 14:35:56 +08:00
silverwind 0585ac3ac6
Update go dev dependencies (#22064)
`golangci-lint`
[deprecated](https://github.com/golangci/golangci-lint/issues/1841) a
bunch of linters, removed them.
2022-12-08 16:21:37 +08:00
hr-98 cf27403e18
Round language stats percentage using largest remainder (#22026)
Fix #22023 

I've changed how the percentages for the language statistics are rounded
because they did not always add up to 100%
Now it's done with the largest remainder method, which makes sure that
total is 100%

Co-authored-by: Lauris BH <lauris@nix.lv>
2022-12-08 10:47:47 +08:00
Jason Song 74874d065f
Fix issue/PR numbers (#22037)
When deleting a closed issue, we should update both `NumIssues`and
`NumClosedIssues`, or `NumOpenIssues`(`= NumIssues -NumClosedIssues`)
will be wrong. It's the same for pull requests.

Releated to #21557.

Alse fixed two harmless problems:

- The SQL to check issue/PR total numbers is wrong, that means it will
update the numbers even if they are correct.
- Replace legacy `num_issues = num_issues + 1` operations with
`UpdateRepoIssueNumbers`.
2022-12-06 15:53:09 +08:00
zeripath ea86c2b56a
Use GhostUser if needed for TrackedTimes (#22021)
When getting tracked times out of the db and loading their attributes
handle not exist errors in a nicer way. (Also prevent an NPE.)

Fix #22006

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-12-04 17:48:18 +00:00
Lunny Xiao 0a7d3ff786
refactor some functions to support ctx as first parameter (#21878)
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-12-03 10:48:26 +08:00
Jason Song f59a74852b
Update gitea-vet to check FSFE REUSE (#22004)
Related to:
- #21840
- https://gitea.com/gitea/gitea-vet/pulls/21

What it looks like when it's working:
https://drone.gitea.io/go-gitea/gitea/64040/1/5

All available SPDX license identifiers: [SPDX License
List](https://spdx.org/licenses/).

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-12-02 22:14:57 +08:00
Lunny Xiao f7ade6de7c
Fix generate index failure possibility on postgres (#21998)
@wxiaoguang Please review

Co-authored-by: silverwind <me@silverwind.io>
2022-12-02 11:15:36 +08:00
Lunny Xiao b2c4870481
Fix parallel creating commit status bug with tests (#21911)
This PR is a follow up of #21469

Co-authored-by: Lauris BH <lauris@nix.lv>
2022-12-01 00:41:49 +08:00
Jason Song f047ee0a40
Use random bytes to generate access token (#21959) 2022-11-28 23:37:42 +08:00
Jason Song 9607750b5e
Replace fmt.Sprintf with hex.EncodeToString (#21960)
`hex.EncodeToString` has better performance than `fmt.Sprintf("%x",
[]byte)`, we should use it as much as possible.

I'm not an extreme fan of performance, so I think there are some
exceptions:

- `fmt.Sprintf("%x", func(...)[N]byte())`
- We can't slice the function return value directly, and it's not worth
adding lines.
    ```diff
    func A()[20]byte { ... }
    - a := fmt.Sprintf("%x", A())
    - a := hex.EncodeToString(A()[:]) // invalid
    + tmp := A()
    + a := hex.EncodeToString(tmp[:])
    ```
- `fmt.Sprintf("%X", []byte)`
- `strings.ToUpper(hex.EncodeToString(bytes))` has even worse
performance.
2022-11-28 11:19:18 +00:00
flynnnnnnnnnn e81ccc406b
Implement FSFE REUSE for golang files (#21840)
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2022-11-27 18:20:29 +00:00