Commit graph

2371 commits

Author SHA1 Message Date
mrsdizzie 2842f6cf42
Include query in sign in redirect (#11579)
* Include query in sign in redirect

Include query string in ctx.Data["SignInLink"] so people are redirected to the right page after singing in. Fixes case of:

Visit:

 https://try.gitea.io/mrsdizzie/testcase/issues?q=&type=all&sort=&state=open&labels=7071&milestone=0&assignee=0

Sign in, then redirected to:

 https://try.gitea.io/mrsdizzie/testcase/issues

Create a new "SignInLink" instead of updating the existing "Link" because too many places in the code assume "Link" won't have a query

* Apply suggestions from code review

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-06-03 23:41:02 -04:00
6543 ce7510a80f
[API] on 500 error only show message if gitea in dev mode (#11641)
* add API specific InternalServerError()

* return 500 error msg only if not Production mode

* rm unnessesary change
2020-06-03 14:17:54 -04:00
John Olheiser a6fd2f23f7
Allow site admin to disable mirrors (#11740)
* Allow site admin to disable mirrors

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* No need to run through Safe

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Clarify only disabling NEW mirrors

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Apply suggestions from @guillep2k

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-06-02 21:21:38 -04:00
zeripath a1ad188326
Fix chardet test and add ordering option (#11621)
* Fix chardet test and add ordering option

Signed-off-by: Andrew Thornton <art27@cantab.net>

* minor fixes

Signed-off-by: Andrew Thornton <art27@cantab.net>

* remove log

Signed-off-by: Andrew Thornton <art27@cantab.net>

* remove log2

Signed-off-by: Andrew Thornton <art27@cantab.net>

* only iterate through top results

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update docs/content/doc/advanced/config-cheat-sheet.en-us.md

* slight restructure of for loop

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-06-02 19:20:19 -03:00
mrsdizzie 3af51f1ab7
Update emoji dataset with skin tone variants (#11678)
* Update emoji dataset with skin tone variants

Since the format of emoji that support skin tone modifiers is predictable we can add different variants into our dataset when generating it so that we can match and properly style most skin tone variants of emoji. No real code change here other than what generates the dataset and the data itself.

* use escape unicode sequence in map

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-06-02 02:22:40 -04:00
6543 dc812f8ba5
Issue/Pull expose IsLocked Property on API (#11708)
Expose IsLocked Property
2020-06-01 22:01:55 +01:00
Cirno the Strongest 9d652002c6
Fix language stat calculation (#11692)
* Fix language stat calculation

* Group languages and ignore 0 size files

* remove unneeded code
2020-05-31 01:58:55 +03:00
Lauris BH ea4c139cd2
Change language statistics to save size instead of percentage (#11681)
* Change language statistics to save size instead of percentage in database

Co-Authored-By: Cirno the Strongest <1447794+CirnoT@users.noreply.github.com>

* Do not exclude if only language

* Fix edge cases with special langauges

Co-authored-by: Cirno the Strongest <1447794+CirnoT@users.noreply.github.com>
2020-05-30 10:46:15 +03:00
zeripath 6e4480835f
Fix issue with DiffIndex on initial commit (#11677)
Unfortunately #11614 introduced a bug whereby the initial commit of a
repository could not be seen due to there being no parent commit to
create a clear diff from.

Here we create a diffstat from the difference between the parentless SHA and the SHA of the empty tree - a constant known to git. (With thanks to @L0veSunshine for informing me of this SHA)

Thanks to @a1012112796 for initial attempt to fix.

Fix #11650

Closes #11674

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-Authored-By: L0veSunshine <xuan199651@gmail.com>
2020-05-29 22:14:00 +01:00
Terence Le Huu Phuong 141d52cc0f
Add API Endpoint for Branch Creation (#11607)
* [FEATURE] [API] Add Endpoint for Branch Creation

Issue: https://github.com/go-gitea/gitea/issues/11376

This commit introduces an API endpoint for branch creation.

The added route is POST /repos/{owner}/{repo}/branches.
A JSON with the name of the new branch and the name of the old branch is
required as parameters.

Signed-off-by: Terence Le Huu Phuong <terence@qwasar.io>

* Put all the logic into CreateBranch and removed CreateRepoBranch

* - Added the error ErrBranchDoesNotExist in error.go
- Made the CreateNewBranch function return an errBranchDoesNotExist error
when the OldBranch does not exist
- Made the CreateBranch API function checks that the repository is not
empty and that branch exists.

* - Added a resetFixtures helper function in integration_test.go to
fine-tune test env resetting
- Added api test for CreateBranch
- Used resetFixture instead of the more general prepareTestEnv in the
repo_branch_test CreateBranch tests

* Moved the resetFixtures call inside the loop for APICreateBranch function

* Put the prepareTestEnv back in repo_branch_test

* fix import order/sort api branch test

Co-authored-by: zeripath <art27@cantab.net>
2020-05-29 19:16:20 +01:00
mrsdizzie 4c1ff57f1a
Update emoji regex (#11584)
When matching emoji, use a regex built from the data we have instead of something generic using unicode ranges. A generic regex can't tell the difference between two separate emoji next to each other or one emoji that is built out of two separate emoji next to each other.

This means that emoji that are next to each other without space in between will be now accurately spanned individually with proper title etc...
2020-05-29 17:08:36 +01:00
Lauris BH bd2335671f
Exclude generated files from language statistics (#11653)
* Update go-enry to v2.5.2
2020-05-29 09:20:01 +03:00
zeripath e8955173a9
Default MSSQL port 0 to allow automatic detection by default (#11642)
Fix #11633

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-05-28 23:59:59 -04:00
6543 9f55769804
Return json on 500 error from API (#11574)
* add API specific InternalServerError()

Co-authored-by: zeripath <art27@cantab.net>

* return 500 error msg only if not Production mode

* Revert "return 500 error msg only if not Production mode"

This reverts commit 8467b2cee6.

* InternalServerError

Co-authored-by: zeripath <art27@cantab.net>
2020-05-28 13:58:11 -03:00
zeripath 3aedc795c4
Use -1 to disable key algorithm type in ssh.minimum_key_sizes (#11635)
Fix #11634

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-05-28 16:29:15 +03:00
zeripath a00974c3b8
When must change password only show Signout (#11600)
When "Must Change Password" simplify the navbar header to only show the
signout button as all other links will redirect back. This prevents the
notifications icon from showing preventing initialization of the
event-source and hence preventing redirect_to being set, however in
addition do not set the redirect_to cookie if we are looking at the
/user/events page.

Fix #11554

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-05-26 23:39:39 +01:00
zeripath 5cb201dc93
Fix numbr of files, total additions, and deletions (#11614)
Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-05-26 01:58:07 -04:00
zeripath 3547e84a05
When initialising repositories ensure that the user doing the creation is the initializer (#11601)
Fix #10760

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-05-24 18:12:40 +01:00
zeripath 814ca9ffea
Allow different HardBreaks settings for documents and comments (#11515)
GH has different HardBreaks behaviour for markdown comments and documents.

Comments have hard breaks and documents have soft breaks - therefore Gitea's rendering will always be different from GH's if we only provide one setting.

Here we split the setting in to two - one for documents and one for comments and other things.

Signed-off-by: Andrew Thornton art27@cantab.net

Changes to index.js as per @silverwind 
Co-authored-by: silverwind <me@silverwind.io>

Changes to docs as per @guillep2k 
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-05-24 09:14:26 +01:00
zeripath f7e3767929
Prevent (caught) panic on login (#11590)
Unfortunately when the virtual session is released it requires that the
real session does not exist. This worked fine when sessions were only
saved at the end of request/response cycle however, now sessions are
saved proactively this does not hold.

The result is a caught panic in the logs during every log-in.  This
panic has no significant side-effects but should not occur.

This PR marks the virtual session as released when released and updates
it if the same session is released again.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-05-23 20:59:04 -03:00
Cirno the Strongest 31df012968
Properly handle and return empty string for dangling commits in GetBranchName (#11587) 2020-05-23 20:49:48 +01:00
silverwind 88fe7b5a72
Move serviceworker to workbox and fix SSE interference (#11538)
* Move serviceworker to workbox and fix SSE interference

Instead of statically hardcoding every frontend asset, this uses a
type-based approach to cache all js,css and manifest.json requests.

This also fixes the issue that the service worker was interfering with
EventSource because it was unconditionally handling all requests which
this new implementation doesn't.

Fixes: https://github.com/go-gitea/gitea/issues/11092
Fixes: https://github.com/go-gitea/gitea/issues/7372

* rethrow error instead of logging

* await .register

* Revert "rethrow error instead of logging"

This reverts commit 043162ba1f.

* improve comment

* remove JSRenderer

* add version-based cache invalidation

* refactor

* more refactor

* remove comment

* rename item to fit cache name

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-05-21 21:45:34 -04:00
zeripath 0b8b81e029
API PullReviewComment HTMLPullURL should return the HTMLURL (#11501)
Fix #11499

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-05-20 22:41:30 -04:00
zeripath c81ab17de9
TrimSpace when reading InternalToken from a file (#11502)
InternalTokens are fixed as alphanum strings therefore TrimSpace from these.
Also use isatty to not add a terminal newline when redirecting generate.

Fix #11498

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
2020-05-20 19:16:06 +03:00
赵智超 0903b1ac8c
Add push commits history comment on PR time-line (#11167)
* Add push commits history comment on PR time-line
* Add notify by email and ui of this comment type also

Signed-off-by: a1012112796 <1012112796@qq.com>

* Add migrations for IsForcePush
* fix wrong force-push judgement
* Apply suggestions from code review
* Remove commit number check
* add own notify fun
* fix some typo

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>

* fix lint

* fix style again, I forgot something before

* Change email notify way

* fix api

* add number check if It's force-push

* Add repo commit link fuction
remove unnecessary check
skip show push commits comment which not have commits alive

* Update issue_comment.go

* Apply suggestions from code review

Co-authored-by: mrsdizzie <info@mrsdizzie.com>

* Apply suggestions from code review

* fix ui view

Co-authored-by: silverwind <me@silverwind.io>

* fix height

* remove unnecessary style define

* simplify GetBranchName

* Apply suggestions from code review

* save commit ids and isForce push by json
* simplify GetBranchName

* fix bug

Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
Co-authored-by: mrsdizzie <info@mrsdizzie.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: silverwind <me@silverwind.io>
2020-05-20 09:47:24 -03:00
zeripath 9a2e47b23a
Refactor Cron and merge dashboard tasks (#10745)
* Refactor Cron and merge dashboard tasks

* Merge Cron and Dashboard tasks
* Make every cron task report a system notice on completion
* Refactor the creation of these tasks
* Ensure that execution counts of tasks is correct
* Allow cron tasks to be started from the cron page

* golangci-lint fixes

* Enforce that only one task with the same name can be registered

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix name check

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per @guillep2k

* as per @lafriks

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Add git.CommandContext variants

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-05-16 19:31:38 -04:00
zeripath 0052d788da
Allow log.xxx.default to set logging settings for the default logger only (#11292)
* Allow log.xxx.default to set logging settings for the default logger only

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update modules/setting/log.go

* as per @silverwind add some documentation

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-05-15 22:38:52 -04:00
zeripath 4a04740daf
Handle panics that percolate up to the graceful module (#11291)
* Handle panics in graceful goroutines

Adds a some deferred functions to handle panics in graceful goroutines

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Handle panic in webhook.Deliver

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Handle panic in mirror.syncMirror

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-05-14 20:06:00 -04:00
Sijmen Schoon 66a9ef9036
Fix ref links in issue overviews for tags (#8742)
* Properly generate ref URLs

Tags used to not generate correct URLs (src/branch/tags/1.0.0 instead of
src/tags/1.0.0).

Also cleans up some code around it with the created helper functions.

* Fix formatting and create migration

* Add copyright head to utils_test

* Use a raw query for the ref migration

* Remove semicolon

* Quote column and table names in migration SQL

* Change || to CONCAT, since MSSQL does not support ||

* Make migration engine aware

* Add missing import

* Move ref EndName and URL to the issue service

* Fix tests

* Add test for commit refs

* Update issue.go

* Use the right command for building JavaScript bundles

* Prepare for merge

* Check for refs/* before prepending in migration

* Update services/issue/issue_test.go

* Update modules/git/utils_test.go

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: techknowlogick <matti@mdranta.net>
2020-05-14 18:55:43 -04:00
zeripath 8d212f778c
Revert "Avoid 0 new commits messages to be send (#11082)" (#11397)
This reverts commit 6034f8bcaa.
2020-05-13 10:50:56 +03:00
6543 7257c39ddf
Refactor Milestone related (#11225) 2020-05-12 22:54:35 +01:00
zeripath 0198bbedc1
Allow compare page to look up base, head, own-fork, forkbase-of-head (#11327)
* Allow compare page to look up base, head, own-fork, forkbase-of-head

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per @guillep2k

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update routers/repo/compare.go

* as per @guillep2k

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Rationalise the names a little

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Rationalise the names a little (2)

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Fix 500 with fork of fork

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Prevent 500 on compare different trees

Signed-off-by: Andrew Thornton <art27@cantab.net>

* dotdotdot is perfectly valid in both usernames and repo names

Signed-off-by: Andrew Thornton <art27@cantab.net>

* ensure we can set the head and base repos too

Signed-off-by: Andrew Thornton <art27@cantab.net>

* ensure we can set the head and base repos too (2)

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix lint

Signed-off-by: Andrew Thornton <art27@cantab.net>

* only set headRepo == baseRepo if isSameRepo

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-05-12 08:52:46 +03:00
burbon d8e6acda8c
Support Range header end in lfs (#11314)
* Initial support of end Range header option in lfs

* Allow end range option to be unspecified

* Declare toByte for consistency

* Factor out content encoding tests from doLfs

This is so Range tests could still use doLfs but without repeating
not related tests

* Add Range header test

* implemented extraHeader
* parametrized expectedStatus

* Add more test cases of Range header

* Fix S1030: should use resp.Body.String()

* Add more tests for edge cases

* Fix tests

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: zeripath <art27@cantab.net>
2020-05-11 11:37:59 +03:00
zeripath 742e26f5a5
Prevent 500 with badly formed task list (#11328)
Fix #11317

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-05-11 00:14:49 +01:00
zeripath f8e015e8e4
Move LFS directory creation out of NewContext (#11362)
Fix #9481
(probably others too)

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
2020-05-10 16:30:46 +03:00
zeripath c58bc4bf80
Prevent timer leaks in Workerpool and others (#11333)
There is a potential memory leak in `Workerpool` due to the intricacies of
`time.Timer` stopping.

Whenever a `time.Timer` is `Stop`ped its channel must be cleared using a
`select` if the result of the `Stop()` is `false`.

Unfortunately in `Workerpool` these were checked the wrong way round.

However, there were a few other places that were not being checked.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2020-05-08 16:46:05 +01:00
zeripath 6f6edb8fab
Add NotifySyncPushCommits to indexer notifier (#11309)
Thanks to @simon-on-gh for tracking down the issue.

Fix #11200

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-05-08 15:58:40 +01:00
zeripath 791353c03b
Add EventSource support (#11235)
If the browser supports EventSource switch to use this instead of
polling notifications.

Signed-off-by: Andrew Thornton art27@cantab.net
2020-05-07 22:49:00 +01:00
zeripath 71e73db2ff
Manage port in submodule refurl (#11305)
* Manage port in submodule refurl

Fix #11304

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix lint

Signed-off-by: Andrew Thornton <art27@cantab.net>

* URLJoin is causes a cyclic dependency and possibly isn't what what we want anyway

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Protect against leading .. in scp syntax

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
2020-05-07 11:25:52 -03:00
zeripath d9de58beee
api.Context.NotFound(...) should tolerate nil (#11288)
There is an unfortunate signature change with the api.Context
NotFound function; whereas the normal modules/context/Context
NotFound function requires an error or nil, the api.Context
variant will panic with an NPE if a nil is provided.

This PR will allow api.Context.NotFound to tolerate a being
passed a nil.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Lauris BH <lauris@nix.lv>
2020-05-05 21:52:13 +03:00
Kyle Evans 38cd9ba47b
Allow unauthenticated users to compare (#11240)
* routers: make /compare route available to unauthenticated users

Remove some bits of the compare interface if the user isn't signed in.
Notably, they don't need to see the "New Pull Request" button box nor the
hidden form that would fail to submit due to the POST request continuing to
require proper privileges.

Follow-up commits will improve the UI a bit around this, removing some
"Pull Request" verbiage in favor of "Compare."

* ui: home: show "compare" button for unauthenticated users

This change requires pulling in the BaseRepo unconditionally and
recording if the pull request is in-fact not allowed
(.PullRequestCtx.Allowed). If the user isn't allowed to create a pull
request, either because this isn't a fork or same-fork branch PRs aren't
allowed, then we'll name the button "Compare" instead of "Pull Request."

* ui: branch list: use the new Compare language when available

When viewing the branch listing as an unauthenticated user, you'll get
"Pull Request" buttons. use the new "Compare" verbiage instead, which
matches GitHub behavior when you can't issue a pull request from the
branches.

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-05-04 19:44:30 -03:00
zeripath ce66ca7f9f
Restore checkbox rendering and prevent poor sanitization of spans (#11277)
* Add test

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Restore checkbox rendering and prevent poor sanitization of spans

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Also fix preview context

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Also fix preview context

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-05-03 17:17:24 -03:00
zeripath 4f9d59be17
Sendmail should create a process on the gitea system and have a default timeout (#11256)
* Make sure that sendmail processes register with the process manager
* Provide a timeout for these (initially of 5 minutes)
* Add configurable value and tie in to documentation
* Tie in to the admin config page.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-05-03 00:04:31 +01:00
6543 c97494a4f4
API: Add pull review endpoints (#11224)
* API: Added pull review read only endpoints

* Update Structs, move Conversion, Refactor

* refactor

* lint & co

* fix lint + refactor

* add new Review state, rm unessesary, refacotr loadAttributes, convert patch to diff

* add DeletePullReview

* add paggination

* draft1: Create & submit review

* fix lint

* fix lint

* impruve test

* DONT use GhostUser for loadReviewer

* expose comments_count of a PullReview

* infent GetCodeCommentsCount()

* fixes

* fix+impruve

* some nits

* Handle Ghosts 👻

* add TEST for GET apis

* complete TESTS

* add HTMLURL to PullReview responce

* code format as per @lafriks

* update swagger definition

* Update routers/api/v1/repo/pull_review.go

Co-authored-by: David Svantesson <davidsvantesson@gmail.com>

* add comments

Co-authored-by: Thomas Berger <loki@lokis-chaos.de>
Co-authored-by: David Svantesson <davidsvantesson@gmail.com>
2020-05-02 03:20:51 +03:00
mrsdizzie 4ed7d2a2bb
Remove last traces of has-emoji class (#11263)
Now that emojify.js has been removed, get rid of all instances of has-emoji class that was only used for that. Support for rendering shortcodes should remain in all of these places so it should still work the same.

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
2020-05-01 20:58:45 +03:00
zeripath 4974b7c120
Fix spelling mistake with Rewiew (#11262)
Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-04-30 21:24:08 +01:00
Alexander Scheel 1bf9e44bda
Fix sanitizer config - multiple rules (#11133)
In #9888, it was reported that my earlier pull request #9075 didn't quite function as expected. I was quite hopeful the `ValuesWithShadow()` worked as expected (and, I thought my testing showed it did) but I guess not. @zeripath proposed an alternative syntax which I like:

```ini
[markup.sanitizer.1]
ELEMENT=a
ALLOW_ATTR=target
REGEXP=something
[markup.sanitizer.2]
ELEMENT=a
ALLOW_ATTR=target
REGEXP=something
```

This was quite easy to adopt into the existing code. I've done so in a semi-backwards-compatible manner:

 - The value from `.Value()` is used for each element.
 - We parse `[markup.sanitizer]` and all `[markup.sanitizer.*]` sections and add them as rules.

This means that existing configs will load one rule (not all rules). It also means people can use string identifiers (`[markup.sanitiser.KaTeX]`) if they prefer, instead of numbered ones.

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-04-29 12:34:59 +01:00
mrsdizzie 4563eb873d
Support unicode emojis and remove emojify.js (#11032)
* Support unicode emojis and remove emojify.js

This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea.

This works in a few ways:

First it adds emoji parsing support into gitea itself. This allows us to

 * Render emojis from valid alias (😄)
 * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling
 * Easily allow for custom "emoji"
 * Support all emoji rendering and features without javascript
 * Uses plain unicode and lets the system render in appropriate emoji font
 * Doesn't leave us relying on external sources for updates/fixes/features

That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also)

For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method.

The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released.

I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens.

I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others.

Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary.

Fixes: https://github.com/go-gitea/gitea/issues/9182
Fixes: https://github.com/go-gitea/gitea/issues/8974
Fixes: https://github.com/go-gitea/gitea/issues/8953
Fixes: https://github.com/go-gitea/gitea/issues/6628
Fixes: https://github.com/go-gitea/gitea/issues/5130

* add new shared function emojiHTML

* don't increase emoji size in issue title

* Update templates/repo/issue/view_content/add_reaction.tmpl

Co-Authored-By: 6543 <6543@obermui.de>

* Support for emoji rendering in various templates

* Render code and review comments as they should be

* Better way to handle mail subjects

* insert unicode from tribute selection

* Add template helper for plain text when needed

* Use existing replace function I forgot about

* Don't include emoji greater than Unicode Version 12

Only include emoji and aliases in JSON

* Update build/generate-emoji.go

* Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have

* final updates

* code review

* code review

* hard code gitea custom emoji to match previous behavior

* Update .eslintrc

Co-Authored-By: silverwind <me@silverwind.io>

* disable preempt

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
2020-04-28 15:05:39 -03:00
mrsdizzie 922a239079
Disable new signal-based asynchronous goroutine preemption from GO 1.14 in git env (#11237)
As seen in trouble shooting #11032 the new feature of Go 1.14 is causing several second delays in startup in certain situations. Debugging shows it spending several seconds handling SIGURG commands during init:

```
6922:04:51.984234 trace init() ./modules/queue/unique_queue_wrapped.go
remote: ) = 69 <0.000012>
remote: [pid 15984] 22:04:51 write(1, "\ttime taken: 236.761\302\265s\n\n", 25    time taken: 236.761µs
remote:
remote: ) = 25 <0.000011>
remote: [pid 15984] 22:04:51 --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=15984, si_uid=0} ---
remote: [pid 15984] 22:04:52 --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=15984, si_uid=0} ---
remote: [pid 15984] 22:04:52 --- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=15984, si_uid=0} ---
```

This causes up to 20 seconds added to a push in some cases as it happens for each call of the gitea hook command. This is likely the cause of #10661 as well and would start to effect users once we release 1.12 which would be the first release compiled with Go 1.14. I suspect this is just a slight issue with the upstream implementatation as there have been a few very similar bugs fixed and reported:

 https://github.com/golang/go/issues/37741
 https://github.com/golang/go/issues/37942

We should revisit this in the future and see if a newer version of Go has solved it, but for now disable this option in the environment that gitea hook runs in to avoid it.
2020-04-28 11:45:32 -04:00
zeripath 0e799c26ba
Prevent panic during wrappedConn close at hammertime (#11219)
* Prevent panic during wrappedConn close at hammertime

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Update modules/graceful/server.go

* Fix extraneous debug in goldmark.go

Signed-off-by: Andrew Thornton <art27@cantab.net>
2020-04-26 21:01:06 -03:00