[feature] Allow loading TLS certs from disk (#1586)

Currently, GtS only supports using the built-in LE client directly for
TLS. However, admins may still want to use GtS directly (so without a
reverse proxy) but with certificates provided through some other
mechanism. They may have some centralised way of provisioning these
things themselves, or simply prefer to use LE but with a different
challenge like DNS-01 which is not supported by autocert.

This adds support for loading a public/private keypair from disk instead
of using LE and reconfigures the server to use a TLS listener if we
succeed in doing so.

Additionally, being able to load TLS keypair from disk opens up the path
to using a custom CA for testing purposes avoinding the need for a
constellation of containers and something like Pebble or Step CA to
provide LE APIs.
This commit is contained in:
Daenney 2023-03-04 18:24:02 +01:00 committed by GitHub
parent ef074752d0
commit d2f6de0185
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
12 changed files with 153 additions and 4 deletions

View file

@ -29,6 +29,7 @@ These contribution guidelines were adapted from / inspired by those of Gitea (ht
- [SQLite](#sqlite) - [SQLite](#sqlite)
- [Postgres](#postgres) - [Postgres](#postgres)
- [CLI Tests](#cli-tests) - [CLI Tests](#cli-tests)
- [Federation](#federation)
- [Updating Swagger docs](#updating-swagger-docs) - [Updating Swagger docs](#updating-swagger-docs)
- [CI/CD configuration](#cicd-configuration) - [CI/CD configuration](#cicd-configuration)
- [Release Checklist](#release-checklist) - [Release Checklist](#release-checklist)
@ -418,6 +419,20 @@ In [./test/envparsing.sh](./test/envparsing.sh) there's a test for making sure t
Although this test *is* part of the CI/CD testing process, you probably won't need to worry too much about running it yourself. That is, unless you're messing about with code inside the `main` package in `cmd/gotosocial`, or inside the `config` package in `internal/config`. Although this test *is* part of the CI/CD testing process, you probably won't need to worry too much about running it yourself. That is, unless you're messing about with code inside the `main` package in `cmd/gotosocial`, or inside the `config` package in `internal/config`.
#### Federation
By using the support for loading TLS files from disk it is possible to have two local instances with TLS to allow for (manually) testing federation.
You'll need to set the following configuration options:
* `GTS_TLS_CERTIFICATE_CHAIN`: poiting to a PEM-encoded certificate chain including the public certificate
* `GTS_TLS_CERTIFICATE_KEY`: pointing to a PEM-encoded private key
Additionally, for the Go HTTP client to recognise certificates issued by a custom CA as valid, you'll need to set one of:
* `SSL_CERT_FILE`: pointing to the public key of your custom CA
* `SSL_CERT_DIR`: a `:`-separated list of directories to load CA certificates from
You'll additionally need functioning DNS for your two instance names which you can achieve through entries in `/etc/hosts` or by running a local DNS server like [dnsmasq](https://thekelleys.org.uk/dnsmasq/doc.html).
### Updating Swagger docs ### Updating Swagger docs
GoToSocial uses [go-swagger](https://goswagger.io) to generate Swagger API documentation from code annotations. GoToSocial uses [go-swagger](https://goswagger.io) to generate Swagger API documentation from code annotations.

View file

@ -1,4 +1,12 @@
# LetsEncrypt # TLS
It's possible to configure TLS support in one of two ways:
* Built-in support for Lets Encrypt / ACME compatible vendors
* Loading TLS files from disk
It is not possible to have both methods enabled at the same time.
Note that when using TLS files loaded from disk you are responsible for restarting the instance when the files change. They are not automatically reloaded.
## Settings ## Settings
@ -39,4 +47,20 @@ letsencrypt-cert-dir: "/gotosocial/storage/certs"
# Examples: ["admin@example.org"] # Examples: ["admin@example.org"]
# Default: "" # Default: ""
letsencrypt-email-address: "" letsencrypt-email-address: ""
##############################
##### MANUAL TLS CONFIG #####
##############################
# String. Path to a PEM-encoded file on disk that includes the certificate chain
# and the public key
# Examples: ["/gotosocial/storage/certs/chain.pem"]
# Default: ""
tls-certificate-chain: ""
# String. Path to a PEM-encoded file on disk containing the private key for the
# associated tls-certificate-chain
# Examples: ["/gotosocial/storage/certs/private.pem"]
# Default: ""
tls-certificate-key: ""
``` ```

View file

@ -92,7 +92,7 @@ For example, let's say you created the `~/gotosocial/data` directory for a user
#### LetsEncrypt (optional) #### LetsEncrypt (optional)
If you want to use [LetsEncrypt](../configuration/letsencrypt.md) for ssl certificates (https), you should also: If you want to use [LetsEncrypt](../configuration/tls.md) for ssl certificates (https), you should also:
1. Change the value of `GTS_LETSENCRYPT_ENABLED` to `"true"`. 1. Change the value of `GTS_LETSENCRYPT_ENABLED` to `"true"`.
2. Remove the `#` before `- "80:80"` in the `ports` section. 2. Remove the `#` before `- "80:80"` in the `ports` section.

View file

@ -575,6 +575,22 @@ letsencrypt-cert-dir: "/gotosocial/storage/certs"
# Default: "" # Default: ""
letsencrypt-email-address: "" letsencrypt-email-address: ""
##############################
##### MANUAL TLS CONFIG #####
##############################
# String. Path to a PEM-encoded file on disk that includes the certificate chain
# and the public key
# Examples: ["/gotosocial/storage/certs/chain.pem"]
# Default: ""
tls-certificate-chain: ""
# String. Path to a PEM-encoded file on disk containing the private key for the
# associated tls-certificate-chain
# Examples: ["/gotosocial/storage/certs/private.pem"]
# Default: ""
tls-certificate-key: ""
####################### #######################
##### OIDC CONFIG ##### ##### OIDC CONFIG #####
####################### #######################

View file

@ -114,6 +114,9 @@ type Configuration struct {
LetsEncryptCertDir string `name:"letsencrypt-cert-dir" usage:"Directory to store acquired letsencrypt certificates."` LetsEncryptCertDir string `name:"letsencrypt-cert-dir" usage:"Directory to store acquired letsencrypt certificates."`
LetsEncryptEmailAddress string `name:"letsencrypt-email-address" usage:"Email address to use when requesting letsencrypt certs. Will receive updates on cert expiry etc."` LetsEncryptEmailAddress string `name:"letsencrypt-email-address" usage:"Email address to use when requesting letsencrypt certs. Will receive updates on cert expiry etc."`
TLSCertificateChain string `name:"tls-certificate-chain" usage:"Filesystem path to the certificate chain including any intermediate CAs and the TLS public key"`
TLSCertificateKey string `name:"tls-certificate-key" usage:"Filesystem path to the TLS private key"`
OIDCEnabled bool `name:"oidc-enabled" usage:"Enabled OIDC authorization for this instance. If set to true, then the other OIDC flags must also be set."` OIDCEnabled bool `name:"oidc-enabled" usage:"Enabled OIDC authorization for this instance. If set to true, then the other OIDC flags must also be set."`
OIDCIdpName string `name:"oidc-idp-name" usage:"Name of the OIDC identity provider. Will be shown to the user when logging in."` OIDCIdpName string `name:"oidc-idp-name" usage:"Name of the OIDC identity provider. Will be shown to the user when logging in."`
OIDCSkipVerification bool `name:"oidc-skip-verification" usage:"Skip verification of tokens returned by the OIDC provider. Should only be set to 'true' for testing purposes, never in a production environment!"` OIDCSkipVerification bool `name:"oidc-skip-verification" usage:"Skip verification of tokens returned by the OIDC provider. Should only be set to 'true' for testing purposes, never in a production environment!"`

View file

@ -91,6 +91,9 @@ var Defaults = Configuration{
LetsEncryptCertDir: "/gotosocial/storage/certs", LetsEncryptCertDir: "/gotosocial/storage/certs",
LetsEncryptEmailAddress: "", LetsEncryptEmailAddress: "",
TLSCertificateChain: "",
TLSCertificateKey: "",
OIDCEnabled: false, OIDCEnabled: false,
OIDCIdpName: "", OIDCIdpName: "",
OIDCSkipVerification: false, OIDCSkipVerification: false,

View file

@ -114,6 +114,10 @@ func (s *ConfigState) AddServerFlags(cmd *cobra.Command) {
cmd.Flags().String(LetsEncryptCertDirFlag(), cfg.LetsEncryptCertDir, fieldtag("LetsEncryptCertDir", "usage")) cmd.Flags().String(LetsEncryptCertDirFlag(), cfg.LetsEncryptCertDir, fieldtag("LetsEncryptCertDir", "usage"))
cmd.Flags().String(LetsEncryptEmailAddressFlag(), cfg.LetsEncryptEmailAddress, fieldtag("LetsEncryptEmailAddress", "usage")) cmd.Flags().String(LetsEncryptEmailAddressFlag(), cfg.LetsEncryptEmailAddress, fieldtag("LetsEncryptEmailAddress", "usage"))
// Manual TLS
cmd.Flags().String(TLSCertificateChainFlag(), cfg.TLSCertificateChain, fieldtag("TLSCertificateChain", "usage"))
cmd.Flags().String(TLSCertificateKeyFlag(), cfg.TLSCertificateKey, fieldtag("TLSCertificateKey", "usage"))
// OIDC // OIDC
cmd.Flags().Bool(OIDCEnabledFlag(), cfg.OIDCEnabled, fieldtag("OIDCEnabled", "usage")) cmd.Flags().Bool(OIDCEnabledFlag(), cfg.OIDCEnabled, fieldtag("OIDCEnabled", "usage"))
cmd.Flags().String(OIDCIdpNameFlag(), cfg.OIDCIdpName, fieldtag("OIDCIdpName", "usage")) cmd.Flags().String(OIDCIdpNameFlag(), cfg.OIDCIdpName, fieldtag("OIDCIdpName", "usage"))

View file

@ -1524,6 +1524,56 @@ func GetLetsEncryptEmailAddress() string { return global.GetLetsEncryptEmailAddr
// SetLetsEncryptEmailAddress safely sets the value for global configuration 'LetsEncryptEmailAddress' field // SetLetsEncryptEmailAddress safely sets the value for global configuration 'LetsEncryptEmailAddress' field
func SetLetsEncryptEmailAddress(v string) { global.SetLetsEncryptEmailAddress(v) } func SetLetsEncryptEmailAddress(v string) { global.SetLetsEncryptEmailAddress(v) }
// GetTLSCertificateChain safely fetches the Configuration value for state's 'TLSCertificateChain' field
func (st *ConfigState) GetTLSCertificateChain() (v string) {
st.mutex.Lock()
v = st.config.TLSCertificateChain
st.mutex.Unlock()
return
}
// SetTLSCertificateChain safely sets the Configuration value for state's 'TLSCertificateChain' field
func (st *ConfigState) SetTLSCertificateChain(v string) {
st.mutex.Lock()
defer st.mutex.Unlock()
st.config.TLSCertificateChain = v
st.reloadToViper()
}
// TLSCertificateChainFlag returns the flag name for the 'TLSCertificateChain' field
func TLSCertificateChainFlag() string { return "tls-certificate-chain" }
// GetTLSCertificateChain safely fetches the value for global configuration 'TLSCertificateChain' field
func GetTLSCertificateChain() string { return global.GetTLSCertificateChain() }
// SetTLSCertificateChain safely sets the value for global configuration 'TLSCertificateChain' field
func SetTLSCertificateChain(v string) { global.SetTLSCertificateChain(v) }
// GetTLSCertificateKey safely fetches the Configuration value for state's 'TLSCertificateKey' field
func (st *ConfigState) GetTLSCertificateKey() (v string) {
st.mutex.Lock()
v = st.config.TLSCertificateKey
st.mutex.Unlock()
return
}
// SetTLSCertificateKey safely sets the Configuration value for state's 'TLSCertificateKey' field
func (st *ConfigState) SetTLSCertificateKey(v string) {
st.mutex.Lock()
defer st.mutex.Unlock()
st.config.TLSCertificateKey = v
st.reloadToViper()
}
// TLSCertificateKeyFlag returns the flag name for the 'TLSCertificateKey' field
func TLSCertificateKeyFlag() string { return "tls-certificate-key" }
// GetTLSCertificateKey safely fetches the value for global configuration 'TLSCertificateKey' field
func GetTLSCertificateKey() string { return global.GetTLSCertificateKey() }
// SetTLSCertificateKey safely sets the value for global configuration 'TLSCertificateKey' field
func SetTLSCertificateKey(v string) { global.SetTLSCertificateKey(v) }
// GetOIDCEnabled safely fetches the Configuration value for state's 'OIDCEnabled' field // GetOIDCEnabled safely fetches the Configuration value for state's 'OIDCEnabled' field
func (st *ConfigState) GetOIDCEnabled() (v bool) { func (st *ConfigState) GetOIDCEnabled() (v bool) {
st.mutex.Lock() st.mutex.Lock()

View file

@ -67,6 +67,19 @@ func Validate() error {
errs = append(errs, fmt.Errorf("%s must be set", WebAssetBaseDirFlag())) errs = append(errs, fmt.Errorf("%s must be set", WebAssetBaseDirFlag()))
} }
tlsChain := GetTLSCertificateChain()
tlsKey := GetTLSCertificateKey()
tlsChainFlag := TLSCertificateChainFlag()
tlsKeyFlag := TLSCertificateKeyFlag()
if GetLetsEncryptEnabled() && (tlsChain != "" || tlsKey != "") {
errs = append(errs, fmt.Errorf("%s cannot be enabled when %s and/or %s are also set", LetsEncryptEnabledFlag(), tlsChainFlag, tlsKeyFlag))
}
if (tlsChain != "" && tlsKey == "") || (tlsChain == "" && tlsKey != "") {
errs = append(errs, fmt.Errorf("%s and %s need to both be set or unset", tlsChainFlag, tlsKeyFlag))
}
if len(errs) > 0 { if len(errs) > 0 {
errStrings := []string{} errStrings := []string{}
for _, err := range errs { for _, err := range errs {

View file

@ -20,6 +20,7 @@ package router
import ( import (
"context" "context"
"crypto/tls"
"fmt" "fmt"
"net" "net"
"net/http" "net/http"
@ -78,6 +79,26 @@ func (r *router) Start() {
// but updated to TLS if LetsEncrypt is enabled. // but updated to TLS if LetsEncrypt is enabled.
listen := r.srv.ListenAndServe listen := r.srv.ListenAndServe
// During config validation we already checked that both Chain and Key are set
// so we can forego checking for both here
if chain := config.GetTLSCertificateChain(); chain != "" {
pkey := config.GetTLSCertificateKey()
cer, err := tls.LoadX509KeyPair(chain, pkey)
if err != nil {
log.Fatalf(
nil,
"tls: failed to load keypair from %s and %s, ensure they are PEM-encoded and can be read by this process: %s",
chain, pkey, err,
)
}
r.srv.TLSConfig = &tls.Config{
MinVersion: tls.VersionTLS12,
Certificates: []tls.Certificate{cer},
}
// TLS is enabled, update the listen function
listen = func() error { return r.srv.ListenAndServeTLS("", "") }
}
if config.GetLetsEncryptEnabled() { if config.GetLetsEncryptEnabled() {
// LetsEncrypt support is enabled // LetsEncrypt support is enabled

View file

@ -65,7 +65,7 @@ nav:
- "configuration/media.md" - "configuration/media.md"
- "configuration/storage.md" - "configuration/storage.md"
- "configuration/statuses.md" - "configuration/statuses.md"
- "configuration/letsencrypt.md" - "configuration/tls.md"
- "configuration/oidc.md" - "configuration/oidc.md"
- "configuration/smtp.md" - "configuration/smtp.md"
- "configuration/syslog.md" - "configuration/syslog.md"

View file

@ -2,7 +2,7 @@
set -eu set -eu
EXPECT='{"account-domain":"peepee","accounts-allow-custom-css":true,"accounts-approval-required":false,"accounts-reason-required":false,"accounts-registration-open":true,"advanced-cookies-samesite":"strict","advanced-rate-limit-requests":6969,"advanced-throttling-multiplier":-1,"advanced-throttling-retry-after":10000000000,"application-name":"gts","bind-address":"127.0.0.1","cache":{"gts":{"account-max-size":99,"account-sweep-freq":1000000000,"account-ttl":10800000000000,"block-max-size":100,"block-sweep-freq":30000000000,"block-ttl":300000000000,"domain-block-max-size":1000,"domain-block-sweep-freq":60000000000,"domain-block-ttl":86400000000000,"emoji-category-max-size":100,"emoji-category-sweep-freq":30000000000,"emoji-category-ttl":300000000000,"emoji-max-size":500,"emoji-sweep-freq":30000000000,"emoji-ttl":300000000000,"media-max-size":500,"media-sweep-freq":30000000000,"media-ttl":300000000000,"mention-max-size":500,"mention-sweep-freq":30000000000,"mention-ttl":300000000000,"notification-max-size":500,"notification-sweep-freq":30000000000,"notification-ttl":300000000000,"report-max-size":100,"report-sweep-freq":30000000000,"report-ttl":300000000000,"status-max-size":500,"status-sweep-freq":30000000000,"status-ttl":300000000000,"tombstone-max-size":100,"tombstone-sweep-freq":30000000000,"tombstone-ttl":300000000000,"user-max-size":100,"user-sweep-freq":30000000000,"user-ttl":300000000000}},"config-path":"internal/config/testdata/test.yaml","db-address":":memory:","db-database":"gotosocial_prod","db-max-open-conns-multiplier":3,"db-password":"hunter2","db-port":6969,"db-sqlite-busy-timeout":1000000000,"db-sqlite-cache-size":0,"db-sqlite-journal-mode":"DELETE","db-sqlite-synchronous":"FULL","db-tls-ca-cert":"","db-tls-mode":"disable","db-type":"sqlite","db-user":"sex-haver","dry-run":true,"email":"","host":"example.com","instance-deliver-to-shared-inboxes":false,"instance-expose-peers":true,"instance-expose-public-timeline":true,"instance-expose-suspended":true,"instance-expose-suspended-web":true,"landing-page-user":"admin","letsencrypt-cert-dir":"/gotosocial/storage/certs","letsencrypt-email-address":"","letsencrypt-enabled":true,"letsencrypt-port":80,"log-db-queries":true,"log-level":"info","media-description-max-chars":5000,"media-description-min-chars":69,"media-emoji-local-max-size":420,"media-emoji-remote-max-size":420,"media-image-max-size":420,"media-remote-cache-days":30,"media-video-max-size":420,"oidc-admin-groups":["steamy"],"oidc-client-id":"1234","oidc-client-secret":"shhhh its a secret","oidc-enabled":true,"oidc-idp-name":"sex-haver","oidc-issuer":"whoknows","oidc-link-existing":true,"oidc-scopes":["read","write"],"oidc-skip-verification":true,"password":"","path":"","port":6969,"protocol":"http","request-id-header":"X-Trace-Id","smtp-from":"queen.rip.in.piss@terfisland.org","smtp-host":"example.com","smtp-password":"hunter2","smtp-port":4269,"smtp-username":"sex-haver","software-version":"","statuses-cw-max-chars":420,"statuses-max-chars":69,"statuses-media-max-files":1,"statuses-poll-max-options":1,"statuses-poll-option-max-chars":50,"storage-backend":"local","storage-local-base-path":"/root/store","storage-s3-access-key":"minio","storage-s3-bucket":"gts","storage-s3-endpoint":"localhost:9000","storage-s3-proxy":true,"storage-s3-secret-key":"miniostorage","storage-s3-use-ssl":false,"syslog-address":"127.0.0.1:6969","syslog-enabled":true,"syslog-protocol":"udp","trusted-proxies":["127.0.0.1/32","docker.host.local"],"username":"","web-asset-base-dir":"/root","web-template-base-dir":"/root"}' EXPECT='{"account-domain":"peepee","accounts-allow-custom-css":true,"accounts-approval-required":false,"accounts-reason-required":false,"accounts-registration-open":true,"advanced-cookies-samesite":"strict","advanced-rate-limit-requests":6969,"advanced-throttling-multiplier":-1,"advanced-throttling-retry-after":10000000000,"application-name":"gts","bind-address":"127.0.0.1","cache":{"gts":{"account-max-size":99,"account-sweep-freq":1000000000,"account-ttl":10800000000000,"block-max-size":100,"block-sweep-freq":30000000000,"block-ttl":300000000000,"domain-block-max-size":1000,"domain-block-sweep-freq":60000000000,"domain-block-ttl":86400000000000,"emoji-category-max-size":100,"emoji-category-sweep-freq":30000000000,"emoji-category-ttl":300000000000,"emoji-max-size":500,"emoji-sweep-freq":30000000000,"emoji-ttl":300000000000,"media-max-size":500,"media-sweep-freq":30000000000,"media-ttl":300000000000,"mention-max-size":500,"mention-sweep-freq":30000000000,"mention-ttl":300000000000,"notification-max-size":500,"notification-sweep-freq":30000000000,"notification-ttl":300000000000,"report-max-size":100,"report-sweep-freq":30000000000,"report-ttl":300000000000,"status-max-size":500,"status-sweep-freq":30000000000,"status-ttl":300000000000,"tombstone-max-size":100,"tombstone-sweep-freq":30000000000,"tombstone-ttl":300000000000,"user-max-size":100,"user-sweep-freq":30000000000,"user-ttl":300000000000}},"config-path":"internal/config/testdata/test.yaml","db-address":":memory:","db-database":"gotosocial_prod","db-max-open-conns-multiplier":3,"db-password":"hunter2","db-port":6969,"db-sqlite-busy-timeout":1000000000,"db-sqlite-cache-size":0,"db-sqlite-journal-mode":"DELETE","db-sqlite-synchronous":"FULL","db-tls-ca-cert":"","db-tls-mode":"disable","db-type":"sqlite","db-user":"sex-haver","dry-run":true,"email":"","host":"example.com","instance-deliver-to-shared-inboxes":false,"instance-expose-peers":true,"instance-expose-public-timeline":true,"instance-expose-suspended":true,"instance-expose-suspended-web":true,"landing-page-user":"admin","letsencrypt-cert-dir":"/gotosocial/storage/certs","letsencrypt-email-address":"","letsencrypt-enabled":true,"letsencrypt-port":80,"log-db-queries":true,"log-level":"info","media-description-max-chars":5000,"media-description-min-chars":69,"media-emoji-local-max-size":420,"media-emoji-remote-max-size":420,"media-image-max-size":420,"media-remote-cache-days":30,"media-video-max-size":420,"oidc-admin-groups":["steamy"],"oidc-client-id":"1234","oidc-client-secret":"shhhh its a secret","oidc-enabled":true,"oidc-idp-name":"sex-haver","oidc-issuer":"whoknows","oidc-link-existing":true,"oidc-scopes":["read","write"],"oidc-skip-verification":true,"password":"","path":"","port":6969,"protocol":"http","request-id-header":"X-Trace-Id","smtp-from":"queen.rip.in.piss@terfisland.org","smtp-host":"example.com","smtp-password":"hunter2","smtp-port":4269,"smtp-username":"sex-haver","software-version":"","statuses-cw-max-chars":420,"statuses-max-chars":69,"statuses-media-max-files":1,"statuses-poll-max-options":1,"statuses-poll-option-max-chars":50,"storage-backend":"local","storage-local-base-path":"/root/store","storage-s3-access-key":"minio","storage-s3-bucket":"gts","storage-s3-endpoint":"localhost:9000","storage-s3-proxy":true,"storage-s3-secret-key":"miniostorage","storage-s3-use-ssl":false,"syslog-address":"127.0.0.1:6969","syslog-enabled":true,"syslog-protocol":"udp","tls-certificate-chain":"","tls-certificate-key":"","trusted-proxies":["127.0.0.1/32","docker.host.local"],"username":"","web-asset-base-dir":"/root","web-template-base-dir":"/root"}'
# Set all the environment variables to # Set all the environment variables to
# ensure that these are parsed without panic # ensure that these are parsed without panic