Warn if postgres database has non-C locale. (#6734)

As using non-C locale can cause issues on upgrading OS.
pull/6787/head
Erik Johnston 2020-01-28 13:44:21 +00:00 committed by GitHub
parent 33f904835a
commit 02b44db922
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 81 additions and 1 deletions

View File

@ -76,6 +76,15 @@ for example:
dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb dpkg -i matrix-synapse-py3_1.3.0+stretch1_amd64.deb
Upgrading to **<NEXT_VERSION>**
===============================
Synapse will now log a warning on start up if used with a PostgreSQL database
that has a non-recommended locale set.
See [docs/postgres.md](docs/postgres.md) for details.
Upgrading to v1.8.0 Upgrading to v1.8.0
=================== ===================

1
changelog.d/6734.bugfix Normal file
View File

@ -0,0 +1 @@
Warn if postgres database has a non-C locale, as that can cause issues when upgrading locales (e.g. due to upgrading OS).

View File

@ -32,7 +32,7 @@ Assuming your PostgreSQL database user is called `postgres`, first authenticate
su - postgres su - postgres
# Or, if your system uses sudo to get administrative rights # Or, if your system uses sudo to get administrative rights
sudo -u postgres bash sudo -u postgres bash
Then, create a user ``synapse_user`` with: Then, create a user ``synapse_user`` with:
createuser --pwprompt synapse_user createuser --pwprompt synapse_user
@ -63,6 +63,24 @@ You may need to enable password authentication so `synapse_user` can
connect to the database. See connect to the database. See
<https://www.postgresql.org/docs/11/auth-pg-hba-conf.html>. <https://www.postgresql.org/docs/11/auth-pg-hba-conf.html>.
### Fixing incorrect `COLLATE` or `CTYPE`
Synapse will refuse to set up a new database if it has the wrong values of
`COLLATE` and `CTYPE` set, and will log warnings on existing databases. Using
different locales can cause issues if the locale library is updated from
underneath the database, or if a different version of the locale is used on any
replicas.
The safest way to fix the issue is to take a dump and recreate the database with
the correct `COLLATE` and `CTYPE` parameters (as per
[docs/postgres.md](docs/postgres.md)). It is also possible to change the
parameters on a live database and run a `REINDEX` on the entire database,
however extreme care must be taken to avoid database corruption.
Note that the above may fail with an error about duplicate rows if corruption
has already occurred, and such duplicate rows will need to be manually removed.
## Tuning Postgres ## Tuning Postgres
The default settings should be fine for most deployments. For larger The default settings should be fine for most deployments. For larger

View File

@ -13,8 +13,12 @@
# See the License for the specific language governing permissions and # See the License for the specific language governing permissions and
# limitations under the License. # limitations under the License.
import logging
from ._base import IncorrectDatabaseSetup from ._base import IncorrectDatabaseSetup
logger = logging.getLogger(__name__)
class PostgresEngine(object): class PostgresEngine(object):
single_threaded = False single_threaded = False
@ -52,6 +56,44 @@ class PostgresEngine(object):
"See docs/postgres.rst for more information." % (rows[0][0],) "See docs/postgres.rst for more information." % (rows[0][0],)
) )
txn.execute(
"SELECT datcollate, datctype FROM pg_database WHERE datname = current_database()"
)
collation, ctype = txn.fetchone()
if collation != "C":
logger.warning(
"Database has incorrect collation of %r. Should be 'C'", collation
)
if ctype != "C":
logger.warning(
"Database has incorrect ctype of %r. Should be 'C'", ctype
)
def check_new_database(self, txn):
"""Gets called when setting up a brand new database. This allows us to
apply stricter checks on new databases versus existing database.
"""
txn.execute(
"SELECT datcollate, datctype FROM pg_database WHERE datname = current_database()"
)
collation, ctype = txn.fetchone()
errors = []
if collation != "C":
errors.append(" - 'COLLATE' is set to %r. Should be 'C'" % (collation,))
if ctype != "C":
errors.append(" - 'CTYPE' is set to %r. Should be 'C'" % (collation,))
if errors:
raise IncorrectDatabaseSetup(
"Database is incorrectly configured:\n\n%s\n\n"
"See docs/postgres.md for more information." % ("\n".join(errors))
)
def convert_param_style(self, sql): def convert_param_style(self, sql):
return sql.replace("?", "%s") return sql.replace("?", "%s")

View File

@ -59,6 +59,11 @@ class Sqlite3Engine(object):
if version < (3, 11, 0): if version < (3, 11, 0):
raise RuntimeError("Synapse requires sqlite 3.11 or above.") raise RuntimeError("Synapse requires sqlite 3.11 or above.")
def check_new_database(self, txn):
"""Gets called when setting up a brand new database. This allows us to
apply stricter checks on new databases versus existing database.
"""
def convert_param_style(self, sql): def convert_param_style(self, sql):
return sql return sql

View File

@ -136,6 +136,11 @@ def _setup_new_database(cur, database_engine, data_stores):
data_stores (list[str]): The names of the data stores to instantiate data_stores (list[str]): The names of the data stores to instantiate
on the given database. on the given database.
""" """
# We're about to set up a brand new database so we check that its
# configured to our liking.
database_engine.check_new_database(cur)
current_dir = os.path.join(dir_path, "schema", "full_schemas") current_dir = os.path.join(dir_path, "schema", "full_schemas")
directory_entries = os.listdir(current_dir) directory_entries = os.listdir(current_dir)