Commit Graph

153 Commits (57fac2a2347e0b386385868cdd1ae435364062b5)

Author SHA1 Message Date
Patrick Cloke e739b20588
Fix up types and comments that refer to Deferreds. (#7945) 2020-07-24 10:53:25 -04:00
Andrew Morgan b7ddece2a6
Lint the contrib/ directory in CI and linting scripts, add synctl to linting script (#7914)
Run `isort`, `flake8` and `black` over the `contrib/` directory and `synctl` script. The latter was already being done in CI, but now the linting script does it too.

Fixes https://github.com/matrix-org/synapse/issues/7910
2020-07-20 21:43:49 +01:00
Will Hunt 62b1ce8539
isort 5 compatibility (#7786)
The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.
2020-07-05 16:32:02 +01:00
Richard van der Hoff a6eae69ffe
Type checking for `FederationHandler` (#7770)
fix a few things to make this pass mypy.
2020-07-01 16:21:02 +01:00
Dagfinn Ilmari Mannsåker b099ef07d6
Make tox actions work on Debian 10 (#7703)
- Remove the requirement for a specific version of Python
- Move dep comment to a separate line, Tox 3.7.0 like trailing ones

Signed-off-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
2020-06-25 17:45:35 +01:00
Richard van der Hoff 1bbc9e2df6
Clean up exception handling in SAML2ResponseResource (#7614)
* Expose `return_html_error`, and allow it to take a Jinja2 template instead of a raw string

* Clean up exception handling in SAML2ResponseResource

  * use the existing code in `return_html_error` instead of re-implementing it
    (giving it a jinja2 template rather than inventing a new form of template)

  * do the exception-catching in the REST layer rather than in the handler
    layer, to make sure we catch all exceptions.
2020-06-03 10:41:12 +01:00
Richard van der Hoff d84bdfe599
mypy for synapse.http.site (#7553) 2020-05-22 10:12:17 +01:00
Richard van der Hoff 164f50f5f2
fix mypy for tests/replication (#7518) 2020-05-18 10:43:05 +01:00
Patrick Cloke c29915bd05
Add type hints to room member handlers (#7513) 2020-05-15 15:05:25 -04:00
Patrick Cloke 5355421295
Add type hints to event_auth code. (#7505) 2020-05-15 11:19:43 -04:00
Andrew Morgan 5cf758cdd6 Merge branch 'release-v1.13.0' into develop
* release-v1.13.0:
  Don't UPGRADE database rows
  RST indenting
  Put rollback instructions in upgrade notes
  Fix changelog typo
  Oh yeah, RST
  Absolute URL it is then
  Fix upgrade notes link
  Provide summary of upgrade issues in changelog. Fix )
  Move next version notes from changelog to upgrade notes
  Changelog fixes
  1.13.0rc1
  Documentation on setting up redis (#7446)
  Rework UI Auth session validation for registration (#7455)
  Fix errors from malformed log line (#7454)
  Drop support for redis.dbid (#7450)
2020-05-11 16:46:33 +01:00
Patrick Cloke 0ad6d28b0d
Rework UI Auth session validation for registration (#7455)
Be less strict about validation of UI authentication sessions during
registration to match client expecations.
2020-05-08 16:08:58 -04:00
Quentin Gliech 616af44137
Implement OpenID Connect-based login (#7256) 2020-05-08 08:30:40 -04:00
Patrick Cloke 22246919e3
Add more type hints to SAML handler. (#7445) 2020-05-07 09:30:45 -04:00
Richard van der Hoff 16b1a34e80
Fix typing annotations in synapse/federation (#7382)
We're pretty close to having mypy working for `synapse.federation`, so let's
finish the job.
2020-05-05 14:27:13 +01:00
Erik Johnston 8123b2f909
Add MultiWriterIdGenerator. (#7281)
This will be used to coordinate stream IDs across multiple writers.

Functions as the equivalent of both `StreamIdGenerator` and
`SlavedIdTracker`.
2020-05-04 17:17:45 +01:00
Patrick Cloke 627b0f5f27
Persist user interactive authentication sessions (#7302)
By persisting the user interactive authentication sessions to the database, this fixes
situations where a user hits different works throughout their auth session and also
allows sessions to persist through restarts of Synapse.
2020-04-30 13:47:49 -04:00
Richard van der Hoff c2e1a2110f
Fix limit logic for EventsStream (#7358)
* Factor out functions for injecting events into database

I want to add some more flexibility to the tools for injecting events into the
database, and I don't want to clutter up HomeserverTestCase with them, so let's
factor them out to a new file.

* Rework TestReplicationDataHandler

This wasn't very easy to work with: the mock wrapping was largely superfluous,
and it's useful to be able to inspect the received rows, and clear out the
received list.

* Fix AssertionErrors being thrown by EventsStream

Part of the problem was that there was an off-by-one error in the assertion,
but also the limit logic was too simple. Fix it all up and add some tests.
2020-04-29 12:30:36 +01:00
Richard van der Hoff 13683a3a22
Extend StreamChangeCache to support multiple entities per stream ID (#7303)
First some background: StreamChangeCache is used to keep track of what "entities" have 
changed since a given stream ID. So for example, we might use it to keep track of when the last
to-device message for a given user was received [1], and hence whether we need to pull any to-device messages from the database on a sync [2].

Now, it turns out that StreamChangeCache didn't support more than one thing being changed at
a given stream_id (this was part of the problem with #7206). However, it's entirely valid to send
to-device messages to more than one user at a time.

As it turns out, this did in fact work, because *some* methods of StreamChangeCache coped
ok with having multiple things changing on the same stream ID, and it seems we never actually
use the methods which don't work on the stream change caches where we allow multiple
changes at the same stream ID. But that feels horribly fragile, hence: let's update
StreamChangeCache to properly support this, and add some typing and some more tests while
we're at it.

[1]: https://github.com/matrix-org/synapse/blob/release-v1.12.3/synapse/storage/data_stores/main/deviceinbox.py#L301
[2]: https://github.com/matrix-org/synapse/blob/release-v1.12.3/synapse/storage/data_stores/main/deviceinbox.py#L47-L51
2020-04-22 13:45:40 +01:00
Patrick Cloke 01294e6b3a
Do not treat display names as globs for push rules. (#7271) 2020-04-16 10:52:55 -04:00
Patrick Cloke d78cb31588
Add typing information to federation_server. (#7219) 2020-04-07 15:03:23 -04:00
Patrick Cloke fa4f12102d
Refactor the CAS code (move the logic out of the REST layer to a handler) (#7136) 2020-03-26 15:05:26 -04:00
Richard van der Hoff 8c75667ad7
Add prometheus metrics for the number of active pushers (#7103) 2020-03-19 10:00:24 +00:00
Patrick Cloke 77d0a4507b
Add type annotations and comments to auth handler (#7063) 2020-03-12 11:36:27 -04:00
Patrick Cloke 13892776ef
Allow deleting an alias if the user has sufficient power level (#6986) 2020-03-04 11:30:46 -05:00
Patrick Cloke 12d4259000
Add some type annotations to the federation base & client classes (#6995) 2020-02-28 07:31:07 -05:00
Richard van der Hoff 132b673dbe
Add some type annotations in `synapse.storage` (#6987)
I cracked, and added some type definitions in synapse.storage.
2020-02-27 11:53:40 +00:00
Erik Johnston 1f773eec91
Port PresenceHandler to async/await (#6991) 2020-02-26 15:33:26 +00:00
Patrick Cloke 509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957)
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
Patrick Cloke 10027c80b0
Add type hints to the spam check module (#6915)
Add typing information to the spam checker modules.
2020-02-14 12:49:40 -05:00
Erik Johnston b08b0a22d5
Add typing to synapse.federation.sender (#6871) 2020-02-07 13:56:38 +00:00
Erik Johnston b9391c9575
Add typing to SyncHandler (#6821)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-02-03 18:05:44 +00:00
Erik Johnston c3d4ad8afd
Fix sending server up commands from workers (#6811)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2020-01-30 16:42:11 +00:00
Erik Johnston b0a66ab83c
Fixup synapse.rest to pass mypy (#6732) 2020-01-20 17:38:21 +00:00
Erik Johnston 0f6e525be3
Fixup synapse.api to pass mypy (#6733) 2020-01-20 17:34:13 +00:00
Erik Johnston e8b68a4e4b
Fixup synapse.replication to pass mypy checks (#6667) 2020-01-14 14:08:06 +00:00
Richard van der Hoff 98247c4a0e
Remove unused, undocumented "content repo" resource (#6628)
This looks like it got half-killed back in #888.

Fixes #6567.
2020-01-03 17:10:52 +00:00
Erik Johnston 495005360c Bump version of mypy 2019-12-12 15:21:12 +00:00
Erik Johnston 324d4f61b8 Include more folders in mypy 2019-12-12 14:52:11 +00:00
Amber Brown fdec84aa42
Add benchmarks for structured logging performance (#6266) 2019-12-03 20:21:25 +11:00
Richard van der Hoff c5abb67e43
Python 3.8 for tox (#6341)
... and update INSTALL.md to include py3.8.

We'll also have to update the buildkite pipeline to run it
2019-11-07 17:14:13 +00:00
Amber Brown 020add5099
Update black to 19.10b0 (#6304)
* update version of black and also fix the mypy config being overridden
2019-11-01 02:43:24 +11:00
Andrew Morgan 2cab02f9d1
Update CI to run isort on scripts and scripts-dev (#6270) 2019-10-30 11:17:14 +00:00
Andrew Morgan 9178ac1b6a
Remove redundant arguments to CI's flake8 (#6277) 2019-10-30 11:07:18 +00:00
Erik Johnston 9d8dc85df2 Add comment as to why we're pinning black in tests 2019-10-29 10:50:42 +00:00
Erik Johnston c28a30da84 Pin black version 2019-10-29 10:37:50 +00:00
Andrew Morgan 409c62b27b
Add config linting script that checks for bool casing (#6203)
Add a linting script that enforces all boolean values in the default config be lowercase.

This has annoyed me for a while so I decided to fix it.
2019-10-23 13:22:54 +01:00
Erik Johnston 5859a5c569 Fix presence timeouts when synchrotron restarts. (#6212)
* Fix presence timeouts when synchrotron restarts.

Handling timeouts would fail if there was an external process that had
timed out, e.g. a synchrotron restarting. This was due to a couple of
variable name typoes.

Fixes #3715.
2019-10-18 06:42:26 +01:00
Amber Brown f743108a94
Refactor HomeserverConfig so it can be typechecked (#6137) 2019-10-10 09:39:35 +01:00
Amber Brown b617864cd9
Fix for structured logging tests stomping on logs (#6023) 2019-09-13 02:29:55 +10:00
Amber Brown e7011280c7
Fix coverage in sytest and use plugins for buildkite (#5922) 2019-08-29 22:19:57 +10:00
Amber Brown 7dc398586c
Implement a structured logging output system. (#5680) 2019-08-28 21:18:53 +10:00
Amber Brown 32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Amber Brown 09e9a26b71
Remove Python 2.7 support. (#5425)
* remove 2.7 from CI and publishing

* fill out classifiers and also make it not be installed on 3.5

* some minor bumps so that the old deps work on python 3.5
2019-06-12 21:31:59 +10:00
Richard van der Hoff bab3eddac4
Pin eliot to <1.8 on python 3.5.2 (#5218)
* Pin eliot to <1.8 on python 3.5.2

Fixes https://github.com/matrix-org/synapse/issues/5199

* Add support for 'markers' to python_dependencies

* tell xargs not to strip quotes
2019-05-21 15:58:01 +01:00
Richard van der Hoff 00714e5102 set PIP_USE_PEP517 = False for tests
pip 19.1 otherwise complains about "editable mode is not supported for
pyproject.toml-style projects"
2019-04-25 14:52:19 +01:00
Erik Johnston b0e767f7bb Add comment back in 2019-03-19 12:51:32 +00:00
Erik Johnston 4d25624ff6 Revert changes 2019-03-19 10:28:00 +00:00
Erik Johnston 00d97668bf Bring py27-old into line with other test envs 2019-03-18 17:45:49 +00:00
Erik Johnston b6ac5e40a0 Add coverage to py27-old 2019-03-18 17:38:22 +00:00
Richard van der Hoff 8e28bc5eee
Include a default configuration file in the 'docs' directory. (#4791) 2019-03-04 17:14:58 +00:00
Richard van der Hoff c594cc8076 Run unit tests against python 3.7 (#4677)
* Run unit tests against python 3.7

... so that we span the full range of our supported python versions

* Switch to xenial

* fix psql fail

* pep8 etc want python 3.6
2019-02-20 18:42:24 +11:00
Erik Johnston b201149c7e
Merge pull request #4420 from matrix-org/jaywink/openid-listener
New listener resource for the federation API "openid/userinfo" endpoint
2019-02-11 09:44:00 +00:00
Amber Brown 58f6c48183
Use native UPSERTs where possible (#4306) 2019-01-24 21:31:54 +11:00
Jason Robinson 899e60be80 Add parameterized Python module to test dependencies
Allows running parameterized tests. BSD license.

Signed-off-by: Jason Robinson <jasonr@matrix.org>
2019-01-23 10:32:41 +02:00
Amber Brown c26f49a664
Make the dependencies more like a standard Python project and hook up the optional dependencies to setuptools (#4298) 2018-12-22 01:37:26 +11:00
Richard van der Hoff 0b26feb422 Merge branch 'master' into develop 2018-12-20 16:48:47 +00:00
Richard van der Hoff 1a6d5bfa08 Debian packaging via dh_virtualenv (#4285) 2018-12-20 11:33:29 +00:00
Erik Johnston 25219b7b4e Make isort tox check print diff when it fails 2018-12-10 13:58:26 +00:00
Amber Brown 8ca53fb53e
Report combined coverage to codecov (#4225) 2018-11-28 20:59:31 +11:00
Amber Brown a51288e5d6
Add a coveragerc (#4180) 2018-11-15 10:50:08 -06:00
Amber Brown 264cb14402
Port hash_password to Python 3 (#4161)
* port hash_password

* changelog
2018-11-08 04:57:28 +11:00
Richard van der Hoff 0f6ec6d1ae Attempt to fix tox installs
It seems that, at some point, the ability to run tox on old servers (with old
setuptools) got broken - and it was only working on our Jenkins instance by
dint of reusing the tox environments.

Let's try to get tox to do the right thing, and remove the guff from
jenkins/prepare_synapse.sh.

(There is a separate question about whether the jenkins builds should be using
tox to prepare the virtualenv at all here, but that is somewhat orthogonal).
2018-10-30 21:00:31 +00:00
Richard van der Hoff 77d3b5772f disable coverage checking
I don't think we ever use this, and it slows things down. If we want to use it,
we should just do so on a couple of builds rather than all of them.
2018-10-25 00:36:00 +01:00
Amber Brown e1728dfcbe
Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase on py3) (#4068) 2018-10-20 11:16:55 +11:00
Amber Brown b3064532d0
Run our oldest supported configuration in CI (#3952) 2018-09-27 23:21:54 +10:00
Amber Brown 1f3f5fcf52
Fix client IPs being broken on Python 3 (#3908) 2018-09-20 20:14:34 +10:00
Amber Brown 741571cf22
Add a way to run tests in PostgreSQL in Docker (#3699) 2018-09-20 18:12:45 +10:00
Amber Brown 7c27c4d51c
merge (#3576) 2018-09-14 03:11:11 +10:00
Amber Brown 99dd975dae
Run tests under PostgreSQL (#3423) 2018-08-13 16:47:46 +10:00
Krombel 9677b1d1c0 rename 'isort' to 'check_isort' as requested 2018-07-16 16:03:41 +02:00
Krombel 4a27000548 check isort by travis 2018-07-16 13:57:33 +02:00
Amber Brown 36cb570641
Use towncrier to build the changelog (#3425) 2018-06-25 14:42:27 +01:00
Krombel 0c87eed294 update tox.ini to cover 292 succeeding tests
Signed-Off-By: Matthias Kesler <krombel@krombel.de>
2018-06-05 23:10:15 +02:00
Krombel d6cd55532b extend tox testing for py3 to avoid regressions 2018-05-29 16:02:41 +02:00
Amber Brown 4abeaedcf3 tests/metrics is gone now 2018-05-22 16:45:11 -05:00
Adrian Tschira 6e005d1382 run config tests on py3
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-30 10:39:45 +02:00
Richard van der Hoff 7767a9fc0e
Update tox.ini
add missing comma
2018-04-30 00:37:32 +01:00
Adrian Tschira 4f5694e2ce Add py3 tests to tox with folders that work
It's just a few tests, but it will at least prevent a few files from
regressing. Also, it makes it easiert to check your code against py36
while writing it.

Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-27 16:29:41 +02:00
Richard van der Hoff 4dd61df6f8 do tox install with pip -e
- this ensures we end up with a working virtualenv which we can use for other
things.
2017-09-05 16:35:23 +01:00
Richard van der Hoff 7dae7087d3 Fix unittests under tox
We now need to set PYTHONPATH when running the unit tests; update tox config to
do so.
2016-12-06 11:35:14 +00:00
Bartek Rutkowski 50f69e2ef2 Change /bin/bash to /bin/sh in tox.ini
No features of Bash are used here, so using /bin/sh makes it more portable to systems that don't have Bash natively (like BSD systems).
2016-06-10 11:33:43 +01:00
Mark Haines 700487a7c7 Fix flake8 warnings for tests 2016-02-19 15:34:38 +00:00
Daniel Wagner-Hall 5b75b637b8 Remove pyc files before running tests 2016-02-09 13:44:04 +00:00
Mark Haines 2f871ad143 Generate code coverage report when running jenkins.sh 2015-12-18 20:44:47 +00:00
Daniel Wagner-Hall 09bb5cf02f Output results files on jenkins
Outputs:
 * results.xml
 * coverage.xml
 * violations.flake8.log
2015-11-16 14:35:57 -05:00
Daniel Wagner-Hall 233af7c74b Pull out jenkins script into a checked in script 2015-11-13 16:22:51 +00:00
Mark Haines d58edd98e9 Update the other place check_joined_room is called 2015-10-30 11:15:37 +00:00
Mark Haines 57df6fffa7 Use the version of "matrix-angular-sdk" hosted on pypi 2015-09-01 14:47:57 +01:00
Mark Haines 4f6fa981ec Make 'setup.py test' run tox 2015-08-26 12:45:29 +01:00