Commit Graph

2770 Commits (07cff7b1214060a4505030fc9eb822d974da9610)

Author SHA1 Message Date
Erik Johnston 65f3fbfbf7
Merge pull request #4737 from matrix-org/erikj/failure_log_tb
Log tracebacks correctly
2019-02-25 17:26:30 +00:00
Erik Johnston c7b333c545 Log tracebacks correctly 2019-02-25 16:56:41 +00:00
Erik Johnston 9c598dddcb Fix typo 2019-02-25 16:32:02 +00:00
Andrew Morgan 8aaf7ffc44 syntax derp 2019-02-25 15:27:17 +00:00
Andrew Morgan 84c0a20dfe Simplify call to generate_room_entry 2019-02-25 15:23:27 +00:00
Erik Johnston 4b9e5076c4 Merge branch 'develop' of github.com:matrix-org/synapse into anoa/public_rooms_federate 2019-02-25 15:08:18 +00:00
Andrew Morgan 07493607a8 Docs and arg name clarification 2019-02-25 15:04:46 +00:00
Andrew Morgan bd398b874e Don't restrict non-fed rooms over client APIs 2019-02-25 15:04:46 +00:00
Andrew Morgan e4b078a600 Config option to prevent showing non-fed rooms in fed /publicRooms 2019-02-25 15:04:46 +00:00
Erik Johnston d730c2c22b More comments 2019-02-25 14:45:02 +00:00
Erik Johnston 890cb048fd Assert rather than clobber the values 2019-02-25 14:42:11 +00:00
Erik Johnston 9342cc6ab1 Add comments and paranoia 2019-02-25 10:02:12 +00:00
Erik Johnston 7b288826b7 Fix backfill storing incorrect state for events 2019-02-22 11:33:51 +00:00
Richard van der Hoff 6d65659b62
Run push_receipts_to_remotes as background job (#4707)
I suspect the CPU usage metrics for this are going to /dev/null at the moment.
2019-02-21 17:50:30 +00:00
Erik Johnston b9d6756b14
Merge pull request #4263 from rkfg/develop
Prevent crash on pagination.
2019-02-21 17:42:15 +00:00
Erik Johnston 71b625d808 Stop backpaginating when events not visible 2019-02-20 18:14:12 +00:00
Erik Johnston dbdc565dfd Fix registration on workers (#4682)
* Move RegistrationHandler init to HomeServer

* Move post registration actions to RegistrationHandler

* Add post regisration replication endpoint

* Newsfile
2019-02-20 18:47:31 +11:00
Brendan Abolivier a288bdf0b1
Merge pull request #4652 from matrix-org/babolivier/acme-delegated
Support .well-known delegation when issuing certificates through ACME
2019-02-19 11:15:38 +00:00
Brendan Abolivier a8626901cd Fetch ACME domain into an instance member 2019-02-19 10:54:33 +00:00
Andrew Morgan 968a30a75c
Merge pull request #4642 from matrix-org/anoa/bans_room_upgrade
Transfer bans on room upgrade
2019-02-19 10:16:42 +00:00
Andrew Morgan 34ac75ce2c lint 2019-02-18 18:23:37 +00:00
Andrew Morgan f8b9ca53ce Move member event processing and changelog fix 2019-02-18 17:28:52 +00:00
Erik Johnston fc2c245a1f
Merge pull request #4666 from matrix-org/erikj/register_login_split
Split out registration to worker
2019-02-18 17:18:06 +00:00
Erik Johnston 459d3d5046
Merge pull request #4668 from matrix-org/erikj/catch_exceptions
Correctly handle HttpResponseException when handling device updates
2019-02-18 16:55:43 +00:00
Erik Johnston af691e415c Move register_device into handler 2019-02-18 16:49:38 +00:00
Erik Johnston c30f73c86a
Merge pull request #4667 from matrix-org/erikj/fix_revoke_guest_access_workers
Fix kicking guest users in worker mode
2019-02-18 15:55:48 +00:00
Brendan Abolivier 45bb55c6de Use a configuration parameter to give the domain to generate a certificate for 2019-02-18 15:46:23 +00:00
Erik Johnston 94960cef03 pep8 2019-02-18 15:24:13 +00:00
Erik Johnston e07cc31cb8 Correctly handle HttpResponseException 2019-02-18 14:55:09 +00:00
Brendan Abolivier 68a53f825f Merge branch 'develop' into babolivier/acme-delegated 2019-02-18 14:52:23 +00:00
Erik Johnston 32e54b472a Fix kicking guest users in worker mode
When guest_access changes from allowed to forbidden all local guest
users should be kicked from the room. This did not happen when
revocation was received from federation on a worker.

Presumably broken in #4141
2019-02-18 14:16:07 +00:00
Andrew Morgan 915421065b Membership events are done later 2019-02-18 14:02:09 +00:00
Erik Johnston d1b060b492
Merge pull request #4643 from matrix-org/erikj/catch_exceptions
Correctly handle RequestSendFailed exceptions
2019-02-18 13:53:24 +00:00
Andrew Morgan 9caab0c364 Transfer bans on room upgrade 2019-02-18 13:52:51 +00:00
Erik Johnston eb2b8523ae Split out registration to worker
This allows registration to be handled by a worker, though the actual
write to the database still happens on master.

Note: due to the in-memory session map all registration requests must be
handled by the same worker.
2019-02-18 12:12:57 +00:00
Andrew Morgan 6d02a13d81
Typo in info log
Co-Authored-By: babolivier <contact@brendanabolivier.com>
2019-02-18 11:36:34 +00:00
Brendan Abolivier f86b695cbd
Various cosmetics to make TravisCI happy 2019-02-15 12:29:34 +00:00
Brendan Abolivier af8a2f679b
Remove unused import 2019-02-15 12:27:43 +00:00
Brendan Abolivier 1895d14e12
Support .well-known delegation when issuing certificates through ACME 2019-02-15 12:05:08 +00:00
Erik Johnston f61b2068e6 Only fetch aliases when publishing rooms 2019-02-14 18:08:09 +00:00
Erik Johnston eaf4d11af9 Add configurable room list publishing rules
This allows specifying who and what is allowed to be published onto the
public room list
2019-02-14 16:02:23 +00:00
Erik Johnston 7fc1196a36 Correctly handle RequestSendFailed exceptions
This mainly reduces the number of exceptions we log.
2019-02-14 14:01:04 +00:00
Amber Brown bb4fd8f927
Run `black` on user directory code (#4635) 2019-02-13 23:05:32 +11:00
Erik Johnston ba3f27b69a
Merge pull request #4608 from matrix-org/anoa/acls_room_upgrade
Transfer Server ACLs on room upgrade
2019-02-12 13:20:06 +00:00
Andrew Morgan a126f86eec Transfer Server ACLs on room upgrade 2019-02-11 11:30:37 +00:00
Hubert Chathi afae8442b5 make sure version is in body and wrap in linearizer queue
also add tests
2019-02-08 01:32:45 -05:00
Hubert Chathi d9e424bf64 re-try to make isort happy 2019-02-06 22:18:41 -05:00
Hubert Chathi 9ff620a518 fix import to make isort happy 2019-02-06 21:32:52 -05:00
Hubert Chathi 8248637173 add new endpoint to update backup versions 2019-02-06 17:57:10 -05:00
Hubert Chathi 664c81e8b7 return proper error codes for some 404s 2019-02-06 17:47:22 -05:00
Andrew Morgan c45fd0dda0
Merge pull request #4530 from matrix-org/anoa/room_upgrade_federatable
Copy over non-federatable trait on room upgrade
2019-01-31 20:28:37 +00:00
Richard van der Hoff 07dfe148de
Add some debug for membership syncing issues (#4538)
I can't figure out what's going on with #4422 and #4436; perhaps this will help.
2019-01-31 18:30:40 +00:00
Andrew Morgan 3ed3cb4339 New function for getting room's create event 2019-01-31 18:21:39 +00:00
Andrew Morgan bbb97a35fd Merge branch 'develop' of github.com:matrix-org/synapse into anoa/room_upgrade_federatable 2019-01-31 18:11:00 +00:00
Andrew Morgan fb50934b8f lint 2019-01-31 11:34:45 +00:00
Andrew Morgan d621c5562e Copy over non-federatable trait on room upgrade 2019-01-30 16:33:51 +00:00
Erik Johnston ed8c5e4cda Fix remote invite rejections not comming down sync
This was broken in PR #4405, commit 886e5ac, where we changed remote
rejections to be outliers.

The fix is to explicitly add the leave event in when we know its an out
of band invite. We can't always add the event as if the server is/was in
the room there might be more events to send down the sync than just the
leave.
2019-01-30 15:46:27 +00:00
Richard van der Hoff 7615a8ced1 ACME config cleanups (#4525)
* Handle listening for ACME requests on IPv6 addresses

the weird url-but-not-actually-a-url-string doesn't handle IPv6 addresses
without extra quoting. Building a string which you are about to parse again
seems like a weird choice. Let's just use listenTCP, which is consistent with
what we do elsewhere.

* Clean up the default ACME config

make it look a bit more consistent with everything else, and tweak the defaults
to listen on port 80.

* newsfile
2019-01-30 14:17:55 +00:00
Amber Brown 3f189c902e
Fix flake8 (#4519) 2019-01-30 10:53:17 +00:00
Erik Johnston 67b82f1336 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah 2019-01-29 22:58:38 +00:00
Erik Johnston a696c48133 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/redactions_eiah 2019-01-29 22:00:33 +00:00
Erik Johnston 55d9024835 Use snder and not event ID domain to check if ours
The transaction queue only sends out events that we generate. This was
done by checking domain of event ID, but that can no longer be used.
Instead, we may as well use the sender field.
2019-01-29 16:54:23 +00:00
Erik Johnston b1fffca345 Remove event ID usage when checking if new room
The event ID is changing, so we can no longer get the domain from it. On
the other hand, the check is unnecessary.
2019-01-29 16:15:02 +00:00
Erik Johnston b8d75ef53e
Merge pull request #4481 from matrix-org/erikj/event_builder
Refactor event building into EventBuilder
2019-01-29 14:07:23 +00:00
Erik Johnston 5891a6edc8 Correctly set context.app_service 2019-01-29 12:09:10 +00:00
Erik Johnston 7709d2bd16 Implement rechecking of redactions 2019-01-29 11:56:20 +00:00
Erik Johnston 6598992b01 Fixup calls to `comput_event_signature`
We currently pass FrozenEvent instead of `dict` to
`compute_event_signature`, which works by accident due to `dict(event)`
producing the correct result.

This fixes PR #4493 commit 855a151
2019-01-29 11:41:58 +00:00
Erik Johnston be47cfa9c9 Refactor event building into EventBuilder
This is so that everything is done in one place, making it easier to
change the event format based on room version
2019-01-29 11:13:00 +00:00
Erik Johnston 073f6c2e5e
Merge pull request #4494 from matrix-org/erikj/fixup_event_validator
Split up event validation between event and builder
2019-01-29 10:55:07 +00:00
Erik Johnston f1a04462eb
Merge pull request #4482 from matrix-org/erikj/event_auth_room_version
Pass through room version to event auth
2019-01-28 20:09:38 +00:00
Erik Johnston d758d5310e Correctly use default room version if none is set 2019-01-28 17:26:39 +00:00
Erik Johnston b872c7b1b4 Split up event validation between event and builder
The validator was being run on the EventBuilder objects, and so the
validator only checked a subset of fields. With the upcoming
EventBuilder refactor even fewer fields will be there to validate.

To get around this we split the validation into those that can be run
against an EventBuilder and those run against a fully fledged event.
2019-01-28 17:00:14 +00:00
Andrew Morgan f0e96ab66a Change return syntax in doc string 2019-01-28 14:09:45 +00:00
Andrew Morgan 4026d555fa Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-28 14:08:24 +00:00
Andrew Morgan 1ce463963d Reuse predecessor method 2019-01-28 14:08:18 +00:00
Erik Johnston ae2a957dba Pass through room version to event auth 2019-01-25 18:31:41 +00:00
Erik Johnston b6dce9b9fd
Merge pull request #4470 from matrix-org/erikj/require_format_version
Require event format version to parse or create events
2019-01-25 15:59:36 +00:00
Andrew Morgan 9244a3089e Fixes 2019-01-25 11:48:38 +00:00
Andrew Morgan da0d2219d2 Clean up direct_rooms access 2019-01-25 11:37:12 +00:00
Andrew Morgan 8265995498 Use python magic 2019-01-25 11:26:06 +00:00
Andrew Morgan c4cdafa81f Destructure account data tuple before use 2019-01-25 11:24:28 +00:00
Andrew Morgan 516456b763 Remove unnecessary null check 2019-01-25 11:22:14 +00:00
Andrew Morgan 6f3fda79ce Move room_tag declaration to be closer to its use 2019-01-25 11:21:25 +00:00
Andrew Morgan 821b65aeb5 Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-25 11:09:53 +00:00
Andrew Morgan 0862d35b8e Move tag and direct state copying into separate function 2019-01-25 11:09:34 +00:00
Andrew Morgan b1b6dba2d2
Merge pull request #4415 from matrix-org/anoa/full_search_upgraded_rooms
Ability to search entire room history after upgrading room
2019-01-25 10:49:58 +00:00
Erik Johnston a50cf929c1 Require event format version to parse or create events 2019-01-25 10:32:19 +00:00
Erik Johnston 62514bb81b Merge branch 'develop' of github.com:matrix-org/synapse into erikj/msc_1813 2019-01-25 10:07:08 +00:00
Erik Johnston edc1e21dbe
Merge pull request #4448 from matrix-org/erikj/get_pdu_versions
Add room_version param to get_pdu
2019-01-24 18:47:15 +00:00
Erik Johnston 80bcca659e
Merge pull request #4405 from matrix-org/erikj/fixup_rejecting_invites
Store rejected remote invite events as outliers
2019-01-24 18:33:27 +00:00
Erik Johnston d148c43050 Review comments 2019-01-24 18:31:23 +00:00
Erik Johnston 9139b87be4 Remove unecessary setting of outlier bit 2019-01-24 18:04:02 +00:00
Erik Johnston b8082a5445 Use term 'out of band membership' instead 2019-01-24 17:33:19 +00:00
Richard van der Hoff 03c85335d1
Apply suggestions from code review
Co-Authored-By: anoadragon453 <1342360+anoadragon453@users.noreply.github.com>
2019-01-24 17:22:09 +00:00
Andrew Morgan 075ff3ede9 Change default timeout value from 0 to None 2019-01-24 15:10:22 +00:00
Andrew Morgan 5b1dc94083 Use self.clock instead of datetime 2019-01-24 14:59:50 +00:00
Andrew Morgan a2d85144e5 isort 2019-01-24 14:22:26 +00:00
Andrew Morgan 5541645e80 lint 2019-01-24 12:45:32 +00:00
Andrew Morgan 068aa1d228 Time out filtered room dir queries after 60s 2019-01-24 12:44:27 +00:00
Erik Johnston 7c288c2250 Clarify the invite flows 2019-01-23 20:07:47 +00:00
Erik Johnston 07f62da55a Remove unnecessary '_sign_event' 2019-01-23 20:04:57 +00:00
Erik Johnston 886e5acc76 Store rejected remote invite events as outliers
Currently they're stored as non-outliers even though the server isn't in
the room, which can be problematic in places where the code assumes it
has the state for all non outlier events.

In particular, there is an edge case where persisting the leave event
triggers a state resolution, which requires looking up the room version
from state. Since the server doesn't have the state, this causes an
exception to be thrown.
2019-01-23 20:04:57 +00:00
Erik Johnston 6a41d2a187 Add room_version param to get_pdu
When we add new event format we'll need to know the event format or room
version when parsing events.
2019-01-23 17:19:58 +00:00
Erik Johnston 67cd4dad81 Implement MSC 1813 - Add room version to make APIs
We also implement `make_membership_event` converting the returned
room version to an event format version.
2019-01-23 16:51:46 +00:00
Andrew Morgan 82a92ba535 Add metric for user dir current event stream position 2019-01-23 15:34:47 +00:00
Amber Brown 6129e52f43
Support ACME for certificate provisioning (#4384) 2019-01-23 19:39:06 +11:00
Andrew Morgan 766a172b99 lint 2019-01-22 13:51:40 +00:00
Andrew Morgan c4875d8c76 Prevent duplicate room IDs in m.direct 2019-01-22 12:13:46 +00:00
Andrew Morgan c9bfb058d8 Fix a bug with single-room search searching all rooms
* Create a new method for getting predecessor rooms
* Remove formatting change
2019-01-22 12:00:41 +00:00
Andrew Morgan 8086a5c05e Fix comments 2019-01-22 11:16:23 +00:00
Andrew Morgan 48951f437f Join logic covers both room creator and arbitrary users 2019-01-22 11:00:04 +00:00
Andrew Morgan 8c85f0833d tags, m.direct copying over correctly 2019-01-22 11:00:04 +00:00
Andrew Morgan 25d64a846a Fix typos 2019-01-22 11:00:04 +00:00
Andrew Morgan ea8903fcc9 Migrating dm and room tags work for migrator 2019-01-22 11:00:04 +00:00
Andrew Morgan 887ca93a1b Prevent crash on user who doesn't have any direct rooms 2019-01-22 11:00:04 +00:00
Andrew Morgan 4ff6d22245 Preserve DM status of a room on upgrade
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
2019-01-22 11:00:04 +00:00
Andrew Morgan 75942af1db Fix typo 2019-01-22 11:00:04 +00:00
Andrew Morgan 702c4b750c Migrate encryption state on room upgrade (#4411)
* Migrate encryption state on room upgrade

Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>

* Add changelog file
2019-01-21 20:42:58 +11:00
Andrew Morgan df3a661e4a Search for messages across predecessor rooms
Signed-off-by: Andrew Morgan <andrew@amorgan.xyz>
2019-01-18 11:19:20 +00:00
Richard van der Hoff 9feb5d0b71
sign_request -> build_auth_headers (#4408)
Just got very confused about the fact that the headers are only an output, not
an input.
2019-01-17 12:40:09 +00:00
Richard van der Hoff 05e1296649
don't store more remote device lists if they have more than 1K devices (#4397) 2019-01-16 23:14:11 +00:00
Matthew Hodgson 27128145e6 fix NPE in /messages by checking if all events were filtered out (#4330) 2019-01-02 23:37:39 +00:00
Nathan Pennie 81b513416e
Fixed line length 2018-12-19 14:12:33 -05:00
Nathan Pennie 668e6625b0
Fix the variable names used for account_data 2018-12-19 06:53:02 -05:00
Neil Johnson d2f7c4e6b1
create support user (#4141)
Allow for the creation of a support user.

A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
2018-12-14 18:20:59 +00:00
Richard van der Hoff 30da50a5b8
Initialise user displayname from SAML2 data (#4272)
When we register a new user from SAML2 data, initialise their displayname
correctly.
2018-12-07 14:44:46 +01:00
Richard van der Hoff c588b9b9e4
Factor SSO success handling out of CAS login (#4264)
This is mostly factoring out the post-CAS-login code to somewhere we can reuse
it for other SSO flows, but it also fixes the userid mapping while we're at it.
2018-12-07 13:10:07 +01:00
rkfg ae19a7db8c Prevent crash on pagination. 2018-12-06 13:32:05 +03:00
Richard van der Hoff b0c24a66ec Rip out half-implemented m.login.saml2 support (#4265)
* Rip out half-implemented m.login.saml2 support

This was implemented in an odd way that left most of the work to the client, in
a way that I really didn't understand. It's going to be a pain to maintain, so
let's start by ripping it out.

* drop undocumented dependency on dateutil

It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
2018-12-06 19:44:38 +11:00
Travis Ralston 158ffb92f1 Add an option to disable search for homeservers which may not be interested in it (#4230)
This is useful for homeservers not intended for users, such as bot-only homeservers or ones that only process IoT data.
2018-12-04 12:01:02 +01:00
Neil Johnson 7039ece8fb Neilj/fix autojoin (#4223)
* Fix auto join failures for servers that require user consent

* Fix auto join failures for servers that require user consent
2018-11-28 22:24:57 +11:00
Travis Ralston ab4526a153 Remove duplicate slashes in generated consent URLs 2018-11-15 20:41:53 -07:00
Erik Johnston dc59ad5334 Remove hack to support rejoining rooms 2018-11-09 14:58:09 +00:00
Erik Johnston 30dd27afff Simplify to always drop events if server isn't in the room 2018-11-09 11:36:45 +00:00
Travis Ralston 0f5e51f726 Add config variables for enabling terms auth and the policy name (#4142)
So people can still collect consent the old way if they want to.
2018-11-06 10:32:34 +00:00
Hubert Chathi f1087106cf
handle empty backups according to latest spec proposal (#4123)
fixes #4056
2018-11-05 17:59:29 -05:00
Erik Johnston bc80b3f454 Add helpers for getting prev and auth events (#4139)
* Add helpers for getting prev and auth events

This is in preparation for allowing the event format to change between
room versions.
2018-11-06 00:35:15 +11:00
Amber Brown cb7a6b2379
Fix typing being reset causing infinite syncs (#4127) 2018-11-03 00:19:23 +11:00
Travis Ralston a8d41c6aff Include a version query string arg for the consent route 2018-10-31 13:19:28 -06:00
Travis Ralston d1e7b9c44c Merge branch 'develop' into travis/login-terms 2018-10-31 13:15:14 -06:00
Richard van der Hoff 1729ba1650
Merge pull request #4101 from matrix-org/rav/aliases_for_upgrades
Attempt to move room aliases on room upgrades
2018-10-31 17:52:18 +00:00
Richard van der Hoff 0f8591a5a8 Avoid else clause on exception for clarity 2018-10-31 15:43:57 +00:00
Richard van der Hoff 94c7fadc98 Attempt to move room aliases on room upgrades 2018-10-31 15:43:57 +00:00
Richard van der Hoff 9b827c40ca
Log some bits about event creation (#4121)
I found these helpful in debugging my room upgrade tests.
2018-10-31 15:42:23 +00:00
Amber Brown 086e1a8f3e Fix search 500ing (#4122) 2018-10-31 06:26:46 +11:00
Amber Brown 3bade14ec0
Fix search 500ing (#4122) 2018-10-31 04:33:41 +11:00
Amber Brown 4cd1c9f2ff
Delete the disused & unspecced identicon functionality (#4106) 2018-10-29 23:57:24 +11:00
Richard van der Hoff 7fbfea062e
Merge pull request #4100 from matrix-org/rav/room_upgrade_avatar
Remember to copy the avatar on room upgrades
2018-10-29 12:49:21 +00:00
Richard van der Hoff db24d7f15e Better handling of odd PLs during room upgrades
Fixes handling of rooms where we have permission to send the tombstone, but not
other state. We need to (a) fail more gracefully when we can't send the PLs in
the old room, and (b) not set the PLs in the new room until we are done with
the other stuff.
2018-10-27 00:54:26 +01:00
Richard van der Hoff 5caf79b312 Remember to copy the avatar on room upgrades 2018-10-26 23:56:40 +01:00