332839f6ea
Update federation client pokes
2017-07-18 14:45:37 +01:00
e5ea6dd021
Add client apis
2017-07-18 14:37:06 +01:00
cccfcfa7b9
Comments
2017-07-18 10:35:18 +01:00
68f34e85ce
Use transport client directly
2017-07-18 10:29:57 +01:00
2f9eafdd36
Add local group server support
2017-07-17 12:03:49 +01:00
e9aec001f4
Use less DB for device list handling in sync
2017-07-12 10:30:10 +01:00
28e8c46f29
Merge pull request #2352 from matrix-org/erikj/group_server_split
...
Initial Group Server
2017-07-12 10:26:29 +01:00
9a6fd3ef29
Don't compute push actions for backfilled events
2017-07-11 10:02:21 +01:00
b8ca494ee9
Initial group server implementation
2017-07-10 15:44:15 +01:00
5a7f561a9b
Fix bug where pusherpool didn't start and broke some rooms
...
Since we didn't instansiate the PusherPool at start time it could fail
at run time, which it did for some users.
This may or may not fix things for those users, but it should happen at
start time and stop the server from starting.
2017-07-06 17:55:51 +01:00
3e279411fe
Improve the error handling for bad invites received over federation
2017-06-30 16:20:30 +01:00
c72058bcc6
Use an ExpiringCache for storing registration sessions
...
This is because pruning them was a significant performance drain on
matrix.org
2017-06-29 14:08:37 +01:00
816605a137
Merge pull request #2307 from matrix-org/erikj/user_ip_batch
...
Batch upsert user ips
2017-06-27 15:08:32 +01:00
a0a561ae85
Fix up client ips to read from pending data
2017-06-27 14:46:12 +01:00
1bce3e6b35
Remove unused variables
2017-06-26 14:03:27 +01:00
e3cbec10c1
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/ensure_round_trip
2017-06-26 14:02:44 +01:00
44c722931b
Make some more params configurable
2017-06-22 14:59:52 +01:00
2d520a9826
Typo. ARGH.
2017-06-22 14:42:39 +01:00
24d894e2e2
Fix thinko in unhandled user spam
2017-06-22 14:39:05 +01:00
1217c7da91
Don't work out users who share room with appservice users
2017-06-21 12:00:41 +01:00
e5ae386ea4
Handle all cases of sending membership events
2017-06-19 16:07:54 +01:00
b490299a3b
Change to create new room and join other users
2017-06-19 14:10:13 +01:00
d7fe6b356c
Add shutdown room API
2017-06-19 12:37:27 +01:00
fcf01dd88e
Reject local events that don't round trip the DB
2017-06-19 11:33:40 +01:00
6aa5bc8635
Initial worker impl
2017-06-16 11:47:11 +01:00
a9d6fa8b2b
Include users who share room with requester in user directory
2017-06-15 10:17:21 +01:00
4564b05483
Implement updating users who share rooms on the fly
2017-06-15 10:17:17 +01:00
72613bc379
Implement initial population of users who share rooms table
2017-06-15 09:59:04 +01:00
d9fd937e39
Fix user directory insertion due to missing room_id
2017-06-13 11:50:24 +01:00
6ae8373d40
Don't assume existance of events when updating user directory
2017-06-13 10:19:26 +01:00
a837765e8c
Merge pull request #2266 from matrix-org/erikj/host_in_room
...
Change is_host_joined to use current_state table
2017-06-12 09:49:51 +01:00
1664948e41
Comment
2017-06-09 13:05:05 +01:00
0185b75381
Change is_host_joined to use current_state table
...
This bypasses a bug where using the state groups to figure out if a host
is in a room sometimes errors if the servers isn't in the room. (For
example when the server rejected an invite to a remote room)
2017-06-09 10:52:26 +01:00
7132e5cdff
Mark remote invite rejections as outliers
2017-06-09 10:08:18 +01:00
f45f07ab86
Merge pull request #2258 from matrix-org/erikj/user_dir
...
Don't start user_directory handling on workers
2017-06-07 14:04:50 +01:00
a053ff3979
Merge pull request #2248 from matrix-org/erikj/state_fixup
...
Faster cache for get_joined_hosts
2017-06-07 14:01:06 +01:00
ecdd2a3658
Don't start user_directory handling on workers
2017-06-07 12:02:53 +01:00
2f34ad31ac
Add some logging to user directory
2017-06-07 11:50:44 +01:00
09e4bc0501
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/state_fixup
2017-06-07 11:05:23 +01:00
1a01af079e
Handle profile updates in user directory
2017-06-01 15:39:51 +01:00
21e255a8f1
Split the table in two
2017-06-01 14:50:46 +01:00
7233341eac
Comments
2017-06-01 13:11:38 +01:00
8be6fd95a3
Check if host is still in room
2017-06-01 13:05:39 +01:00
9c7db2491b
Fix removing users
2017-06-01 11:36:50 +01:00
0fe6f3c521
Bug fixes and logging
...
- Check if room is public when a user joins before adding to user dir
- Fix typo of field name "content.join_rules" -> "content.join_rule"
2017-06-01 11:09:49 +01:00
cc7609aa9f
Comment briefly on how we keep user_directory up to date
2017-05-31 17:11:18 +01:00
b2d8d07109
Lifts things into separate function
2017-05-31 17:00:24 +01:00
f9791498ae
Typos
2017-05-31 16:50:57 +01:00
f091061711
Fix tests
2017-05-31 16:34:40 +01:00
dc51af3d03
Pull max id from correct table
2017-05-31 15:13:49 +01:00
350622a107
Handle the server leaving a public room
2017-05-31 15:11:36 +01:00
63fda37e20
Add comments
2017-05-31 15:00:29 +01:00
3b5f22ca40
Add search
2017-05-31 14:00:01 +01:00
b5db4ed5f6
Update room column when room becomes unpublic
2017-05-31 13:40:28 +01:00
168524543f
Add call later
2017-05-31 11:59:36 +01:00
3e123b8497
Start later
2017-05-31 11:56:27 +01:00
42137efde7
Don't go round in circles
2017-05-31 11:55:13 +01:00
eeb2f9e546
Add user_directory to database
2017-05-31 11:51:01 +01:00
23da638360
Fix typing tests
2017-05-26 10:02:04 +01:00
11f139a647
Merge pull request #2247 from matrix-org/erikj/auth_event
...
Only store event_auth for state events
2017-05-24 16:46:34 +01:00
c049472b8a
Only store event_auth for state events
2017-05-24 15:23:31 +01:00
b4f59c7e27
Add count of one time keys to sync stream
2017-05-19 15:47:55 +01:00
cafe659f72
Store ActionGenerator in HomeServer
2017-05-19 10:09:56 +01:00
66d8ffabbd
Faster push rule calculation via push specific cache
...
We add a push rule specific cache that ensures that we can reuse
calculated push rules appropriately when a user join/leaves.
2017-05-17 16:55:40 +01:00
6fa8148ccb
Merge pull request #2228 from matrix-org/erikj/speed_up_get_hosts
...
Speed up get_joined_hosts
2017-05-16 17:40:55 +01:00
13f540ef1b
Speed up get_joined_hosts
2017-05-16 16:05:22 +01:00
ec5c4499f4
Make presence use cached users/hosts in room
2017-05-16 16:01:43 +01:00
ccad2ed824
Modify condition on empty localpart
2017-05-10 17:34:30 +01:00
369195caa5
Modify register/available to be GET with query param
...
- GET is now the method for register/available
- a query parameter "username" is now used
Also, empty usernames are now handled with an error message on registration or via register/available: `User ID cannot be empty`
2017-05-10 17:23:55 +01:00
a3648f84b2
Merge pull request #2208 from matrix-org/erikj/ratelimit_overrid
...
Add per user ratelimiting overrides
2017-05-10 15:54:48 +01:00
b990b2fce5
Add per user ratelimiting overrides
2017-05-10 11:05:43 +01:00
aedaba018f
Replace some instances of preserve_context_over_deferred
2017-05-09 19:04:56 +01:00
de042b3b88
Do some logging when one-time-keys get claimed
...
might help us figure out if https://github.com/vector-im/riot-web/issues/3868
has happened.
2017-05-09 19:04:56 +01:00
a7e9d8762d
Allow clients to upload one-time-keys with new sigs
...
When a client retries a key upload, don't give an error if the signature has
changed (but the key is the same).
Fixes https://github.com/vector-im/riot-android/issues/1208 , hopefully.
2017-05-09 19:04:56 +01:00
27c28eaa27
Merge pull request #2190 from matrix-org/erikj/mark_remote_as_back_more
...
Always mark remotes as up if we receive a signed request from them
2017-05-05 14:08:12 +01:00
653d90c1a5
Comment
2017-05-05 14:01:17 +01:00
7b222fc56e
Remove redundant reset of destination timers
2017-05-05 11:14:09 +01:00
b843631d71
Add comment and TODO
2017-05-05 10:59:32 +01:00
f346048a6e
Handle exceptions thrown in handling remote device list updates
2017-05-05 10:34:10 +01:00
60833c8978
Merge pull request #2147 from matrix-org/dbkr/http_request_propagate_error
...
Propagate errors sensibly from proxied IS requests
2017-05-03 11:23:25 +01:00
0c27383dd7
Merge pull request #2170 from matrix-org/erikj/fed_hole_state
...
Don't fetch state for missing events that we fetched
2017-05-03 10:49:37 +01:00
ef862186dd
Merge together redundant calculations/logging
2017-05-03 10:06:43 +01:00
3e5a62ecd8
Add more granular event send metrics
2017-05-02 14:23:26 +01:00
2347efc065
Fixup
2017-04-28 12:46:53 +01:00
9b147cd730
Remove unncessary call in _get_missing_events_for_pdu
2017-04-28 11:55:25 +01:00
3a9f5bf6dd
Don't fetch state for missing events that we fetched
2017-04-28 11:26:46 +01:00
25a96e0c63
Merge pull request #2163 from matrix-org/erikj/fix_invite_state
...
Fix invite state to always include all events
2017-04-27 17:36:30 +01:00
87ae59f5e9
Typo
2017-04-27 15:16:21 +01:00
e42b4ebf0f
Add some extra logging for edge cases of federation
2017-04-27 14:38:21 +01:00
34e682d385
Fix invite state to always include all events
2017-04-26 16:18:08 +01:00
82ae0238f9
Revert accidental commit
2017-04-26 11:43:16 +01:00
81804909d3
Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error
2017-04-26 11:31:55 +01:00
1a9255c12e
Use CodeMessageException subclass instead
...
Parse json errors from get_json client methods and throw special
errors.
2017-04-25 19:30:55 +01:00
2e165295b7
Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error
2017-04-21 11:35:52 +01:00
a90a0f5c8a
Propagate errors sensibly from proxied IS requests
...
When we're proxying Matrix endpoints, parse out Matrix error
responses and turn them into SynapseErrors so they can be
propagated sensibly upstream.
2017-04-21 11:32:48 +01:00
0cdb32fc43
Remove redundant try/except clauses
...
The `except SynapseError` clauses were pointless because the wrapped functions
would never throw a `SynapseError` (they either throw a `CodeMessageException`
or a `RuntimeError`).
The `except CodeMessageException` is now also pointless because the caller
treats all exceptions equally, so we may as well just throw the
`CodeMessageException`.
2017-04-21 01:32:01 +01:00
838810b76a
Broaden the conditions for locally_rejecting invites
...
The logic for marking invites as locally rejected was all well and good, but
didn't happen when the remote server returned a 500, or wasn't reachable, or
had no DNS, or whatever.
Just expand the except clause to catch everything.
Fixes https://github.com/matrix-org/synapse/issues/761 .
2017-04-21 01:31:37 +01:00
736b9a4784
Remove redundant function
...
inline `reject_remote_invite`, which only existed to make tracing the callflow
more difficult.
2017-04-21 01:31:09 +01:00
3fb8784c92
m.read_marker -> m.fully_read ( #2128 )
...
Also:
- change the REST endpoint to have a "S" on the end (so it's now /read_markers)
- change the content of the m.read_up_to event to have the key "event_id" instead of "marker".
2017-04-18 17:46:15 +01:00
78f0ddbfad
Merge pull request #2120 from matrix-org/luke/read-markers
...
Implement Read Marker API
2017-04-13 14:21:31 +01:00
b9557064bf
Simplify is_event_after logic
2017-04-12 14:36:20 +01:00
247c736b9b
Merge pull request #2115 from matrix-org/erikj/dedupe_federation_repl
...
Reduce federation replication traffic
2017-04-12 11:07:13 +01:00
69a18514e9
Only notify user, not entire room
2017-04-12 10:50:37 +01:00
122cd52ce4
Remove comment, simplify null-guard
2017-04-12 10:48:32 +01:00
c7ddb5ef7a
Reuse get_interested_parties
2017-04-12 10:16:26 +01:00
77fb2b72ae
Handle no previous RM
2017-04-12 09:47:29 +01:00
867822fa1e
flake8
2017-04-11 17:36:04 +01:00
73880268ef
Refactor event ordering check to events store
2017-04-11 17:34:09 +01:00
131485ef66
Copyright
2017-04-11 17:33:51 +01:00
0127423027
flake8
2017-04-11 17:07:07 +01:00
414522aed5
Move get_interested_parties
2017-04-11 15:33:26 +01:00
2be8a281d2
Comments
2017-04-11 15:28:24 +01:00
6308ac45b0
Move get_interested_remotes back to presence handler
2017-04-11 15:19:26 +01:00
d892079844
Finish implementing RM endpoint
...
- This change causes a 405 to be sent if "m.read_marker" is set via /account_data
- This also fixes-up the RM endpoint so that it actually Works.
2017-04-11 15:01:39 +01:00
e263c26690
Initial commit of RM server-side impl
...
(See https://docs.google.com/document/d/1UWqdS-e1sdwkLDUY0wA4gZyIkRp-ekjsLZ8k6g_Zvso/edit#heading=h.lndohpg8at5u )
2017-04-11 11:55:30 +01:00
f3cf3ff8b6
Merge branch 'master' of github.com:matrix-org/synapse into develop
2017-04-11 11:13:32 +01:00
29574fd5b3
Reduce federation presence replication traffic
...
This is mainly done by moving the calculation of where to send presence
updates from the presence handler to the transaction queue, so we only
need to send the presence event (and not the destinations) across the
replication connection. Before we were duplicating by sending the full
state across once per destination.
2017-04-10 16:48:30 +01:00
64765e5199
When we do an invite rejection, save the signed leave event to the db
...
During a rejection of an invite received over federation, we ask a remote
server to make us a `leave` event, then sign it, then send that with
`send_leave`.
We were saving the *unsigned* version of the event (which has a different event
id to the signed version) to our db (and sending it to the clients), whereas
other servers in the room will have seen the *signed* version. We're not aware
of any actual problems that caused, except that it makes the database confusing
to look at and generally leaves the room in a weird state.
2017-04-07 14:39:32 +01:00
98ce212093
Merge pull request #2103 from matrix-org/erikj/no-double-encode
...
Don't double encode replication data
2017-04-07 09:39:52 +01:00
877c029c16
Use iteritems
2017-04-06 15:51:22 +01:00
96b9b6c127
Don't double json encode typing replication data
2017-04-05 11:34:20 +01:00
62b89daac6
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/repl_tcp_server
2017-04-04 09:46:16 +01:00
773e64cc1a
Merge pull request #2095 from matrix-org/rav/cull_log_preserves
...
Cull spurious PreserveLoggingContexts
2017-04-03 17:02:25 +01:00
30bcbf775a
Accept join events from all servers
...
Make sure that we accept join events from any server, rather than just the
origin server, to make the federation join dance work correctly.
(Fixes #1893 ).
2017-04-03 15:58:07 +01:00
7eb9f34cc3
Remove spurious yield
...
In `MessageHandler`, remove `yield` on call to `Notifier.on_new_room_event`:
it doesn't return anything anyway.
2017-04-03 15:44:19 +01:00
0b08c48fc5
Remove more spurious `PreserveLoggingContext`s
...
Remove `PreserveLoggingContext` around calls to `Notifier.on_new_room_event`;
there is no problem if the logcontext is set when calling it.
2017-04-03 15:43:37 +01:00
36d2b66f90
Add a timestamp to USER_SYNC command
...
This timestamp is used to indicate when the user last sync'd
2017-03-31 15:42:22 +01:00
9d0170ac6c
Fix up presence
2017-03-31 11:36:32 +01:00
63fcc42990
Remove user from process_presence when stops syncing
2017-03-30 14:26:08 +01:00
e9dd8370b0
Add functions to presence to support remote syncs
...
The TCP replication protocol streams deltas of who has started or
stopped syncing. This is different from the HTTP API which periodically
sends the full list of users who are syncing. This commit adds support
for the new TCP style of sending deltas.
2017-03-30 13:25:14 +01:00
24d35ab47b
Add new storage functions for new replication
...
The new replication protocol will keep all the streams separate, rather
than muxing multiple streams into one.
2017-03-30 11:48:35 +01:00
7fc1f1e2b6
Cache hosts in room
2017-03-24 11:46:24 +00:00
9397edb28b
Merge pull request #2050 from matrix-org/rav/federation_backoff
...
push federation retry limiter down to matrixfederationclient
2017-03-23 22:27:01 +00:00
5a16cb4bf0
Ignore backoff history for invites, aliases, and roomdirs
...
Add a param to the federation client which lets us ignore historical backoff
data for federation queries, and set it for a handful of operations.
2017-03-23 12:23:22 +00:00
4bd597d9fc
push federation retry limiter down to matrixfederationclient
...
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
2017-03-23 09:28:46 +00:00
e08f81d96a
Add a missing yield in device key upload
...
(this would only very very rarely actually be a useful thing, so the main
problem was the logcontext leak...)
2017-03-23 00:16:43 +00:00
eddce9d74a
Merge pull request #2027 from matrix-org/rav/logcontext_leaks
...
A few fixes to logcontext things
2017-03-20 11:53:36 +00:00
f40c2db05a
Stop preserve_fn leaking context into the reactor
...
Fix a bug in ``logcontext.preserve_fn`` which made it leak context into the
reactor, and add a test for it.
Also, get rid of ``logcontext.reset_context_after_deferred``, which tried to do
the same thing but had its own, different, set of bugs.
2017-03-18 00:07:43 +00:00
5068fb16a5
Refactoring and cleanups
...
A few non-functional changes:
* A bunch of docstrings to document types
* Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more
readable.
* Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid
mind-bending conditional.
* Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid
mind-bending conditional.
2017-03-17 15:06:07 +00:00
2abe85d50e
Merge pull request #2016 from matrix-org/rav/queue_pdus_during_join
...
Queue up federation PDUs while a room join is in progress
2017-03-17 11:32:44 +00:00
da146657c9
Comments
2017-03-16 13:04:07 +00:00
a158c36a8a
Comment
2017-03-16 11:57:45 +00:00
6957bfdca6
Don't recreate so many sets
2017-03-16 11:54:26 +00:00
2ccf3b241c
Implement no op for room stream in sync
2017-03-16 11:06:41 +00:00
9ce53a3861
Queue up federation PDUs while a room join is in progress
...
This just takes the existing `room_queues` logic and moves it out to
`on_receive_pdu` instead of `_process_received_pdu`, which ensures that we
don't start trying to fetch prev_events and whathaveyou until the join has
completed.
2017-03-15 18:01:11 +00:00
54d2b7e596
Merge pull request #2014 from Half-Shot/hs/fix-appservice-presence
...
Add fallback to last_active_ts if it beats the last sync time on a presence timeout.
2017-03-15 17:37:15 +00:00
f83ac78201
Cache set of users whose presence the other user should see
2017-03-15 15:29:19 +00:00
e6032054bf
Add a great comment to handle_timeout for active vs sync times.
2017-03-15 15:24:48 +00:00
ebf5a6b14c
Add fallback to last_active_ts if it beats the last sync time.
2017-03-15 15:17:16 +00:00
e892457a03
Comment
2017-03-15 15:01:39 +00:00
6c82de5100
Format presence events on the edges instead of reformatting them multiple times
2017-03-15 14:27:34 +00:00
7827251daf
Merge pull request #1994 from matrix-org/dbkr/msisdn_signin_2
...
Phone number registration / login support v2
2017-03-15 09:59:54 +00:00
bb256ac96f
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/public_list_speed
2017-03-14 11:35:05 +00:00
bbeeb97f75
Implement _simple_delete_many_txn, use it to delete devices
...
(But this doesn't implement the same for deleting access tokens or e2e keys.
Also respond to code review.
2017-03-13 17:53:23 +00:00
73a5f06652
Support registration / login with phone number
...
Changes from https://github.com/matrix-org/synapse/pull/1971
2017-03-13 17:27:51 +00:00
3c69f32402
Merge remote-tracking branch 'origin/develop' into rav/refactor_received_pdu
2017-03-13 12:20:47 +00:00
0162994983
Comments
2017-03-13 11:53:26 +00:00
7eae6eaa2f
Revert "Support registration & login with phone number"
2017-03-13 09:59:33 +00:00
79926e016e
Assume rooms likely haven't changed
2017-03-13 09:50:10 +00:00
8ffbe43ba1
Get current state by using current_state_events table
2017-03-10 17:39:35 +00:00
64d62e41b8
Noop repated delete device inbox calls from /sync
2017-03-10 10:36:43 +00:00
29235901b8
Move FederationServer._handle_new_pdu to FederationHandler
...
Unfortunately this significantly increases the size of the already-rather-big
FederationHandler, but the code fits more naturally here, and it paves the way
for the tighter integration that I need between handling incoming PDUs and
doing the join dance.
Other than renaming the existing `FederationHandler.on_receive_pdu` to
`_process_received_pdu` to make way for it, this just consists of the move, and
replacing `self.handler` with `self` and `self` with `self.replication_layer`.
2017-03-09 16:20:13 +00:00
663396e45d
Merge pull request #1971 from matrix-org/dbkr/msisdn_signin
...
Support registration & login with phone number
2017-03-09 10:18:51 +00:00
0e0aee25c4
Fix log line
2017-03-08 11:46:22 +00:00
88df6c0c9a
Factor out msisdn canonicalisation
...
Plus a couple of other minor fixes
2017-03-08 11:03:39 +00:00
402a7bf63d
Fix pep8
2017-03-08 09:33:40 +00:00
f2581ee8b8
Don't keep around old stream IDs forever
2017-03-03 16:02:53 +00:00
9834367eea
Spelling
2017-03-03 15:31:57 +00:00
ad882cd54d
Just return the deferred straight off
...
defer.returnValue doth not maketh a generator: it would need a
yield to be a generator, and this doesn't need a yield.
2017-03-01 18:08:51 +00:00
3557cf34dc
Merge remote-tracking branch 'origin/develop' into dbkr/msisdn_signin
2017-03-01 17:20:37 +00:00
36be39b8b3
Fix device list update to not constantly resync
2017-03-01 14:12:11 +00:00
848cf95ea0
Pop with default value to stop throwing
2017-02-28 10:02:54 +00:00
64a2cef9bb
Pop rather than del from dict
2017-02-27 19:15:36 +00:00
17673404fb
Remove unused param
2017-02-20 15:02:01 +00:00
7f026792e1
Fix /context/ visibiltiy rules
2017-02-20 14:54:50 +00:00
8c87bb550e
Merge pull request #1922 from matrix-org/dbkr/allow_forget_for_ban
...
Allow forgetting rooms you're banned from
2017-02-17 10:52:30 +00:00
474c9aadbe
Allow forgetting rooms you're banned from
2017-02-15 19:32:20 +00:00
ce3e583d94
WIP support for msisdn 3pid proxy methods
2017-02-14 15:05:55 +00:00
fc2f29c1d0
Fix bugs in the /keys/changes api
...
* `get_forward_extremeties_for_room` takes a numeric `stream_ordering`. We were
passing a `RoomStreamToken`, which meant that it returned the *current*
extremities, rather than those corresponding to the `from_token`. However:
* `get_state_ids_for_events` required a second ('types') parameter; this meant
that a `TypeError` was thrown and we ended up acting as though there was *no*
prev state.
* `get_state_ids_for_events` actually returns a map from event_id to state
dictionary - just looking up the state keys in it again meant that we acted
as though there was no prev state. We now check if each member's state has
changed since *any* of the extremities.
Also add/fix some comments.
2017-02-14 13:59:50 +00:00
9e617cd4c2
Cache get_presence storage
2017-02-13 13:50:03 +00:00
af6da6db2d
Merge pull request #1784 from morteza-araby/user-admin
...
Administration functionalities
2017-02-06 16:21:10 +01:00
a597994fb6
Measure new device list stuff
2017-02-02 18:36:17 +00:00
1232ae41cf
Use new get_users_who_share_room_with_user
2017-02-02 15:25:00 +00:00
99fa03e8b5
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/device_list_fixes
2017-02-02 15:23:45 +00:00
a8331897aa
Merge pull request #1876 from matrix-org/erikj/shared_member_store
...
Make presence.get_new_events a bit faster
2017-02-02 15:20:14 +00:00
46ecd9fd6d
Use stream_ordering_to_exterm for /keys/changes
2017-02-02 14:27:19 +00:00
54a79c1d37
Make presence.get_new_events a bit faster
...
We do this by caching the set of users a user shares rooms with.
2017-02-02 13:07:18 +00:00
2849d3f29d
admin,storage: added more administrator functionalities
...
administrators can now:
- Set displayname of users
- Update user avatars
- Search for users by user_id
- Browse all users in a paginated API
- Reset user passwords
- Deactivate users
Helpers for doing paginated queries has also been added to storage
Signed-off-by: Morteza Araby <morteza.araby@ericsson.com>
2017-02-02 14:02:26 +01:00
51adaac953
Fix email push in pusher worker
...
This was broken when device list updates were implemented, as Mailer
could no longer instantiate an AuthHandler due to a dependency on
federation sending.
2017-02-02 10:53:36 +00:00
fbfe44bb4d
Doc args
2017-02-01 17:52:57 +00:00
7e919bdbd0
Include newly joined users in /keys/changes API
2017-02-01 17:33:16 +00:00
6d6591880e
Wake sync up for device changes
2017-02-01 15:15:16 +00:00
97479d0c54
Implement /keys/changes
2017-02-01 10:30:03 +00:00
bfb3d255b1
Merge pull request #1862 from matrix-org/erikj/presence_update
...
Use DB cache of joined users for presence
2017-01-31 13:23:24 +00:00
c7a26b7c32
Fix unit tests
2017-01-30 17:11:24 +00:00
fd1c18c088
Use DB cache of joined users for presence
2017-01-30 17:00:24 +00:00
c2c9a78db9
Noop device key changes if they're the same
2017-01-30 16:55:04 +00:00
828db669ec
Use get_users_in_room and declare it iterable
2017-01-30 16:37:22 +00:00
9636b2407d
Merge pull request #1857 from matrix-org/erikj/device_list_stream
...
Implement device lists updates over federation
2017-01-30 14:35:21 +00:00
252b503fc8
Hook device list updates to replication
2017-01-27 14:31:35 +00:00
888c59c955
Better name
2017-01-27 10:29:47 +00:00
76d40f4904
Handle users leaving rooms
2017-01-26 16:39:33 +00:00
fbfad76c03
Add comments
2017-01-26 16:33:21 +00:00
c974116f19
Implement device key caching over federation
2017-01-26 16:07:24 +00:00
51e9fe36e4
Fix up sending of m.device_list_update edus
2017-01-25 16:55:21 +00:00
2367c5568c
Add basic implementation of local device list changes
2017-01-25 14:27:27 +00:00
10e48d8310
Don't clobber a displayname or avatar_url if provided by an m.room.member event
2017-01-24 18:06:07 +00:00
a55fa2047f
Insert delta of current_state_events to be more efficient
2017-01-20 17:10:18 +00:00
09eb08f910
Derive current_state_events from state groups
2017-01-20 11:52:51 +00:00
f9058ca785
Merge pull request #1822 from matrix-org/erikj/statE_logging
...
Change resolve_state_groups call site logging to DEBUG
2017-01-18 11:02:03 +00:00
15f012032c
Merge pull request #1818 from matrix-org/erikj/state_auth_splitout_split
...
Optimise state resolution
2017-01-18 10:53:00 +00:00
5f027d1fc5
Change resolve_state_groups call site logging to DEBUG
2017-01-17 17:07:15 +00:00
e6153e1bd1
Fix couple of federation state bugs
2017-01-17 13:22:34 +00:00
5d6bad1b3c
Optimise state resolution
2017-01-17 13:22:19 +00:00
7b62d0bc70
Add missing None check
2017-01-11 10:57:03 +00:00
657488113e
Merge pull request #1789 from matrix-org/erikj/decouple_presence
...
Don't block messages sending on bumping presence
2017-01-10 14:06:05 +00:00
3b4de17d2b
Comment
2017-01-10 14:05:53 +00:00
7d0981b312
Merge pull request #1787 from matrix-org/erikj/linearize_member
...
Linearize updates to membership via PUT /state/
2017-01-10 14:04:54 +00:00
586f474a44
Don't block messages sending on bumping presence
2017-01-10 12:46:00 +00:00
6823fe5241
Linearize updates to membership via PUT /state/
2017-01-09 18:25:13 +00:00
f7085ac84f
Name linearizer's for better logs
2017-01-09 17:17:10 +00:00
06c00bd19b
Merge branch 'release-v0.18.6' into develop
2017-01-06 14:46:27 +00:00
1ef6084b75
handlers/room_member: fix guest access check when joining rooms
...
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2017-01-06 14:36:56 +00:00
bd85434cb3
Merge branch 'release-v0.18.6' into develop
2017-01-05 13:58:19 +00:00
883ff92a7f
Fix case
2017-01-05 13:45:02 +00:00
d79d165761
add logging for all the places we call resolve_state_groups. my kingdom for a backtrace that actually works.
2017-01-05 13:40:39 +00:00
b31ed22738
Merge branch 'release-v0.18.6' into develop
2017-01-05 13:03:02 +00:00
f784980d2b
Only send events that originate on this server.
...
Or events that are sent via the federation "send_join" API.
This should match the behaviour from before v0.18.5 and #1635 landed.
2017-01-05 11:26:30 +00:00
84cf00c645
Fix another comment typo
2016-12-21 09:51:43 +00:00
0c88ab1844
Add /account/3pid/delete endpoint
...
Also fix a typo in a comment
2016-12-20 18:27:30 +00:00
f2a5aebf98
fix ability to change password to a non-ascii one
...
https://github.com/vector-im/riot-web/issues/2658
2016-12-18 22:25:21 +00:00
b2f8642d3d
Cache network room list queries.
2016-12-16 16:11:43 +00:00
edc6a1e4f9
Add some logging for syncing to_device events
...
Attempt to track down the loss of to_device events
(https://github.com/vector-im/riot-web/issues/2711 etc).
2016-12-15 18:16:10 +00:00
35129ac998
Merge pull request #1698 from matrix-org/erikj/room_list
...
Fix caching on public room list
2016-12-15 15:40:28 +00:00
b8255eba26
Comment
2016-12-14 13:49:54 +00:00
b2999a7055
Fix /backfill returning events it shouldn't
2016-12-14 13:41:45 +00:00
1de53a7a1a
Fix caching on public room list
2016-12-13 17:33:24 +00:00
1574b839e0
Merge pull request #1676 from matrix-org/erikj/room_list
...
Add new API appservice specific public room list
2016-12-12 17:00:10 +00:00
d53a80af25
Merge pull request #1620 from matrix-org/erikj/concurrent_room_access
...
Limit the number of events that can be created on a given room concurrently
2016-12-12 10:30:23 +00:00
85cd30b1fd
Merge pull request #1686 from matrix-org/rav/fix_federation_key_fails
...
E2E key query: handle federation fails
2016-12-12 09:33:39 +00:00
deca951241
Remove unused import
2016-12-12 09:24:35 +00:00
efa4ccfaee
E2E key query: handle federation fails
...
Don't fail the whole request if we can't connect to a particular server.
2016-12-09 18:31:01 +00:00
be14c24cea
Fix rare notifier bug where listeners dont timeout
...
There was a race condition that caused the notifier to 'miss' the
timeout notification, since there were no other checks for the timeout
this caused listeners to get stuck in a loop until something happened.
2016-12-09 15:43:18 +00:00
b08f76bd23
Fix ignored read-receipts
...
Don't ignore read-receipts which arrive in the same EDU as a read-receipt for
an old event.
2016-12-08 12:13:01 +00:00
2687af82d4
Comments
2016-12-07 09:58:33 +00:00
f32fb65552
Add new API appservice specific public room list
2016-12-06 16:12:27 +00:00
1529c19675
Prevent user tokens being used as guest tokens ( #1675 )
...
Make sure that a user cannot pretend to be a guest by adding 'guest = True'
caveats.
2016-12-06 15:31:37 +00:00
9609c91e7d
Merge pull request #653 from matrix-org/erikj/preset_guest_join
...
Enable guest access for private rooms by default
2016-12-05 17:47:14 +00:00
338df4f409
Merge pull request #1649 from matrix-org/dbkr/log_ui_auth_args
...
Log the args that we have on UI auth completion
2016-12-05 16:40:58 +00:00
aa09d6b8f0
Rip out more refresh_token code
...
We might as well treat all refresh_tokens as invalid. Just return a 403 from
/tokenrefresh, so that we don't have a load of dead, untestable code hanging
around.
Still TODO: removing the table from the schema.
2016-11-30 17:40:18 +00:00
dc4b23e1a1
Merge branch 'develop' into rav/no_more_refresh_tokens
2016-11-30 17:10:04 +00:00
1c4f05db41
Stop putting a time caveat on access tokens
...
The 'time' caveat on the access tokens was something of a lie, since we weren't
enforcing it; more pertinently its presence stops us ever adding useful time
caveats.
Let's move in the right direction by not lying in our caveats.
2016-11-29 16:49:41 +00:00
5c4edc83b5
Stop generating refresh tokens
...
Since we're not doing refresh tokens any more, we should start killing off the
dead code paths. /tokenrefresh itself is a bit of a thornier subject, since
there might be apps out there using it, but we can at least not generate
refresh tokens on new logins.
2016-11-28 10:13:01 +00:00
c9d4e7b716
Clarify that creds doesn not contain passwords.
2016-11-24 10:54:59 +00:00
f681aab895
Log the args that we have on UI auth completion
...
This will be super helpful for debugging if we have more
registration woes.
2016-11-24 10:11:45 +00:00
feec718265
Shuffle receipt handler around so that worker apps don't need to load it
2016-11-23 15:14:24 +00:00
b69f76c106
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/split_out_fed_txn
2016-11-23 11:31:53 +00:00
83bcdcee61
Return early on /sync code paths if a '*' filter is used
...
This is currently very conservative in that it only does this if there is no
`since` token. This limits the risk to clients likely to be doing one-off
syncs (like bridges), but does mean that normal human clients won't benefit
from the time savings here. If the savings are large enough, I would consider
generalising this to just check the filter.
2016-11-22 16:38:35 +00:00
9687e039e7
Remove explicit calls to send_pdu
2016-11-21 14:48:51 +00:00
f8ee66250a
Handle sending events and device messages over federation
2016-11-17 15:48:04 +00:00
59ef517e6b
Use new federation_sender DI
2016-11-16 14:47:52 +00:00
d56c39cf24
Use external ldap auth pacakge
2016-11-15 13:03:19 +00:00
64038b806c
Comments
2016-11-11 10:42:08 +00:00
2bd4513a4d
Limit the number of events that can be created on a given room concurretnly
2016-11-10 16:44:35 +00:00
a4632783fb
Sample correctly
2016-11-08 11:20:26 +00:00
eeda4e618c
Limit the number of prev_events of new events
2016-11-08 11:02:29 +00:00
9084720993
Don't error on non-ascii passwords
2016-11-03 10:42:14 +00:00
2ef617bc06
Fix infinite typing bug
...
There's a bug somewhere that causes typing notifications to not be timed
out properly. By adding a paranoia timer and using correct inequalities
notifications should stop being stuck, even if it the root cause hasn't
been fixed.
2016-10-24 15:51:22 +01:00
d04e2ff3a4
Fix incredubly slow back pagination query
...
If a client didn't specify a from token when paginating backwards
synapse would attempt to query the (global) maximum topological token.
This a) doesn't make much sense since they're room specific and b) there
are no indices that lets postgres do this efficiently.
2016-10-24 13:35:51 +01:00
e01a1bc92d
Merge pull request #1175 from matrix-org/luke/feature-configurable-as-rate-limiting
...
Allow Configurable Rate Limiting Per AS
2016-10-20 16:21:10 +01:00
f09db236b1
as_user->app_service, less redundant comments, better positioned comments
2016-10-20 12:04:54 +01:00
1b17d1a106
Use real AS object by passing it through the requester
...
This means synapse does not have to check if the AS is interested, but instead it effectively re-uses what it already knew about the requesting user
2016-10-20 11:43:05 +01:00
3aa8925091
Merge pull request #1176 from matrix-org/erikj/eager_ratelimit_check
...
Check whether to ratelimit sooner to avoid work
2016-10-19 14:25:52 +01:00
f2f74ffce6
Comment
2016-10-19 14:21:28 +01:00
df2a616c7b
Convert emails to lowercase when storing
...
And db migration sql to convert existing addresses.
2016-10-19 11:13:55 +01:00
550308c7a1
Check whether to ratelimit sooner to avoid work
2016-10-19 10:45:24 +01:00
5b54d51d1e
Allow Configurable Rate Limiting Per AS
...
This adds a flag loaded from the registration file of an AS that will determine whether or not its users are rate limited (by ratelimit in _base.py). Needed for IRC bridge reasons - see https://github.com/matrix-org/matrix-appservice-irc/issues/240 .
2016-10-18 17:04:09 +01:00
35e2cc8b52
Merge pull request #1155 from matrix-org/erikj/pluggable_pwd_auth
...
Implement pluggable password auth
2016-10-12 11:41:20 +01:00
0061e8744f
Merge pull request #1166 from matrix-org/rav/grandfather_broken_riot_signup
...
Work around email-spamming Riot bug
2016-10-11 11:58:58 +01:00
fa74fcf512
Work around email-spamming Riot bug
...
5d9546f9https://github.com/matrix-org/matrix-doc/pull/397 ).
That change exposed a bug in Riot which would make it request a new validation
token (and send a new email) each time it got a 401 with a `flows` parameter
(see https://github.com/vector-im/vector-web/issues/2447 and the fix at
https://github.com/matrix-org/matrix-react-sdk/pull/510 ).
To preserve compatibility with broken versions of Riot, grandfather in the old
behaviour for the email validation stage.
2016-10-11 11:34:40 +01:00
a2f2516199
Merge pull request #1157 from Rugvip/nolimit
...
Remove rate limiting from app service senders and fix get_or_create_user requester
2016-10-11 11:20:54 +01:00
8681aff4f1
Merge pull request #1160 from matrix-org/rav/401_on_password_fail
...
Interactive Auth: Return 401 from for incorrect password
2016-10-07 10:57:43 +01:00
5d9546f9f4
Interactive Auth: Return 401 from for incorrect password
...
This requires a bit of fettling, because I want to return a helpful error
message too but we don't want to distinguish between unknown user and invalid
password. To avoid hardcoding the error message into 15 places in the code,
I've had to refactor a few methods to return None instead of throwing.
Fixes https://matrix.org/jira/browse/SYN-744
2016-10-07 00:00:00 +01:00
7b5546d077
rest/client/v1/register: use the correct requester in createUser
...
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-10-06 22:12:32 +02:00
3de7c8a4d0
handlers/profile: added admin override for set_displayname and set_avatar_url
...
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-10-06 15:24:59 +02:00
2ff2d36b80
handers: do not ratelimit app service senders
...
Signed-off-by: Patrik Oldsberg <patrik.oldsberg@ericsson.com>
2016-10-06 15:24:59 +02:00
9bfc617791
storage/appservice: make appservice methods only relying on the cache synchronous
2016-10-06 15:24:59 +02:00
850b103b36
Implement pluggable password auth
...
Allows delegating the password auth to an external module. This also
moves the LDAP auth to using this system, allowing it to be removed from
the synapse tree entirely in the future.
2016-10-03 10:36:40 +01:00
3027ea22b0
Restructure ldap authentication
...
- properly parse return values of ldap bind() calls
- externalize authentication methods
- change control flow to be more error-resilient
- unbind ldap connections in many places
- improve log messages and loglevels
2016-09-29 15:30:08 +01:00
655891d179
Move FEDERATION_PING_INTERVAL timer. Update log line
2016-09-23 15:43:34 +01:00
22578545a0
Time out typing over federation
2016-09-23 14:00:52 +01:00
667fcd54e8
Merge pull request #1136 from matrix-org/erikj/fix_signed_3pid
...
Allow invites via 3pid to bypass sender sig check
2016-09-22 13:41:49 +01:00
81964aeb90
Merge pull request #1132 from matrix-org/erikj/initial_sync_split
...
Support /initialSync in synchrotron worker
2016-09-22 12:45:02 +01:00
2e9ee30969
Add comments
2016-09-22 11:59:46 +01:00
1168cbd54d
Allow invites via 3pid to bypass sender sig check
...
When a server sends a third party invite another server may be the one
that the inviting user registers with. In this case it is that remote
server that will issue an actual invitation, and wants to do it "in the
name of" the original invitee. However, the new proper invite will not
be signed by the original server, and thus other servers would reject
the invite if it was seen as coming from the original user.
To fix this, a special case has been added to the auth rules whereby
another server can send an invite "in the name of" another server's
user, so long as that user had previously issued a third party invite
that is now being accepted.
2016-09-22 10:56:53 +01:00
90c070c850
Add total_room_count_estimate to /publicRooms
2016-09-21 13:30:05 +01:00
87528f0756
Support /initialSync in synchrotron worker
2016-09-21 11:46:28 +01:00
39af634dd2
Merge pull request #1130 from matrix-org/erikj/fix_pubroom_pag
...
Handle fact that _generate_room_entry may not return a room entry
2016-09-19 10:13:59 +01:00
4d49e0bdfd
PEP8
2016-09-17 18:09:22 +01:00
81570abfb2
Handle fact that _generate_room_entry may not return a room entry
2016-09-17 18:01:54 +01:00
eb24aecf8c
Merge pull request #1129 from matrix-org/erikj/fix_pubroom_pag
...
Fix and clean up publicRooms pagination
2016-09-17 15:30:34 +01:00
a298331de4
Spelling
2016-09-17 14:59:40 +01:00
71edaae981
Fix and clean up publicRooms pagination
2016-09-17 14:46:19 +01:00
5336acd46f
Make public room search case insensitive
2016-09-16 19:02:42 +01:00
b7b62bf9ea
Comment
2016-09-16 11:00:29 +01:00
d84319ae10
Add remote reoom cache
2016-09-16 10:31:59 +01:00
23b6701a28
Support filtering remote room lists
2016-09-16 10:24:15 +01:00
e58a9d781c
Filter remote rooms lists locally
2016-09-16 10:19:32 +01:00
74d4cdee25
Don't cache searched in /publicRooms
2016-09-16 09:05:11 +01:00
098db4aa52
Add very basic filter API to /publicRooms
2016-09-15 17:50:16 +01:00
c33b25fd8d
Change the way we calculate new_limit in /publicRooms and add POST API
2016-09-15 17:35:20 +01:00
e457034e99
Merge pull request #1121 from matrix-org/erikj/public_room_paginate
...
Add pagination support to publicRooms
2016-09-15 13:27:09 +01:00
4fb65a1091
Base public room list off of public_rooms stream
2016-09-15 11:27:04 +01:00
5810cffd33
Pass since/from parameters over federation
2016-09-15 10:36:19 +01:00
f3eead0660
Allow paginating both forwards and backwards
2016-09-15 10:15:37 +01:00
4131381123
Remove support for aggregate room lists
2016-09-15 09:28:15 +01:00
4f181f361d
Accept optional token to public room list
2016-09-15 09:08:57 +01:00
c566f0ee17
Calculate the public room list from a stream_ordering
2016-09-14 17:42:47 +01:00
772c6067a3
Refactor public rooms to not pull out the full state for each room
2016-09-14 17:29:25 +01:00
264a48aedf
Merge pull request #1117 from matrix-org/erikj/fix_state
...
Ensure we don't mutate state cache entries
2016-09-14 16:50:37 +01:00
a70a43bc51
Move RoomListHandler into a separate file
2016-09-14 14:07:37 +01:00
d5ae1f1291
Ensure we don't mutate state cache entries
2016-09-14 10:03:48 +01:00
1b01488d27
Merge pull request #1111 from matrix-org/matthew/device-ids
...
make device IDs more useful for human disambiguation
2016-09-14 01:18:02 +01:00
0f73f0e70e
Merge pull request #1116 from matrix-org/paul/tiny-fixes
...
Fix typo "persiting"
2016-09-13 13:27:42 +01:00
ca35e54d6b
Fix typo "persiting"
2016-09-13 13:26:33 +01:00
ad816b0add
Limit how often we ask for keys from dead servers
2016-09-13 11:53:50 +01:00
18ab019a4a
Move the E2E key handling into the e2e handler
2016-09-13 11:35:35 +01:00
76b09c29b0
Merge pull request #1110 from matrix-org/markjh/e2e_timeout
...
Add a timeout parameter for end2end key queries.
2016-09-13 10:50:45 +01:00
edbcb4152b
make device IDs more useful for human disambiguation
2016-09-13 00:02:39 +01:00
949c2c5435
Add a timeout parameter for end2end key queries.
...
Add a timeout parameter for controlling how long synapse will wait
for responses from remote servers. For servers that fail include how
they failed to make it easier to debug.
Fetch keys from different servers in parallel rather than in series.
Set the default timeout to 10s.
2016-09-12 18:17:09 +01:00
48a5a7552d
Add is_direct param to /createRoom
2016-09-12 16:34:20 +01:00
3265def8c7
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/batch_edus
2016-09-09 18:06:01 +01:00
52b2318777
Clobber EDUs in send queue
2016-09-09 15:59:08 +01:00
56f38d1776
Merge pull request #1091 from matrix-org/paul/third-party-lookup
...
Improvements to 3PE lookup API
2016-09-09 15:43:11 +01:00
ed44c475d8
Reject malformed 3PE query metadata results earlier in AS API handling code
2016-09-09 15:07:04 +01:00
f25d74f69c
Minor fixes from PR comments
2016-09-09 14:54:16 +01:00
6c4d582144
Deduplicate presence in _update_states
2016-09-09 14:28:22 +01:00
25eb769b26
Efficiency fix for lookups of a single protocol
2016-09-09 13:25:02 +01:00
033d43e419
Don't corrupt shared cache on subsequent protocol requests
2016-09-09 13:10:36 +01:00
2f267ee160
Collect up all the "instances" lists of individual AS protocol results into one combined answer to the client
2016-09-08 17:43:53 +01:00
257025ac89
Merge pull request #1082 from matrix-org/erikj/remote_public_rooms
...
Add server param to /publicRooms
2016-09-08 16:04:22 +01:00
8b93af662d
Check the user_id for presence/typing matches origin
2016-09-08 15:07:38 +01:00
fa9d36e050
Merge branch 'develop' into markjh/direct_to_device_federation
2016-09-08 13:43:43 +01:00
791658b576
Add server param to /publicRooms
2016-09-08 11:53:05 +01:00
43954d000e
Add a new method to enqueue the device messages rather than sending a dummy EDU
2016-09-07 16:10:51 +01:00
d4a35ada28
Send device messages over federation
2016-09-06 18:16:20 +01:00
3c4208a057
Record counts of state changes
2016-09-06 11:31:01 +01:00
438ef47637
Short circuit if presence is the same
2016-09-06 10:28:35 +01:00
74a3b4a650
Fiddle should_notify to better report stats
2016-09-06 10:23:38 +01:00
8c93e0bae7
Merge pull request #1070 from matrix-org/erikj/presence_stats
...
Record why we have chosen to notify
2016-09-05 15:01:30 +01:00
Erik Johnston