Commit Graph

6554 Commits (2d9f341c3e4fb4117e3986d1c64bdd14f4beaefb)

Author SHA1 Message Date
Erik Johnston 33ba90c6e9 Merge pull request #2240 from matrix-org/erikj/cache_list_fix
Update list cache to handle one arg case
2017-05-22 16:10:46 +01:00
Erik Johnston bd7bb5df71 Pull out if statement from for loop 2017-05-22 15:12:19 +01:00
Erik Johnston e3417a06e2 Update list cache to handle one arg case
We update the normal cache descriptors to handle caches with a single
argument specially so that the key wasn't a 1-tuple. We need to update
the cache list to be aware of this.
2017-05-22 15:04:42 +01:00
Erik Johnston 7fb80b5eae Check if current event is a membership event 2017-05-22 15:02:12 +01:00
Erik Johnston 2d17b09a6d Add debug logging 2017-05-22 15:01:36 +01:00
Erik Johnston 24c8f38784 Comment 2017-05-22 14:59:27 +01:00
Erik Johnston 25f03cf8e9 Use tuple unpacking 2017-05-22 14:58:22 +01:00
Erik Johnston 270e1c904a Speed up calculating push rules 2017-05-19 16:51:05 +01:00
Erik Johnston b4f59c7e27 Add count of one time keys to sync stream 2017-05-19 15:47:55 +01:00
Erik Johnston 58ebb96cce Fix invalidation of get_users_with_read_receipts_in_room 2017-05-19 14:38:50 +01:00
Erik Johnston 1c1c0257f4 Move invalidation cb to its own structure 2017-05-19 11:44:11 +01:00
Erik Johnston cafe659f72 Store ActionGenerator in HomeServer 2017-05-19 10:09:56 +01:00
Erik Johnston 72ed8196b3 Don't push users who have left 2017-05-18 17:48:36 +01:00
Erik Johnston 107ac7ac96 Increase size of push rule caches 2017-05-18 17:17:53 +01:00
Erik Johnston 760625acba Make get_if_app_services_interested_in_user faster 2017-05-18 16:34:44 +01:00
Erik Johnston c57789d138 Remove size of push get_rules cache 2017-05-18 16:17:23 +01:00
Erik Johnston f33df30732 Merge branch 'master' of github.com:matrix-org/synapse into develop 2017-05-18 13:56:37 +01:00
Erik Johnston a5425b2e5b Bump changelog and version 2017-05-18 13:53:48 +01:00
Erik Johnston 6e381180ae Merge pull request #2177 from matrix-org/erikj/faster_push_rules
Make calculating push actions faster
2017-05-18 11:46:18 +01:00
Erik Johnston 056ba9b795 Add comment 2017-05-18 11:45:56 +01:00
Erik Johnston d9e3a4b5db Merge pull request #2230 from matrix-org/erikj/speed_up_get_state
Make get_state_groups_from_groups faster.
2017-05-17 17:23:04 +01:00
Erik Johnston 66d8ffabbd Faster push rule calculation via push specific cache
We add a push rule specific cache that ensures that we can reuse
calculated push rules appropriately when a user join/leaves.
2017-05-17 16:55:40 +01:00
Erik Johnston ace23463c5 Merge pull request #2216 from slipeer/app_services_interested_in_user
Fix users claimed non-exclusively by an app service don't get notific…
2017-05-17 16:28:50 +01:00
Erik Johnston bbfe4e996c Make get_state_groups_from_groups faster.
Most of the time was spent copying a dict to filter out sentinel values
that indicated that keys did not exist in the dict. The sentinel values
were added to ensure that we cached the non-existence of keys.

By updating DictionaryCache to keep track of which keys were known to
not exist itself we can remove a dictionary copy.
2017-05-17 15:12:15 +01:00
Erik Johnston a8bc7cae56 Merge branch 'develop' of github.com:matrix-org/synapse into release-v0.21.0 2017-05-17 13:11:43 +01:00
Erik Johnston 3a431a126d Bump changelog and version 2017-05-17 11:26:57 +01:00
Erik Johnston 85e8092cca Comment 2017-05-17 10:03:09 +01:00
Erik Johnston ad53fc3cf4 Short circuit when we have delta ids 2017-05-17 09:57:34 +01:00
Erik Johnston 6fa8148ccb Merge pull request #2228 from matrix-org/erikj/speed_up_get_hosts
Speed up get_joined_hosts
2017-05-16 17:40:55 +01:00
Erik Johnston 7c69849a0d Merge pull request #2227 from matrix-org/erikj/presence_caches
Make presence use cached users/hosts in room
2017-05-16 17:40:47 +01:00
Erik Johnston 13f540ef1b Speed up get_joined_hosts 2017-05-16 16:05:22 +01:00
Erik Johnston ec5c4499f4 Make presence use cached users/hosts in room 2017-05-16 16:01:43 +01:00
Erik Johnston f2a5b6dbfd Speed up get_domain_from_id 2017-05-16 15:59:37 +01:00
Erik Johnston b8492b6c2f Merge pull request #2224 from matrix-org/erikj/prefill_state
Prefill state caches
2017-05-16 15:50:11 +01:00
Erik Johnston 331570ea6f Remove spurious merge artifacts 2017-05-16 15:33:07 +01:00
Krombel 55af207321 Merge branch 'develop' into avoid_duplicate_filters 2017-05-16 15:29:59 +02:00
Richard van der Hoff d648f65aaf Merge pull request #2218 from matrix-org/rav/event_search_index
Add an index to event_search
2017-05-16 13:26:07 +01:00
Erik Johnston 608b5a6317 Take a copy before prefilling, as it may be a frozendict 2017-05-16 12:55:29 +01:00
Krombel 64953c8ed2 avoid access-error if no filter_id matches 2017-05-15 18:36:37 +02:00
Erik Johnston f451b64c8f Merge branch 'develop' of github.com:matrix-org/synapse into erikj/prefill_state 2017-05-15 16:09:32 +01:00
Erik Johnston 2c9475b58e Merge pull request #2221 from psaavedra/sync_timeline_limit_filter_by_name
Configurable maximum number of events requested by /sync and /messages
2017-05-15 16:08:46 +01:00
Erik Johnston d12ae7fd1c Don't log exceptions for NotRetryingDestination 2017-05-15 15:42:18 +01:00
Pablo Saavedra 224137fcf9 Fixed syntax nits 2017-05-15 16:21:02 +02:00
Erik Johnston e4435b014e Update comment 2017-05-15 15:11:30 +01:00
Erik Johnston 871605f4e2 Comments 2017-05-15 15:11:30 +01:00
Erik Johnston e0d2f6d5b0 Add more granular event send metrics 2017-05-15 15:11:30 +01:00
Erik Johnston bfbc907cec Prefill state caches 2017-05-15 15:11:13 +01:00
Pablo Saavedra 627e6ea2b0 Fixed implementation errors
* Added HS as property in SyncRestServlet
* Fixed set_timeline_upper_limit function implementat¡ion
2017-05-15 14:51:43 +02:00
Pablo Saavedra 9da4316ca5 Configurable maximum number of events requested by /sync and /messages (#2220)
Set the limit on the returned events in the timeline in the get and sync
operations. The default value is -1, means no upper limit.

For example, using `filter_timeline_limit: 5000`:

POST /_matrix/client/r0/user/user:id/filter
{
room: {
    timeline: {
      limit: 1000000000000000000
    }
}
}

GET /_matrix/client/r0/user/user:id/filter/filter:id

{
room: {
    timeline: {
      limit: 5000
    }
}
}

The server cuts down the room.timeline.limit.
2017-05-13 18:17:54 +02:00
Krombel eb7cbf27bc insert whitespace to fix travis build 2017-05-12 12:09:42 +02:00
Krombel 6b95e35e96 add check to only add a new filter if the same filter does not exist previously
Signed-off-by: Matthias Kesler <krombel@krombel.de>
2017-05-11 16:05:30 +02:00
Richard van der Hoff ff3d810ea8 Add a comment to old delta 2017-05-11 12:48:50 +01:00
Richard van der Hoff 34194aaff7 Don't create event_search index on sqlite
... because the table is virtual
2017-05-11 12:46:55 +01:00
Richard van der Hoff 114f290947 Add more logging for purging
Log the number of events we will be deleting at info.
2017-05-11 12:08:47 +01:00
Richard van der Hoff baafb85ba4 Add an index to event_search
- to make the purge API quicker
2017-05-11 12:05:22 +01:00
Richard van der Hoff 29ded770b1 Merge pull request #2214 from matrix-org/rav/hurry_up_purge
When purging, don't de-delta state groups we're about to delete
2017-05-11 12:04:25 +01:00
Richard van der Hoff dc026bb16f Tidy purge code and add some comments
Try to make this clearer with more comments and some variable renames
2017-05-11 10:56:12 +01:00
Slipeer 328378f9cb Fix users claimed non-exclusively by an app service don't get notifications #2211 2017-05-11 11:42:08 +03:00
Richard van der Hoff 8e345ce465 Don't de-delta state groups we're about to delete 2017-05-10 18:44:22 +01:00
Richard van der Hoff b64d312421 add some logging to purge_history 2017-05-10 18:44:22 +01:00
Luke Barnard ccad2ed824 Modify condition on empty localpart 2017-05-10 17:34:30 +01:00
Luke Barnard 369195caa5 Modify register/available to be GET with query param
- GET is now the method for register/available
- a query parameter "username" is now used

Also, empty usernames are now handled with an error message on registration or via register/available: `User ID cannot be empty`
2017-05-10 17:23:55 +01:00
Erik Johnston a3648f84b2 Merge pull request #2208 from matrix-org/erikj/ratelimit_overrid
Add per user ratelimiting overrides
2017-05-10 15:54:48 +01:00
Richard van der Hoff 5331cd150a Merge pull request #2206 from matrix-org/rav/one_time_key_upload_change_sig
Allow clients to upload one-time-keys with new sigs
2017-05-10 14:18:40 +01:00
Luke Barnard f7278e612e Change register/available to POST (from GET) 2017-05-10 11:40:18 +01:00
Erik Johnston b990b2fce5 Add per user ratelimiting overrides 2017-05-10 11:05:43 +01:00
Richard van der Hoff aedaba018f Replace some instances of preserve_context_over_deferred 2017-05-09 19:04:56 +01:00
Richard van der Hoff de042b3b88 Do some logging when one-time-keys get claimed
might help us figure out if https://github.com/vector-im/riot-web/issues/3868
has happened.
2017-05-09 19:04:56 +01:00
Richard van der Hoff a7e9d8762d Allow clients to upload one-time-keys with new sigs
When a client retries a key upload, don't give an error if the signature has
changed (but the key is the same).

Fixes https://github.com/vector-im/riot-android/issues/1208, hopefully.
2017-05-09 19:04:56 +01:00
Erik Johnston ca238bc023 Merge branch 'release-v0.21.0' of github.com:matrix-org/synapse into develop 2017-05-08 17:35:11 +01:00
Erik Johnston 40dcf0d856 Merge pull request #2203 from matrix-org/erikj/event_cache_hit_ratio
Don't update event cache hit ratio from get_joined_users
2017-05-08 16:52:39 +01:00
Erik Johnston d3c3026496 Merge pull request #2201 from matrix-org/erikj/store_device_cache
Cache check to see if device exists
2017-05-08 16:23:04 +01:00
Erik Johnston 093f7e47cc Expand docstring a bit 2017-05-08 16:14:46 +01:00
Erik Johnston 6a12998a83 Add missing yields 2017-05-08 16:10:51 +01:00
Erik Johnston ffad4fe35b Don't update event cache hit ratio from get_joined_users
Otherwise the hit ration of plain get_events gets completely skewed by
calls to get_joined_users* functions.
2017-05-08 16:06:17 +01:00
Erik Johnston 94e6ad71f5 Invalidate cache on device deletion 2017-05-08 15:55:59 +01:00
Erik Johnston 8571f864d2 Cache one time key counts 2017-05-08 15:34:27 +01:00
Erik Johnston fc6d4974a6 Comment 2017-05-08 15:33:57 +01:00
Erik Johnston 738ccf61c0 Cache check to see if device exists 2017-05-08 15:32:18 +01:00
Erik Johnston dcabef952c Increase client_ip cache size 2017-05-08 15:09:19 +01:00
Erik Johnston 771c8a83c7 Bump version and changelog 2017-05-08 13:23:46 +01:00
Erik Johnston 6631985990 Merge pull request #2200 from matrix-org/erikj/revert_push
Revert speed up push
2017-05-08 13:20:52 +01:00
Erik Johnston e0f20e9425 Revert "Remove unused import"
This reverts commit ab37bef83b.
2017-05-08 13:07:43 +01:00
Erik Johnston fe7c1b969c Revert "We don't care about forgotten rooms"
This reverts commit ad8b316939.
2017-05-08 13:07:43 +01:00
Erik Johnston 78f306a6f7 Revert "Speed up filtering of a single event in push"
This reverts commit 421fdf7460.
2017-05-08 13:07:41 +01:00
Erik Johnston 9ac98197bb Bump version and changelog 2017-05-08 11:07:54 +01:00
Erik Johnston 27c28eaa27 Merge pull request #2190 from matrix-org/erikj/mark_remote_as_back_more
Always mark remotes as up if we receive a signed request from them
2017-05-05 14:08:12 +01:00
Erik Johnston be2672716d Merge pull request #2189 from matrix-org/erikj/handle_remote_device_list
Handle exceptions thrown in handling remote device list updates
2017-05-05 14:01:27 +01:00
Erik Johnston 653d90c1a5 Comment 2017-05-05 14:01:17 +01:00
Erik Johnston 310b1ccdc1 Use preserve_fn and add logs 2017-05-05 13:41:19 +01:00
Kegsay a59b0ad1a1 Merge pull request #2192 from matrix-org/kegan/simple-http-client-timeouts
Rewrite SimpleHttpClient.request to include timeouts
2017-05-05 11:52:43 +01:00
Erik Johnston 7b222fc56e Remove redundant reset of destination timers 2017-05-05 11:14:09 +01:00
Kegan Dougal d0debb2116 Remember how twisted works 2017-05-05 11:00:21 +01:00
Erik Johnston 66f371e8b8 Merge pull request #2176 from matrix-org/erikj/faster_get_joined
Make get_joined_users faster
2017-05-05 10:59:55 +01:00
Erik Johnston b843631d71 Add comment and TODO 2017-05-05 10:59:32 +01:00
Kegan Dougal c2ddd773bc Include the clock 2017-05-05 10:52:46 +01:00
Kegan Dougal 7dd3bf5e24 Rewrite SimpleHttpClient.request to include timeouts
Fixes #2191
2017-05-05 10:49:19 +01:00
Erik Johnston db7d0c3127 Always mark remotes as up if we receive a signed request from them 2017-05-05 10:34:53 +01:00
Erik Johnston f346048a6e Handle exceptions thrown in handling remote device list updates 2017-05-05 10:34:10 +01:00
Erik Johnston e3aa8a7aa8 Merge pull request #2185 from matrix-org/erikj/smaller_caches
Optimise caches for single key
2017-05-05 10:19:05 +01:00
Erik Johnston cf589f2c1e Fixes 2017-05-05 10:17:56 +01:00
Erik Johnston 8af4569583 Merge pull request #2174 from matrix-org/erikj/current_cache_hosts
Add cache for get_current_hosts_in_room
2017-05-05 10:15:24 +01:00
Erik Johnston 587f07543f Revert "Prefill state caches" 2017-05-04 15:07:27 +01:00
Erik Johnston aa93cb9f44 Add comment 2017-05-04 14:59:28 +01:00
Erik Johnston 537dbadea0 Intern host strings 2017-05-04 14:55:28 +01:00
Erik Johnston 07a07588a0 Make caches bigger 2017-05-04 14:52:28 +01:00
Erik Johnston dfaa58f72d Fix comment and num args 2017-05-04 14:50:24 +01:00
Erik Johnston 9ac263ed1b Add new storage functions to slave store 2017-05-04 14:29:03 +01:00
Erik Johnston d2d8ed4884 Optimise caches with single key 2017-05-04 14:18:46 +01:00
Erik Johnston 5d8290429c Reduce size of get_users_in_room 2017-05-04 13:43:19 +01:00
Luke Barnard 6aa423a1a8 Merge pull request #2183 from matrix-org/luke/username-availability
Implement username availability checker
2017-05-04 09:58:40 +01:00
Luke Barnard 3669065466 Appease the flake8 gods 2017-05-03 18:05:49 +01:00
Erik Johnston 7ebf518c02 Make get_joined_users faster 2017-05-03 15:55:54 +01:00
Luke Barnard 34ed4f4206 Implement username availability checker
Outlined here: https://github.com/vector-im/riot-web/issues/3605#issuecomment-298679388

```HTTP
GET /_matrix/.../register/available
{
    "username": "desiredlocalpart123"
}
```

If available, the response looks like
```HTTP
HTTP/1.1 200 OK
{
    "available": true
}
```

Otherwise,
```HTTP
HTTP/1.1 429
{
    "errcode": "M_LIMIT_EXCEEDED",
    "error": "Too Many Requests",
    "retry_after_ms": 2000
}
```
or
```HTTP
HTTP/1.1 400
{
    "errcode": "M_USER_IN_USE",
    "error": "User ID already taken."
}

```
or
```HTTP
HTTP/1.1 400
{
    "errcode": "M_INVALID_USERNAME",
    "error": "Some reason for username being invalid"
}
```
2017-05-03 12:04:12 +01:00
David Baker 60833c8978 Merge pull request #2147 from matrix-org/dbkr/http_request_propagate_error
Propagate errors sensibly from proxied IS requests
2017-05-03 11:23:25 +01:00
David Baker 482a2ad122 No need for the exception variable 2017-05-03 11:02:59 +01:00
David Baker c0380402bc List caught expection types 2017-05-03 10:56:22 +01:00
Erik Johnston cdbf38728d Merge pull request #2175 from matrix-org/erikj/prefill_state
Prefill state caches
2017-05-03 10:54:11 +01:00
Erik Johnston 0c27383dd7 Merge pull request #2170 from matrix-org/erikj/fed_hole_state
Don't fetch state for missing events that we fetched
2017-05-03 10:49:37 +01:00
Erik Johnston ef862186dd Merge together redundant calculations/logging 2017-05-03 10:06:43 +01:00
Erik Johnston 2c2dcf81d0 Update comment 2017-05-03 10:00:29 +01:00
Erik Johnston 1827057acc Comments 2017-05-03 09:56:05 +01:00
Erik Johnston 8346e6e696 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/prefill_state 2017-05-03 09:46:40 +01:00
Erik Johnston e4c15fcb5c Merge pull request #2178 from matrix-org/erikj/message_metrics
Add more granular event send metrics
2017-05-02 17:57:34 +01:00
Erik Johnston 3e5a62ecd8 Add more granular event send metrics 2017-05-02 14:23:26 +01:00
Richard van der Hoff 82475a18d9 Merge pull request #2180 from matrix-org/rav/fix_timeout_on_timeout
Instantiate DeferredTimedOutError correctly
2017-05-02 13:32:58 +01:00
Richard van der Hoff 2e996271fe Instantiate DeferredTimedOutError correctly
Call `super` correctly, so that we correctly initialise the `errcode` field.

Fixes https://github.com/matrix-org/synapse/issues/2179.
2017-05-02 13:26:17 +01:00
Erik Johnston a2c89a225c Prefill state caches 2017-05-02 10:40:31 +01:00
Erik Johnston 7166854f41 Add cache for get_current_hosts_in_room 2017-05-02 10:36:35 +01:00
Erik Johnston 3033261891 Merge pull request #2080 from matrix-org/erikj/filter_speed
Speed up filtering of a single event in push
2017-04-28 14:17:13 +01:00
Erik Johnston 2347efc065 Fixup 2017-04-28 12:46:53 +01:00
Erik Johnston 9b147cd730 Remove unncessary call in _get_missing_events_for_pdu 2017-04-28 11:55:25 +01:00
Erik Johnston 3a9f5bf6dd Don't fetch state for missing events that we fetched 2017-04-28 11:26:46 +01:00
Erik Johnston ab37bef83b Remove unused import 2017-04-28 09:57:23 +01:00
Erik Johnston ad8b316939 We don't care about forgotten rooms 2017-04-28 09:52:36 +01:00
Erik Johnston 421fdf7460 Speed up filtering of a single event in push 2017-04-28 09:52:36 +01:00
Erik Johnston 25a96e0c63 Merge pull request #2163 from matrix-org/erikj/fix_invite_state
Fix invite state to always include all events
2017-04-27 17:36:30 +01:00
Erik Johnston 46826bb078 Comment and remove spurious logging 2017-04-27 17:25:44 +01:00
Erik Johnston f87b287291 Merge pull request #2127 from APwhitehat/alreadystarted
print something legible if synapse already running
2017-04-27 15:46:53 +01:00
Erik Johnston bb9246e525 Merge pull request #2131 from matthewjwolff/develop
web_client_location documentation fix
2017-04-27 15:46:40 +01:00
Richard van der Hoff c84770b877 Fix bgupdate error if index already exists (#2167)
When creating a new table index in the background, guard against it existing already. Fixes
https://github.com/matrix-org/synapse/issues/2135.

Also, make sure we restore the autocommit flag when we're done, otherwise we
get more failures from other operations later on. Fixes
https://github.com/matrix-org/synapse/issues/1890 (hopefully).
2017-04-27 15:27:48 +01:00
Erik Johnston 380fb87ecc Merge pull request #2168 from matrix-org/erikj/federation_logging
Add some extra logging for edge cases of federation
2017-04-27 15:19:12 +01:00
Erik Johnston 87ae59f5e9 Typo 2017-04-27 15:16:21 +01:00
Erik Johnston e42b4ebf0f Add some extra logging for edge cases of federation 2017-04-27 14:38:21 +01:00
Erik Johnston d3c150411c Merge pull request #2130 from APwhitehat/roomexists
Check that requested room_id exists
2017-04-27 09:20:26 +01:00
Erik Johnston 1e166470ab Fix tests 2017-04-26 16:23:30 +01:00
Erik Johnston 34e682d385 Fix invite state to always include all events 2017-04-26 16:18:08 +01:00
David Baker 5fd12dce01 Remove debugging 2017-04-26 12:36:26 +01:00
David Baker 82ae0238f9 Revert accidental commit 2017-04-26 11:43:16 +01:00
David Baker 81804909d3 Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error 2017-04-26 11:31:55 +01:00
David Baker c366276056 Fix get_json 2017-04-26 10:07:01 +01:00
David Baker 1a9255c12e Use CodeMessageException subclass instead
Parse json errors from get_json client methods and throw special
errors.
2017-04-25 19:30:55 +01:00
Erik Johnston f053a1409e Make state caches cache in ascii 2017-04-25 17:22:55 +01:00
Erik Johnston acb58bfb6a fix up 2017-04-25 15:39:19 +01:00
Erik Johnston f7181615f2 Don't specify default as dict 2017-04-25 15:22:59 +01:00
Erik Johnston f144365281 Comment 2017-04-25 15:18:26 +01:00
Erik Johnston d9aa645f86 Reduce size of joined_user cache
The _get_joined_users_from_context cache stores a mapping from user_id
to avatar_url and display_name. Instead of storing those in a dict,
store them in a namedtuple as that uses much less memory.

We also try converting the string to ascii to further reduce the size.
2017-04-25 14:38:51 +01:00
Erik Johnston 22f3d3ae76 Reduce _get_state_group_for_event cache size 2017-04-25 11:43:03 +01:00
Erik Johnston b4da08cad8 Merge pull request #2158 from matrix-org/erikj/reduce_cache_size
Reduce cache size by not storing deferreds
2017-04-25 11:18:07 +01:00
Erik Johnston efab1dadde Remove DEBUG_CACHES 2017-04-25 10:54:09 +01:00
Erik Johnston 119cb9bbcf Reduce cache size by not storing deferreds
Currently the cache descriptors store deferreds rather than raw values,
this is a simple way of triggering only one database hit and sharing the
result if two callers attempt to get the same value.

However, there are a few caches that simply store a mapping from string
to string (or int). These caches can have a large number of entries,
under the assumption that each entry is small. However, the size of a
deferred (specifically the size of ObservableDeferred) is signigicantly
larger than that of the raw value, 2kb vs 32b.

This PR therefore changes the cache descriptors to store the raw values
rather than the deferreds.

As a side effect cached storage function now either return a deferred or
the actual value, as the cached list decriptor already does. This is
fine as we always end up just yield'ing on the returned value
eventually, which handles that case correctly.
2017-04-25 10:23:11 +01:00
Mark Haines e6e2627636 Fix code for reporting old verify keys in synapse 2017-04-24 18:51:25 +01:00
Richard van der Hoff 30f7bfa121 Merge pull request #2145 from matrix-org/rav/reject_invite_to_unreachable_server
Fix rejection of invites to unreachable servers
2017-04-24 15:20:52 +01:00
Erik Johnston 7af825bae4 Merge pull request #2155 from matrix-org/erikj/string_intern
Only intern ascii strings
2017-04-24 14:28:41 +01:00
Erik Johnston d134d0935e Only intern ascii strings 2017-04-24 14:07:48 +01:00
Erik Johnston e4f3431116 Remove unused cache 2017-04-24 13:27:38 +01:00
David Baker a46982cee9 Need the HTTP status code 2017-04-21 16:20:12 +01:00
David Baker 70caf49914 Do the same for get_json 2017-04-21 16:09:03 +01:00
David Baker a1595cec78 Don't error for 3xx responses 2017-04-21 11:51:17 +01:00
David Baker 2e165295b7 Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error 2017-04-21 11:35:52 +01:00
David Baker a90a0f5c8a Propagate errors sensibly from proxied IS requests
When we're proxying Matrix endpoints, parse out Matrix error
responses and turn them into SynapseErrors so they can be
propagated sensibly upstream.
2017-04-21 11:32:48 +01:00
Richard van der Hoff 91b3981800 Try harder when sending leave events
When we're rejecting invites, ignore the backoff data, so that we have a better
chance of not getting the room out of sync.
2017-04-21 01:50:36 +01:00
Richard van der Hoff 0cdb32fc43 Remove redundant try/except clauses
The `except SynapseError` clauses were pointless because the wrapped functions
would never throw a `SynapseError` (they either throw a `CodeMessageException`
or a `RuntimeError`).

The `except CodeMessageException` is now also pointless because the caller
treats all exceptions equally, so we may as well just throw the
`CodeMessageException`.
2017-04-21 01:32:01 +01:00
Richard van der Hoff 838810b76a Broaden the conditions for locally_rejecting invites
The logic for marking invites as locally rejected was all well and good, but
didn't happen when the remote server returned a 500, or wasn't reachable, or
had no DNS, or whatever.

Just expand the except clause to catch everything.

Fixes https://github.com/matrix-org/synapse/issues/761.
2017-04-21 01:31:37 +01:00
Richard van der Hoff 736b9a4784 Remove redundant function
inline `reject_remote_invite`, which only existed to make tracing the callflow
more difficult.
2017-04-21 01:31:09 +01:00
Richard van der Hoff 4903ccf159 Fix some lies, and other clarifications, in docstrings
The documentation on get_json has been wrong ever since the very first commit
to synapse...
2017-04-21 01:31:09 +01:00
Matthew Wolff d4040e9e28
Queried CONDITIONAL_REQUIREMENTS 2017-04-18 16:19:48 -05:00
Luke Barnard 3fb8784c92 m.read_marker -> m.fully_read (#2128)
Also:
 - change the REST endpoint to have a "S" on the end (so it's now /read_markers)
 - change the content of the m.read_up_to event to have the key "event_id" instead of "marker".
2017-04-18 17:46:15 +01:00
Matthew Wolff 54f9a4cb59
Fixed travis build failure
Signed-off-by: Matthew Wolff <matthewjwolff@gmail.com>
2017-04-17 01:38:27 -05:00
Matthew Wolff 8e780b113d
web_server_root documentation fix
Signed-off-by: Matthew Wolff <matthewjwolff@gmail.com>
2017-04-17 00:49:11 -05:00
Anant Prakash 574d573ac2 Check that requested room_id exists 2017-04-14 23:50:59 +05:30
Luke Barnard 78f0ddbfad Merge pull request #2120 from matrix-org/luke/read-markers
Implement Read Marker API
2017-04-13 14:21:31 +01:00
Luke Barnard 6a70647d45 Correct logic in is_event_after 2017-04-13 13:46:17 +01:00
Anant Prakash c1f52a321d synctl.py: Check if synapse is already running 2017-04-13 18:00:02 +05:30
Luke Barnard b9557064bf Simplify is_event_after logic 2017-04-12 14:36:20 +01:00
Luke Barnard cf6121e3da More null-guard changes 2017-04-12 14:02:03 +01:00
Erik Johnston 247c736b9b Merge pull request #2115 from matrix-org/erikj/dedupe_federation_repl
Reduce federation replication traffic
2017-04-12 11:07:13 +01:00
Paul Evans 8fbc0d29ee Merge pull request #2121 from matrix-org/paul/sent-transactions-metric
Add a counter metric for successfully-sent transactions
2017-04-12 11:04:31 +01:00
Erik Johnston c06c00190f Merge pull request #2116 from matrix-org/erikj/dedupe_federation_repl2
Dedupe KeyedEdu and Devices federation repl traffic
2017-04-12 10:57:24 +01:00
Luke Barnard c0aba0a23e Remove Unused ref to hs 2017-04-12 10:52:11 +01:00
Luke Barnard b9676a75f6 Move a space 2017-04-12 10:51:17 +01:00
Luke Barnard 69a18514e9 Only notify user, not entire room 2017-04-12 10:50:37 +01:00
Luke Barnard 122cd52ce4 Remove comment, simplify null-guard 2017-04-12 10:48:32 +01:00
Erik Johnston 26ae5178a4 Add some comments 2017-04-12 10:36:29 +01:00
Erik Johnston bf9060156a Merge pull request #2117 from matrix-org/erikj/remove_http_replication
Remove HTTP replication APIs
2017-04-12 10:21:42 +01:00
Erik Johnston 1745069543 Comment 2017-04-12 10:17:10 +01:00
Erik Johnston c7ddb5ef7a Reuse get_interested_parties 2017-04-12 10:16:26 +01:00
Erik Johnston 7b41013102 Merge pull request #2118 from matrix-org/erikj/no_devices
Fix getting latest device IP for user with no devices
2017-04-12 10:14:32 +01:00
Luke Barnard 77fb2b72ae Handle no previous RM 2017-04-12 09:47:29 +01:00
Luke Barnard 7f94709066 travis flake8.. 2017-04-11 18:35:45 +01:00
Luke Barnard 867822fa1e flake8 2017-04-11 17:36:04 +01:00
Luke Barnard 73880268ef Refactor event ordering check to events store 2017-04-11 17:34:09 +01:00
Luke Barnard 131485ef66 Copyright 2017-04-11 17:33:51 +01:00
Paul "LeoNerd" Evans 11dbceb761 Add a counter metric for successfully-sent transactions 2017-04-11 17:16:12 +01:00
Luke Barnard 0127423027 flake8 2017-04-11 17:07:07 +01:00
Erik Johnston 85657eedf8 Bail on where clause instead 2017-04-11 16:24:31 +01:00
Erik Johnston b48045a8f5 Don't bother with outer check for now 2017-04-11 16:23:24 +01:00
Erik Johnston 9c712a366f Move get_presence_list_* to SlaveStore 2017-04-11 16:07:33 +01:00
Erik Johnston a8c8e4efd4 Comment 2017-04-11 15:35:49 +01:00
Erik Johnston 414522aed5 Move get_interested_parties 2017-04-11 15:33:26 +01:00
Erik Johnston 2be8a281d2 Comments 2017-04-11 15:28:24 +01:00
Erik Johnston 6308ac45b0 Move get_interested_remotes back to presence handler 2017-04-11 15:19:26 +01:00
Erik Johnston b9b72bc6e2 Comments 2017-04-11 15:15:34 +01:00
Luke Barnard d892079844 Finish implementing RM endpoint
- This change causes a 405 to be sent if "m.read_marker" is set via /account_data
 - This also fixes-up the RM endpoint so that it actually Works.
2017-04-11 15:01:39 +01:00
lukebarnard e263c26690 Initial commit of RM server-side impl
(See https://docs.google.com/document/d/1UWqdS-e1sdwkLDUY0wA4gZyIkRp-ekjsLZ8k6g_Zvso/edit#heading=h.lndohpg8at5u)
2017-04-11 11:55:30 +01:00
Erik Johnston f3cf3ff8b6 Merge branch 'master' of github.com:matrix-org/synapse into develop 2017-04-11 11:13:32 +01:00
Erik Johnston 34840cdcef Fix getting latest device IP for user with no devices 2017-04-11 09:56:54 +01:00
Erik Johnston 28a4649785 Remove HTTP replication APIs 2017-04-11 09:52:11 +01:00
Matthew Hodgson 7c551ec445 trust a hypothetical future riot.im IS 2017-04-10 17:58:36 +01:00
Erik Johnston 84fbb80c8f Use generators 2017-04-10 16:55:56 +01:00
Erik Johnston 40453b3f84 Dedupe KeyedEdu and Devices federation repl traffic 2017-04-10 16:49:51 +01:00
Erik Johnston 29574fd5b3 Reduce federation presence replication traffic
This is mainly done by moving the calculation of where to send presence
updates from the presence handler to the transaction queue, so we only
need to send the presence event (and not the destinations) across the
replication connection. Before we were duplicating by sending the full
state across once per destination.
2017-04-10 16:48:30 +01:00
Erik Johnston 2e6f5a4910 Typo 2017-04-10 16:17:40 +01:00
David Baker 405ba4178a Merge pull request #2102 from DanielDent/add-auth-email
Support authenticated SMTP
2017-04-10 15:42:16 +01:00
Erik Johnston efcb6db688 Merge pull request #2109 from matrix-org/erikj/send_queue_fix
Fix up federation SendQueue and document types
2017-04-10 13:09:25 +01:00
Erik Johnston 0018491af2 Rename variable 2017-04-10 12:44:43 +01:00
Erik Johnston 0364d23210 Up replication ping timeout 2017-04-10 11:32:05 +01:00
Erik Johnston 8c5f03cec7 Revert to sending the same data type as before 2017-04-10 10:07:18 +01:00
Erik Johnston f8434db549 Change name 2017-04-10 10:03:07 +01:00
Erik Johnston ab904caf33 Comments 2017-04-10 10:02:17 +01:00
Richard van der Hoff 64765e5199 When we do an invite rejection, save the signed leave event to the db
During a rejection of an invite received over federation, we ask a remote
server to make us a `leave` event, then sign it, then send that with
`send_leave`.

We were saving the *unsigned* version of the event (which has a different event
id to the signed version) to our db (and sending it to the clients), whereas
other servers in the room will have seen the *signed* version. We're not aware
of any actual problems that caused, except that it makes the database confusing
to look at and generally leaves the room in a weird state.
2017-04-07 14:39:32 +01:00
Erik Johnston 2a3e822f44 Comment 2017-04-07 13:47:04 +01:00
Erik Johnston a828a64b75 Comment 2017-04-07 11:54:03 +01:00
Erik Johnston d4d176e5d0 Add logging 2017-04-07 11:51:28 +01:00
Erik Johnston 449d1297ca Fix up federation SendQueue and document types 2017-04-07 11:48:33 +01:00
Erik Johnston d72667fcce Speed up get_current_state_ids
Using _simple_select_list is fairly expensive for functions that return
a lot of rows and/or get called a lot. (This is because it carefully
constructs a list of dicts).

get_current_state_ids gets called a lot on startup and e.g. when the IRC
bridge decided to send tonnes of joins/leaves (as it invalidates the
cache). We therefore replace it with a custon txn function that builds
up the final result dict without building up and intermediate
representation.
2017-04-07 10:10:49 +01:00
Erik Johnston a41fe500d6 Bump version and changelog 2017-04-07 10:03:48 +01:00
Erik Johnston 54f59bd7d4 Merge pull request #2107 from HarHarLinks/patch-1
fix typo in synctl help
2017-04-07 09:54:37 +01:00
Erik Johnston 98ce212093 Merge pull request #2103 from matrix-org/erikj/no-double-encode
Don't double encode replication data
2017-04-07 09:39:52 +01:00
Kim Brose 8a1137ceab fix typo in synctl help 2017-04-06 17:10:20 +02:00
Erik Johnston 877c029c16 Use iteritems 2017-04-06 15:51:22 +01:00
Erik Johnston 944692ef69 Merge pull request #2106 from matrix-org/erikj/reduce_user_sync
Reduce rate of USER_SYNC repl commands
2017-04-06 13:35:31 +01:00
Erik Johnston 391712a4f9 Comment 2017-04-06 13:35:00 +01:00
Erik Johnston ad544c803a Document types of the replication streams 2017-04-06 13:28:52 +01:00
Erik Johnston dbf87282d3 Docs 2017-04-06 13:11:21 +01:00
Erik Johnston 69b3fd485d Fix incorrect type when using InvalidateCacheCommand 2017-04-06 09:36:38 +01:00
Daniel Dent 5058292537 Support authenticated SMTP
Closes (SYN-714) #1385

Signed-off-by: Daniel Dent <matrixcontrib@contactdaniel.net>
2017-04-05 21:01:08 -07:00
Erik Johnston fcc803b2bf Add log lines 2017-04-05 17:13:44 +01:00
Erik Johnston 3f213d908d Rearrange metrics 2017-04-05 14:15:09 +01:00
Erik Johnston 1ca0e78ca1 Fix typo 2017-04-05 13:43:39 +01:00