Commit Graph

1394 Commits (4ef7a25c103d14ca4192b64100e0c6bb2a5fd9e9)

Author SHA1 Message Date
Mark Haines 2a37467fa1 Use google style doc strings.
pycharm supports them so there is no need to use the other format.

Might as well convert the existing strings to reduce the risk of
people accidentally cargo culting the wrong doc string format.
2016-04-01 16:12:07 +01:00
Mark Haines f2b916534b Merge pull request #684 from matrix-org/markjh/backfill_id_gen
Use a stream id generator for backfilled ids
2016-04-01 15:13:14 +01:00
Mark Haines 9bc5b4c663 Assert that the step != 0 2016-04-01 15:08:20 +01:00
Mark Haines 35b5c4ba1b use google style doc strings 2016-04-01 15:07:01 +01:00
Mark Haines a2866e2e6a Rename direction to step, apply checks consistently 2016-04-01 13:50:54 +01:00
Mark Haines e36bfbab38 Use a stream id generator for backfilled ids 2016-04-01 13:29:05 +01:00
Mark Haines 7753fc6570 Fix the invalidation of the names and aliases cache 2016-04-01 10:34:51 +01:00
Mark Haines 5d06929169 Move the check for backfilled outside the for loop 2016-03-31 15:09:09 +01:00
Mark Haines 76503f95ed Remove the is_new_state argument to persist event.
Move the checks for whether an event is new state inside persist
event itself.

This was harder than expected because there wasn't enough information
passed to persist event to correctly handle invites from remote servers
for new rooms.
2016-03-31 15:00:42 +01:00
Mark Haines 62e395f0e3 Merge pull request #676 from matrix-org/markjh/replicate_stateIII
Add replication streams for ex outliers and current state resets
2016-03-31 11:20:57 +01:00
Mark Haines 2ec5426035 Use a namedtuple rather than tuple unpacking 2016-03-31 10:33:02 +01:00
Mark Haines 1fbb094c6f Add replication streams for ex outliers and current state resets 2016-03-30 17:19:56 +01:00
Mark Haines 8b8052909f return the state_group for backfill 2016-03-30 16:20:07 +01:00
Mark Haines 61407986b4 Add a entry to current_state_resets table when the current state is reset 2016-03-30 16:18:46 +01:00
Mark Haines 31a9eceda5 Add a replication stream for state groups 2016-03-30 16:01:58 +01:00
Mark Haines 1e25f62ee6 Use a stream id generator to assign state group ids 2016-03-30 12:55:02 +01:00
Erik Johnston 2f0180b09e Don't bother interning keys that are already interned 2016-03-23 16:29:46 +00:00
Erik Johnston acdfef7b14 Intern all the things 2016-03-23 16:25:54 +00:00
Erik Johnston 75daede92f String intern 2016-03-23 14:53:53 +00:00
Erik Johnston fbdeb1778d Merge pull request #664 from matrix-org/erikj/public_room_list
Don't require alias in public room list.
2016-03-23 14:42:01 +00:00
Erik Johnston b275765545 Comment about weird SQL 2016-03-23 14:15:32 +00:00
Erik Johnston 0c1a27b787 SQLite and postgres doesn't share a true literal 2016-03-23 14:10:49 +00:00
Erik Johnston 84afeb41f3 Ensure all old public rooms have aliases 2016-03-23 13:59:34 +00:00
Erik Johnston 2749da542c Merge pull request #663 from matrix-org/erikj/invite_for_user
Make get_invites return RoomsForUser
2016-03-23 13:19:26 +00:00
Erik Johnston e14baa7a3b Merge pull request #665 from matrix-org/erikj/dont_cache_events
Only cache events in the event cache
2016-03-23 13:19:12 +00:00
Erik Johnston d87a846ebc Don't cache events in get_recent_events_for_room 2016-03-23 11:42:50 +00:00
Erik Johnston 8b0dfc9fc4 Don't cache events in get_current_state_for_key 2016-03-23 11:42:17 +00:00
Erik Johnston b6507869cd Make get_invites return RoomsForUser 2016-03-23 10:32:10 +00:00
Erik Johnston 99f929f36b Make StateHandler._state_cache only store event_ids. 2016-03-22 16:06:04 +00:00
Erik Johnston 6cf0ba1466 Bump get_unread_event_push_actions_by_room_for_user cache 2016-03-22 14:18:21 +00:00
Erik Johnston 76d18a5776 Bump get_aliases_for_room cache 2016-03-22 14:08:13 +00:00
Erik Johnston fa2f96c2e3 Merge pull request #660 from matrix-org/erikj/state_cache
Don't cache events in _state_group_cache
2016-03-22 13:12:06 +00:00
Erik Johnston f93304e77f Merge pull request #659 from matrix-org/erikj/state_cache_factor
Make stateGroupCache honour CACHE_SIZE_FACTOR
2016-03-22 13:12:01 +00:00
Erik Johnston 2c86187a1b Don't cache events in _state_group_cache
Instead, simply cache the event ids, relying on the event cache to cache
the actual events.

The problem was that while the state groups cache was limited in the
number of groups it could hold, each individual group could consist of
thousands of events.
2016-03-22 12:00:09 +00:00
Erik Johnston d6ac752538 Merge pull request #657 from matrix-org/erikj/roomlist
Add published room list edit API
2016-03-22 11:57:39 +00:00
Erik Johnston b591277620 Make stateGroupCache honour CACHE_SIZE_FACTOR 2016-03-22 10:32:50 +00:00
Mark Haines 5244c0b48e Remove unused backfilled parameter from persist_event 2016-03-21 18:06:08 +00:00
Erik Johnston 3e7fac0d56 Add published room list edit API 2016-03-21 15:06:07 +00:00
Erik Johnston 67ed8065db Dedupe requested event list in _get_events 2016-03-18 14:31:31 +00:00
Mark Haines 673c96ce97 Remove dead code left over from presence changes 2016-03-17 11:03:47 +00:00
Mark Haines ba660ecde2 Add a comment to offer a hint to an explanation for why we have a unique constraint on (app_id, pushkey, user_id) 2016-03-16 10:35:00 +00:00
Mark Haines ee32d622ce Fix a couple of errors when deleting pushers 2016-03-15 17:47:36 +00:00
Mark Haines b6e8420aee Add replication stream for pushers 2016-03-15 17:33:10 +00:00
Erik Johnston 15122da0e2 Thats not how transactions work. 2016-03-11 16:45:27 +00:00
Erik Johnston b13035cc91 Implement logout 2016-03-11 16:27:50 +00:00
Erik Johnston c081228439 Fix SQL statement 2016-03-11 15:09:17 +00:00
Erik Johnston b5afe6bc38 Merge pull request #641 from matrix-org/dbkr/fix_change_password
Fix logout on password change
2016-03-11 14:48:20 +00:00
David Baker 2dee03aee5 more pep8 2016-03-11 14:38:23 +00:00
David Baker af59826a2f Make select more sensible when dseleting access tokens, rename pusher deletion to match access token deletion and make exception arg optional. 2016-03-11 14:34:09 +00:00
David Baker f523177850 Delete old, unused methods and rename new one to just be `user_delete_access_tokens` with an `except_token_ids` argument doing what it says on the tin. 2016-03-11 14:29:01 +00:00
David Baker 57c444b3ad Dear PyCharm, please indent sensibly for me. Thx. 2016-03-11 14:25:05 +00:00
Erik Johnston 58443a022d Merge pull request #635 from matrix-org/erikj/sync_order
Use topological orders for initial sync timeline
2016-03-11 13:17:49 +00:00
David Baker aa11db5f11 Fix cache invalidation so deleting access tokens (which we did when changing password) actually takes effect without HS restart. Reinstate the code to avoid logging out the session that changed the password, removed in 415c2f0549 2016-03-11 13:14:18 +00:00
Daniel Wagner-Hall 465605d616 Store appservice ID on register 2016-03-10 15:58:22 +00:00
Erik Johnston 9669a99d1a Update users table in a batched manner 2016-03-10 15:12:19 +00:00
Erik Johnston 8a88684736 Add comment 2016-03-09 16:51:22 +00:00
Erik Johnston af2fe6110c Return the correct token form 2016-03-09 16:11:53 +00:00
Erik Johnston 3ecaabc7fd Use topological orders for initial sync timeline 2016-03-09 15:45:34 +00:00
Erik Johnston 158a322e82 Ensure integer is an integer 2016-03-09 10:20:48 +00:00
Mark Haines 5f5817ab05 Merge pull request #632 from matrix-org/markjh/py3v2
Fix relative imports so they work in both py3 and py27
2016-03-08 18:11:10 +00:00
Daniel Wagner-Hall 3b97797c8d Merge branch 'develop' into daniel/ick 2016-03-08 17:35:09 +00:00
Daniel Wagner-Hall edca2d9891 Idempotent-ise schema update script
If any ASes don't have an ID, the schema will fail, and then it will
error when trying to add the column again.
2016-03-08 17:32:29 +00:00
Mark Haines 7076082ae6 Fix relative imports so they work in both py3 and py27 2016-03-08 11:45:50 +00:00
Mark Haines 9848b54cac Prefill from the correct stream 2016-03-04 16:20:22 +00:00
Mark Haines deda48068c prefill the push rules stream change cache 2016-03-04 16:19:42 +00:00
Mark Haines ebcbb23226 s/stream_ordering/event_stream_ordering/ in push 2016-03-04 16:15:23 +00:00
Mark Haines 7e9fc9b6af /FNARG/d 2016-03-04 15:54:09 +00:00
Mark Haines ec7460b4f2 Merge branch 'develop' into markjh/pushrule_stream 2016-03-04 14:44:34 +00:00
Mark Haines 1b4f4a936f Hook up the push rules stream to account_data in /sync 2016-03-04 14:44:01 +00:00
Erik Johnston dd463e246d Merge pull request #614 from matrix-org/erikj/alias_delete
Allow alias creators to delete aliases
2016-03-04 09:02:33 +00:00
Daniel Wagner-Hall 95481e7ba7 Merge pull request #571 from matrix-org/daniel/asids
Mark AS users with their AS's ID
2016-03-03 13:56:28 +00:00
Mark Haines 2223204eba Hook push rules up to the replication API 2016-03-02 17:26:20 +00:00
Mark Haines a1cf9e3bf3 Add a stream for push rule updates 2016-03-01 18:16:37 +00:00
Mark Haines a612ce6659 Merge pull request #489 from matrix-org/markjh/replication
Add a /replication API for extracting the updates that happened on synapse.
2016-03-01 15:08:24 +00:00
Mark Haines 60a0f81c7a Add a /replication API for extracting the updates that happened on
synapse

This is necessary for replicating the data in synapse to be visible to a
separate service because presence and typing notifications aren't stored
in a database so won't be visible to another process.

This API can be used to either get the raw data by requesting the tables
themselves or to just receive notifications for updates by following the
streams meta-stream.

Returns updates for each table requested a JSON array of arrays with a
row for each row in the table.

Each table is prefixed by a header row with the: name of the table,
current stream_id position for the table, number of rows, number of
columns and the names of the columns.
This is followed by the rows that have been added to the server since
the requester last asked.

The API has a timeout and is hooked up to the notifier so that a slave
can long poll for updates.
2016-03-01 14:49:41 +00:00
Erik Johnston f9af8962f8 Allow alias creators to delete aliases 2016-03-01 14:46:31 +00:00
Mark Haines 54172924c8 Load the current id in the IdGenerator constructor
Rather than loading them lazily. This allows us to remove all
the yield statements and spurious arguments for the get_next
methods.

It also allows us to replace all instances of get_next_txn with
get_next since get_next no longer needs to access the db.
2016-03-01 14:32:56 +00:00
Mark Haines de27f7fc79 Add support for changing the actions for default rules
See matrix-org/matrix-doc#283

Works by adding dummy rules to the push rules table with a negative priority class and then using those rules to clobber the default rule actions when adding the default rules in ``list_with_base_rules``
2016-02-26 14:28:19 +00:00
Daniel Wagner-Hall 33300673b7 Generate guest access token on 3pid invites
This means that following the same link across multiple sessions or
devices can re-use the same guest account.

Note that this is somewhat of an abuse vector; we can't throw up
captchas on this flow, so this is a way of registering ephemeral
accounts for spam, whose sign-up we don't rate limit.
2016-02-24 14:41:25 +00:00
Erik Johnston 6451fcd085 Create a new stream_id per presence update 2016-02-23 15:51:39 +00:00
Erik Johnston e5ad2e5267 Merge pull request #582 from matrix-org/erikj/presence
Rewrite presence for performance.
2016-02-19 09:37:50 +00:00
Erik Johnston e6c5e3f28a Close cursor 2016-02-18 16:39:28 +00:00
Erik Johnston 42109a62a4 Remove unused param from get_max_token 2016-02-18 16:37:28 +00:00
Mark Haines b9977ea667 Remove dead code for setting device specific rules.
It wasn't possible to hit the code from the API because of a typo
in parsing the request path. Since no-one was using the feature
we might as well remove the dead code.
2016-02-18 16:05:13 +00:00
Erik Johnston 112283e230 Prefix TS fields with _ts 2016-02-18 10:11:43 +00:00
Daniel Wagner-Hall 7e90fb6a57 Merge branch 'develop' into daniel/roomcleanupincremental
Conflicts:
	synapse/rest/client/v1/room.py
2016-02-17 15:53:59 +00:00
Erik Johnston e5999bfb1a Initial cut 2016-02-17 15:40:50 +00:00
Daniel Wagner-Hall a4e278bfe7 Respond to federated invite with non-empty context
Currently, we magically perform an extra database hit to find the
inviter, and use this to guess where we should send the event. Instead,
fill in a valid context, so that other callers relying on the context
actually have one.
2016-02-17 15:25:12 +00:00
Mark Haines a9c9868957 Make adding push rules idempotent
Also remove the **kwargs from the add_push_rule method.

Fixes https://matrix.org/jira/browse/SYN-391
2016-02-16 15:53:38 +00:00
Daniel Wagner-Hall 763360594d Mark AS users with their AS's ID 2016-02-11 17:26:42 +00:00
Erik Johnston ce14c7a995 Fix SYN-627, events are in incorrect room in /sync 2016-02-11 15:02:56 +00:00
Erik Johnston 0eff740523 Return events in correct order for /events 2016-02-11 10:07:27 +00:00
Erik Johnston 5189bfdef4 Batch fetch _get_state_groups_from_groups 2016-02-10 13:24:42 +00:00
Erik Johnston 24f00a6c33 Use _simple_select_many for _get_state_group_for_events 2016-02-10 12:57:50 +00:00
Erik Johnston f7ef5c1d57 Merge pull request #568 from matrix-org/erikj/unread_notif
Atomically persit push actions when we persist the event
2016-02-10 11:25:32 +00:00
Erik Johnston 0214745239 Rename functions 2016-02-10 11:09:56 +00:00
Erik Johnston 7b0d846407 Atomically persit push actions when we persist the event 2016-02-09 16:19:15 +00:00
Erik Johnston 78d6c1b5be Change a log from debug to info 2016-02-09 14:44:12 +00:00
Erik Johnston 70a8608749 Invalidate get_last_receipt_event_id_for_user cache 2016-02-09 14:27:29 +00:00
Erik Johnston eff12e838c Don't load all ephemeral state for a room on every sync 2016-02-09 13:55:59 +00:00