c5a296b10c
Add support for persisting event format versions
...
Currently we only have the one event format version defined, but this
adds the necessary infrastructure to persist and fetch the format
versions alongside the events.
We specify the format version rather than the room version as:
1. We don't necessarily know the room version, existing events may be
either v1 or v2.
2. We'd need to be careful to prevent/handle correctly if different
events in the same room reported to be of different versions, which
sounds annoying.
2019-01-23 11:30:01 +00:00
bc80b3f454
Add helpers for getting prev and auth events ( #4139 )
...
* Add helpers for getting prev and auth events
This is in preparation for allowing the event format to change between
room versions.
2018-11-06 00:35:15 +11:00
7232917f12
Disable frozen dicts by default ( #3987 )
2018-10-02 22:53:47 +10:00
89a76d1889
Fix handling of redacted events from federation
...
If we receive an event that doesn't pass their content hash check (e.g.
due to already being redacted) then we hit a bug which causes an
exception to be raised, which then promplty stops the event (and
request) from being processed.
This effects all sorts of federation APIs, including joining rooms with
a redacted state event.
2018-09-13 15:44:12 +01:00
c1f80effbe
Handle delta_ids being None in _update_context_for_auth_events
...
it's easier to create the new state group as a delta from the existing one.
(There's an outside chance this will help with
https://github.com/matrix-org/synapse/issues/3364 )
2018-07-23 22:06:50 +01:00
8b9f164fff
Comments
2018-07-23 17:43:01 +01:00
50c60e5fad
Only get cached state from context in persist_event
...
We don't want to bother pulling out the current state from the DB since
until we know we have to. Checking the context for state is just an
optimisation.
2018-07-23 17:21:40 +01:00
0faa3223cd
Fix missing attributes on workers.
...
This was missed during the transition from attribute to getter for
getting state from context.
2018-07-23 16:28:00 +01:00
999bcf9d01
Fix EventContext when using workers
...
We were:
1. Not correctly setting all attributes
2. Using defer.inlineCallbacks in a non-generator
2018-07-23 15:24:21 +01:00
4797ed000e
Update docstrings to make sense
2018-07-23 15:05:56 +01:00
027bc01a1b
Add support for updating state
2018-07-23 13:17:25 +01:00
440b8845b5
Make EventContext lazy load state
2018-07-23 12:56:56 +01:00
842cdece42
pep8
2018-07-23 12:31:35 +01:00
acbfdc3442
Refcator EventContext to accept state during init
2018-07-23 12:20:55 +01:00
3132b89f12
Make the rest of the .iterwhatever go away ( #3562 )
2018-07-21 15:47:18 +10:00
49af402019
run isort
2018-07-09 16:09:20 +10:00
c936a52a9e
Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy ( #3307 )
2018-05-31 19:03:47 +10:00
d9fe2b2d9d
Replace some more comparisons with six
...
plus a bonus b"" string I missed last time
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:56:31 +02:00
0c9db26260
add comment explaining attributeerror
2018-04-30 09:49:10 +02:00
bfc2ade9b3
Make event properties raise AttributeError instead
...
They raised KeyError before. I'm changing this because the code uses
hasattr() to check for the presence of a key. This worked accidentally
before, because hasattr() silences all exceptions in python 2. However,
in python3, this isn't the case anymore.
I had a look around to see if anything depended on this raising a
KeyError and I couldn't find anything. Of course, I could have simply
missed it.
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 20:16:59 +02:00
4810f7effd
Remove context.push_actions
2018-02-15 15:47:06 +00:00
106906a65e
Don't serialize current state over replication
2018-02-15 13:53:18 +00:00
24dd73028a
Add replication http endpoint for event sending
2018-02-07 10:32:32 +00:00
3d33eef6fc
Store state groups separately from events ( #2784 )
...
* Split state group persist into seperate storage func
* Add per database engine code for state group id gen
* Move store_state_group to StateReadStore
This allows other workers to use it, and so resolve state.
* Hook up store_state_group
* Fix tests
* Rename _store_mult_state_groups_txn
* Rename StateGroupReadStore
* Remove redundant _have_persisted_state_group_txn
* Update comments
* Comment compute_event_context
* Set start val for state_group_id_seq
... otherwise we try to recreate old state groups
* Update comments
* Don't store state for outliers
* Update comment
* Update docstring as state groups are ints
2018-02-06 14:31:24 +00:00
eaaabc6c4f
replace 'except:' with 'except Exception:'
...
what could possibly go wrong
2017-10-23 15:52:32 +01:00
631d7b87b5
Remove pointless create() method
...
It just calls the constructor, so we may as well kill it rather than having
random codepaths.
2017-10-20 22:14:55 +01:00
f878e6f8af
Spam checking: add the invitee to user_may_invite
2017-10-05 14:02:28 +01:00
d8ce68b09b
spam check room publishing
2017-10-04 14:29:33 +01:00
197c14dbcf
Add room creation checks to spam checker
...
Lets the spam checker deny attempts to create rooms and add aliases
to them.
2017-10-04 10:47:54 +01:00
1e2ac54351
s/roomid/room_id/
2017-10-03 17:41:38 +01:00
1e375468de
pass room id too
2017-10-03 17:13:14 +01:00
537088e7dc
Actually write warpper function
2017-10-03 14:28:12 +01:00
ef3a5ae787
Don't test is spam_checker not None
...
Sometimes it's a Mock object which is not none but is still not
what we're after
2017-09-27 11:24:19 +01:00
8c06dd6071
Remove unintentional debugging
2017-09-27 10:31:14 +01:00
60c78666ab
pep8
2017-09-27 10:26:13 +01:00
6cd5fcd536
Make the spam checker a module
2017-09-26 19:20:23 +01:00
f496399ac4
fix thinko'd docstring
2017-09-22 15:34:14 +01:00
5ed109d59f
PoC for filtering spammy events ( #2456 )
...
Demonstration of how you might add some hooks to filter out spammy events.
2017-09-19 12:20:11 +01:00
3e5a62ecd8
Add more granular event send metrics
2017-05-02 14:23:26 +01:00
46826bb078
Comment and remove spurious logging
2017-04-27 17:25:44 +01:00
1e166470ab
Fix tests
2017-04-26 16:23:30 +01:00
34e682d385
Fix invite state to always include all events
2017-04-26 16:18:08 +01:00
5068fb16a5
Refactoring and cleanups
...
A few non-functional changes:
* A bunch of docstrings to document types
* Split `EventsStore._persist_events_txn` up a bit. Hopefully it's a bit more
readable.
* Rephrase `EventFederationStore._update_min_depth_for_room_txn` to avoid
mind-bending conditional.
* Rephrase rejected/outlier conditional in `_update_outliers_txn` to avoid
mind-bending conditional.
2017-03-17 15:06:07 +00:00
5d6bad1b3c
Optimise state resolution
2017-01-17 13:22:19 +00:00
2f4b2f4783
gah, fix mangled merge of 0.18.7 into develop
2017-01-07 04:00:42 +00:00
f784980d2b
Only send events that originate on this server.
...
Or events that are sent via the federation "send_join" API.
This should match the behaviour from before v0.18.5 and #1635 landed.
2017-01-05 11:26:30 +00:00
c3d963ac24
Review comments
2016-11-22 13:42:11 +00:00
6d4e6d4cba
Also check for dict since sometimes they aren't frozen
2016-11-22 10:39:41 +00:00
cea4e4e7b2
Glue only_event_fields into the sync rest servlet
2016-11-22 10:14:05 +00:00
0a8b0eeca1
More tests
2016-11-22 09:59:27 +00:00
70a2157b64
Start adding some tests
2016-11-21 17:52:45 +00:00
f97511a1f3
Move event_fields filtering to serialize_event
...
Also make it an inclusive not exclusive filter, as the spec demands.
2016-11-21 17:42:16 +00:00
628e65721b
Add comments
2016-09-05 10:41:27 +01:00
9e25443db8
Move to storing state_groups_state as deltas
2016-09-01 14:31:26 +01:00
c10cb581c6
Correctly handle the difference between prev and current state
2016-08-31 14:26:22 +01:00
54cc69154e
Make None optional
2016-08-26 11:20:59 +01:00
a3dc1e9cbe
Replace context.current_state with context.current_state_ids
2016-08-25 17:32:22 +01:00
17f4f14df7
Pull out event ids rather than full events for state
2016-08-25 13:42:44 +01:00
1d40373c9d
Include prev_content in redacted state events
2016-08-11 10:24:41 +01:00
76503f95ed
Remove the is_new_state argument to persist event.
...
Move the checks for whether an event is new state inside persist
event itself.
This was harder than expected because there wasn't enough information
passed to persist event to correctly handle invites from remote servers
for new rooms.
2016-03-31 15:00:42 +01:00
1e25f62ee6
Use a stream id generator to assign state group ids
2016-03-30 12:55:02 +01:00
acdfef7b14
Intern all the things
2016-03-23 16:25:54 +00:00
f96526ffc2
Intern sender, event_id and room_id in events
2016-03-23 15:04:11 +00:00
fe9794706a
Intern type and state_key on events
2016-03-23 14:58:08 +00:00
58e207cd77
Don't assume existence of event_id in __str__
2016-03-18 14:31:44 +00:00
7b0d846407
Atomically persit push actions when we persist the event
2016-02-09 16:19:15 +00:00
2068678b8c
Make Event objects behave more like dicts
2016-01-18 14:43:50 +00:00
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
306415391d
Only add the user_id if the sender is present
2015-12-01 11:14:48 +00:00
bde8d78b8a
Copy rather than move the fields to shuffle between a v1 and a v2 event.
...
This should make all v1 APIs compatible with v2 clients. While still
allowing v1 clients to access the fields.
This makes the documentation easier since we can just document the v2
format and explain that some of the fields, in some of the APIs are
duplicated for backwards compatibility, rather than having to document
two separate event formats.
2015-11-30 17:46:35 +00:00
3c3fc6b268
Flatten the /sync response to remove the event_map
2015-11-19 10:51:11 +00:00
d58edd98e9
Update the other place check_joined_room is called
2015-10-30 11:15:37 +00:00
243a79d291
Surely we don't need to preserve 'events_default' twice
2015-10-16 18:25:19 +01:00
49ae42bbe1
Bundle in some room state in the unsigned bit of the invite when sending to invited servers
2015-09-10 14:25:54 +01:00
1f7642efa9
Fix bug where we didn't correctly serialize the redacted_because key over federation
2015-08-21 09:36:07 +01:00
fb47c3cfbe
Rename key and values for m.room.history_visibility. Support 'invited' value
2015-07-06 13:05:52 +01:00
00ab882ed6
Add m.room.history_visibility to list of auth events
2015-07-03 10:31:24 +01:00
ca580ef862
Don't copy twice
2015-05-29 15:02:55 +01:00
a7b65bdedf
Add config option to turn off freezing events. Use new encode_json api and ujson.loads
2015-05-29 12:17:33 +01:00
1b988b051b
Store the rejected reason in (Frozen)Event structs
2015-03-25 19:06:05 +00:00
b2e6ee5b43
Remove concept of context.auth_events, instead use context.current_state
2015-03-16 13:06:23 +00:00
7eef84a95b
pyflakes
2015-02-11 16:52:22 +00:00
ddb816cf60
Don't unfreeze when using FreezeEvent.get_dict, as we are using a JSONEncoder that understands FrozenDict
2015-02-11 15:44:28 +00:00
c0462dbf15
Rearrange persist_event so that do all the queries that need to be done before returning early if we have already persisted that event.
2015-02-04 10:16:51 +00:00
9bace3a367
Actually, the old prune_event function was non-deterministic, so no point keeping it around :(
2015-02-03 15:32:17 +00:00
0dd3aea319
Keep around the old (buggy) version of the prune_event function so that we can use it to check signatures for events on old servers
2015-02-03 14:58:30 +00:00
7d897f5bfc
Merge pull request #43 from matrix-org/rejections
...
Rejections
2015-01-30 16:11:14 +00:00
6dc92d3427
Merge pull request #41 from matrix-org/client_v2_sync
...
Client v2 sync
2015-01-30 15:54:25 +00:00
e0b7c521cb
Merge branch 'develop' of github.com:matrix-org/synapse into rejections_storage
...
Conflicts:
synapse/storage/__init__.py
synapse/storage/schema/delta/v12.sql
2015-01-30 14:08:28 +00:00
2ebf795c0a
Merge branch 'develop' of github.com:matrix-org/synapse into rejections
...
Conflicts:
synapse/storage/__init__.py
synapse/storage/schema/delta/v12.sql
2015-01-30 11:10:37 +00:00
c1c7b39827
Fix bug where we changes in outlier in metadata dict propogated to other events
2015-01-30 10:30:54 +00:00
78015948a7
Initial implementation of auth conflict resolution
2015-01-29 16:52:33 +00:00
b9c442c85c
Include transaction ids in unsigned section of events in the sync results for the clients that made those requests
2015-01-29 02:46:00 +00:00
1b4a164c02
Add support for formatting events in the way a v2 client expects
2015-01-29 02:34:35 +00:00
a56008842b
Start implementing incremental initial sync
2015-01-27 16:24:22 +00:00
436513068d
Start implementing the non-incremental sync portion of the v2 /sync API
2015-01-26 18:53:31 +00:00
7f6f3f9d62
Pass the current time to serialize event, rather than passing an
...
HS and getting a clock from it and calling time_msec on the clock.
Remove the serialize_event method from the HS since it is no longer
needed.
2015-01-26 16:11:28 +00:00
b1b85753d7
Add support for storing rejected events in EventContext and data stores
2015-01-22 15:50:17 +00:00
3891597eb3
Remove unused functions
2015-01-13 15:57:26 +00:00
38e3241eb7
Merge branch 'hotfixes-v0.6.1b' of github.com:matrix-org/synapse into develop
2015-01-13 10:01:22 +00:00
39585bf556
Insert 'age' into top level when returning events to clients
2015-01-13 09:57:32 +00:00
edb557b2ad
Return the raw federation event rather than adding extra keys for federation data.
2015-01-08 14:28:08 +00:00
5720ab59e0
Add 'raw' query parameter to expose the event graph and signatures to savvy clients.
2015-01-08 13:57:40 +00:00
a039e2544c
Remove unused import
2015-01-07 09:48:03 +00:00
dfa05f0cd6
Optimize FrozenEvent creation
2015-01-06 18:51:03 +00:00
adb04b1e57
Update copyright notices
2015-01-06 13:21:39 +00:00
5dbe820e9a
Remove unneeded federation keys from events
2014-12-19 12:16:26 +00:00
3c7857e49b
clean up coding style a bit
2014-12-16 15:24:03 +00:00
882dc8dcab
Persist internal_metadata
2014-12-16 13:17:09 +00:00
65b2e49429
Fix pyflakes
2014-12-15 17:35:37 +00:00
f280929a12
Use frozenutils
2014-12-15 17:31:36 +00:00
670dcdfc14
Remove unused functions
2014-12-15 16:16:58 +00:00
b75adaedca
Finish up upgrade script
2014-12-15 16:14:34 +00:00
c8dd3314d6
Fix bug where we ignored event_edge_hashes table
2014-12-15 13:55:22 +00:00
63810c777d
Validate message, topic and name event contents
2014-12-12 11:01:09 +00:00
9191292b0f
Fix prev_content
2014-12-11 15:16:55 +00:00
8cdebce470
Fix redactions. Fix 'age' key
2014-12-11 13:25:19 +00:00
b245ee34ed
Add some basic event validation
2014-12-10 17:59:47 +00:00
cabead6194
Actually fix bug when uploading state with empty state_key
2014-12-10 14:49:52 +00:00
02db7eb209
Fix bug when uploading state with empty state_key
2014-12-10 14:02:48 +00:00
02db1fd2e7
Fix AttributeError
2014-12-10 12:00:05 +00:00
102d2373b4
Add __str__ to FrozenEvent
2014-12-10 11:38:08 +00:00
95aa903ffa
Try and figure out how and why signatures are being changed.
2014-12-10 11:37:47 +00:00
8c48450682
Add PEP8 newlines
2014-12-09 10:58:31 +00:00
609c31e8df
More bug fixes
2014-12-08 17:50:56 +00:00
d044121168
Various typos and bug fixes.
2014-12-08 09:08:26 +00:00
6630e1b579
Start making more things use EventContext rather than event.*
2014-12-05 16:20:48 +00:00
c31dba86ec
Convert rest and handlers to use new event structure
2014-12-04 15:50:01 +00:00
5d7c9ab789
Begin converting things to use the new Event structure
2014-12-04 11:27:59 +00:00
75b4329aaa
WIP for new way of managing events.
2014-12-03 16:07:21 +00:00
c1e66800a9
Begin fleshing out a new Event object
2014-12-02 11:40:22 +00:00
Erik Johnston