940a161192
Fix the background update
2015-11-11 13:59:40 +00:00
cf437900e0
Return world_readable and guest_can_join in /publicRooms
2015-11-10 17:10:27 +00:00
90b503216c
Use a background task to update databases to use the full text search
2015-11-10 16:20:13 +00:00
a412b9a465
Run the background updates when starting synapse.
2015-11-10 15:50:58 +00:00
2ede7aa8a1
Add background update task for reindexing event search
2015-11-09 19:29:32 +00:00
c6a01f2ed0
Add storage module for tracking background updates.
...
The progress for each background update is stored as a JSON blob in the
database. Each background update is broken up into separate batches.
The batch size is automatically tuned to try avoid blocking single
threaded databases for too long.
2015-11-09 17:26:27 +00:00
bbe10e8be7
Merge branch 'develop' into markjh/SYN-513
...
Conflicts:
synapse/storage/tags.py
2015-11-09 15:01:59 +00:00
c4135d85e1
SYN-513: Include updates for rooms that have had all their tags deleted
2015-11-09 14:53:08 +00:00
dd40fb68e4
fix comedy important missing comma breaking recent-ordered FTS on sqlite
2015-11-08 16:04:37 +00:00
767c20a869
add a key existence check to tags_by_room to avoid /events 500'ing when testing against vector
2015-11-06 20:49:57 +01:00
f2c4ee41b9
Remove accidentally added ID column
2015-11-06 14:27:49 +00:00
3640ddfbf6
Error handling
2015-11-05 16:10:54 +00:00
729ea933ea
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search
2015-11-05 15:43:52 +00:00
7301e05122
Implement basic pagination for search results
2015-11-05 15:04:08 +00:00
ca2f90742d
Open up /events to anonymous users for room events only
...
Squash-merge of PR #345 from daniel/anonymousevents
2015-11-05 14:32:26 +00:00
05c326d445
Implement order and group by
2015-11-04 17:57:44 +00:00
4e62ffdb21
Merge branch 'develop' of github.com:matrix-org/synapse into develop
2015-11-04 17:31:01 +00:00
f522f50a08
Allow guests to register and call /events?room_id=
...
This follows the same flows-based flow as regular registration, but as
the only implemented flow has no requirements, it auto-succeeds. In the
future, other flows (e.g. captcha) may be required, so clients should
treat this like the regular registration flow choices.
2015-11-04 17:29:07 +00:00
33b3e04049
Merge branch 'develop' into daniel/removesomelies
...
Conflicts:
synapse/notifier.py
2015-11-04 16:01:00 +00:00
6a3a840b19
Merge pull request #343 from matrix-org/erikj/fix_retries
...
Fix broken cache for getting retry times.
2015-11-03 17:51:49 +00:00
6797fcd9ab
Merge pull request #335 from matrix-org/markjh/room_tags
...
Add APIs for adding and removing tags from rooms
2015-11-03 16:45:53 +00:00
7ce264ce5f
Fix broken cache for getting retry times. This meant we retried remote destinations way more frequently than we should
2015-11-03 16:24:03 +00:00
5897e773fd
Spell "deferred" more correctly
2015-11-03 14:27:35 +00:00
771ca56c88
Remove more unused parameters
2015-11-02 15:31:57 +00:00
ddd8566f41
Store room tag content and return the content in the m.tag event
2015-11-02 15:11:31 +00:00
0e36756383
Merge branch 'develop' into markjh/room_tags
2015-11-02 10:57:00 +00:00
621e84d9a0
Add missing column
2015-10-30 16:25:53 +00:00
f40b0ed5e1
Inform the client of new room tags using v1 /events
2015-10-29 15:21:09 +00:00
a89b86dc47
Fix pyflakes errors
2015-10-28 16:45:57 +00:00
892e70ec84
Add APIs for adding and removing tags from rooms
2015-10-28 16:06:57 +00:00
56dbcd1524
Docs
2015-10-28 14:05:50 +00:00
5cb298c934
Add room context api
2015-10-28 13:45:56 +00:00
4cf633d5e9
Pull out sender when computing search results
2015-10-23 15:41:36 +01:00
0c36098c1f
Implement rank function for SQLite FTS
2015-10-23 13:23:48 +01:00
671ac699f1
Actually filter results
2015-10-22 16:54:56 +01:00
fb0fecd0b9
LESS THAN
2015-10-22 16:18:35 +01:00
232beb3a3c
Use namedtuple as return value
2015-10-22 15:02:35 +01:00
ba02bba88c
Limit max number of SQL vars
2015-10-22 13:25:27 +01:00
8a98f0dc5b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search
2015-10-22 13:16:35 +01:00
f142898f52
PEP8
2015-10-22 11:18:01 +01:00
4d25bc6c92
Move FTS to delta 25
2015-10-22 11:12:28 +01:00
e94ffd89d6
Merge pull request #316 from matrix-org/markjh/v2_sync_archived
...
Add rooms that the user has left under archived in v2 sync.
2015-10-21 15:46:41 +01:00
e3d75f564a
Include banned rooms in the archived section of v2 sync
2015-10-21 11:15:48 +01:00
68b7fc3e2b
Add rooms that the user has left under archived in v2 sync.
2015-10-19 17:26:18 +01:00
46d39343d9
Explicitly check for Sqlite3Engine
2015-10-16 16:58:00 +01:00
f2d698cb52
Typing
2015-10-16 16:46:48 +01:00
33646eb000
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search
2015-10-16 15:35:35 +01:00
fc012aa8dc
Fix FilteringStore.get_user_filter to work with postgres
2015-10-16 15:28:43 +01:00
12122bfc36
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search
2015-10-16 14:46:32 +01:00
edb998ba23
Explicitly check for Sqlite3Engine
2015-10-16 14:37:14 +01:00
3cf9948b8d
Add docstring
2015-10-16 11:28:12 +01:00
73260ad01f
Comment on the LIMIT 500
2015-10-16 11:24:02 +01:00
22a8c91448
Split up run_upgrade
2015-10-16 11:19:44 +01:00
6296590bf7
Encode the filter JSON as UTF-8 before storing in the database.
...
Because we are using a binary column type to store the filter JSON.
2015-10-16 10:50:32 +01:00
bcfb653816
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/search
2015-10-15 16:37:32 +01:00
a059760954
Merge pull request #305 from matrix-org/markjh/v2_sync_api
...
Update the v2 sync API to work as specified in the current spec.
2015-10-14 13:56:23 +01:00
f50c43464c
Merge branch 'develop' into markjh/v2_sync_api
2015-10-14 11:40:45 +01:00
99c7fbfef7
Fix to work with SQLite
2015-10-14 09:52:40 +01:00
858634e1d0
Remove unused room_id arg
2015-10-14 09:31:20 +01:00
7ecd11accb
Add paranoia limit
2015-10-13 15:50:56 +01:00
3e2a1297b5
Remove constraints in preperation of using filters
2015-10-13 15:22:14 +01:00
cacf0688c6
Add a get_invites_for_user method to the storage to find out the rooms a user is invited to
2015-10-13 14:08:38 +01:00
88971fd034
Merge branch 'erikj/store_engine' into erikj/search
2015-10-13 14:03:30 +01:00
17c80c8a3d
rename schema_prepare to prepare_database
2015-10-13 13:56:22 +01:00
cfd39d6b55
Add SQLite support
2015-10-13 13:47:50 +01:00
f9340ea0d5
Merge branch 'erikj/store_engine' into erikj/search
2015-10-13 13:29:02 +01:00
ec398af41c
Expose error more nicely
2015-10-13 11:43:43 +01:00
40b6a5aad1
Split out the schema preparation and update logic into its own module
2015-10-13 11:38:48 +01:00
1a40afa756
Add sqlite schema
2015-10-13 10:36:25 +01:00
ca53ad7425
Filter events to only thsoe that the user is allowed to see
2015-10-12 15:52:55 +01:00
927004e349
Remove unused room_id parameter
2015-10-12 15:06:14 +01:00
61561b9df7
Keep FTS indexes up to date. Only search through rooms currently joined
2015-10-12 10:49:53 +01:00
c85c912562
Add basic full text search impl.
2015-10-09 15:48:31 +01:00
301141515a
Merge pull request #288 from matrix-org/markjh/unused_definitions
...
Remove some of the unused definitions from synapse
2015-09-28 14:22:44 +01:00
cf1100887b
Fix order of ON constraints in _get_rooms_for_user_where_membership_is_txn
2015-09-24 17:35:10 +01:00
1cd65a8d1e
synapse/storage/state.py: _make_group_id was unused
2015-09-23 10:37:58 +01:00
973ebb66ba
Remove unused functions from synapse/storage/signatures.py
2015-09-23 10:36:33 +01:00
e51aa4be96
synapse/storage/roommember.py:_get_members_query was unused
2015-09-23 10:35:10 +01:00
92d8d724c5
Remove unused functions from synapse/storage/events.py
2015-09-23 10:33:06 +01:00
c292dba70c
Remove unused functions from synapse/storage/event_federation.py
2015-09-23 10:31:25 +01:00
396834f1c0
synapse/storage/_base.py:_simple_max_id was unused
2015-09-23 10:30:38 +01:00
1d9036aff2
synapse/storage/_base.py:_simple_delete was unused
2015-09-23 10:30:25 +01:00
1ee3d26432
synapse/storage/_base.py:_simple_selectupdate_one was unused
2015-09-23 10:30:03 +01:00
3559a835a2
synapse/storage/event_federation.py:_get_auth_events is unused
2015-09-22 18:39:46 +01:00
7dd4f79c49
synapse/storage/_base.py:_execute_and_decode was unused
2015-09-22 18:37:07 +01:00
527d95dea0
synapse/storage/_base.py:Table was unused
2015-09-22 18:14:15 +01:00
6d59ffe1ce
Add some docstrings
2015-09-22 13:47:40 +01:00
eb011cd99b
Add docstring
2015-09-22 13:29:36 +01:00
7213588083
Implement configurable stats reporting
...
SYN-287
This requires that HS owners either opt in or out of stats reporting.
When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
2015-09-22 12:57:40 +01:00
ee2d722f0f
Merge pull request #276 from matrix-org/markjh/history_for_rooms_that_have_been_left
...
SPEC-216: Allow users to view the history of rooms that they have left.
2015-09-21 14:38:13 +01:00
2048388cfd
Merge pull request #281 from matrix-org/erikj/update_extremeties
...
When updating a stored event from outlier to non-outlier, remember to update the extremeties
2015-09-15 16:57:25 +01:00
8148c48f11
"Comments"
2015-09-15 16:54:48 +01:00
3b05b67c89
When updating a stored event from outlier to non-outlier, remember to update the extremeties
2015-09-15 16:34:42 +01:00
dffc9c4ae0
Drop unused index
2015-09-14 14:41:37 +01:00
09cb5c7d33
Allow users that have left a room to get the messages that happend in the room before they left
2015-09-09 17:31:09 +01:00
3c166a24c5
Remove undocumented and unimplemented 'feedback' parameter from the Client-Server API
2015-09-09 16:05:09 +01:00
89ae0166de
Allow room initialSync for users that have left the room, returning a snapshot of how the room was when they left it
2015-09-09 13:25:22 +01:00
417485eefa
Include the event_id and stream_ordering of membership events when looking up which rooms a user is in
2015-09-08 18:14:54 +01:00
b62c1395d6
Merge branch 'release-v0.10.0' of github.com:matrix-org/synapse into develop
2015-09-01 13:11:55 +01:00
cd800ad99a
Lower size of 'stateGroupCache' now that we have data from matrix.org to support doing so
2015-09-01 10:09:03 +01:00
a7122692d9
Merge branch 'release-v0.10.0' into develop
...
Conflicts:
synapse/handlers/auth.py
synapse/python_dependencies.py
synapse/rest/client/v1/login.py
2015-08-28 11:15:27 +01:00
d3c0e48859
Merge erikj/user_dedup to develop
2015-08-26 13:42:45 +01:00
a2355fae7e
Merge pull request #251 from matrix-org/removeadmin
...
Stop looking up "admin", which we never read
2015-08-25 17:23:05 +01:00
ee3fa1a99c
Merge pull request #248 from matrix-org/deviceid
...
Remove completely unused concepts from codebase
2015-08-25 17:19:06 +01:00
a9d8bd95e7
Stop looking up "admin", which we never read
2015-08-25 16:29:39 +01:00
a0b181bd17
Remove completely unused concepts from codebase
...
Removes device_id and ClientInfo
device_id is never actually written, and the matrix.org DB has no
non-null entries for it. Right now, it's just cluttering up code.
This doesn't remove the columns from the database, because that's
fiddly.
2015-08-25 16:23:06 +01:00
1c847af28a
Merge pull request #243 from matrix-org/markjh/remove_syutil
...
Replace syutil dependency with smaller, single-purpose libraries
2015-08-25 10:52:16 +01:00
037481a033
Remove autoincrement since we incrementing the ID in the storage layer
2015-08-24 17:48:57 +01:00
01fc3943f1
Fix indent
2015-08-24 17:18:58 +01:00
78323ccdb3
Remove syutil dependency in favour of smaller single-purpose libraries
2015-08-24 16:17:38 +01:00
457970c724
Don't insert events into 'event_*_extremeties' tables if they're outliers
2015-08-23 13:44:23 +01:00
1bd1a43073
Actually check if event_id isn't returned by _get_state_groups
2015-08-21 14:30:34 +01:00
745b72660a
Merge branch 'release-v0.10.0' of github.com:matrix-org/synapse into develop
2015-08-21 11:39:38 +01:00
42f12ad92f
When logging in fetch user by user_id case insensitively, *unless* there are multiple case insensitive matches, in which case require the exact user_id
2015-08-21 11:38:44 +01:00
aa3c9c7bd0
Don't allow people to register user ids which only differ by case to an existing one
2015-08-21 10:57:47 +01:00
1f7642efa9
Fix bug where we didn't correctly serialize the redacted_because key over federation
2015-08-21 09:36:07 +01:00
e8cf77fa49
Merge branch 'develop' into refresh
...
Conflicts:
synapse/rest/client/v1/login.py
2015-08-20 16:25:40 +01:00
cecbd636e9
/tokenrefresh POST endpoint
...
This allows refresh tokens to be exchanged for (access_token,
refresh_token).
It also starts issuing them on login, though no clients currently
interpret them.
2015-08-20 16:21:35 +01:00
13a6517d89
s/by_token/by_access_token/g
...
We're about to have two kinds of token, access and refresh
2015-08-20 16:01:29 +01:00
a0b8e5f2fe
Merge pull request #211 from matrix-org/email_in_use
...
Changes for unique emails
2015-08-20 10:04:04 +01:00
e624cdec64
Merge pull request #228 from matrix-org/erikj/_get_state_for_groups
...
Ensure we never return a None event from _get_state_for_groups
2015-08-18 16:30:17 +01:00
38a965b816
Merge pull request #227 from matrix-org/erikj/receipts_take2
...
Re-enable receipts API.
2015-08-18 16:30:04 +01:00
a82938416d
Remove newline because vertical whitespace makes mjark sad
2015-08-18 16:28:13 +01:00
0bfdaf1f4f
Rejig the code to make it nicer
2015-08-18 16:26:07 +01:00
128ed32e6b
Bump size of get_presence_state cache
2015-08-18 15:51:23 +01:00
f704c10f29
Rename unhelpful variable name
2015-08-18 11:54:03 +01:00
8199475ce0
Ensure we never return a None event from _get_state_for_groups
2015-08-18 11:44:10 +01:00
85d0bc3bdc
Reduce cache size from obscenely large to quite large
2015-08-18 11:00:38 +01:00
cfc503681f
Comments
2015-08-18 10:49:23 +01:00
8f4165628b
Add index receipts_linearized_room_stream
2015-08-17 14:43:54 +01:00
d3d582bc1c
Remove unused import
2015-08-17 13:38:09 +01:00
afef6f5d16
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts_take2
2015-08-17 13:23:44 +01:00
2d97e65558
Remember to invalidate caches
2015-08-17 10:46:55 +01:00
1a9510bb84
Implement a batched presence_handler.get_state and use it
2015-08-17 10:40:23 +01:00
47abebfd6d
Add batched version of store.get_presence_state
2015-08-17 09:50:50 +01:00
f9d4da7f45
Fix bug where we were leaking None into state event lists
2015-08-17 09:39:45 +01:00
30883d8409
Merge pull request #221 from matrix-org/auth
...
Simplify LoginHander and AuthHandler
2015-08-14 17:02:22 +01:00
68b255c5a1
Batch _get_linearized_receipts_for_rooms
2015-08-14 15:06:22 +01:00
129ee4e149
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/receipts_take2
2015-08-13 17:28:43 +01:00
2bb2c02571
Remove some vertical space
2015-08-13 17:11:30 +01:00
57877b01d7
Replace list comprehension
2015-08-13 17:00:17 +01:00
5db5677969
Add metrics to the receipts cache
2015-08-13 16:58:23 +01:00
0fbed2a8fa
Comment
2015-08-12 17:22:54 +01:00
c10ac7806e
Explain why we're prefilling dict with Nones
2015-08-12 17:16:30 +01:00
101ee3fd00
Better variable name
2015-08-12 17:08:05 +01:00
df361d08f7
Split _get_state_for_group_from_cache into two
2015-08-12 17:06:21 +01:00
415c2f0549
Simplify LoginHander and AuthHandler
...
* Merge LoginHandler -> AuthHandler
* Add a bunch of documentation
* Improve some naming
* Remove unused branches
I will start merging the actual logic of the two handlers shortly
2015-08-12 15:49:37 +01:00
2df8dd9b37
Move all the caches into their own package, synapse.util.caches
2015-08-11 18:00:59 +01:00
53a817518b
Comments
2015-08-11 11:40:40 +01:00
6eaa116867
Comment
2015-08-11 11:35:24 +01:00
4762c276cb
Docs
2015-08-11 11:33:41 +01:00
1b994a97dd
Fix application of ACLs
2015-08-11 10:41:40 +01:00
10b874067b
Fix state cache
2015-08-11 09:12:41 +01:00
017b798e4f
Clean up StateStore
2015-08-10 15:01:06 +01:00
2c019eea11
Remove unused function
2015-08-10 14:44:41 +01:00
bb0a475c30
Comments
2015-08-10 14:27:38 +01:00
559c51debc
Use TypeError instead of ValueError and give a nicer error mesasge
...
when someone calls Cache.invalidate with the wrong type.
2015-08-10 14:07:17 +01:00
7ce71f2ffc
Merge branch 'erikj/cache_varargs_interface' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-10 13:47:51 +01:00
8c3a62b5c7
Merge pull request #215 from matrix-org/erikj/cache_varargs_interface
...
Change Cache to not use *args in its interface
2015-08-10 13:47:45 +01:00
86eaaa885b
Rename keyargs to args in CacheDescriptor
2015-08-10 13:44:44 +01:00
e0b6e49466
Merge branch 'erikj/cache_varargs_interface' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-10 10:39:22 +01:00
2cd6cb9f65
Rename keyargs to args in Cache
2015-08-10 10:38:47 +01:00
aa88582e00
Do bounds check
2015-08-10 10:08:15 +01:00
5119e416e8
Line length
2015-08-10 10:05:30 +01:00
8f04b6fa7a
Merge branch 'erikj/cache_varargs_interface' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-07 19:30:25 +01:00
7dec0b2bee
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-07 19:28:39 +01:00
06218ab125
Merge pull request #212 from matrix-org/erikj/cache_deferreds
...
Make CacheDescriptor cache deferreds rather than the deferreds' values
2015-08-07 19:28:05 +01:00
2352974aab
Merge branch 'erikj/cache_deferreds' of github.com:matrix-org/synapse into erikj/cache_varargs_interface
2015-08-07 19:26:54 +01:00
9c5385b53a
s/observed/observer/
2015-08-07 19:26:38 +01:00
ffab798a38
Merge branch 'erikj/cache_deferreds' of github.com:matrix-org/synapse into erikj/cache_varargs_interface
2015-08-07 19:18:47 +01:00
62126c996c
Propogate stale cache errors to calling functions
2015-08-07 19:17:58 +01:00
3213ff630c
Remove unnecessary cache
2015-08-07 19:14:05 +01:00
20addfa358
Change Cache to not use *args in its interface
2015-08-07 18:32:47 +01:00
9eb5b23d3a
Batch up various DB requests for event -> state
2015-08-07 18:16:02 +01:00
0211890134
Implement a CacheListDescriptor
2015-08-07 18:14:49 +01:00
0db40d3e93
Don't complain about extra .pyc files we find while hunting for database schemas
2015-08-07 17:22:11 +01:00
b2c7bd4b09
Cache get_recent_events_for_room
2015-08-07 14:42:34 +01:00
b3768ec10a
Remove unncessary cache
2015-08-07 13:41:05 +01:00
b8e386db59
Change Cache to not use *args in its interface
2015-08-07 11:52:21 +01:00
fe994e728f
Store absence of state in cache
2015-08-07 10:17:38 +01:00
1d08bf7c17
Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache
2015-08-06 14:03:15 +01:00
63b1eaf32c
Docs
2015-08-06 14:02:50 +01:00
b811c98574
Remove failed deferreds from cache
2015-08-06 14:01:27 +01:00
433314cc34
Re-implement DEBUG_CACHES flag
2015-08-06 14:01:05 +01:00
f596ff402e
Merge branch 'erikj/cache_deferreds' into erikj/dictionary_cache
2015-08-06 13:37:56 +01:00
2efb93af52
Merge branch 'erikj/cached_keyword_args' into erikj/cache_deferreds
2015-08-06 13:35:28 +01:00
953dbd28a7
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/cached_keyword_args
2015-08-06 13:35:03 +01:00
7eea3e356f
Make @cached cache deferreds rather than the deferreds' values
2015-08-06 13:33:34 +01:00
3e1b77efc2
Merge branch 'erikj/cached_keyword_args' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-05 16:45:56 +01:00
b52b4a84ec
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dictionary_cache
2015-08-05 15:41:20 +01:00
1e62a3d3a9
Up the cache size for 'get_joined_hosts_for_room' and 'get_users_in_room'
2015-08-05 15:40:40 +01:00
a89559d797
Use LRU cache by default
2015-08-05 15:39:47 +01:00
07507643cb
Use dictionary cache to do group -> state fetching
2015-08-05 15:11:42 +01:00
c67ba143fa
Move DictionaryCache
2015-08-04 15:58:28 +01:00
e7768e77f5
Add basic dictionary cache
2015-08-04 15:56:56 +01:00
e124128542
Bump schema version
2015-08-04 14:50:31 +01:00
c77048e12f
Add endpoint that proxies ID server request token and errors if the given email is in use on this Home Server.
2015-08-04 14:37:09 +01:00
413a4c289b
Add comment
2015-08-04 11:08:07 +01:00
4d6cb8814e
Speed up event filtering (for ACL) logic
2015-08-04 09:32:23 +01:00
39e21ea51c
Add support for using keyword arguments with cached functions
2015-07-27 13:57:29 +01:00
Mark Haines