Commit Graph

78 Commits (69b74d9330e42fc91a9c7423d00a06cd6d3732bf)

Author SHA1 Message Date
Eric Eastwood 8ddb2de553
Document `looping_call()` functionality that will wait for the given function to finish before scheduling another (#15772)
Thanks to @erikjohnston for clarifying, https://github.com/matrix-org/synapse/pull/15743#discussion_r1226544457

We don't have to worry about calls stacking up if the given function takes longer than the scheduled time.
2023-06-13 16:34:54 -05:00
Patrick Cloke c01343de43
Add stricter mypy options (#15694)
Enable warn_unused_configs, strict_concatenate, disallow_subclassing_any,
and disallow_incomplete_defs.
2023-05-31 07:18:29 -04:00
David Robertson 3b0083c92a
Use immutabledict instead of frozendict (#15113)
Additionally:

* Consistently use `freeze()` in test

---------

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
2023-03-22 17:15:34 +00:00
David Robertson cb20b885cb
Always close _all_ `ijson` coroutines, even if doing so raises Exceptions (#14065) 2022-10-06 18:17:50 +00:00
David Robertson 6ba732fefe
Type `tests.utils` (#13028)
* Cast to postgres types when handling postgres db

* Remove unused method

* Easy annotations

* Annotate create_room

* Use `ParamSpec` to annotate looping_call

* Annotate `default_config`

* Track `now` as a float

`time_ms` returns an int like the proper Synapse `Clock`

* Introduce a `Timer` dataclass

* Introduce a Looper type

* Suppress checking of a mock

* tests.utils is typed

* Changelog

* Whoops, import ParamSpec from typing_extensions

* ditch the psycopg2 casts
2022-07-05 15:13:47 +01:00
David Robertson f30bcbd84a
Fix Synapse git info missing in version strings (#12973) 2022-06-07 15:24:11 +01:00
reivilibre 72e7f1c420
Remove workaround introduced in Synapse v1.50.0rc1 for Mjolnir compatibility. Breaks compatibility with Mjolnir v1.3.1 and earlier. (#11700) 2022-03-10 15:53:23 +00:00
Richard van der Hoff 9d11fee8f2
Improve exception handling for concurrent execution (#12109)
* fix incorrect unwrapFirstError import

this was being imported from the wrong place

* Refactor `concurrently_execute` to use `yieldable_gather_results`

* Improve exception handling in `yieldable_gather_results`

Try to avoid swallowing so many stack traces.

* mark unwrapFirstError deprecated

* changelog
2022-03-01 09:34:30 +00:00
reivilibre eec34b1f2a
Work around Mjolnir compatibility issue by adding an import for `glob_to_regex` in `synapse.util`, where it moved from. (#11696) 2022-01-06 16:36:26 +00:00
reivilibre 84bfe47b01
Re-apply: Move glob_to_regex and re_word_boundary to matrix-python-common #11505 (#11687)
Co-authored-by: Sean Quah <seanq@element.io>
2022-01-05 11:41:49 +00:00
Sean Quah 088d748f2c
Revert "Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common` (#11505) (#11527)
This reverts commit a77c369897.
2021-12-07 13:51:11 +00:00
Sean Quah a77c369897
Move `glob_to_regex` and `re_word_boundary` to `matrix-python-common` (#11505) 2021-12-06 11:36:08 +00:00
Patrick Cloke b64b6d12d4
Add more type hints to synapse.util. (#11321) 2021-11-12 13:43:06 +00:00
David Robertson 797ee7812d
Relax `ignore-missing-imports` for modules that have stubs now and update mypy (#11006)
Updating mypy past version 0.9 means that third-party stubs are no-longer distributed with typeshed. See http://mypy-lang.blogspot.com/2021/06/mypy-0900-released.html for details.
We therefore pull in stub packages in setup.py

Additionally, some modules that we were previously ignoring import failures for now have stubs. So let's use them.

The rest of this change consists of fixups to make the newer mypy + stubs pass CI.

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2021-10-08 14:49:41 +01:00
Hillery Shay 0f007fe009
Update utility code to handle C implementations of frozendict (#10902)
* update _handle_frozendict to work with c implementations of frozen dict

* add changelog

* add clarifying comment to _handle_frozendict
2021-09-28 09:13:23 -07:00
reivilibre 524b8ead77
Add types to synapse.util. (#10601) 2021-09-10 17:03:18 +01:00
Richard van der Hoff 03318a766c
Merge pull request from GHSA-x345-32rc-8h85
* tests for push rule pattern matching

* tests for acl pattern matching

* factor out common `re.escape`

* Factor out common re.compile

* Factor out common anchoring code

* add word_boundary support to `glob_to_regex`

* Use `glob_to_regex` in push rule evaluator

NB that this drops support for character classes. I don't think anyone ever
used them.

* Improve efficiency of globs with multiple wildcards

The idea here is that we compress multiple `*` globs into a single `.*`. We
also need to consider `?`, since `*?*` is as hard to implement efficiently as
`**`.

* add assertion on regex pattern

* Fix mypy

* Simplify glob_to_regex

* Inline the glob_to_regex helper function

Signed-off-by: Dan Callahan <danc@element.io>

* Moar comments

Signed-off-by: Dan Callahan <danc@element.io>

Co-authored-by: Dan Callahan <danc@element.io>
2021-05-11 11:47:23 +02:00
Jonathan de Jong 4b965c862d
Remove redundant "coding: utf-8" lines (#9786)
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Richard van der Hoff b6ca69e4f1 Remove frozendict_json_encoder and support frozendicts everywhere
Not being able to serialise `frozendicts` is fragile, and it's annoying to have
to think about which serialiser you want. There's no real downside to
supporting frozendicts, so let's just have one json encoder.
2020-10-28 15:56:57 +00:00
Patrick Cloke aec294ee0d
Use slots in attrs classes where possible (#8296)
slots use less memory (and attribute access is faster) while slightly
limiting the flexibility of the class attributes. This focuses on objects
which are instantiated "often" and for short periods of time.
2020-09-14 12:50:06 -04:00
Patrick Cloke a9dbe98ef9 Synapse 1.20.0rc3 (2020-09-11)
==============================
 
 Bugfixes
 --------
 
 - Fix a bug introduced in v1.20.0rc1 where the wrong exception was raised when invalid JSON data is encountered. ([\#8291](https://github.com/matrix-org/synapse/issues/8291))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl9bbA8ACgkQM/xY9qcR
 MEjjJg//ZGcvPLr8y3F5JXptcjWA3AFja3DgztBA3uNFoDrwVhR9m3+F4rL/Fbat
 gnaccWsKIXOjZw5WOThQEUIfYMKmHEsaDl7xtlUPC4ylmSV+31ypu1KiRSxND2kT
 TmOupvqw+4/E4RajpYX6WT3e2oQUpIcBAKsUyZL3ZXECR/yNYrA6w2w7i4wrv3cU
 QDzBFrCcr5aJql7VUc88BlaZUG6xom2/kkPtjmO6imPnPGBHLN22uPU7zQ4n7Vgi
 Nrg45v5WHVCx57WoXqAEap1zdKgoRCna1x0NkqYh0OAXq1l6aVwlf/Pdynt91f2x
 yyaMYjMskjyatHlzONMV4kz0w03dUrGJXiAx2ldEDd32SKBUxLJ/CYtNEdoXZ4Mm
 o0OJfDbaSJwlqK7FhdZJE3kCSkUvlmVasDErXfQRDVHy5sx9Ufr4P0kkaZyXDys1
 UAWFcCJwy9dgFR2679PoXk1s/gHf10wuk6FOs8ESbcJJOvCljxWrOIqKNzSUL3HQ
 Hj5V1zHYXB6HtgiI7tbKxSn/d2uajHI9b8LVOkyV9RKsb1DGGRgNQQ5Kz4Zud7dC
 vuQ2jVw7JIo41W3QIXvcL1KdQeje2nwYwuwNREZaEDZJsGLjxaJ6dNyBVgkmYYFP
 gxhplSOSsFjQRGEsBrc2utWxDnLVGPHEAt7iHVSqjwVUyARvxjo=
 =e2T1
 -----END PGP SIGNATURE-----

Merge tag 'v1.20.0rc3' into develop

Synapse 1.20.0rc3 (2020-09-11)
==============================

Bugfixes
--------

- Fix a bug introduced in v1.20.0rc1 where the wrong exception was raised when invalid JSON data is encountered. ([\#8291](https://github.com/matrix-org/synapse/issues/8291))
2020-09-11 08:30:36 -04:00
Patrick Cloke b86764662b
Fix the exception that is raised when invalid JSON is encountered. (#8291) 2020-09-10 14:55:25 -04:00
Patrick Cloke 72bec36d50
Directly import json from the standard library. (#8259)
By importing from canonicaljson the simplejson module was still being used
in some situations. After this change the std lib json is consistenty used
throughout Synapse.
2020-09-08 07:33:48 -04:00
Patrick Cloke c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Patrick Cloke eebf52be06
Be stricter about JSON that is accepted by Synapse (#8106) 2020-08-19 07:26:03 -04:00
David Vo 4dd27e6d11
Reduce unnecessary whitespace in JSON. (#7372) 2020-08-07 08:02:55 -04:00
Patrick Cloke 38e1fac886
Fix some spelling mistakes / typos. (#7811) 2020-07-09 09:52:58 -04:00
Richard van der Hoff 14d8f342d5 move batch_iter to a separate module 2020-01-16 22:25:32 +00:00
Brendan Abolivier 244953be3f
Add kwargs and doc 2019-07-29 10:03:14 +02:00
Brendan Abolivier 08352d44f8
Add ability to pass arguments to looping calls 2019-07-29 09:54:37 +02:00
Amber Brown 4806651744
Replace returnValue with return (#5736) 2019-07-23 23:00:55 +10:00
Amber Brown 1ee268d33d
Improve the backwards compatibility re-exports of synapse.logging.context (#5617)
* Improve the backwards compatibility re-exports of synapse.logging.context.

* reexport logformatter too
2019-07-05 02:32:02 +10:00
Amber Brown 463b072b12
Move logging utilities out of the side drawer of util/ and into logging/ (#5606) 2019-07-04 00:07:04 +10:00
Richard van der Hoff 91753cae59
Fix a number of "Starting txn from sentinel context" warnings (#5605)
Fixes #5602, #5603
2019-07-03 09:31:27 +01:00
Amber Brown 32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Erik Johnston 9fafdfa97d Anchor returned regex to start and end of string 2018-10-19 10:22:45 +01:00
Erik Johnston 084046456e Add config option to control alias creation 2018-10-19 10:22:45 +01:00
Erik Johnston 6982320572 Remove unnecessary extra function call layer 2018-10-08 14:06:19 +01:00
Erik Johnston 8a1817f0d2 Use errback pattern and catch async failures 2018-10-08 13:29:47 +01:00
Erik Johnston f7199e8734 Log looping call exceptions
If a looping call function errors, then it kills the loop entirely.
Currently it throws away the exception logs, so we should make it
actually log them.

Fixes #3929
2018-10-05 11:24:12 +01:00
Amber Brown 49af402019 run isort 2018-07-09 16:09:20 +10:00
Amber Brown 07cad26d65
Remove all global reactor imports & pass it around explicitly (#3424) 2018-06-25 14:08:28 +01:00
Amber Brown 77ac14b960
Pass around the reactor explicitly (#3385) 2018-06-22 09:37:10 +01:00
Adrian Tschira 45b55e23d3 Add batch_iter to utils
There's a frequent idiom I noticed where an iterable is split up into a
number of chunks/batches. Unfortunately that method does not work with
iterators like dict.keys() in python3. This implementation works with
iterators.

Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:48:30 +02:00
Richard van der Hoff 1ea904b9f0 Use deferred.addTimeout instead of time_bound_deferred
This doesn't feel like a wheel we need to reinvent.
2018-04-23 00:53:18 +01:00
Richard van der Hoff 0be99858f3 fix vars named `l`
E741 says "do not use variables named ‘l’, ‘O’, or ‘I’".
2017-10-23 15:56:38 +01:00
Richard van der Hoff eaaabc6c4f replace 'except:' with 'except Exception:'
what could possibly go wrong
2017-10-23 15:52:32 +01:00
Richard van der Hoff 2e996271fe Instantiate DeferredTimedOutError correctly
Call `super` correctly, so that we correctly initialise the `errcode` field.

Fixes https://github.com/matrix-org/synapse/issues/2179.
2017-05-02 13:26:17 +01:00
Richard van der Hoff b88a323ffb Fix time_bound_deferred to throw the right exception
Due to a failure to instantiate DeferredTimedOutError, time_bound_deferred
would throw a CancelledError when the deferred timed out, which was rather
confusing.
2017-03-23 12:07:11 +00:00
Erik Johnston fbaf868f62 Correctly handle timeout errors 2016-12-09 16:30:29 +00:00