1619802228
Various clean-ups to the logging context code ( #8935 )
2020-12-14 14:19:47 -05:00
8b42a4eefd
Gracefully handle a pending logging connection during shutdown. ( #8685 )
2020-10-29 12:53:57 -04:00
00b24aa545
Support generating structured logs in addition to standard logs. ( #8607 )
...
This modifies the configuration of structured logging to be usable from
the standard Python logging configuration.
This also separates the formatting of logs from the transport allowing
JSON logs to files or standard logs to sockets.
2020-10-29 07:27:37 -04:00
2b7c180879
Start fewer opentracing spans ( #8640 )
...
#8567 started a span for every background process. This is good as it means all Synapse code that gets run should be in a span (unless in the sentinel logging context), but it means we generate about 15x the number of spans as we did previously.
This PR attempts to reduce that number by a) not starting one for send commands to Redis, and b) deferring starting background processes until after we're sure they're necessary.
I don't really know how much this will help.
2020-10-26 09:30:19 +00:00
20a67aa70d
Separate the TCP and terse JSON formatting code. ( #8587 )
...
This should (theoretically) allow for using the TCP code with a different output type
and make it easier to use the JSON code with files / console.
2020-10-21 06:59:54 -04:00
4ff0201e62
Enable mypy checking for unreachable code and fix instances. ( #8432 )
2020-10-01 08:09:18 -04:00
5e3ca12b15
Create a mechanism for marking tests "logcontext clean" ( #8399 )
2020-09-28 17:58:33 +01:00
8a4a4186de
Simplify super() calls to Python 3 syntax. ( #8344 )
...
This converts calls like super(Foo, self) -> super().
Generated with:
sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py
2020-09-18 09:56:44 -04:00
aec294ee0d
Use slots in attrs classes where possible ( #8296 )
...
slots use less memory (and attribute access is faster) while slightly
limiting the flexibility of the class attributes. This focuses on objects
which are instantiated "often" and for short periods of time.
2020-09-14 12:50:06 -04:00
d2a3eb04a4
Fix typos in comments.
2020-09-14 11:46:58 -04:00
703e2b8a96
Use the right constructor for log records ( #8278 )
...
Update `log_function` to use the right factory to create log records, to make
sure that they have `request` attributes.
Fixes : #8267 .
2020-09-08 14:52:51 +01:00
c619253db8
Stop sub-classing object ( #8249 )
2020-09-04 06:54:56 -04:00
dbc630a628
Use the JSON encoder without whitespace in more places. ( #8124 )
2020-08-20 10:32:33 -04:00
eebf52be06
Be stricter about JSON that is accepted by Synapse ( #8106 )
2020-08-19 07:26:03 -04:00
1ef9efc1e0
Fix error reporting when using `opentracing.trace` ( #7961 )
2020-07-27 16:20:24 +01:00
15997618e2
Clean up PreserveLoggingContext ( #7877 )
...
This had some dead code and some just plain wrong docstrings.
2020-07-22 00:40:27 +01:00
5662e2b0f3
Remove unused code from synapse.logging.utils. ( #7897 )
2020-07-20 15:20:53 -04:00
d1d5fa66e4
Fix the trace function for async functions. ( #7872 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-07-17 13:32:01 -04:00
62b1ce8539
isort 5 compatibility ( #7786 )
...
The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.
2020-07-05 16:32:02 +01:00
5cdca53aa0
Merge different Resource implementation classes ( #7732 )
2020-07-03 19:02:19 +01:00
e07a8caf58
Add support for using rust-python-jaeger-reporter ( #7697 )
2020-06-17 14:13:41 +01:00
a3f11567d9
Replace all remaining six usage with native Python 3 equivalents ( #7704 )
2020-06-16 08:51:47 -04:00
d4676910c9
remove miscellaneous PY2 code
2020-05-15 19:37:41 +01:00
d7c2df2fa3
Improve per-block CPU and DB usage metrics ( #7426 )
2020-05-06 16:43:39 -04:00
fe69fb6263
Add backwards compatibility codepath to LoggingContext. ( #7408 )
2020-05-05 09:21:34 -04:00
37f6823f5b
Add instance name to RDATA/POSITION commands ( #7364 )
...
This is primarily for allowing us to send those commands from workers, but for now simply allows us to ignore echoed RDATA/POSITION commands that we sent (we get echoes of sent commands when using redis). Currently we log a WARNING on the master process every time we receive an echoed RDATA.
2020-04-29 16:23:08 +01:00
0f05fd1530
Reduce the number of calls to `resource.getrusage` ( #7183 )
...
Let's just call `getrusage` once on each logcontext change, rather than twice.
2020-04-03 13:21:30 +01:00
60adcbed91
Fix "'NoneType' has no attribute start|stop" logcontext errors ( #7181 )
...
Fixes #7179 .
2020-03-31 15:18:41 +01:00
39230d2171
Clean up some LoggingContext stuff ( #7120 )
...
* Pull Sentinel out of LoggingContext
... and drop a few unnecessary references to it
* Factor out LoggingContext.current_context
move `current_context` and `set_context` out to top-level functions.
Mostly this means that I can more easily trace what's actually referring to
LoggingContext, but I think it's generally neater.
* move copy-to-parent into `stop`
this really just makes `start` and `stop` more symetric. It also means that it
behaves correctly if you manually `set_log_context` rather than using the
context manager.
* Replace `LoggingContext.alive` with `finished`
Turn `alive` into `finished` and make it a bit better defined.
2020-03-24 14:45:33 +00:00
dc6fb56c5f
Hopefully mypy is happy now
2020-03-10 14:40:28 +00:00
fe593ef990
Attempt at appeasing the gods of mypy
2020-03-10 14:19:06 +00:00
2bff4457d9
Add type hints to logging/context.py ( #6309 )
...
* Add type hints to logging/context.py
Signed-off-by: neiljp (Neil Pilgrim) <github@kepier.clara.net>
2020-03-07 17:57:26 +00:00
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 ( #6957 )
...
Ensure good comprehension hygiene using flake8-comprehensions.
2020-02-21 07:15:07 -05:00
da4e52544e
comment for run_in_background
2020-01-12 21:53:47 +00:00
495005360c
Bump version of mypy
2019-12-12 15:21:12 +00:00
ffeafade48
Update comment
2019-12-10 13:17:39 +00:00
9a2223d4c8
Fix make_deferred_yieldable to work with coroutines
2019-12-10 11:22:12 +00:00
fdec84aa42
Add benchmarks for structured logging performance ( #6266 )
2019-12-03 20:21:25 +11:00
9eebd46048
Improve the performance of structured logging ( #6322 )
2019-11-26 03:45:50 +11:00
73d091be48
A couple more instances
2019-11-12 13:12:25 +00:00
020add5099
Update black to 19.10b0 ( #6304 )
...
* update version of black and also fix the mypy config being overridden
2019-11-01 02:43:24 +11:00
54fef094b3
Remove usage of deprecated logger.warn method from codebase ( #6271 )
...
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
f3ceaf4323
Trace non-JSON APIs, /media, /key etc
2019-10-11 11:58:52 +01:00
864f144543
Fix up some typechecking ( #6150 )
...
* type checking fixes
* changelog
2019-10-02 05:29:01 -07:00
a86a290850
Fix logcontext spam on non-Linux platforms ( #6059 )
...
This checks whether the current platform supports thread resource usage tracking
before logging a warning to avoid log spam.
Fixes https://github.com/matrix-org/synapse/issues/6055
2019-09-18 21:55:37 +01:00
b617864cd9
Fix for structured logging tests stomping on logs ( #6023 )
2019-09-13 02:29:55 +10:00
d8517da85b
Don't assume there is a 'self' arg in @trace decorator
2019-09-06 10:07:12 +01:00
bc604e7f94
Gracefully handle log context slips and missing opentracing import errors. ( #5988 )
2019-09-05 17:33:29 +01:00
909827b422
Add opentracing to all client servlets ( #5983 )
2019-09-05 14:46:04 +01:00
b9cfd3c375
Fix opentracing contexts missing from outbound replication requests ( #5982 )
2019-09-05 14:22:15 +01:00
Patrick Cloke