Richard van der Hoff
47815edcfa
ConsentResource to gather policy consent from users
...
Hopefully there are enough comments and docs in this that it makes sense on its
own.
2018-05-15 15:11:59 +01:00
Richard van der Hoff
318711e139
Set Server header in SynapseRequest
...
(instead of everywhere that writes a response. Or rather, the subset of places
which write responses where we haven't forgotten it).
This also means that we don't have to have the mysterious version_string
attribute in anything with a request handler.
Unfortunately it does mean that we have to pass the version string wherever we
instantiate a SynapseSite, which has been c&ped 150 times, but that is code
that ought to be cleaned up anyway really.
2018-05-10 18:50:27 +01:00
Richard van der Hoff
645cb4bf06
Remove redundant request_handler decorator
...
This is needless complexity; we might as well use the wrapper directly.
Also rename wrap_request_handler->wrap_json_request_handler.
2018-05-10 12:19:53 +01:00
Richard van der Hoff
09f570b935
Factor wrap_request_handler_with_logging out of wrap_request_handler
...
... so that it can be used on non-JSON endpoints
2018-05-10 12:19:52 +01:00
Richard van der Hoff
9589a1925e
Remove include_metrics param
...
The metrics are now available via the request, so this is redundant and can go
away at last.
2018-05-10 12:19:52 +01:00
Richard van der Hoff
49e5a613f1
Move outgoing_responses_counter handling to RequestMetrics
...
it's much neater there.
2018-05-10 12:19:52 +01:00
Richard van der Hoff
b8700dd7d0
Bump requests_counter in wrapped_request_handler
...
less magic
2018-05-10 12:19:52 +01:00
Richard van der Hoff
c6f730282c
Move RequestMetrics handling into SynapseRequest.processing()
...
It fits quite nicely here, and opens the path to getting rid of the
"include_metrics" mess.
2018-05-10 12:19:51 +01:00
Richard van der Hoff
09b29f9c4a
Make RequestMetrics take a raw time rather than a clock
...
... which is going to make it easier to move around.
2018-05-10 12:18:52 +01:00
Richard van der Hoff
8460e48d06
Move request_id management into SynapseRequest
2018-05-10 11:48:17 +01:00
Richard van der Hoff
18e144fe08
Move RequestsMetrics to its own file
...
This is useful in its own right, because server.py is full of stuff; but more
importantly, I want to do some refactoring that will cause a circular reference
as it is.
2018-05-09 19:55:03 +01:00
Richard van der Hoff
2117f84323
Merge pull request #3182 from Half-Shot/hs/fix-twisted-shutdown
...
Fix 'Unhandled Error' logs with Twisted 18.4
2018-05-03 12:40:11 +01:00
Will Hunt
2e7a94c36b
Don't abortConnection() if the transport connection has already closed.
2018-05-03 12:31:47 +01:00
Richard van der Hoff
1dfd650348
add missing param to cancelled_to_request_timed_out_error
...
This gets two arguments, not one.
2018-05-02 22:42:36 +01:00
Richard van der Hoff
049b0b5af2
Merge pull request #3154 from NotAFile/py3-stringio
...
Replace stringIO imports with six
2018-04-30 00:59:04 +01:00
Richard van der Hoff
dbf6f28d64
Merge pull request #3155 from NotAFile/py3-bytes-1
...
more bytes strings
2018-04-30 00:38:21 +01:00
Richard van der Hoff
9e2601f830
Merge pull request #3108 from NotAFile/py3-six-urlparse
...
Use six.moves.urlparse
2018-04-30 00:33:05 +01:00
Adrian Tschira
e9143b6593
more bytes strings
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-29 00:13:57 +02:00
Adrian Tschira
4f2f5171b7
replace stringIO imports
2018-04-28 13:46:23 +02:00
Richard van der Hoff
9d2c1b8429
Backport deferred.addTimeout
...
Twisted 16.0 doesn't have addTimeout, so let's backport it.
2018-04-27 12:52:30 +01:00
Richard van der Hoff
1ea904b9f0
Use deferred.addTimeout instead of time_bound_deferred
...
This doesn't feel like a wheel we need to reinvent.
2018-04-23 00:53:18 +01:00
Adrian Tschira
2a3c33ff03
Use six.moves.urlparse
...
The imports were shuffled around a bunch in py3
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-15 21:22:43 +02:00
Richard van der Hoff
87478c5a60
Merge pull request #3061 from NotAFile/add-some-byte-strings
...
Add b prefixes to some strings that are bytes in py3
2018-04-09 23:54:05 +01:00
Richard van der Hoff
664adb4236
Merge pull request #3016 from silkeh/improve-service-lookups
...
Improve handling of SRV records for federation connections
2018-04-09 23:40:06 +01:00
Richard van der Hoff
13decdbf96
Revert "Merge pull request #3066 from matrix-org/rav/remove_redundant_metrics"
...
We aren't ready to release this yet, so I'm reverting it for now.
This reverts commit d1679a4ed7
, reversing
changes made to e089100c62
.
2018-04-09 12:59:12 +01:00
Erik Johnston
7b824f1475
Add response size metrics
2018-04-06 13:20:11 +01:00
Krombel
1d71f484d4
use PUT instead of POST for federating groups/m.join_policy
2018-04-06 12:54:09 +02:00
Richard van der Hoff
518f6de088
Remove redundant metrics which were deprecated in 0.27.0.
2018-04-04 19:46:28 +01:00
Adrian Tschira
6168351877
Add b prefixes to some strings that are bytes in py3
...
This has no effect on python2
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-04 13:48:51 +02:00
Silke
72251d1b97
Remove address resolution of hosts in SRV records
...
Signed-off-by: Silke Hofstra <silke@slxh.eu>
2018-04-04 12:26:50 +02:00
Erik Johnston
1c41b05c8c
Add Cache-Control headers to all JSON APIs
...
It is especially important that sync requests don't get cached, as if a
sync returns the same token given then the client will call sync with
the same parameters again. If the previous response was cached it will
get reused, resulting in the client tight looping making the same
request and never making any progress.
In general, clients will expect to get up to date data when requesting
APIs, and so its safer to do a blanket no cache policy than only
whitelisting APIs that we know will break things if they get cached.
2018-03-21 17:46:26 +00:00
Erik Johnston
fa72803490
Merge branch 'master' of github.com:matrix-org/synapse into develop
2018-03-19 11:41:01 +00:00
Erik Johnston
a8ce159be4
Replace some ujson with simplejson to make it work
2018-03-16 00:27:09 +00:00
Erik Johnston
926ba76e23
Replace ujson with simplejson
2018-03-15 23:43:31 +00:00
Richard van der Hoff
58dd148c4f
Add some docstrings to help figure this out
2018-03-09 18:05:41 +00:00
Richard van der Hoff
88541f9009
Add a metric which increments when a request is received
...
It's useful to know when there are peaks in incoming requests - which isn't
quite the same as there being peaks in outgoing responses, due to the time
taken to handle requests.
2018-03-09 16:30:26 +00:00
Richard van der Hoff
dbe80a286b
refactor JsonResource
...
rephrase the OPTIONS and unrecognised request handling so that they look
similar to the common flow.
2018-03-09 16:22:16 +00:00
Richard van der Hoff
10b34dbb9a
Merge pull request #2858 from matrix-org/rav/purge_updates
...
delete_local_events for purge_room_history
2018-02-09 14:11:00 +00:00
Richard van der Hoff
74fcbf741b
delete_local_events for purge_history
...
Add a flag which makes the purger delete local events
2018-02-09 13:07:41 +00:00
Richard van der Hoff
78e7e05188
Merge pull request #2838 from matrix-org/rav/fix_logging_on_dns_fail
...
Remove spurious log argument
2018-01-31 22:18:46 +01:00
Richard van der Hoff
421d68ca8c
Merge pull request #2817 from matrix-org/rav/http_conn_pool
...
Use a connection pool for the SimpleHttpClient
2018-01-31 22:14:22 +01:00
Richard van der Hoff
af19f5e9aa
Remove spurious log argument
...
... which would cause scary-looking and unhelpful errors in the log on dns fail
2018-01-30 17:52:03 +00:00
Richard van der Hoff
02ba118f81
Increase http conn pool size
2018-01-29 14:30:15 +00:00
Matthew Hodgson
ab9f844aaf
Add federation_domain_whitelist option ( #2820 )
...
Add federation_domain_whitelist
gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
2018-01-22 19:11:18 +01:00
Richard van der Hoff
2c8526cac7
Use a connection pool for the SimpleHttpClient
...
In particular I hope this will help the pusher, which makes many requests to
sygnal, and is currently negotiating SSL for each one.
2018-01-20 00:55:44 +00:00
Richard van der Hoff
e8f7541d3f
Merge remote-tracking branch 'origin/develop' into rav/track_db_scheduling
2018-01-17 14:01:57 +00:00
Richard van der Hoff
fb6563b4be
Merge pull request #2793 from matrix-org/rav/db_txn_time_in_millis
...
Track db txn time in millisecs
2018-01-17 13:52:42 +00:00
Richard van der Hoff
936482d507
Fix 'NoneType' object has no attribute 'writeHeaders'
...
Avoid throwing a (harmless) exception when we try to write an error response to
an http request where the client has disconnected.
This comes up as a CRITICAL error in the logs which tends to mislead people
into thinking there's an actual problem
2018-01-16 17:58:16 +00:00
Richard van der Hoff
3d12d97415
Track DB scheduling delay per-request
...
For each request, track the amount of time spent waiting for a db
connection. This entails adding it to the LoggingContext and we may as well add
metrics for it while we are passing.
2018-01-16 17:23:32 +00:00
Richard van der Hoff
6324b65f08
Track db txn time in millisecs
...
... to reduce the amount of floating-point foo we do.
2018-01-16 15:53:18 +00:00
Richard van der Hoff
39f4e29d01
Reorganise request and block metrics
...
In order to circumvent the number of duplicate foo:count metrics increasing
without bounds, it's time for a rearrangement.
The following are all deprecated, and replaced with synapse_util_metrics_block_count:
synapse_util_metrics_block_timer:count
synapse_util_metrics_block_ru_utime:count
synapse_util_metrics_block_ru_stime:count
synapse_util_metrics_block_db_txn_count:count
synapse_util_metrics_block_db_txn_duration:count
The following are all deprecated, and replaced with synapse_http_server_response_count:
synapse_http_server_requests
synapse_http_server_response_time:count
synapse_http_server_response_ru_utime:count
synapse_http_server_response_ru_stime:count
synapse_http_server_response_db_txn_count:count
synapse_http_server_response_db_txn_duration:count
The following are renamed (the old metrics are kept for now, but deprecated):
synapse_util_metrics_block_timer:total ->
synapse_util_metrics_block_time_seconds
synapse_util_metrics_block_ru_utime:total ->
synapse_util_metrics_block_ru_utime_seconds
synapse_util_metrics_block_ru_stime:total ->
synapse_util_metrics_block_ru_stime_seconds
synapse_util_metrics_block_db_txn_count:total ->
synapse_util_metrics_block_db_txn_count
synapse_util_metrics_block_db_txn_duration:total ->
synapse_util_metrics_block_db_txn_duration_seconds
synapse_http_server_response_time:total ->
synapse_http_server_response_time_seconds
synapse_http_server_response_ru_utime:total ->
synapse_http_server_response_ru_utime_seconds
synapse_http_server_response_ru_stime:total ->
synapse_http_server_response_ru_stime_seconds
synapse_http_server_response_db_txn_count:total ->
synapse_http_server_response_db_txn_count
synapse_http_server_response_db_txn_duration:total
synapse_http_server_response_db_txn_duration_seconds
2018-01-15 17:09:44 +00:00
Richard van der Hoff
d90e8ea444
Update http request metrics before calling servlet
...
Make sure that we set the servlet name in the metrics object *before* calling
the servlet, in case the servlet throws an exception.
2018-01-09 18:27:35 +00:00
Richard van der Hoff
df6c72ede3
Merge pull request #2711 from matrix-org/rav/fix_dns_errhandler
...
Fix error handling on dns lookup
2017-11-27 12:19:18 +00:00
Richard van der Hoff
6be01f599b
Improve tracebacks on exceptions
...
Use failure.Failure to recover our failure, which will give us a useful
stacktrace, unlike the rethrown exception.
2017-11-27 12:05:58 +00:00
Richard van der Hoff
8b38096a89
Fix error handling on dns lookup
...
pass the right arguments to the errback handler
Fixes "TypeError('eb() takes exactly 2 arguments (1 given)',)"
2017-11-24 16:47:48 +00:00
Richard van der Hoff
f90649eb2b
Fix 500 on invalid utf-8 in request
...
If somebody sends us a request where the the body is invalid utf-8, we should
return a 400 rather than a 500. (json.loads throws a UnicodeError in this
situation)
We might as well catch all Exceptions here: it seems very unlikely that we
would get a request that *isn't caused by invalid json.
2017-11-10 09:15:39 +00:00
Richard van der Hoff
fcdfc911ee
Add a hook for custom rest endpoints
...
Let the user specify custom modules which can be used for implementing extra
endpoints.
2017-11-02 14:36:55 +00:00
Richard van der Hoff
8b56977b6f
Merge pull request #2586 from matrix-org/rav/frontend_proxy_auth_header
...
Front-end proxy: pass through auth header
2017-10-27 11:01:50 +01:00
Richard van der Hoff
173567a7f2
Docstring for post_urlencoded_get_json
2017-10-27 10:59:50 +01:00
Richard van der Hoff
0d8e3ad48b
Fix logcontext leaks in httpclient
...
`preserve_context_over_fn` is borked
2017-10-26 18:17:10 +01:00
Richard van der Hoff
12ef02dc3d
SimpleHTTPClient: add support for headers
...
Sometimes we need to pass headers into these methods
2017-10-26 17:59:50 +01:00
Richard van der Hoff
eaaabc6c4f
replace 'except:' with 'except Exception:'
...
what could possibly go wrong
2017-10-23 15:52:32 +01:00
David Baker
27955056e0
Merge branch 'develop' into erikj/groups_merged
2017-10-02 16:20:41 +01:00
Richard van der Hoff
e43de3ae4b
Improve logging of failures in matrixfederationclient
...
* don't log exception types twice
* not all exceptions have a meaningful 'message'. Use the repr rather than
attempting to build a string ourselves.
2017-09-28 15:38:09 +01:00
Richard van der Hoff
75e67b9ee4
Handle SERVFAILs when doing AAAA lookups for federation ( #2477 )
...
... to cope with people with broken dnssec setups, mostly
2017-09-28 15:24:00 +01:00
Richard van der Hoff
f65e31d22f
Do an AAAA lookup on SRV record targets ( #2462 )
...
Support SRV records which point at AAAA records, as well as A records.
Fixes https://github.com/matrix-org/synapse/issues/2405
2017-09-22 20:26:47 +01:00
Erik Johnston
e884ff31d8
Add DELETE
2017-07-18 16:41:44 +01:00
Erik Johnston
05c13f6c22
Add 'args' param to post_json
2017-07-18 16:40:21 +01:00
Erik Johnston
b8ca494ee9
Initial group server implementation
2017-07-10 15:44:15 +01:00
Krombel
1ff419d343
allow Authorization header which handling got implemented in #1098
...
Signed-off-by: Matthias Kesler <krombel@krombel.de>
2017-06-16 11:21:14 +02:00
Kegan Dougal
d0debb2116
Remember how twisted works
2017-05-05 11:00:21 +01:00
Kegan Dougal
c2ddd773bc
Include the clock
2017-05-05 10:52:46 +01:00
Kegan Dougal
7dd3bf5e24
Rewrite SimpleHttpClient.request to include timeouts
...
Fixes #2191
2017-05-05 10:49:19 +01:00
David Baker
482a2ad122
No need for the exception variable
2017-05-03 11:02:59 +01:00
David Baker
c0380402bc
List caught expection types
2017-05-03 10:56:22 +01:00
David Baker
5fd12dce01
Remove debugging
2017-04-26 12:36:26 +01:00
David Baker
81804909d3
Merge remote-tracking branch 'origin/develop' into dbkr/http_request_propagate_error
2017-04-26 11:31:55 +01:00
David Baker
c366276056
Fix get_json
2017-04-26 10:07:01 +01:00
David Baker
1a9255c12e
Use CodeMessageException subclass instead
...
Parse json errors from get_json client methods and throw special
errors.
2017-04-25 19:30:55 +01:00
David Baker
a46982cee9
Need the HTTP status code
2017-04-21 16:20:12 +01:00
David Baker
70caf49914
Do the same for get_json
2017-04-21 16:09:03 +01:00
David Baker
a1595cec78
Don't error for 3xx responses
2017-04-21 11:51:17 +01:00
David Baker
a90a0f5c8a
Propagate errors sensibly from proxied IS requests
...
When we're proxying Matrix endpoints, parse out Matrix error
responses and turn them into SynapseErrors so they can be
propagated sensibly upstream.
2017-04-21 11:32:48 +01:00
Richard van der Hoff
4903ccf159
Fix some lies, and other clarifications, in docstrings
...
The documentation on get_json has been wrong ever since the very first commit
to synapse...
2017-04-21 01:31:09 +01:00
Richard van der Hoff
a380f041c2
try not to drop context after federation requests
...
preserve_context_over_fn uses a ContextPreservingDeferred, which only restores
context for the duration of its callbacks, which isn't really correct, and
means that subsequent operations in the same request can end up without their
logcontexts.
2017-03-23 22:36:21 +00:00
Richard van der Hoff
5a16cb4bf0
Ignore backoff history for invites, aliases, and roomdirs
...
Add a param to the federation client which lets us ignore historical backoff
data for federation queries, and set it for a handful of operations.
2017-03-23 12:23:22 +00:00
Richard van der Hoff
4bd597d9fc
push federation retry limiter down to matrixfederationclient
...
rather than having to instrument everywhere we make a federation call,
make the MatrixFederationHttpClient manage the retry limiter.
2017-03-23 09:28:46 +00:00
Richard van der Hoff
ad8a26e361
MatrixFederationHttpClient: clean up
...
rename _create_request to _request, and push ascii-encoding of `destination`
and `path` down into it
2017-03-23 00:27:04 +00:00
Erik Johnston
7827251daf
Merge pull request #1994 from matrix-org/dbkr/msisdn_signin_2
...
Phone number registration / login support v2
2017-03-15 09:59:54 +00:00
David Baker
73a5f06652
Support registration / login with phone number
...
Changes from https://github.com/matrix-org/synapse/pull/1971
2017-03-13 17:27:51 +00:00
Richard van der Hoff
170ccc9de5
Fix routing loop when fetching remote media
...
When we proxy a media request to a remote server, add a query-param, which will
tell the remote server to 404 if it doesn't recognise the server_name.
This should fix a routing loop where the server keeps forwarding back to
itself.
Also improves the error handling on remote media fetches, so that we don't
always return a rather obscure 502.
2017-03-13 16:30:36 +00:00
Erik Johnston
7eae6eaa2f
Revert "Support registration & login with phone number"
2017-03-13 09:59:33 +00:00
David Baker
2e27339add
Refector out assert_params_in_request
...
and replace requestEmailToken where we meant requestMsisdnToken
2017-03-08 11:37:34 +00:00
Mark Haines
342fb8dae9
Merge branch 'release-v0.18.6' into develop
2016-12-29 17:33:46 +00:00
Mark Haines
97ffc5690b
Manually abort the underlying TLS connection.
...
The abort() method calls loseConnection() which tries to shutdown the
TLS connection cleanly. We now call abortConnection() directly which
should promptly close both the TLS connection and the underlying TCP
connection.
I also added some TODO markers to consider cancelling the old previous
timeout rather than checking time.time(). But given how urgently we want
to get this code released I'd rather leave the existing code with the
duplicate timeouts and the time.time() check.
2016-12-29 15:51:04 +00:00
Erik Johnston
b4bc6fef5b
Respect long_retries param and default to off
2016-12-29 00:58:34 +00:00
Erik Johnston
68030fd37b
Spelling and comments
2016-12-29 00:10:49 +00:00
Erik Johnston
b7336ff32d
Clean up
2016-12-29 00:09:33 +00:00
Erik Johnston
5b6672c66d
Wrap connections in an N minute timeout to ensure they get reaped correctly
2016-12-29 00:06:53 +00:00
Matthew Hodgson
a58e4e0d48
Merge pull request #1696 from kyrias/ipv6
...
IPv6 support
2016-12-19 23:49:07 +00:00
Johannes Löthberg
0648e76979
Remove spurious newline
...
Apparently I just removed the spaces instead...
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-12 18:41:30 +01:00
Johannes Löthberg
d3bd94805f
Fixup for #1689 and #1690
...
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2016-12-12 16:32:47 +01:00
Glyph
9f07f4c559
IPv6 support for endpoint.py
...
Similar to https://github.com/matrix-org/synapse/pull/1689 , but for endpoint.py
2016-12-11 11:10:32 +01:00
Glyph
6e18805ac2
IPv6 support for client.py
...
This is an (untested) general sketch of how to use wrapClientTLS to implement TLS over IPv6, as well as faster connections over IPv4.
2016-12-11 11:10:32 +01:00
Erik Johnston
f32fb65552
Add new API appservice specific public room list
2016-12-06 16:12:27 +00:00
Richard van der Hoff
b5b3a7e867
More intelligent Content-Type parsing
...
Content-Type is allowed to contain options (`; charset=utf-8`, for
instance). We should allow that.
2016-11-30 15:07:32 +00:00
Mark Haines
b1c27975d0
Set CORs headers on responses from the media repo
2016-11-02 11:29:25 +00:00
Erik Johnston
5810cffd33
Pass since/from parameters over federation
2016-09-15 10:36:19 +01:00
Mark Haines
949c2c5435
Add a timeout parameter for end2end key queries.
...
Add a timeout parameter for controlling how long synapse will wait
for responses from remote servers. For servers that fail include how
they failed to make it easier to debug.
Fetch keys from different servers in parallel rather than in series.
Set the default timeout to 10s.
2016-09-12 18:17:09 +01:00
Erik Johnston
33f3624ff7
Add exception logging. Fix typo
2016-08-22 10:49:31 +01:00
Erik Johnston
8c52160b07
Allow request handlers to override metric name
2016-08-22 10:44:45 +01:00
Erik Johnston
a093fab253
Use top level measure
2016-08-22 10:18:12 +01:00
Erik Johnston
27e0178da9
Add a top level measure
2016-08-19 18:49:37 +01:00
Erik Johnston
6d1a94d218
Remove redundant measure
2016-08-19 18:40:31 +01:00
Erik Johnston
39b900b316
Measure http.server render
2016-08-19 18:23:44 +01:00
Erik Johnston
ca8abfbf30
Clean up TransactionQueue
2016-08-10 16:24:16 +01:00
Richard van der Hoff
33d08e8433
Log when adding listeners
2016-07-25 17:22:15 +01:00
Richard van der Hoff
255c229f23
Work around TLS bug in twisted
...
Wrap up twisted's FileBodyProducer to work around
https://twistedmatrix.com/trac/ticket/8473 . Hopefully this fixes
https://matrix.org/jira/browse/SYN-700 .
2016-06-15 10:39:08 +01:00
Mark Haines
eb79110beb
Clean up the blacklist/whitelist handling.
...
Always set the config key with an empty list, even if a list isn't specified.
This means that the codepaths are the same for both the empty list and
for a missing key. Since the behaviour is the same for both cases this
makes the code somewhat easier to reason about.
2016-05-16 13:03:59 +01:00
Matthew Hodgson
792def4928
add a url_preview_ip_range_whitelist config param so we can whitelist the matrix.org IP space
2016-05-01 12:44:24 +01:00
Mark Haines
351b50a887
Fix more typos in per-request metrics
2016-04-28 15:29:46 +01:00
Mark Haines
dcfc10b129
Fix typo in request metrics
2016-04-28 15:11:06 +01:00
Mark Haines
1a12766e3b
Add a comment explaining why automatic metric reporting is disabled for JsonResource
2016-04-28 12:31:26 +01:00
Mark Haines
6037349512
Check if report_metrics is True
2016-04-28 12:26:07 +01:00
Mark Haines
8d7ad44331
Report per request metrics for all of the things using request_handler
2016-04-28 10:57:49 +01:00
Mark Haines
e856036f4c
Move SynapseSite to its own file
2016-04-22 16:09:55 +01:00
Erik Johnston
f505575f69
Make InsecureInterceptableContextFactory work with SpiderEndpoint
2016-04-19 16:08:14 +01:00
Matthew Hodgson
4bd3d25218
Merge pull request #688 from matrix-org/matthew/preview_urls
...
URL previewing support
2016-04-11 10:40:29 +01:00
Matthew Hodgson
83b2f83da0
actually throw meaningful errors
2016-04-08 21:36:59 +01:00
Matthew Hodgson
dafef5a688
Add url_preview_enabled config option to turn on/off preview_url endpoint. defaults to off.
...
Add url_preview_ip_range_blacklist to let admins specify internal IP ranges that must not be spidered.
Add url_preview_url_blacklist to let admins specify URL patterns that must not be spidered.
Implement a custom SpiderEndpoint and associated support classes to implement url_preview_ip_range_blacklist
Add commentary and generally address PR feedback
2016-04-08 18:37:15 +01:00
Erik Johnston
a28d066732
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/dns_cache
2016-04-07 11:11:17 +01:00
Matthew Hodgson
9f7dc2bef7
Merge branch 'develop' into matthew/preview_urls
2016-04-04 00:38:21 +01:00
Matthew Hodgson
8b98a7e8c3
pep8
2016-04-03 12:56:29 +01:00
Matthew Hodgson
7426c86eb8
add a persistent cache of URL lookups, and fix up the in-memory one to work
2016-04-03 00:31:57 +01:00
Matthew Hodgson
d1b154a10f
support gzip compression, and don't pass through error msgs
2016-04-02 03:06:39 +01:00
Mark Haines
2a37467fa1
Use google style doc strings.
...
pycharm supports them so there is no need to use the other format.
Might as well convert the existing strings to reduce the risk of
people accidentally cargo culting the wrong doc string format.
2016-04-01 16:12:07 +01:00
Matthew Hodgson
683e564815
handle spidered relative images correctly
2016-03-31 23:52:58 +01:00
Matthew Hodgson
0d3d7de6fc
sync in changes from matrixfederationclient
2016-03-31 12:42:27 +01:00
Erik Johnston
f9d3665c88
Allow clock to be passed in to func
2016-03-31 10:23:48 +01:00
Erik Johnston
f699b8f997
Read from DNS cache if within TTL
2016-03-31 10:04:28 +01:00
Matthew Hodgson
a8a5dd3b44
handle requests with missing content-length headers (e.g. YouTube)
2016-03-31 01:55:21 +01:00
Matthew Hodgson
721b2bfa85
implement redirects
2016-03-29 03:32:52 +01:00
Matthew Hodgson
64b4aead15
make it work
2016-03-29 03:13:25 +01:00
Matthew Hodgson
dd4287ca5d
make it build
2016-03-29 02:07:57 +01:00
Matthew Hodgson
e0c2490a14
Merge branch 'develop' into matthew/preview_urls
2016-03-29 01:20:25 +01:00
Matthew Hodgson
d9d48aad2d
Merge branch 'develop' into matthew/preview_urls
2016-03-27 22:54:42 +01:00
Erik Johnston
acdfef7b14
Intern all the things
2016-03-23 16:25:54 +00:00
Mark Haines
398cd1edfb
Fix regression where synapse checked whether push rules were valid JSON before the compatibility hack that handled clients sending invalid JSON
2016-03-14 14:16:41 +00:00
Mark Haines
e9c1cabac2
Use parse_json_object_from_request to parse JSON out of request bodies
2016-03-11 16:41:03 +00:00
Mark Haines
b7dbe5147a
Add a parse_json_object function
...
to deduplicate all the copy+pasted _parse_json functions. Also document
the parse_.* functions.
2016-03-09 11:26:26 +00:00