25eb769b26
Efficiency fix for lookups of a single protocol
2016-09-09 13:25:02 +01:00
3328428d05
Allow lookup of a single 3PE protocol query metadata
2016-09-09 13:19:04 +01:00
b8f84f99ff
Merge pull request #1081 from matrix-org/dbkr/notifications_only_highlight
...
Implement `only=highlight` on `/notifications`
2016-09-09 00:09:51 +01:00
257025ac89
Merge pull request #1082 from matrix-org/erikj/remote_public_rooms
...
Add server param to /publicRooms
2016-09-08 16:04:22 +01:00
3f9889bfd6
Use parse_string
2016-09-08 15:51:10 +01:00
fa9d36e050
Merge branch 'develop' into markjh/direct_to_device_federation
2016-09-08 13:43:43 +01:00
4ef222ab61
Implement `only=highlight` on `/notifications`
2016-09-08 13:43:35 +01:00
791658b576
Add server param to /publicRooms
2016-09-08 11:53:05 +01:00
d4a35ada28
Send device messages over federation
2016-09-06 18:16:20 +01:00
d51b8a1674
Add quotes and be explicity about script-src
2016-09-05 17:35:01 +01:00
662b031a30
Allow PDF to be rendered from media repo
2016-09-05 17:25:26 +01:00
7ed5acacf4
Fix up the calls to the notifier for device messages
2016-09-01 18:08:40 +01:00
1aa3e1d287
Add a replication stream for direct to device messages
2016-08-31 10:38:58 +01:00
4bbef62124
Merge remote-tracking branch 'origin/develop' into markjh/direct_to_device
2016-08-26 14:35:31 +01:00
1294d4a329
Move ThirdPartyEntityKind into api.constants so the expectation becomes that the value is significant
2016-08-25 18:34:47 +01:00
ab34fdecb7
Merge branch 'develop' into markjh/direct_to_device
2016-08-25 18:34:46 +01:00
b162cb2e41
Add some TODOs
2016-08-25 18:18:53 +01:00
641efb6a39
Fix the deduplication of incoming direct-to-device messages
2016-08-25 18:14:02 +01:00
e993925279
Add store-and-forward direct-to-device messaging
2016-08-25 17:35:37 +01:00
d9dcb2ba3a
Merge pull request #1041 from matrix-org/paul/third-party-lookup
...
Extend 3PE lookup APIs for metadata query
2016-08-25 17:06:53 +01:00
8e1ed09dff
Move static knowledge of protocol metadata into AS handler; cache the result
2016-08-24 13:01:53 +01:00
965f33c901
Declare 'gitter' known protocol, with user lookup
2016-08-24 12:34:03 +01:00
9899824b85
Initial hack at the 3PN protocols metadata lookup API
2016-08-24 12:33:01 +01:00
9219139351
Preserve some logcontexts
2016-08-24 11:58:40 +01:00
63c19e1df9
Move 3PU/3PL lookup APIs into /thirdparty containing entity
2016-08-24 11:55:57 +01:00
4b31426a02
Pass through user-supplied content in /join/$room_id
...
It was always intended to allow custom keys on the join event, but this has
at some point been lost. Restore it.
If the user specifies keys like "avatar_url" then they will be clobbered.
2016-08-23 16:32:04 +01:00
6e80c03d45
Merge branch 'develop' into dbkr/notifications_api
2016-08-20 00:16:18 +01:00
0acdd0f1ea
Use tuple comparison
...
Hopefully easier to read
2016-08-18 17:51:08 +01:00
b515f844ee
Avoid so much copypasta between 3PU and 3PL query by unifying around a ThirdPartyEntityKind enumeration
2016-08-18 17:19:55 +01:00
602c84cd9c
Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api
2016-08-18 17:15:26 +01:00
105ff162d4
Authenticate 3PE lookup requests
2016-08-18 16:19:23 +01:00
06964c4a0a
Copypasta the 3PU support code to also do 3PL
2016-08-18 16:09:50 +01:00
f3afd6ef1a
Remove TODO note about request fields being strings - they're always strings
2016-08-18 15:53:01 +01:00
d5bf7a4a99
Merge remote-tracking branch 'origin/develop' into paul/thirdpartylookup
2016-08-18 14:21:01 +01:00
3856582741
Ensure that 3PU lookup request fields actually get passed in
2016-08-18 14:06:02 +01:00
0af9e1a637
Set `Content-Security-Policy` on media repo
...
This is to inform browsers that they should sandbox the returned
media. This is particularly cruical for javascript/HTML files.
2016-08-17 16:27:39 +01:00
fa87c981e1
Thread 3PU lookup through as far as the AS API object; which currently noöps it
2016-08-17 16:17:28 +01:00
f90b3d83a3
Add None check to _iterate_over_text
2016-08-17 15:17:17 +01:00
e3e3fbc23a
Initial empty implementation that just registers an API endpoint handler
2016-08-17 12:46:49 +01:00
109a560905
Flake8
2016-08-16 14:57:21 +01:00
48b5829aea
Fix up preview URL API. Add tests.
...
This includes:
- Splitting out methods of a class into stand alone functions, to make
them easier to test.
- Adding unit tests to split out functions, testing HTML -> preview.
- Handle the fact that elements in lxml may have tail text.
2016-08-16 14:53:24 +01:00
4e1cebd56f
Make synchrotron accept /events
2016-08-12 15:31:44 +01:00
866a5320de
Dont invoke get_handlers fromClientV1RestServlet
...
hs.get_handlers() can not be invoked from split out processes. Moving
the invocations down a level means that we can slowly split out
individual servlets.
2016-08-12 10:03:19 +01:00
b4ecf0b886
Merge remote-tracking branch 'origin/develop' into dbkr/notifications_api
2016-08-11 14:09:13 +01:00
2510db3e76
Don't change status_msg on /sync
2016-08-10 12:59:59 +01:00
fa1ce4d8ad
Don't print stack traces when failing to get remote keys
2016-08-10 10:44:37 +01:00
cd41c6ece2
Merge pull request #995 from matrix-org/rav/clean_up_cas_login
...
Clean up CAS login code
2016-08-09 10:21:56 +01:00
65666fedd5
Clean up CAS login code
...
Remove some apparently unused code.
Clean up parse_cas_response, mostly to catch the exception if the CAS response
isn't valid XML.
2016-08-08 17:17:25 +01:00
0682ca04b3
Fix CAS login
...
Attempting to log in with CAS was giving a 500 error.
2016-08-08 17:01:30 +01:00
5bcccfde6c
Don't include html comments in description
2016-08-05 14:45:11 +01:00
b5525c76d1
Typo
2016-08-04 16:10:08 +01:00
e97648c4e2
Test summarization
2016-08-04 16:09:09 +01:00
b3682df2ca
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/xpath_fix
2016-08-04 15:29:45 +01:00
530bc862dc
Merge branch 'rav/null_default_device_displayname' into develop
2016-08-03 14:30:32 +01:00
4fec5e57be
Default device_display_name to null
...
It turns out that it's more useful to return a null device display name (and
let clients decide how to handle it: eg, falling back to device_id) than using
a constant string like "unknown device".
2016-08-03 11:53:00 +01:00
921f17f938
Merge branch 'develop' into rav/refactor_device_query
2016-08-03 11:12:47 +01:00
58c9653c6b
Don't infer paragrahs from newlines
2016-08-02 18:50:24 +01:00
6b58ade2f0
Comment on why we clone
2016-08-02 18:41:22 +01:00
9e66c58ceb
Spelling.
2016-08-02 18:37:31 +01:00
f83f5fbce8
Make it actually compile
2016-08-02 18:32:42 +01:00
aecaec3e10
Change the way we summarize URLs
...
Using XPath is slow on some machines (for unknown reasons), so use a
different approach to get a list of text nodes.
Try to generate a summary that respect paragraph and then word
boundaries, adding ellipses when appropriate.
2016-08-02 18:25:53 +01:00
986615b0b2
Move e2e query logic into a handler
2016-08-01 18:02:07 +01:00
271d3e7865
Fix adding emails on registration
...
Synapse was not adding email addresses to accounts registered with an email address, due to too many different variables called 'result'. Rename both of them. Also remove the defer.returnValue() with no params because that's not a thing.
2016-07-29 15:25:24 +01:00
f6f8f81a48
Add r0.1.0 to the "supported versions" list
2016-07-28 10:14:07 +01:00
fda078f995
Add r0.2.0 to the "supported versions" list
2016-07-28 09:14:21 +01:00
ccec25e2c6
key upload tweaks
...
1. Add v2_alpha URL back in, since things seem to be using it.
2. Don't reject the request if the device_id in the upload request fails to
match that in the access_token.
2016-07-27 16:41:06 +01:00
d47115ff8b
Delete e2e keys on device delete
2016-07-27 12:24:52 +01:00
2e3d90d67c
Make the device id on e2e key upload optional
...
We should now be able to get our device_id from the access_token, so the
device_id on the upload request is optional. Where it is supplied, we should
check that it matches.
For active access_tokens without an associated device_id, we ought to register
the device in the devices table.
Also update the table on upgrade so that all of the existing e2e keys are
associated with real devices.
2016-07-26 23:38:12 +01:00
eb359eced4
Add `create_requester` function
...
Wrap the `Requester` constructor with a function which provides sensible
defaults, and use it throughout
2016-07-26 16:46:53 +01:00
012b4c1913
Implement updating devices
...
You can update the displayname of devices now.
2016-07-26 07:35:48 +01:00
436bffd15f
Implement deleting devices
2016-07-26 07:35:48 +01:00
2c28e25bda
Merge pull request #943 from matrix-org/rav/get_device_api
...
Implement GET /device/{deviceId}
2016-07-21 13:41:42 +01:00
1e6e370b76
Merge pull request #942 from matrix-org/rav/fix_register_deviceid
...
Preserve device_id from first call to /register
2016-07-21 13:16:31 +01:00
406f7aa0f6
Implement GET /device/{deviceId}
2016-07-21 12:00:29 +01:00
1a64dffb00
Preserve device_id from first call to /register
...
device_id may only be passed in the first call to /register, so make sure we
fish it out of the register `params` rather than the body of the final call.
2016-07-21 11:34:16 +01:00
7314bf4682
Merge branch 'develop' into rav/get_devices_api
...
(pick up PR #938 in the hope of fixing the UTs)
2016-07-20 17:40:00 +01:00
bc8f265f0a
GET /devices endpoint
...
implement a GET /devices endpoint which lists all of the user's devices.
It also returns the last IP where we saw that device, so there is some dancing
to fish that out of the user_ips table.
2016-07-20 16:42:32 +01:00
b97a1356b1
Register a device_id in the /v2/register flow.
...
This doesn't cover *all* of the registration flows, but it does cover the most
common ones: in particular: shared_secret registration, appservice
registration, and normal user/pass registration.
Pull device_id from the registration parameters. Register the device in the
devices table. Associate the device with the returned access and refresh
tokens. Profit.
2016-07-20 16:38:27 +01:00
c68518dfbb
Merge pull request #933 from matrix-org/rav/type_annotations
...
Type annotations
2016-07-20 12:26:32 +01:00
e967bc86e7
Merge pull request #932 from matrix-org/rav/register_refactor
...
Further registration refactoring
2016-07-20 11:03:33 +01:00
1e2a7f18a1
Merge pull request #922 from matrix-org/erikj/file_api2
...
Feature: Add filter to /messages. Add 'contains_url' to filter.
2016-07-20 10:40:48 +01:00
3413f1e284
Type annotations
...
Add some type annotations to help PyCharm (in particular) to figure out the
types of a bunch of things.
2016-07-19 18:56:16 +01:00
40cbffb2d2
Further registration refactoring
...
* `RegistrationHandler.appservice_register` no longer issues an access token:
instead it is left for the caller to do it. (There are two of these, one in
`synapse/rest/client/v1/register.py`, which now simply calls
`AuthHandler.issue_access_token`, and the other in
`synapse/rest/client/v2_alpha/register.py`, which is covered below).
* In `synapse/rest/client/v2_alpha/register.py`, move the generation of
access_tokens into `_create_registration_details`. This means that the normal
flow no longer needs to call `AuthHandler.issue_access_token`; the
shared-secret flow can tell `RegistrationHandler.register` not to generate a
token; and the appservice flow continues to work despite the above change.
2016-07-19 18:46:19 +01:00
b9e997f561
Merge pull request #931 from matrix-org/rav/refactor_register
...
rest/client/v2_alpha/register.py: Refactor flow somewhat.
2016-07-19 16:13:45 +01:00
8f6281ab0c
Don't bind email unless threepid contains expected fields
2016-07-19 15:50:01 +01:00
0da0d0a29d
rest/client/v2_alpha/register.py: Refactor flow somewhat.
...
This is meant to be an *almost* non-functional change, with the exception that
it fixes what looks a lot like a bug in that it only calls
`auth_handler.add_threepid` and `add_pusher` once instead of three times.
The idea is to move the generation of the `access_token` out of
`registration_handler.register`, because `access_token`s now require a
device_id, and we only want to generate a device_id once registration has been
successful.
2016-07-19 13:12:22 +01:00
f863a52cea
Add device_id support to /login
...
Add a 'devices' table to the storage, as well as a 'device_id' column to
refresh_tokens.
Allow the client to pass a device_id, and initial_device_display_name, to
/login. If login is successful, then register the device in the devices table
if it wasn't known already. If no device_id was supplied, make one up.
Associate the device_id with the access token and refresh token, so that we can
get at it again later. Ensure that the device_id is copied from the refresh
token to the access_token when the token is refreshed.
2016-07-18 16:39:44 +01:00
93efcb8526
Merge pull request #928 from matrix-org/rav/refactor_login
...
Refactor login flow
2016-07-18 16:12:35 +01:00
dcfd71aa4c
Refactor login flow
...
Make sure that we have the canonical user_id *before* calling
get_login_tuple_for_user_id.
Replace login_with_password with a method which just validates the password,
and have the caller call get_login_tuple_for_user_id. This brings the password
flow into line with the other flows, and will give us a place to register the
device_id if necessary.
2016-07-18 15:23:54 +01:00
511a52afc8
Use body.get to check for 'user'
2016-07-16 18:44:08 +01:00
e885e2a623
Fall back to 'username' if 'user' is not given for appservice reg.
2016-07-16 18:33:48 +01:00
a3036ac37e
Merge pull request #921 from matrix-org/erikj/account_deactivate
...
Feature: Add an /account/deactivate endpoint
2016-07-14 17:25:15 +01:00
a98d215204
Add filter param to /messages API
2016-07-14 16:30:56 +01:00
209e04fa11
Merge pull request #918 from negzi/bugfix_for_token_expiry
...
Bug fix: expire invalid access tokens
2016-07-14 15:51:52 +01:00
848d3bf2e1
Add hs object
2016-07-14 10:25:52 +01:00
b55c770271
Only accept password auth
2016-07-14 10:00:38 +01:00
d543b72562
Add an /account/deactivate endpoint
2016-07-14 09:56:53 +01:00
0136a522b1
Bug fix: expire invalid access tokens
2016-07-13 15:00:37 +02:00
c55ad2e375
be more pythonic
2016-07-12 14:15:10 +01:00
aaa9d9f0e1
on_OPTIONS isn't neccessary
2016-07-12 14:13:14 +01:00
75fa7f6b3c
Remove other debug logging
2016-07-12 14:08:57 +01:00
a5db0026ed
Separate out requestTokens to separate handlers
2016-07-11 09:57:07 +01:00
9c491366c5
Oops, remove debug logging
2016-07-11 09:07:40 +01:00
385aec4010
Implement https://github.com/matrix-org/matrix-doc/pull/346/files
2016-07-08 17:42:48 +01:00
67f2c901ea
Add rest servlet. Fix SQL.
2016-07-06 15:56:59 +01:00
a17e7caeb7
Merge branch 'erikj/shared_secret' into erikj/test2
2016-07-06 14:46:31 +01:00
76b18df3d9
Check that there are no null bytes in user and passsword
2016-07-06 11:17:53 +01:00
0da24cac8b
Add null separator to hmac
2016-07-06 11:05:16 +01:00
651faee698
Add an admin option to shared secret registration
2016-07-05 17:30:22 +01:00
caf33b2d9b
Protect password when registering using shared secret
2016-07-05 17:18:19 +01:00
2d21d43c34
Add purge_history API
2016-07-05 10:28:51 +01:00
bb069079bb
Fix style violations
...
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-04 22:07:11 +09:00
2e5a31f197
Use .get() instead of [] to access password_hash
2016-07-04 22:00:13 +09:00
fc8007dbec
Optionally include password hash in createUser endpoint
...
Signed-off-by: Kent Shikama <kent@kentshikama.com>
2016-07-03 15:08:15 +09:00
f328d95cef
Feature: Add deactivate account admin API
...
Allows server admins to "deactivate" accounts, which:
- Revokes all access tokens
- Removes all threepids
- Removes password
The API is a POST to `/admin/deactivate/<user_id>`
2016-06-30 15:40:58 +01:00
f52cb4cd78
Remove race
2016-06-29 15:24:50 +01:00
a70688445d
Implement purge_media_cache admin API
2016-06-29 14:57:59 +01:00
314b146b2e
Track approximate last access time for remote media
2016-06-29 11:41:20 +01:00
13e334506c
Remove the legacy v0 content upload API.
...
The existing content can still be downloaded. The last upload to the
matrix.org server was in January 2015, so it is probably safe to remove
the upload API.
2016-06-21 11:47:39 +01:00
09a17f965c
Line lengths
2016-06-15 16:58:12 +01:00
1e9026e484
Handle floats as img widths
2016-06-15 16:58:05 +01:00
a60169ea09
Handle og props with not content
2016-06-15 16:57:48 +01:00
eba4ff1bcb
502 on /thumbnail when can't contact remote server
2016-06-09 11:29:43 +01:00
95f305c35a
Remove redundant exception log in /events
2016-06-09 11:15:04 +01:00
690029d1a3
Don't make rooms visibile by default
2016-06-08 14:47:42 +01:00
efeabd3180
Log user that is making /publicRooms calls
2016-06-08 14:23:15 +01:00
6a0afa582a
Load push rules in storage layer, so that they get cached
2016-06-03 11:10:00 +01:00
1f31cc37f8
Working unsubscribe links going straight to the HS
...
and authed by macaroons that let you delete pushers and nothing else
2016-06-02 17:21:31 +01:00
3a3fb2f6f9
Merge branch 'dbkr/split_out_auth_handler' into dbkr/email_unsubscribe
2016-06-02 13:35:25 +01:00
4a10510cd5
Split out the auth handler
2016-06-02 13:31:45 +01:00
991af8b0d6
WIP on unsubscribing email notifs without logging in
2016-06-01 17:40:52 +01:00
d240796ded
Basic, un-cached support for secondary_directory_servers
2016-05-31 17:20:07 +01:00
887c6e6f05
Split out the room list handler
...
So I can use it from federation bits without pulling in all the handlers.
2016-05-31 11:05:16 +01:00
37b7e84620
Include the ts the notif was received at
2016-05-24 11:33:32 +01:00
b791a530da
Actually make the 'read' flag correct
2016-05-23 18:48:02 +01:00
a24bc5b2dc
Add GET /notifications API
2016-05-23 18:33:51 +01:00
332d7e9b97
Allow clients to specify a server_name to avoid 'No known servers'
...
Multiple server_names are supported via ?server_name=foo&server_name=bar
2016-05-19 13:50:52 +01:00
0cb441fedd
Move typing handler out of the Handlers object
2016-05-17 15:58:46 +01:00
f68eea808a
Move SyncHandler out of the Handlers object
2016-05-16 20:19:26 +01:00
3b86ecfa79
Move the presence handler out of the Handlers object
2016-05-16 18:56:37 +01:00
eb79110beb
Clean up the blacklist/whitelist handling.
...
Always set the config key with an empty list, even if a list isn't specified.
This means that the codepaths are the same for both the empty list and
for a missing key. Since the behaviour is the same for both cases this
makes the code somewhat easier to reason about.
2016-05-16 13:03:59 +01:00
40aa6e8349
Create user with expiry
...
- Add unittests for client, api and handler
Signed-off-by: Negar Fazeli <negar.fazeli@ericsson.com>
2016-05-13 15:34:15 +02:00
997db04648
Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs
2016-05-10 14:40:19 +02:00
c00b484eff
More consistent config naming
2016-05-10 14:39:16 +02:00
94040b0798
Add config option to not send email notifs for new users
2016-05-10 14:34:53 +02:00
573ef3f1c9
Rename openid/token to openid/request_token
2016-05-05 15:15:00 +01:00
9c272da05f
Add an openidish mechanism for proving to third parties that you own a given user_id
2016-05-05 13:42:44 +01:00
8e6a163f27
Add timestamp and auto incrementing ID
2016-05-04 15:19:12 +01:00
5650e38e7d
Move event_id to path
2016-05-04 13:19:39 +01:00
984d4a2c0f
Add /report endpoint
2016-05-04 11:28:10 +01:00
2df75de505
Merge remote-tracking branch 'origin/develop' into dbkr/email_notifs
2016-04-29 20:28:47 +01:00
dc2c527ce9
Fix password reset
...
Default requester to None, otherwise it isn't defined when resetting using email auth
2016-04-29 12:07:54 +01:00
b2c04da8dc
Add an email pusher for new users
...
If they registered with an email address and email notifs are enabled on the HS
2016-04-29 11:43:57 +01:00
8d7ad44331
Report per request metrics for all of the things using request_handler
2016-04-28 10:57:49 +01:00
52ecbc2843
Make pyjwt dependency optional
2016-04-25 14:30:15 +01:00
b9675ef6e6
Merge pull request #687 from nikriek/jwt-fix
...
Fix issues with JWT login
2016-04-21 17:42:25 +01:00
565c2edb0a
Fix issues with JWT login
2016-04-21 18:10:48 +02:00
e8884e5e9c
Add self.media_repo to PreviewUrlResource
2016-04-19 14:51:34 +01:00
a7001c311b
_make_dirs was moved to MediaRepository
2016-04-19 14:49:31 +01:00
9181e2f4c7
Add store to PreviewUrlResource
2016-04-19 14:48:24 +01:00
fb76a81ff7
Reorder imports
2016-04-19 14:45:05 +01:00
0c93df89b6
Move MediaRepository to media_repository module
2016-04-19 11:31:43 +01:00
43f0941e8f
Split out BaseMediaResource into MediaRepository
...
This is so that a single MediaRepository can be shared across all
resources, rather than having a "copy" per resource.
In particular this allows us to guard against both the thumbnail and
download resource triggering a download of remote content at the same
time.
2016-04-19 11:24:59 +01:00
aaabbd3e9e
explicitly pass in the charset from Content-Type to lxml to fix cyrillic woes better
2016-04-15 14:32:25 +01:00
84f9cac4d0
fix cyrillic URL previews by hardcoding all page decoding to UTF-8 for now, rather than relying on lxml's heuristics which seem to get it wrong
2016-04-15 13:20:08 +01:00
f78b479118
fix urlparse import thinko breaking tiny URLs
2016-04-14 15:23:55 +01:00
83776d6219
Make v2_alpha reg follow the AS API specification
...
The spec is clear the key should be 'user' not 'username' and this is indeed
the case for v1. This is not true for v2_alpha though, which is what this
commit is fixing.
2016-04-14 14:52:26 +01:00
bd77216d06
comment out 2c838f6459 due to risk of https://en.wikipedia.org/wiki/Billion_laughs attacks - thanks @torhve
2016-04-14 14:39:24 +01:00
d0633e6dbe
Sanitize the optional dependencies for spider API
2016-04-13 13:38:09 +01:00
a04c076b7f
Make the /set part mandatory
2016-04-12 13:54:41 +01:00
7b39bcdaae
Mis-named function
2016-04-12 13:35:08 +01:00
d937f342bb
Split into separate servlet classes
2016-04-12 13:33:30 +01:00
8a76094965
Add get endpoint for pushers
...
As per https://github.com/matrix-org/matrix-doc/pull/308
2016-04-11 18:00:03 +01:00
17515bae14
PEP8
2016-04-11 11:02:50 +01:00
5ffacc5e84
fix typos and needless try/except from PR review
2016-04-11 10:39:16 +01:00
83b2f83da0
actually throw meaningful errors
2016-04-08 21:36:59 +01:00
b36270b5e1
Fix pep8 warning
2016-04-08 19:52:23 +01:00
1ccabe2965
more PR feedback
2016-04-08 18:58:08 +01:00
dafef5a688
Add url_preview_enabled config option to turn on/off preview_url endpoint. defaults to off.
...
Add url_preview_ip_range_blacklist to let admins specify internal IP ranges that must not be spidered.
Add url_preview_url_blacklist to let admins specify URL patterns that must not be spidered.
Implement a custom SpiderEndpoint and associated support classes to implement url_preview_ip_range_blacklist
Add commentary and generally address PR feedback
2016-04-08 18:37:15 +01:00
9f7dc2bef7
Merge branch 'develop' into matthew/preview_urls
2016-04-04 00:38:21 +01:00
cf51c4120e
report image size (bytewise) in OG meta
2016-04-03 23:57:05 +01:00
0834b152fb
char encoding
2016-04-03 12:59:27 +01:00
8b98a7e8c3
pep8
2016-04-03 12:56:29 +01:00
eab4d462f8
fix etag typing error. fix timestamp typing error
2016-04-03 02:02:46 +01:00
c3916462f6
rebase all image URLs
2016-04-03 01:33:12 +01:00
110780b18b
remove stale todo
2016-04-03 00:48:31 +01:00
b09e29a03c
Ensure only one download for a given URL is active at a time
2016-04-03 00:47:40 +01:00
7426c86eb8
add a persistent cache of URL lookups, and fix up the in-memory one to work
2016-04-03 00:31:57 +01:00
d1b154a10f
support gzip compression, and don't pass through error msgs
2016-04-02 03:06:39 +01:00
9377157961
how was _respond_default_thumbnail ever meant to work?
2016-04-02 02:31:45 +01:00
2c838f6459
pass back SVGs as their own thumbnails
2016-04-02 02:30:07 +01:00
5037ee0d37
handle missing dimensions without crashing
2016-04-02 02:29:57 +01:00
b26e8604f1
make meta comparisons case insensitive
2016-04-02 01:35:44 +01:00
5fd07da764
refactor calc_og; spider image URLs; fix xpath; add a (broken) expiringcache; loads of other fixes
2016-04-02 00:35:49 +01:00
2a37467fa1
Use google style doc strings.
...
pycharm supports them so there is no need to use the other format.
Might as well convert the existing strings to reduce the risk of
people accidentally cargo culting the wrong doc string format.
2016-04-01 16:12:07 +01:00
c60b751694
fix assorted redirect, unicode and screenscraping bugs
2016-04-01 02:17:48 +01:00
683e564815
handle spidered relative images correctly
2016-03-31 23:52:58 +01:00
72550c3803
prevent choking on invalid utf-8, and handle image thumbnailing smarter
2016-03-31 15:14:14 +01:00
bb9a2ca87c
synthesise basig OG metadata from pages lacking it
2016-03-31 14:15:09 +01:00
a8a5dd3b44
handle requests with missing content-length headers (e.g. YouTube)
2016-03-31 01:55:21 +01:00
Paul "LeoNerd" Evans