607bd27c83
fix pep8
2018-06-05 18:10:35 +01:00
d62162bbec
doc fixes
2018-06-05 18:09:13 +01:00
522bd3c8a3
Merge remote-tracking branch 'origin/master' into develop
2018-06-05 17:42:49 +01:00
d6e3c2c79b
Let's try labels instead of label, that might work
2018-06-05 17:30:45 +01:00
f7869f8f8b
Port to sortedcontainers (with tests!) ( #3332 )
2018-06-06 00:13:57 +10:00
604cff1a06
Add metrics to track appservice transactions
2018-06-05 13:21:30 +01:00
f29b41fde9
Merge pull request #3324 from matrix-org/rav/remove_dead_method
...
Remove was_forgotten_at
2018-06-04 18:18:08 +01:00
28b0490dfd
Merge pull request #3334 from matrix-org/rav/cache_factor_override
...
Cache factor override system for specific caches
2018-06-04 17:19:33 +01:00
b7e7fd2d0e
Fix replication metrics
...
fix bug introduced in #3256
2018-06-04 16:23:05 +01:00
244ab974e7
bump version and changelog
2018-06-04 16:09:58 +01:00
694968fa81
Hopefully, fix LaterGuage error handling
2018-06-04 15:59:14 +01:00
042eedfa2b
Add hacky cache factor override system
2018-06-04 15:39:28 +01:00
c5930d513a
Docstring
2018-06-04 12:05:58 +01:00
6a29e815fc
Fix comment
2018-06-04 12:01:23 +01:00
e44150a6de
Missing yield
2018-06-04 12:01:13 +01:00
f731e42baf
docstring
2018-06-04 12:00:51 +01:00
7d9d75e4e8
federation/send_queue.py: fix usage of LaterGauge
...
Fixes a startup crash due to commit df9f72d9e5
2018-06-03 14:16:17 +03:00
09503126df
Strip `access_token` from outgoing requests using existing regex
2018-06-02 23:25:13 +01:00
c1f4118bb6
Remove was_forgotten_at
...
This is unused. IT MUST DIE!!!1
̧̪͈̱̹̳͖͙H̵̰̤̰͕̖e̛ ͚͉̗̼̞w̶̩̥͉̮h̩̺̪̩͘ͅọ͎͉̟ ̜̩͔̦̘ͅW̪̫̩̣̲͔̳a͏͔̳͖i͖͜t͓̤̠͓͙s̘̰̩̥̙̝ͅ ̲̠̬̥Be̡̙̫̦h̰̩i̛̫͙͔̭̤̗̲n̳͞d̸ ͎̻͘T̛͇̝̲̹̠̗ͅh̫̦̝ͅe̩̫͟ ͓͖̼W͕̳͎͚̙̥ą̙l̘͚̺͔͞ͅl̳͍̙̤̤̮̳.̢
̟̺̜̙͉Z̤̲̙̙͎̥̝A͎̣͔̙͘L̥̻̗̳̻̳̳͢G͉̖̯͓̞̩̦O̹̹̺!̙͈͎̞̬ *
2018-06-01 18:21:49 +01:00
a9e97dcd65
Merge pull request #3317 from thegcat/feature/3312-add_ipv6_to_blacklist_example_config
...
Add private IPv6 addresses to example config for url preview blacklist
2018-06-01 14:45:14 +01:00
71477f3317
Merge pull request #3264 from matrix-org/neil/sign-up-stats
...
daily user type phone home stats
2018-06-01 13:42:01 +00:00
9f797a24a4
Handle RRs which arrive before their events
2018-06-01 14:01:43 +01:00
857e6fd8b6
Ignore depth when updating read-receipts
...
Order read receipts by stream ordering instead of depth
2018-06-01 12:18:11 +01:00
4ef76f3ac4
Add private IPv6 addresses to preview blacklist #3312
...
The added addresses are expected to be local or loopback addresses and
shouldn't be spidered for previews.
Signed-off-by: Felix Schäfer <felix@thegcat.net>
2018-06-01 12:18:35 +02:00
4986b084f8
remove unnecessary INSERT
2018-06-01 10:50:40 +01:00
febe0ec8fd
Run Prometheus on a different port, optionally. ( #3274 )
2018-05-31 19:04:50 +10:00
c936a52a9e
Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy ( #3307 )
2018-05-31 19:03:47 +10:00
219c2a322b
remove trailing whitespace
2018-05-30 19:42:19 +01:00
2e4be8bfd9
fix english and wrap comment
2018-05-30 19:24:12 +01:00
872cf43516
Merge pull request #3303 from NotAFile/py3-memoryview
...
use memoryview in py3
2018-05-30 12:51:42 +10:00
debff7ae09
Merge pull request #3281 from NotAFile/py3-six-isinstance
...
remaining isintance fixes
2018-05-30 12:44:46 +10:00
34b85df7f5
Update some comments and docstrings in SyncHandler
2018-05-29 22:31:18 +01:00
4a9cbdbc15
Exempt AS-registered users from doing gdpr
2018-05-29 19:54:32 +01:00
ab0ef31dc7
create users index on creation_ts
2018-05-29 17:51:08 +01:00
558f3d376a
create index in background
2018-05-29 17:47:55 +01:00
c379acd4fd
bump version
2018-05-29 17:47:28 +01:00
4b9d0cde97
add remaining memoryview changes
2018-05-29 17:42:43 +02:00
7873cde526
pep8
2018-05-29 17:35:55 +02:00
1afafb3497
use memoryview in py3
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-29 17:14:34 +02:00
ff1bc0a279
pep8
2018-05-29 02:32:15 +01:00
0a240ad36e
disable CPUMetrics if no /proc/self/stat
...
fixes build on macOS again
2018-05-29 02:23:30 +01:00
57ad76fa4a
fix up tests
2018-05-28 19:51:53 +10:00
3ef5cd74a6
update to more consistently use seconds in any metrics or logging
2018-05-28 19:39:27 +10:00
5c40ce3777
invalid syntax :(
2018-05-28 19:16:09 +10:00
357c74a50f
add comment about why unreg
2018-05-28 19:14:41 +10:00
a2eb5db4a0
update metrics to be in seconds
2018-05-28 19:10:27 +10:00
754826a830
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-28 18:57:23 +10:00
08ea5fe635
add link to thorough instruction how to configure consent
2018-05-25 23:19:55 +02:00
66bdae986f
Fix default for send_server_notice_to_guests
...
bool("False") == True...
2018-05-25 11:42:05 +01:00
ba1b163590
Avoid sending consent notice to guest users
...
we think it makes sense not to send the notices to guest users.
2018-05-25 11:36:43 +01:00
41921ac01b
Merge pull request #3287 from matrix-org/rav/allow_leaving_server_notices_room
...
Let users leave the server notice room after joining
2018-05-25 11:15:45 +01:00
757ed27258
Let users leave the server notice room after joining
...
They still can't reject invites, but we let them leave it.
2018-05-25 11:07:21 +01:00
4ee4450d66
fix recursion error
2018-05-24 21:44:10 +02:00
9c36c150e7
Merge pull request #3283 from NotAFile/py3-state
...
py3-ize state.py
2018-05-24 14:23:33 -05:00
cc1349c06a
Merge pull request #3279 from NotAFile/py3-more-iteritems
...
more six iteritems
2018-05-24 14:23:13 -05:00
0e61705661
py3-ize state.py
2018-05-24 20:59:00 +02:00
dd068ca979
remaining isintance fixes
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-24 20:55:08 +02:00
17a70cf6e9
Misc. py3 fixes
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-24 20:20:33 +02:00
6c16a4ec1b
more iteritems
2018-05-24 20:19:06 +02:00
7ea07c7305
Merge pull request #3278 from NotAFile/py3-storage-base
...
Py3 storage/_base.py
2018-05-24 13:08:09 -05:00
1f69693347
Merge pull request #3244 from NotAFile/py3-six-4
...
replace some iteritems with six
2018-05-24 13:04:07 -05:00
c4fb15a06c
Merge pull request #3246 from NotAFile/py3-repr-string
...
use repr, not str
2018-05-24 13:00:20 -05:00
36501068d8
Merge pull request #3247 from NotAFile/py3-misc
...
Misc Python3 fixes
2018-05-24 12:58:37 -05:00
2aff6eab6d
Merge pull request #3245 from NotAFile/batch-iter
...
Add batch_iter to utils
2018-05-24 12:54:12 -05:00
095292304f
Py3 storage/_base.py
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-24 18:24:12 +02:00
77a23e2e05
Merge remote-tracking branch 'origin/develop' into dbkr/unbind
2018-05-24 16:20:53 +01:00
ecc4b88bd1
Merge pull request #3277 from matrix-org/dbkr/remove_from_user_dir
...
Remove users from user directory on deactivate
2018-05-24 16:12:12 +01:00
46345187cc
Merge pull request #3243 from NotAFile/py3-six-3
...
Replace some more comparisons with six
2018-05-24 16:08:57 +01:00
037c6db85d
Merge branch 'master' into develop
2018-05-24 16:03:44 +01:00
7a1af504d7
Remove users from user directory on deactivate
2018-05-24 15:59:58 +01:00
f72d5a44d5
Merge pull request #3261 from matrix-org/erikj/pagination_fixes
...
Fix federation backfill bugs
2018-05-24 14:52:03 +01:00
68399fc4de
Merge pull request #3267 from matrix-org/erikj/iter_filter
...
Use iter* methods for _filter_events_for_server
2018-05-24 14:44:57 +01:00
91d95a1d8e
bump version
2018-05-24 14:05:07 +01:00
9700d15611
pep8
2018-05-24 11:23:15 +01:00
a21a41bad7
comment
2018-05-24 11:19:59 +01:00
b3bff53178
Unbind 3pids when they're deleted too
2018-05-24 11:08:05 +01:00
6abcb5d22d
Merge pull request #3273 from matrix-org/rav/server_notices_avatar_url
...
Allow overriding the server_notices user's avatar
2018-05-24 10:31:43 +01:00
389dac2c15
pepeightttt
2018-05-23 13:08:59 -05:00
472a5ec4e2
add back CPU metrics
2018-05-23 13:03:56 -05:00
e987079037
fixes
2018-05-23 13:03:51 -05:00
9bf4b2bda3
Allow overriding the server_notices user's avatar
...
probably should have done this in the first place, like @turt2live suggested.
2018-05-23 17:43:30 +01:00
a0b3946fe2
Merge branch 'release-v0.30.0' into rav/localpart_in_consent_uri
2018-05-23 16:06:03 +01:00
2f7008d4eb
Merge pull request #3271 from matrix-org/rav/consent_uri_in_messages
...
Support for putting %(consent_uri)s in messages
2018-05-23 16:04:30 +01:00
dda40fb55d
fix typo
2018-05-23 15:30:26 +01:00
3ff6f50eac
Use the localpart in the consent uri
...
... because it's shorter.
2018-05-23 15:28:23 +01:00
82191b08f6
Support for putting %(consent_uri)s in messages
...
Make it possible to put the URI in the error message and the server notice that
get sent by the server
2018-05-23 15:24:31 +01:00
2c7866d664
Hit the 3pid unbind endpoint on deactivation
2018-05-23 14:38:56 +01:00
321f02d263
Block attempts to send server notices to remote users
2018-05-23 14:30:47 +01:00
3601a240aa
bump version and changelog
2018-05-23 12:03:23 +01:00
5aaa3189d5
s/values/itervalues/
2018-05-23 10:13:05 +01:00
0a4bca4134
Use iter* methods for _filter_events_for_server
2018-05-23 10:04:23 +01:00
b6063631c3
more cleanup
2018-05-22 17:36:20 -05:00
53cc2cde1f
cleanup
2018-05-22 17:32:57 -05:00
071206304d
cleanup pep8 errors
2018-05-22 16:54:22 -05:00
85ba83eb51
fixes
2018-05-22 16:28:23 -05:00
228f1f584e
fix the test failures
2018-05-22 15:02:38 -05:00
e85b5a0ff7
Use iter* methods
2018-05-22 19:02:48 +01:00
586b66b197
Fix that states is a dict of dicts
2018-05-22 19:02:36 +01:00
a17e901f4d
Remove unused string formatting param
2018-05-22 18:24:32 +01:00
5494c1d71e
Don't support limitless pagination
...
The pagination storage function supported not specifiying a limit on the
number of events returned. This was triggered when using the search or
context API with a limit of zero, which the storage function took to
mean not being limited.
2018-05-22 18:15:21 +01:00
d8cb7225d2
daily user type phone home stats
2018-05-22 18:09:09 +01:00
ad2823ee27
fix synchrotron
2018-05-22 17:47:42 +01:00
08bfc48abf
custom error code for not leaving server notices room
2018-05-22 17:27:27 +01:00
0a078026ea
comment typo
2018-05-22 17:14:06 +01:00
8f5a688d42
cleanups, self-registration
2018-05-22 10:56:03 -05:00
a8990fa2ec
Merge remote-tracking branch 'origin/develop' into 3218-official-prom
2018-05-22 10:50:26 -05:00
cb2a2ad791
get_domains_from_state returns list of tuples
2018-05-22 16:23:39 +01:00
08a14b32ae
Merge pull request #3262 from matrix-org/rav/has_already_consented
...
Add a 'has_consented' template var to consent forms
2018-05-22 15:35:10 +01:00
7b36d06a69
Add a 'has_consented' template var to consent forms
...
fixes #3260
2018-05-22 14:58:34 +01:00
669400e22f
Enable auto-escaping for the consent templates
...
... to reduce the risk of somebody introducing an html injection attack...
2018-05-22 14:58:34 +01:00
b5b2d5d64b
Fix dependency on jinja2
...
Delay the import of ConsentResource, so that we can get away without jinja2 if
people don't have the consent resource enabled.
Fixes #3259
2018-05-22 14:03:45 +01:00
a5e2941aad
Reject attempts to send event before privacy consent is given
...
Returns an M_CONSENT_NOT_GIVEN error (cf
https://github.com/matrix-org/matrix-doc/issues/1252 ) if consent is not yet
given.
2018-05-22 12:00:47 +01:00
8aeb529262
Merge pull request #3236 from matrix-org/rav/consent_notice
...
Send users a server notice about consent
2018-05-22 11:56:09 +01:00
8810685df9
Stub out ServerNoticesSender on the workers
...
... and have the sync endpoints call it directly rather than obsure indirection
via PresenceHandler
2018-05-22 11:54:51 +01:00
d5dca9a04f
Move consent config parsing into ConsentConfig
...
turns out we need to reuse this, so it's better in the config class.
2018-05-22 11:54:51 +01:00
9ea219c514
Send users a server notice about consent
...
When a user first syncs, we will send them a server notice asking them to
consent to the privacy policy if they have not already done so.
2018-05-22 11:54:51 +01:00
d14d7b8fdc
Rename 'version' param on user consent config
...
we're going to use it for the version we require too.
2018-05-22 11:54:51 +01:00
7948ecf234
Comment
2018-05-22 11:39:43 +01:00
020377a550
Fix logcontext resource usage tracking
2018-05-22 11:16:07 +01:00
13a8dfba0d
Merge pull request #3252 from matrix-org/erikj/in_flight_requests
...
Add in flight request metrics
2018-05-22 09:54:30 +01:00
c435b0b441
Don't store context
2018-05-22 09:34:26 +01:00
fb2806b186
Move in_flight_requests_count to be a callback metric
2018-05-22 09:31:53 +01:00
fcc525b0b7
rest of the changes
2018-05-21 19:48:57 -05:00
df9f72d9e5
replacing portions
2018-05-21 19:47:37 -05:00
c60e0d5e02
don't need the resource portion
2018-05-21 17:03:20 -05:00
02c1d29133
look at the Prometheus metrics instead
2018-05-21 17:02:20 -05:00
f258deffcb
remove old metrics libs
2018-05-21 17:01:15 -05:00
413482f578
Merge pull request #3255 from matrix-org/rav/fix_transactions
...
Stop the transaction cache caching failures
2018-05-21 17:58:08 +01:00
6e1cb54a05
Fix logcontext leak in HttpTransactionCache
...
ONE DAY I WILL PURGE THE WORLD OF THIS EVIL
2018-05-21 16:58:20 +01:00
6d6e7288fe
Stop the transaction cache caching failures
...
The transaction cache has some code which tries to stop it caching failures,
but if the callback function failed straight away, then things would happen
backwards and we'd end up with the failure stuck in the cache.
2018-05-21 16:49:59 +01:00
dfa70adc33
Add in flight request metrics
...
This tracks CPU and DB usage while requests are in flight, rather than
when we write the response.
2018-05-21 16:23:06 +01:00
933bf2dd35
replace some iteritems with six
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:59:26 +02:00
d9fe2b2d9d
Replace some more comparisons with six
...
plus a bonus b"" string I missed last time
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:56:31 +02:00
45b55e23d3
Add batch_iter to utils
...
There's a frequent idiom I noticed where an iterable is split up into a
number of chunks/batches. Unfortunately that method does not work with
iterators like dict.keys() in python3. This implementation works with
iterators.
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:48:30 +02:00
dcc235b47d
use stand-in value if maxint is not available
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:35:44 +02:00
73cbdef5f7
fix py3 intern and remove unnecessary py3 encode
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:35:31 +02:00
aafb0f6b0d
py3-ize url preview
2018-05-19 17:35:20 +02:00
b932b4ea25
use repr, not str
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-05-19 17:28:42 +02:00
644aac5f73
Tighter filtering for user_daily_visits
2018-05-18 17:10:35 +01:00
ef466b3a13
fix psql compatability bug
2018-05-18 15:51:21 +01:00
2725223f08
Merge branch 'master' into develop
2018-05-18 14:07:50 +01:00
6d9dc67139
Merge pull request #3232 from matrix-org/rav/server_notices_room
...
Infrastructure for a server notices room
2018-05-18 11:28:04 +01:00
ed3125b0a1
Merge pull request #3235 from matrix-org/rav/fix_receipts_deferred
...
Fix error in handling receipts
2018-05-18 11:23:11 +01:00
67af392712
Merge pull request #3233 from matrix-org/rav/remove_dead_code
...
Remove unused `update_external_syncs`
2018-05-18 11:22:43 +01:00
011e1f4010
Better docstrings
2018-05-18 11:22:12 +01:00
26305788fe
Make sure we reject attempts to invite the notices user
2018-05-18 11:18:39 +01:00
d10707c810
Replace inline docstrings with "Attributes" in class docstring
2018-05-18 11:00:55 +01:00
fa30ac38cc
Merge pull request #3221 from matrix-org/erikj/purge_token
...
Make purge_history operate on tokens
2018-05-18 10:35:23 +01:00
8b1c856d81
Fix error in handling receipts
...
Fixes an error which has been happening ever since #2158 (v0.21.0-rc1):
> TypeError: argument of type 'ObservableDeferred' is not iterable
fixes #3234
2018-05-18 09:15:35 +01:00
6958459b50
bump version, change log
2018-05-17 21:35:07 +01:00
David Baker