95481e7ba7
Merge pull request #571 from matrix-org/daniel/asids
...
Mark AS users with their AS's ID
2016-03-03 13:56:28 +00:00
60a0f81c7a
Add a /replication API for extracting the updates that happened on
...
synapse
This is necessary for replicating the data in synapse to be visible to a
separate service because presence and typing notifications aren't stored
in a database so won't be visible to another process.
This API can be used to either get the raw data by requesting the tables
themselves or to just receive notifications for updates by following the
streams meta-stream.
Returns updates for each table requested a JSON array of arrays with a
row for each row in the table.
Each table is prefixed by a header row with the: name of the table,
current stream_id position for the table, number of rows, number of
columns and the names of the columns.
This is followed by the rows that have been added to the server since
the requester last asked.
The API has a timeout and is hooked up to the notifier so that a slave
can long poll for updates.
2016-03-01 14:49:41 +00:00
763360594d
Mark AS users with their AS's ID
2016-02-11 17:26:42 +00:00
2c1fbea531
Fix up logcontexts
2016-02-08 14:26:45 +00:00
737c4223ef
Host /media/r0 as well as /media/v1
2016-02-05 10:47:46 +00:00
6a9f1209df
Error if macaroon key is missing from config
...
Currently we store all access tokens in the DB, and fall back to that
check if we can't validate the macaroon, so our fallback works here, but
for guests, their macaroons don't get persisted, so we don't get to
find them in the database. Each restart, we generate a new ephemeral
key, so guests lose access after each server restart.
I tried to fix up the config stuff to be less insane, but gave up, so
instead I bolt on yet another piece of custom one-off insanity.
Also, add some basic tests for config generation and loading.
2016-02-05 01:58:23 +00:00
f8aae79a72
Simplify get_rooms
2016-02-03 13:24:35 +00:00
d83d004ccd
Fix flake8 warnings for new flake8
2016-02-02 17:18:50 +00:00
167d1df699
Merge pull request #534 from matrix-org/erikj/setup
...
Add a Homeserver.setup method
2016-01-28 14:45:36 +00:00
7ed2bbeb11
Clean up a bit. Add comment
2016-01-28 14:32:05 +00:00
a6477d5933
Remove chdir
2016-01-28 09:19:55 +00:00
fa90c180ee
Merge branch 'develop' of github.com:matrix-org/synapse into erikj/setup
2016-01-27 17:37:33 +00:00
9fda8b5193
Don't turn on profiling
2016-01-26 18:27:23 +00:00
87f9477b10
Add a Homeserver.setup method.
...
This is for setting up dependencies that require work on startup. This
is useful for the DataStore that wants to read a bunch from the database
before initiliazing.
2016-01-26 15:51:06 +00:00
9959d9ece8
Remove redundated BaseHomeServer
2016-01-26 13:52:29 +00:00
44b4fc5f50
Use compiled regex
2016-01-13 11:47:32 +00:00
8f8b884430
Don't log urlencoded access_tokens
2016-01-08 17:48:08 +00:00
33bef689c1
Turn on core dumps
2016-01-07 15:34:30 +00:00
5727922106
Merge pull request #473 from matrix-org/erikj/ssh_manhole
...
Change manhole to use ssh
2016-01-07 14:36:16 +00:00
c9ae1d1ee5
Change manhole to use ssh
2016-01-07 13:59:02 +00:00
6c28ac260c
copyrights
2016-01-07 04:26:29 +00:00
11a974da21
Add /_matrix/versions to report supported versions
2016-01-06 18:08:52 +00:00
bf32922e5a
Log when starting stats reporting
2016-01-06 14:13:34 +00:00
797691f908
Log on stats scheduling
2016-01-06 14:04:27 +00:00
7a8ba4c9a0
Actually host r0 and unstable prefixes
2015-12-08 15:26:52 +00:00
6a5ff5f223
Track the time spent in the database per request.
...
and track the number of transactions that request started.
2015-12-07 17:56:11 +00:00
d6059bdd2a
Fix warnings
2015-12-04 11:34:23 +00:00
a412b9a465
Run the background updates when starting synapse.
2015-11-10 15:50:58 +00:00
fa1cf5ef34
Move static folder into synapse
...
This is because otherwise it won't get picked up by python packaging.
This also fixes the problem where the "static" folder was found if
synapse wasn't started from that directory.
2015-10-26 15:37:44 +00:00
9020860479
Only turn on the twisted deferred debugging if full_twisted_stacktraces is set in the config
2015-10-13 17:50:44 +01:00
7639c3d9e5
Bounce all deferreds through the reactor to make debugging easier.
...
If all deferreds wait a reactor tick before resolving then there is
always a chance to add an errback to the deferred so that stacktraces
get reported, rather than being discarded.
2015-10-13 17:13:04 +01:00
17c80c8a3d
rename schema_prepare to prepare_database
2015-10-13 13:56:22 +01:00
ec398af41c
Expose error more nicely
2015-10-13 11:43:43 +01:00
7a5818ed81
Note that GzipFile was removed in comment that referenced it
2015-09-22 18:27:22 +01:00
184ba0968a
synapse/app/homeserver.py:GzipFile was unused
2015-09-22 18:25:30 +01:00
b6e0303c83
Catch stats-reporting errors
2015-09-22 13:34:29 +01:00
7213588083
Implement configurable stats reporting
...
SYN-287
This requires that HS owners either opt in or out of stats reporting.
When --generate-config is passed, --report-stats must be specified
If an already-generated config is used, and doesn't have the
report_stats key, it is requested to be set.
2015-09-22 12:57:40 +01:00
0b5c9adeb5
Merge pull request #267 from matrix-org/markjh/missing_requirements
...
Print an example "pip install" line for a missing requirement
2015-09-18 18:52:08 +01:00
2c746382e0
Merge branch 'daniel/insecureclient' into develop
2015-09-09 14:27:30 +01:00
6485f03d91
Fix random formatting
2015-09-09 13:05:00 +01:00
81a93ddcc8
Allow configuration to ignore invalid SSL certs
...
This will be useful for sytest, and sytest only, hence the aggressive
config key name.
2015-09-09 12:02:07 +01:00
73e4ad4b8b
Merge branch 'master' into develop
...
Conflicts:
setup.py
2015-09-07 15:06:46 +01:00
1002bbd732
Change log level to info
2015-09-03 09:51:01 +01:00
3cdfd37d95
Print an example "pip install" line for a missing requirement
2015-09-01 16:47:26 +01:00
d71af2ee12
don't log the whole DB config (including postgres password...)
2015-08-29 22:23:21 +01:00
d33f31d741
Print the correct pip install line when failing due to lack of matrix-angular-sdk
2015-08-25 15:33:23 +01:00
37403ab06c
Update the log message
2015-08-25 14:19:09 +01:00
d9088c923f
Remove dependency on matrix-angular-sdk
2015-08-25 13:34:50 +01:00
86cef6a91b
Allow specifying a directory to host a web client from
2015-08-25 12:01:23 +01:00
efe60d5e8c
Only print the pidfile path on startup if requested by a commandline flag
2015-08-07 16:36:42 +01:00
0e58d19163
Merge pull request #187 from matrix-org/erikj/sanitize_logging
...
Sanitize logging
2015-06-19 11:35:59 +01:00
eb928c9f52
Add site_tag to logger
2015-06-19 10:16:48 +01:00
9e5a353663
Make upload dir a configurable path.
...
Fixes SYN-425.
Signed-off-by: Eric Myhre <hash@exultant.us>
2015-06-18 23:38:20 -05:00
aaa749d366
Disable twisted access logging. Move access logging to SynapseRequest object
2015-06-15 18:18:05 +01:00
cee69441d3
Log more when we have processed the request
2015-06-15 17:11:44 +01:00
b5209c5744
Create SynapseRequest that overrides __repr__ to not print access_token
2015-06-15 16:37:04 +01:00
a005b7269a
Add backwards compat support for metrics, manhole and webclient config options
2015-06-12 17:44:23 +01:00
942e39e87c
PEP8
2015-06-12 17:13:54 +01:00
9c5fc81c2d
Correctly handle x_forwaded listener option
2015-06-12 17:13:23 +01:00
fd2c07bfed
Use config.listeners
2015-06-12 15:33:07 +01:00
c42ed47660
Fix up create_resource_tree
2015-06-12 11:52:52 +01:00
522f285f9b
Add config option to disable compression of http responses
2015-06-01 13:36:30 +01:00
a7b65bdedf
Add config option to turn off freezing events. Use new encode_json api and ujson.loads
2015-05-29 12:17:33 +01:00
1ce1509989
s/metric_interface/metric_bind_host/
2015-05-22 14:51:22 +01:00
8bb85c8c5a
Update log line
2015-05-22 14:48:06 +01:00
59a0682f3e
Enable changing the interface the metrics listener binds to
2015-05-22 13:13:07 +01:00
19505e0392
Disable GZip encoding on static file resources as per comment
2015-05-19 13:20:25 +01:00
677be13ffc
Revert accidental commit
2015-05-19 13:12:28 +01:00
2b7120e233
SYN-383: Handle the fact the server might not have signed things
2015-05-19 12:49:38 +01:00
6e1ad283cf
Support gzip encoding for client, client v2 and web client resources (SYN-176).
2015-05-14 16:39:19 +01:00
4a7a4a5b6c
Optional profiling using cProfile
2015-05-06 17:08:00 +01:00
2d4d2bbae4
Merge branch 'develop' into markjh/config_cleanup
...
Conflicts:
synapse/config/captcha.py
2015-04-30 16:54:55 +01:00
054aa0d58c
Do access log using python's logging stuff, just under a separate logger name
2015-04-30 16:21:38 +01:00
d89a9f7283
Add an access_log
...
SYN-161 #resolve
2015-04-30 13:58:13 +01:00
6b69ddd17a
remove duplicate parse_size method
2015-04-30 04:26:29 +01:00
d624e2a638
Manually generate the default config yaml, remove most of the commandline arguments for synapse anticipating that people will use the yaml instead. Simpify implementing config options by not requiring the classes to hit the super class
2015-04-30 04:24:44 +01:00
e7a6edb0ee
Revert previous
2015-04-29 18:37:30 +01:00
c27d6ad6b5
Only start pushers when synapse has fully started
2015-04-29 18:25:24 +01:00
f2d90d5c02
Fix whitespace
2015-04-29 14:53:23 +01:00
845b0b2c97
Check requirements before doing anything else when running the homeserver
2015-04-29 14:52:42 +01:00
64991b0c8b
Merge pull request #129 from matrix-org/key_distribution
...
Key distribution v2
2015-04-29 13:34:38 +01:00
4ad8b45155
Merge branch 'develop' into key_distribution
...
Conflicts:
synapse/config/homeserver.py
2015-04-29 13:15:14 +01:00
cd0864121b
Make postgres database error slightly more helpful
2015-04-29 12:12:25 +01:00
204132a998
Check that postgres database has correct charset set
2015-04-29 11:42:28 +01:00
073b891ec1
Remove unused imports
2015-04-28 13:44:23 +01:00
327ca883ec
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-28 13:39:42 +01:00
42b7139dec
Remove unused import
2015-04-27 15:59:56 +01:00
1ef66cc3bd
Move database configuration into config module
2015-04-27 15:57:43 +01:00
416a3e6c4f
Ensure check_same_thread is enabled for sqlite3
2015-04-27 15:44:30 +01:00
df75914791
pep8
2015-04-27 11:48:33 +01:00
b02e1006b9
Run database check before daemonizing, at the cost of database hygiene.
2015-04-27 11:46:00 +01:00
f8152f2708
rename db method to be more informative
2015-04-27 10:16:26 +01:00
a7b51f4539
Check users in our table aren't on a different domain to the one we're configured with to try & fix SYN-266
2015-04-24 18:11:21 +01:00
ffad75bd62
Remove mysql/maria support
2015-04-15 17:00:50 +01:00
6606f7c659
Merge branch 'develop' of github.com:matrix-org/synapse into postgres
2015-04-15 10:27:20 +01:00
d488463fa3
Add a version 2 of the key server api
2015-04-14 16:04:52 +01:00
58d8339966
Add support for postgres instead of mysql. Change sql accourdingly. blob + varbinary -> bytea. No support for UNSIGNED or CREATE INDEX IF NOT EXISTS.
2015-04-14 13:53:20 +01:00
3cbc286d06
Move server key api into rest/key/v1
2015-04-14 13:28:11 +01:00
c5365dee56
Use case sensitive collations
2015-04-10 18:46:33 +01:00
4fe95094d1
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-04-07 18:05:39 +01:00
0bfa78b39b
PEP8
2015-04-07 12:16:05 +01:00
d0e444a648
Explicitly name the __main__ module logger
2015-04-07 12:04:02 +01:00
779f7b0f44
Fix unicode support
2015-04-02 10:06:22 +01:00
e24c32e6f3
Fix SQLite support
2015-04-01 15:09:51 +01:00
9236136f3a
Make work in both Maria and SQLite. Fix tests
2015-04-01 14:12:33 +01:00
3470cb36a8
Pyflakes
2015-03-31 13:03:31 +01:00
e7887e37a8
Remove appservice REST servlets
2015-03-31 09:32:40 +01:00
0e8f5095c7
Fix unicode database support
2015-03-25 17:15:20 +00:00
f6583796fe
Merge branch 'develop' of github.com:matrix-org/synapse into mysql
2015-03-20 16:31:48 +00:00
455579ca90
Make database selection configurable
2015-03-20 10:55:55 +00:00
7564dac8cb
Wire up the webclient option
...
It existed but was hardcoded to True.
Give it an underscore for consistency.
Also don't pull in syweb unless we're actually using the web client.
2015-03-17 12:45:37 +01:00
c8032aec17
actually uphold the bind_host parameter. in theory should make ipv6 binds work like bind_host: 'fe80::1%lo0'
2015-03-14 00:12:20 +00:00
b98b4c135d
Option to serve metrics from their own localhost-only TCP port instead of muxed on the main listener
2015-03-12 16:24:51 +00:00
a2cdd11d4a
Fold the slightly-odd bind_port/secure_port/etc.. logic into SynapseHomeServer.start_listening()
2015-03-12 16:24:51 +00:00
e0214a263b
Build MetricsResource as a specific HomeServer dependency
2015-03-12 16:24:51 +00:00
e07fc62833
A trivial 'hello world'-style resource on /_synapse/metrics, with optional commandline flag
2015-03-12 16:24:50 +00:00
211c14c391
No need to explicitly pass 'web_client' in to create_resource_tree as it can be found via config
2015-03-12 16:24:50 +00:00
045afd6b61
in_thread takes no arguments
2015-03-10 10:19:03 +00:00
c37eceeb9e
Split out the 'run' from 'setup'
2015-03-10 09:58:33 +00:00
b8a6692657
Add documentation. When starting via twistd respect soft_file_limit config option.
2015-03-10 09:39:42 +00:00
f31e65ca8b
Merge branch 'develop' of github.com:matrix-org/synapse into erikj-perf
2015-03-09 13:29:41 +00:00
c54773473f
Merge branch 'master' of github.com:matrix-org/synapse into develop
2015-03-06 14:23:41 +00:00
82b34e813d
SYN-67: Finish up implementing new database schema management
2015-03-04 12:04:19 +00:00
3fce185c77
Merge pull request #83 from matrix-org/nofile_limit_config
...
Add config option to set the soft fd limit on start
2015-03-02 13:52:16 +00:00
28d8614f48
Trailing comma
2015-02-23 17:36:37 +00:00
0696dfd94b
Actually treat this as static content, not random Resources.
2015-02-23 15:35:09 +00:00
22399d3d8f
Add RegisterFallbackResource to /_matrix/static/client/register
...
Try to keep both forms of registration logic (native/fallback) close
together for sanity.
2015-02-23 15:14:56 +00:00
1f29fafc95
Don't exit if we can't work out if we're running in a git repo
2015-02-21 13:44:46 +00:00
7c56210f20
By default set soft limit to hard limit
2015-02-20 16:09:44 +00:00
939273c4b0
Rename resource variable so as to not shadow module import
2015-02-19 11:53:13 +00:00
c3eb7dd9c5
Add config option to set the soft fd limit on start
2015-02-19 11:50:49 +00:00
6311ae8968
Conform to header spec take two
2015-02-18 17:34:26 +00:00
b6771037a6
Make version_string conform to User-Agent and Server spec
2015-02-18 17:31:50 +00:00
1df8bad63e
pyflakes
2015-02-18 16:54:25 +00:00
aa577df064
When computing git version run git commands in same dir as source files
2015-02-18 16:52:04 +00:00
d122e215ff
Generate a version string that includes git details if run from git checkout
2015-02-18 16:52:04 +00:00
2635d4e634
Merge branch 'develop' of github.com:matrix-org/synapse into develop
2015-02-18 10:29:54 +00:00
ec847059f3
Rename _fail_fetch_pdu_cache to _get_pdu_cache
2015-02-18 10:14:10 +00:00
1177245e86
Merge branch 'hotfixes-v0.7.0g' of github.com:matrix-org/synapse into develop
2015-02-17 17:30:11 +00:00
8b919c00f3
Start the get_pdu cache
2015-02-17 15:44:01 +00:00
ea1d6c16cd
Don't write bytecode
2015-02-17 10:54:06 +00:00
0d872f5aa6
Merge pull request #50 from matrix-org/application-services
...
Application Services
2015-02-13 15:06:14 +00:00
183b3d4e47
Prepare the database whenever a connection is opened from the db_pool so that in-memory databases will work
2015-02-13 14:38:24 +00:00
5758dafb4e
Merge branch 'develop' of github.com:matrix-org/synapse into state-chache
2015-02-11 10:35:13 +00:00
9978c5c103
Merge branch 'develop' into application-services
2015-02-11 10:03:24 +00:00
84a769cdb7
Fix code-style
2015-02-10 17:58:36 +00:00
c86ebe7673
Merge branch 'develop' of github.com:matrix-org/synapse into state-chache
...
Conflicts:
synapse/app/homeserver.py
synapse/state.py
2015-02-10 11:04:37 +00:00
53557fc532
Merge branch 'develop' into application-services
2015-02-09 15:20:56 +00:00
75656712e3
Time how long we're spending on the database thread
2015-02-09 14:22:52 +00:00
24cc6979fb
Log when we receive a request, when we send a response and how long it took to process it.
2015-02-09 13:46:22 +00:00
5bf318e9a6
Bug fixes.
2015-02-06 16:52:22 +00:00
c059c9fea5
Merge branch 'develop' into application-services
...
Conflicts:
synapse/handlers/__init__.py
synapse/storage/__init__.py
2015-02-02 15:57:59 +00:00
c291a4d522
Merge branch 'develop' into pushers
...
Conflicts:
synapse/handlers/events.py
synapse/server.py
2015-01-28 12:51:05 +00:00
fa8e6ff900
Add stub application services REST API.
2015-01-27 14:01:51 +00:00
7b814d3f7f
Add client v2_alpha resource to synapse server resource tree
2015-01-23 18:55:19 +00:00
7256def8e4
Merge rest servlets into the client json resource object
2015-01-23 10:37:38 +00:00
5c6189ea3e
Merge branch 'develop' into pushers
...
Conflicts:
synapse/rest/__init__.py
2015-01-22 17:46:16 +00:00
97c68c508d
Move rest APIs back under the rest directory
2015-01-22 16:10:07 +00:00
7f058c5ff7
Merge branch 'develop' of github.com:matrix-org/synapse into erikj-perf
...
Conflicts:
synapse/app/homeserver.py
2015-01-22 13:35:34 +00:00
42529cbced
Fix pyflakes errors
2015-01-19 15:33:04 +00:00
00e9c08609
Fix syntax
2015-01-19 15:30:48 +00:00
3e85e52b3f
Allow ':memory:' as the database path for sqlite3
2015-01-19 15:26:19 +00:00
c06a9063e1
Merge branch 'develop' into pushers
2015-01-13 13:15:51 +00:00
63403aa7a5
Check the existance and versions of necessary modules when starting synapse, log which modules are used
2015-01-08 17:08:57 +00:00
42507b0011
Log server version on startup
2015-01-07 17:25:28 +00:00
82be4457de
Add twisted Service interface
2015-01-07 13:46:37 +00:00
5cf45c4319
Merge branch 'master' into develop
2015-01-06 19:48:53 +00:00
adb04b1e57
Update copyright notices
2015-01-06 13:21:39 +00:00
407c299828
improve error msg
2014-12-24 17:50:42 +00:00
b56730bb6e
Merge branch 'develop' into pushers
...
Conflicts:
synapse/api/errors.py
synapse/server.py
synapse/storage/__init__.py
2014-12-18 15:15:22 +00:00
2e44714214
Make failure to run appropraite upgrade scripts more helpful.
2014-12-16 14:20:32 +00:00
5da65085d1
Get uploads working with new media repo
2014-12-02 19:51:47 +00:00
58f82e2e54
Merge branch 'develop' into pushers
2014-11-20 18:25:31 +00:00
23465a30b6
Merge branch 'develop' into pushers
2014-11-20 18:17:46 +00:00
f1c7f8e813
Merge branch 'develop' into http_client_refactor
2014-11-20 17:49:48 +00:00
e377d33652
Separate out the matrix http client completely because just about all of its code it now separate from the simple case we need for standard HTTP(S)
2014-11-20 17:41:56 +00:00
db9ce032a4
Fix pep8 codestyle warnings
2014-11-20 17:26:36 +00:00
dfdda2c871
Use module loggers rather than the root logger. Exceptions caused by bad clients shouldn't cause ERROR level logging. Fix sql logging to use 'repr' rather than 'str'
2014-11-20 17:10:37 +00:00
ebf6c08a47
Merge branch 'http_client_refactor' into pushers
2014-11-20 14:01:41 +00:00
dc60eee50e
Refactor the HTTP clients a little.
2014-11-20 13:53:34 +00:00
74c3879760
Start creating a module to do generic notifications (just prints them to stdout currently!)
2014-11-19 18:20:59 +00:00
c6ea29d916
Revert accidental commit of bad file
2014-11-18 15:57:00 +00:00
95614e5220
Fix auth to correctly handle initial creation of rooms
2014-11-18 15:36:41 +00:00
e903c941cb
Merge branch 'develop' into request_logging
...
Conflicts:
setup.py
synapse/storage/_base.py
synapse/util/async.py
2014-11-14 11:16:50 +00:00
3cb678f84c
Merge branch 'develop' of github.com:matrix-org/synapse into federation_authorization
2014-11-07 10:55:28 +00:00
dca3ba2f77
Determine webclient path by the python module it is contained in
2014-11-04 16:19:16 +00:00
af7ae048f8
Add option to not bind to HTTPS port. This is useful if running behind an ssl load balancer
2014-11-03 15:06:40 +00:00
7a756e5d9d
Remove unused 'context' variables to appease pyflakes
2014-10-30 11:15:39 +00:00
b29517bd01
Add a request-id to each log line
2014-10-30 01:21:33 +00:00
984e207b59
Merge branch develop into server2server_signing
...
Conflicts:
synapse/app/homeserver.py
2014-10-13 10:58:50 +01:00
4f11518934
Split PlainHttpClient into separate clients for talking to Identity servers and talking to Capatcha servers
2014-10-02 14:03:26 +01:00
e3117a2a23
Add a _matrix/key/v1 resource with the verification keys of the local server
2014-09-23 18:43:34 +01:00
55397f6347
prepare_database() on db_conn, not plain name, so we can pass in the connection from outside
2014-09-10 16:23:58 +01:00
2faffc52ee
Make sure not to open our TCP ports until /after/ the DB is nicely prepared ready for use
2014-09-10 16:16:24 +01:00
6c1f0055dc
No need for a tiny run() function any more, just use reactor.run() directly
2014-09-10 16:07:44 +01:00
ce55a8cc4b
Move database preparing code out of homserver.py into storage where it belongs
2014-09-10 15:42:15 +01:00
9dd4570b68
Generate m.room.aliases event when the HS creates a room alias
2014-09-05 21:35:56 +01:00
8a7c1d6a00
fix the copyright holder from matrix.org to OpenMarket Ltd, as matrix.org hasn't been incorporated in time for launch.
2014-09-03 17:31:57 +01:00
756e171ad0
Store SQL DDL deltas as well; attempt to upgrade the database on startup if it's too old
2014-09-03 14:14:39 +01:00
b50ea730b1
Merge branch 'develop' of github.com:matrix-org/synapse into develop
...
Conflicts:
synapse/http/server.py
synapse/http/content_repository.py
2014-09-03 12:01:14 +01:00
Daniel Wagner-Hall