element-web/test/end-to-end-tests
David Baker b8013fc52a
Add a Cypress Test 🌲 (#8295)
* A first, maybe working cypress test

Plus cypress plugins to manage synapses in docker containers

* Fix yaml

* This file is important

* try & find where it's put the artifact

* Download artifact to a directory

* pics or it didn't happen

* Add conditional, otherwise no artifacts on failure...

* Try increasing timeout

also actually give the test a name

* Try in chrome

* Get docker logs to see why it's failing

also document the chrome setting

* Try changing mode on homeserver.yaml

* debug

* More debugging

* more file permissions debugging

* ARGH

* more debug

* sigh

* Eugh, that's not how arguments work

* Add the option to really allow open registration

and remove debug logging / comment fixes

* failure to yaml

* Upload docker logs as artifacts

and temporarily remove contional to test

* Put the conditional back

* Upgrade types in end to end tests

to be compatible with fs-extra types

* Try reducing timeout a bit

also make password more... sensible

* Hex is not octal

* Remove file mode

Seems to be unnecessary since the signing key is perfectly fine

* Give the log files extensions

* Rename workflow file now it also does tests

* Add cypress scripts

* copyright headers

* Use ? operator

Co-authored-by: Travis Ralston <travisr@matrix.org>

* Use develop synapse image

* Tidy up any remaining synapses after each spec run

Also:
 * Move the synapseStart / synapseStop functions out to the top level
   so they can be reused
 * Add a tsconfig file
 * Give the containers names

* Don't upload video on test pass

We don't upload it anyway so tell cypress not to so it can not
bother encoding them

* Enable linting on cypress files

and fix existing lint errors

* Type check cypress files

and make it pass the type checks, specifically:
 * Upgrade sinon fake timers to a version that has the right types
 * Set module resolution
 * Type check cypress files separately

* Rename workflow file again

Probably better to just call it an element web build

* Don't plus + characters in container name

* Fix yaml

* Stream logs to file

* Add note to end to end tester to sya what's been ported

* Put docker rm in finally block

Co-authored-by: Travis Ralston <travisr@matrix.org>
2022-04-14 10:41:58 +01:00
..
element Revert "Pin end-to-end test Python dependencies" (#8106) 2022-03-21 22:30:36 +00:00
src Add a Cypress Test 🌲 (#8295) 2022-04-14 10:41:58 +01:00
synapse Fix end-to-end tests for recent Synapse features (#8159) 2022-03-25 19:45:52 +00:00
.gitignore End to end tests for threads (#8267) 2022-04-12 17:46:08 -06:00
README.md Triple the speed of E2E tests and stop them exploding if a circular datastructure is logged (#8095) 2022-03-21 10:26:26 +00:00
TODO.md move everything to subfolder to merge into react-sdk 2019-10-09 16:52:48 +02:00
Windows.md Remove unused step from e2e test docs for Windows (#8148) 2022-03-25 09:13:20 -06:00
has-custom-app.js Fix spacing errors (#7484) 2022-01-10 12:57:20 +00:00
install.sh Do not generate a lockfile when running in CI 2021-07-07 16:11:47 +02:00
package.json Add an e2e test to check the app reloads when a new version is available (#7956) 2022-03-03 15:44:34 +00:00
pick-synapse-log-file.js Fix end-to-end tests for recent Synapse features (#8159) 2022-03-25 19:45:52 +00:00
run.sh Fix end-to-end tests for recent Synapse features (#8159) 2022-03-25 19:45:52 +00:00
start.ts Improve end-to-end test logging by rendering JSHandles (#7959) 2022-03-03 07:59:29 -07:00
tsconfig.json Step 2: Remove the decorator 2022-03-28 14:02:31 -06:00
yarn.lock Add a Cypress Test 🌲 (#8295) 2022-04-14 10:41:58 +01:00

README.md

Matrix React SDK End-to-End tests

This directory contains tests for matrix-react-sdk. The tests fire up a headless Chrome and simulate user interaction (end-to-end). Note that end-to-end has little to do with the end-to-end encryption Matrix supports, just that we test the full stack, going from user interaction to expected DOM in the browser.

Setup

Run ./install.sh. This will:

  • install Synapse, fetches the develop branch at the moment. If anything fails here, please refer to the Synapse README to see if you're missing one of the prerequisites.
  • install Element Web, this fetches the develop branch at the moment.
  • install dependencies (will download copy of Chrome)

Running the tests

Run tests with ./run.sh.

Debug tests locally.

./run.sh will run the tests against the Element copy present in element/element-web served by a static Python HTTP server. You can symlink your element-web develop copy here but that doesn't work well with Webpack recompiling. You can run the test runner directly and specify parameters to get more insight into a failure or run the tests against your local Webpack server.

./synapse/stop.sh && \
./synapse/start.sh && \
node start.js <parameters>

It's important to always stop and start Synapse before each run of the tests to clear the in-memory SQLite database it uses, as the tests assume a blank slate.

start.js accepts these parameters (and more, see node start.js --help) that can help running the tests locally:

  • --app-url <url> don't use the Element Web copy and static server provided by the tests, but use a running server like the Webpack watch server to run the tests against.
  • --slow-mo type at a human speed, useful with --windowed.
  • --throttle-cpu <factor> throttle cpu in the browser by the given factor. Useful to reproduce failures because of insufficient timeouts happening on the slower CI server.
  • --windowed run the tests in an actual browser window Try to limit interacting with the windows while the tests are running. Hovering over the window tends to fail the tests, dragging the title bar should be fine though.
  • --dev-tools open the devtools in the browser window, only applies if --windowed is set as well.

For god level debug (e.g. for debugging slow tests):

env DEBUG="puppeteer:*" ./test/end-to-end-tests/run.sh --app-url http://localhost:8080 --log-directory pwd/logs --dev-tools --windowed 2>&1 | cat

(piping everything through cat means you get proper timestamps on the debugging, and the chromiums hang around at the end)

Developer Guide

Please follow the standard Matrix contributor's guide: https://github.com/matrix-org/synapse/tree/master/CONTRIBUTING.rst

Please follow the Matrix JS/React code style as per: https://github.com/matrix-org/matrix-react-sdk/blob/master/code_style.md