Run Playwright tests against Dendrite & Pinecone periodically (#28888)
* Switch to TestContainers for manging services in Playwright Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Flip fixture dependency order Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Remove mas dep Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Update matrix-authentication-service in Playwright tests Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * delint Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Fix SMTP port Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Comments Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Strip ansi from playwright logs to make them more readable Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Actually do the update Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Remove access to homeserver.config.baseUrl field in favour of homeserver.baseUrl Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Use sane default_server_config and specify server.invalid in the specific tests which demand it Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Fix mas run Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * break cycle Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * typo Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * prettier Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Wire up basics of dendriteHomeserver Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Run Playwright tests against Dendrite & Pinecone periodically Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Fix playwright-image-updates.yaml workflow Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Add `X-Run-All-Tests` label for running all tests Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Ignore failing tests in stale-screenshot-reporter.ts to avoid confusing errors Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Specify Synapse ui_auth.session_timeout only on tests which require it As Dendrite lacks this configuration option Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * React to MatrixEvent sender/target sentinels being updated for rendering state events Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Remove test code Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * React to sentinel changes in EventListSummary Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Docs Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Avoid reusing user1234 Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Fix stale-screenshot-reporter.ts Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Clean up public rooms between tests on reused homeserver Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Deflake spotlight when homeserver is reused Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Deflake more tests using existing username Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Clean mailhog between tests Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Fix more flakes Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Fix missing _request Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Fix playwright flaky tests Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Wipe mailhog between test runs Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Delint Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * delint Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Deflake more tests Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Fix flaky tests Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Fix flaky tests Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Fix mas config Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Fix another flaky test Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Fix playwright flakes due to floating promises Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Fix flaky playwright tests Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Iterate Signed-off-by: Michael Telatynski <7t3chguy@gmail.com> * Update services.ts --------- Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
This commit is contained in:
Michael Telatynski
committed by
GitHub
GitHub
parent
bb41616d5f
commit
dbce48b23d
@@ -16,6 +16,8 @@ import { StartedMatrixAuthenticationServiceContainer } from "./testcontainers/ma
|
||||
import { HomeserverContainer, StartedHomeserverContainer } from "./testcontainers/HomeserverContainer.ts";
|
||||
import { MailhogContainer, StartedMailhogContainer } from "./testcontainers/mailhog.ts";
|
||||
import { OAuthServer } from "./plugins/oauth_server";
|
||||
import { DendriteContainer, PineconeContainer } from "./testcontainers/dendrite.ts";
|
||||
import { HomeserverType } from "./plugins/homeserver";
|
||||
|
||||
export interface TestFixtures {
|
||||
mailhogClient: mailhog.API;
|
||||
@@ -37,7 +39,9 @@ export interface Services {
|
||||
oAuthServer?: OAuthServer;
|
||||
}
|
||||
|
||||
export interface Options {}
|
||||
export interface Options {
|
||||
homeserverType: HomeserverType;
|
||||
}
|
||||
|
||||
export const test = base.extend<TestFixtures, Services & Options>({
|
||||
logger: [
|
||||
@@ -104,21 +108,35 @@ export const test = base.extend<TestFixtures, Services & Options>({
|
||||
},
|
||||
|
||||
synapseConfig: [{}, { scope: "worker" }],
|
||||
homeserverType: ["synapse", { option: true, scope: "worker" }],
|
||||
_homeserver: [
|
||||
// eslint-disable-next-line no-empty-pattern
|
||||
async ({}, use) => {
|
||||
const container = new SynapseContainer();
|
||||
async ({ homeserverType }, use) => {
|
||||
let container: HomeserverContainer<any>;
|
||||
switch (homeserverType) {
|
||||
case "synapse":
|
||||
container = new SynapseContainer();
|
||||
break;
|
||||
case "dendrite":
|
||||
container = new DendriteContainer();
|
||||
break;
|
||||
case "pinecone":
|
||||
container = new PineconeContainer();
|
||||
break;
|
||||
}
|
||||
|
||||
await use(container);
|
||||
},
|
||||
{ scope: "worker" },
|
||||
],
|
||||
homeserver: [
|
||||
async ({ logger, network, _homeserver: homeserver, synapseConfig, mas }, use) => {
|
||||
async ({ homeserverType, logger, network, _homeserver: homeserver, synapseConfig, mas }, use) => {
|
||||
if (homeserver instanceof SynapseContainer) {
|
||||
homeserver.withConfig(synapseConfig);
|
||||
}
|
||||
const container = await homeserver
|
||||
.withNetwork(network)
|
||||
.withNetworkAliases("homeserver")
|
||||
.withLogConsumer(logger.getConsumer("synapse"))
|
||||
.withConfig(synapseConfig)
|
||||
.withLogConsumer(logger.getConsumer(homeserverType))
|
||||
.withMatrixAuthenticationService(mas)
|
||||
.start();
|
||||
|
||||
@@ -137,7 +155,11 @@ export const test = base.extend<TestFixtures, Services & Options>({
|
||||
{ scope: "worker" },
|
||||
],
|
||||
|
||||
context: async ({ logger, context, request, homeserver, mailhogClient }, use, testInfo) => {
|
||||
context: async ({ homeserverType, synapseConfig, logger, context, request, homeserver }, use, testInfo) => {
|
||||
testInfo.skip(
|
||||
!(homeserver instanceof SynapseContainer) && Object.keys(synapseConfig).length > 0,
|
||||
`Test specifies Synapse config options so is unsupported with ${homeserverType}`,
|
||||
);
|
||||
homeserver.setRequest(request);
|
||||
await logger.onTestStarted(context);
|
||||
await use(context);
|
||||
|
||||
Reference in New Issue
Block a user