Jest + Playwright Testing by jhildenbiddle · Pull Request #1276 · docsifyjs/docsify

jhildenbiddle · 2020-07-08T02:30:49Z

Summary

Added Jest test environment for unit and integration tests
Added Playwright test environment for e2e tests
Removed previous unit and integration test environments (mocha + chai + jsdom)
Removed previous e2e test environment (Cypress.io)
Ported existing unit, integration, and e2e tests of value to new environments

Details

An implementation of Jest+Playwright. This setup offers a number of advantages over the existing test structures (mocha+chai+jsdom & Cypress):

Ability to perform unit, integration, and e2e tests with one test configuration.
Separate test environments for unit + integration (Jest+jsdom) and e2e (Jest + Playwright) tests.
Testing in real browsers (Chromium, Firefox, and Webkit) for e2e tests.
Significantly faster and easier to create complex tests that require custom docsify configurations and site content thanks to a new docsifyInit() helper. This abstraction allows new docsify instances to be dynamically generated declaratively with a fraction of the code that would otherwise be required. It also removes the need to manage a large collection of fixture files (although you can use them if you prefer).
Documentation content is mapped to /docs/ on the local testing server, making it easy to use our documentation as fixtures without having to duplicate the content.
The local ./lib/ directory is also mapped to /lib/ on the local testing server, making it easy to use the latest docsify JS and CSS.
Example tests provided to help reduce the learning curve for new contributors

Changes have been made to the GitHub workflows to accommodate these changes:

A new "Build & Test" workflow has been added. This workflow runs all tests in a single workflow to prevent separate checkout+build operations for each test type. It updates the test matrix to build and test docsify on Linux (ubuntu-latest), macOS (macos-latest), and Windows (windows-latest) on the three latest LTS versions of Node (10, 12, 14).
The linting workflow has been updated to prevent building docsify, as there is no need to perform a build to link source code (the build output is ignored by eslint).

Part of the motivation for doing this work was to allow me to create proper tests for #1271. The other part was to try and clean up our test configuration so that we can offer a reliable and easy-to-learn environment for devs to work in. I looked at our existing test structures and realized there was no way for me do what I needed to do easily without creating complex tests with repeated boilerplate code and a large number of fixtures. Specifically, testing #1271 requires multiple docsify configurations that must be run in a browser with and without additional third-party scripts (vue.js). With this setup, every custom docisfy test is between 8 and 11 lines of code and zero test-specific file fixtures.

A few tips for reviewers:

All Jest and Playwright code is isolated in a new ./test/ directory. This means this PR can safely be merged and it will not affect existing tests.
Start by reviewing /test/README.md. This file provides an overview of the test environments, global variables, and CLI commands available.
Review ./jest.config.js and note that we are using Jest projects to simplify unique environment configurations for unit, integration, and e2e tests.
Review ./test/helpers/docsify-init.js. This helper greatly simplifies the process of creating dynamic docsify sites for individual tests.
For Jest (and jsdom) testing, review /test/unit/example.test.js and /test/integration/example.test.js.
For Jest + Playwright testing, review /test/e2e/example.test.js.
I've created real-world tests for vue.js reactivity (the reason Fix Vue Reactivity #1271 exists). These are available for review in /test/e2e/vue.test.js;
VSCode debug configurations have been added to make testings a single test or file easier. You can access these by clicking the debug icon on VSCode's sidebar (or pressing SHFT+CMD/CTRL+D). The two tests currently in place are:
- Run current test file
- Run selected test name (requires highlighting describe() or test() name)
- Update current test file snapshot(s)
- Update selected test name snapshot(s) (requires highlighting describe() or test() name)

What kind of change does this PR introduce? (check at least one)

Does this PR introduce a breaking change? (check one)

Yes
No

The PR fulfills these requirements:

When resolving a specific issue, it's referenced in the PR's title (e.g. fix #xxx[,#xxx], where "xxx" is the issue number)

Fix #1217

You have tested in the following browsers: (Providing a detailed version will be better.)

If adding a new feature, the PR's description includes:

A convincing reason for adding this feature
Related documents have been updated
Related tests have been updated

vercel · 2020-07-08T02:30:55Z

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/docsify-core/docsify-preview/fs42flso2
✅ Preview: https://docsify-preview-git-jest-playwright.docsify-core.vercel.app

codesandbox-ci · 2020-07-08T02:34:58Z

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

Latest deployment of this branch, based on commit d58b3ad:

Sandbox	Source
docsify-template	Configuration

trusktr · 2020-07-08T04:50:25Z

Nice! I will take a look this weekend!

- Setup projects for e2e, integration, and unit tests in Jest configuration - Setup /tests/e2e/ tests for Jest+PlayWright - Setup /tests/integration/ tests for Jest - Setup /tests/unit/ tests for Jest - Setup eslint for Jest and Jest+Playwright environments - Move e2e configuration files into separate folder - Update e2e example tests - Update unit example tests - Update HTML fixtures

- Add `debug` option - Append `Markdown` suffix to markdown-related options - Reorder options alphabetically

…2e only) - Create identical docsifyInit() tests using Jest (integration) and Playwright (e2e) - Update docsifyInit() to convert relative URLs to absolute URLs to work in both JSDOM and Playwright - Update docsifyInit() to append style- and js-related tags using createElement instead of insertAdjacentHTML - Update paths in test files to use unified docsifyInit() - Added option to docsifyInit() to enable/disable formatted HTML output to console - Removed vue.css as default docsify theme from docsifyInit() - Removed outdated files

- Allows mocking all XMLHttpRequests instead of just those initiated via /src/core/fetch/ajax.js - Allows JSDOM tests to use /lib/docsify.js instead of /src/core/index.js (same as Playwright tests) - Allows JSDOM tests to use /docs site as test content

jhildenbiddle · 2020-09-26T08:42:46Z

@anikethsaha @trusktr

All done. :) Feel free to kick the tires and let me know what you think. Note the updated details in the first comment as well.

I have noticed that this branch tends to get stuck on the "build (10.x)" task. This is odd, since it appears as though GitHub is not picking up the changes I made to our CI workflows: https://github.com/docsifyjs/docsify/tree/jest-playwright/.github/workflows. Not sure what's happening there. For now, you can check out the branch, update/install the necessary dependencies, and we can wrap up the review.

- Import globals from various files instead of manually adding them to ensure they are availability in Jest and eslint configurations - Add middleware to server configuration for serving virtual “_blank.html” file - Add BLANK_URL - Rename TEST_URL to TEST_HOST - Removed ./test/fixtures/ directory (blank page now served via server.js middleware) - Added page.goto(BLANK_URL) call to global Playwright beforeEach() setup

sy-records · 2020-10-01T03:02:40Z

LGTM.

jhildenbiddle · 2020-10-01T16:19:50Z

@anikethsaha --

Few non-blocking suggestion/question

is it safe to remove the cypress now ?

Yep. The Cypress tests we had were limited (only tested sidebar links on /doc site which we can do much more easily with snapshots), slow, and brittle. There's no reason to hang on to the tests or the dependencies.

can we add the new script in the github action

GitHub Actions have already been updated in this PR. Unfortunately, the workflows aren't running because of GitHub issue that prevents workflows from running on PRs that have merge conflicts. The conflicts that need to be resolved are very minor and easily resolved. Once resolved and merged, workflows should run as expected.

not sure what is unit in the tests. I couldn't notice any jsdom imports, are those just mockups ?

Jest comes with jsdom baked in. No need for a separate jsdom dependency or import.

jhildenbiddle · 2020-10-01T18:35:02Z

@sy-records --

LGTM.

I just flipped the PR from "draft" to "ready". Can you approve the PR if everything looks good?

jhildenbiddle · 2020-10-01T18:36:36Z

@anikethsaha --

The whole setup and tests looks good to me.

I just flipped the PR from "draft" to "ready". Can you approve the PR if everything looks good?

Koooooo-7

@jhildenbiddle Awesome.

anikethsaha

Thanks

- Relocate carbon plugin script with other scripts - Update zh-cn docs URL to align with other translatins (GitHub, not jsdelivr) - Add major version locks to URLs

# Conflicts: # cypress/fixtures/tpl/docs.index.html # docs/index.html # package-lock.json # test/unit/render.test.js

jhildenbiddle added 15 commits July 7, 2020 06:31

Initial Jest+Playwright setup

6eb7c90

Fix eslint warnings

f299c26

Add placeholder for common options

7e6a7b0

Fix load order of scriptURLs

c35aea5

Add docsifyURL and waitForSelector options

08e1bbf

Add executeScript scenarios

a8c5cb7

Enable firefox and webkit tests by default

5229727

Prevent prettier from reformatting browsers list

045e91e

Update options and add JSDoc comments

83fd5fd

Initial commit

f704f0b

Complete initial example tests

25747d6

Minor tweaks

e0dffb6

Change suite name

efab952

Rename file

2c7e66d

Add npm script to run jest+playwright example test

36c5c33

jhildenbiddle added the tests label Jul 8, 2020

jhildenbiddle requested review from Koooooo-7, QingWei-Li, anikethsaha, sy-records and trusktr July 8, 2020 02:31

jhildenbiddle added 6 commits July 12, 2020 11:38

Clean up server properties

e8da21d

Update docsifyInit helper

3ce73f1

- Add `debug` option - Append `Markdown` suffix to markdown-related options - Reorder options alphabetically

Add jestPlaywright.debug() to e2e examples

2b7c54f

Removed need to pass page as arg to docsifyInit()

1ab3de2

Add VSCode debug configurations

d167031

jhildenbiddle added 5 commits September 25, 2020 15:35

Reorganize test files

4fec434

Fix basePath option

da36452

Added new waitFor helpers

e4887ae

jhildenbiddle added 6 commits September 30, 2020 15:43

Add try/catch for waitForFunction callback

4cbfc0a

Move playwright config into jest.config.js

79aeaa9

Add runInBand option to tests for reliability

473bb5a

Remove unnecessary XHR Mock teardown

2440a0c

Add —runInBand to test script

9f5e562

sy-records previously approved these changes Oct 2, 2020

View reviewed changes

Koooooo-7 previously approved these changes Oct 2, 2020

View reviewed changes

anikethsaha previously approved these changes Oct 3, 2020

View reviewed changes

jhildenbiddle added 8 commits October 3, 2020 18:25

Merge develop

f6c34f8

Cleanup

0983d53

- Relocate carbon plugin script with other scripts - Update zh-cn docs URL to align with other translatins (GitHub, not jsdelivr) - Add major version locks to URLs

Removed fixed host value from docs

5ccea9d

Updated test after merge (docsify version change)

af71553

Added startPath for manual instance

1c365fb

Merge branch 'develop' into jest-playwright

4623677

# Conflicts: # cypress/fixtures/tpl/docs.index.html # docs/index.html # package-lock.json # test/unit/render.test.js

Remove Node 10 from CI tests

befa408

Remove Node 10 from CI lint tests

d58b3ad

jhildenbiddle mentioned this pull request Oct 8, 2020

fix: Cannot serve off /.../index.html #1372

Merged

19 tasks

trusktr mentioned this pull request Nov 2, 2020

[WIP] Finish initial SSR (server side rendering) and add tests. #1227

Draft

21 tasks

sy-records mentioned this pull request Feb 5, 2021

[Release]: v4.12.0 #1493

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Jest + Playwright Testing#1276

Jest + Playwright Testing#1276
jhildenbiddle merged 131 commits intodevelopfrom
jest-playwright

jhildenbiddle commented Jul 8, 2020 •

edited

Loading

Uh oh!

vercel bot commented Jul 8, 2020 •

edited

Loading

Uh oh!

codesandbox-ci bot commented Jul 8, 2020 •

edited

Loading

Uh oh!

trusktr commented Jul 8, 2020

Uh oh!

jhildenbiddle commented Sep 26, 2020

Uh oh!

sy-records commented Oct 1, 2020

Uh oh!

jhildenbiddle commented Oct 1, 2020

Uh oh!

jhildenbiddle commented Oct 1, 2020

Uh oh!

jhildenbiddle commented Oct 1, 2020

Uh oh!

Koooooo-7 left a comment

Uh oh!

anikethsaha left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

jhildenbiddle commented Jul 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vercel bot commented Jul 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codesandbox-ci bot commented Jul 8, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

trusktr commented Jul 8, 2020

Uh oh!

jhildenbiddle commented Sep 26, 2020

Uh oh!

sy-records commented Oct 1, 2020

Uh oh!

jhildenbiddle commented Oct 1, 2020

Uh oh!

jhildenbiddle commented Oct 1, 2020

Uh oh!

jhildenbiddle commented Oct 1, 2020

Uh oh!

Koooooo-7 left a comment

Choose a reason for hiding this comment

Uh oh!

anikethsaha left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

jhildenbiddle commented Jul 8, 2020 •

edited

Loading

vercel bot commented Jul 8, 2020 •

edited

Loading

codesandbox-ci bot commented Jul 8, 2020 •

edited

Loading