Gleb Bahmutov PRO
JavaScript ninja, image processing expert, software quality fanatic
Network Control for End-to-End Web Testing
https://slides.com/bahmutov/network-control-e2e
Gleb Bahmutov
LA/Canada/world Fires
brought to you by the fossil fuel companies
image source: https://www.dailysabah.com/world/americas/more-damage-anticipated-as-california-fire-season-sets-records
Control your life
Join an organization
Vote
Greenpeace 350 Sunrise Citizen Climate Lobby
Speaker: Gleb Bahmutov PhD
C / C++ / C# / Java / CoffeeScript / JavaScript / Node / Angular / Vue / Cycle.js / functional programming / testing
Gleb Bahmutov
Sr Director of Engineering
🌎 🔥 350.org 🌎 🔥 citizensclimatelobby.org 🌎 🔥
Mercari Does A Lot Of Testing
A typical Mercari US Cypress E2E test
E2E Is Great
The focus is on the user
Downsides of full E2E
- Deployed web application
- Deployed server(s)
- Full roundtrip latency
Example app: TodoMVC
Example app: TodoMVC
Web vs App server
- Web server: http, css, static js
- nginx
- apache
- github pages
Web vs App server
- Web server: http, css, static js
- nginx
- apache
- github pages
-
App server: server-side code execution
- Node.js
- Go
- PHP
Web vs App server
- Web server: http, css, static js
- Simple!
- Caching
- Edge caching
Web vs App server
- Web server: http, css, static js
- Simple!
- Caching
- Edge caching
-
App server: server-side code execution
- Slow
- Shared data
- Security
Typical App Server Test
- reset the data (if possible)
// branch a1
// cypress/e2e/add-todo.cy.js
beforeEach(() => {
cy.request('POST', '/reset', { todos: [] })
})
it('adds a todo', () => {
cy.visit('/')
cy.log('**confirm the items are loaded**')
cy.get('.loaded')
cy.get('.new-todo').type('item 1{enter}')
cy.get('li.todo').should('have.length', 1)
cy.get('.new-todo').type('item 2{enter}')
cy.get('li.todo').should('have.length', 2)
cy.log('**confirm the items are saved**')
cy.reload()
cy.get('li.todo').should('have.length', 2)
})Reminder: Switch to branch "a1"
Typical App Server Test
- reset the data (if possible)
- interact with the application
// branch a1
// cypress/e2e/add-todo.cy.js
beforeEach(() => {
cy.request('POST', '/reset', { todos: [] })
})
it('adds a todo', () => {
cy.visit('/')
cy.log('**confirm the items are loaded**')
cy.get('.loaded')
cy.get('.new-todo').type('item 1{enter}')
cy.get('li.todo').should('have.length', 1)
cy.get('.new-todo').type('item 2{enter}')
cy.get('li.todo').should('have.length', 2)
cy.log('**confirm the items are saved**')
cy.reload()
cy.get('li.todo').should('have.length', 2)
})Reminder: Switch to branch "a1"
Reminder: Switch to branch "a1"
add-todo.cy.js spec
The test is slow!
Goal:
- Avoid app server during testing
- Testing is SO much simpler
- Make the tests faster 🏎️
- Test in parallel 🤖 🤖 🤖 🤖 ...
Step 1: Record API Calls 🎥
// branch a2
// cypress.config.js
// https://github.com/bahmutov/cypress-magic-backend
magicBackend: {
// this app makes "XHR" calls to load and update "/todos"
// match calls like
// GET /todos
// POST /todos
// DELETE /todos/1234
apiCallsToIntercept: [
// TODO: insert the intercept definitions
],
},
Press the "🪄 🎥" Record button
Reminder: Switch to branch "a2" and restart the app
Step 1: Record API Calls 🎥
// branch a2
// cypress.config.js
// https://github.com/bahmutov/cypress-magic-backend
magicBackend: {
// this app makes "XHR" calls to load and update "/todos"
// match calls like
// GET /todos
// POST /todos
// DELETE /todos/1234
apiCallsToIntercept: [
{
method: '+(GET|POST)',
pathname: '/todos',
},
{
method: 'DELETE',
pathname: '/todos/*',
},
],
},
Recording API Calls During The Test
Saved JSON file with API calls
{
"pluginName": "cypress-magic-backend",
"pluginVersion": "1.11.0",
"specName": "cypress/e2e/add-todo.cy.js",
"testName": "adds a todo",
"apiCallsInThisTest": [
{
"method": "GET",
"url": "/todos",
"request": "",
"response": [],
"duration": 1013
},
{
"method": "POST",
"url": "/todos",
"request": {
"title": "item 1",
"completed": false,
"id": "6895127373"
},
"response": {
"title": "item 1",
"completed": false,
"id": "6895127373"
},
"duration": 1009
},
{
"method": "POST",
"url": "/todos",
"request": {
"title": "item 2",
"completed": false,
"id": "6787924392"
},
"response": {
"title": "item 2",
"completed": false,
"id": "6787924392"
},
"duration": 1008
},
{
"method": "GET",
"url": "/todos",
"request": "",
"response": [
{
"title": "item 1",
"completed": false,
"id": "6895127373"
},
{
"title": "item 2",
"completed": false,
"id": "6787924392"
}
],
"duration": 1010
}
]
}cypress/magic-backend/cypress/e2e/add-todo.cy.js_adds_a_todo_api_calls.json
Replay 🪄🎞️
Replaying API Calls During The Test
Magic Replay Mode
2 seconds!
We do not need the "POST /reset" API call in the replay mode
// branch a2
// cypress/e2e/add-todo.cy.js
beforeEach(() => {
const mode = Cypress.env('magic_backend_mode')
if (mode !== 'playback') {
mode && cy.log(`during the test the mode is "${mode}"`)
cy.request('POST', '/reset', { todos: [] })
}
})cypress-magic-backend: from 6s to 0.6s
cypress-watch-and-reload
// branch a3
// cypress.config.js
// list the files and file patterns to watch
// https://github.com/bahmutov/cypress-watch-and-reload
'cypress-watch-and-reload': {
watch: ['index.html', 'app.js'],
},Use the "locked" Replay Mode "🪄 🎞️ ☑️"
Reminder: Switch to branch "a3" and restart the app
Normal vs Locked replay mode while watching the "index.html" file
🪄 🎞️ = Replay
🪄 🎥 = Record
What does this "🪄 🧐" button do?
The "Inspect" button is my ❤️
Reminder: Switch to branch "a4" and restart the app
Normal vs Inspect replay mode "🪄 🧐"
Hmm, something has changed in our API,
the "POST /todos" call is now twice slower!
You can see more details about the observed timing differences by clicking on the warning in the Command Log
It is not just the timing...
Reminder: Switch to branch "a5" and restart the app
Hmm, something isn't right
Even if the test is green...
Normal vs Inspect replay mode
But where could the problem be ...
Run in the "Inspect Mode" and see exactly what the problem is and where it started
A change in the front-end code started the entire chain of errors!
cypress-magic-backend
Inspect Mode 🪄 🧐
- Compared API calls against recorded JSON files
- Reports timing changes
-
Reports schema changes
- But not the values!
Magic-backed as a Service
- No need to manage JSON files
- More information for the inspect mode
Reminder: Switch to branch "a6"
run "npm install" and restart the app
using the "as-a ." command to use the API key
// branch a6
// cypress.config.js
magicBackend: {
// where to store recorded API calls?
// local: store all API calls locally in JSON files
// remote: send API calls to a remote server at cypress.tips
store: 'remote',
}Works just like before but without local JSON files
Magic-backed as a Service
Recorded API Calls For Faster Debugging
Numbers example
Run the "number.cy.js" spec
Change the input number
// branch a6
// cypress/e2e/number.cy.js
// change to produce some negative numbers
const random = Cypress._.random(3, 30)Try getting both passing and failing tests
then open DevTools console and see a failed test
Can you guess which input leads to a failure?
Magic-backed Debugging
cypress-magic-backend with remote API recordings
Automatic context for AI prompt
+++
---
-
Speed
-
Independence
-
Debugging
-
Record of API calls
-
Same calls order
-
Random / time-sensitive data
-
Maintenance
cypress-magic-backend
API Calls Recording And Replaying
Learn More 🎓
- https://github.com/bahmutov/cypress-magic-backend
- 📝 read the blog post "Magic Backed For E2E Testing" https://glebbahmutov.com/blog/magic-backend/
- 📝 read the blog post "Magic Backed Inspection Mode" https://glebbahmutov.com/blog/magic-backend-inspect-mode/
- 📝 read the blog post "Watch Mock And Reload" https://glebbahmutov.com/blog/watch-mock-reload/
- 📝 read the blog post "Mock But Verify" https://glebbahmutov.com/blog/mock-but-verify/
- 📺 videos on the plugin at https://youtube.com/@gleb
OSS and/or Money
400 OSS
projects
plus example repos, blog posts, videos, GH issues triage
My time could be...
My time could be...
Reality
Can this continue?
Is it fair?
- OSS
- Talks
- Blog posts
- Videos
- Workshops
- Online courses
Is There A Better Way?
Dual license:
< 100 employees, free to use non-transferable
>= 100 employees, small license, non-transferable
- Free use for small companies and individuals to capture market share
- Small / large fee for large companies to pay for support
I don't know what the future looks like
I would like to give OSS the effort it deserves. OSS is worth it.
Network Control for End-to-End Web Testing
Gleb Bahmutov
👏 Thank you 👏
https://slides.com/bahmutov/network-control-e2e
Network Control for End-to-End Web Testing
By Gleb Bahmutov
Network Control for End-to-End Web Testing
A modern web application probably makes tens of network calls during a typical user flow. A good end-to-end web test must observe and in some cases mock the network calls to make the test reliable, complete, and easy to debug. I will show some of my tips and tricks for using and controlling network API calls during tests. I will use Cypress E2E test runner for my examples plus the "record - replay - inspect" pattern for API mocking. Presented at Testmu online conference, 40 minutes
