Creating a Visual Comparison Tool for Front-End Developers 🔬

Regressions are the worst, and they’re inevitable in any software development cycle. We can mitigate their occurances as much as possible through automated testing, but getting 100% test coverage is a time-consuming task and sometimes not viable. Something I’ve run into a lot as a Front-End developer are visual regressions, where a change will be made to a stylesheet that at first glance seems fine, but later causes chaos somewhere else in the application. As the length and complexity of a project increases, making sure your styles don’t regress can be a tricky task.

I’ve used a couple of tools in the past that show visual diffs between two pages, but I’ve always wondered what it would take to make my own version. So join me as I dive down the rabbit hole once again to create my own CRUD (Crete, Read, Update, Delete) application to do exactly that. For this project I’ll be using NodeJS, React and Redux.

Creating the API

The first order of business was setting up a database and creating an API to store the tests. To handle this I decided to utilize Express for my routes and MongoDB for the database. To get started I laid out the database connections using the Mongoose package, requiring the path from the /config/database.config.js file. Once the connection has been established I required the routes for the API and then started Express.

const express = require('express');
const bodyParser = require('body-parser');
const app = express();
const mongoose = require('mongoose');
const MongoConfig = require('./config/database.config');
const morgan = require('morgan');
const cors = require('cors');
const port = 9090;

app.use(morgan('combined'));
app.use(cors());
app.use(bodyParser.urlencoded({ extended: true }));
app.use(bodyParser.json());

mongoose.Promise = global.Promise;

// Establish connection to the database...
mongoose.connect(MongoConfig.url, { useNewUrlParser: true })
.then(() => {
  require('./routes/tests.routes.js')(app);

  app.listen(port, () => {
    console.log('Now listening on port ', port)
  })

}).catch(error => {
  console.error('Encountered an error while starting the API: ', error)
  process.exit();
});

Afterwards I started setting up the API endpoints. As I intend for this to be a CRUD application I started with the usual suspects, I needed an endpoint to create a test, retrieve all tests, find a specific one, update and delete. Inside of my routes/tests.route.js file I exported a module that defined these endpoints, and then tied them to a function that would later be defined within controllers/tests.controller.js.

const tests = require('../controllers/tests.controller.js')

module.exports = app => {
  app.post('/tests', tests.create);
  app.get('/tests', tests.findAll);
  app.get('/tests/:testId', tests.findOne);
  app.put('/tests/:testId', tests.update);
  app.delete('/tests/:testId', tests.delete);
}

I knew what type of information I’d need in the database in order to run a test, so the next step was creating a schema for the test object before we started interfacting with the database. I set this up within models/tests.model.js. The schema tells the database what fields it can expect. However, it doesn’t define which fields are required. I’ll set that up in our controller within the tests.create and tests.update methods.

const mongoose = require('mongoose')

const TestSchema = mongoose.Schema({
  name: String,
  description: String,
  live: String,
  dev: String,
  size: Number
})

module.exports = mongoose.model('Test', TestSchema);

Because the tests will rely on a Node service to run I’d need some sort of interface that would allow our Front-End to trigger a process on the backend. Therefore in addition to the endpoints I’ve already created I setup two more, one that will run all tests, and one that will run a specific test.

module.exports = app => {
  // ...
  app.get('/run', tests.testAll);
  app.get('/run/:testId', tests.testOne);
}

The intention here is to fire a Node function using the test data in the database when the endpoint is requested, generating the visual diff images. From an API perspective it would return a success or failure boolean when it’s done processing so we can inform our Front-End of the status. If this application was being deployed to a server there would need to be more considerations made for this in regards to performance and authentication, but for local use only it’s fine.

exports.testAll = (req, res) => {
  Test.find().then((tests) => {
    // We should be firing a function here to generate the diffs...
  })
}

exports.testOne = (req, res) => {
  const tests = []
  Test.findById(req.params.testId).then(test => {

    tests.push(test);

    /* This function should behave like the one in testAll, except the array will only contain one test.
    I'll be keeping them both arrays for consistency. */
  })
}

Generating the Diffs

To generate the diff images I used two Node libraries. One is Puppeteer, which provides a high-level API to control the Chrome browser; I’ll be using this to take the page screenshots. The other is Pixelmatch, a library by Mapbox which highlighlights pixel differences between two images. I started off by creating and exporting a module which would capture the initial browser shots. This module should accept an array of test data as its arguement and return the success flag when it was done creating the visual diff images on the localdisk. In order to achieve this I had the module return a promise.

async function runTest(test) {
  // This is the function that will actually run the test!
}

module.exports = tests => {
  return new Promise((resolve, reject) => {
    // This promise will resolve once we have our visual diffs
  })
}

As it’s expecting an array of data, I’ll need to fire off the runTest function for every item in the array, therefore I’ll need to lean even harder on the promise library. Using Array.map I converted my array of data into promises. That way I can then pass that mapped array into a Promise.all statement and resolve our wrapping promise once all of our items in the array are done processing.

I also created an if statement that would fire at the start of the block that will create our folder to hold the images in if it doesn’t exist already, if the user accidently deletes the folder it will error out the entire process and the application would no longer work.

const mkdirp = require('mkdirp');

module.exports = tests => {
  return new Promise((resolve, reject) => {
    
    // Makes sure that our save directory for our images actually exist
    if (!fs.existsSync(directory)) {
      mkdirp(directory, (() => {
        console.log('Created directory')
      }));
    };

    let testPromises = tests.map(test =>  {
      return runTest(test);
    });
  
    Promise.all(testPromises).then(results => {
      resolve(results);
    }).catch(() => {
      reject('Encountered processing error.');
    });
  })
}

The goal of the runTest function is to start a headless Chrome browser using Puppeteer and then do the following:

  1. Set the viewport width if a size value is provided.
  2. Navigate to the live page and wait for a few seconds to let any animations finish playing.
  3. Take a screenshot of the live page.
  4. Do the same for the dev page.
  5. Generate the pixel comparison overlay using the live and dev page images.
  6. Close the browser and return.

The documentation for Puppeteer is quite clear, and getting an initial proof of concept going was quite simple.

async function timeout(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

async function runTest(test) {
  const { _id, name, live, dev, size } = test;

  // Launches the browser.
  const browser = await puppeteer.launch();

  // Creates a new 'tab'.
  const page = await browser.newPage();

  /* If a size is present we set the viewport, otherwise we leave it
  open to the default setting. */
  if (size) {
    await page.setViewport({
      width: size,
      height: 0
    })
  };

  /* Navitates puppeteer to the page to take the screenshot. */
  await page.goto(live)
      
  /* We set a timeout here to make sure that all initial load animations
  have finished playing before we take the screenshots. */
  await timeout(5000);

  // Takes the screenshot for the live page.
  await page.screenshot({
    path: `${directory}/live_${_id}.png`,
    fullPage: true
  });

  // Navigates to the dev page.
  await page.goto(dev)

  await timeout(5000);

  // Takes the screenshot for the dev page.
  await page.screenshot({
    path: `${directory}/dev_${_id}.png`,
    fullPage: true
  });

  /* Once we have both of our screenshots we send them over to the compare function
  so we can get a pixel overlay for each image. */
  await compare(_id);

  // Closes the browser.
  await browser.close()

  // Sets 'success': true in our test object and returns.
  test['success'] = true;
  return test;
}

This initial pass of the function does mostly everything I need it to, except there’s a problem, primarily in the error handling department. If I run through this function and provide it two valid paths it will complete and everything will be fine, however if I provide it at least one invalid path, it will error and not pass back any form of usable data for our API response. To improve this I utilized try, catch and finally to throw errors when they are encountered. In the example below I setup catch cases on the goto calls, and then throw an error if it’s triggered, moving it to the wrapping catch case, this allows me to set the property success to false. If it reaches the end of the try case then finally is triggered so I can then set success to true.

async function runTest(test) {
  const { _id, name, live, dev, size } = test;
  const browser = await puppeteer.launch();

  try {
    const page = await browser.newPage();

    /* If a size is present we set the viewport, otherwise we leave it
    open to the default setting. */
    if (size) {
      await page.setViewport({
        width: size,
        height: 0
      })
    };

    /* If an error is encountered we throw an exception cancelling the rest of the process.
    These errors are collected in an API response so we can alert the front-end. */
    await page.goto(live).catch((error) => {
      throw `${error} ${live}`
    });

    /* We set a timeout here to make sure that all initial load animations
    have finished playing before we take the screenshots. */
    await timeout(5000);

    await page.screenshot({
      path: `${directory}/live_${_id}.png`,
      fullPage: true
    });

    await page.goto(dev).catch((error) => {
      throw `${error} ${dev}`
    });

    await timeout(5000);

    await page.screenshot({
      path: `${directory}/dev_${_id}.png`,
      fullPage: true
    });

    /* Once we have both of our screenshots we send them over to the compare function
    so we can get a pixel overlay for each image. */
    await compare(_id);

  } catch(error) {
    await browser.close();
    test['success'] = false;
    return test;
  } finally {
    await browser.close();
    test['success'] = true;
    return test;
  }
}

The compare function that gets fired near the end of runTest also resolves a promise when it’s done processing, signalling the async function to move onto the next step. The code is mostly unchanged from the Pixelmatch README example.

module.exports = id => {
  return new Promise((resolve, reject) => {
    const directory = DirectoryConfig.path;
    let filesRead = 0;
  
    const live = fs.createReadStream(`${directory}/live_${id}.png`).pipe((new PNG()).on('parsed', doneReading));
    const dev = fs.createReadStream(`${directory}/dev_${id}.png`).pipe((new PNG()).on('parsed', doneReading));
  
    function doneReading() {
      if (++filesRead < 2) return;
    
      const diff = new PNG({width: live.width, height: live.height})
      pixelmatch(live.data, dev.data, diff.data, live.width, live.height, { threshold: 0.1 });
    
      diff.pack().pipe(fs.createWriteStream(`${directory}/diff_${id}.png`));
      resolve('Resolving')
    }
  })
}

With these building blocks in place I can now call the capture module when our endpoint is requested, and trigger a response with usable data. The response will be delayed until the runTest function has been fired for every item in the array.

const capture = require('../utilities/capture');

exports.testAll = (req, res) => {
  Test.find().then((tests) => {
    capture(tests).then(testResults => {
      res.send(testResults);
    }).catch(error => {
      res.send(error);
    })
  })
}

exports.testOne = (req, res) => {
  const tests = []
  Test.findById(req.params.testId).then(test => {

    tests.push(test);

    capture(tests).then(testResults => {
      res.send(testResults)
    }).catch(error => {
      res.send(error)
    })
  })
}

If you’d like to see the other API controllers you can view them on GitHub here. The delete function also interacts with the localdisk and deletes screenshots from tests which have been previously run when the delete endpoint is requested.

For the Front-End I decided to make a simple application using React and Redux that will display a list of tests, with a test page to display our diffs, and a form page to create/edit the tests. You can view the full code for the Front-End on GitHub.

In Conclusion

After numerous amounts of testing and making sure everything worked together, I finally ended up with an application that worked! This post was intended to give you insight into my thought process while building this project, however if you’d like to go into more detail you can check out the source on GitHub.

As always if you have any questions or feedback feel free to reach out on Twitter or via my contact form.