CypressIO Simplifies Web App Testing

After discussing projects over the past week, one of the guys at Automattic brought up CypressIO.   If you’ve been following the recent posts on Lance.Bio you know that the path to running automated web testing has run from Selenium IDE, the QA tool of choice for the past few years for Store Locator Plus, to recent forays into Selenium Webdriver.      Webdriver is powerful but difficult to code and soon led to the discovery of WebdriverIO, then Mocha and Chai on top of that.      The new stack makes it easier to write more advanced tests than we could in Selenium IDE — but it was like pulling teeth to get all the right pieces installed and working.

More info, less pain

That is what CypressIO promotes on their home page.   “Test your code, not your patience.” and “No more async hell.”  — if you’ve gone through the Selenium Webdriver setup you’ll understand what both of these mean.   It takes several false starts to understand all the pieces you MUST install to get Selenium Webdriver working.  CypressIO has made it easy; it is even easier than they promote on their website with the most recent builds.

The CypressIO Test Execution Window

The Cypress Test Execution Window

I’ve only started simple test writing but in less than 20 minutes I had a fully functional test running with all the extras from a completely “clean” install base.    If you have NodeJS and NPM up-to-date and a faster Internet connection than we have in tech-orphan Charleston South Carolina, you can be up and running in 5 minutes.
Read More

WebdriverIO and Mocha For Better Test Reports

In a continuing series on using WebdriverIO with Selenium we are going to focus on getting better reporting output now that our environment is setup.  The previous article on setting up WebdriverIO should have you up-and-running with basic tests.   Now it is time to use some of that “Mocha flavoring” to get useful reports out of our tests.   WebdriverIO and Mocha gives you the tools to group together tests and report the results in plain text.

Adding Mocha to a test

Mocha adds a ways to group together our test units and send meta information out to the report modules that are employed by WebdriverIO and the default test runner, conveniently named “TestRunner”.   Nothing special needs to be added if you’ve setup the project and employed the Mocha framework as shown in the prior article.

Since we are using the BDD test style with Mocha we have two primary functions we want to employ.

BDD = Behavior Driven Development,  in case you were wondering.

describe()

This is the Mocha syntax that is used to group together tests.   The first parameter in that function is the plain text that will be used to describe what tests are being run.   The second parameter is a standard JavaScript anonymous function that will include the test code.    Typically you use a series of it() functions within.

it()

This is the Mocha syntax that describes each individual thing to be tested.    The first parameter is again the plain text of what is being tested and the second is the function that contains the code.

A simple example

This test is going to open the home page for the website that was configured in the wdio.conf.js file when we setup our wdio configuration.

This does nothing more than make sure it can open the site URL.

When the test is run we now get a more detailed report of what happened.

Here is a video showing how the test run looks compared to the standard non-Mocha version.

Optimizing JavaScript for Chrome

A recent discussion with a tech guru at Automattic posed a great question — how does JavaScript asynchronous processing work on a single-threaded app?  Great question, but before I found out how that worked I decided to refresh my knowledge on how JavaScript manages the call stack.   Turns out a LOT has changed in 10 years and it turns out Google’s V8 engine was launched.

What is V8?

V8 was very likely a result of Google Maps.    Google Maps was one of the most-used JavaScript applications to hit the Internet (and has been my primary focus for the past 5 years as part of my Store Locator Plus project).  The problem with Google Maps is that the JavaScript library behind it is HUGE.    So big that it crushed most browsers of the day.   Not only did laptops and mobile devices have far less compute power — the JavaScript engines were not very efficient.

Enter V8.   Back in 2008 Google launched V8 as a new JavaScript engine for Chrome.   They employed several tricks to speed up code execution making their maps, and every other JavaScript applet, run faster.     It also happens to have become the core engine used in NodeJS which has itself become another “killer JS app”.
Read More

Selenium on JavaScript : New WordPress Site Config Script

This article continues the journey into learning Selenium on JavaScript (SoJa).  It builds on the lessons of the previous articles and adds some common library functions to make our tests more readable and starts to employ the Don’t Repeat Yourself (DRY) concept.

In this case we are going to build a common library module that we will re-use in future scripts. It will load our configuration and environment.  It will also export some methods that make our test script code a lot easier to read. The configuration and environment setup functionality is covered in prior articles so we’ll leave that out of this discussion.

What This Script Does

This script automates the very first steps of setting up a WordPress site after the software has been installed.  We use this on our Store Locator Plus development systems after doing a “fresh install” of WordPress so our dev team doesn’t have to keep typing in the same 5 things over-and-over every week when we spin up a new release to be tested “from scratch”.

The script will select our language (English, the default) wait for the next screen to load and then type in the title of the site, our username, email, check some boxes and click the install button.      The other elements of setting up our site such as the database name, etc. has been completed as part of our base dev box setup so you won’t see that here.   You can easily extend this test to cover the other WordPress install forms.

The Scripts

We’ll leave out the configuration and environment scripts as they have been covered a few times in other articles. Our main test script and the common lib are below.

Main Test

Common Lib

Discussion

The Main Script

The main script itself is fairly self-explanatory thanks to our new helper methods in our common library.   The main script load the common library and assigns it to the local test variable.    That library has setup logging for us, the Selenium Webdriver, and some “nice to have” functions that we’ll get into later.

We output that our activate site test has started to the log window, load up our WordPress site URL with the get command and when the page is loaded click the continue button, wait for various form elements like the blog title to appear and fill it in.  We also click some things as well.

Simple, right?

The Magic, aka Common Library

The real magic happens in the common library.   It employs the tricks we’ve learned in previous lessons to build reusable code and make our main scripts easier to read.   It uses the standard NodeJS module export technique to define names for our methods as well as their functionality.

There are some things that are common for most web tests, type a value into a form field, click a button on a form.    We’ve added some “make this stuff behave” logic using the Selenium Promise-based style to ensure things are more stable than randomly typing stuff and hoping they are on our page.

Let’s review our new methods in more detail.

When ID Exists Type

This method is used to look for an input on a page by its HTML ID and type something into the form.    We could just “jam some key strokes” into a field and assume it is on our page.  However we often are waiting for dynamic JavaScript elements to appear on modern websites.   We also have pre-filled input fields like password suggestions to deal with.   Things are not as simple as they seem.

In this method we first tell Selenium to wait for the element to appear on our page.   Our wait loop will give the web page 6 seconds to present the element or throw an error.   That is the driver.wait() part with until.elementLocated() within.

If the element is located run the success function defined within the then() follow-on method attached to the driver.wait() call.

Notice we assign findElement() to var el.   Why?  We want to make sure the input field is EMPTY when we start.    We do that by click on the input field first, then clearing it out, then ending our value in as keystrokes.   This addresses any pre-filled fields like the WordPress password suggestion as well as any add ons your browser may be employing that auto-fill forms (if you did not disable them, which you should).

The when_id_exists_click() is the same idea as the input method but in this case we have less work to do since we are clicking a button and don’t have to clear the element out.   No need to explain further.

When Name Visible Click

This method is a bit different than the ID Exists Click or ID Exists Type methods.   While the concept is the same the difference is in the execution.    The Selenium until.elementIsVisible() method has been constructed differently than the other methods in the library.    This is one of those language inconsistencies, and every language has them, that drives developer crazy.

 

Many of the other until.blah() methods take a LOCATOR as the parameter, in other words the By.id or By.name method where the locator is what is being processed.   elementIsVisible() takes the actual web element itself.    It’s a gotcha that has tripped up more than a couple of Selenium coders.   It is why our Visible Click method is more complex.

To not repeat ourselves and ensure consistency we setup our locator in the by variable to locate an element by name.    We let the driver wait until our element is found by name on the page, using the standard 6 second delay.   Once we know the element is on the page we assign it to the el variable and then do another wait until the element is visible.

Remember, just because an element is within the web page (part of the DOM structure) it does not mean it is visible.   It is also important to know that many Selenium functions will NOT let you interact with an element that is not visible.

 

Selenium on JavaScript : User List Test

I wasn’t quite sure what to name this article.  The Selenium on JS example here can be used to scan any table to ensure every entry on a list of strings exists; should I name it Test Web Page Has All Your Important Data?   The test also uses a separate NodeJS module to configure that list of important string, in my case user account names; Using NodeJS modules to configure repetitive data lists?  It also employes the Promise construct inherent in Selenium on NodeJS so maybe The Right Way To Wait Before Testing Data?

Instead I chose “User List Test” because that is what my test case does.

In this example I want to test a list of user accounts I deem “critical test users” to ensure that none of the user accounts have gone missing after upgrading our staging or production server.  Each user account was selected because it stress tests specific parts of our MySLP SaaS application.    That means I also want to make sure the user accounts are still active before delving deeper into the test suite.

MySLP Dashboard 2017-09-26_22-06-56.png

This example is a bit more complex than the prior Selenium articles written this week and builds on an excellent resource I found online.  It talks about the Promise model and how to properly wait for things to exist versus using the “hack” driver.sleep() command to wait for stuff to appear.

Let’s get into it.    Here is a test that loops through a list of users and checks to make sure they all appear in an HTML table that is the list of users.

The App

Before I get into the code I want to show you the web app interface.  It is a modified variant of the WordPress User table.   This is where we will test that the table cells have the users I’m looking for.

MySLP Customer Blur Partial 2017-09-26_20-56-09.png

The Source

The primary test case.

The customer tests module.

The config module.

The customer list module.

The sample credentials module.

The Walk Through

I’ve already written articles on how to setup Selenium on JavaScript to drive Safari (SoJa / Safari).   It requires Safari 10+ with automation enabled plus NodeJS with the Selenium Driver and Webdriver modules installed.  I run on a MacOS from within phpStorm. You’ll find my examples and screen shots based on that environment.

The Setup

The setup has been covered in prior articles as well.  You can review those for details.  The short version of this step:

  • load the config module using the NodeJS require() method
  • add some aliases for the web driver modules we use most often
  • configure logging via the logging module
  • start up the webdriver to get our NodeJS environment talking to Safari and set the browser window size and position

Starting The Test

We start by logging in to the dashboard.   A standard driver.get() with our dashboard URL is going to kick things off.   We string together our Promise fulfillments via a series of then() methods.   In case you missed the prior articles the then() executes when the method they are attached to has “fulfilled a promise” (is done by failure or success).   Basically we are saying “go to this URL and when the page loads then do some other things.

Our other things?  Find the email element then type in our admin username.   Find the password element and type that in.  Then submit the form.

If you are wondering where the values are being set, that is coming from the config module that reads staging-example.js or production-example.js depending on what I’ve set NODE_ENV to when starting up my node script.    This way I can have separate credentials for live or production servers and only need to change my environment variable to switch not only which URL I’m testing against but my username/password to login.   You can read more about this and why the config file loads -real.js instead of -example.js in the configuration article.

Timing The After Login Stuff

One of the first things I started to realize is a simple string of .then commands was not going to work.  Selenium runs the browser command FAR faster than the server responds or a typical user types things in.   This means building commands based on waiting for things to appear in the browser.      As mentioned previously, the sleep() trick is a horrible way to manage this.

If we boil down the test case itself to its essence we get something like this:

If you look at the source that last submit button and click it code looks different.    Let’s get into that for a moment, but first here is that code snippet:

 

The first thing to know is that then() is a function that takes 2 parameters.  The first parameter is a function to run when the prior thing that then is attached to was succesful.   The second parameter is a function to run when the prior thing failed; we’ll get to that later.   In this case for the sake of simplicity we ignore the failure.   Another important tidbit here is to know that both the success and failure functions get passed a parameter.    That parameter is the result of the promise that the .then() is attached to; our driver.get() in this case.  We’ll ignore what that parameter actually is for now.

On the first 2 then calls the format looks a little different. What is that underscore and pointer thing?   It is syntactic sugar that keeps the code readable with less braces and brackets.   .then( _ => blah() ) is basically saying “assign the paramter we get for the success function to _ so we can get to it later and then run the blah() function with it.   Since we are going to throw away the parameter we use _ because it is not visually distracting.   So the first 2 then() calls say “ignore the parameter driver.get() spit out and just go find the HTML element with the ID email , or password, and type some text in there.

The third call after we find the email and password fields says “when you’ve succesfully finished those two things go run the following function”.    We first find the submit button element using the css type=submit locator and assign that to the previous_el variable.   Why we do that will be clear in the next task.    Since previous_el is now assigned to our submit button we can use that shorthand to click it.

Side Note: In general it is always faster to assign an element that a JavaScript library like our Selenium Driver here , or jQuery lookups, to a variable so you don’t keep scanning the DOM to locate the thing you just located.  

So we’re done with logging in.    We go to a specific site URL, when it finishes loading the page we find the email and password inputs and type in our login info, then find the submit button (that we saved ot our previous_el variable) and click it.

Onward and upward.

We Logged In, Now What?

Moving down a line we see another then() attached to our prior then that clicked the submit button.   Again we are using a more explicit form of the success function.   Why?  Because we want to log something to the console to tell us the login worked.   Not critical, but it is nice to know your tests are doing what you expect even when they are working.

You’ll also notice we are now using a new until.stalenessOf() method in this function.    This is where our previous_el comes in.    Selenium WebDriver has a timer-based method built in that will loop around until a specific condition is met or until the specified timeout has been reached.  In this case we are going to wait until our submit button from the previous page has been marked “stale” for up to 6 seconds (config.setting.standard_wait = 6000 in our config file).

Now you know why we recorded the WebElement for the submit button in our previous then test when we found it.     Using stalenessOf( element from a prior page ) is a great way to ensure you’ve left the previous page.    It does not require that you know anything about the next page you are about to load.     We’ll worry about that in a moment.

Loading and Testing The Customer List

Now that we’ve performed the login and waited to ensure we got past the login page we can now continue with our next step in the original driver.get().then() sequence.   Here we are going to go a little deeper into the then() nesting by “latching on” to the a new driver.wait() call.

The essense of the code:

OK, that is a LOT simpler than the code we have written.   Let’s dig a little deeper and expand on the “run our list testing function” statement to this:

Here it is important to note that we do not just string along another then() on our original driver.get() series.    It is important to specifically wait not only for the new customer list page URL to load but also to ensure that page has content.    For this example we are going to assume that if the header tag “MySLP Customers” is rendered that our customer table is complete.  In reality we will check for a specific HTML marker that is output only after the customer list is fully rendered.

The code is loading the URL then calling a function to do a lot more stuff.   That function says “don’t do anything until we see that MySLP Customers header”.     We use the standard Promise .then() method once we do see that  header to start our processing.  The firs thing we do is output something to the log to tell use the customer list page is open and ready for testing.

Testing The List

Let’s start with the code that does the test plus our couple of actual code lines outlined above.

What is var customers = require() doing?

It is loading our active_customers module.  That module sets up a JavaScript object that is assigned to customers that we can later reference.  This lets us later update the active_customer.js module in our config directory so we don’t have to hack up the main test code to add users.   It keeps our future edits somewhat isolated and will allow us ot later use other non-code-editor tools to allow our QA team to easily load a “check this user” list.    In this case the active customers we want to test will end up in the customers.active[] array.  It is an array of objects with a username key that holds the usernames we want to test.

Next Up: test_customer…

We assign our customer_tests library of customer test functions to the test_customer constant (we could have used a var just as easily) which gives us access to all our re-usable test methods.    The library, outside of loading up a bunch of our shorthand aliases and enabling logging for our module scope, provides two simple methods.   An on_list() method and a reset_tested() method.    reset_tested() only re-sets our test counter to 1, fairly simple and not really doing much in this single test.

on_list() is passed the driver, config, and customer name we are looking for.  Driver and config are only so we dont’ have to re-invoke another copy of each within every submodule.   Customer is the string that will change every time our loop moves to the next customer.

The guts of the customer_tests.js module:

Within the function we do a simple “find a TD tag that has text containing the customer string we are seeking”.   You can see that we are using the more advanced form of then() where we not only have a success and failure function but also name our parameters.    If the thing we are looking for is found within 6 seconds we print out a found message with the customer string preceding by a count of how many we’ve found so far.    If the TD with the customer cannot be located on the page we log a could not found message instead.

When It Runs

Our NodeJS app fires up a Safari window, goes to the production or staging site, logs in our admin user, goes to the customer list page, then looks for a dozen-or-so key customer accounts to ensure they are still on our customer list.   We wait 10 seconds then close the test Safari browser.

This is what we see in our phpStorm NodeJS test execution window when the test runs:

phpStorm NodeJS Customer List Test Output 2017-09-26_22-04-47.png

%d bloggers like this: