How to start using Testissimo

Add the following snipped to your index.html

<script id="testissimo" src="https://app.testissimo.io/testissimo.min.js"></script>

or use Google Chrome plugin: https://chrome.google.com/webstore/detail/testissimo/kbndfdpfemdihkbgpaggicjhmfaeeobh?hl=en

or FireFox plugin: firefox-0.2.8

Testissimo should appear overlapping your application and prompting you to login.


After you have successfully logged in, you will need to create new repository (or clone existing one in case someone already created it and assigned rights to you. The repository is named place where you store all your tests, components and everything you have done. It is like GIT repository but build-in to Testissimo infrastructure. Your tests are stored safely in cloud and are accessible from everywhere.

In “Repos -> Remote” section you can either checkout existing repository or create a new one.



Writing tests

Targeting HTML components

When you write UI automation test, most of your work will be telling your test automation tool how to find HTML elements which you want to interact with. This is no different with Testissimo. In Testissimo you can use full set of CSS selectors and some build-in helpers to always target element on page with ease.

As CSS selectors are standard feature in web development you will find lot of resources on the internet dealing with how to write them and how they work. For absolute beginners, we recommend playing this game which can train the basics of CSS selectors: https://flukeout.github.io/. As cheat-sheet we like to use this website where there is a full description of CSS selectors capabilities https://www.w3schools.com/cssref/css_selectors.asp. Testissimo supports all of css3 selectors and adds custom, extended syntax to fulfill all searching needs.

Selector Combinators

Standard css combinators

  • " " all descendants (has alias ">>")
  • ">" direct children
  • "+" adjanced sibling (first direct next sibling)
  • "~" general siblings (all next siblings)

Extended combinators

  • "<" direct parent (alias for "!>")
  • "<<" all ancestors include parent up to HTML document (alias for "!")
  • "^" first direct child (alternative is "> :first")
  • "!" all ancestors (has alias "<<")
  • "!>" direct parent (has alias "<")
  • "!+" previous adjanced sibling (first direct previous sibling)
  • "!~" previous siblings
  • "!^" last direct child (alternative is "> :last")
  • "~~" all next and previous siblings - not implemented yet
  • "++" next and previous direct sibling - not implemented yet

Attributes Matching

Extractable - when drag-dopping component from preview in app into test, some variables from selector definition can be suggested to user, e.g. input:value({variable}) can be extracted from element and suggested

Same as standard css attribute matching, but allways true if value is missing or empty, e.g. [attribute=""] or [attribute*=""] is same like [attribute]

  • [attribute] - has attribute, value is irelevant (extractable)
  • [attribute=value] - attribute equals to value (extractable)
  • [attribute~=value] - attribute value is in space separated list (extractable)
  • [attribute|=value] - attribute value is in hyphen separated list (extractable)
  • [attribute^=value] - attribute starts with value (extractable)
  • [attribute$=value] - attribute ends with value (extractable)
  • [attribute*=value] - attribute contains value (extractable)

Extended attribute matching

  • [attribute!=value] - attribute not equals to

For strict selection, e.g. if you want to select element with attribute equals to empty value "", use double "==" syntax

  • [attribute==value], [attribute*==value], [attribute!==value], etc…

Extended Pseudo Selectors

Position in current selection (not in child elements, but in all selected elements in current selection)    
  • :first - first in selection
  • :last - last in selection
  • :even - every even element in current selection
  • :odd - every odd element in current selection

Positional selectors with argument value, for scaffolding reason, if value is empty, it is allways true, e.g. index() or :index will match every element    

  • :index(0…9) - zero based position number (extractable)
  • :order(1…9) - same as :index but starting from one (extractable)
  • :eq(0…9) - means equal, it is alias for :index (extractable)
  • :gt(0…9) - greater than
  • :gte(0…9) - greater or equal than
  • :lt(0…9) - lower than
  • :lte(0…9) - lower or equal than    

Inner text selectors, will select all elements containing text, not only nearest but all ancestors, so it is often used with pseudo :first, :last    

  • :contains(text) - contains substring - (extractable)
  • :text(text) - equals substring - (extractable)
  • :text-contains(text) - alias for :contains - (extractable)
  • :text-begins(text) - begins with text (extractable)
  • :text-ends(text) - (extractable)
  • :text-regex(regextext) - (extractable)    

Allowed boolean values are false,False,no,No,0 and true,True,yes,Yes,1    
Visibility selections, based on element or ancestors "display:none" style

  • :visible, :visible(true/false) - (extractable)
  • :hidden, :hidden(true/false) - (extractable)    

Input state selectors

  • :enabled, :enabled(true/false) - (extractable)
  • :disabled, :disabled(true/false) - (extractable)
  • :checked, :checked(true/false) - (extractable)
  • :selected, :selected(true/false) - (extractable)
  • :focused, :focused(true/false) - (extractable)

Selecting by element value property

  • :value(value) - (extractable)
  • :value-contains(value) - (extractable)
  • :value-begins(value) - (extractable)
  • :value-ends(value) - (extractable)

Location url based filtering - usefull if you have many components, but not all of them are suitable for some pages, it helps to filter them    

  • :url-contains(text)
  • :url-query-contains(text) - exclude "?"
  • :url-hash-contains(text) - exclude "#"
  • :url-equals(text)
  • :url-query-equals(text) - exclude "?"
  • :url-hash-equals(text) - exclude "#"

Variable based filtering - usefull if you have multiple comma separated options how to select element, and you have to switch between them, e.g. input:value({value}):if-var({value}), input:index({index}):if-not-var({value})    

  • :if-var({variable}) - allways true in suggestiom mode (because variables are not yet defined)
  • :if-not-var({variable}) - allways true in suggestiom mode (because variables are not yet defined)

Build-in tools

This is a set of basic build-in actions which can be used to create more complex constructs like component methods. These actions are the Testissimo language which all other component are composed of.

Most of the simple action has a CSS selector which defined the html object on which this action will be executed. If the selector is omitted, the selector from owning component or the last selector from previous execution is used.

As of the current version, the set of Simple actions is given by Testissimo it-self. They do have an undocumented API and thus even custom simple action can be plugged into Testissimo. If you are interested in doing so, please contact the Testissimo development team for further information.

Fill file input

TODO

Keyboard (Write/Key press/Focus)

The action has tree variants. It can be used

  • to simulate typing sequence of character on some html object
  • to simulate typing single character on some html object
  • to focuse some html object

Following example demonstrates a component method which is writing sequence of character into an INPUT html object


Mouse click (Left/Middle/Right/Double)

The action simulates click on some html object. The action has four variants depending on which mouse button you want to click and if the click should be single or double.

The following example demonstrates component method which click on a button identified by its label


Navigate to (Custom URL, Back, Reload Page)

The action can perform various navigation tasks. It can

  • navigate to custom url, which is specified as its parameter
  • simulate the back button click
  • reload the current page

The following example demonstrated the navigating the application to some relative path


Select option

TODO

Wait

The action forces Testissimo to wait for certain amount of time. In clear majority of cases, you will not need to use this action. Testissimo will do most of the synchronization for you.

Extract element values

TODO

For each

Using this construct, Testissimo can iterate over some collection and execute some set of actions for each element of that collection. There are various ways how to define the collection. The options are

  • Parent component occurrences
  • Html elements defined using selector
  • Values defined in the semi-colon separated list
  • JSON (in the form of array of object)
  • Using values in variable which is of type array of objects

If – Else

Using this construct, you can implement decision making in Testissimo.The decision can be based

  • on some properties of HTML object
  • on count of HTML object
  • on the URL parameters
  • on the value of some variable

In the following example, the action check if the variable ‘do-cleanup’ has the value true and if so perform call to some method


Section

This construct can be used to define some section on the HTML page. This is very like component definition (we encourage you to almost always create component instead of section) with the difference that section cannot have method of its own. Using section, one can fix constant part of CSS selectors for underlying actions.

Select element

TODO

Set Variable

Using this command, user can set (or declare and set) some variable which can be later used as a method parameter of for decision making in some if statement. Using this command, user can save some HTML properties if he needs them later in the test to make for example some assertions.

Assert Elements

Using this command, user can assert if some HTML elements parameter has a given value. If this condition is not met, Testissimo stops the test execution and related error message is shown.

Assert Elements Count

Using this command user can assert HTML elements count for given CSS Selector. This can also be used to check the existence of some HTML element in the given context.
In the example below, Testissimo will check if there is exactly one ‘strong’ element within the given ‘Document grid’ component




Assert URL

TODO

Assert Variable

TODO

Parameters

In the examples above we have seen cases where we need to parametrize some CSS selectors or we want that the caller defines the parameter value of some other action parameter. We can use parameter for this purpose. Parameters are parsed automatically from actions and CSS selectors using the {parameter name} syntax.
In the example below we can see parametrized CSS selector and parametrized method call setValue.



When using this method, we can provide parameter values which will be used during the test execution



Page Object pattern and Components

As soon as you have at least one repository and you know the basic set of Testissimo tools and action which are described in chapter

Build-in tools, you could start writing test. However, the test you would produce would be very be of poor quality, minimal code reuse and wouldn’t be very resilient to any UI changes. To produce test of high code reuse and thus test with low maintenance cost Testissimo adapt the Page Object patter and implements it in very intuitive and easy to use way. To understand the idea behind Page object patter we advise you to read following article written by Martin Fowler (https://martinfowler.com/bliki/PageObject.html) and also to read the following chapter: Test maintainability.

Component in Testissimo can by anything from very simple things like button and test fields all the way to complex Grids and component which are very specific to your application. The component is defined by its selector (which can have parameters) and set of method which can be performed on this component.

Simple button component



This is an example of button component which is defined by its selector:
button[class*=”btn”][@text*={text}] and which has one method “click”. The component as we see in here is just the definition. It can be executed in development phase for testing purposes but it doesn’t do anything unless you use it in test. To see if the defined component is on current page, you can click the eye button next to the component name and Testissimo will highlight all occurrences of that component


You can choose which button you want to act on, choose the method and drag drop the method to the test


You will end-up with simple test which click on the “Search” button


You can execute your test by clicking on “play” button in the right corner.

By using components rather than writing test using simple action, we can achieve that all CSS selectors are defined just ones so for example if some completely redesign some grid, we also need to only redesign this grids Testissimo component. No matter how many tests are using this component, if they interact with it inly using this component, they will stay unchanged.

Furthermore, using component we can create sort of DSL specific to our application and Test can written on much higher level of abstraction than if they would just use simple low-level action. This way we can achieve great amount of code reuse and
nice readability of our test.

Complex component and components composition

As soon as we have defined the basic components like button, text fields, combos and so on, we can start defining more complex component which are very specific for our application. Of course, we can use already defined simple component (or any
component) to implement methods of these higher level component. Let’s look at the following example


The application we want to test has one grid which show the documents matching the filter criteria. Let’s call it “Document grid”. As the content of this grid are also rather complex component it wouldn’t be good design if we would create all the methods in the “Document Grid”. Let’s create two more components. One which represents the row of this grid and thus one document with name “Grid document”. Second which represents the component for document tags manipulation with name
“Document tags editor”.

As “Grid document” always exists in the context of “Document grid”, we don’t need to be too specific in its CSS selector definition, because it should always be used in test as part of “Document grid” component and its selector must be selective
only within this section. The component definition could look like this:


We use document name as the CSS selector parameter so if we want to use it in test we need to specify this name so that the component targets exactly one row. We can also use it to implement method of some different component. For example, let’s implement delete method on “Documents grid” component.


We see that to delete one document, we first need to select the document by checking the checkbox for that document. We can easily use our “Grid document” component to accomplish this task, propagate its method parameters to our caller by simple defining the variable name in {} as the parameter value. Clicking on the delete button is also accomplished using another component which we have already defined. To use this method in test we simply need to call it and provide the document name parameter like this


Another way how we can use components composition is using the components to narrow the selection scope. Imagine that you want to start working with tags for some document. In order to correctly select the right “Document tag editor” we would
need to make its CSS selector quite complex to be always able to identify the right tag editor in the whole result set. To avoid this complexity, we can use already defined component to narrow down the selection space. Let’s use component “Documents grid” in test without method just to define section on the webpage. We can better visualize the section be clicking on the eye icon next to the component name.


Let’s further narrow down the scope to single document by using component “Grid Document” with name of some specific document.


And now we can start manipulating tags with the “Document tags editor”


If we look at the selector definition of “Document tags editor” it is very simple, just div[multiselect-doctags]. If we would use this component outside of “Grid document” component, the selector would need to be much more precise. Using
component composition we can save quite some complexity in the CSS selector and have one aspect of CSS selection defined just once.

Marco

Sometimes we need to define reusable piece of execution which spans more than one component which means that it is not convenient to implement it as method of some component. For this purpose, we can create macro which is simple a set of
component method execution or simple action which can be reused. As any other method, macros can be parametrized.

Test

Test is our final construct where we put the logic of single test. Technically tests and macros are equal so they have the same properties. Test can be organized in executable groups called Test suites which can be executed with a single click.

Test execution

You can run any test by clicking on the play button in the right corner of the test. In development phase of some test you can also run single step of the test by clicking on the item “Run only this step” from the context menu


In development phase, you it is also convenient to run methods of component even without using them in test. You can to this using the “Run” menu item from the context menu of the chose component method.

Synchronization

Synchronization is important part of writing GUI automation test. The web applications of today are very dynamic and can change their HTML almost any time. In the test tools like Selenium, which don’t have much information about what is happening in the browser you need to handle synchronization by yourself and use wait for element commands to stop test execution until browser updates the page.

In Testissimo, you don’t need to bother about synchronization for most of cases. Testissimo runs in the same browser as the application and has good information about if there are server call which have not yet returned or if the browser is doing some rendering task. That is why Testissimo will always wait, even if you don’t specify it explicitly until the performing the next step. This greatly simplifies the writing the tests and overall maintainability because you don’t need to prolong your waits if running on a slower hardware.

The only cases where you will need to handle synchronization by itself are the cases when even the browser doesn’t know. This are things like server push or when the application uses functions like denounce.

Test maintainability

The biggest issue in software testing is tests maintainability. The less effort takes to fix tests if tested application change, the happier development team, and software development speed is.

Let’s assume you have simple book search web application. There is only search input and books result list. How will be logical steps of simple search test looks like?

  1. Do search “some book”
  2. Assert book result list – it should contain only relevant results with no data missing

Now, imagine pseudo code of this test:

  1. Select search input by id=”books-search”
  2. Select book result list container by id=”result-list”
  3. Select all result items inside result list containers
  4. For each item in result list, do:
  5. Select field name by some selector
  6. Assert field name value
  7. Etc.

Now, let’s do some changes in tested app, and see how would our test handle it:

Redesign of search, now it has another id, and search suggestions, so it acts more like drop down, and has input wrapped by another element, which are generated, input has no more id attribute

Removed some book fields, added new, with conditionally hiding/showing behavior

Tests pseudo code must be changed entirely, but logical steps doesn’t. In real world, there are no few and simple tests, but hundreds or thousands, robust tests with similar and repeated behaviors, like search in categories, tags, search in
authors, search by ISBN, etc. Rewriting it entirely is very time consuming.

Maintainability with component based approach

In Testissimo, maintainability is provided by reusable components and their methods. Best practice is to follow logical steps. So, in our book search example, it would look like:

We will create component “Search-Input” with methods

  • “doSearch” with parameter “searchPhrase”, with actions:
  • action 1: select search input inside search drop down
  • action 2: keyboard write text {searchPhrase} into search input
  • action 3: assert search input value, it should be equal to {searchPhrase}
  • action 4: select start search button, or keyboard keypress enter

And Search-Result-List component with method

  • “assertResults”, with actions
  • action 1: …
  • action 2: …

Now, we can write test like this:

  1. Search-Input.doSearch ( searchPhrase = “some book” )
  2. Search-Result-List.assertResults

If we redesign web app, we need to do changes only in tests components, in one place, not in tests. Therefore, components are crucial for tests maintainability.

Maintenance using macros

If you are repeating some application flow in tests, like login, or fill user profile, or add product to cart, etc. where this flow span more than one component, it is better to use macros. Macros are reusable test part with parameters. Imagine,
you are testing ecommerce cart, with several payment and delivery options, physical products, goods, localized offers, etc. You don’t want to rewrite all tests because of someone added cart confirmation step. You just need to update one macro “make order”.

Run anywhere

TODO

Build-in version control system

TODO

Test reports

TODO

©2017 Objectify s.r.o. All Rights Reserved.