Art Of Test WebAii Looks Promising For Testing The Web

0 comments suggest edit

A coworker of mine ran into some problems using WATIN to test our website. Specifically issues with Javascript and AJAX. Yes, I know there’s Selenium out there, but I hoped for something we could run from within NUnit/MbUnit, so that it’s nicely and easily integrated into our build process and development environment.

He did a bit of digging and found this free web automation infrastructure product by Art Of Test Inc. called WebAii (they also have a blog). It seems to do everything we want and more. Check out some of these features (the full feature list is much longer).

  • Supports IE and Firefox
  • Supports DOM actions and pure UI Mouse/Keyboard actions for Ajax Testing
  • For ASP.NET testing, you can use an In-Process Host for fast browser-less testing without a webserver
  • Use ASP.NET Development Server (aka Cassini) to run tests without IIS
  • Use name, id, xpath identifications. (Need to see if they support Regex too)
  • Unit test your JavaScript by calling JS methods from your .NET code.
  • Extensions for NUnit and VS Team Test (I’ll ping them to build one for MbUnit)

These are just a small sampling of the many features. It sounds like this would be a killer web testing app. Does anyone have experience with it?

We’re going to evaluate it and I’ll keep you posted, but I thought I’d mention it here because it sounds great.

Features I’d Like To See In A Web Unit Testing Tool

  • Ability to set the UserAgent and Referrer
  • Ability to set request headers
  • Ability to hook into and verify the status of a redirect. For example, if a page redirects, I want to be able to assert that the HTTP Status code for the redirect

Technorati Tags: Unit Testing, TDD, Web Testing, Web Automation, WATIN, WebAii

Found a typo or error? Suggest an edit! If accepted, your contribution is listed automatically here.

Comments

avatar

20 responses

  1. Avatar for Kevin Dente
    Kevin Dente August 1st, 2007

    Looks intriguing. I look forward to your report.

  2. Avatar for Joe Brinkman
    Joe Brinkman August 2nd, 2007

    Myself and a few others are looking at this for DotNetNuke testing. One downside on this initial release is that there is no script recorder. For UI testing I think that a recorder is very beneficial so that non-developers and general community members can create scripts without fully understanding the page framework or the testing tool. Even if the recorder did 80% of the work and then let me edit the script to add in some assertions or tweak the script to make it more resilient to change.

  3. Avatar for Dave G
    Dave G August 2nd, 2007

    We've been using WebAii now for a week, with a view to getting a lot of UI tests up very quickly. We chose it purely for it's ability to abstract the id's of controls outside of the tests, and for test regions, that should allow us to make our component tests reuseable for testing functionality when integrated into a site.
    I wouldn't worry too much about the MbUnit testing extensions. I did have a crack at trying to write my own MbUnit extension, but found most of the hooks for doing this are internal methods. Apart from the new attributes for test cases (Dialog and another one), all the other shortcuts stuff works fine in MbUnit, if you just use the NUnitTest template. If you want to use the Initialize() call with some non default/config settings though, you can just pass in Console.WriteLine() instead of the NUnit.Logger method.
    Although WebAii are looking at building a script recorder tool, I'm looking at modifying the WatinRecorder to allow us to record WebAii tests.. so far it looks like it'll be a fairly easy task.

  4. Avatar for Dave G
    Dave G August 2nd, 2007

    Thought I should add a few things..
    We've quickly noticed, there are quite a few things in WebAii which can be painful to achieve, compared to previous testing frameworks we've used. Things like checking visibility of controls (i.e. for ASP.NET validation with client side script .. among many other cases). I've already asked for that one, but been told it's not likely for v1.0. Even the VERY regular tasks of checking for text on a page is the unreadable affair, of:
    Assert.IsNotNull(Find.ByContent("p:sometext", InnerText));
    compared to, say in selenium:
    Assert.IsTrue(isTextPresent("someText"));
    BTW just to add, there is nothing stopping you from driving Selenium tests from MbUnit, or CruiseControl .. We've been doing that for a while now as native C# tests, similar to WebAii/WATIN. The FireFox IDE/recorder will also generate NUnit C# tests as well as Selenese/HTML (and easily modified for MbUnit), though this is not obvious without a bit of digging. To drive from xUnit, you need to get the Selenium Remote Control package for .NET, and fire up it's java service before running the tests.
    I've got a NANT script for doing all this (bit tricky as the java server needs to be spawned .. something not easily done in NANT at the moment), and examples of MbUnit/ASP.NET helpers if anybody is interested.
    We've just decided to try out WebAii as it is a bit lighter, and easier to setup, and has much better support for us reusing tests ... we're missing the recorder though!

  5. Avatar for Vadim
    Vadim August 2nd, 2007

    There's a new kid on the block, SWAT. http://sourceforge.net/proj...
    SWAT stands for Simple Web Automation Toolkit.

  6. Avatar for Jay Kimble
    Jay Kimble August 2nd, 2007

    Hey Phil,
    Check out InCsif.net (trhat's the product name and the URL). It's very similar and it has recording capabilities. I actually found it to be superior to WebAii. The only downside for InCsif.net is the price. If you are looking for free then this won't fit the bill.
    Jay

  7. Avatar for Tod Birdsall
    Tod Birdsall August 3rd, 2007

    I have been pretty happy with SWExplorerAutomation (SWEA). I have been using it for over a year and comes with a recorder. Although it is not free, it is inexpensive at $79. The bugs that I have run into have been quickly fixed by the creator (Alex Furman).

  8. Avatar for Kevin Michalsky
    Kevin Michalsky August 3rd, 2007

    Phil,
    I have to agree with you, this is very exciting. I downloaded the RC0 version that is listed on ArtOfTest's Website today and I cannot stop digging into it. Their RC0 now has an HTML control set in it. You can do Generic finds and have a strongly typed html control....VERY NICE!
    Thanks for the find!
    Kevin

  9. Avatar for Jeroen van Menen
    Jeroen van Menen August 3rd, 2007

    Hi Phill,
    Could you (or your coworker) explain some more about the problems you experienced with WatiN. Love to improve WatiN even if you decide to move over to another web testing framework (although I think WatiN being open source gives you all the posibilities you might ever need).
    Thanks in advance,
    Jeroen van Menen
    main developer of WatiN

  10. Avatar for Haacked
    Haacked August 3rd, 2007

    @Jeroen Thanks for commenting. Here's the list of issues:
    Details of WatiN issues
    - WatiN doesn't focus control (control.focus() doesn't work)
    - WatiN doesn't send events for typetext
    - Watin doesn't set keyCode event parameter when keyDown/keyPress/keyUp are sent
    The problem I had was I couldn't figure out how to verify a document.write statement. For example, I wanted to make sure that a document.write statement was executed.

  11. Avatar for Haacked
    Haacked August 3rd, 2007

    Also, the Firefox, IE, and Safari on Windows testing is useful for us.

  12. Avatar for Ftorres
    Ftorres August 3rd, 2007

    InCisif.net has already been mentioned, thanks Jay Kimble.
    InCisif has a record mode and also a different Api from WatiN or Selenium.
    We ship with the product a lot of samples in C#, VB.NET and IronPython.
    We have a lot of videos on our web site - www.InCisif.net.
    So you can judge for yourself.
    InCisif.net does not support FireFox or Safari yet, that is a plus from WebAii.
    InCisif Software is giving away free licenses for personal this summer.
    Don't be afraid to try!

  13. Avatar for Steve Trefethen
    Steve Trefethen August 3rd, 2007

    FWIW, Selenium does integrate with NUnit and in fact I just got done adding a bunch of Selenium/NUnit tests to our CI server. Check out Selenium RC and it's .NET support. I've written a blog post about it here.
    I haven't tried WebAii so thanks for the pointer, I'll definitely take a look.

  14. Avatar for Jeroen van Menen
    Jeroen van Menen August 3rd, 2007

    Phil,
    Thanks for the info. The keycode issue is fixed in the current developement source and will be part of the next release. I will have a look at the other issues you mentioned.
    Regarding the document.write issue. There must be something that has changed on your page after this statement has been executed. The simplest check you could do is IE.ContainsText("").
    Supporting other browsers is on my TODO list but very near to the end of it.
    Regards, Jeroen

  15. Avatar for Haacked
    Haacked August 3rd, 2007

    It seemed to me that IE.ContainsText was finding the text every time because it found it in the original document.write statement. For example, with something like this:
    <script type="text/javascript">
    if(false) document.write('blah blah');
    </script>
    It appeared I was getting true for IE.ContainsText. When I get a chance, I'll try out that scenario again to double check.

  16. Avatar for Jeroen van Menen
    Jeroen van Menen August 4th, 2007

    Hmmm, your right about that.

  17. Avatar for Kieran
    Kieran March 30th, 2008

    Does anyone have any more opinions on webaii versus other testing frameworks? I've been using webaii for about 3 months though, and like it, but have never tried anything else. Is it worth learning something new or sticky with webaii?

  18. Avatar for DriveU
    DriveU April 30th, 2010

    WebAii is Free ?
    Who says that?
    Could someone clarify this?
    Telerik is advertising WebAii or WebAii is a product from Telerik?
    But This tag shows a Whopping $2K+
    Using WatiN with one of the Unit testing Framework should be enough to solve and manage automation this is my experience.

  19. Avatar for Nathan Bain
    Nathan Bain November 15th, 2010

    Telerik have confirmed that the WebAii testing framework will always be free.
    See here:- www.telerik.com/.../is-it-free.aspx
    Commercial versions (with test recorder etc.) are also available.

  20. Avatar for Justin Collum
    Justin Collum July 5th, 2011

    So WebAii doesn't have a test recorder out of the box? Looks like Selenium is still better -- a working test recorder is what sets Selenium apart from every other func tester I've seen. Also, getting Selenium to kick out tests in your test tool of choice is just a matter of altering the formatter. I've got one around for C# / VStest if you need it (I may have posted it on the Selenium newsgroup).