Complete Custom Test Automation Framework - Using C#, MS EXCEL and Telerik Testing Framework - Part 1

In this series of posts, I will share with you a complete custom test automation framework that I built using Excel as a test driver, coded in C# on top of the Telerik Testing Framework, that you can use for your own test automation. You are free to extend this solution subject to GNU Public Licensing.

Okay, lets start....First off, when designing a test automation framework, you need to understand the AUT (or application under test) that you need to automate. Based on my personal experience (your's may vary depending on what your application is etc.), I have found that Unit or Acceptance tests are best when you have more access to your application's code and is a lot more stable and easier to maintain.

I am a huge fan of this diagram, and it's a great starting off point when discussing options available to you when you need to start writing your own automation framework (source: http://www.velocitypartners.net/blog/2014/01/28/agile-testing-the-agile-test-automation-pyramid/):

Fig 1- Agile Test Automation Pyramid

However, there are a lot of times too that you may need to write UI automation tests (integration tests) and understanding which tools out there suits your need, depends on several factors.

I've compiled a table below of some of my personal experiences with these tools:

Test Automation Tool	Pros	Cons
Coded UI	·        Tightly Coupled with Visual Studio. ·        Great for recognizing UI objects especially WPF controls and MS-native controls. ·        Can be integrated with other MS Testing tools – Load Test, Unit, Web Performance Tests. ·        You can code using either C# or Visual Basic	Tightly Coupled with Visual Studio. You can’t use it outside of Visual Studio. Needs some work in terms of recording and playback, especially when debugging code and managing multiple UI maps.
Test Complete	·        Superior object recognition capability. ·        You can code using a variety of scripting options – C#Script, Jscript, VBScript, etc. ·        Can be integrated with other SmartBear tools like AQTime (used for profiling and detecting memory leaks). ·        Has great UI for recording/playback and debugging code.	Not open source, no free option.
Telerik Test Studio	·        Superior object recognition capability. ·        You can code using a variety full blown C# code or Visual Basic code which is awesome if you are already coding in either languages ·        Has great UI for recording/playback and debugging code. ·        Works awesome when recognizing complex Telerik UI objects. ·        Has an open source testing framework – Telerik Testing Framework that can be used outside of the Visual Studio integration.	Has some minor nuances that you need to be aware of. A bit on the heavy side and comes with a lot of software/tools that you may not need or use.

I did not include in this table Selenium because although it is great, and would be my first go to tool if I'm doing test automation for a web-based app. It is not a full fledged tool (does not have a strong UI support for debugging your code), and obviously if you are learning to do automation, this can be difficult at first to learn as it forces you to code right away. QTP may be the grand-daddy of all UI testing tools out there, but I find a lot of issues with QTP when recognizing new UI controls. Plus it is extremely expensive compared to these other tools. So, unless you have very deep pockets to invest in you test automation right out the gate, this may not be the right tool for you.

After looking at the pros and cons of using these tools, I've decided to pick and use Telerik as my solution because of the following:

1. Tight integration with Visual Studio. And since the web-app that I was tasked to automate was built in .NET, this was a natural choice for me.

2. You have C# language behind you. You're not limited to just a scripting language, but the full stack of C# .NET technology behind you that you can use at your disposal. For example, you can use .NET reflection here with relative ease.

3. Has an open source option in the form of the Telerik Testing Framework. You are free to create and distribute your code (of course subject to some restrictions in the EULA) as you need.

In my old job, my primary responsibility as Senior Test Automation Engineer was to evaluate tools and recommend the best technology stack appropriate for our needs (and obviously also cost effective at the same time). Which is why we have a lot of unit tests, and SDETs code integration/acceptance tests using Specflow and MSTEST, and of course Telerik Test Studio as our UI testing solution.

In this post, we will start looking at how a similar (because the one that we're actually using back then for our company obviously cannot be made public here and of course uses a more advanced hybrid solution) custom test automation that I earlier built. 

Lets get started...

First you need to have Visual Studio 2012 or better installed in your machine. Start Visual Studio and create 2 Projects in Visual Studio. The first project is a class project, name it as CustomTestController. This project will contain all your code for running your test cases, we call this your Test Controller. 

Next, you need to create a second Visual Studio project, this time you need to create a Console Based Application Project. You should name this project as TestRunner, and this project will basically be your test runner, responsible for instantiating your Test Controller and managing your environment.

To understand further the design of this custom test automation framework, I created a diagram below with the different components working in this solution:

Fig. 2 - Custom Test Automation Framework Design

Other parts of the solution - Test Driver, Test Cases, Test Config and Object Repository are XML based files that are built and managed using EXCEL. In the next post, we will delve more into the structure of these files as well as how to manage them.

By the third post, I will post in Github the full source code for this solution, that you can fork and use as you need. Thanks and see you next time.