Friday 30 May 2014

Good Bye Blog !!

I have migrated my Blog to a new address http://pankajmalhotra.com Readers stay tuned there's a lot more to come ahead !!

Friday 16 May 2014

From a PIE to an API !!

Last week we were pretty much concerned about what to use and what not ?
and what changes we shall introduce in our model of Testing ?

So may be if you have curiosity about what we used and what we decided, This post is for you !

Overall Scenario last week: We wanted to use FactoryBoy for database object modelling and reliable data for tests.
So using FactoryBoy meant that we had to keep our Functional Tests with the application itself.
This was the change we decided to have and pretty much kicked of the work and initialized the test objects in application repository.

So what next ?
In our old testing model we ran the Functional tests on staging/production application already running. So now the challenge was:
How will we trigger the server ?

One of the solution which instantly came to mind was using LiveServerTestCase 
So things were *almost* finalized !!

But things come to mind as we start working, and this time it was not something which could be ignored.
Man, we would not be able to run the tests on Staging and Production or in general any other remotely hosted clone of OneAndDone, Also running the tests locally would mean setting up of a local instance of OneAndDone. This was what came to mind as soon as I began writing the first test and it was for sure not what we wanted, because  of mainly two reasons:

  1. Why will a contributor developing a test case want to set up the application locally ? We definitely don't want to assume our contributors to be Application developers. So this was a hindrance for contributors.
  2. We wanted to run the tests on staging and production because it gives us a better idea of current state of Application.

So we decided to give it a rethought.

After many discussions with other team members it settled down to two options:
  1. Keep the tests local for better data population and faster test runs.
  2. Have the tests in a separate repository making it easy for contributors to work on and to make us able to run the tests on stage/prod.

Finally we decided to go with the second one because of the above mentioned limitations of first, or the benefits of second overpowered the benefits of first. We realized our current model to be more advance than most of the other methods discovered.

The main changes we decided to have in these tests that make them better and advance than the tests for most of the other Web QA projects are:

  • Develop and Use a REST API which interacts with database objects remotely. So it helps us to create/delete the test data. For developing this API we decided to use Django REST framework over tastypie for some reasons based on prior experiences of developers.
  • We decided to use pytest fixtures instead of SetUp and TearDown methods which will allow creation/deletion of mock objects using the REST API.

We again kicked off the work by initializing the Page Objects and Base Class and opened some Test Case bugs.
Now I am looking forward to developing the REST API and start writing the tests next week.

Sunday 4 May 2014

We need it !! Oh wait... do we ??

Ok after stumbling upon various IRC channels, having talk to some top most QA testers, Automation geeks, and  API developers this week I am here with a few thoughts. So this week was a part of my research period, My main goals for this week were :


Goals for this week
First Task
This week I will mainly concentrate on exploring and researching about the model of the testing that we are going to use for this application. Whether a change is required or not ? If yes are we in a position to make that change ? If no , explore more about other methods that can tackle the problem for database object modelling.


  • Explore more about the number of tests that require modelling of database objects to make sure that do we really need FactoryBoy and how will it be useful.
  • If we need FactoryBoy then can we merge the code for tests with the application code ? Explore advantages and disadvantages for merging the code.
  • If we cannot merge the code are there any other alternatives that will allow better modelling of Database objects ?


  



After answering these questions with concrete arguments and proofs I would like to discuss it with the team and may be have a poll ?
Some of the references I am exploring:

I will also like to explore the problems and limitations(if any) that might arise if we are ready to use FactoryBoy using past examples from Rails applications testing using FactoryGirl and see if we have a solution to them.

Second Task
The second task I will like to take up this week is to discuss with lizzard, pragmatic, rbillings, marcia and team to come up with a more specific design for template for task creation as discussed in meeting yesterday.

The template would later help to autofill the tasks after scraping bugs data from other API’s,
inturn helping in Task creation automation that I proposed in my proposal.

Along with this I will use this time to become familiar with OneandDone codebase.


So I have roughly achieved most of the tasks on the list, and have explored many resources related to Test Automation.

Here are some resources that I went through and found really very useful

1 Talk on testing django applications using pytest by Andreas Pelme(who himself is an author of pytest-django) Link to Video
2 Talk on Integration and Functional Testing using LiveServerTestCase and Selenium Link to Video
3 A Presentation on Testing and Django Link to Slides

I also came across some of talks from very experienced Python developers who highlighted the advantages of pytest for easier and better testing.

1 Holger Krekel: Improving automated testing with py.test Link to Video
2 Andy Todd: Why I use py.test and maybe you should too  Link to Video

Also while talking to a one of the pytest developers I came across a completely new framework for automated testing : Robot Framework

I believe this is as far the most advance tool for acceptance testing that I came across which integrates with Selenium to provide better test coverage and test outputs as shown in one of videos Link to Video
But our existing test suites are quite different from the Robot Framework and we are more concerned about Integration + Functional testing. So in our case it might not be of much use, but really it was a fun to get to know more about this framework.

Exploring these tools and talking to so many developers, I am in an opinion to keep the test suite with application itself so that we can use FactoryBoy for better data population along with pytest fixtures but since this time the tests will be with the application itself we have an option for using LiveServerTestCase  or the original pytest-mozwebqa plugin. 

I am still not sure about how pytest-mozwebqa will integrate with Fixtures and LiveServerTestCase if we use it, but I am looking forward to talk with my mentor and team about it and finalize the tools by next week.