Working with STAF (part 2)

Well we have discussed some of the concepts of staf in post WAORKING WITH STAF. NOW we will have some thing interesting interaction by using JMeter.

Now let us know the concepts of Jmeter inorder to go further.

JMeter is a open source tool we can download from the site Apache Jmeter.
Jmeter is a performance tool used for finding the avg. response time, devaition etc parameters like the other tools load runner. The only use of the Jmeter is a open source and also it is a less memory eater when compared to the other tools.

NOW create a test plan according to the manual given in the Apache Jakarata site site itself, After creating a test plan save the file with desired name. LETs go to STAF now.

IF for suppose this file is in the remote system. we can execute this file and and the results can be obtained in our systems. INORDER to do this we have to give the trust levels to the remote systems. There are 5 trust levels, according to each trust level the access is differnet for the remote systems. After giving the trust level. Now we can execute the test plan by making a batch file.

In this Batch file we need the controller where the system we handle load generators where the test case is present, Well, i will make batch file very simple to understand and an idea how to execute the test plan by using the batch file.


REM set the controllers
REM set the load generators
REM set the plan

SET controller= webserver1
SET load = webserver2
SET plan=myplan.jmx

REM checking the STAF connecting to the remote systems

FOR %%H IN (%controller% %load%) DO STAF %%H PING PING

REM executing the test plan using jmeter

for %%H IN (%load%) Do staf %%H PROCESS START SHELL COMMAND C:\jmeter\bin\jmeter-n.cmd C:\jmeter\bin\myplan.jmx

REM copying the results from the remote system to our system

for %%H IN (%load%) Do staf %%H fs copy directory c:\jmeter\bin todirectory c:\logs\ tomachine %controller% EXT log CASEINSENSITIVE


This is a small batch file where we can use staf and make wonders.


Dear Upcoming Testers...

Testers can't go away with "click here,click there and see the result" kind of a thing while testing. Testers always need the "DEEP THINKING" while they test. One of the famous example given by Mr.James Bach which i already mentioned in About test cases. Testers always need Deep thinking.I have spoken with some of the Upcoming testers, they are in situation what is the role of a tester after getting into the company ! i don't want to pinpoint saying that "here goes the wrong so you don't know the role of a tester". I meet some of my friends who are trained like me, we are trained in different schools in different arts of styles but final output we get is knowledge. We also have a discussion several times, what is the job/role of testers in a company ? we always have to compromise by saying one thing or the other in our battle.
And more important most of the upcoming tester think automation means "QTP.LOAD Runner". This is not at all the correct way of thinking, automation includes many concepts to discuss tools and as far as approach. for example spell check in MS Office or in some application can also called as automated way. we have resources and the only thing is we have to improve our knowledge by implementing and sharing it. i got few examples for testers,

f'ing counting

interactive puzzle

The above examples are really awesome and we can also correlate this examples with Testing.

Working with STAF

STAF (software testing automation framework) is designed for reusable components called services. STAF is a remote agent used to control the test on various machines.for making easier to create automated test cases and workloads. it works on Windows,Linux,AS/400,MVS.

Concepts of STAF

1).STAFProc

2).STAF services
a)Internal services are within the STAFProc which means they are always available
and having the constant names.
b)External services are outside the STAFProc which means they are always from
outside for example from the java,c++ etc.
c)Custom services are always the external services then can also be written in our
custom and can be plugged into STAF.

3).Queues and handles

4).Variables

5).Security

6).Submitting Staf requests.

Configuration file:
After installing the Staf, in order to get the access to local machines, trust level has to be given and java sdk is needed for running the staf.


TRUST LEVEL 5 MACHINE tcp://local machine name or ip address.*


you can find the STAF.CFG file in the bin folder of STAF. You can also alter many features in the configuration file like

1).Specify the network interfaces

2).Define operational parameters

3).Define global variables

4).Specify the security access

5).Define startup/shutdown process

6).Enable and configure tracing

7).Register and configure external services

The idea behind STAF is to run a very simple agent on all the machines that participate in the STAF testbed. Every machine can then run services on any other machine, subject to a so-called trust level. In practice, one machine will act as what I called the 'test management' machine, and will coordinate the test runs by sending jobs to the test clients. STAX is one of the services offered on top of the low-level STAF plumbing. It greatly facilitates the distribution of jobs to the test clients and the collection and logging of test results. STAX jobs are XML files spiced up with special tags that contain Python code (actually Jython, but there are no differences for the purpose of this tutorial). This in itself was for us a major reason for choosing STAF over other solutions.

Common mistakes of software developers (cont.)

Some of the common problems are listed:

MY PROBLEM IS DIFFERENT

Many designers and programmers refuse to listen to the experiences of others, claiming that their application is different, and of course much more complicated. Designers should be more open-minded about the similarities in their work. In response, ask “what is different in the LCD display software in a cellular phone versus one on a temperature controller? Are they really different?” Comparing control and communication systems side-by-side, both are characterized by modules that have inputs and outputs, with a function that maps the input to the output. A 256 by 256 image processed by a algorithm might not be very different from graphical code for a LCD dot matrix display of size 320 by 200. Furthermore, both use hardware with limited memory and processing power relative to the size of the application; both require development of software on a platform other than the target, and many of the issues in developing software for a micro-controller. The timing and volume of data is different. But if the system is designed correctly, these are just variables in equations. Methods to analyze resources such as memory and processing time are the same, both may require similar real-time scheduling, and both may also have high-speed interrupt handlers that can cause priority inversion. Perhaps if control systems and communication systems are similar, so are two different control applications or two different communication systems. Every application is unique, but more often than not the procedure to specify, designs, and build the software is the same. Embedded software designers should learn as much as possible from the experiences of others, and not shrug off experience just because it was acquired in a different application area.


Large if-then-else and case statements

It is not uncommon to see large if-else statements or case statements in embedded code. These are problematic from three perspectives

1.) They are extremely difficult to test, because code ends up having so many different paths. If statements are nested it becomes even more complicated.

2.). The difference between best-case and worst-case execution time becomes
significant. This leads either to under-utilizing the CPU, or possibilities of timing errors when the longest path is taken.

3.) The difficulty of structure code coverage testing grows exponentially with the number of branches, thus branches should be minimized.

This example confuses new testers who lack in programming experience.
Developers think their code is always correct and as mentioned earlier 99 % errors are corrected by themselves and remaining 1%.errors will be found out by testers. In the below example the

IF (0 < x < 12) then
SYSTEM.OUT.PRINTLN (“Month is” & i);
Else
SYSTEM.OUT.PRINTLN (“Invalid input”);


Consider how this code could fail. Here are some of the simple programming errors that are very common mistakes that can go wrong:

a) Suppose the programmers said less than or equals instead of less than. The program would eject 0 as bad character. The only way to catch the error I by testing with 0.

b) If the code is written as less than 12 instead of less than or equal to 12, the program would go wrong.


“Testers with just the four boundary characters, /, 0, 9, and: will reveal every classification
Error that the programmer could make by getting an inequality wrong or by mistyping”

Error Handling:

Errors in dealing with errors are common. Error handling errors include failure to anticipate the possibility of errors and protect against them, failure to notice error conditions, and failure to deal with a detected error in a reasonable way. Many programmers correctly detect errors but then branch into untested error recovery routines. These routines’ bugs can cause more damage than the original problem.

Some times the errors are more even large while executing the tests and the Microsoft’s worst scenario is we can’t copy the error messages. There are some tools for copying the text of such error messages and also we can take the screen shots.


Conclusion:


Testers should have a common sense in smelling the non obvious things. There is a relation between developers and testers in fixing the bugs. The probability of fixing the bugs is always depends on the way the test cases are communicated. Test cases play a major role in the QA's life. Test Cases are written in such a way that they are traceable, self contained and should not be duplicated in preparing they should always be atomic.



"Why go into something to test the waters? Go into it to make waves"





HAPPY TESTING

Common mistakes of Software developers

Introduction:

Most Software developers are not even aware that there favorites methods are problematic. Quite often experts are self thought, hence they tend to have the same bad habit as when they first began, usually because they never witnessed the better ways of performing their embedded systems. These experts then train novices who subsequently acquire the same bad habits. The purpose of this presentation is to improve the awareness to common problems, and to provide a start towards eliminating mistakes and thus creating software that is more reliable and easier to maintain.

It is easy for spending a million on testing a program. Common estimation of the cost of finding and fixing the errors in program range from 40% to 80% of total development cost. Companies don’t spend this kind of money to “verify that a program works”. They spend it because the program doesn’t work, it has bugs and they want them found. No matter what development methodology they follow, their programs still end up with bugs. Beizer’s (1990) review estimation the average number of errors in program released to testing at 1 to 3 bugs per 100 executable statements. There are big differences between programmers, but no one’s work is error-free.

One error per 100 statements is an estimate of public bugs; the ones still left in program after the programmer declares it error-free Beizer (1984) reported his private bug rate, how many mistakes he made in designing and coding a program, as 1.5 errors per executable statement. This includes all mistakes including typing errors.

“At this rate, if your programming language allows one executable statements per lines, you make 150 errors while writing a 100 lines program.”

Most programmers catch and fix more than 99% of their mistake before releasing a program for testing. Having found so many, no wonder they think they most found a lot. But they haven’t .Tester’s job is to find the remaining 1%.

Correcting just one of the mistakes within a project can lead to week or months of savings in manpower (especially during the maintenance place of a software life cycle.).

Mail from Mr. Pradeep Soundararajan

For a long time i wasn't unable to update my blog after recieving the mail from Mr.Pradeep Soundararajan.


Hi Amardeep,

I am sure you know at least a little about me and you might not need my introduction. I found my blog linked to yours and as it seemed to be on testing, I did want to peruse it.

I was impressed by the fact that you linked people James and Dr Cem Kaner with respect of Mr. I intended to make you a few suggestions to help you write better posts.

1. The copy paste stuff never makes someone read your blog, since there are so many copy pasted stuff. All people might want to read is your experience, be it little or whatever or your day to day testing activities, the problem you face, the problems you solve, the testers you meet and lots more.
2. You might also want to read this: http://testertested.blogspot.com/2006/11/indian-testing-community-start.html

Best wishes and Happy Testing!



I feel really good after recieving the mail from
Mr.Pradeep Soundararajan. I always want to write my own experiences and thoughts after reading his mail. i will do my best in improving my blog and skills.


DEAR UPCOMING TESTERS : this mail is not only for me, but for all of you who just want to become the stars in TESTING INDUSTRY

why do software fails !

Generally speaking, there are bugs in software because of unclear requirements, software complexity, programming errors, changes in requirements, errors made in bug tracking, time pressure, poorly documented code and/or bugs in tools used in software development.

* There are unclear software requirements because there is miscommunication as what the software should or shouldn't do.

* Software complexity. All of the followings contribute to the exponential growth in software and system complexity.

* Programming errors occur because programmers and software engineers, like everyone else, can make mistakes.

* As to changing requirements, in some fast-changing business environments, continuously modified requirements are a fact of life. Sometimes customers do not understand the effects of changes, or understand them but request them anyway. And the changes require redesign of the software, rescheduling of resources and some of the work already completed have to be redone or discarded and hardware requirements can be effected, too.

* Bug tracking can result in errors because the complexity of keeping track of changes can result in errors, too.

* Time pressures can cause problems, because scheduling of software projects is not easy and it often requires a lot of guesswork and when deadlines loom and the crunch comes, mistakes will be made.

* Code documentation is tough to maintain and it is also tough to modify code that is poorly documented. The result is bugs. Sometimes there is no incentive for programmers and software engineers to document their code and write clearly documented, understandable code. Sometimes developers or programmers feel they cannot have job security if everyone can understand the code they write, or they believe if the code was hard to write, it should be hard to read.

* Software development tools, including visual tools, class libraries, compilers, scripting tools, can introduce their own bugs. Other times the tools are poorly documented, which can create additional bugs.

Globalization Testing

Globalization Testing ensures the application can function in any culture/locale. The goal of globalization testing is to detect potential problems in application design that could inhibit globalization. It makes sure that the code can handle all international support without breaking functionality that would cause either data loss or display problems. The following must be part of your globalization-testing plan:

1.) Decide the priority of each component :

To make globalization testing more effective, assign a testing priority to all tested components. Components that should receive top priority:

* Support text data in the ANSI (American National Standards Institute)format.

* Extensively handle strings (for example, components with many edit controls).

* Use files for data storage or data exchange (e.g., *Windows metafiles, security configuration tools, and Web-based tools).

2.) Select a test platform,

3.) Create the test environment,

4.) Execute tests,

5.) Recognize the problems.


LIFE begins when we loose,

LIFE ends when we win,

So, make sure that you always win.


-------AMAR DEEP MARADANA

Localization testing

Localization : is a service that are acceptable for a particular culture or locale that makes users to accept the adaptable software. For example MS OFFICE is used in many languages as a packagable software, as i use the software in english and others may use in Japanese or Hindi etc.

Localization testing checks how well the build has been translated into a particular target language. This test is based on the results of globalized testing where the functional support for that particular locale has already been verified. If the product is not globalized enough to support a given language, you probably will not try to localize it into that language in the first place!

Bug Tracking System

Bug Tracking System is an application used for programmers to track of software bugs in there work. it is essential in companies and used in doveloping software products. A major component of bug tracking system is database that records facts about known bugs. Facts may include the time of the bug was reported, its severity, the erroneous program behavior, and details on how to reproduce the bug; as well as the identity of the person who reported it and any programmers who may be working on fixing it.

In a corporate environment, a bug-tracking system may be used to generate reports on the productivity of programmers at fixing bugs. However, this may sometimes yield inaccurate results because different bugs may have different levels of severity and complexity. The severity of a bug may not be directly related to the complexity of fixing the bug. There may be different opinions among the managers, architects and the developers about the relative ease of fixing bugs.

About Test Cases

Test cases plays a major role in the QA's life. The probability of fixing the bugs is always depends on the way the test cases are communicated. They should never be confused. Test Cases are written in such a way that they are tracable,self contained and should not be duplicated in preparing they should always be atomic. So every tester has to spend some time on the test cases before writing. Test conditons has to be clear and most of the test conditions wont be clear,so deep thinking is necessary for the testers. In the context to the DEEP THINKING, I do like to discuss an example given by the Jsmes Bach.

REQUIREMENTS :
1.THERE ARE FOUR CARDS. (cards are E,7.V.4)
2.THERE IS A LETTER ONE SIDE AND A NO. ON REVERSE SIDE.
3.IF THERE IS A VOWEL ON ONE SIDE AND A EVEN NO. ON OTHER SIDE.

In this Requirements, if we consider the first case which is clear, where as in second and third cases where the deep thinking should be there.

So for a Tester, deep thinking is always an important.

One of my friend asked a question about test requirement, i wondered how to answer to his question. One of the requirement in the web page is to check the back ground colour of the Username and Password.HE asked me whther i will check or not. I thought for a second and gave the answer, as "i will check the background colour of both the fields",but the thing is we have to make sure of the TEXT of the font colour also is checked. Even though iam wrong in this Example i learnt a new thing in Exploring a my knowledge towards the SOFTWARE TESTING.


NEVER GIVE UP TILL YOU SUCCEED

KEY PRINCIPLES OF SOFTWARE TESTING

1) TESTING ALWAYS SHOWS THE PRESENCE OF DEFECTS.

2) EXHAUSTIVE TESTING IS POSSIBLE.

3) EARLY TESTING.

4) DEFECTIVE CLUSTERING.

5) PESTICIDE PARADOX.

6) TESTING IS CONTEXT DEPENDENT.

7) ABSENCE OF ERRORS IS FAILURE.

Bugs per lines of code

Books that reports or web site that addresses the topic of bugs per line of code The book "Code Complete" by Steve McConnell has a brief section about error expectations. He basically says that the range of possibilities can be as follows 
(a) Industry Average: "about 15 - 50 errors per 1000 lines of delivered code." He further says this is usually representative of code that has some level of structured programming behind it, but probably includes a mix of coding techniques. 

(b) Microsoft Applications: "about 10 - 20 defects per 1000 lines of code during in-house testing, and 0.5 defect per KLOC (KLOC IS CALLED AS 1000 lines of code) in released product (Moore 1992)." He attributes this to a combination of code-reading techniques and independent testing (discussed further in another chapter of his book). 

(c) "Harlan Mills pioneered 'cleanroom development', a technique that has been able to achieve rates as low as 3 defects per 1000 lines of code during in-house testing and 0.1 defect per 1000 lines of code in released product (Cobb and Mills 1990). A few projects - for example, the space-shuttle software - have achieved a level of 0 defects in 500,000 lines of code using a system of format development methods, peer reviews, and statistical testing."

Vinnie Murdico Software with Brains, Inc. 
SWBTracker - Value-Priced Defect Management Software 
http://www.softwarewithbrains.com

dear tester's

this was my first post. i do like to update this blog every day and i need all your assistance for the people who just started learning the testing.