Testing Active Vs Passive or Explicit vs Implicit

[Rama Krishna Yalamanchili]

Hi Andy, Dave,
As a programmer, i do test the things that i program at every milestone.
If i write a business block i will test it, and assume any serious programmer will test his code. We do spend time with debugging, and the natural causes are typos or some ill formed sytax mistakes apart from logical programming mistakes.
It is good to know these principles of testing for a programmer.
As i look at the index, it suggests concepts like checking boundary conditions, forcing error conditions, performance testing ect... which are typically done by testers. I am critical about the factor that you suggest that the programmer focuses more on writing and implementing tests than actual "programming logic".
I appreciate that you have pointed out many practical problems like "runs on my machine"..But are you trying to give an answer for these general problems or pointing them out?..
May be i have to go through the complete book over a weekend to get a better understanding..I wouldnt know if my Manager (Line) will appreciate more lines of testing code than programming code.
Yours Sincierly ,
YRK

[Dave Thomas]

Originally posted by Rama Krishna Yalamanchili:
As i look at the index, it suggests concepts like checking boundary conditions, forcing error conditions, performance testing ect... which are typically done by testers. I am critical about the factor that you suggest that the programmer focuses more on writing and implementing tests than actual "programming logic".

But as a developer, don't you have to consider all these things as you are coding? You need to think about boundary conditions, becase they affect your design and your algorithms. You need to think about error handling for exactly the same reasons. That's the great benefit of proper unit testing: it forces you to consider these special cases as you are writing the code, just when its easiest to make changes. If instead you leave it to testers to try these things, then if they find problems you're potentially talking about redesigning your already written code to fix the issues. And this is far more expensive.
As to the problem with your boss not liking the number of lines of test code: a couple of suggestions. First, ask him to let a team experiment with doing proper unit testing, and then to look not at lines of code but instead at overall productivity. I'm betting the tested code will be delivered earlier and with less bugs than the same code developered without testing.
Second, there's a story in the book about a manager with similar doubts. We helped him see the light by plotting the number of tests witeen for each of the classes in his system. There was a direct correlations between the classes with the fewest tests and those that caused him the biggest headaches. In fact, he initially thought we'd simply plotted QA complaints per class!
After that, he was a convert.
Cheers

Dave

[Ilja Preuss]

Originally posted by Dave Thomas:
If instead you leave it to testers to try these things, then if they find problems you're potentially talking about redesigning your already written code to fix the issues. And this is far more expensive.

Additionally, it is also more risk regarding the schedule, as the feedback on wether you are really finished implementing some functionality is delayed.

[Glenn Murray]

Originally posted by Dave Thomas:

...
As to the problem with your boss not liking the number of lines of test code: a couple of suggestions. First, ask him to let a team experiment with doing proper unit testing, and then to look not at lines of code but instead at overall productivity. I'm betting the tested code will be delivered earlier and with less bugs than the same code developered without testing.
...
Dave

I've heard this and even suspected it was true. Do you know of any
published data for this? I'd really like to see some hard data to
support the claim that JUnit tested code is developed faster. All
I've seen so far is hearsay, anecdotes, and wishful thinking.
Cheers,
Glenn

[Andy Hunt]

We have hard data, but unfortunately it's not publishable (NDA's and such).
But all we did was write a simple script that counted up the number of JUNit asserts in a given test class and calculated a ratio of the number of asserts versus the number of methods in the production class. It's a crude measurement, but it gave us something to go on.
And low and behold, after running this for a few weeks a very clear pattern emerged. The classes and modules without unit tests (or with very few unit tests) were the huge, giant headaches. Those with more unit tests jsut weren't a problem.
We could publish such a study, but it's much more effective to demonstrate this to the powers that be on your own code.

[Ko Ko Naing]

Originally posted by Andy Hunt:
The classes and modules without unit tests (or with very few unit tests) were the huge, giant headaches. Those with more unit tests jsut weren't a problem.

But there is a trade-off between the time spent for the unit testing and the coding process... I guess classes and modules with more unit tests will take more time than those with less unit tests...
What if we can just neglect some un-important classes and modules to be with less unit tests and emphasize more unit tests on important classes and modules...
Just my 2 cents...

[Lasse Koskela]

Originally posted by Ko Ko Naing:
I guess classes and modules with more unit tests will take more time than those with less unit tests...

Yes, typing 200 lines of code takes more time than typing 100 lines of code. However, the vast majority of development time is not spent typing the code but fixing the defects that the developer introduced while typing those lines of code.

Originally posted by Ko Ko Naing:
What if we can just neglect some un-important classes and modules to be with less unit tests and emphasize more unit tests on important classes and modules...

This is exactly what every single unit testing author has promoted since the beginning of time; there is a point of diminishing returns in unit testing as well.

[Jeanne Boyarsky]

Rama,
I firmly believe that programmers should consider everything that "tester" do. (with the possible exception of performance testing) Since a programmer has access to the code, it is often easier to induce errors. Also, why should you have to wait for formal testing to start before you find a boundary condition error?
While managers want the code writter quickly, they also want it written correctly. They don't want you to have to waste time later to fix bugs that could have been caught.
One of the big benefits of junit is that it makes sure your code "stays working." A chunk of business logic may work when you test it during development. But what's to say it is working two weeks later?

[Rama Krishna Yalamanchili]

Hi Ranchers,
So, back to my point after all your valuble inputs....
Why should we test th code every 5th line... if i am sure of my programming logic..
Testing has evolved over a paradigm, where what we have engineered is to be faultless. (I hope everyone agrees).
Ex: Civil engineering
If I am building something physical then i would not test every brick. If i know that i have my concrete mixture as prescribed by my building blue print specs then i test the building for its overall parameters.
Ex: Mechanical engineering
If I am assembling a car with x(Horse)Ranch(Power) then i would not test every nut and bolt and tyre as for every car i build. I would quality test a sample car with various mechanical parameters.(Object orientation applies)in the sense every single component of the car is pre-manufactured by different vendors and is tested.
Coming to the software paradigm:::
I have my prebuilt OO building blocks. They are tested (I am sure SUN tests its classes before putting it in the Java API).
Ex: I built a security package with RSA. I needed random numbers with (Big Integers)at that time. I would write a piece of code that produces the keys. I would trust Java math.random. And if my extended eucliedien algorthim programming logic is correct which would be around 450 lines. Then i expect the keys to be produced.
What i would test in this scenario if i go by unit testing?
What as programmer i have tested around 6 years ago was that atleast for 100 samples of the keys produced were not identical before i handed it to testing. I did write a piece of testing code which would run the program for n number of times and compare all the results. I had to do a little syntax debugging for typos. And my code still runs with a Major Bank(Big Bucks) I did achieve a piece of code without using unit testing.
If i read my software specifications, design document and i am not new to programming logic, syntax and OO concepts of use then i would not test my code every 5th line.
Where i would agree is that a person new to programming will benifit if he knows the rules of testing. Then his brain will automatically think for a logic and syntax while programming itself.
Summary:
Learning testing rules in general will give a good starting ride in the programming ranch. As you become a seasoned rider you implicitly do pseudo testing while programming without much effort for explicit unit testing.

[Lasse Koskela]

What if you have to go back into your code 12 months later, to make it do something differently? Wouldn't you want to run your regression tests to make sure that you didn't break anything while tweaking?

[Lasse Koskela]

One more thing.
When you say "I don't test my code after every 5 lines of code", are you actually saying that you're saving time by writing bigger chunks of code before testing it? If you are, I'd strongly suggest you to think about how much harder it is to find and fix the defect you find when you finally test your code. Furthermore, how much harder it is to find and fix the defect if it slips through to production and an end-user reports it?

[Jeanne Boyarsky]

Ralma,
Yes, but those bricks, nuts and bolts are not changing once they are built. Code is not guaranteed to stay exactly the same in the future. And they do randomly check bricks, nuts and bolts on an assembly line. If the random sample has too high a threshold for errors, the process is examined.
In your RSA example, I would trust math.random. However, you still want to test the extended eucliedien algorthim. How do you know it works? I would mock out the random calls for testing to make sure the rest of the algorithm works. If you test the code "every five lines" as it is being written, you will find bugs quickly and have a full suite of tests when done. Otherwise, you will have 450 lines of untested code. This would be overwhelming to test and unlikely to get done.
I agree that it is possible to write code without unit testing. However, you did write a (high level) test. Even this could be automated and run regularly.
I strongly disagree that testing is only for new programmers. As you get better at testing, it gets faster and more valuable. The regression aspect is priceless. The tests also document what you were thinking when you wrote the code. If someone else (or even you) have to touch the code in the future, why should they have to waste time coming up with the same test cases you already had? Why should they have to waste time figuring out what you were thinking when you wrote the code originally?

[Ilja Preuss]

Originally posted by Rama Krishna Yalamanchili:
Why should we test th code every 5th line... if i am sure of my programming logic...

If you never have bugs in your code, don't write tests.
If you sometimes do have bugs, only write tests for the code you think the bugs could be in. If you find that there actually *was* a bug in some code you thought there couldn't possibly be one, learn from it. Seriously.

Testing has evolved over a paradigm, where what we have engineered is to be faultless. (I hope everyone agrees).

I am not sure I am understanding you correctly here. But I want to point out that testing cannot prove the absence of bugs, it can only improve our convidence.
And it can help us design our code - as testable code typically needs to be reasonably decoupled, trying to write unit tests can help us spot problems with coupling in the design.

Ex: Civil engineering
If I am building something physical then i would not test every brick. If i know that i have my concrete mixture as prescribed by my building blue print specs then i test the building for its overall parameters.

Yes. How does that translate to software development? What is our concrete mixture, what are the blue print specs?

Ex: Mechanical engineering
If I am assembling a car with x(Horse)Ranch(Power) then i would not test every nut and bolt and tyre as for every car i build. I would quality test a sample car with various mechanical parameters.

Yes. And if the tests aren't satisfying or new requirements emerge, you change the design and retest. Once you are satisfied, you go into mass-production.
The former is more what software developers are doing. The latter is very easy in software development - press the binaries on CD.

Coming to the software paradigm:::
I have my prebuilt OO building blocks. They are tested (I am sure SUN tests its classes before putting it in the Java API).
Ex: I built a security package with RSA. I needed random numbers with (Big Integers)at that time. I would write a piece of code that produces the keys. I would trust Java math.random. And if my extended eucliedien algorthim programming logic is correct which would be around 450 lines. Then i expect the keys to be produced.

Yes, I would trust math.random, too. But how can you trust yourself to code a 450 lines algorithm correctly without any testing to back you up?

What i would test in this scenario if i go by unit testing?

You would test that the lines *you* wrote are actually doing what you thought them to do.

What as programmer i have tested around 6 years ago was that atleast for 100 samples of the keys produced were not identical before i handed it to testing. I did write a piece of testing code which would run the program for n number of times and compare all the results. I had to do a little syntax debugging for typos. And my code still runs with a Major Bank(Big Bucks) I did achieve a piece of code without using unit testing.

Of course it's possible. The question is wether incorporating unit testing could have been more effective, isn't it?

[Siva Jagadeesan]

I feel really bad for Rama and guys like him, who think testing is not a part of developer's job. They think they could write any code and throw it to the poor testers to find their mistakes. If you have so much pride as a developer, isn't it embarassing for you to allow some one else to find that the code you wrote is a crap.Tell me how many times have you wrote a program and someoneelse as found a bug( some are bugs , but mostly they are developer's mistake) in it.I think you better have to do some soul search my buddy, before comparing Software engineering with Civil and Mechanical engineering.
"It is not possible to wake up someone who is pretend to be asleep"

[Kishore Dandu]

Siva Dude,
Give some respect to other desi with his own point of view.
Hope in the future you will put your mind into a topic before posting nasty opinions about some one else's thinking.
Dan.
[ February 29, 2004: Message edited by: Chris Daniel ]

[Siva Jagadeesan]

Yah Chris, I guess you are right.

[Rama Krishna Yalamanchili]

Thanks guys for valuble input and never mind the explicit harsh lingo.
My point of view (Active vs Passive Testing) is in a sense has a very subtle conflict. As a programmer everyone must test his code(Agreed). No one has commented yet on the logical part yet. Being a programmer for me is to be strong at programming logic and programming syntax.
If i know a language in and out as well have an implicit testing instinct i would bring it out in the code i write. Explicit testing for every few lines of code means we are not matured enough to write code(Or for that matter not a qualified programmer).
"Quote" Test a programmer first before he tests his code "Unquote"
I would like to end this discussion thread with this.

[Jeff Langr]

Originally posted by Rama Krishna Yalamanchili:
My point of view (Active vs Passive Testing) is in a sense has a very subtle conflict. As a programmer everyone must test his code(Agreed). No one has commented yet on the logical part yet. Being a programmer for me is to be strong at programming logic and programming syntax.
If i know a language in and out as well have an implicit testing instinct i would bring it out in the code i write. Explicit testing for every few lines of code means we are not matured enough to write code(Or for that matter not a qualified programmer).
"Quote" Test a programmer first before he tests his code "Unquote"

Greetings Rama,
I think this presents a good solution to the current issue of the lack of software development jobs. If we got rid of every programmer who wasn't disciplined enough to make sure his or her code worked without testing, 99% of us would be out of work.
Perhaps you're in that other 1%. I think I'm pretty good at writing clean, correct code, but even after gobs of years of it I still make mistakes. And it's the little mistakes, the ones that can occur in "every few lines of code" that matter.
Many of us have worked on systems where we've survived without writing unit tests. Yes, it's workable. But those of us that are sold on extensive unit tests have found, in addition to a lower defect rate, many other benefits in unit testing (particularly when doing test-driven development). We also don't want to have to hope that all other developers have matured to the point where we can implicitly trust their code.
I would like to end this discussion thread with this.
Good try.
-Jeff-

[Ilja Preuss]

Originally posted by Rama Krishna Yalamanchili:
Explicit testing for every few lines of code means we are not matured enough to write code(Or for that matter not a qualified programmer).

I am not sure I understand your reasoning here.
First, let me explain that "writing a test for every few lines of code" actually does *not* sound like the thing I am doing. What I will do when I am disciplined enough is writing a new test every few minutes, after which I will change, add or even delete some lines of code.
I don't do this because I think that I will otherwise introduce a bug every few minutes. I do this because of several reasons:
- I know that sometimes I *do* introduce bugs. Sadly, I don't know in advance which minute I will introduce it. Additionally I also found that the earlier I know that theere is a bug, the less costly it is to fix. That is, I write a new test every few minutes because it reduces the time until I find a bug.
- The tests are a tool to reason about my code. Of course I could also analyze the code to understand what it is doing and if I got my indices right, but letting a test tell me is just *so much easier*!
- The tests give me a feeling of success. Every new test passing is a little motivating event to celebrate.
- And last but not least, writing the tests helps me to think about the design. They focus my thoughts on the needs of the clients, and they force me to decouple my designs.
The matured programmer will try a tool and evaluate it based on its usefullness in helping him write good code - regardless of what his ego says...

[Jeanne Boyarsky]

On the topic of logical vs syntax error, I think this whole discussion relates to logical errors. The compiler can tell you about language/syntax type errors. Junit tests can tell you about logical errors.

[Warren Dew]

Originally posted by Glenn Murray:
Do you know of any published data for this? I'd really like to see
some hard data to support the claim that JUnit tested code is
developed faster.

Not published and not exactly development time data, but I usually keep a bug log, and at one job a few years back kept statistics on how many bugs various code QA methods caught. Here were the results over the month or so I kept track:

Statistics collected at [client], a dot-com startup, for 
programming in Java using automated unit testing tools
                                       Summarizer
                               Token   Sentencer      Token         Long
hours (bugs caught) | Assert | Revamp | Stripper | Distribution | Sentence || Total 
--------------------+--------+--------+----------+--------------+----------++-------
design/write/compile|   4    |  16    |   32     |      9       |    3     || 64
code review (bugs)  |   1(3) |   2(3) |    5(4)  |      2(3)    |    1(0)  || 11(13)
automated unit test |        |        |(1 shared)|  (1 shared)  |          ||(2 shared)
            (bugs)  |   4(0) |   7(0) |   18(4)  |      3(5)    |    4(0)  || 36(9)
unit test to JUnit  |        |        |          |              |          ||
      (later bugs)  |1.5*(0) |   2(1) |    2     |      1(0)    |    1(0)  ||7.5(1)
                                         (2)      (1 bug missed)                (3)
*two people for learning JUnit
Note:  code review bugs tended to be issues that would not affect immediate 
operation, but would affect future maintainability.

I would note a couple of things:
- there's a difference between testing, coding automated tests, and using a regression test framework like JUnit. I always test, but on most projects I only write formal unit test code for particularly tricky modules, and I've only used JUnit on two projects. I believe that informal, unautomated tests would still have caught most or all of the bugs listed under "automated unit test" above. Still, it was the above statistics that made me a believer in JUnit.
- as Jeanne notes, the main advantage to a regression testing framework like JUnit is not with the initial code, but with later refactoring or redesign. The time period covered above didn't include any such activities, but on another project that did, the refactoring went far faster because I had some confidence that unit tests would catch any unexpected consequences of my changes.
Warren