Dynamics NAV Testing Automation and False Positives and Negatives
One of the cool add-ons you can get for Microsoft Dynamics NAV is the Application Test Toolset. For details and a download link, visit Microsoft Dynamics NAV Team Blog. (NOTE: you’ll need a PartnerSource login to access the link.) In the blog entry, there are details about setting up the test tool, and it would be redundant for me to write a set of instructions.
Testing Automation
The idea behind the Application Test Toolset is that you write various tests for business processes and exceptional cases into special testing codeunits, and then you can hit a few buttons and test all the processes within a functional area of your organization—or even all the functional areas in your organization.
The challenge of the Application Test Toolset is that it doesn’t come out of the box and include the customized processes you’ve put together. You’ll have to sit down and write your own testing codeunits to cover all of those different business cases, but it’s generally worth the time and effort.
All that said, it takes a lot of coding effort and business analysis to get a thorough set of test codeunits built, and that is unfortunately often not practical for a business to implement. If I ever get enough time to develop and harness my psychic powers to the point where I can just read a company’s database with my mind and then write all the test cases within an hour or two, I’ll make a public announcement. Until then, we’ll have to muddle through with what we’ve got.
Although every Dynamics NAV installation would benefit from implementing the Application Test Toolset, actual implementations in the wild are extremely rare. I’ve actually never seen one myself, but I can definitely think of several points in my career where a business could have avoided a lot of problems by putting an automated toolset in place that tested everything and let us know where the errors were. I do want to talk about something that applies to testing—not just automated Dynamics NAV testing, or even software testing, but all the testing that everyone is doing everywhere. That something is the concept of the false positives and false negatives in a test.
False Positives and False Negatives
A false positive result for a test is a test that looks like it passed successfully, but did not really. And a false negative result is a test that looks like it failed, but actually passed. These are the really scary scenarios when you’re building tests, because you run them and things look OK, but they’re not OK at all. (And if you’re worried about testing your software, imagine the headaches that false positives can cause for people doing something like medical testing or security testing.)
The most common time I see false positive results for a test is when I’m not casting a wide enough net. I’ll build some changes to sales orders, and they work out OK for customers A and B, so I think that my changes will work out for all the customers, but then someone tries with customer C and the process doesn’t function. It’s tough to find a balance between getting the project finished in a timely manner and testing it thoroughly enough; I always try to tell my clients to test with as many scenarios as they can, but it’s not always practical.
There is one thing I try to do when I develop reports to find false positives—but it only works if you have a good set of historical data. I’ll do my initial testing of the report with a small subset of data (like a few weeks of the year, or all the customers beginning with “A”) until everything looks OK, and then I’ll do a few runs of the report for as much data as is practicable—like all of the past year, or all of the customers in the database. Then I’ll double-check that against the General Ledger or another big source of data, and spend some time doing detective work to run down any discrepancies that show up.
So, next time you’ve got some shiny new code to test out, hopefully you’ve got something as cool as the Application Test Toolset to help you out. But even if you don’t, try to cast a wide net and try different things, and hopefully you’ll be able to work the bugs out.
For more information on topics related to Microsoft Dynamics NAV development, read more on the ArcherPoint Developer Blog, written specifically for Dynamics NAV and Business Central developers.