How Software Testing Professionals view Functional and Security Testing
It is a common belief that security testing & functional testing are just the same. However the fact remains is that there are few points of differences among security testing & functional testing needing some sort of paradigm shift in the viewpoint of the testing community about testing as a philosophy.
Functional testing being a type of software testing activity is carried out by the testers as if actual product user would have used it to confirm as to whether the product meets the intended functionality or not. Almost all functional testing activities are carried out from the customer�s perspective.
There happens to be a pitfall in this approach that testing done with such a perspective we tend to bypass large number of security related tests. Majority of the security related vulnerabilities generally get unearthed by chance by the desired users, but there remain slight chances of their exploitation by such users. Reason being such users or customers are generally harmless & are not able to visualize the gravity of the security vulnerability of the problem detected by them. Rather such users prefer to contact the technical customer support for reporting their accidental catch of the software problem or at the most would prefer to talk about the poor quality of the product among their acquaintances.
The focus of the functional testing effort generally lies in
1) Enabling the customers using the software product easily and conveniently as far as possible.
2) Offering enough checks & safety provisions so that unintentional security related harms are prevented.
On the contrary, the focus of the security testing effort is
1) Based upon permissions & privileges.
2) Based upon the belief that the user is using the intended functionality.
Basic premise of Security Testing & Functional Testing:
The primary intent of conventional functional testing of software under test is validation of the important aspect that it complies with the specified requirements & performs the desired functions.
Functional testing includes many basic security tests like permissions, passwords & method of authentication or login etc. Even such testing activities are aimed at testing within the expectations of the system. For example testing of permissions focus on the aspect that administrative functions can be allowed to be carried out only when you are logged in as an administrator. However the method of acquiring administrator�s privileges falls beyond the purview of user�s login.
Security is always viewed as a simple aspect under the purview of functional testing, yet it planned independently. Although security testing goes side by side with functional testing, still their focus happens to be exactly contrasting.
The different focuses of security testing & functional testing have distinct purposes, while each of them often compensates the failures of the other. Security bugs are often found while running the functional tests & vice versa. It happens on occasions that the tests related to both these aspects are mixed up & run simultaneously, hence their outcome are also separately evaluated.
It is good to act according to the philosophy of removing all the redundant test cases to cut down the time & cost of testing. Yet it is seen that pure functional test cases and security test cases often overlap with each other. Reason being there can be totally different fail or pass criteria for these two kinds of tests. Testers are likely to miss out more number of bugs while evaluating the findings of a test case in the process of simultaneous evaluation of both security & functional criteria.
Security testing from the perspectives of different people in software domain:
Nowadays as the market for e-commerce and e-business is growing, the need for making security testing more & more powerful and sophisticated is also increasing. Reason being, there are people who would want to break the product in several ways and for various reasons, starting from unintentional mistakes, for fun or even to harm others.
Personnel from different software disciplines tend to focus on security testing according to few of the following generalized perspectives & may not be applicable to every person in the particular group.
1) Security testing from the perspective of software developers:
Developers have a mindset of working on compact piece of code designed to perform its desired functions without extraneous code or functionality. This is often against the security concerns because it can lead to taking unsafe shortcuts or not validating input as often as necessary in a drive to keep the coding compact. Developers always wish their code to be secure, but their natural inclination is to trust the inputs and data more & more.
Developers also tend to be very focused on just the system or subsystem they are writing code for. Due to this, they tend to develop some sort of tunneled vision when it comes to implicit interfaces with other systems.
Majorities of developers remain somewhat optimistic. When they write code, their first concern is to make it function and, once they make it work, they do not test it themselves any more, and they retain an optimistic view that it will not break easily.
2) Security testing from the perspective of software testing engineers:
Software testing engineers really want to find bugs before the customers or consumers do. Their other main drive is to solve the puzzle of how to convince the system to do something it is not supposed to do.
Because there are always more tests to run, security testing generally seems to add yet more cases to be fit into roughly the same amount of test time, so sometimes there is some resistance to adding more tasks without adding resources. It is difficult to find out software testing engineers those who do not feel that more testing would be even better, including security testing.
3) Security testing from the perspective of project managers:
The project managers are generally focused on the needs of the customers and trying to align those with the needs of the company producing the system.
The system has to fulfill the needs of the customers since, without customers, companies cannot survive. However, the focus on this means that program manager tend to have thing as easy as possible for the customers, and this is sometimes at the cost of the best security practices.
Part of the desire to make things easier for customers has been the ever-increasing move toward tight integration of different systems with each other, allowing them to interact and share functionality and data with each other. This also involves a more expanded trust relationship with each other that creates additional security concerns as a by-product.
4) Security testing from the perspective of management:
For continued sustenance in business every management wants their company to flourish & make more profits. Hence their focus lies on finding ways & means to increase its sales; and at the lowest possible costs as well.
Generally the reaction of top management to security testing and software security drives is dependent upon the way in which the process is presented to it. To most effectively inform management of the factors it cares about, Software testing managers have to be able to present the security concerns in a way that makes sense to the bottom line it is responsible for.
Effective communication of the security concerns:
One of the real issues faced by the software testing engineers when they have found a security vulnerability is how to get it fixed. Sometimes everyone is on the same page, especially if there is a formal security drive in place, and getting security issues approved for a fix is relatively easy.
However, what if you are not so lucky, and your bugs may be repeatedly returned to you with some reason or the other that is not important or urgent enough to be fixed. Sometimes this means they are being pushed out for a possible fix in the future, and sometimes they are being permanently decided as a “No need to fix at all.”
It is comparatively easy to explain the repercussions of not fixing a functional bug, and the process of prioritizing that fix is straightforward. Because it is a very customer centric view, it is simple to explain to the management.
Security vulnerabilities, on the other hand, are not nearly so easy for someone not familiar with security to understand and be able to assign an appropriate priority to. There are, however, a few steps you can take when you file these bugs that will give that bug the best chance of being correctly prioritized and to have an appropriate decision made on when to fix it. These are actually useful steps to take for all your bugs but are particularly good for security bugs.
Broad tips for Software Testing Managers & Engineers:
1) Carefully examine all the factors playing significant role in determining how urgent a bug fix is.
2) Make your bug reports fairly accurate. Never try to overstate your bugs by artificially inflating the reported severity of your bugs.
3) Include appropriate information in your bug report not only the core issue and steps to reproduce the behavior but also a brief overview of why you have assigned the particular severity to your bug..
4) Avoid fighting every decision about the priority of your bugs to avoid being earning a reputation for being short-sighted and argumentative.
5) Create a security conscious environment. Try to be a driver of positive changes in your team or even your company.
6) Be persistent in your explanation of your bugs. Never get put off by any lack of immediate understanding or eagerness on the part of the other team members.
7) Share your knowledge frequently with others. The knowledge you share with others in your team or even your company can help keep attention on the subject of security and educate others as well.
8) Publicize the success and failures in your security efforts at the level of both your peers and the management.
9) Carry out a detailed assessment of every release or stage of the product life cycle and track items like the number of security bugs found and fixed, any new security concerns in the industry and, details of action plan to address them..
10) Do not believe that people who use your product will do so according to its intended purpose. It is better to realize that some users would be more than willing to misuse the entire or portion of your product.
An expert on R&D, Online Training and Publishing. He is M.Tech. (Honours) and is a part of the STG team since inception.