Penetration Testing: Step-by-Step Guide, Stages, Methods and Application

Introduction

The architecture of companies today is complex- networks, applications, servers, storage devices, WAF, DDOS protection mechanisms, cloud technology and so much more is involved. With such options in hand, the system becomes complex. Since a single person is not handling these things, complete knowledge is impossible. Some teams handle network and create rules on business demand, some handle the configuration part and ensure that the functionality is taken care of; these scenarios leave space for weaknesses. An attacker can identify these vulnerabilities and launch attacks that can do a lot of damage. This possibility cannot be brought down to zero but can be reduced to an acceptable level. The need is to bring an ethical hacker to the environment and get the things tested. He/she will be responsible for performing penetration tests on the target agreed upon.

What is Penetration Testing?

Penetration testing is the art of finding vulnerabilities and digging deep to find out how much a target can be compromised, in case of a legitimate attack. A penetration test will involve exploiting the network, servers, computers, firewalls, etc., to uncover vulnerabilities and highlight the practical risks involved with the identified vulnerabilities.

Stages of Penetration Testing

Penetration testing can be broken down into multiple phases; this will vary depending on the organization and the type of test conducted– internal or external. Let’s discuss each phase:

1) Agreement phase:

In this phase, there is a mutual agreement between the parties; the agreement covers high-level details- methods followed and the exploitation levels. The attacker cannot bring down the production server even if the testing has been done at non-peak hours. What if the attacker changes the data that has been contained in the database in production? This will unveil the vulnerabilities but at the cost of business. A non-disclosure agreement has to be signed between the parties before the test starts.

2) Planning and reconnaissance:

In this phase, the attacker gathers as much information about the target as possible. The information can be IP addresses, domain details, mail servers, network topology, etc. An expert hacker will spend most of the time in this phase, this will help with further phases of the attack.

3) Scanning:

This is the phase where the attacker will interact with the target with an aim to identify the vulnerabilities. An attacker will send probes to the target and records the response of the target to various inputs. This phase includes- scanning the network with various scanning tools, identification of open share drives, open FTP portals, services that are running, and much more. In case of a web application, the scanning part can be either dynamic or static. In static scanning, the application code is scanned by either a YTool or an expert application vulnerability analyst. The aim is to identify the vulnerable functions, libraries and logic implemented. In dynamic analysis, the tester will pass various inputs to the application and record the responses; various vulnerabilities like injection, cross-site scripting, remote code execution can be identified in this phase.

4) Gaining Access:

Once the vulnerabilities have been identified, the next step is to exploit the vulnerabilities with an aim to gain access to the target. The target can be a system, firewall, secured zone or server. Be aware that not all vulnerabilities will lead you to this stage. You need to identify the ones that are exploitable enough to provide you with access to the target.

5) Maintaining access:

The next step is to ensure that the access is maintained; i.e., persistence. This is required to ensure that the access is maintained even if the system is rebooted, reset or modified. This kind of persistence is used by attackers who live in the system and gain knowledge about them over a period of time, and when the environment is suitable, they exploit.

6) Exploitation:

This is the phase where the actual damage is done. An attacker will try to get the data, compromise the system, launch dos attacks, etc. Usually, this phase is controlled in penetration testing so as to ensure that the mayhem on the network is limited. This phase is modified in this way- a dummy flag is placed in the critical zone, may be in the database; the aim of the exploitation phase will be to get the flag. Revealing the contents of the flag will be enough to ensure practical exploitation of the network or data theft.

7) Evidence collection and report generation:

Once the penetration test is complete, the final aim is to collect the evidence of the exploited vulnerabilities and report it to the executive management for review and action. Now, it is the management’s decision on how this risk has to be addressed. Whether they want to accept the risk, transfer it or ignore it (least likely option).

Different Types and Methods of Penetration Testing

Types of penetration testing can be categorized on the basis of either, the knowledge of the target or the position of the penetration tester. There are a few other parameters to the categorization of penetration.

• Black Box, Gray Box, and White Box:

When the penetration tester is given the complete knowledge of the target, this is called a white box penetration test. The attacker has complete knowledge of the IP addresses, controls in place, code samples, etc. When the attacker has no knowledge of the target, this is referred to as a black box penetration test. Please note that the tester can still have all the information that is publically available about the target. When the tester is having partial information about the target, this is referred to as gray box penetration testing. In this case, the attacker is having some knowledge of the target like URLs, IP addresses, etc., but does not have complete knowledge or access. This is with respect to the knowledge.

• Internal and External Penetration test:

If the penetration test is conducted from outside the network, this is referred to as external penetration testing. If the attacker is present inside the network, simulation of this scenario is referred to as internal penetration testing. Since the attacker is an internal person, the knowledge about the system and the target will be abundant when compared to a test conducted from outside.

• In-house and Third party Penetration test:

When the test is conducted by an in-house security team, it is another form of internal penetration testing. Companies often hire third-party organizations to conduct these tests, this is referred to as third-party penetration testing.

• Blind and Double-Blind Penetration test:

In a blind penetration test, the penetration tester is provided with no prior information but the organization name. The penetration tester will have to do all the homework, just like a legitimate attacker would do. This will surely take more time, but the results would be more close to the practical attacks. A double-blind test is like a blind test but the security professionals will not know when the testing will start. Only the senior management will have this information. This will test the processes, controls and the awareness of the security teams if and when a real attack occurs.

Importance of penetration testing in business

For an organization, the most important thing is business continuity. Second most important thing is the supporting services that ensure the business runs smoothly. Thus, to ensure that senior management is involved and pays attention, a penetration tester should highlight the risks that a business might face due to the findings. Let’s discuss a few important pointers that cover two things:

• What is in this for the business, in terms of capital?
• What is there for the security teams?

A penetration test will ensure that:

1) Weaknesses in the architecture are identified and fixed before a hacker can find and exploit them; thus, causing a business loss or unavailability of services.

2) Organisations these days need to comply with various standards and compliance procedures. A penetration test will ensure that the gaps are fixed in time to meet compliance. One of the examples is PCI-DSS; an organization which deals with customer’s credit card information (store, process or transmit) have to get them PCI-DSS certified. One of the requirement is to get penetration testing done.

3) Penetration tests will be an eye-opener or a check on the organization’s internal security team. How much time do they take to identify attacks and take responsive steps? Do they realize that a breach has happened? If yes, what do they do? And, when they do, is it sufficient?

4) What will be the effect if a real attack occurs? What damage can be done? We can actually calculate the potential loss to the organization if an attack occurs.

Tools and techniques

Now that we have talked enough about what is the need of a penetration test. We need to talk about the tools that a penetration tester can use to conduct this test.

1. Nessus

Nessus is a network and web application vulnerability scanner, it can perform different types of scans and help a penetration tester identify vulnerabilities. The attacker can then spend time in determining what can be exploited further. The free version of the tool is having some interesting features disabled. The full version is powerful and has a lot of features that will help during the scanning phase of the penetration test.

2. Dirbuster

Dirbuster is a directory busting tool, this will help the attacker to find the directories that are present. The tool will take an input list and will help in testing their availability. This will allow for footprinting of the directory structure and find directories that will be difficult to find.

3. Metasploit

Metasploit is an exploitation framework that has been packed with various capabilities. A skilled attacker can generate payloads, shellcodes, gain access, and perform privilege escalation attacks. The knowledge of python and ruby will be helpful since the framework uses them for most of the scripts.

4. Burp Suite

This tool is specifically used for testing web applications. Let us assume that you have uncovered a test web application that is no longer used after production push. You can use this tool to dig deeper into the application and hunt vulnerabilities. The high severity vulnerabilities can be further exploited to move forward with the attack.

Why do we need tools?

1. Saves time and effort- a well-known vulnerability will take a significant amount of time to be identified. Tools will identify them and you can work on the next stage.
2. Will be more accurate with findings; there will be false positives, but that can be minimized over a period of time.
3. A penetration tester cannot be an expert in all phases of the test. Thus, tools will be of much help.
4. They help in generating easy to understand reports that can be used by the business teams and executive management. Most of the tools offer various reporting formats that can be used by developers, testers, management or fed to other tools for further usage.
5. Automates the manual tasks- teams can focus on skilled work rather than redundant tasks.
6. The tool will gather a lot of data that will be reported to the tester; this data may not be exploitable always, though it offers a lot of knowledge. The data is used by internal teams to create strong architecture.

Conclusion

By now, you would have understood-

1) What is penetration testing, and why is it necessary for business and organization as a whole?

2) What is done after a penetration test is complete?

If you do not have these questions already, then you might be thinking from only one side. Once the test is done, the management has to take a call on what is the risk and what they can do- do they put in place a security control to mitigate the risk? You might think that, yes, that is necessary; but this is wrong. Sometimes, the loss due to vulnerability is less than the cost of control. In these cases, the organization may opt to accept the risk. It takes time and effort to be an expert penetration tester; today, most of the penetration testers are just vulnerability analysts. To be a fine penetration tester, you should know the art of exploitation. You need to sharpen your instincts at identifying, what can be exploited and what can be extended.