Web Security, Application Pen Testing Methods.
Need a web application penetration test? Use our quick contact form!

Your Email or Phone:

Enter code:

Web Security - Application Penetration Testing Methodology

Safety and Stability - Network - Web Applications - Wireless - Compliance -

Web Security - Web Application Penetration Testing

Most of our penetration testing engagements include web security testing, usually one or more web applications or web services. While there are many cases when the procedures outlined below may be changed, this is our standard approach to testing web security for web applications and web services. If we are given special circumstances, we may modify the methods, but this is the standard case.

Before getting into the tools and methods we use, it is worth discussing a more general question: Is there any difference between manual web security testing and automated web security testing, and if there is, how do you, the client, know what you are getting?

Manual, Automated, or Both?

Both of these approaches to penetration testing have value, and we use both. Automation is necessary for full testing coverage, and in some cases is actually better than manual testing. Automation by itself, however, is entirely incapable of identifying, let alone validating, some of the most important security flaws found in web applications.

There are a number of vendors currently offering 'penetration testing services' for advertised prices of $895, or even $700 or less. We do not object to this, as long as the 'service' is accurately described. Our objection is with vendors who devote an entire web page to the quality of their testing, including statements about the manual effort they employ, including the qualifications of the 'testers', and then disclose in the fine print at the bottom of the page that the testing is automated.

It's not the testing methods we object to, it's the honesty. If you know what you are buying, and it meets your objectives, then we have no objection to purely automated testing. If you are told, however, that automated testing is sufficient by itself, and if thorough web security testing is an important consideration for you, then we encourage you to take a look at the table we present below.

Web Security, Testing Coverage

OWASP (Open Web Application Security Project) is perhaps the most respected organization in the world on the subject of web application security. The table below comes directly from the table of contents of the OWASP testing guide v4, but we've taken the liberty of adding one column. In this table, we added the Method column, which identifies testing that requires Manual methods (M), Automated methods (A), or both (MA).

Testing Method Index:

M

Manual - If an M appears, it means that the comments from OWASP indicate at least some Manual testing is needed. If only an M is present, we could find no indication from OWASP or our own experience that automation alone can be used even to consistently identify these faults.

A

Automated - If an A is present, then either OWASP or we can see at least some value in automation. If only an A is present, then either OWASP or we believe that automation alone could be used to accurately identify the faults, though not necessarily for validation of the faults.

MA

Both - If A and M are both designated, then both manual testing and automation are necessary for full coverage.


We should mention one more thing about the table that follows: Our interpretation of the OWASP comments and resulting method classification is, of course, our own. That is why we are including the link to the OWASP testing guide, so you can review the OWASP comments yourself. There may be some difference of opinion in some areas, but we think most informed professionals will agree with our overall conclusions. Even if you are not an informed professional, some of the OWASP comments make it obvious that human intuition and persistence is a necessary component.

Source: OWASP_Testing_Guide_v4_Table_of_Contents

4.2 Information Gathering

Method

Ref. No.

Category

Test Name

M

4.2.1

OTG-INFO-001

Conduct Search Engine Discovery and Reconnaissance for Information Leakage

A

4.2.2

OTG-INFO-002

Fingerprint Web Server

A

4.2.3

OTG-INFO-003

Review Webserver Metafiles for Information Leakage

MA

4.2.4

OTG-INFO-004

Enumerate Applications on Webserver

MA

4.2.5

OTG-INFO-005

Review Webpage Comments and Metadata for Information Leakage

MA

4.2.6

OTG-INFO-006

Identify application entry points

MA

4.2.7

OTG-INFO-007

Map execution paths through application

A

4.2.8

OTG-INFO-008

Fingerprint Web Application Framework

A

4.2.9

OTG-INFO-009

Fingerprint Web Application

MA

4.2.10

OTG-INFO-010

Map Application Architecture

 

4.3 Configuration and Deploy Management Testing

Method

Ref. No.

Category

Test Name

MA

MAR 4.3.1

OTG-CONFIG-001

Test Network/Infrastructure Configuration

MA

MA 4.3.2

OTG-CONFIG-002

Test Application Platform Configuration

A

A 4.3.3

OTG-CONFIG-003

Test File Extensions Handling for Sensitive Information

MA

MA 4.3.4

OTG-CONFIG-004

Backup and Unreferenced Files for Sensitive Information

MA

MA 4.3.5

OTG-CONFIG-005

Enumerate Infrastructure and Application Admin Interfaces

A

A 4.3.6

OTG-CONFIG-006

Test HTTP Methods

A

A 4.3.7

OTG-CONFIG-007

Test HTTP Strict Transport Security

A

A 4.3.8

OTG-CONFIG-008

Test RIA cross domain policy

 

4.4 Identity Management Testing

Method

Ref. No.

Category

Test Name

M

4.4.1

OTG-IDENT-001

Test Role Definitions

M

4.4.2

OTG-IDENT-002

Test User Registration Process

M

4.4.3

OTG-IDENT-003

Test Account Provisioning Process

M

4.4.4

OTG-IDENT-004

Testing for Account Enumeration and Guessable User Account

M

4.4.5

OTG-IDENT-005

Testing for Weak or unenforced username policy

M

4.4.6

OTG-IDENT-006

Test Permissions of Guest/Training Accounts (no current OWASP content)

M

4.4.7

OTG-IDENT-007

Test Account Suspension/Resumption Process (no current OWASP content)

 

4.5 Authentication Testing

Method

Ref. No.

Category

Test Name

A

4.5.1

OTG-AUTHN-001

Testing for Credentials Transported over an Encrypted Channel

A

4.5.2

OTG-AUTHN-002

Testing for default credentials

M

4.5.3

OTG-AUTHN-003

Testing for Weak lock out mechanism

MA

4.5.4

OTG-AUTHN-004

Testing for bypassing authentication schema

MA

4.5.5

OTG-AUTHN-005

Test remember password functionality

MA

4.5.6

OTG-AUTHN-006

Testing for Browser cache weakness

M

4.5.7

OTG-AUTHN-007

Testing for Weak password policy

M

4.5.8

OTG-AUTHN-008

Testing for Weak security question/answer

M

4.5.9

OTG-AUTHN-009

Testing for weak password change or reset functionalities

M

4.5.10

OTG-AUTHN-010

Testing for Weaker authentication in alternative channel

 

4.6 Authorization Testing

Method

Ref. No.

Category

Test Name

MA

4.6.1

OTG-AUTHZ-001

Testing Directory traversal/file include

MA

4.6.2

OTG-AUTHZ-002

Testing for bypassing authorization schema

MA

4.6.3

OTG-AUTHZ-003

Testing for Privilege Escalation

M

4.6.4

OTG-AUTHZ-004

Testing for Insecure Direct Object References

 

4.7 Session Management Testing

Method

Ref. No.

Category

Test Name

MA

4.7.1

OTG-SESS-001

Testing for Bypassing Session Management Schema

MA

4.7.2

OTG-SESS-002

Testing for Cookies attributes

MA

4.7.3

OTG-SESS-003

Testing for Session Fixation

A

4.7.4

OTG-SESS-004

Testing for Exposed Session Variables

MA

4.7.5

OTG-SESS-005

Testing for Cross Site Request Forgery

M

4.7.6

OTG-SESS-006

Testing for logout functionality

M

4.7.7

OTG-SESS-007

Test Session Timeout

MA

4.7.8

OTG-SESS-008

Testing for Session puzzling

 

4.8 Data Validation Testing

Method

Ref. No.

Category

Test Name

MA

4.8.1

OTG-INPVAL-001

Testing for Reflected Cross Site Scripting

MA

4.8.2

OTG-INPVAL-002

Testing for Stored Cross Site Scripting

MA

4.8.3

OTG-INPVAL-003

Testing for HTTP Verb Tampering

MA

4.8.4

OTG-INPVAL-004

Testing for HTTP Parameter pollution

MA

4.8.5

OTG-INPVAL-005

Testing for SQL Injection

MA

4.8.6

OTG-INPVAL-006

Testing for LDAP Injection

MA

4.8.7

OTG-INPVAL-007

Testing for ORM Injection

MA

4.8.8

OTG-INPVAL-008

Testing for XML Injection

MA

4.8.9

OTG-INPVAL-009

Testing for SSI Injection

MA

4.8.10

OTG-INPVAL-010

Testing for XPath Injection

MA

4.8.11

OTG-INPVAL-011

IMAP/SMTP Injection

MA

4.8.12

OTG-INPVAL-012

Testing for Code Injection

MA

4.8.12.1

 

Testing for Local File Inclusion

MA

4.8.12.2

 

Testing for Remote File Inclusion

MA

4.8.13

OTG-INPVAL-013

Testing for Command Injection

MA

4.8.14

OTG-INPVAL-014

Testing for Buffer overflow

MA

4.8.14.1

 

Testing for Heap overflow

MA

4.8.14.2

 

Testing for Stack overflow

MA

4.8.14.3

 

Testing for Format string

MA

4.8.15

OTG-INPVAL-015

Testing for incubated vulnerabilities

MA

4.8.16

OTG-INPVAL-016

Testing for HTTP Splitting/Smuggling

 

4.9 Error Handling

Method

Ref. No.

Category

Test Name

MA

4.9.1

OTG-ERR-001

Analysis of Error Codes

MA

4.9.2

OTG-ERR-002

Analysis of Stack Traces

 

4.1 Cryptography

Method

Ref. No.

Category

Test Name

MA

4.10.1

OTG-CRYPST-001

Testing for Weak SSL/TSL Ciphers, Insufficient Transport Layer Protection

MA

4.10.2

OTG-CRYPST-002

Testing for Padding Oracle

MA

4.10.3

OTG-CRYPST-003

Testing for Sensitive information sent via unencrypted channels

 

4.11 Business Logic Testing

Method

Ref. No.

Category

Test Name

M

4.11.1

OTG-BUSLOGIC-001

Test Business Logic Data Validation

M

4.11.2

OTG-BUSLOGIC-002

Test Ability to Forge Requests

M

4.11.3

OTG-BUSLOGIC-003

Test Integrity Checks

MA

4.11.4

OTG-BUSLOGIC-004

Test for Process Timing

M

4.11.5

OTG-BUSLOGIC-005

Test Number of Times a Function Can be Used Limits

M

4.11.6

OTG-BUSLOGIC-006

Testing for the Circumvention of Work Flows

M

4.11.7

OTG-BUSLOGIC-007

Test Defenses Against Application Mis-use

MA

4.11.8

OTG-BUSLOGIC-008

Test Upload of Unexpected File Types

M

4.11.9

OTG-BUSLOGIC-009

Test Upload of Malicious Files

 

4.12 Client Side Testing

Method

Ref. No.

Category

Test Name

MA

4.12.1

OTG-CLIENT-001

Testing for DOM based Cross Site Scripting

MA

4.12.2

OTG-CLIENT-002

Testing for JavaScript Execution

MA

4.12.3

OTG-CLIENT-003

Testing for HTML Injection

MA

4.12.4

OTG-CLIENT-004

Testing for Client Side URL Redirect

MA

4.12.5

OTG-CLIENT-005

Testing for CSS Injection

M

4.12.6

OTG-CLIENT-006

Testing for Client Side Resource Manipulation

MA

4.12.7

OTG-CLIENT-007

Test Cross Origin Resource Sharing

MA

4.12.8

OTG-CLIENT-008

Testing for Cross Site Flashing

MA

4.12.9

OTG-CLIENT-009

Testing for Clickjacking

MA

4.12.10

OTG-CLIENT-010

Testing WebSockets

M

4.12.11

OTG-CLIENT-011

Test Web Messaging

 

Web Security, Testing Coverage Summary

The number of testing points that require manual testing is obviously a substantial part of the total, so it's no wonder that many of the vendors who rely solely on automated scanning do their best to at least appear to be performing manual testing. To what degree they actually do so, and to what degree you need it, we leave to your judgment. If you suspect you are being told one thing and sold another, we can give you a few pointers on what to ask:

  • Ask the vendor to provide a copy of the automated scan report as a seperate deliverable, along with your penetration test report.
  • If they claim their scanner doesn't produce a report, ask them to zip up whatever automated scanner output their testers look at and send you that.

    If they claim they don't do any automated scans, then you might wonder how are they going to check hundreds of thousands of payload/parameter combinations manually, without using a scanner at all, along with all of the rest of the OWASP table above, for the price they quoted?

    In our opinion, there is no legitimate reason to resist this request. The automated scanner report is, after all, a report on the security posture of your systems, paid for by you, generated at your request, and should be part of any thorough web security test. If you can think of a legitimate reason for a company to refuse to provide it to you, we'd like to hear it.

    If they resist, it's probably because they don't want you comparing their 'pen test' report with the automated scan report. You can probably figure out why.

  • Ask for and check references.
  • When you call or email their references, make sure you ask if they felt comfortable that the vendor used substantial manual effort.

  • Ask the vendor if they will provide a manual activity report along with the penetration testing report.
  • This may be one of the most important questions to ask. Any vendor who is actually performing manual penetration testing is sure to be keeping records that will demonstrate it and should be glad to have the chance to prove it to you.

    If they claim that this will take too much time and will increase the price, tell them it doesn't have to be pretty and you're not expecting to pay for it.

    In our case, we are happy to provide our manual testing notes, which are full of half formed suspicions, false starts, dead ends, on-the-fly vulnerability research, screen captures taken for later use, typos, and yes, even mistakes. In other words, exactly the kind of thing you would expect to see from live human beings trying to figure out what is going on with your web application. When you see it, you will know three things: It ain't pretty, it wasn't generated by a scanner, and we actually did the work.

    One word of caution about evidence: Be wary of any vendor who tells you that you will see the evidence in your logs. It is very difficult to tell the difference between scanner traffic and manual effort, especially if the scanner is configured to randomize timing. Some scanners even have configuration options intended to mimic human behavior.

So, that's our current guidance on how to ensure that you are actually getting at least some manual testing. It's worth repeating that we are not opposed to testing that is entirely automated. Automated testing alone will not be thorough but there may be cases where it is appropriate. Our objection is with vendors who try to sell a simple automated scan to clients who are expecting significant manual effort.

Our view is that no penetration test can be considered reasonably thorough without both automated scanning and substantial manual testing. We are not alone in that opinion. The PCI- DSS council as of PCI-DSS v3, along with a growing number of software purchasing departments, and the OWASP testing guide all support that view.

Our web application testing covers the entire OWASP testing guide, not just the top 10 or top 25, and makes extensive use of manual testing using qualified, certified testers as well as automation. The remainder of this section deals with the tools and methods we use to achieve this coverage.


Web Security, Application Pen Test Tools.

The primary tools we use for Web Application Penetration Testing are:

  • Web Browsers
  • Burp Suite Professional
  • SoapUI
  • Custom Perl Scripts

This is not a complete list, but these are the major tools. We look for simple, powerful, flexible and proven tools.

Web Browsers. We use many different web browsers depending on circumstances, but the two we use the most are FireFox (or derivatives), and Google Chrome (or derivatives). Whichever browser we are using at any given time, we are using it for manual inspection and analysis.

Burp Suite Professional. Burp Suite is a penetration testing platform that integrates several important testing tools, including a web application scanner, spidering tools, intercepting proxy, entropy analysis tools for session tokens or other (presumably) random tokens, and tools for crafting and testing many kinds of attack payloads. The creators at Portswigger.net do a great job of keeping the suite current with the latest exploits and continually incorporate improvements. For our purposes, the factors that make it a great tool are:

  • Flexibility.
  • Because it is a platform intended for manual penetration testing of web applications, it allows great flexibility in the use of it's manual and automated tools. The automation that it provides is under our tight control. We can decide if we should scan, what to scan for, when we should scan, and how we should scan.
  • Manual tools.
  • The suite offers a surprisingly robust framework for crafting and testing custom attacks through it's repeater and intruder tools, and allows for real-time interception and manipulation of traffic between the client and the server.
  • Extensibility.
  • When we encounter anything we want to do that Burp Suite doesn't already handle, the suite allows us to write and incorporate our own plugins.
  • Standard Logging.
  • The suite allows us to capture and log every request and response, in sequence, and in formats parsable by other tools.

Unlike ordinary application scanners, this is a penetration testing suite. The emphasis is on fine grained control for penetration testers and robust support for manual testing methods, and not just push-button automation. That makes it a near perfect tool for our purposes.

SoapUI is a tool designed for functional testing of SOAP, and more recently, REST web services. It is not intended as a penetration testing tool, but we find it very useful for it's ability to rapidly create functional test cases for web services. Those test cases can then be used with our other tools that are intended for penetration testing.

Perl is our scripting language of choice. We use Perl for day to day on-the-fly scripting for all kinds of penetration testing tasks. You never know when you will need to do something special with a web application, and we can write what we need with Perl.

Methods and sequence.

Manual Application Review. There are two primary reasons for starting our actual testing with a manual review of your application. The first is safety and stability: we want to know if there are any factors that could result in unintended consequences before we configure and launch any automated tools. The second reason is quality. A brief examination of the alternative approach will help to illuminate some of our reasons for performing a manual review as a first step.

The alternative to a manual review as the first step is to perform automated scanning first, and that means that the scanner has to be configured. One common approach to configuring an automated scanner is to provide the scanner with an initial URL, along with any scope limitations, and then allow the scanner to spider the approved target. In other words, the tool is allowed to follow all of the links it can find, and perform a security scan of all of the parameters it can identify, for all of the links it finds, as long as the URL is in scope. This is the cheapest and (often) fastest way to scan a web application, and for vendors who do not actually perform a manual review, it is the only approach. It also has a host of potential problems. Here are a few:

  • Pre-Knowledge.
  • The spidering approach requires that you know all of the potential application specific problems that a scanner might cause or encounter, without ever looking at the application itself. You can't configure the scanner to avoid something that you are not even aware of, so your only option is to do the best you can to anticipate, cross your fingers and pull the trigger.

  • Spiders in the weeds.
  • Spiders have a problem dealing with certain application structures, such as dynamically generated URLs. In some circumstances, applications can generate URLs in a nearly infinite sequence. A human tester can quickly recognize a dynamically generated URL, and discern that every subsequent URL of the same form is essentially identical, and that the functionality exposed is always the same. All of those judgments are required in order to make the right decision about whether to continue scanning URLs of the same form. In spite of many industry attempts to code a solution to the problem, it still requires human judgment. Spiders simply can't reliably make that determination, and as a result they often end up 'off in the weeds', stuck on what is essentially a dead end loop, or skipping important content because of a badly coded solution to this issue. When scanners end up in a loop, inexperienced testers may let them run for days, then finally stop them near the end of the testing window, and assume that the application was 'adequately' scanned. In fact, how much of the application was scanned depends on the point at which the spider took a trip into the weeds.

  • Noise.
  • Spiders generate a lot of traffic, and when combined with scanning activity, they are normally about as stealthy as a circus setting up in the town square. The activity will often trip Web Application Firewalls or other Intrusion Prevention Systems, and that can cause avoidable complications in subsequent manual testing. For vendors who actually intend to perform any manual testing, it makes sense to do it quietly, behaving as much as possible like a normal user, and doing as much of it as possible before setting off the fireworks show.

  • Lock outs, email forms and databases.
  • If you think of an application scanner as a machine gun, with a machine brain, on a school playground, you will have an accurate if dramatized picture of what can happen when scanners fail to make adjustments for login pages, contact forms and database input forms. Account lockouts, thousands of emails directed at sales and support staff, and database corruption are all problems that scanners can, and routinely do cause when left to define their own targets.

So, if allowing the scanner to find it's own targets through spidering is not a good idea, what is? Before getting to our approach, there is one intuitive solution that needs to be addressed, and that is defining all of the targets in advance. While this seems like a reasonable approach, it is entirely impractical to actually do, at least directly. You might be able to list all of the known web pages, but modern web applications make dozens, sometimes hundreds of requests through java script, style sheets, web services and other similar requests, for each web address that actually appears in your browser's address bar. While theoretically possible, the chance of missing something important is quite high, and so is the amount of time required. This notion is headed in the right general direction though, and there is a practical way to develop such a list. It involves a careful manual review using a proxy, and that gets to our approach.

All of the problems enumerated above can be effectively mitigated if you conduct a manual review before scanning. Here is what our manual application review includes:

  • Full Exercise with Proxy Capture.
  • The first thing our testers do is set up the testing browser to use a proxy tool that captures every web request. From that point on the tester will fully exercise the application, while taking notes about what they see. The main purpose here is to exercise the application fully, capture all of the traffic, gain a full understanding of the application, and take notes.

  • Scope Checks.
  • One of the things our testers look for in this phase is any indication of scope problems that require clarification, such as mixed protocol schemes (https is in scope, but http is not mentioned and we find both), or the application is making requests to different host specifiers than expected, and we need to know if your intent was different than what actually ended up in our scope documents. We try to ensure that scope is accurate before we even begin, but it is important for testers to be very 'scope aware' and to identify any such issues as early as possible and get clarification.

  • Safety and Stability Checks.
  • Our testers are taking notes about many things, but none are more important than identifying potential safety and stability factors. Any login forms, email forms, database forms, or other potential problem areas are identified. Again, we try to identify and address problem areas in our scope documents before testing even begins, but our testers are trained to look for anything that might have been missed in planning.

  • Side Channel Vulnerabilities.
  • While exercising the application, we look at any email, text messaging or other out of band communication sent to us by the web application. This is something that scanners can't do very well, and we look closely at everything the application does.

  • Logical Faults.
  • There may be some issues that can be fully documented into finding reports immediately, without perfoming any testing that would raise flags with an application firewall or IPS, and if so, we do it. This often involves so called 'logical faults', and we find them often in login pages, password reset pages, and account registration pages. For findings that require further intrusive testing, we just take notes for later follow up.

Confirmation.

At this point our initial manual review is complete. Hopefully our tester does not have any scope questions or safety and stability concerns that have not been addressed in our scope documents, but if so, this is the point at which testing will pause until clarified.

Manual Testing.

After the initial review we perform any manual intrusive testing that may be indicated from our manual review of the application. We do it at this point, as quietly as possible, and before launching automated scans.

Automated Scans.

Automated scanning is critical to ensure full testing coverage, but scanners need careful attention. During manual review, our tester has developed a full proxy record of every page request, and a full proxy record of every subsequent request that was generated as a result of exercising the application. All of the java script, style sheet, web service and image requests that would have been nearly impossible to list have been captured. Our tester has also made rational, informed human decisions about what not to do, like continuing to pursue dynamic URLs forever. That resource list now becomes the scanner target list. After checking to insure that the scanner is configured as prescribed in our scope documents, including any scope or safety clarifications we may have received, we launch automated scans.

After automated scans have completed, which may require adjustments for IPS or WAF evasion, we will have a scanner report. We look very closely at the vulnerability scan results. We take note of any identified vulnerabilities and start sorting them into two buckets - those that require further validation and those that are reliable and need no further validation. We are not just looking for vulnerabilities that the scanner identified though. We look at vulnerability scans differently than most. For us, the results are a record of tens or hundreds of thousands of interactions with your application, and we look for anything in those results that seems out of the ordinary at all. It is surprising how often you can find hints that lead you to really serious vulnerabilities when you combine knowledgeable, informed human intuition with scanner output. We look hard.

Manual Testing Again.

Finally, we look at everything that has been identified for further testing. This is the point at which it is impossible to list tools or methods because there is simply too much potential ground to cover, but very often we will use Burp Suite Intruder and/or Repeater. In general, vulnerabilities will fall into three categories at this point:

  • Vulnerabilities that were identified by automation and are reliable. A finding report is prepared, along with any validating evidence from the automated tool.
  • Vulnerabilities that were identified by automation but are not reliable until validated. These are validated using whatever tools or methods are appropriate. Screen captures and other evidence is collected and a finding report is created.
  • Possible vulnerabilities or simple suspicions identified manually. These are all tested, one way or another, until we are convinced that we know what we are seeing, and can either dismiss them or report them.

Web Application Penetration Testing Summary.

This is our standard methodology for a standard testing approach. We adjust as necessary for different testing objectives. Our standard approach is neither fully automated, nor fully manual. It is our opinion that one cannot expect full breadth of coverage without the use of at least some automation, nor can automation be expected to exhibit human intuition and experience.

At High Bit Security, we employ an approach that balances depth and breadth. We use carefully configured automated tools to aid in breadth of testing. We also use extensive manual effort to dig deep into potential security faults, but we stop pursuing depth when we have proven the fault, and have documented the finding in detail. This balanced approach allows for thorough breadth of coverage, sufficient depth, detailed documentation, and above all, safer testing.

Ask us for a free, quick, no hassle quote using the contact form above.