Abstract

As enterprises seek to maximize efficiency, a popular strategy has emerged—outsourcing some parts of business operations to application service providers (ASPs). By engaging an ASP, an enterprise can get expert help in a particular area while lowering costly internal specialization expenses. However, for security professionals, the move to use the ASP model comes at an often-high cost. The ASP may be an expert in its domain, but its security function may be immature. Cisco Systems is offering these ASP security evaluation criteria as a step toward mitigating the risks of outsourcing to ASPs. These criteria are specific measurements of an ASP's security posture and maturity.

Introduction

As enterprises seek to maximize efficiency, a popular strategy has emerged—outsourcing some parts of business operations to application service providers (ASPs). By engaging an ASP, an enterprise can get expert help in a particular area while lowering costly internal specialization expenses. The question enterprises are asking is: “What do we do that is core to our business, and what should we let an expert do for us?”

For instance, a sales organization trying to increase its sales in the public school market might choose to work with an ASP that has focused on gathering the latest information and bid requests from public school districts in the United States. Perhaps the ASP’s service carefully combs this data for the sort of items that the sales organization is selling, and tailors the service for each of its customers. Since the ASP will likely sell this “boutique” service to many clients, it can focus its efforts on doing as thorough a job as possible. And enterprise customers can dispense with internal and perhaps less-effective efforts in favor of the ASP’s expertise and focus.

However, for security professionals, the move to use the ASP model comes at an often-high cost. The ASP may be an expert in its area of expertise, but its security function may be immature. While some ASPs are large and mature enterprises, others are niche players or small “mom-and-pop” shops. Their security can range from excellent to nonexistent. In engaging with an ASP, the corporate security department faces risks, including:

• Loss of control of corporate data
• Loss of control of corporate image
• Insufficient ASP security to mitigate perceived risks
• No ongoing working relationship with the third-party ASP
• The ASP’s possible financial instability
• Exposure of corporate data to the ASP’s other customers
• Compromise or tampering with corporate data

These issues make the task of securing corporate data and image even more difficult than it is within the corporate firewall boundaries. Within corporate boundaries, there is a common management chain. But across corporate boundaries, interests may be divergent. And, legal or purchasing personnel may not consider information security issues as contracts are drawn up.

What can the security person do with these issues? One approach is to ask the ASP to show sufficient security maturity to mitigate the risks of the engagement.

Cisco Systems has established ASP Security Evaluation Criteria to help mitigate the risks of outsourcing to ASPs. These criteria are specific measurements of an ASP’s security posture and maturity. The criteria do not replace certifications like ISO 17799, which measures maturity in ten non-specific categories, or TruSecure, which measures many of the same operational and functional areas as Cisco’s ASP Security Evaluation Criteria. However, not all ASPs are certified. Many are too small—or the engagement with Cisco is too small—to bear the cost burden of such certifications. And, at Cisco, we prefer to assess on a wider scope than the TruSecure certification provides. There is a need to create a set of specific metrics against which to measure an ASP—or any third-party entity whose security is to be measured.

Methodology

The ASP Security Evaluation Criteria were developed at Cisco Systems over several years of reviewing ASP security. A simple questionnaire was used at first; this yielded responses that varied greatly—and were often vague and required extra work on the part of security reviewers to gain clarification. In addition, at some ASPs, the questionnaire was handled by sales people who were reluctant to provide statements that would imply that security was an issue.

Cisco was reaching out to more ASPs. The evaluation team needed a different, streamlined approach, and decided to build a standard of standards—the ASP Security Evaluation Criteria.

The criteria in this document are based upon existing standards that are readily available, either in print form or on the Internet. The criteria use well-known bodies of accepted industry practice, but also allow for “home-grown” solutions that are demonstrably equivalent to the cited standards.

Where a standard is not publicly available, local standards have been surveyed and the most basic, common practice principles available have been summarized. Notably, there is no disaster recovery standard. Since disaster recovery certification is a for-profit business model at the time of this writing, certifiers are not sharing their standards. However, many state governments have posted their local disaster recovery standards; these have been combined into a workable metric against which to assess an ASP’s maturity.

Scope

When an ASP is evaluated, its overall security function is first assessed. Operational aspects include:

• Patch maintenance
• Alert processing
• Incident handling
• The types of firewalls in place and the logical security architecture
• How the physical architecture supports the logical well
• Hardening guides

We also assess the ASP security organization’s level of understanding of the challenges inherent in building strong security. Systems are complex, and security is constantly evolving. We need to know the level of risk in Cisco’s engagement with an ASP, and how those risks will be mitigated. Further, we need to know that we can work with the ASP in the event of a security incident. We need to know that they have given thought, time, and resources to the area of security, and have some level of experience. Questions we ask include:

• Does the ASP have processes in place?
• Does the ASP have a body of policy from which it is working?
• How formal are the ASP’s practices?
• Does the ASP screen its employees before hiring them? If so, how?

The ASP Security Evaluation Criteria cover all of the areas that we use to evaluate an ASP for its security maturity. This includes:

• Network protection
• Incident response
• Host hardening
• Security policies and plans
• Security patching
• Application development
• Encryption requirements
• Password policies and procedures
• Privacy
• Employee background checks
• Disaster recovery
• The ability to respond to security incidents

These areas are similar to ISO 17799; however, the ISO framework does not specify particular answers against which an ASP may be measured. We needed to be very specific so that the ASPs we were evaluating would be able to understand and respond appropriately to our queries. By being specific, the ASP has a clearer understanding of our metrics on every point—and whether their procedures are close to our metrics. The ASP can understand this fairly clearly, as can any reviewer.

We also ask the ASP about their physical security capabilities and their financial viability. However, these questions are outside the scope of the ASP Security Evaluation Criteria.

Using the ASP Security Evaluation Criteria

We ask the ASP to provide documentation that demonstrates that they meet each of our criteria. It is our working theory that a mature security organization will have at least some documentation, policies, standards, and procedures in each area that we evaluate. The content in this body of documentation should cover the content specified in the criteria. The ASP Security Evaluation Criteria do not specify a format or a manner of composition—one organization’s policy may be quite specific, while another organization may have placed the specifics in a standard or a guideline. These distinctions are immaterial; the criteria focus on content rather than format.

There is some risk inherent in an evaluation approach that does not audit its findings. Each engagement with a third party should first be evaluated for obvious risk areas and exposures. A reviewer can use this information to assess whether there is sufficient risk exposure to warrant any fact checking or audit. For low-risk engagements, it may be enough to understand that the third party has an idea of what are reasonable security measures. As risk exposure increases, reviewers may wish to build more assurance into their review processes. Assurance can be added through simple network scans, port mapping and fingerprinting, or even physical visits. These are beyond the scope of the ASP Criteria.

After the ASP provides its documentation, we then evaluate the provided documentation against the criteria. Each criteria section is graded: “pass,” “caution,” or “fail.” Pass/fail alone did not give us all of the information that we needed—when a specific area of criteria is considered a necessary part of a reasonable security mix, we might not be willing to compromise with a partial fulfillment. For instance, if Cisco’s data will be exposed to the public Internet, we would insist upon firewall layers (at least two), and a DMZ area, and other strong security practices. But, if the data was not being stored by the ASP, we might allow less-stringent database controls.

We recognize the importance of the insight of an experienced security engineer in our process. We perform a pre-review to assess exposure, high-level architecture, and data flow. And we do a post-review with the ASP over the findings to sort out any issues for remediation. In all of these areas, the judgment based upon a body of experience is invaluable. And, performing ASP reviews provides a strong experiential training ground for security—the reviewer touches many areas.

In your evaluation process, you may choose to outsource your reviews and provide the criteria to a third-party organization. At Cisco, we sometimes work with a trusted external organization, who performs the criteria-based review and provides findings for the post-review work with the ASP. This is a “core vs. context” decision. Since using the criteria for the review creates a repeatable process, it is possible for us to outsource this area without losing any control over the relationship with the ASP.

Depending upon the risk profile of the engagement, we may also audit the results of the review. Some of the auditing options that we use might be:

• Port/service network scans
• Application vulnerability scans
• Interviews of ASP personnel
• Physical visits

While we rarely perform penetration testing, scans can be quite useful. One ASP was subcontracting its hosting to a third party. Though the ASP had specified a specific firewall configuration for specific ports, it had never tested the firewall. The third party had failed to provide any configuration to the firewall. In effect, there was no firewall, although the equipment had been deployed. A simple port scan showed the problem.

These hosting arrangements are typical. While the enterprise contract is with the first party, there may be second, third, or even fourth parties as well. We ask that all of the parties participate in the review.

It is Cisco’s hope that the ASP Security Evaluation Criteria can become an evolving resource in the toolkit of corporate information security departments. As our body of practice and standards mature, so will our ability to assess third-party security capabilities. In the meantime, we can use the ASP Security Evaluation Criteria to help us assess how well we can partner with other entities who may have responsibility for our corporate data.

Application Service Provider Security Evaluation Criteria

Overview

1. This document sets for the criteria by which Application Service Providers (ASP) security posture may be evaluated. Evaluators may be potential customers or business partners of the ASP, or the evaluation might be conducted by an independent auditor.
2. The auditor will evaluate whether or not the ASP’s security related policies and procedures meet industry standard best practices in those areas where customer or partner data may be processed by the ASP. For the purposes of this document, it will be assumed that “customer” means both the ASP customers and ASP partners.
3. The ASP must meet or exceed the criteria in this document for all relevant sections.

Scope

1. The ASP must meet all requirements in this document for which the ASP has control and responsibility. If responsibility lies with another party, then that party must be reviewed for the requirements under that party’s control.
2. All the security requirements in this document pertain to networks, equipment, and user access that has any network connectivity, data storage/caching, or processing relationship with any of:
• Data owned by ASP customers.
• Data or results produced for ASP customers.
• Any application that will be accessed by employees, business partners, or customers of ASP customers.
  
  

Best Practice Security Requirements

Network Protection

• If ASP has an Internet connection, the network design of layers exposed to the Internet must meet or exceed the following specifications:
• SMTP server - must:
• Act only as a relay between the Internet and the Internal mail
• Inspect content
• Firewall - must
• Be placed between the Internet and all other networks of the ASP.
• Provide network-level protection of resources and stateful filtering of traffic.
• Be placed between exposed networks and any other ASP networks. For example, the ASP must have layered security architecture.
• If the ASP allows VPN access into customer related networks, its network design must implement the following:
• VPN Concentrator must authenticate individual remote users using Extended Authentication (XAUTH) and terminate their IPSec tunnels
• VPN Router - must authenticate trusted remote sites and provide connectivity using GRE/IPSec tunnels
• Dial-In Server - must authenticate individual remote users using TACACS+/Radius and terminate their analog connections
• Passwords used for remote access must meet the following criteria
• Passwords should be 2 factor. Smartcards or one-time-passwords fulfill this criteria.
• If single factor, password authentication is used, then passwords must be
• A minimum of 10 characters.
• Contain no dictionary words.
• Be of mixed case.
• Include both alphanumeric and non-alphanumeric characters.
• Be changed at least every 180 days.
• Password strength must be ensured through automated means.
• Fixed passwords must not be guessable or recoverable.
• Firewall - must provide differentiated security for any of the three different types of remote access
• NIDS appliance - must provide Layer 4 to Layer 7 monitoring of key network segments used for remote access.
• If the ASP exposes any applications to a public network like the Internet, its network design must implement the following:
• Unauthorized Access - must employ stateful firewalling and ACL's to limit exposure to specific protocols
• IP Spoofing - Must implement RFC 2827 prevent locally originated spoofed packets and limit remote spoof attempts. May also implement RFC 1918 filtering for this purpose.
• Network Reconnaissance - ports must be limited to only what is necessary and ICMP must be restricted
• Trust Exploitation - firewalls must be installed and configured to ensure communication flows only in the proper direction on the proper service
• An ASP may be required to have the following when data sensitivity is high or more sensitive.
• IDS - must provide Layer 4 to Layer 7 monitoring of network segments on which sensitive customer data will be routed, processed or stored.
• Router hardening.
• Must be hardened to the requirements of the NSA's router hardening specification:
• http://nsa2.www.conxion.com/cisco/guides/cis-2.pdf

Incident Response

There must be an incident response team and process. These should meet or exceed:

• http://www.faqs.org/rfcs/rfc2350.html
• The ASP must provide and keep current ASP incident response personnel contact information.
• The ASP must provide a process to customer for notifying customer's information security in the event of a security incident at the ASP that meets any of the following conditions:
• • Impacts any operation of the ASP that falls within the "Scope" section of this document.
  • Impacts any part of the ASP's management infrastructure, whether or not it has any relationship to the ASP's engagement with customer. However management infrastructure that is used exclusively by other customers of the ASP is except from this notification requirement.

Host Hardening

1. Hosts which are exposed to a public network, such as the Internet or hosts which contain highly sensitive or greater sensitivity of customer data must be hardened to the relevant CIS hardening guide: http://www.cisecurity.org
2. If there is no CIS hardening guide published for an ASP's host OS, or OS revision, the ASP may show adequate host hardening practices through one or more of the following:
3. • The ASP may substitute another published industry consensus approved hardening guide for the host. Or
   • The ASP may submit a hardening guide and logs of the ASP hardening process or other proof of having performed the hardening process to the reviewer. The reviewer may analyze and consider these documents as sufficient to fulfill the requirement of industry best practice host hardening.

Security Policies and Plans

• An ASP must have sufficient security policies and plans. These should meet or exceed RFC 2196 Site Security Handbook in all areas which are relevant to the services and protocols being run by the ASP, and the network connections and architecture. For example, if a particular protocol, like FTP is not used by the ASP, then the provisions in RFC 2196 for FTP may be disregarded.

Security Patching

The ASP must address every subsection of section 6, "Ongoing Activities" in RFC 2196 Site Security Handbook. And, specifically, the ASP must demonstrate the following:

1. The ASP must have documented: a maintenance patch schedule, a process for applying maintenance patches, and a testing process for testing patches before applying them.
2. A security patch policy and process must meet the following requirements:
3. • A means for receiving timely security notifications must be demonstrated. These may include but are not limited to:
   • • Membership or participation in security alert organizations.
     • Membership in one or more security alert mailing lists, like Bugtraq.
     • Personnel who's duties include monitoring of security alerts postings or emailings.
     • • A documented process for reacting to relevant security alerts.
   • A means to determine criticality of patches.
   • • An alert evaluation process
     • An alert evaluation team
     • Documented criteria for determining the criticality of patches.
   • A means to apply patches at varying levels of criticality
   • A matrix of criticality of patch versus time to apply patches. E.g., non-critical patches might be applied every quarter, while highly critical ones must have a short turn around time
   • The ASP must provide the shortest timeline that it has for applying a high risk patch.
4. There must be a test procedure for patches before applying them.
5. The must be quality assurance criteria that patches must meet before application.

Application Development

The ASP must:

1. Demonstrate that it meets of exceeds the coding practices in Security Focus Secure Coding Practices, Writing Secure Code by Michael Howard and David LeBlanc, Microsoft Press, 2002, SANS secure coding papers, http://www.sans.org/rr/catindex.php?cat_id=46, or similar document.
2. Demonstrate that it meets of exceeds the Software Industry QA practices, by meeting or exceeding, 1061-1998 IEEE Standard for a Software Quality Metrics Methodology 1998, or similar standard or certification.
3. Demonstrate that it meets of exceeds the CERT Malicious Code Mitigation web security coding best practice.
4. Maintain up-to-date anti-virus scanning and detection software on any of the following systems:
5. • All Microsoft Windows systems.
   • Any systems involved in the uploading of content. (This would include Web servers, file servers, and FTP servers)

Encryption Requirements

All encryption used in conjunction with customer data or users must meet the following requirements:

• Proven, standard algorithms such as AES, 3DES, Blowfish, RSA, RC5 and IDEA must be used as the basis for encryption technologies. These algorithms represent the actual cipher used for an approved application. For example, PGP Corp's Pretty Good Privacy (PGP) uses a combination of IDEA and RSA, while the Secure Sockets Layer (SSL) technology utilizes RSA and RC5.
• Symmetric cryptosystem key lengths must be at least 128 bits. Asymmetric crypto-system keys must be of a length that yields equivalent strength. customer's key length requirements will be reviewed annually and upgraded as technology allows.
• Cryptographic authentication must utilize standardized Keyed Hash Message Authentication Code (HMAC)-based algorithms, such as HMAC-MD5 or HMAC-SHA-1
• Due to the fact that the standard Diffie-Hellman-Merkle algorithm is vulnerable to man-in-the middle attacks, the use of unauthenticated Diffie-Hellman-Merkle exchanges is prohibited.
• The use of proprietary encryption algorithms is not allowed for any purpose.

Password Policies and Procedures

1. All passwords must be encrypted while kept in non-volatile storage.
2. Application password policies must have the following characteristics at a minimum: Contain both upper and lower case characters, have digits and punctuation characters as well as letters, and are at least eight characters long.
3. The application password policy must be enforced by the application and not rely upon voluntary compliance.
4. If your application creates default passwords for new users, those passwords must be changed on first login. The application should force this behavior and not rely on voluntary compliance by the user.

Privacy

1. All ASPs interacting with individual users (customer employees, customers, partners, the general public, etc.) must have a privacy policy available for viewing by those users. The policy must include the following:
2. • A statement informing the user that this is a site maintained by a third party and NOT customer systems (even if the site is customer-branded).
   • Information stating what personal information is collected, and why.
   • What information, if any, is transmitted to additional third parties.
   • Contact information (e.g. privacy@yourcompany.com) for users to report privacy concerns/questions. Someone at the third party must have the responsibility of responding to privacy concerns and interacting with customer's privacy function.
   • A statement informing users that customer is notified whenever there is reasonable belief that unencrypted personal information has been disclosed to an unauthorized party.
   • The ASP must put in place facilities to encrypt the following protected information for residents of California:
   • • First & Last Name
     • Social Security Number
     • Driver's License or California ID number
     • Account number, credit or debit card number, in combination with any required security code, access code, or password that would permit access to an individual's financial account.
3. The requirement to encrypt covers both the transmission and storage of the protected information.
4. The ASP must demonstrate that they have the ability to protect the data as described above as well as the ability detect a breach of security which may result in the unencrypted disclosure of the protected information. The responsibility is upon the ASP to prove to customer that they meet this test, and if they cannot meet this test, they will not be approved for the purposes of hosting protected data.
5. The ASP must have a reporting and notification mechanism in place to notify their customer's information security teams when there is reasonable belief that their unencrypted personal information has been disclosed to an unauthorized party. This mechanism must be documented and provided to the customer's information security teams, as well as noted in the privacy policy of the ASP.

Miscellaneous Requirements

1. All personnel who will have access to the environment and/or customer's data must have a professionally performed background check completed prior to working on the customer environment. The background check must meet the following criteria:
2. • For each of ASP's employees assigned to work on a customer contract, the ASP has conducted a criminal records check for all known residences or obtained a police certificate of good conduct (for countries that will not provide criminal history records).
   • For each of the ASP's employees assigned to work on a customer contract, the ASP has conducted a Social Security check to verify all of the counties the ASP employee has resided in for the last five years. For international employees, you have verified residences through the National Identification Number or driving records.
   • ASP is required to establish a Records Retention Policy of 3.5 years plus the current year for the background verification file of each ASP employee assigned to work on a customer contract.
3. The ASP must have a disaster recovery plan which meets the following criteria:
4. • An applications and data criticality analysis (an entities formal assessment of the sensitivity, vulnerabilities, and security of its programs and information it receives, manipulates, stores, and/or transmits).
   • A data backup plan (a documented and routinely updated plan to create and maintain, for a specific period of time, retrievable exact copies of information).
   • A disaster recovery plan (the part of an overall contingency plan that contains a process enabling an enterprise to restore any loss of data in the event of fire, vandalism, natural disaster, or system failure).
   • An emergency mode operation plan (the part of an overall contingency plan that contains a process enabling an enterprise to continue to operate in the event of fire, vandalism, natural disaster, or system failure).
   • Testing and revision procedures (the documented process of periodic testing of written contingency plans to discover weaknesses and the subsequent process of revising the documentation, if necessary).
   • The ASP must have put into service sufficient equipment and business processes that will allow the ASP to effectively execute its Disaster Recovery plan.
   • Provide an SLA to the customer for the amount of time within which critical customer operations would be restored.
5. The ASP must provide documentation or other proof of the ability to immediately disable all or part of the functionality of the ASP's application should a security issue be identified.
6. The ASP equipment hosting the application for customer must be located in a physically secure facility, which requires badge access at a minimum.
7. The infrastructure (hosts, network equipment, etc.) hosting the customer application must be located in a locked cage-type environment.
8. If the value of the customer data is high, the ASP may be required to provide an infrastructure that is air gapped from other networks the ASP may have which is separate from any other customer's infrastructure as well as the ASP's corporate environment.
9. If the ASP is handling sensitive customer data, the ASP must demonstrate proof of sufficient general liability insurance to cover the value of the customer data's loss.

Acknowledgments

The author Brook Schoenfield is a Senior Security Architect with Corporate Security Programs Organization at Cisco Systems Inc. and specializes in Internet security.

---

This document is part of the Cisco Security Center.

This document is provided on an "as is" basis and does not imply any kind of guarantee or warranty, including the warranties of merchantability or fitness for a particular use. Your use of the information on the document or materials linked from the document is at your own risk. Cisco reserves the right to change or update this document at any time.

Back to Top

Cisco Security Center