Software Development Security · Topic 3 of 4
Assessing Software Security and Acquired Software
Measuring software security with audits, logging, risk analysis and metrics, and assessing commercial, open source, third-party and managed service software.
- 3 min
- Measure software security with audits, change logs, risk analysis and trend metrics.
- Evaluate tools by accuracy on known flaws, not finding counts.
- Assess COTS, open source, custom and managed services differently.
What it is
Managers need evidence that software security is working, both for software built in-house and for software the organization acquires. Acquired software brings risks the organization cannot directly control, so it must be assessed before purchase and monitored during use.
How it works
Assessing software security effectiveness
| Method | What it gives |
|---|---|
| Auditing | Independent check that secure development practices are followed |
| Logging of changes | Traceability of who changed what in code and configuration |
| Risk analysis and mitigation | Prioritizing weaknesses by business impact |
| Security testing results | Findings from SAST, DAST, penetration tests over time |
| Metrics | Trends that show whether the program improves |
Useful metrics:
| Metric | Shows |
|---|---|
| Mean time to remediate | How quickly confirmed vulnerabilities are fixed |
| Defect density | Defects per unit of code size |
| Code coverage | Proportion of code exercised by tests |
| Vulnerability recurrence | Whether the same flaw types keep returning |
| Change failure rate | How often releases need rollback or hotfix |
Evaluating security tools objectively
When comparing tools, measure them on your own technology: scan a test application seeded with known flaws and compare detection and false positive rates. Vendor demonstrations and raw finding counts reward noise.
The number of findings a tool reports is not a measure of quality. Accuracy against known flaws is.
Acquired software types
| Type | Key risks | Assessment and controls |
|---|---|---|
| Commercial off-the-shelf (COTS) | Vendor flaws, slow patching, end of support | Vendor security evidence, patch commitments, testing in your environment |
| Open source | Abandoned projects, vulnerable dependencies, licence obligations | SCA, community health, maintenance plans, licence review |
| Third-party / custom developed | Quality of the developer's practices | Contract security requirements, code review rights, escrow |
| Managed services (SaaS, PaaS, IaaS) | Data handled by the provider, shared responsibility | SOC 2 or ISO reports, contracts, configuration reviews, exit plans |
Before and after acquisition
| Stage | Activities |
|---|---|
| Before | Define security requirements; review vendor evidence; test; check licensing and data handling |
| Contract | Security obligations, vulnerability disclosure and patch timelines, audit rights, breach notification, data return and deletion |
| During use | Patch promptly after testing; monitor advisories; review access and configuration |
| End | Plan migration before end of support; retrieve and delete data |
Patching acquired software safely
When a vendor patch might break a custom integration, test it against that integration in a staging environment first, then deploy on evidence. Piloting directly in production risks the feared breakage; isolating systems indefinitely delays the fix.
Software escrow
Source code escrow places the vendor's source with a third party, released to the customer if the vendor fails, so critical software can still be maintained.
How the exam asks about this
- "Which metric shows how fast confirmed vulnerabilities are fixed?" Mean time to remediate.
- "Compare two SAST products on our own stack." Scan a seeded test application and compare detection and false positive rates.
- "A critical vendor may go out of business." Source code escrow.
What to remember for the exam
- Measure software security with audits, change logs, risk analysis and trend metrics.
- Evaluate tools by accuracy on known flaws, not finding counts.
- Assess COTS, open source, custom and managed services differently.
- Contracts carry security, patching, audit and data obligations.
- Test vendor patches in staging; use escrow for critical vendor software.
Open-source components are acquired software too. Unmaintained libraries inside your products are a risk you own.
Common mistakes
Trusting vendor marketing instead of evidence.
Ignoring licence obligations of open-source components.
Buying SaaS without an exit plan for the data.
Check yourself
What does code coverage measure?
The proportion of code executed when the test suite runs.
What protects a customer if a software vendor stops supporting critical custom software?
Source code escrow.
What evidence helps assess a SaaS provider's controls?
Independent audit reports such as SOC 2 or ISO/IEC 27001 certification, plus contract terms.
Topic 3 of 4 in Software Development Security
Test yourself on Software Development Security
The full ISC2 CISSP question bank has 1,113 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by ISC2. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.