SUMMARY

The Cybersecurity and Infrastructure Security Agency (CISA) and the following partners (hereafter referred to as the authoring organizations) are releasing this joint Cybersecurity Advisory to warn that cyber threat actors are exploiting previously identified vulnerabilities in Ivanti Connect Secure and Ivanti Policy Secure gateways. CISA and authoring organizations appreciate the cooperation of Volexity, Ivanti, Mandiant and other industry partners in the development of this advisory and ongoing incident response activities. Authoring organizations:

• Federal Bureau of Investigation (FBI)
• Multi-State Information Sharing & Analysis Center (MS-ISAC)
• Australian Signals Directorate’s Australian Cyber Security Centre (ASD’s ACSC)
• United Kingdom National Cyber Security Centre (NCSC-UK)
• Canadian Centre for Cyber Security (Cyber Centre), a part of the Communications Security Establishment
• New Zealand National Cyber Security Centre (NCSC-NZ)
• CERT-New Zealand (CERT NZ)

Of particular concern, the authoring organizations and industry partners have determined that cyber threat actors are able to deceive Ivanti’s internal and external Integrity Checker Tool (ICT), resulting in a failure to detect compromise.

Cyber threat actors are actively exploiting multiple previously identified vulnerabilities—CVE-2023-46805, CVE-2024-21887, and CVE-2024-21893—affecting Ivanti Connect Secure and Ivanti Policy Secure gateways. The vulnerabilities impact all supported versions (9.x and 22.x) and can be used in a chain of exploits to enable malicious cyber threat actors to bypass authentication, craft malicious requests, and execute arbitrary commands with elevated privileges.

During multiple incident response engagements associated with this activity, CISA identified that Ivanti’s internal and previous external ICT failed to detect compromise. In addition, CISA has conducted independent research in a lab environment validating that the Ivanti ICT is not sufficient to detect compromise and that a cyber threat actor may be able to gain root-level persistence despite issuing factory resets.

The authoring organizations encourage network defenders to (1) assume that user and service account credentials stored within the affected Ivanti VPN appliances are likely compromised, (2) hunt for malicious activity on their networks using the detection methods and indicators of compromise (IOCs) within this advisory, (3) run Ivanti’s most recent external ICT, and (4) apply available patching guidance provided by Ivanti as version updates become available. If a potential compromise is detected, organizations should collect and analyze logs and artifacts for malicious activity and apply the incident response recommendations within this advisory.

Based upon the authoring organizations’ observations during incident response activities and available industry reporting, as supplemented by CISA’s research findings, the authoring organizations recommend that the safest course of action for network defenders is to assume a sophisticated threat actor may deploy rootkit level persistence on a device that has been reset and lay dormant for an arbitrary amount of time. For example, as outlined in PRC State-Sponsored Actors Compromise and Maintain Persistent Access to U.S. Critical Infrastructure), sophisticated actors may remain silent on compromised networks for long periods. The authoring organizations strongly urge all organizations to consider the significant risk of adversary access to, and persistence on, Ivanti Connect Secure and Ivanti Policy Secure gateways when determining whether to continue operating these devices in an enterprise environment.

Note: On February 9, 2024, CISA issued Emergency Directive (ED) 24-01: Mitigate Ivanti Connect Secure and Ivanti Policy Secure Vulnerabilities, which requires emergency action from Federal Civilian Executive Branch (FCEB) agencies to perform specific actions on affected products.

The Canadian Centre for Cyber Security also issued an alert, Ivanti Connect Secure and Ivanti Policy Secure gateways zero-day vulnerabilities, which provides periodic updates for IT professionals and managers affected by the Ivanti vulnerabilities.

Download the PDF version of this report:

For a downloadable copy of IOCs, see:

TECHNICAL DETAILS

This advisory uses the MITRE ATT&CK^® for Enterprise framework, version 14. See the MITRE ATT&CK Tactics and Techniques in Appendix C for a table of the threat actors’ activity mapped to MITRE ATT&CK tactics and techniques. For assistance with mapping malicious cyber activity to the MITRE ATT&CK framework, see CISA and MITRE ATT&CK’s Best Practices for MITRE ATT&CK Mapping and CISA’s Decider Tool.

Overview

On January 10, 2024, Volexity reported on two vulnerabilities in Ivanti Connect Secure and Ivanti Policy Secure gateways observed being chained to achieve unauthenticated remote code execution (RCE):[1]

Volexity first identified active exploitation in early December 2023, when they detected suspicious lateral movement [TA0008] on the network of one of their network security monitoring service customers. Volexity identified that threat actors exploited the vulnerabilities to implant web shells, including GLASSTOKEN and GIFTEDVISITOR, on internal and external-facing web servers [T1505.003]. Once successfully deployed, these web shells are used to execute commands on compromised devices.[1]

After Ivanti provided initial mitigation guidance in early January, threat actors developed a way to bypass those mitigations to deploy BUSHWALK, LIGHTWIRE, and CHAINLINE web shell variants.[2] Following the actors’ developments, Ivanti disclosed three additional vulnerabilities:

• CVE-2024-21893 is a server-side request forgery vulnerability in the SAML component of Ivanti Connect Secure (9.x, 22.x) Ivanti Policy Secure (9.x, 22.x), and Ivanti Neurons for ZTA that allows an attacker to access restricted resources without authentication.
• CVE-2024-22024 is an XML vulnerability in the SAML component of Ivanti Connect Secure (9.x, 22.x), Ivanti Policy Secure (9.x, 22.x), and ZTA gateways that allows an attacker to access restricted resources without authentication.
• CVE-2024-21888 is a privilege escalation vulnerability found in the web component of Ivanti Connect Secure and Ivanti Policy Secure. This vulnerability allows threat actors to gain elevated privileges to that of an administrator.

Observed Threat Actor Activity

CISA has responded to multiple incidents related to the above vulnerabilities in Ivanti Connect Secure and Policy Secure Gateways. In these incidents, actors exploited these CVEs for initial access to implant web shells and to harvest credentials stored on the devices. Post-compromise, the actors moved laterally into domain environments and have been observed leveraging tools that are native to the Ivanti appliances—such as freerdp, ssh, telnet, and nmap libraries—to expand their access to the domain environment. The result, in some cases, was a full domain compromise.

During incident response investigations, CISA identified that Ivanti’s internal and external ICT failed to detect compromise. The organizations leveraged the integrity checker to identify file mismatches in Ivanti devices; however, CISA incident response analysis confirmed that both the internal and external versions of the ICT were not reliable due to the existence of web shells found on systems that had no file mismatches according to the ICTs. Additionally, forensic analysis showed evidence the actors were able to clean up their efforts by overwriting files, time-stomping files, and re-mounting the runtime partition to return the appliance to a “clean state.” This reinforces that ICT scans are not reliable to indicate previous compromise and can result in a false sense of security that the device is free of compromise.

As detailed in Appendix A, CISA conducted independent research in a lab environment validating that the ICT is likely insufficient for detecting compromise and that a cyber threat actor may be able to maintain root level persistence despite issuing factory resets and appliance upgrades.

INDICATORS OF COMPROMISE

See Tables 1 – 4 in Appendix B for IOCs related to cyber actors exploiting multiple CVEs related to Ivanti appliances.

For additional indicators of compromise, see:

Memory and disk forensics were used during forensic analysis, combined with the Integrity Checker Tool, to identify malicious files on the compromised Ivanti Connect Secure VPN appliance. This advisory provides a list of combined authoring organization IOCs and open source files identified by Volexity via network analysis.

Disclaimer: Some IP addresses in this advisory may be associated with legitimate activity. Organizations are encouraged to investigate the activity around these IP addresses prior to taking action such as blocking. Activity should not be attributed as malicious without analytical evidence to support it is used at the direction of, or controlled by, threat actors.

DETECTION METHODS

YARA Rules

See Appendix D for additional open source YARA rules, provided by Volexity, that may aid network defenders in detecting malicious activity within Ivanti Connect Secure VPN appliances. For more information on detection methods, visit Mandiant’s blog post Cutting Edge, Part 2: Investigating Ivanti Connect Secure VPN Zero-Day Exploitation or the Volexity GitHub page.

INCIDENT RESPONSE

The authoring organizations encourage you to assess your organization’s user interface (UI) software and systems for evidence of compromise and to hunt for malicious activity using signatures outlined within this advisory. If compromise is suspected or detected, organizations should assume that threat actors hold full administrative access and can perform all tasks associated with the Ivanti Connect Secure VPN appliance as well as executing arbitrary code and installing malicious payloads.

Note: These are vendor-managed appliances and systems may be encrypted with limited access. Thus, collecting artifacts may be limited on some versions of appliances. The authoring organizations recommend investigating associated devices on the network to identify lateral movement in the absence of access to the Secure Connect appliance.

If a potential compromise is detected, organizations should:

1. Quarantine or take offline potentially affected hosts.
2. Reimage compromised hosts.
3. Reset all credentials that may have been exposed during the compromise, including user and service accounts.
4. Identify Ivanti hosts with Active Directory (AD) access, threat actors can trivially export active domain administrator credentials during initial compromise. Until there is evidence to the contrary, it is assumed that AD access on compromised systems is connected to external authentication systems such as Lightweight Directory Access Protocol (LDAP) and AD.
5. Collect and review artifacts such as running processes/services, unusual authentications, and recent network connections.
   • Note: Removing malicious administrator accounts may not fully mitigate risk considering threat actors may have established additional persistence mechanisms.
6. Report the compromise to FBI Internet Crime Complaint Center (IC3) at IC3.gov, local FBI field Office, or CISA via the agency’s Incident Reporting System or its 24/7 Operations Center (report@cisa.gov or 888-282-0870). State, local, tribal, or territorial government entities can also report to MS-ISAC (SOC@cisecurity.org or 866-787-4722). Organizations outside of the United States should contact their national cyber center. (See the Reporting section.)

MITIGATIONS

These mitigations apply to all critical infrastructure organizations and network defenders using Ivanti Connect Secure VPN and Ivanti Policy Secure. The authoring organizations recommend that software manufacturers incorporate Secure by Design principles and tactics into their software development practices. These principles and tactics can limit the impact of exploitation—such as threat actors leveraging newly discovered, unpatched vulnerabilities within Ivanti appliances—thus, strengthening the secure posture for their customers.

For more information on secure by design, see CISA’s Secure by Design webpage and joint guide.

The authoring organizations recommend organizations implement the mitigations below to improve your cybersecurity posture based on threat actor activity and to reduce the risk of compromise associated with Ivanti vulnerabilities. These mitigations align with the cross-sector Cybersecurity Performance Goals (CPGs) developed by CISA and the National Institute of Standards and Technology (NIST). The CPGs provide a minimum set of practices and protections that CISA and NIST recommend all organizations implement. CISA and NIST based the CPGs on existing cybersecurity frameworks and guidance to protect against the most common and impactful threats, tactics, techniques, and procedures. Visit CISA’s Cross-Sector Cybersecurity Performance Goals for more information on the CPGs, including additional recommended baseline protections.

• As organizations make risk decisions in choosing a VPN, to include decisions regarding continued operation of Ivanti Connect Secure and Policy Secure gateways, avoid VPN solutions that use proprietary protocols or non-standard features. VPNs as a class of devices carry some specific risks that a non-expert implementer may trigger (e.g., authentication integration and patching). When choosing a VPN, organizations should consider vendors who:
  • Provide a Software Bill of Materials (SBOM) to proactively identify, and enable remediation of, embedded software vulnerabilities, such as deprecated operating systems.
  • Allow a restore from trusted media to establish a root of trust. If the software validation tooling can be modified by the software itself, there is no way to establish a root of trust other than returning the device to the manufacturer (return material authorization [RMA]).
  • Are a CVE Numbering Authority (CNA) so that CVEs are assigned to emerging vulnerabilities in a timely manner.
  • Have a public Vulnerability Disclosure Policy (VDP) to enable security researchers to proactively share and disclose vulnerabilities through coordinated vulnerability disclosure (CVD).
  • Have in place a clear end-of-life policy (EoL) to prepare customers for updating to supported product versions.
• Limit outbound internet connections from SSL VPN appliances to restrict access to required services. This will limit the ability of an actor to download tools or malware onto the device or establish outbound connections to command and control (C2) servers.
• Ensure SSL VPN appliances configured with Active Directory or LDAP authentication use low privilege accounts for the LDAP bind.
• Limit SSL VPN connections to unprivileged accounts only to help limit the exposure of privileged account credentials.
• Keep all operating systems, software, and firmware up to date. Timely patching is one of the most efficient and cost-effective steps an organization can take to minimize its exposure to cybersecurity threats. Organizations should patch vulnerable software and hardware systems within 24 to 48 hours of vulnerability disclosure. Prioritize patching known exploited vulnerabilities in internet-facing systems [CPG 1.E].
• Secure remote access tools.
  • Implement application controls to manage and control execution of software, including allowlisting remote access programs. Application controls should prevent installation and execution of portable versions of unauthorized remote access and other software. A properly configured application allowlisting solution will block any unlisted application execution. Allowlisting is important because antivirus solutions may fail to detect the execution of malicious portable executables when the files use any combination of compression, encryption, or obfuscation.
• Strictly limit the use of Remote Desktop Protocols (RDP) and other remote desktop services. If RDP is necessary, rigorously apply best practices, for example [CPG 2.W]:

• Configure the Windows Registry to require User Account Control (UAC) approval for any PsExec operations requiring administrator privileges to reduce the risk of lateral movement by PsExec.
• Implement a recovery plan to maintain and retain multiple copies of sensitive or proprietary data and servers in a physically separate, segmented, and secure location (e.g., hard drive, storage device, or the cloud).
• Require all accounts with password logins (e.g., service account, admin accounts, and domain admin accounts) to comply with NIST’s standards for developing and managing password policies.
  • Use longer passwords consisting of at least 15 characters [CPG 2.B].
  • Store passwords in hashed format using industry-recognized password managers.
  • Add password user “salts” to shared login credentials.
  • Avoid reusing passwords [CPG 2.C].
  • Implement multiple failed login attempt account lockouts [CPG 2.G].
  • Disable password “hints.”
  • Require administrator credentials to install software.
• Review the CISA and NSA joint guidance for Selecting and Hardening Remote Access VPN Solutions.

VALIDATE SECURITY CONTROLS

In addition to applying mitigations, the authoring organizations recommend exercising, testing, and validating your organization’s security program against the threat behaviors mapped to the MITRE ATT&CK for Enterprise framework in this advisory. The authoring organizations recommend testing your existing security controls inventory to assess how the controls perform against the ATT&CK techniques described in this advisory.

To get started:

1. Select an ATT&CK technique described in this advisory (Appendix C).
2. Align your security technologies against the technique.
3. Test your technologies against the technique.
4. Analyze your detection and prevention technologies’ performance.
5. Repeat the process for all security technologies to obtain a set of comprehensive performance data.
6. Tune your security program, including people, processes, and technologies, based on the data generated by this process.

The authoring organizations recommend continually testing your security program, at scale, in a production environment to ensure optimal performance against the MITRE ATT&CK techniques identified in this advisory.

REPORTING

U.S. organizations should report every potential cyber incident to the U.S. government. When available, each report submitted should include the date, time, location, type of activity, number of people, and type of equipment used for the activity, the name of the submitting company or organization, and a designated point of contact. Reports can be submitted to the FBI’s Internet Crime Complaint Center (IC3), local FBI Field Office, or CISA via the agency’s Incident Reporting System or its 24/7 Operations Center at report@cisa.gov or (888) 282-0870.

The FBI encourages organizations to report information concerning suspicious or criminal activity to their local FBI Field Office.

Australian organizations that have been impacted or require assistance regarding Ivanti compromise, contact ASD’s ACSC via 1300 CYBER1 (1300 292 371), or by submitting a report to cyber.gov.au.

UK organizations that have been impacted by Ivanti compromise, should report the incident to the National Cyber Security Centre.

Organizations outside of the United States or Australia should contact their national cyber center.

REFERENCES

1. Active Exploitation of Two Zero-Day Vulnerabilities in Ivanti Connect Secure VPN | Volexity
2. Ivanti Connect Secure VPN Exploitation Goes Global | Volexity
3. KB CVE-2023-46805 (Authentication Bypass) & CVE-2024-21887 (Command Injection) for Ivanti Connect Secure and Ivanti Policy Secure Gateways
4. Cutting Edge, Part 2: Investigating Ivanti Connect Secure VPN Zero-Day Exploitation | Mandiant

DISCLAIMER

The information in this report is being provided “as is” for informational purposes only. CISA and authoring organizations do not endorse any commercial entity, product, company, or service, including any entities, products, or services linked within this document. Any reference to specific commercial entities, products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply endorsement, recommendation, or favoring by CISA and authoring organizations.

ACKNOWLEDGEMENTS

Volexity, Mandiant, and Ivanti contributed to this advisory.

VERSION HISTORY

February 29, 2024: Initial version.

APPENDIX A: CISA’S PRODUCT EVALUATION FINDINGS

Research Approach

As part of ongoing efforts to effectively serve the cybersecurity community with actionable insights and guidance, CISA conducted research by using a free and downloadable version of the Ivanti Connect Secure virtual appliance to assess potential attack paths and adversary persistence mechanisms. The virtual appliances were not connected to the internet, and were deployed in a closed virtualized network, with a non-internet connected Active Directory. This research included a variety of tests on version 22.3R1 Build 1647, connected to Active Directory credentials, to leverage the access obtained through CVE-2023-46805, CVE-2024-21887 and CVE-2024-21893. Put simply, CISA’s research team wanted to answer the question: “How far could an attacker go if they set were to exploit these CVEs remotely?”

Persistent Post-Reset and -Upgrade Access

Leveraging these vulnerabilities, CISA researchers were able to exfiltrate domain administrator cleartext credentials [TA0006], gain root-level persistence [TA0003], and bypass integrity checks used by the Integrity Checker application. CISA’s Incident Response team observed these specific techniques leveraged during the agency’s incident response engagements, along with the native tools and libraries to conduct internal reconnaissance and compromise domains behind the Ivanti appliances. CISA researchers assess that threat actors are able to use the credentials to move deeper into the environment.

The ability to exfiltrate domain administrator cleartext credentials, if saved when adding an “Active Directory Authentication server” during setup, was accomplished by using the root-level access obtained from the vulnerabilities to interface directly with the internal server and retrieve the cached credentials as shown in Figure 4, APPENDIX A. Users who currently have active sessions to the appliance could have their base64 encoded active directory cleartext passwords, in addition to the New Technology LAN Manager (NTLM) password hashes, retrieved with the same access, as shown in Figure 10, APPENDIX A. In addition to users with active sessions, users previously authenticated can have base64 encoded active directory plaintext passwords and NTLM hashes harvested from the backups of the data.mdb database files stored on the appliance, as shown in Figure 15 and 16, APPENDIX A.

The root-level access allows adversaries to maintain persistence despite issuing factory resets and appliance upgrades while deceiving the provided integrity checkers, creating the illusion of a clean installation. Due to the persistence mechanism being stored on the encrypted partition of the drive and inaccurate integrity check results, it is untenable for network administrators to validate their application has not been compromised without also decrypting the partition and validating against a clean installation of the appliance, which are actions not easily accomplished at present. Without major alterations of the integrity checking process, it is conceivable that new vulnerabilities that afford root-level access to the appliance could also result in root-kit level persistence to the appliance.

Below is proof of concept being released by CISA, which demonstrates the capacity of and opportunity for a threat actor to exfiltrate Domain Administrator credentials that were used during appliance configuration:

Below is a demonstration of the capacity for post exploitation exfiltration of base64 encoded cleartext credentials for active directory users and their associated NTLM password hashes:

APPENDIX B: INDICATORS OF COMPROMISE

APPENDIX C: MITRE ATT&CK TACTICS AND TECHNIQUES

APPENDIX D: DETECTION METHODS