Apply the Latest Security Updates

To address this vulnerability, Microsoft recommends customers apply the latest security updates to remediate this vulnerability. Please review the Apache CVEs and the Apache security advisory for further details: 

• Apache Log4j 2.x CVEs: CVE-2021-44228 and CVE-2021-45046 
• Apache security advisory: Apache Log4j Security Vulnerabilities 

All systems, including those that are not internet facing, are potentially vulnerable to these vulnerabilities, so backend systems and microservices should also be upgraded.  No Java version can mitigate these vulnerabilities. The recommended action is to update Apache Log4j 2. An application restart will be required.  

• Java 8 or newer: update Log4j to 2.16.0
• Java 7: update to Log4j 2.12.2 when it becomes available

Systems that have already been updated to 2.15.0, should plan to move to 2.16.0 as soon as possible for extra protection against a potential DoS attack under specific conditions as described in CVE-2021-45046.  

Systems running on Log4j 1.x are not impacted by these vulnerabilities. In 2015, Apache announced Log4j 1.x has reached end-of-life. Microsoft recommends customers to upgrade to Log4j 2.16.0 for the latest security updates.  

• Apache Announcement: Log4j 1.x End of Life 
• Apache Log4j 1.x vulnerability – 1.2 up to 1.2.17: CVE-2019-17571 

Workarounds

To help mitigate the risk of these vulnerabilities in Log4j 2.x until the more complete security update can be applied, customers should consider the following mitigations steps. These workarounds should not be considered a complete solution to resolve these vulnerabilities: 

• In case the Log4j 2 vulnerable component cannot be updated, Log4j versions 2.10 to 2.14.1 support the parameter log4j2.formatMsgNoLookups to be set to ‘true’, to disable the vulnerable feature. Ensure this parameter is configured in the startup scripts of the Java Virtual Machine:  
  -Dlog4j2.formatMsgNoLookups=true. 
• Alternatively, customers using Log4j 2.10 to 2.14.1 may set the LOG4J_FORMAT_MSG_NO_LOOKUPS=”true” environment variable to force this change. 
• Kubernetes administrators may use “kubectl set env” to set the LOG4J_FORMAT_MSG_NO_LOOKUPS=”true” environment variable to apply the mitigation across Kubernetes clusters where the Java applications are running Log4j 2.10 to 2.14.1, effectively reflecting on all pods and containers automatically. 

• For all releases between 2.0-beta9 and 2.15.0, a better mitigation approach is to prevent the JndiLookup.class file from being loaded in the applications’s classpath. Customers can do this by deleting the class from affected JAR files. For example: 
  $ zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class 
• Log4j may also be present in other files as a bundle or as a shaded library. Microsoft advises customers to do an extensive search beyond log4j-core-*.jar files.
• An application restart will be required for these changes to take effect. 

Background of Log4j

The vulnerability, tracked as CVE-2021-44228 and referred to as “Log4Shell,” affects Java-based applications that use Log4j 2 versions 2.0 through 2.14.1. Log4j 2 is a Java-based logging library that is widely used in business system development, included in various open-source libraries, and directly embedded in major software applications. The scope of impact has expanded to thousands of products and devices, including Apache products such as Struts 2, Solr, Druid, Flink, Swift, Karaf, and others.

Because this vulnerability is in a Java library, the cross-platform nature of Java means the vulnerability is exploitable on many platforms, including Windows, macOS, and Linux. As many Java-based applications can leverage Log4j 2 directly or indirectly, organizations should contact application vendors or ensure their Java applications are running the latest up-to-date version. Developers using Log4j 2 should ensure that they are incorporating the latest version of Log4j into their applications as soon as possible to protect users and organizations.

Analysis of the CVE-2021-44228 vulnerability

The vulnerability is a remote code execution vulnerability that can allow an unauthenticated attacker to gain complete access to a target system. It can be triggered when a specially crafted string is parsed and processed by the vulnerable Log4j 2 component. This could happen through any user provided input.

Successful exploitation allows for arbitrary code execution in the targeted application. Attackers do not need prior access to the system to log the string and can remotely cause the logging event by using commands like curl against a target system to log the malicious string in the application log. When processing the log, the vulnerable system reads the string and executes it, which in current attacks is used to execute the code from the malicious domain. Doing so can grant the attacker full access and control of the affected application.

Given the fact that logging code and functionalities in applications and services are typically designed to process a variety of external input data coming from upper layers and from many possible vectors, the biggest risk factor of this vulnerability is predicting whether an application has a viable attack vector path that will allow the malformed exploit string to reach the vulnerable Log4j 2 code and trigger the attack.

A common pattern of exploitation risk, for example, is a web application with code designed to process usernames, referrer, or user-agent strings in logs. These strings are provided as external input (e.g., a web application built with Apache Struts). An attacker can send a malformed username or set user-agent with the crafted exploit string hoping that this external input will be processed at some point by the vulnerable Log4j 2 code and trigger code execution.

Figure 1. CVE-2021-44228 exploit vectors and attack chain 

Mitigation Guidance for Microsoft Services

After further analysis of our services and products, below are a few mitigation strategies given by various Microsoft services.

Azure App Service (Windows, Linux and Containers)

Azure App Service and Functions does not distribute Log4J in the managed runtimes such as Tomcat, Java SE, JBoss EAP, or the Functions Runtime. However, your applications may use Log4J and be susceptible to this vulnerability.

Customers are recommended to apply the latest Log4j security updates and re-deploy applications.

If you are not able to re-package your application with a newer version of Log4j and you are using Log4j versions 2.10 to 2.14, you can mitigate this behavior by creating an application setting for the environment variable LOG4J_FORMAT_MSG_NO_LOOKUPS with value true, with the Azure CLI as follows:

 
$ az webapp config appsettings set \
 --resource-group <group-name> \ 
 --name <app-name> \ 
 --settings LOG4J_FORMAT_MSG_NO_LOOKUPS=true

Note that this command will also restart your App Service. If you are using Log4J version 2.9 or lower, this configuration mitigation will not work, and you must apply the latest security updates or consider other mitigation steps described previously in this article.

Azure Application Gateway, Azure Front Door, and Azure WAF 

In our investigation so far, we have not found any evidence that these services are vulnerable however customer applications running behind these services might be vulnerable to this exploit. We highly recommend customers to follow mitigations and workarounds mentioned in this blog to protect their applications. Additional guidance for Azure WAF is located here.

Azure Functions

Customers are recommended to apply the latest Log4j security updates and re-deploy applications. If you are not able to and you are using Log4j versions 2.10 to 2.14.1, configuring the environment variable or system property will depend on your choice of hosting option: dedicated, premium or consumption.

• Dedicated and Premium Functions: Create two application settings:
  1. LOG4J_FORMAT_MSG_NO_LOOKUPS with value =true
  2. WEBSITE_USE_PLACEHOLDER with value =0
• This can be done with the following Azure CLI command:

$ az functionapp config appsettings set \
--subscription \
--name \
--resource-group \
--settings "LOG4J_FORMAT_MSG_NO_LOOKUPS=true" "WEBSITE_USE_PLACEHOLDER=0"

• Consumption Functions:
  • Linux: Create an application setting named “languageWorkers__java__arguments” with a value of “-Dlog4j2.formatMsgNoLookups=true”.
  • Windows: Create an application setting named “languageWorkers:java:arguments” with a value of “-Dlog4j2.formatMsgNoLookups=true”. 

Note that updating application settings will restart your Function apps, which could affect cold start performance. If you are using Log4j version 2.9 or lower, these mitigations will not work and you must apply the latest security updates for Log4j or follow other mitigations as described earlier in this article.

Azure HDInsights

Automatic updates are enabled by default for existing clusters and customers do not need to take any action. Customers that have turned off automatic updates must apply the updates by running the following script on every cluster node.    

Azure Spring Cloud

Azure Spring Cloud does NOT directly distribute Log4j. However, your applications deployed to Azure Spring Cloud may use Log4j and be susceptible to this vulnerability. Log4j usage may originate from:  

• Your application sources. 
• Application Performance Monitoring tools activated for the application. 

Spring Boot Applications

Spring Boot applications are only affected if they have switched the default logging framework to Log4j 2. The log4j-to-slf4j and log4j-api jar files that are included in spring-boot-starter-logging cannot be exploited on their own. Only applications using log4j-core are vulnerable. If your application is impacted and you can redeploy the application, we recommend that you upgrade your application with the latest security updates for Log4j, and redeploy to Azure Spring Cloud – see more details at Log4j 2 vulnerability and Spring Boot. 

If you are not able to re-deploy, you may mitigate impacted applications that are using Log4j 2.10 to 2.14.1 by setting the log4j2.formatMsgNoLookups system property to true OR by setting the environment variable LOG4J_FORMAT_MSG_NO_LOOKUPS to true. You can set the system property or environment variable using: 

• Azure Portal 
• Azure CLI 
• ARM Template 
• Bicep or  
• Terraform.  

For Example – set the system property log4j2.formatMsgNoLookups via the Azure Portal or CLI 

In the Azure Portal, navigate to your application in Azure Spring Cloud and change the configuration as illustrated below: 

You can set the log4j2.formatMsgNoLookups system property to true using the Azure CLI: 

$ az spring-cloud app update -s ${SERVICE_NAME} \
      -n ${APP_NAME} -d ${DEPLOYMENT_NAME} -g ${RESOURCE_GROUP} \
      --jvm-options='-Dlog4j2.formatMsgNoLookups=true' 

For Example – set the LOG4J_FORMAT_MSG_NO_LOOKUPS environment variable via the Azure Portal or CLI 

In the Azure Portal, navigate to your application in Azure Spring Cloud and change the configuration as illustrated below: 

You can set the LOG4J_FORMAT_MSG_NO_LOOKUPS environment variable to true using the Azure CLI: 

$ az spring-cloud app update -s ${SERVICE_NAME} \
      -n ${APP_NAME} -d ${DEPLOYMENT_NAME} -g ${RESOURCE_GROUP} \
      --env 'LOG4J_FORMAT_MSG_NO_LOOKUPS=true' 

Application Performance Monitoring tools activated by your application 

Applications in Azure Spring Cloud are only impacted by the Log4j vulnerability if users activated New Relic and AppDynamics Java Agents. Applications monitored by Application Insights or Dynatrace Java Agents do not carry any potential risk associated with the Log4j vulnerability.  

We already patched-in updated New Relic and AppDynamics Java Agents. If you activated New Relic or AppDynamics Agents for your applications, we recommend that you restart your applications. Azure Spring Cloud will take steps to automatically protect customers and auto-restart any application with activated New Relic or AppDynamics Java Agents by Tuesday, December 21st, 2021 to ensure the latest fixes take effect.   

Microsoft Azure AD

While the industry is determining and mitigating overall exposure, attackers are probing all endpoints for vulnerability. Applying rigorous least privilege access policies to all resources in your environment is critical. If you use Azure Active Directory for single-sign on in your environment, we recommend you do the following with a special focus on applications you deploy or manage directly (SaaS apps, including those deployed by Microsoft, must be secured by their vendors). Note that log4j2 usage may be pre-auth for some of your applications, but these steps will help prevent post-authentication exploitation. Templates and examples for these policies are built in to facilitate deployment:

• To facilitate these steps and minimize business impact, you may wish to use application tagging capabilities to identify those applications which you haven’t validated as patched and target stricter policies to them.
• Enable MFA for all access to these resources to prevent probing using accounts with compromised passwords.
• If you are using Azure AD Identity Protection, enable blocking on risky logins (we recommend blocking on medium or higher). This will prevent access via ToR exit nodes and anonymizing VPNs.
• If you use Azure AD Conditional Access, restrict access to your resources to known/trusted locations or networks.
• Require Azure AD joined or better, MDM managed devices to access these resources.
• Monitor the risky sign in reports or use the risk workbook to track anomalous logins to your applications to help focus your investigations.
• While ADFS, as a Windows service, does not use the impacted libraries, other federation providers do. If you use a non-Microsoft federation provider (for example, for SAML 2.0), watch for token anomalies which indicate compromise of those systems.

For key guidance on securing your identity deployment, see https://aka.ms/securitysteps.  

Minecraft: Java Edition

We’ve taken steps to keep our Minecraft customers safe and protected, which included rolling out a fix that blocks this issue for Minecraft Java Edition 1.18.1. Minecraft customers running their own servers are encouraged to deploy the latest Minecraft server update to protect their users. More information is available at Security Vulnerability in Minecraft: Java Edition.

Information for Security Operations and Hunters

Microsoft security teams have put together the following guidance and resources to help customers understand this vulnerability and to help detect and hunt for exploits:

• Microsoft Security blog describing the nature of current attacks Microsoft is observing. The blog also contains guidance on how to use Microsoft security products to detect and hunt for malicious activity, and apply protections: Guidance for preventing, detecting, and hunting for CVE-2021-44228 Log4j 2 exploitation
• RiskIQ (acquired by Microsoft in August 2021) published threat intelligence article to the community portal with information about the vulnerability and exploitation of it, as well as detections and mitigations: CVE-2021-44228 Apache Log4j Remote Code Execution Vulnerability
• Microsoft 365 Defender threat analytics article with detection information and potential impacts to customer environments: Threat Insights: CVE-2021-44228 Log4j active exploitation (sign in is required)

We will further update this guidance as we continue to learn from our investigation.

The MSRC Team

Revision History:
12/16/2021 – Clarified customer guidance in the summary and linked to affected software in the security update guide.
12/15/2021 – Clarified guidance on Azure service. Added guidance for Java 7. Added guidance on Azure libraries for Java.
12/14/2021 – Added HDInsights guidance, updated guidance for CVE-2021-45046, and updated workaround guidance.
12/13/2021 – Added table to index guidance, added additional guidance for Azure WAF and Azure Spring Cloud.
12/11/2021 – Initial publish.