Azure NSG Flow Logs: Setup Guide

Azure NSG Flow Logs are being retired. Starting 30 June 2025, you won’t be able to create new logs, and the feature will be fully retired by 30 September 2027. Transition to Virtual Network Flow Logs to maintain network monitoring capabilities.

Quick Facts:

• Purpose: NSG Flow Logs track inbound/outbound Layer 4 IP traffic for Azure Network Security Groups (NSGs).
• Data: Logs include source/destination IPs, ports, protocols, and traffic decisions.
• Storage: Logs are saved in Azure Storage (JSON format) and can integrate with tools like SIEMs or visualisation platforms.
• Costs: 5GB free per month; additional charges for storage and Traffic Analytics.

Setup Summary:

1. Prerequisites: Azure subscription, permissions (Owner/Contributor), and a Network Security Group.
2. Enable Network Watcher: Activate in the same Azure region as your NSG.
3. Configure Flow Logs:
   • Create a Storage Account (General-purpose v2).
   • Enable flow logs in Network Watcher.
   • Set retention periods (up to 1 year).

Key Benefits:

• Security: Detect unusual traffic and verify NSG rules.
• Compliance: Meet UK data protection laws.
• Cost Control: Use selective logging and retention policies to manage storage expenses.

⚠️ Action Required: Migrate to Virtual Network Flow Logs before the retirement deadline to avoid disruption.

For step-by-step details, cost tips, and advanced settings, read on.

How to Properly configure Network Security Group Flow Logs

Prerequisites and Planning

Getting everything in order before you start is crucial to avoid any hiccups during setup.

Azure Subscription and Permissions

To ensure everything runs smoothly, assign the Owner or Contributor role to your Azure subscription. These roles provide the necessary permissions to manage NSG flow logs, configure storage accounts, and handle related tasks. If you're using the Network Contributor role, don’t forget to manually add permissions for storage account actions like reading, writing, and configuring SAS tokens.

Another important step is verifying that the Microsoft.Insights provider is registered in your subscription. This provider is critical for NSG flow logs to function. If its status is "Unregistered", make sure to register it before proceeding.

Once permissions are sorted, double-check that all the required resources are ready to go.

Required Resources

First, ensure you have a Network Security Group (NSG). If one doesn’t exist, you’ll need to create it. Next, set up a General-purpose v2 or Blob storage account with hierarchical namespace disabled. This is essential for storing the flow logs.

Additionally, make sure Network Watcher is enabled in your Azure region. This tool provides the infrastructure needed to collect flow log data. To minimise data transfer costs, it’s a good idea to place your NSG and storage account in the same Azure region.

If you’re planning to process flow logs programmatically, you’ll also need to create a Storage Queue within the same storage account. Set up an event subscription for "Blob created" events, filtering specifically for the insights-logs-networksecuritygroupflowevent container.

Once these resources are in place, take some time to plan how you’ll manage costs and data retention.

Cost and Retention Planning

NSG flow logs come with costs based on the volume of data. While there’s a 5GB free tier each month, additional storage and retention will increase expenses. If you enable Traffic Analytics for deeper insights, note that it’s charged per gigabyte of data processed and doesn’t include a free tier.

To manage costs effectively, adopt a retention strategy that aligns with your needs. For instance:

Instead of enabling flow logs for every NSG, focus on critical resources or those subject to specific regulations. This selective approach can help you maintain visibility where it matters most while keeping costs in check.

To further optimise storage costs, use lifecycle management rules for Blob Storage. These rules can automatically move logs between Hot, Cool, and Archive tiers based on their age. You can also automate log deletion or movement using tools like Azure Logic Apps or Azure Functions, ensuring consistent cost management without manual effort.

It’s wise to keep an eye on your spending with Azure's Cost Management and Billing tools. These tools can help you spot areas for optimisation and avoid unexpected charges. Setting up budget alerts is another smart move - they’ll notify you as costs approach your defined limits.

Lastly, keep in mind that NSG flow logs are automatically deleted after one year when stored in General-purpose v2 storage accounts. For more tips on managing costs and performance in Azure, check out Azure Optimization Tips, Costs & Best Practices.

Step-by-Step Configuration Guide

This guide walks you through three key steps: enabling Network Watcher, setting up NSG flow logs, and optionally enhancing monitoring with advanced settings.

Important note: NSG flow logs will be retired on 30 September 2027. Starting 30 June 2025, you won't be able to create new logs, so it's best to migrate to virtual network flow logs as soon as possible.

Enable Network Watcher

The first step is to activate Network Watcher in the region where your NSG is deployed. This service is essential for NSG flow logs to function. Here's how you can enable it:

1. Go to the Azure portal and select All services.
2. Search for "Network Watcher" and select it.
3. Under Regions, you'll see a list of Azure regions and their Network Watcher statuses.
4. Find the region where your NSG is located, click the three dots (...) next to it, and choose Enable Network Watcher.

The process usually takes just a few moments. Once completed, the status for that region will update to "Enabled". Without this step, NSG flow logs won't work.

Configure NSG Flow Logs

Once Network Watcher is enabled, you can proceed to configure NSG flow logs:

1. Set up a storage account:
   • In the Azure portal, go to Storage Accounts and click Add.
   • Select your subscription and resource group, then provide a unique name for the storage account.
   • Ensure the storage account is in the same region as your NSG to avoid additional data transfer costs.
   • Choose Standard for performance and StorageV2 (general-purpose v2) for account kind.
   • Click Review + Create, then hit Create.
2. Enable flow logs:
   • In the Azure portal, search for and select Network Watcher.
   • Under Logs, click Flow logs to see a list of available NSGs. Choose the NSG you want to monitor.
   • In the Flow logs settings, toggle the setting to On.
   • Select Version 2 for the flow logs, as it includes additional details like flow state and throughput.
   • Pick the storage account you just created and set your preferred retention period.
   • Click Save.

The changes might take a few minutes to apply. Once active, the flow logs will start collecting data, which will be stored in JSON format in your designated storage account.

Advanced Settings (Optional)

After setting up basic flow logging, you can enhance monitoring with advanced features. One popular option is Traffic Analytics, which processes flow logs to provide visual insights via Network Watcher. Keep in mind that Traffic Analytics charges per gigabyte of data processed and does not offer a free tier.

For more custom analysis, you can explore other tools:

• Power BI: Offers strong business intelligence and graphical reporting.
• Azure Data Explorer: Provides Kusto-based analysis for deeper insights.
• Open-source tools: These can be useful but may require additional plugins.
• Python scripts: Ideal for extracting the latest flow logs quickly, especially helpful during incident response scenarios.

"Flow logs are the source of truth for all network activity in your cloud environment. Whether you're in a startup that's trying to optimise resources or a large enterprise that's trying to detect intrusion, flow logs can help." - Microsoft Learn

Accessing and Analysing NSG Flow Logs

Once your NSG flow logs are collected, the next step is to locate and interpret them. These logs, set up during earlier configurations, are now available for analysis, offering valuable insights into your network's security. They are stored in your specified Azure Storage account and follow an organised structure, making them straightforward to access and review.

Locating Flow Logs in Azure Storage

NSG flow logs are saved as block blobs in your Azure Storage account. Each log file is generated hourly and updated every few minutes. To access them via the Azure portal, go to your storage account, then navigate to the Containers section under Data storage. Look for the container named insights-logs-networksecuritygroupflowevent - this is where all your NSG flow logs are stored.

The logs are arranged in a hierarchy based on date, time, and network interface details. If navigating through the Azure portal feels cumbersome, you can use Azure Storage Explorer. This tool provides a more user-friendly way to browse, download, and manage your flow log files without dealing with the portal's folder structure.

Once located, the next step is to interpret the logs and derive actionable insights.

Understanding Log Structure and Data

NSG flow logs capture Layer 4 IP traffic, recording all inbound and outbound IP flows associated with a network security group. These logs are formatted as JSON records and collected at 1-minute intervals. Each log entry includes metadata and key traffic details encapsulated in the flowTuples. These tuples are comma-separated strings containing information such as IP addresses, ports, protocols, traffic direction, and the decision (allow or deny).

Version 2 logs add even more detail, including markers for flow states (B for Begin, C for Continuing, and E for End), as well as packet and byte counts. This additional information can help you better understand traffic behaviour over time.

Tools for Log Analysis

To streamline the analysis process, you can use several tools designed for handling NSG flow logs:

• Azure Monitor: This tool enables you to set up alerts and automated responses to unusual traffic patterns. It also includes Azure Monitor Workbooks, which provide built-in visualisation features to help you identify trends and anomalies.
• Power BI: For creating detailed dashboards and reports, Power BI integrates seamlessly with Azure Storage, offering a business intelligence perspective on your flow log data.
• Enterprise Tools: Solutions like Splunk, Grafana, and Stealthwatch are ideal for processing and visualising NSG flow logs at scale.
• Open-Source Options: Tools such as Graylog and the Elastic Stack (Elasticsearch, Logstash, Kibana) offer cost-effective alternatives for log management and analysis .

Centralising your logging setup - whether in Azure Storage, Azure Monitor, or a third-party SIEM solution - makes it easier to correlate events across different parts of your network. This approach can help you identify patterns that point to potential security threats or performance issues.

Monitoring and Optimisation Best Practices

With NSG flow logs now active, it's time to focus on monitoring and managing costs effectively. Proper monitoring not only enhances security but also helps UK SMBs keep expenses under control. Below, we’ll explore retention policies, alert automation, and ways to improve performance.

Retention and Storage Management

The cost of logs depends heavily on the volume of data stored, making retention policies a key part of cost management. These policies strike a balance between staying compliant and avoiding unnecessary expenses. For general-purpose v2 storage accounts, you can set automatic retention policies, such as deleting data after up to one year, to help manage storage costs. To further optimise, enable NSG flow logs only for critical subnets - like those hosting sensitive workloads or internet-facing resources. This approach reduces log volumes while maintaining visibility into important network activities.

Setting Up Alerts and Automation

Once your flow logs are configured, Azure Monitor can help you detect and respond to potential security threats. By setting up alerts for scenarios like unusual outbound traffic, connections to known malicious IPs, or unexpected protocol use, you can turn your flow log data into a proactive monitoring system. For example, anomaly detection can flag traffic from malicious IP addresses by cross-referencing threat intelligence databases. When paired with monitoring tools, NSG flow logs can automatically trigger security alerts based on network activity.

Centralising logs in Azure Storage, Azure Monitor, or a SIEM solution allows for better event correlation across your network. A tiered alerting system can also be useful - high-severity threats could trigger immediate automated responses, while medium-severity events might generate notifications for manual investigation. This method helps avoid alert fatigue while ensuring critical threats are addressed promptly.

Performance and Cost Optimisation Tips

To optimise your setup, consider strategies that reduce NSG flow log costs while improving monitoring efficiency. For instance, while Traffic Analytics provides valuable insights, it charges per gigabyte processed and does not include a free tier. Evaluate whether its benefits align with your specific needs before enabling it. Also, ensure your storage capacity matches expected log volumes to prevent throttling or resource wastage.

Blocking unnecessary inbound traffic is another way to cut down on log entries. Use precise NSG rules to block known malicious actors and unused protocols, reducing irrelevant data before it’s logged. Additionally, migrating to Virtual Network Flow Logs can provide broader coverage, including traffic from Azure Application Gateway, ExpressRoute Gateway, and VPN Gateway. This solution also consolidates multiple logging requirements into a single system, offering a more streamlined approach.

For more tailored advice on balancing security monitoring with cost management, visit Azure Optimization Tips, Costs & Best Practices.

Regularly reviewing your monitoring setup - examining log volumes, alert configurations, and storage costs - can uncover further opportunities to refine and improve your Azure environment over time.

Conclusion

Configuring Azure NSG flow logs transforms your approach to network monitoring. Instead of reacting to issues as they arise, you gain the tools to anticipate and address potential challenges, ensuring a more secure and efficient cloud environment. By following the steps in this guide - from activating Network Watcher to setting up retention policies - you've built a solid framework for maintaining network security and meeting compliance demands.

The advantages go far beyond basic monitoring. With this setup, you can pinpoint high-traffic sources, forecast capacity needs, and eliminate overly restrictive traffic rules that might slow down performance. Additionally, exporting logs to SIEM or IDS tools enables detailed forensic analysis during security events, offering critical insights when it matters most.

From a budgeting perspective, Azure’s 5 GB free tier and pay-as-you-go pricing model make this solution accessible even for smaller deployments. Automatic retention policies, allowing for log deletion up to a year after creation, provide a practical way to balance compliance needs with storage costs. These cost management features complement the advanced monitoring capabilities.

Your current setup ensures network isolation for compliance, monitors bandwidth for capacity planning, and flags unusual traffic for improved security. Whether you're troubleshooting, performing forensic investigations, or optimising operations, NSG flow logs provide the detailed Layer 4 traffic data needed for smarter decision-making.

By investing in this configuration, you’ve enhanced your organisation’s security, compliance, and network performance. This proactive approach ensures your Azure network can adapt and thrive alongside your business's evolving demands.

For more tips on improving performance and managing costs, visit Azure Optimization Tips, Costs & Best Practices.

FAQs

What are the main differences between Azure NSG Flow Logs and Virtual Network Flow Logs, and why is it important to switch to the latter?

The key distinction between Azure NSG Flow Logs and Virtual Network (VNet) Flow Logs lies in the range and detail of traffic monitoring they offer. VNet Flow Logs deliver a wider view, capturing all traffic flowing into and out of your subnets, regardless of the Network Security Group (NSG) rules. They also include extra features like tracking the encryption status of traffic, making them a stronger choice for in-depth traffic analysis.

Microsoft has announced that NSG Flow Logs will be retired by September 2027, recommending users transition to VNet Flow Logs. By making the switch, you'll ensure your network monitoring remains compatible with future updates, while gaining enhanced insights that support better network management. Adopting VNet Flow Logs now positions you to stay ahead of these changes and refine your monitoring approach.

How can I manage Azure NSG Flow Log costs efficiently while ensuring security and compliance?

To keep Azure NSG Flow Log costs under control while ensuring security and compliance, you can take a few strategic steps. One effective approach is leveraging Azure Policy. This tool allows you to automate and standardise logging configurations, cutting down on manual work, keeping costs manageable, and ensuring your setup adheres to security standards.

Since NSG flow logs will be retired in September 2027, transitioning to virtual network flow logs is a smart move. These logs offer improved monitoring features, which can strengthen both your compliance and security posture.

For cost management, consider using Azure Blob Storage paired with lifecycle management policies. These policies can automatically archive or delete older logs, helping to lower storage expenses. Additionally, reviewing your log retention policies regularly and removing unnecessary data can further help you manage costs effectively.

How can I analyse Azure NSG Flow Logs to improve network security?

To dive into Azure NSG Flow Logs and improve your network security, several tools can make the process more efficient. Platforms like Network Watcher Traffic Analytics, Elastic Stack, Graylog, and Power BI allow you to visualise and manage logs with ease, offering a clearer view of your network's activity.

You can export flow logs to these tools for in-depth analysis or use PowerShell for targeted log examination. For even stronger security measures, integrating with SIEM solutions provides real-time alerts and advanced analytics, helping you spot vulnerabilities and fine-tune your network's security setup.

Related posts

• Azure Performance Baseline: Step-by-Step Guide
• Configuring Failover Routing in Azure: Step-by-Step
• Ultimate Guide to Azure Data Security Auditing
• Checklist for Monitoring Scalable Azure Systems