How network traffic analysis works in NDR

Network traffic analysis describes how an NDR platform collects, processes, and examines traffic across a hybrid enterprise network to detect attacker behavior in real time. Effective network traffic analysis covers how traffic is captured from on-premises, cloud, and remote environments; how it is processed into security-relevant metadata; how encrypted sessions are examined without requiring decryption; how identity is derived from network protocols; and how analysis components are deployed and scaled. This page explains each layer for security teams evaluating or deploying NDR.

How network traffic analysis works in NDR architecture

Network traffic analysis in NDR is built on three interdependent layers: traffic capture, behavioral analysis, and response integration. Sensors deployed across network segments capture raw traffic and forward metadata to a processing component — commonly called a Brain appliance, which analyzes that metadata using behavioral AI models. Detection results are surfaced in a management interface and fed to downstream systems such as SIEM and SOAR.

Unlike endpoint-centric tools, NDR operates at the network level, meaning it sees all devices communicating across the environment, managed, unmanaged, cloud-hosted, and IoT/OT, regardless of whether a security agent is installed. This gives security teams visibility into lateral movement, east-west traffic, and identity-based activity that endpoint tools cannot observe.

The architecture is designed to be agentless at the point of capture. Sensors receive a copy of network traffic through SPAN ports, TAPs, or cloud-native mirroring, and do not require inline placement or traffic interception. This minimizes deployment complexity and avoids introducing latency into production traffic flows.

East-west vs. north-south traffic monitoring

Network traffic monitoring covers two primary traffic directions, each revealing a distinct class of attacker behavior. North-south traffic moves between internal systems and external destinations, this is where command-and-control communications, data exfiltration, and botnet activity become visible. East-west traffic moves laterally between internal systems, this is where reconnaissance, privilege escalation, and lateral movement unfold after an attacker has already established a foothold.

Most perimeter security tools focus on north-south traffic at the network edge. This leaves east-west visibility largely uncovered. When an attacker uses valid credentials to move between servers, workloads, or identity systems, that activity flows east-west, and it will not trigger alerts in tools that only inspect ingress and egress traffic.

NDR addresses this gap by capturing traffic at internal network segments, within data centers, across cloud environments, and between campus locations, not just at the perimeter. This allows behavioral models to observe lateral movement as it occurs, before an attack reaches its target. For a deeper look at how attackers exploit coverage gaps beyond the endpoint, the attack paths that emerge in those blind spots are where network-level analysis provides the most differentiated value.

The table below maps each traffic direction to the attack behaviors it surfaces. Security teams should ensure Sensor placement covers both flows across all network segments where sensitive systems operate.

What traffic should network traffic analysis capture (and exclude)?

Effective network traffic analysis captures the protocols that reveal behavioral intent, authentication events, service-to-service communications, and session-level activity, while excluding high-volume traffic that adds noise without security value. Protocol selection directly affects detection accuracy and appliance performance.

The tables below reflect operational guidance derived directly from Vectra NDR deployment specifications. Read them together: what to capture determines detection coverage, what to exclude determines model fidelity and appliance efficiency, and what improves HostID determines how reliably the platform attributes activity to specific named hosts and identities.

Protocols important to capture vs. traffic to exclude

The left column lists protocols that carry behavioral signal — authentication, service calls, session data. The right column lists traffic categories that consume capacity without improving detection. Capturing excluded traffic types increases appliance load and introduces noise that degrades AI model precision.

What helps improve Vectra's HostID

Host identification accuracy depends on the platform observing the protocols that carry device and identity naming signals. The following inputs, both protocol-based and integration-based, improve how reliably the platform attributes network activity to specific named hosts and identities, even as IP addresses change.

Supported encapsulations

Vectra NDR Sensors support the following network encapsulation formats, ensuring compatibility across modern data center, campus, and cloud network architectures.

Typical traffic capture sources

Network traffic reaches Vectra Sensors through the following capture mechanisms. The appropriate source depends on network architecture, environment type, and whether the deployment spans on-premises, cloud, or hybrid infrastructure.

‍

Placement guidance: Sensors should be deployed on the south side of any proxy or NAT device to accurately identify the originating host. When traffic from multiple hosts is aggregated behind a proxy to a single IP, the NDR platform should be configured to recognize that proxy to preserve detection accuracy. Small remote branch locations do not typically require a dedicated Sensor — traffic from those sites is generally visible at central locations where Sensors are already deployed. For physical and virtual appliance sizing guidance by throughput tier, see the appliance and Sensor specifications.

How encrypted traffic analysis works in NDR

The vast majority of NDR detections do not require traffic decryption. Behavioral AI models analyze metadata derived from encrypted sessions, packet timing, session size, connection patterns, certificate attributes, and protocol behavior, to identify malicious activity without inspecting payload content. This makes network traffic analysis effective against how attackers hide command-and-control in encrypted traffic, one of the most consistently underdetected attack behaviors in enterprise environments.

This is a critical architectural distinction. Inline decryption introduces latency, privacy concerns, regulatory complexity, and significant infrastructure cost. NDR avoids these trade-offs by operating on metadata rather than plaintext payloads — a position supported by why modern architectures no longer require inline decryption to achieve meaningful behavioral detection coverage.

There are limited scenarios where access to decrypted traffic metadata provides additional value:

• Teams using advanced investigation or recall capabilities that require decrypted session content for forensic analysis
• Deployments using pattern-matching rules where certain signatures only activate when payload content is inspectable

When decryption is needed: the parallel pipeline approach

When both encrypted and decrypted traffic analysis is required, this must be handled through parallel processing pipelines — not by sending both traffic types to the same Sensor. Sending both streams to a single Sensor causes deduplication logic to discard one stream (typically the decrypted version, because decryption adds processing overhead that causes it to arrive later). The correct approach is to deploy a dedicated Sensor paired to a separate Brain appliance for the decrypted traffic stream. Virtual Sensors and Brain appliances are supported at no additional licensing cost for this purpose.

How network identity detection works from traffic metadata

Network identity detection derives identity context directly from network protocols rather than relying on agent-reported telemetry or log ingestion alone. By analyzing authentication traffic flowing across the network, Kerberos tickets, LDAP queries, NTLM exchanges, DCE/RPC calls, the platform builds a continuous picture of which identities are active, what resources they are accessing, and how their behavior compares to established baselines. This makes the approach particularly effective at detecting privilege-based identity attacks, where compromised accounts operate within expected access boundaries but behave anomalously across authentication patterns.

Additionally, service accounts authenticating between systems, workloads making API calls, and AI agents traversing infrastructure all generate identity-visible network traffic, even when no endpoint agent is present on those systems.

The table below maps each identity-relevant protocol to the attacker behavior it surfaces and why it matters operationally.

‍

Network identity detection also feeds host identification accuracy. DNS, reverse DNS, multicast DNS, Kerberos, DHCP, and NetBIOS are the primary signals used to associate network activity with specific named hosts and identities — enabling consistent attribution even as IP addresses change.

Brain and Sensor architecture in NDR deployments

The core NDR architecture consists of two primary appliance types: The Brain and the Sensor. Each performs a distinct function, and their interaction is what transforms raw network traffic analysis into prioritized security detections.

Sensor appliances capture raw network traffic at designated capture points, data centers, campus environments, cloud environments, and remote locations. Sensors deduplicate traffic before forwarding metadata to the Brain. They maintain a rolling capture buffer that allows packet-level retrieval (PCAP) when needed for investigation. The vSensor is the virtual variant, deployable on hypervisors or IaaS cloud environments.

Brain appliances receive metadata from one or more paired Sensors and process it locally using behavioral AI models to generate detections. Understanding how metadata-driven detection improves threat visibility is central to evaluating why this architecture outperforms log-centric approaches for east-west traffic monitoring and identity-based attack scenarios. In cloud-delivered (Respond UX) deployments, the Brain sends detection data and metadata to the Vectra cloud for presentation in the management interface. In on-premises (Quadrant UX) deployments, the Brain hosts the management UI locally.

Mixed-mode appliances combine Brain and Sensor functions in a single unit, suited for smaller deployments or environments where a two-appliance architecture is not justified.

‍

Communication and pairing:

• Sensors communicate with their paired Brain over TCP/22 (SSH) and TCP/443 (HTTPS)
• Sensors generally communicate only with the Brain and DNS — not directly with external systems
• Brain appliances initiate all outbound connections to the Vectra cloud
• Updates are delivered from the Vectra cloud to the Brain and propagated automatically to paired Sensors
• Air-gapped deployments are supported (Quadrant UX only), with detections generated locally on the Brain
• Physical Sensors can retrieve the Brain IP from the Vectra cloud at initial boot via TCP/443, enabling automatic pairing with DHCP

Deployment recommendation: Brain and Sensor appliances should be placed in network locations not directly visible from the public internet. Private connectivity or a VPN tunnel terminated outside the Vectra appliances is the preferred approach. NDR detections focus on in-to-in and in-to-out communications — capturing traffic outside edge firewalls in the DMZ is not recommended.

‍

Deployment, scaling, and Global View considerations

NDR deployments scale from single-site environments to large, distributed global enterprises. Understanding appliance capacity and multi-instance architecture is essential for teams planning deployments at enterprise scale.

Appliance sizing

The table below reflects current appliance performance tiers. Actual performance varies based on traffic composition — work with your account team to determine the right physical and virtual appliance mix for your environment.

‍

Note: Vectra periodically releases new appliance configurations to support updated throughput requirements, hypervisors, and cloud providers. Always refer to the appliance and Sensor specifications for current guidance before sizing a deployment.

Global View for distributed enterprises

Organizations operating multiple sites or business units can deploy Global View for distributed SOC environments to maintain a unified detection picture across separate NDR instances. Global View aggregates prioritized entity data from child Respond UX instances into an anchor instance, providing a single management view without centralizing stored data.

‍

Network traffic analysis in practice: Deployment outcomes, 2023–2025

Network traffic analysis produces measurable outcomes when deployed correctly, but those outcomes depend directly on architectural decisions: where Sensors are placed, what traffic is captured, how identity is derived from protocols, and how detections are prioritized across the environment. The following cases illustrate what the architecture produces in production deployments across different network environments.

Goodwood Estate — east-west visibility from 20% to 95% (2024) Before deploying NDR, Goodwood Estate had east-west traffic monitoring covering approximately 20% of its distributed infrastructure. Blind spots across campus, IoT, and operational technology environments left significant lateral movement paths unmonitored. After deploying Sensors at internal network segments, not just the perimeter, east-west visibility reached 95%, closing the coverage gaps that traditional perimeter tools cannot address by design.

Schaefer Kalk — stopping a ransomware attack that bypassed EDR (2023) Schaefer Kalk's endpoint controls failed to catch an active ransomware campaign moving laterally across the network. Network traffic analysis, examining east-west flows that EDR cannot observe, detected the lateral movement and contained the attack before it reached production systems.

Globe Telecom — 99% alert noise reduction and 75% faster response (2024) Globe Telecom reduced incident response time from 16 hours to 3.5 hours. Alert noise dropped by 99% and escalations fell by 96%, allowing analysts to focus on six real incidents rather than hundreds of thousands of low-value alerts. The underlying factor: risk-based prioritization built on behavioral AI operating across network, identity, and cloud metadata, not signature-matching against individual alerts.

A global healthcare organization — detecting credential abuse that SIEM missed (2024) Within days of deployment, network traffic analysis detected stolen credentials, cloud reconnaissance, privilege escalation attempts, and AWS persistence activity that the organization's SIEM had failed to surface. Identity detection from network protocols — Kerberos, LDAP, DCE/RPC — provided the signal that log-based systems could not reconstruct. The SOC intervened before data or operations were impacted.

Key takeaway: outcomes depend on Sensor placement covering east-west segments, protocol capture including authentication traffic, and behavioral models that analyze identity movement across the network rather than isolated alerts at individual endpoints.

How Vectra AI implements network traffic analysis

Vectra AI's network traffic analysis is built on four engineering layers that work together to convert raw network and identity telemetry into prioritized, actionable detections. These are not general NDR capabilities — they describe the specific architectural decisions that determine how the platform performs in production across hybrid, multi-cloud, and identity-driven environments.

Layer 1: Attacker behavior modeling

Every detection is grounded in security research, not statistical anomaly. Models are mapped directly to MITRE ATT&CK tactics and techniques across network, identity, cloud, and SaaS domains.

Detection engineers and data scientists define three things for each attacker technique:

• What must be detected
• What telemetry is required to detect it
• Which AI/ML model is appropriate for that specific problem

Models are selected based on the problem — credential abuse, lateral movement, command-and-control, persistence — not applied generically. This ensures detections are explainable, repeatable, and defensible, with clear linkage between attacker technique and defensive outcome. Vectra AI has contributed to and influenced how network- and identity-based ATT&CK techniques are understood and mapped by the broader security community.

Layer 2: Jetstream — real-time streaming engine

Jetstream processes network and identity telemetry in motion — not after the fact.

Unlike batch-based, log-centric systems that store traffic and analyze it later, Jetstream ingests, enriches, and correlates telemetry continuously as events occur across the hybrid enterprise. This is what enables detection of behavioral patterns while an attack is unfolding rather than minutes or hours later.

For east-west traffic monitoring in particular, streaming matters. Lateral movement that unfolds across minutes cannot be reliably detected by systems built around batch processing cycles.

Layer 3: Metadata signal fabric

Rather than relying on full packet capture or raw log ingestion, Vectra AI extracts, normalizes, and enriches security-relevant metadata from across the hybrid network.

Sources include:

• Network flows
• Identity events
• Cloud activity
• SaaS interactions
• Infrastructure telemetry

Each metadata record is continuously enriched with context — identity, asset role, behavior history, attack stage, and risk posture — creating a shared intelligence layer across detection, investigation, and response workflows. Analysts gain deep visibility without the storage and performance overhead of managing full packet capture at scale.

Layer 4: Multi-layer attribution

In environments where attackers abuse identities, impersonate services, and move laterally, attribution must go beyond IP addresses and single-event correlation.

Vectra AI's multi-layer attribution continuously links activity across users, service accounts, workloads, hosts, and infrastructure by combining network behavior, identity context, and privilege intelligence. Three capabilities work together to make this possible:

‍

Together, these layers enable the platform to answer, with confidence, who is doing what, where, and with what level of privilege, even when attackers use valid credentials and blend into normal traffic patterns.

Conclusión

Network detection and response is not a single product decision, it is a set of interconnected engineering choices that determine whether your security team can see lateral movement as it happens, derive identity context from network protocols, analyze encrypted traffic without decryption, and scale detection across distributed environments without centralizing stored data.

The fundamentals covered in this page translate directly into operational outcomes: East-west Sensor placement determines whether lateral movement is observed or inferred; protocol capture decisions determine whether identity-based attacks are visible or invisible; parallel pipeline design determines whether encrypted traffic analysis degrades detection accuracy; and Global View architecture determines whether distributed deployments produce unified visibility or fragmented coverage.

Explore how Vectra AI's platform delivers network traffic analysis that catches lateral movement, identity abuse, and encrypted command-and-control, seeing what perimeter tools and endpoint agents miss.