In modern IT and security environments, collecting logs is only the beginning. The real value lies in how you process, filter, enrich and route telemetry before it reaches your SIEM or storage layer.
This is where custom telemetry pipelines in NXLog truly shine.
In this blog, we’ll walk step-by-step through building a structured telemetry pipeline that transforms noisy raw events into meaningful, actionable data.
📌 What is a Telemetry Pipeline?
A telemetry pipeline is the structured flow of:
Input → Processing → Enrichment → Filtering → Output → Storage / SIEM
Instead of forwarding everything blindly, a smart pipeline:
- Reduces SIEM ingestion cost
- Improves detection quality
- Eliminates noise
- Adds contextual intelligence
- Enhances compliance visibility
🏗️ Step 1: Define Your Data Sources (Inputs)
- Before setting up NXLog agents, ensure:
- Administrative privileges on target systems
- Correct binaries/packages for your OS
- Network connectivity to NXLog Manager/Server (if applicable)
- Firewall rules allowing communication between agent and server
Start by identifying what you want to collect:
- Windows Event Logs
- Linux Syslog
- Application Logs
- Firewall Logs
- Cloud Telemetry
Example: Windows Event Log Input
Example: Linux Syslog Input
✔️ Best Practice: Collect only relevant log categories instead of everything.
🔍 Step 2: Parse and Normalize Logs
Raw logs are messy. Normalize them into structured fields.
Why this matters:
- Makes searching easier
- Improves dashboard visualization
- Helps SIEM correlation rules
🎯 Step 3: Filter Noise
Not every log deserves to go to your SIEM.
Impact:
- Reduce ingestion cost
- Improve signal-to-noise ratio
- Speed up investigations
➕ Step 4: Enrich Telemetry
Enrichment adds context.
Examples:
- Add hostname
- Add environment tag (Prod / Dev)
- Add geo-location
- Map event IDs to readable categories
Why enrichment is powerful:
- Faster incident response
- Better compliance tracking
- Clearer SOC visibility
🔐 Step 5: Securely Route to Destination
Once telemetry is processed and enriched, the next step is delivering it securely to the right destination. NXLog provides both:
- Generic secure outputs (TCP/SSL/Syslog)
- Specialized integration modules for SIEMs, observability and monitoring platforms
This allows organizations to send the same telemetry stream to multiple platforms — each formatted exactly the way the destination expects.
📡 Common Telemetry Destinations
NXLog can forward telemetry to:
- SIEM platforms (Google Chronicle, Splunk, Sentinel)
- Monitoring tools (Prometheus)
- NXLog Platform storage
- Elasticsearch
- Kafka
- Cloud endpoints
- Log archive servers
🛠️ Generic Secure SSL Output
For standard secure forwarding, NXLog supports encrypted SSL/TLS transport.
This method is commonly used for:
- Generic SIEM forwarding
- Syslog collectors
- Secure log relays
🚀 Using Specialized Output Modules (Recommended)
NXLog also provides destination-specific modules that automatically handle formatting, field mapping and protocol requirements.
This significantly reduces manual parsing and integration complexity.
📊 Example: Sending Metrics to Prometheus
Here, NXLog automatically formats telemetry in a structure Prometheus understands.
🔎 Why Specialized Modules Matter
Using dedicated output modules provides:
✅ Better compatibility
✅ Faster integrations
✅ Cleaner telemetry formatting
✅ Reduced engineering effort
✅ Improved reliability
✅ Lower operational complexity
This is especially valuable in enterprise environments where telemetry must flow simultaneously into:
- SIEM
- Monitoring
- Compliance
- Observability platforms
🔄 Step 6: Define the Route
Finally, connect everything.
This creates a complete custom pipeline.
🔐 Best Practices for Secure Routing
Always ensure:
- TLS encryption enabled
- Certificate-based authentication
- Destination-specific modules whenever possible
- Load balancing for critical environments
- Monitoring output performance and failures
📊 Real-World Use Case
Imagine a SOC handling 500 endpoints:
Without pipeline:
- 100% logs forwarded
- High SIEM bill
- Alert fatigue
With custom pipeline:
- 40–60% noise removed
- Critical events enriched
- Faster triage
- Lower operational cost
That’s the power of intelligent telemetry design.
📊 NXLog Custom Telemetry Pipeline Architecture 📊
Diagram Flow (Up to Down)
🧠 Design Principles for Strong Pipelines
✔️ Collect with purpose
✔️ Normalize early
✔️ Filter aggressively
✔️ Enrich strategically
✔️ Encrypt everything
✔️ Monitor pipeline performance
🏁 Final Thoughts
Telemetry pipelines are not just technical configurations, they are strategic visibility frameworks.
By building custom pipelines in NXLog, you move from simple log forwarding to:
✨ Intelligent data flow
⚡ Faster detection
💰 Optimized SIEM costs
🔍 Clearer operational insights
This is where logging evolves into observability.
👉 Stay tuned for the next blog, where we’ll explore Advanced Filtering, Correlation and Performance Tuning in NXLog Platform.
Custom Telemetry Pipelines nxlog xml file URL:- Reference config file