Development Notes: Processing 43000 Snort IDS Alerts

0
642
43639 Snort Alerts
43639 Snort Alerts

Development Notes:

The visual above represents the following back-end Linux processing:

IDS Alerts File Size:   15,898,993
IDS Alerts:                 43,639            
Nodes:                       3,379            Note 1
Edges:                       3,378            
XML File Size:           1,152,487 Bytes      
Time to Create XML FDG:          5 minutes    Note 2,3
XML File Size:           1,188,220 Bytes      Note 3,4

Notes:

1. 43,639 Alerts Clustered into 3379 Nodes
2. 16 Core AMD Opteron(tm) Processor 6128 64 MB RAM
3. Coordinates Added in Force-Directed Graph Process
4. Network Load Across Internet, Unity 3D Load Time ~30 Seconds

FDG Processing Time
FDG Processing Time

Brief Discussion:

Creating the FDG on the back-end servers, outside the visualization engine,  for large data sets is better for the performance of the visualization engine.  As the graph grows larger, the graph processing will require a distributing graph processing architecture.