openmon.org | FreeBSD

FreeBSD - Using SiLK

Hi !

When we talk about seeing things, it is tempting to imagine the presence of images, but this is not always the case. Take the network, for example, knowing what’s happening at the data level involves capturing that data and analyzing it.

Why not use the “IPFIX” protocol? (https://www.rfc-editor.org/rfc/rfc7011.html) Here I’ll use “SiLK” ( https://tools.netsa.cert.org/silk/ ).

As mentioned in the mentioned web page: “SiLK, the System for Internet-Level Knowledge, is a collection of traffic analysis tools developed by the CERT Network Situational Awareness Team (CERT NetSA) to facilitate security analysis of large networks.”

Given the profusion of things that the “SiLK” suite can do, I will just show two types of uses. To do this, I will use some binaries offered in the software suite.

But first, as I use FreeBSD, let’s install the package “SiLK”.

# pkg install silktools

Note: I must admit I use this inelegant way to launch the system processes used.

$cat /etc/cron.d/silk

Content:

@reboot root:wheel /usr/local/bin/run_rwflowpack.sh
@reboot root:wheel /usr/local/bin/run_yaf.sh

Here is the “run_yaf.sh” file:

$ cat /usr/local/bin/run_yaf.sh

Content:

#!/bin/sh

/usr/local/bin/yaf --in em0 --live pcap --ipfix tcp --out 10.0.1.16 --log /var/data/silk/log/yaf.log --silk --ipfix-port=9903 --applabel --max-payload 2048 --plugin-name=/usr/local/lib/yaf/dpacketplugin.so &

Some informations about the program, see (man yaf)

yaf is Yet Another Flowmeter and yaf is a suite of tools to do flow
       metering.  yaf is used as a sensor to capture flow information on a
       network and export that information in IPFIX format. It reads packet
       data from pcap(3) dumpfiles as generated by tcpdump(1), from live
       capture from an interface using pcap(3), pf_ring, an Endace DAG capture
       device, a Napatech adapter, or Netronome NFE card aggregates these
       packets into flows, and exports flow records via IPFIX over SCTP, TCP
       or UDP, Spread, or into serialized IPFIX message streams (IPFIX files)
       on the local file system.

Here is the “run_rwflowpack.sh” file:

$ cat /usr/local/bin/run_rwflowpack.sh

Content:

#!/bin/sh

rwflowpack --sensor-configuration=/usr/local/etc/silk/sensor.conf --input-mode=stream --site-config-file=/usr/local/etc/silk/silk.conf --root-directory=/var/data/silk --log-directory=/var/data/silk/log

Some informations about the program, see (man rwflowpack)

man rwflowpack
    rwflowpack is a daemon that runs as part of the SiLK flow collection
    and packing tool-chain.  The primary job of rwflowpack is to convert
    each incoming flow record to the SiLK Flow format, categorize each
    incoming flow record (e.g., as incoming or outgoing), set the sensor
    value for the record, and determine which hourly file will ultimately
    store the record.

Here is the “sensor.conf” file.

 # cat /usr/local/etc/silk/sensor.conf

Content:

probe S0 ipfix
         listen-on-port 9903
         protocol tcp
         listen-as-host 10.0.1.16
end probe

group my-network
      ipblocks 10.0.1.0/24 192.168.2.0/24
end group

sensor S0
       ipfix-probes S0
       internal-ipblocks @my-network
       external-ipblocks remainder
end sensor

Here is the “silk.conf” file:

 # cat /usr/local/etc/silk/silk.conf

Content:

version 2

sensor 0 S0

class all
    sensors S0
end class

class all
    type  0 in      in
    type  1 out     out
    type  2 inweb   iw
    type  3 outweb  ow
    type  4 innull  innull
    type  5 outnull outnull
    type  6 int2int int2int
    type  7 ext2ext ext2ext
    type  8 inicmp  inicmp
    type  9 outicmp outicmp
    type 10 other   other

    default-types in inweb inicmp
end class

default-class all

packing-logic "packlogic-twoway.so"

Let’s do a brief overview of the binaries mentioned:

rwfilter: Choose which SiLK Flow records to process
rwsort: Sort SiLK Flow records on one or more fields
rwcut: Print selected fields of binary SiLK Flow records
rwscan: Detect scanning activity in a SiLK dataset

Let’s use some shell files to put it all together (more easy):

See network flows (shflows.sh)

#!/usr/local/bin/bash

# STARTDATE YYY/MM/DD:HH
# ENDDATE YYY/MM/DD:HH

STARTDATE=$1
ENDDATE=$2

if [[ $# -eq 1 ]]
then
   rwfilter --start-date=$STARTDATE --type=all --proto=0- --pass-destination=stdout --site-config-file=/usr/local/etc/silk/silk.conf | rwsort --field=stime | rwcut
elif [[ $# -eq 2 ]]
then
   rwfilter --start-date=$STARTDATE --end-date=$ENDDATE --type=all --proto=0- --pass-destination=stdout --site-config-file=/usr/local/etc/silk/silk.conf | rwsort --field=stime | rwcut
fi

Check if network scans are present (shscans.sh)

#!/bin/sh

DATE=$1

rwfilter --type=all --sensor=S0 --start=$DATE --proto=0- --pass=stdout | rwsort --fields=stime | rwfilter --input-pipe=stdin --proto=0- --pass=stdout | rwscan --scan-model=2

Let’s look at some flows observed between 01 AM and 02 AM:

# shflows.sh 2023/03/17:01 2023/03/17:02
root@silk:/usr/local/bin # shflows.sh 2023/03/17:01 2023/03/17:02
                                    sIP|                                    dIP|sPort|dPort|pro|   packets|     bytes|   flags|                  sTime| duration|                  eTime|sen|
                           192.168.2.64|                                8.8.4.4|20877|   53| 17|         1|        83|        |2023/03/17T01:00:02.560|    0.007|2023/03/17T01:00:02.567| S0|
                           192.168.2.64|                                8.8.4.4|20877|   53| 17|         1|        83|        |2023/03/17T01:00:02.560|    0.007|2023/03/17T01:00:02.567| S0|
                                8.8.4.4|                           192.168.2.64|   53|20877| 17|         1|       160|        |2023/03/17T01:00:02.567|    0.000|2023/03/17T01:00:02.567| S0|
                                8.8.4.4|                           192.168.2.64|   53|20877| 17|         1|       160|        |2023/03/17T01:00:02.567|    0.000|2023/03/17T01:00:02.567| S0|
                          AA.BBB.CC.DDD|                           192.168.2.64|    0| 2048|  1|         1|        36|        |2023/03/17T01:00:03.085|    0.000|2023/03/17T01:00:03.085| S0|
                          AA.BBB.CC.DDD|                           192.168.2.64|    0| 2048|  1|         1|        36|        |2023/03/17T01:00:03.085|    0.000|2023/03/17T01:00:03.085| S0|
                           192.168.2.64|                          AA.BBB.CC.DDD|    0|    0|  1|         1|        36|        |2023/03/17T01:00:03.085|    0.000|2023/03/17T01:00:03.085| S0|
                           192.168.2.64|                          AA.BBB.CC.DDD|    0|    0|  1|         1|        36|        |2023/03/17T01:00:03.085|    0.000|2023/03/17T01:00:03.085| S0|

Let’s see if we got some network scans.

 # ./shscans.sh 2023/03/17
             sip| proto|                   stime|                   etime|     flows|   packets|     bytes|
   x.y.z.189|     6|     2023-03-17 12:23:09|     2023-03-17 12:23:09|        58|        58|      2552|
   a.b.c.172|     6|     2023-03-17 12:47:53|     2023-03-17 12:47:54|        47|        47|      2068|
   a.b.c.172|     6|     2023-03-17 12:47:54|     2023-03-17 12:47:54|        58|        60|      2632|
   a.b.c.172|     6|     2023-03-17 12:47:54|     2023-03-17 12:47:55|       576|       576|     25344|
   a.b.c.172|     6|     2023-03-17 12:47:55|     2023-03-17 12:47:57|      1184|      1186|     52176|
   a.b.c.172|     6|     2023-03-17 12:47:57|     2023-03-17 12:47:58|      2088|      2089|     91912|

To summarize, I showed here only a weak outline of what one can do with “SiLK”. I will come back in the future with other points concerning this very good tool.

Cheers

Tree Of Things

FreeBSD - Using SiLK