The Flexible NetFlow—Prevent Export
Storms feature prevents export storms at a NetFlow Collecting (NFC) device,
especially when multiple Flexible NetFlow (FNF) entities are configured to export FNF
records to the same NFC at the same synchronized wallclock time. Export storms
occur due to the creation of the synchronized cache type. Export spreading reduces the severity of export storms and mitigates
their impact.
Synchronized cache with spreading requires adding the interval
timestamp field for the synchronized cache. When no spreading is configured, it is recommended to add
the interval as a key, but the configuration is not rejected to maintain
backward compatibility. If no export spread is specified, the default behavior
is immediate export. The spread time must be smaller than half of the interval.
Therefore, it will be set to half the interval time or to the configured spread interval, whichever is lower (but not lower
than 1 second).
You must not enable spreading when the interval sync timeout is
lower than 10 seconds (5- second spreading). This requirement comes from the
need for asynchronous monitors to aggregate the data within a few seconds.
Spreading might start a couple of seconds after the interval ends in order to
complete the aggregation. If a synchronized interval value is lower than 10
seconds, no spreading option is visible in the command-line interface (CLI). The default spread interval, if unspecified,
is 30 seconds. The maximum synchronized interval timeout value
is 300 seconds. For native FNF monitors, the maximum synchronized interval timeout value could be larger. The rate
calculation is provisioned as follows:
- The simple implementation
is a constant rate based on the cache-size/spread-interval.
- An improved implementation
is based on the current-previous-interval-cache-size/spread-interval. This
provides better results when the cache is not full.
The NetFlow/IPFIX header timestamp is set to the time when the record leaves
the device (and not when the record leaves the NetFlow cache). The timestamp
fields in the record itself capture the timestamp of the packets and are
accounted for in the NetFlow cache. A new, implementation-driven concept of a
“small interval” is now implicitly introduced and understood to be directly in
contrast with the concept of a “large interval”. The “large
interval” can be thought of as simply the sync interval as configured by the
CLI. This is the interval at the beginning of which the entire export process is
to be initiated. It corresponds to the "synchronized interval" that is driven
and defined by the CLI. At the beginning of a “large interval”, we must take the
number of records in the cache and divide that number by the number of seconds available in
which to export these records, thus yielding the calculated or derived quantity of
“records per second”.
For example, if there are 100,000 records in the cache and 100 seconds
in which to export these records, we would calculate and store the value 1000
records/second. Because this quantity is expressed in seconds, it follows that we will need to count the records exported
in small intervals that are
one second in duration.
This then, implicitly, defines the notion of a “small interval”, which
is, to be succinct and equal to one second. Combining this idea of small and large
intervals with the need for a state or context, it quickly becomes evident that a timer thread must be able to discern
if it is beginning a “small interval” or a “large interval”.
Feedback