Administration

Logging

Each system component logs its own activity via Syslog protocol. This enables to create an auditory track of all events having been executed in the system.

The system also makes it possible to direct logging into one of the pipelines and thus ensure one´s own installation monitoring.

Monitoring

Internal control mechanisms of the BitSwan product monitor the whole system and thus provide the user interface with important information about the state and correctness of real-time data processing. This set of functionalities is essential for debugging of real-time data processing, which requires the whole system to run precisely.

Monitoring BSPump

BitSwan implicitly provides the following main metrics for each process (i.e. pipeline):

  • ● Throughput: measures a so called IOPS (number of input / output operations per second) and indicates the number of events incoming and outgoing from given pipeline
  • ● Duty cycle: displays the ratio of data processing in a pump as compared to waiting for their coming
  • ● Time drift: provides information about event processing delay as compared to real-time event occurrence
  • ● Error ratio: provides information about the number of damaged data in a pipeline
  • ● Warning ratio: provides information about the number of potential error data in a pipeline

Monitoring pipeline in Grafana

Known tools - e.g. Grafana or Zabbix - can further process outputs from the monitoring in the form of logs and metrics for a more specific performance analysis.

Data in Kibana

Přehled výkonosti v Grafaně

Moreover, BitSwan can provide performance data to standard monitoring tools like Zabbix, HP Open View etc.

LogMan.io

BitSwan is able to dispatch its logs and metrics (i.e. telemetrics) via the ITGuard product produced by theTeskaLabs company and thus enable a detailed insight into processing, analysis and storage of data by means of Grafana and Kibana tools.

It suffices to configure the BitSwan product correctly with a login username and password.

[logman.io]
url=amqps://{username}:{password}@logman-bridge:5477/{virtualhost}
username={YOUR_USERNAME}
password={YOUR_PASSWORD}
virtualhost={YOUR_VHOST}

InfluxDB

By means of configuration BitSwan makes it possible to dispatch the above mentioned metrics into InfluxDB database, from which they can be loaded and visualized in Grafana tool.

[asab:metrics:influxdb]
url={YOUR_INFLUXDB_URL}
db={YOUR_INFLUXDB_DATABASE}
username={YOUR_INFLUXDB_USERNAME}
password={YOUR_INFLUXDB_PASSWORD}

Grafana

Grafana is a tool which enables to create graphs and other visualisations over the BitSwan metrics. Metrics stored in InfluxDB database can be the input.

Performance in Grafana