Configuring notification forwarding to Kafka

Virtuozzo Hybrid Infrastructure allows forwarding notifications to Apache Kafka, an open source platform for streaming events. You can configure the following security protocols to communicate with Kafka brokers:

PLAINTEXT. This protocol provides no security and is used by default if no other security protocol is specified.
SSL. The Secure Sockets Layer (SSL) protocol is used for traffic encryption and two-way authentication between the client and server. SSL uses private-key/certificates pairs, which are used during the SSL handshake process.
SASL_PLAINTEXT. Simple Authentication and Security Layer (SASL) is used with PLAINTEXT as the transport layer, where requests are not encrypted.
SASL_SSL. SASL is used with SSL as the transport layer, where encryption is enabled.

Virtuozzo Hybrid Infrastructure supports SASL with Salted Challenge Response Authentication Mechanism (SCRAM) as the default authentication mechanism. You can choose between the SCRAM-SHA-256 and SCRAM-SHA-512 mechanisms, which use the SHA-256 and SHA-512 hashing functions, respectively. For details on how SASL/SCRAM works, see RFC 5802.

To enable Kafka notification forwarding

Use the following command:

vinfra service compute notification set [--transport-url <transport-url>]
                                        [--kafka-security-protocol {PLAINTEXT,SASL_PLAINTEXT,SSL,SASL_SSL}]
                                        [--kafka-sasl-mechanism {SCRAM-SHA-256,SCRAM-SHA-512}]
                                        [--kafka-ssl-ca-cert <path>] [--kafka-ssl-client-cert <path>]

--transport-url <transport-url>

Enable notification forwarding through the specified transport URL in the format driver://[user:pass@]host:port[,[userN:passN@]hostN:portN], where:

driver is the supported transport driver (kafka, ampq, or rabbit)
user:pass are the username and password used for authentication with the messaging broker
host:port specifies the hostname or IP address and port number of the messaging broker

Messages will be published to the "notifications" topic.

Example: kafka://10.10.10.10:9092

--kafka-security-protocol {PLAINTEXT,SASL_PLAINTEXT,SSL,SASL_SSL}

Protocol used to communicate with brokers

--kafka-sasl-mechanism {SCRAM-SHA-256,SCRAM-SHA-512}

Authentication mechanism to use for the SASL protocol

--kafka-ssl-ca-cert <path>

Path to a PEM file with the CA certificate that is used to verify the server

--kafka-ssl-client-cert <path>

Path to a PEM file with the SSL client certificate that is used for client authentication

For example, to configure notification forwarding to the Kafka server with the IP address 10.10.10.10 with the SASL_SSL security protocol and the SCRAM-SHA-512 authentication mechanism, run the following command specifying two PEM files:

# vinfra service compute notification set --transport-url kafka://10.10.10.10:9092 --kafka-security-protocol SASL_SSL \
--kafka-sasl-mechanism SCRAM-SHA-512 --kafka-ssl-ca-cert kafka-server.pem --kafka-ssl-client-cert kafka-client.pem

You can check the notification forwarding options in the vinfra service compute notification show output:

# vinfra service compute notification show
+---------------+-------------------------------------------------+
| Field         | Value                                           |
+---------------+-------------------------------------------------+
| kafka         | sasl_mechanism: SCRAM-SHA-512                   |
|               | security_protocol: SASL_SSL                     |
|               | ssl_ca_file: kafka_ssl_ca_cert.pem              |
|               | ssl_client_cert_file: kafka_ssl_client_cert.pem |
| transport_url | kafka://10.10.10.10:9092                        |
+---------------+-------------------------------------------------+

To disable encryption of Kafka messaging

Run the following command:

# vinfra service compute notification disable --kafka-encryption

The Kafka security protocol will be set to the default value PLAINTEXT.

You can check that the encryption is successfully disabled in the vinfra service compute notification show output:

# vinfra service compute notification show
+---------------+--------------------------+
| Field         | Value                    |
+---------------+--------------------------+
| transport_url | kafka://10.10.10.10:9092 |
+---------------+--------------------------+

To disable Kafka notification forwarding

Run the following command:

# vinfra service compute notification disable --notification-forwarding