[OCP4] Forwarding Logs from a VM on OCP Virtualization to OpenShift Logging

To forward /var/log/messages or other logs using rsyslog from a VM running on OCP virtualization to OpenShift Logging, you must configure a receiver in the ClusterLogForwarder. Below is an example configuration for OpenShift.

By default, OCP Virtualization logs from /dev/console can be collected by the collector, which can be enabled through the web console(figure below). However, for other paths, an agent needs to be installed to forward the logs.

enable /dev/console logs collect for VM

Prerequitsite

• OpenShift Virtualization
• OpenShift-Logging 6 + Lokistack

Configuration on OpenShift

apiVersion: observability.openshift.io/v1
kind: ClusterLogForwarder
metadata:
  name: collector
  namespace: openshift-logging
spec:
  inputs:
    - name: syslogserver
      receiver:
        port: 5514
        type: syslog
      type: receiver
  managementState: Managed
  outputs:
    - lokiStack:
        authentication:
          token:
            from: serviceAccount
        target:
          name: lokistack-demo
          namespace: openshift-logging
      name: default-lokistack
      tls:
        ca:
          configMapName: openshift-service-ca.crt
          key: service-ca.crt
      type: lokiStack
  pipelines:
    - inputRefs:
        - application
        - infrastructure
        - syslogserver
      name: default-logstore
      outputRefs:
        - default-lokistack
  serviceAccount:
    name: collector

The OpenShift Logging component will create a secret for the collector to use with the syslog receiver, located in the openshift-logging project. You need to retrieve these certificates to use rsyslog on the VM for authentication with the collector.

To obtain the necessary certificates and keys, execute the following commands:

## get collector cert,key
oc -n openshift-logging extract secret/collector-syslogserver --confirm

## get CA
oc extract secret/signing-key --confirm -n openshift-service-ca

Configuration on the VM(Rhel)

install openssl for rsyslog module

yum -y install rsyslog-openssl

Allow the use of port 5514 in SELinux:

semanage port -a -t syslogd_port_t -p tcp 5514

Configure /etc/rsyslog.conf to forward logs to the collector service, which can be found using oc get service -n openshift-logging:

global(
  DefaultNetstreamDriver="ossl"
  DefaultNetstreamDriverCAFile="/etc/pki/ca-trust/source/anchors/client-ca.pem"
  DefaultNetstreamDriverCertFile="/etc/pki/ca-trust/source/anchors/client-cert.pem"
  DefaultNetstreamDriverKeyFile="/etc/pki/ca-trust/source/anchors/client-key.pem"
)


*.* action(
  type="omfwd"
  StreamDriverMode="1"
  StreamDriverPermittedPeers="collector-syslogserver.openshift-logging.svc"
  StreamDriverAuthMode="x509/name"
  target="collector-syslogserver.openshift-logging.svc.cluster.local" port="5514" protocol="tcp"
) 
...

Restart the rsyslog service.

Testing

We tested by generating a test entry in /var/log/messages on the VM, and it should be forwarded to the collector as we configured.

command to send messages to /var/log/message

Log from the syslog receiver will be stored in the infrastructure index.

Logs from the VM are received by the syslog collector and stored in Loki.

Disclaimer

This is intended solely for testing purposes and is not officially affiliated with or supported by Red Hat solutions. The information provided herein is for educational and testing use only.