Skip to main content

Connecting a RabbitMQ and Redis cluster

Introduction​

Important

The following guide is valid only for ONLYOFFICE Docs Enterprise Edition and Developer Edition.

Use this guide to connect ONLYOFFICE Docs to a RabbitMQ cluster and a Redis Cluster. To run ONLYOFFICE Docs itself as a cluster, follow ONLYOFFICE Docs on Kubernetes.

note

ONLYOFFICE Docs Community Edition doesn't use Redis. Starting from version 9.4.0, it doesn't use RabbitMQ either.

Editing the configuration​

The settings below are made in the local.json file:

  • For Linux: /etc/onlyoffice/documentserver/local.json
  • For Windows: %ProgramFiles%\ONLYOFFICE\DocumentServer\config\local.json

Add the parameters to the existing sections of the file, then restart ONLYOFFICE Docs services.

Connecting a RabbitMQ cluster​

ONLYOFFICE Docs connects to a single RabbitMQ URL. The rabbitmq.url parameter takes one string; a list of broker URLs is not supported. Make the RabbitMQ cluster reachable through one endpoint:

  • a TCP load balancer, for example HAProxy, with a highly available address (keepalived or VRRP);
  • a DNS name that resolves to the running nodes.

Specify this endpoint in local.json:

{
  "rabbitmq": {
    "url": "amqp://<user>:<password>@rabbitmq.example.com:5672"
  }
}

For Docker, pass the same URL in the AMQP_URI environment variable.

When a RabbitMQ node goes down, ONLYOFFICE Docs reconnects automatically. With classic queues, it starts consuming again when the node that owns a queue restarts.

Heartbeat​

ONLYOFFICE Docs enables TCP keepalive and uses the AMQP heartbeat timeout offered by RabbitMQ, 60 seconds by default, to detect broken connections. Set the idle timeout of the RabbitMQ load balancer longer than the heartbeat timeout. To request another timeout, set heartbeat in seconds in rabbitmq.url:

{
  "rabbitmq": {
    "url": "amqp://<user>:<password>@rabbitmq.example.com:5672?heartbeat=30"
  }
}

Quorum queues​

By default, ONLYOFFICE Docs uses classic queues. You can switch to quorum queues, which RabbitMQ replicates across the cluster nodes. Set x-queue-type to quorum for four queues:

{
  "rabbitmq": {
    "queueconverttask": {"options": {"arguments": {"x-queue-type": "quorum"}}},
    "queueconvertresponse": {"options": {"arguments": {"x-queue-type": "quorum"}}},
    "queueconvertdead": {"options": {"arguments": {"x-queue-type": "quorum"}}},
    "queuedelayed": {"options": {"arguments": {"x-queue-type": "quorum"}}}
  }
}

Keep queuepubsub classic: it is an exclusive auto-delete queue, which quorum queues don't allow.

Quorum queues support two priority levels in RabbitMQ 4.0 to 4.2 and 32 levels starting from RabbitMQ 4.3. Earlier versions don't support priorities, so conversion tasks are no longer prioritized.

warning

RabbitMQ cannot change the type of an existing queue. Before restarting ONLYOFFICE Docs with the new settings, stop it and delete the ds.converttask6, ds.convertresponse, ds.convertdead and ds.delayed queues, or set new names in the name parameters of these queues. Do this during a maintenance window: messages left in the deleted queues are lost.

Connecting a Redis Cluster​

ONLYOFFICE Docs supports Redis Cluster with both Redis clients it includes, node-redis (default) and ioredis. Specify the cluster nodes in the services.CoAuthoring.redis section of local.json.

For node-redis, use optionsCluster:

{
  "services": {
    "CoAuthoring": {
      "redis": {
        "name": "redis",
        "optionsCluster": {
          "rootNodes": [
            {"url": "redis://192.168.0.150:7000"},
            {"url": "redis://192.168.0.150:7001"},
            {"url": "redis://192.168.0.150:7002"}
          ],
          "defaults": {"username": "default", "password": "<password>"}
        }
      }
    }
  }
}

For ioredis, use iooptionsClusterNodes and iooptionsClusterOptions:

{
  "services": {
    "CoAuthoring": {
      "redis": {
        "name": "ioredis",
        "iooptionsClusterNodes": [
          {"host": "192.168.0.150", "port": 7000},
          {"host": "192.168.0.150", "port": 7001},
          {"host": "192.168.0.150", "port": 7002}
        ],
        "iooptionsClusterOptions": {"redisOptions": {"password": "<password>"}}
      }
    }
  }
}

Redis Cluster supports only database 0.

Mapping node addresses​

The cluster nodes tell the client their own addresses. If ONLYOFFICE Docs cannot reach these addresses, for example because of NAT or Docker networks, errors such as MOVED ... or Cannot read properties of undefined (reading 'master') appear in the logs.

Map each announced address to one that ONLYOFFICE Docs can reach. For node-redis, add nodeAddressMap to optionsCluster:

"optionsCluster": {
  "rootNodes": [{"url": "redis://redis1.example.com:7000"}],
  "nodeAddressMap": {
    "10.0.0.5:7000": {"host": "redis1.example.com", "port": 7000},
    "10.0.0.6:7000": {"host": "redis2.example.com", "port": 7000},
    "10.0.0.7:7000": {"host": "redis3.example.com", "port": 7000}
  }
}

For ioredis, add natMap to iooptionsClusterOptions:

"iooptionsClusterOptions": {
  "natMap": {
    "10.0.0.5:7000": {"host": "redis1.example.com", "port": 7000},
    "10.0.0.6:7000": {"host": "redis2.example.com", "port": 7000},
    "10.0.0.7:7000": {"host": "redis3.example.com", "port": 7000}
  }
}

Checking the setup​

Steps 3 and 4 require quorum queues replicated across at least three RabbitMQ nodes and a Redis Cluster with a replica for each master.

  1. Restart ONLYOFFICE Docs and check the DocService log, /var/log/onlyoffice/documentserver/docservice/out.log on Linux.
    • RabbitMQ connection errors are logged as [AMQP] connect error, with the attempt number and the time until the next retry.
    • Redis errors contain redisClient.
  2. Open a document from two browsers and edit it together.
  3. While the document is open, stop one RabbitMQ node. Check that editing continues and that the document is saved after you close it.
  4. Start the RabbitMQ node again, then repeat the test with one Redis master node. Start the Redis node again after the test.