Overcome the MongoDB Backup is down alert

I’ve set up a new MongoDB single node replicaset using MongoDB 4.2.7.    Although I’m using the same server and agents configuration settings as other replicasets in the same deployment – there is a regular MongoDB alert being triggered. 

Backup is down
Backup Agent

Alert Details

CREATED 2020/05/29 13:12 GMT
DATA-BEARING HOSTS (INSTALL AGENT ON AT LEAST ONE) rsdevelop (at least one of the following mongod(s) needs a co-located agent: [muserver.xxxxx.net:27017])
OPEN

The scheduled backups are being completed , and there aren’t any errors. Why is the Backup is down  alert being triggered?

I checked through the MongoDB documentation and the related text for this alert is:

Backup is down

The Backup for a project with at least one active replica set or cluster is down for more than 1 hour.

To resolve this alert:

  1. To see which host serves the Backup, click Deployment, then the Servers tab.
  2. Check the Backup log file on that host.

In the MongoDB documentation there is a specification about how MongoDB alert notifications works:

“When a condition triggers an alert, you receive the alert at regular intervals until the alert resolves or Ops Manager cancels it. You can acknowledge an alert for a period of time, but if the alert condition persists, you will again receive notifications once the acknowledgment period ends”

I followed the path to the Backup log file on the host and reviewed all messages  – with no error messages – apart from the initial one.

So I started reviewing the Backup Agent status of the specific host – just doublecking the status and any other  observations. I noticed the host Backup Agent was in standby , and the other host in the deployment was Active. But remember reading in the MongoDB documentation, suggesting it’s not a problem:

“Only one host can backup a deployment at a time. On the Server tab, the host that is backing up the deployment displays Backup – active. Any other host with Backup activated displays Backup – standby”

Next step was to restart the automation agent for the MongoDB server.

–Stop the Automation Agent 

sudo service mongodb-mms-automation-agent stop

–Start the Automation Agent 

sudo service mongodb-mms-automation-agent start

The restart of the Automation agent – did not resolve the issue. I went to the next step , and restarted the MongoDB server – this stopped the alert reappearing. The Backup Agent is working fine , all snapshots and oplog slices are successfully and up to date.

Author: Jack Vamvas

Leave a Reply

Discover more from ciquery

Subscribe now to keep reading and get access to the full archive.

Continue reading