Skip to content

Add new backup report feature - #13939

Open
JoaoJandre wants to merge 5 commits into
apache:mainfrom
scclouds:add-backup-report-feature
Open

JoaoJandre wants to merge 5 commits into
apache:mainfrom
scclouds:add-backup-report-feature

Conversation

@JoaoJandre

@JoaoJandre JoaoJandre commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Description

This PR adds a feature for periodically sending basic backup reports to operators. To enable backup reports, the environment's alert email settings must be configured beforehand. To configure the report task, the following settings have been added:

Configuration Description Default Value
backup.report.task.enabled Enables the periodic backup report task. false
backup.report.period Period, in minutes, between task executions. The generated report will contain backups from the last backup.report.period minutes, as well as a forecast of the next execution of backup schedules that may potentially run during the next backup.report.period minutes. 1440
backup.report.timeout Timeout in minutes for the report generation process. If a Management Server finds a task that has been running longer than the configured value, the task will be considered canceled. 10

Additionally, the getBackupReport API has been added to generate reports through the API. This API returns the report in the API response and does not send emails to operators. The API supports the following parameters:

Parameter Description Required?
zoneid ID of the zone for which the report should be generated. No
domainid ID of the domain for which the report should be generated. No
accountid ID of the account for which the report should be generated. Mutually exclusive with projectid. No
projectid ID of the project for which the report should be generated. Mutually exclusive with accountid. No
startdate Report start date. Yes
enddate Report end date. If it is in the future, a forecast of the next execution of backup schedules within the period will also be returned. Yes

This API is only available to Admin accounts.

API Usage Example
(labinterno) 🐱 > get backupreport startdate='2026-06-23' enddate='2026-06-24'
{
  "backupreport": {
    "backupstoragetotal": 6.224914558231831,
    "domainreport": [
      {
        "accountreport": [
          {
            "account": "admin",
            "accountid": "e3c10cf4-b564-11ee-afc2-5254000a59f7",
            "backupstoragetotal": 3.1328735426068306,
            "deletedbackup": [
              {
                "backupofferingname": "standard",
                "created": "2026-06-23T19:01:24+0000",
                "id": "b980ef1e-6a95-4aa5-803e-328029020d4e",
                "name": "da-2026-06-23T19:01:22.803027606Z",
                "removed": "2026-06-24T19:01:09+0000",
                "virtualmachineid": "dd3e719e-9063-46a2-8596-85d4f1319199",
                "virtualmachinename": "vm1",
                "zone": "zona",
                "zoneid": "eff81807-ee5f-4397-98ea-855a4b8b541b"
              },
              {
                "backupofferingname": "standard",
                "id": "8bda5fa7-ff71-4216-b251-c653d5441295",
                "name": "VM-ff96a116-d856-4899-87d2-3ec3cab8a486-2026-06-24T18:08:46.052496381Z",
                "removed": "2026-06-24T18:11:08+0000",
                "virtualmachineid": "ff96a116-d856-4899-87d2-3ec3cab8a486",
                "virtualmachinename": "VM-ff96a116-d856-4899-87d2-3ec3cab8a486",
                "zone": "zona",
                "zoneid": "eff81807-ee5f-4397-98ea-855a4b8b541b"
              },
              {
                "backupofferingname": "standard",
                "id": "a5a44ec9-7532-482b-93bf-4cd6b64fcab5",
                "name": "VM-ff96a116-d856-4899-87d2-3ec3cab8a486-2026-06-24T18:14:24.053079158Z",
                "removed": "2026-06-24T18:15:38+0000",
                "virtualmachineid": "ff96a116-d856-4899-87d2-3ec3cab8a486",
                "virtualmachinename": "VM-ff96a116-d856-4899-87d2-3ec3cab8a486",
                "zone": "zona",
                "zoneid": "eff81807-ee5f-4397-98ea-855a4b8b541b"
              }
            ],
            "failedbackup": [
              {
                "backupofferingname": "standard",
                "created": "2026-06-24T18:15:22+0000",
                "failurereason": "No available host found to create backup [ae522d32-97af-4069-b311-0e6a25a7f227] of VM [dd3e719e-9063-46a2-8596-85d4f1319199]. Setting the backup as Failed.",
                "id": "ae522d32-97af-4069-b311-0e6a25a7f227",
                "logid": "f3e8d632",
                "name": "da-2026-06-24T18:15:22.779064677Z",
                "virtualmachineid": "dd3e719e-9063-46a2-8596-85d4f1319199",
                "virtualmachinename": "vm1",
                "zone": "zona",
                "zoneid": "eff81807-ee5f-4397-98ea-855a4b8b541b"
              }
            ],
            "providerinfo": [],
            "sucessfulbackup": [
              {
                "backupofferingname": "standard",
                "created": "2026-06-24T18:15:00+0000",
                "id": "50d971b4-bdec-4a37-a79b-564058e6f3a8",
                "name": "da-2026-06-24T18:15:00.489465709Z",
                "virtualmachineid": "dd3e719e-9063-46a2-8596-85d4f1319199",
                "virtualmachinename": "vm1",
                "zone": "zona",
                "zoneid": "eff81807-ee5f-4397-98ea-855a4b8b541b"
              },
              {
                "backupofferingname": "standard",
                "created": "2026-06-24T19:01:07+0000",
                "id": "a9e93b0f-5992-4793-8956-25d7fccdebfe",
                "name": "da-2026-06-24T19:01:05.567776285Z",
                "virtualmachineid": "dd3e719e-9063-46a2-8596-85d4f1319199",
                "virtualmachinename": "vm1",
                "zone": "zona",
                "zoneid": "eff81807-ee5f-4397-98ea-855a4b8b541b"
              },
              {
                "backupofferingname": "standard",
                "created": "2026-06-23T19:02:50+0000",
                "id": "af163123-12a5-4f76-b683-bc18a67ce141",
                "name": "VM-ff96a116-d856-4899-87d2-3ec3cab8a486-2026-06-23T19:02:48.527866466Z",
                "virtualmachineid": "ff96a116-d856-4899-87d2-3ec3cab8a486",
                "virtualmachinename": "VM-ff96a116-d856-4899-87d2-3ec3cab8a486",
                "zone": "zona",
                "zoneid": "eff81807-ee5f-4397-98ea-855a4b8b541b"
              }
            ]
          }
        ],
        "backupstoragetotal": 3.1328735426068306,
        "domain": "ROOT",
        "domainid": "8e33df7b-b564-11ee-afc2-5254000a59f7"
      },
      {
        "accountreport": [
          {
            "backupstoragetotal": 3.092041015625,
            "deletedbackup": [],
            "failedbackup": [],
            "project": "project1",
            "projectid": "7ba37e0a-d40e-4812-a030-229ed506c26b",
            "providerinfo": [],
            "sucessfulbackup": [
              {
                "backupofferingname": "standard",
                "created": "2026-06-24T18:18:24+0000",
                "id": "aaf5f9a1-710c-4755-b973-0e7fec2a3aee",
                "name": "Test-Project-VM-2026-06-24T18:18:23.624968182Z",
                "virtualmachineid": "494e2476-ea06-4b88-8af3-3b6c8c533a16",
                "virtualmachinename": "Test-Project-VM",
                "zone": "zona",
                "zoneid": "eff81807-ee5f-4397-98ea-855a4b8b541b"
              }
            ]
          }
        ],
        "backupstoragetotal": 3.092041015625,
        "domain": "A",
        "domainid": "c5bcdc6a-c138-4483-939e-8b31ccf8b912"
      }
    ],
    "enddate": "2026-06-24T23:59:59+0000",
    "schedulereport": [],
    "startdate": "2026-06-23T00:00:00+0000"
  }
}

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • Build/CI
  • Test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

Example generated email:

image

How Has This Been Tested?

Test Result
Call the getBackupReport API passing only startdate and enddate All backups belonging to the specified period were reported
Call the getBackupReport API specifying the domain Only backups for the domain were returned
Call the getBackupReport API specifying the account Only backups for the account were returned
Call the getBackupReport API specifying the project Only backups for the project were returned
Configure alert email notifications and the backup reporting period to 3 days Email received with backups from the last 3 days and the forecast for the next 3 days

@codecov

codecov Bot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 1.21413% with 895 lines in your changes missing coverage. Please review.
✅ Project coverage is 19.88%. Comparing base (602d9ec) to head (56c14a8).

Files with missing lines Patch % Lines
...che/cloudstack/backup/BackupReportServiceImpl.java 0.00% 246 Missing ⚠️
...che/cloudstack/backup/BackupReportKbossJoinVO.java 0.00% 90 Missing ⚠️
...g/apache/cloudstack/backup/BackupReportJoinVO.java 0.00% 75 Missing ⚠️
...apache/cloudstack/api/response/BackupResponse.java 0.00% 57 Missing ⚠️
...tack/api/response/BackupReportAccountResponse.java 0.00% 55 Missing ⚠️
.../cloudstack/api/response/BackupReportResponse.java 0.00% 45 Missing ⚠️
...loudstack/api/response/BackupScheduleResponse.java 0.00% 45 Missing ⚠️
.../apache/cloudstack/backup/KbossBackupProvider.java 11.76% 44 Missing and 1 partial ⚠️
...ck/api/command/user/backup/GetBackupReportCmd.java 0.00% 34 Missing ⚠️
...stack/api/response/BackupReportDomainResponse.java 0.00% 25 Missing ⚠️
... and 15 more
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #13939      +/-   ##
============================================
- Coverage     19.91%   19.88%   -0.03%     
+ Complexity    20200    20199       -1     
============================================
  Files          6373     6388      +15     
  Lines        577230   578126     +896     
  Branches      70696    70732      +36     
============================================
+ Hits         114974   114983       +9     
- Misses       449690   450576     +886     
- Partials      12566    12567       +1     
Flag Coverage Δ
uitests 3.71% <ø> (ø)
unittests 21.15% <1.21%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@GaOrtiga GaOrtiga left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JoaoJandre I ran some tests, overall LGTM, just a few minor nits

If a backup was created within the gathered period, but has been removed after, it appears as a deleted backup and shows 0 total storage. Given that the deletion happened after the period, it should show as a succesful backup. Same for a failed backup, where the failure reason is not showed.

API return
(admin) 🐱 > get backupreport startdate='2026-08-01' enddate='2026-08-23T11:59:59-0300' domainid=517e680a-dc14-4cac-98ab-5b6a21a2ac36
{
  "backupreport": {
    "backupstoragetotal": 0,
    "domainreport": [
      {
        "accountreport": [
          {
            "account": "userc",
            "accountid": "62f2b1f0-4373-4983-84a0-5e74b1558002",
            "backupstoragetotal": 0,
            "deletedbackup": [
              {
                "backupofferingname": "a",
                "created": "2026-08-23T11:25:46-0300",
                "id": "05c18465-4986-4d85-a119-00a469c35cb7",
                "name": "VM-75e8ffef-9ae6-47fb-bca8-b1724fab8f63-2026-08-24T11:25:45.937948704-03:00",
                "removed": "2026-08-24T11:30:21-0300",
                "virtualmachineid": "75e8ffef-9ae6-47fb-bca8-b1724fab8f63",
                "virtualmachinename": "VM-75e8ffef-9ae6-47fb-bca8-b1724fab8f63",
                "zone": "zone",
                "zoneid": "b25d0ef2-b171-4885-beec-21a27044f07c"
              }
            ],
            "failedbackup": [],
            "sucessfulbackup": []
          }
        ],
        "backupstoragetotal": 0,
        "domain": "c",
        "domainid": "517e680a-dc14-4cac-98ab-5b6a21a2ac36"
      }
    ],
    "enddate": "2026-08-23T11:59:59-0300",
    "providerinfo": [
      {
        "compressionreport": []
      },
      {
        "validationreport": []
      }
    ],
    "schedulereport": [],
    "startdate": "2026-08-01T00:00:00-0300"
  }
}

Comment thread engine/schema/src/main/java/com/cloud/upgrade/dao/Upgrade42020to42030.java Outdated
Comment thread server/src/main/java/org/apache/cloudstack/backup/BackupReportServiceImpl.java Outdated
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.


public class Upgrade42210to42300 extends DbUpgradeAbstractImpl implements DbUpgrade, DbUpgradeSystemVmTemplate {

// This must be moved to the new upgrade class when 4.23 is released

@abh1sar abh1sar Sep 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update/resolve conflicts and fix the upgrade path

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the upgrade path to 24 is not created still. #14033. I'll update again when the PR is merged

@abh1sar abh1sar moved this from Ready to conflict/waiting in CloudStack Testing Sep 7, 2026
@JoaoJandre

Copy link
Copy Markdown
Contributor Author

@JoaoJandre I ran some tests, overall LGTM, just a few minor nits

If a backup was created within the gathered period, but has been removed after, it appears as a deleted backup and shows 0 total storage. Given that the deletion happened after the period, it should show as a succesful backup. Same for a failed backup, where the failure reason is not showed.
API return

Should be fixed now, can you test it?

@JoaoJandre
JoaoJandre force-pushed the add-backup-report-feature branch from aaaf43a to 56c14a8 Compare September 18, 2026 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: conflict/waiting

Development

Successfully merging this pull request may close these issues.

5 participants