Skip to content
SakthiQPublic

About

A multi-tenant database backup platform for MySQL, PostgreSQL and MongoDB — with encrypted credentials, cloud storage, and cron scheduling.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

13 Commits

Folders and files

Repository files navigation

BitbyBit

A multi-tenant database backup platform for MySQL, PostgreSQL and MongoDB — with encrypted credentials, cloud storage, and cron scheduling.

Java Spring Boot MySQL Build


What it does

BitbyBit backs up other people's databases. Each user signs in, registers connection profiles for the databases they own, and runs backups on demand or on a cron schedule. Backups are compressed, checksummed, and pushed to local disk or a cloud bucket — and verified against that checksum before they are ever restored.

🗄️ Three engines MySQL, PostgreSQL and MongoDB, via mysqldump, pg_dump and mongodump
☁️ Four destinations Local disk, AWS S3, Azure Blob Storage, Google Cloud Storage
🔐 Encrypted at rest Target-database passwords stored with AES-256-GCM, never in plaintext
⏰ Cron scheduling Quartz expressions per profile, replayed from the database on restart
✅ Verified restores SHA-256 recorded at backup time and checked before restore
📋 Job history Every run recorded with status, size, duration, checksum and error detail
👥 Multi-tenant Users only ever see their own profiles, jobs and schedules

Quick start

Prerequisites: JDK 21, a MySQL 8 server for the platform's own metadata, and the client binaries for whichever databases you intend to back up (mysqldump, pg_dump, mongodump). No Maven install needed — the wrapper is included.

1. Create the platform database

mysql -u root -p < BitbyBit/docs/schema.sql

Caution

This script begins with DROP DATABASE IF EXISTS bitbybit_platform. It is a clean-slate installer — running it again destroys every registered profile, job record and schedule.

2. Supply your secrets

Create BitbyBit/src/main/resources/application-secrets.properties. This file is gitignored and is the only place credentials belong:

spring.datasource.username=your_mysql_user
spring.datasource.password=your_mysql_password

# Exactly 32 characters. See the warning under Security.
app.encryption.key=change_me_to_32_characters_long!

# Optional — only needed for the cloud destinations you actually use
aws.s3.access-key=
aws.s3.secret-key=
azure.blob.connection-string=
gcp.storage.credentials-json-path=

Environment variables work too (SPRING_DATASOURCE_USERNAME, APP_ENCRYPTION_KEY, …). Startup fails with a clear message if neither source supplies them.

3. Point at your client binaries

Edit BitbyBit/src/main/resources/config.properties if your tools aren't in the default install locations. Include the trailing separator:

mysql.bin=C:\\Program Files\\MySQL\\MySQL Server 8.0\\bin\\

4. Run

cd BitbyBit && ./mvnw.cmd spring-boot:run

On macOS or Linux use ./mvnw spring-boot:run.

This starts the interactive terminal UI. To run headless as a server instead — for the REST API once it lands (Phase 4) — start with the web profile, which suppresses the CLI so it doesn't hold stdin:

cd BitbyBit && ./mvnw.cmd spring-boot:run -Dspring-boot.run.profiles=web

Using it

BitbyBit is an interactive terminal application. The menus nest in the order you'd actually work:

Sign up / Log in
   └── Pick a database flavor          MySQL · MongoDB · PostgreSQL
         └── Pick a connection profile  (or add one)
               ├── Test Connection
               ├── Backup ─────────────► Local · S3 · Azure · GCS
               ├── Restore ────────────► browse local or cloud backups
               ├── View Backup History
               └── Manage Schedules ───► list · add cron · enable/disable
Cron expression examples

Schedules use Quartz syntax, which has a seconds field ahead of the familiar five:

Expression Meaning
*/10 * * * * ? Every 10 seconds — handy for testing
0 0/5 * * * ? Every 5 minutes
0 0 2 * * ? Daily at 2:00 AM
0 30 3 ? * SUN Sundays at 3:30 AM

Expressions are validated before the schedule is saved.


Security

Concern How it's handled
User passwords BCrypt, cost factor 12
Brute force Account locks for 15 minutes after 5 failed attempts
Target DB passwords AES-256-GCM with a random IV per record; decrypted only in memory
Credentials on the command line MySQL uses an option file, Postgres uses PGPASSWORD — neither appears in ps
Backup integrity SHA-256 recorded per job, verified before restore; mismatches require explicit confirmation
Untrusted archives Restore sources are validated and zip expansion is capped
Secrets in git No credential lives in a version-controlled file

Warning

app.encryption.key cannot be changed casually. Every connection_profiles row is encrypted with it. Rotating it requires decrypting with the old key and re-encrypting with the new one, or re-entering every stored database password.

Note

Known gap: MongoDB's mongodump accepts a password only as a command-line argument, so it is briefly visible in the process list on shared hosts. MySQL and PostgreSQL do not have this exposure.


Architecture

Package layout
com.Database.BitbyBit
├── auth/         Sign-up, login, lockout, Bearer-token sessions
├── dto/          Request/response shapes — no controller returns a JPA entity
├── web/          REST controllers (auth, profiles, backups)
├── model/        10 JPA entities
├── repository/   Spring Data JPA interfaces
├── service/      Encryption, connection profiles, the backup orchestrator, scheduling
├── strategy/     Per-engine connect / backup / restore
├── storage/      StorageProvider: local, S3, Azure, GCS
├── scheduler/    Quartz job that runs a scheduled backup
├── config/       Security filter chain, cloud clients, binary paths, executor pool
├── util/         Archive, checksum and validation helpers
└── runner/       The interactive CLI (default run mode; suppressed under --spring.profiles.active=web)
How a backup runs
InteractiveCliMenu / BackupQuartzJob / (Phase 4) BackupController
   │  user or scheduler picks profile + destination
   ▼
BackupOrchestrator                          ── the one path all three share
   │
   ├── resolveDestination()                 ── bucket/region defaults per storage type
   ▼
BackupJobService.startBackupJob()          ── row written, status RUNNING
   │
   ▼
DatabaseStrategy (MYSQL | POSTGRESQL | MONGODB)
   │  ProcessBuilder → mysqldump / pg_dump / mongodump
   │  (submitBackup(): runs on a bounded executor — caller gets the job id immediately)
   ▼
BackupArchiveUtils                          ── zip, then SHA-256
   │
   ▼
StorageProvider.uploadFile()                ── skipped for local-only
   │
   ▼
BackupJobService.completeBackupJob()        ── size, checksum, duration, COMPLETED

Strategies are resolved by name — Spring injects a Map<String, DatabaseStrategy> keyed by @Component("MYSQL") and friends — so adding an engine means adding one class, not editing a switch. BackupOrchestrator exists so this whole sequence lives in exactly one place rather than being reimplemented per caller — see docs/frontend_plan.md §0.1 for why that mattered.

Database schema

A single MySQL database, bitbybit_platform, with 10 tables. The schema is managed manually via docs/schema.sql; Hibernate runs with ddl-auto=validate and never alters it.

users (1) ──┬── (N) user_services
            ├── (N) connection_profiles ── (N) backup_jobs ── (N) backup_manifests
            ├── (N) storage_profiles     ── (N) backup_jobs
            ├── (N) scheduled_backups
            ├── (N) user_sessions
            ├── (N) user_audit_log
            └── (N) password_reset_tokens

Four of these tables (user_services, user_audit_log, password_reset_tokens, and backup_manifests) are scaffolding for features not yet built — see Roadmap. user_sessions is now active, backing Bearer-token auth. Full column-by-column reference in docs/database_design.md.


Testing

cd BitbyBit && ./mvnw.cmd test
Suite Needs a database
EncryptionServiceTest, UserServiceTest, SessionServiceTest, BackupOrchestratorTest, ConnectionProfileServiceTest, BackupArchiveUtilsTest — pure unit tests no
AuthControllerIntegrationTest, ProfileControllerIntegrationTest, BackupControllerIntegrationTest — real HTTP through the full security filter chain yes
BitbyBitApplicationTests, WebProfileContextTest — full context loads, all entities validated yes

110 tests total. The integration suites sign up and log in for real, then use the returned Bearer token against protected endpoints — auth isn't mocked. See docs/architecture.md for what each covers, including why one test deliberately skips @Transactional — it's what caught a real LazyInitializationException bug that the other 109 tests structurally could not.


Roadmap

Phase Status
1 ✅ Interactive CLI, Spring Boot foundation
1.5 ✅ 10-table schema, BCrypt, AES-256, lockout
2 ✅ MySQL, PostgreSQL and MongoDB backup/restore engines
3 ✅ Cloud storage and Quartz scheduling
3.5 ✅ Secrets out of version control, verified restores, test suite
4.0–4.2 ✅ Backup orchestrator, Bearer-token auth, profiles + backups REST API
4.3 ✅ React frontend shell — verified live in a browser
4.4+ 🔲 Backup/restore UI, schedules UI, audit, dashboard

Full detail on Phase 4 — API surface, security checklist, screen-by-screen status — lives in docs/frontend_plan.md, which is kept current as the phase progresses rather than duplicated here.

Designed but not yet built — the tables and entities exist, nothing writes to them:

  • backup_manifests → selective, table-level restore
  • user_audit_log → immutable action trail
  • user_services → per-user feature entitlements
  • password_reset_tokens → password recovery

(user_sessions is no longer in this list — Phase 4.1 wired it up for opaque Bearer-token issuance, validation and revocation.)

Also worth building: a retention policy (nothing currently deletes old backups), per-user cloud credentials (the buckets are global today), encryption of the backup archive itself, and failure notifications for unattended scheduled runs.


Documentation

Document Contents
docs/architecture.md Package structure, security model, configuration reference
docs/database_design.md Every table and column, with the reasoning
docs/schema.sql Master DDL — run this to create the database
docs/project_log.md Change history: what changed, and why
docs/frontend_plan.md Phase 4 — REST API and React frontend: build order, screen-by-screen status, security checklist

Troubleshooting

"No suitable driver" or connection tests fail

Confirm the target server is reachable and the profile's host, port and credentials are right — "Test Connection" performs a real authenticated handshake for all three engines, so a failure here means a backup would also fail.

Backup fails immediately with a non-zero exit code

The dump tool couldn't be launched or rejected the connection. Check mysql.bin / postgres.bin / mongodb.bin in config.properties point at the directory containing the executable, trailing separator included. The tool's own stderr is captured in logs/backup_error_<timestamp>.log.

Application won't start

Most often a missing secret. Confirm application-secrets.properties exists and defines spring.datasource.username, spring.datasource.password and app.encryption.key — the last must be present, as there is deliberately no default. Otherwise, check that schema.sql has been run: ddl-auto=validate refuses to start if a table is missing.

A schedule didn't fire

Quartz uses an in-memory job store; triggers are re-registered from scheduled_backups at startup, so the application must be running for a schedule to fire. Confirm the schedule is enabled via Manage Backup Schedules → List, which shows the next run time.

About

A multi-tenant database backup platform for MySQL, PostgreSQL and MongoDB — with encrypted credentials, cloud storage, and cron scheduling.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages