A multi-tenant database backup platform for MySQL, PostgreSQL and MongoDB — with encrypted credentials, cloud storage, and cron scheduling.
BitbyBit backs up other people's databases. Each user signs in, registers connection profiles for the databases they own, and runs backups on demand or on a cron schedule. Backups are compressed, checksummed, and pushed to local disk or a cloud bucket — and verified against that checksum before they are ever restored.
| 🗄️ Three engines | MySQL, PostgreSQL and MongoDB, via mysqldump, pg_dump and mongodump |
| ☁️ Four destinations | Local disk, AWS S3, Azure Blob Storage, Google Cloud Storage |
| 🔐 Encrypted at rest | Target-database passwords stored with AES-256-GCM, never in plaintext |
| ⏰ Cron scheduling | Quartz expressions per profile, replayed from the database on restart |
| ✅ Verified restores | SHA-256 recorded at backup time and checked before restore |
| 📋 Job history | Every run recorded with status, size, duration, checksum and error detail |
| 👥 Multi-tenant | Users only ever see their own profiles, jobs and schedules |
Prerequisites: JDK 21, a MySQL 8 server for the platform's own metadata, and the client binaries for whichever databases you intend to back up (
mysqldump,pg_dump,mongodump). No Maven install needed — the wrapper is included.
1. Create the platform database
mysql -u root -p < BitbyBit/docs/schema.sqlCaution
This script begins with DROP DATABASE IF EXISTS bitbybit_platform. It is a clean-slate
installer — running it again destroys every registered profile, job record and schedule.
2. Supply your secrets
Create BitbyBit/src/main/resources/application-secrets.properties. This file is
gitignored and is the only place credentials belong:
spring.datasource.username=your_mysql_user
spring.datasource.password=your_mysql_password
# Exactly 32 characters. See the warning under Security.
app.encryption.key=change_me_to_32_characters_long!
# Optional — only needed for the cloud destinations you actually use
aws.s3.access-key=
aws.s3.secret-key=
azure.blob.connection-string=
gcp.storage.credentials-json-path=Environment variables work too (SPRING_DATASOURCE_USERNAME, APP_ENCRYPTION_KEY, …).
Startup fails with a clear message if neither source supplies them.
3. Point at your client binaries
Edit BitbyBit/src/main/resources/config.properties if your tools aren't in the default
install locations. Include the trailing separator:
mysql.bin=C:\\Program Files\\MySQL\\MySQL Server 8.0\\bin\\4. Run
cd BitbyBit && ./mvnw.cmd spring-boot:runOn macOS or Linux use ./mvnw spring-boot:run.
This starts the interactive terminal UI. To run headless as a server instead — for the
REST API once it lands (Phase 4) — start with the web profile, which suppresses the CLI
so it doesn't hold stdin:
cd BitbyBit && ./mvnw.cmd spring-boot:run -Dspring-boot.run.profiles=webBitbyBit is an interactive terminal application. The menus nest in the order you'd actually work:
Sign up / Log in
└── Pick a database flavor MySQL · MongoDB · PostgreSQL
└── Pick a connection profile (or add one)
├── Test Connection
├── Backup ─────────────► Local · S3 · Azure · GCS
├── Restore ────────────► browse local or cloud backups
├── View Backup History
└── Manage Schedules ───► list · add cron · enable/disable
Cron expression examples
Schedules use Quartz syntax, which has a seconds field ahead of the familiar five:
| Expression | Meaning |
|---|---|
*/10 * * * * ? |
Every 10 seconds — handy for testing |
0 0/5 * * * ? |
Every 5 minutes |
0 0 2 * * ? |
Daily at 2:00 AM |
0 30 3 ? * SUN |
Sundays at 3:30 AM |
Expressions are validated before the schedule is saved.
| Concern | How it's handled |
|---|---|
| User passwords | BCrypt, cost factor 12 |
| Brute force | Account locks for 15 minutes after 5 failed attempts |
| Target DB passwords | AES-256-GCM with a random IV per record; decrypted only in memory |
| Credentials on the command line | MySQL uses an option file, Postgres uses PGPASSWORD — neither appears in ps |
| Backup integrity | SHA-256 recorded per job, verified before restore; mismatches require explicit confirmation |
| Untrusted archives | Restore sources are validated and zip expansion is capped |
| Secrets in git | No credential lives in a version-controlled file |
Warning
app.encryption.key cannot be changed casually. Every connection_profiles row is
encrypted with it. Rotating it requires decrypting with the old key and re-encrypting
with the new one, or re-entering every stored database password.
Note
Known gap: MongoDB's mongodump accepts a password only as a command-line
argument, so it is briefly visible in the process list on shared hosts. MySQL and
PostgreSQL do not have this exposure.
Package layout
com.Database.BitbyBit
├── auth/ Sign-up, login, lockout, Bearer-token sessions
├── dto/ Request/response shapes — no controller returns a JPA entity
├── web/ REST controllers (auth, profiles, backups)
├── model/ 10 JPA entities
├── repository/ Spring Data JPA interfaces
├── service/ Encryption, connection profiles, the backup orchestrator, scheduling
├── strategy/ Per-engine connect / backup / restore
├── storage/ StorageProvider: local, S3, Azure, GCS
├── scheduler/ Quartz job that runs a scheduled backup
├── config/ Security filter chain, cloud clients, binary paths, executor pool
├── util/ Archive, checksum and validation helpers
└── runner/ The interactive CLI (default run mode; suppressed under --spring.profiles.active=web)
How a backup runs
InteractiveCliMenu / BackupQuartzJob / (Phase 4) BackupController
│ user or scheduler picks profile + destination
▼
BackupOrchestrator ── the one path all three share
│
├── resolveDestination() ── bucket/region defaults per storage type
▼
BackupJobService.startBackupJob() ── row written, status RUNNING
│
▼
DatabaseStrategy (MYSQL | POSTGRESQL | MONGODB)
│ ProcessBuilder → mysqldump / pg_dump / mongodump
│ (submitBackup(): runs on a bounded executor — caller gets the job id immediately)
▼
BackupArchiveUtils ── zip, then SHA-256
│
▼
StorageProvider.uploadFile() ── skipped for local-only
│
▼
BackupJobService.completeBackupJob() ── size, checksum, duration, COMPLETED
Strategies are resolved by name — Spring injects a Map<String, DatabaseStrategy> keyed
by @Component("MYSQL") and friends — so adding an engine means adding one class, not
editing a switch. BackupOrchestrator exists so this whole sequence lives in exactly one
place rather than being reimplemented per caller — see
docs/frontend_plan.md §0.1 for why that mattered.
Database schema
A single MySQL database, bitbybit_platform, with 10 tables. The schema is managed
manually via docs/schema.sql; Hibernate runs with
ddl-auto=validate and never alters it.
users (1) ──┬── (N) user_services
├── (N) connection_profiles ── (N) backup_jobs ── (N) backup_manifests
├── (N) storage_profiles ── (N) backup_jobs
├── (N) scheduled_backups
├── (N) user_sessions
├── (N) user_audit_log
└── (N) password_reset_tokens
Four of these tables (user_services, user_audit_log, password_reset_tokens, and
backup_manifests) are scaffolding for features not yet built — see
Roadmap. user_sessions is now active, backing Bearer-token auth. Full
column-by-column reference in
docs/database_design.md.
cd BitbyBit && ./mvnw.cmd test| Suite | Needs a database |
|---|---|
EncryptionServiceTest, UserServiceTest, SessionServiceTest, BackupOrchestratorTest, ConnectionProfileServiceTest, BackupArchiveUtilsTest — pure unit tests |
no |
AuthControllerIntegrationTest, ProfileControllerIntegrationTest, BackupControllerIntegrationTest — real HTTP through the full security filter chain |
yes |
BitbyBitApplicationTests, WebProfileContextTest — full context loads, all entities validated |
yes |
110 tests total. The integration suites sign up and log in for real, then use the
returned Bearer token against protected endpoints — auth isn't mocked. See
docs/architecture.md for what each covers,
including why one test deliberately skips @Transactional — it's what caught a real
LazyInitializationException bug that the other 109 tests structurally could not.
| Phase | Status | |
|---|---|---|
| 1 | ✅ | Interactive CLI, Spring Boot foundation |
| 1.5 | ✅ | 10-table schema, BCrypt, AES-256, lockout |
| 2 | ✅ | MySQL, PostgreSQL and MongoDB backup/restore engines |
| 3 | ✅ | Cloud storage and Quartz scheduling |
| 3.5 | ✅ | Secrets out of version control, verified restores, test suite |
| 4.0–4.2 | ✅ | Backup orchestrator, Bearer-token auth, profiles + backups REST API |
| 4.3 | ✅ | React frontend shell — verified live in a browser |
| 4.4+ | 🔲 | Backup/restore UI, schedules UI, audit, dashboard |
Full detail on Phase 4 — API surface, security checklist, screen-by-screen status — lives
in docs/frontend_plan.md, which is kept current as
the phase progresses rather than duplicated here.
Designed but not yet built — the tables and entities exist, nothing writes to them:
backup_manifests→ selective, table-level restoreuser_audit_log→ immutable action trailuser_services→ per-user feature entitlementspassword_reset_tokens→ password recovery
(user_sessions is no longer in this list — Phase 4.1 wired it up for opaque
Bearer-token issuance, validation and revocation.)
Also worth building: a retention policy (nothing currently deletes old backups), per-user cloud credentials (the buckets are global today), encryption of the backup archive itself, and failure notifications for unattended scheduled runs.
| Document | Contents |
|---|---|
docs/architecture.md |
Package structure, security model, configuration reference |
docs/database_design.md |
Every table and column, with the reasoning |
docs/schema.sql |
Master DDL — run this to create the database |
docs/project_log.md |
Change history: what changed, and why |
docs/frontend_plan.md |
Phase 4 — REST API and React frontend: build order, screen-by-screen status, security checklist |
"No suitable driver" or connection tests fail
Confirm the target server is reachable and the profile's host, port and credentials are right — "Test Connection" performs a real authenticated handshake for all three engines, so a failure here means a backup would also fail.
Backup fails immediately with a non-zero exit code
The dump tool couldn't be launched or rejected the connection. Check mysql.bin /
postgres.bin / mongodb.bin in config.properties point at the directory containing
the executable, trailing separator included. The tool's own stderr is captured in
logs/backup_error_<timestamp>.log.
Application won't start
Most often a missing secret. Confirm application-secrets.properties exists and defines
spring.datasource.username, spring.datasource.password and app.encryption.key — the
last must be present, as there is deliberately no default. Otherwise, check that
schema.sql has been run: ddl-auto=validate refuses to start if a table is missing.
A schedule didn't fire
Quartz uses an in-memory job store; triggers are re-registered from scheduled_backups
at startup, so the application must be running for a schedule to fire. Confirm the
schedule is enabled via Manage Backup Schedules → List, which shows the next run time.