A key-value storage engine library

• Install clang-format-11.
• Run ./init.sh to install pre-commit hook for formatting the code.

• cmake version >= 3.16.*
• gcc version >= 10.2.0 (This is needed for proper c++20 support)

To use ProjectDb, follow the following steps:

• Clone this repo into some directory (<root> for example)

• Run make init_build && make projectdb

  This will build a libprojectdb.a static library, which needs to be linked against the application that uses ProjectDb.

• Set the c++ standard with -std=gnu++20

• Add <root>/include to the include path, and #include <projectdb/projectdb.h> in places where ProjectDb is used

• Add libprojectdb.a to the linker path, and add -lprojectdb -lpthread in at the end of link line

The directory structure for this example is as following:

.
..
ProjectDbTest
    main.cpp .............> This is the application that's trying to use ProjectDb
ProjectDb ................> This is cloned ProjectDb repo 
    cmake-build-release
        libprojectdb.a ...> This is generated after running "make init_build && make projectdb"

main.cpp, which represents the application, is as following:

// main.cpp

#include "projectdb/projectdb.h"

int main() {
    // A config file template can be found in ./config/config.template
    // To use a customized config, create object with:
    // ProjectDb::ProjectDb db {"<config_file_path>"};
    projectdb::ProjectDb db;
    db.set("Hello", "World!");
    return 0;
}

The command used to build main.cpp is as following:

g++ -std=gnu++20 -I../ProjectDb/include main.cpp -o main \
    -L../ProjectDb/cmake-build-release/ -lprojectdb -lpthread

• The current ProjectDb apis are not thread safe by itself. ProjectDb object needs to be locked by user if the apis are called in a multi-threaded fashion.

• The current implementation doesn't provide multi-process support.

  It is important to note that only one ProjectDb should be created for one DB_FILE_PATH, otherwise, there's a risk of data corruption.

• Build the Docker using docker build .

• Open an interactive bash with current directory mounted to projectdb: docker run -it --rm -v ${PWD}:/projectdb projectdb bash

All the documents can be found in docs/, specifically:

• DesignDocument.pdf: Documents the description of the algorithm implemented, and some design decisions made while implementing.
• Document.pdf: Detailed document about the directory, files, classes and functions implemented.
• Measurements.pdf: Documentation about the benchmarks on the library.
• Presentation.pdf: The slides for the presentation for the project.
• ProjectProposal.pdf: The initial proposal for this project.
• Tutorial.pdf: Documents on how to use the library.

coverage

• pre-commit hook for clang-format: https://github.com/barisione/clang-format-hooks/

• paper for tech used by nosql: https://www.cs.utexas.edu/~rossbach/cs378h/papers/nosql-survey.pdf

• LevelDb: The database used at a single node of BigTable: https://github.com/google/leveldb

• Google big table: http://static.googleusercontent.com/media/research.google.com/en//archive/bigtable-osdi06.pdf

• Designing Data-Intensive Applications: p76 - p79

• Readings on future and promise: https://thispointer.com//c11-multithreading-part-9-stdasync-tutorial-example/

• Code coverage setup: https://github.com/codecov/codecov-action

• Codecov:

  • Supported languate: https://docs.codecov.io/docs/supported-languages
  • Supported xml format: https://docs.codecov.io/docs/supported-report-formats

1. Support range query for read.
2. Add bloom filter.
3. Work on making this distributed.
4. ...

1. Add more unit tests.
2. Add more tests for crash recovery. (Need to investigate the concept of commit and rollback.)
3. Rewrite benchmark code and compare with leveldb.
4. ...

1. Add badge for code coverage.
2. ...