PufferDB

PufferDB is a key-value storage powered by Protocol Buffers (aka Protobuf) and Coroutines.

The purpose of this library is to provide an efficient, reliable and Android independent storage.

Why Android independent? The SharedPreferences and many great third-party libraries (like Paper and MMKV) requires the Android Context to work. But if you are like me and want a kotlin-only data module (following the principles of Clean Architecture), this library is for you!

This project started as a library module in one of my personal projects, but I decided to open source it and add more features for general use. Hope you like!

About Protobuf

Protocol Buffers are a language-neutral, platform-neutral extensible mechanism for serializing structured data. Compared to JSON, Protobuf files are smaller and faster to read/write because the data is stored in an efficient binary format.

Features

  • Fast
  • Works on Android and JVM
  • Simple API
  • Thread-safe
  • Wrappers for Coroutines and RxJava

Supported types

So far, PufferDB supports the following types:

  • [x] Double and List<Double>
  • [x] Float and List<Float>
  • [x] Int and List<Int>
  • [x] Long and List<Long>
  • [x] Boolean and List<Boolean>
  • [x] String and List<String>

Getting Started

Import to your project

  1. Add the JitPack repository in your root build.gradle at the end of repositories:
allprojects {
    repositories {
        maven { url 'https://jitpack.io' }
    }
}
  1. Next, add the desired dependencies to your module:
dependencies {
    // Core library
    implementation "com.github.adrielcafe.pufferdb:core:$currentVersion"

    // Android helper
    implementation "com.github.adrielcafe.pufferdb:android:$currentVersion"

    // Coroutines wrapper
    implementation "com.github.adrielcafe.pufferdb:coroutines:$currentVersion"

    // RxJava wrapper
    implementation "com.github.adrielcafe.pufferdb:rxjava:$currentVersion"
}

Current version: JitPack

Platform compatibility

core android coroutines rxjava
Android
JVM

Core

As the name suggests, Core is a standalone module and all other modules depends on it.

To create a new Puffer instance you must tell which file to use.

val pufferFile = File("path/to/puffer/file")
val puffer = PufferDB.with(pufferFile)

If you are on Android, I recommend to use the Context.filesDir as the parent folder. If you want to save in the external storage remember to ask for write permission first.

Its API is similar to SharedPreferences:

puffer.apply {
    val myValue = get<String>("myKey")
    val myValueWithDefault = get("myKey", "defaultValue")
    
    put("myOtherKey", 123)

    getKeys().forEach { key ->
        // ...
    }

    if(contains("myKey")){
        // ...
    }

    remove("myOtherKey")

    removeAll()
}

But unlike SharedPreferences, there's no apply() or commit(). Changes are saved asynchronously every time a write operation (put(), remove() and removeAll()) happens.

Threading

PufferDB uses a ConcurrentHashMap to manage a thread-safe in-memory cache for fast read and write operations.

Changes are saved asynchronously with the help of a Conflated Channel (to save the most recent state in a race condition) and Mutex locker (to prevent simultaneous writes).

It is possible to run the API methods on the Android Main Thread, but you should avoid that. You can use one of the wrapper modules or built in extension functions for that.

Android

The Android module contains an AndroidPufferDB helper class:

class MyApp : Application() {

    override fun onCreate() {
        super.onCreate()
        // Init the PufferDB when your app starts
        AndroidPufferDB.init(this)
    }
}

// Now you can use it anywhere in your app
class MyActivity : AppCompatActivity() {

    // Returns a default Puffer instance, the file is located on Context.filesDir
    val corePuffer = AndroidPufferDB.withDefault()

    // Returns a File that should be used to create a Puffer instance
    val pufferFile = AndroidPufferDB.getInternalFile("my.db")
    val coroutinePuffer = CoroutinePufferDB.with(pufferFile)
}

Coroutines

The Coroutines module contains a CoroutinePufferDB wrapper class and some useful extension functions:

val pufferFile = File("path/to/puffer/file")
val puffer = CoroutinePufferDB.with(pufferFile)

// All methods are suspend functions that runs on Dispatchers.IO context
launch {
    puffer.apply {
        val myValue = get<String>("myKey")
        val myValueWithDefault = get("myKey", "defaultValue")
        
        put("myOtherKey", 123)

        getKeys().forEach { key ->
            // ...
        }

        if(contains("myKey")){
            // ...
        }

        remove("myOtherKey")

        removeAll()
    }
}

If you don't want to use this wrapper class, there's some built in extension functions that can be used with the Core module:

val pufferFile = File("path/to/puffer/file")
val puffer = PufferDB.with(pufferFile) // <- Note that we're using the Core PufferDB

launch {
    puffer.apply {
        val myValue = getSuspend<String>("myKey")

        val myValue = getAsync<String>("myKey").await()
        
        // You can use your custom coroutine context...
        putSuspend("myOtherKey", 123, Dispatchers.Unconfined)

        // ... And your custom coroutine scope
        putAsync("myOtherKey", 123, myActivityScope).await()
    }
}

RxJava

The RxJava module contains a RxPufferDB wrapper class and some useful extension functions:

val pufferFile = File("path/to/puffer/file")
val puffer = RxPufferDB.with(pufferFile)

puffer.apply {
    // Some methods returns Single<T>...
    get<String>("myKey") // OR get("myKey", "defaultValue")
        .subscribeOn(Schedulers.io())
        .observeOn(AndroidSchedulers.mainThread())
        .subscribe { myValue ->
            // ...
        }

    // ... And others returns Completable
    put("myOtherKey", 123)
        // ...
        .subscribe {
            // ...
        }

    getKeys()
        // ...
        .subscribe { keys ->
            // ...
        }

    contains("myKey")
        // ...
        .subscribe { contains ->
            // ...
        }

    remove("myOtherKey")
        // ...
        .subscribe {
            // ...
        }

    removeAll()
        // ...
        .subscribe {
            // ...
        }
}

Like the Coroutines module, the RxJava module also provides some useful built in extension functions that can be used with the Core module:

val pufferFile = File("path/to/puffer/file")
val puffer = PufferDB.with(pufferFile) // <- Note that we're using the Core PufferDB

puffer.apply {
    val myValue = getSingle<String>("myKey").blockingGet()

    putCompletable("myOtherKey", 123).blockingAwait()

    getKeysObservable().blockingSubscribe { keys ->
        // ...
    }
}

GitHub