MongoDB Realm Multi-Threading in Swift

MongoDB Realm is the leading offline-first synchronizing platform for developing collaborative cross-platform applications that do not require continuous connectivity. It is a real-time client side database that allows mobile or desktop apps to synchronize data to a MongoDB Atlas cluster. The Beta version of MongoDB Realm was released in June 2020, and has steadily been improving ever since. This platform is truly transformational because it is ushering in a whole new class of computing — notably collaborative software — where multiple users access shared data in a structured architected fashion.

At Cosync, Inc we are building a number of collaborative extensions on top of the MongoDB Realm real-time database. In the process of implementing our products, we have had to resort to multi-threading to optimize performance and increase responsiveness. Fortunately, MongoDB Realm is designed with multi-threading in mind, and handles concurrency very well. The same cannot be said for Apple’s new Swift language, which tends to hide traditional multi-threading constructs behind the callback structure of the language itself. The rules for multi-threaded programming within MongoDB Realm are fairly explicit, and any deviation from those rules generally results in a runtime exception. This article aims to elucidate what those rules are and how to enforce conformance to them within the Swift programming language.

In MongoDB Realm, a specific client side implementation opens a Realm by specifying a partition key value. All objects with the same partition key value are said to belong to the same Realm. All object collections in Atlas that are synced through MongoDB Realm specify a partition property. The mapping between Realms and MongoDB Atlas is explained in more depth in the Realm documentation section on how to Partition Atlas Data into Realms.

Insofar as multi-threading with MongoDB Realm on iOS is concerned, there are only three rules that the programmer needs to remember:

1. A call to Realm asyncOpen() always opens a Realm on the primary thread, regardless of what thread it was called on.
2. When opening a Realm on a secondary thread, the programmer must use a synchronized open “try! Realm(configuration: configuration)” to open the Realm.
3. If a Realm is opened on a secondary thread, all objects read and written to that Realm must take place on that same secondary thread.

For a more in depth review, the threading model for MongoDB Realm is documented here. The thing to remember about Realm is the Las Vegas motto — whatever happens on a thread, must stay on that thread.

The Swift programming language extensively uses function callbacks as a control mechanism for handling asynchronous coding. Although Swift is not a functional language per se, it has adopted passing functions as parameters as one of its best practices. In the case of blocking functions, the callbacks often occur on secondary threads that are different from the primary thread upon which they were called. In other words, Swift tends to optimize your code under the hood for multi-threading. Furthermore, the recommended multi-threading technique in Swift is to use Grand Central Dispatch or Operation Queues as a means of offloading work onto secondary threads. The problem is that both of these mechanisms employ thread pools and there is no guarantee as to which secondary thread is assigned the background task. In the case of Operation Queues, this problem is not even solved by setting the maxConcurrentOperationCount to 1. When it comes to multi-threading, Swift is very much like a fully automatic car that prevents a racing driver from having full control over the gears.

In a typical MongoDB Realm application, the programmer opens Realms on the primary thread because that is where the user interface is operating out of. The data garnished from these Realms is used to populate the user interface. Since MongoDB Realm requires data access to be done on the same thread upon which the tread was opened, it follows that this must be done on the primary thread exclusively.

For this reason, one of the most commonly used constructs within a Swift Realm application is the DispatchQueue.main.async call to force execution back on to the main thread.

DispatchQueue.main.async {
    self.initRealms(onCompletion: { (err) in
        if err==nil {
            UploadManager.shared.setup()
        }
        completion(err)
    })
}

Although the Swift Dispatch Queue has a mechanism for forcing execution upon the primary thread, it provides no such mechanism for forcing execution on a specific secondary thread in particular. The Dispatch Queue mechanism is somewhat military in its orientation. There is one officer in charge (the primary thread), and there are a number of interchangeable privates (the secondary threads) who carry out whatever tasks the officer has ordered. Unfortunately, the secondary threads because of their subordinate nature are not even given the privilege of being distinguished from one another. It is the operating system that decides which private cleans the latrine, and it could be a different soldier depending on the day of the week.

In most Swift programming scenarios the automatic transmission approach to multi-threading is actually a blessing. The programmer does not have to worry about thread synchronization: specifically semaphores, mutually exclusive critical sections, and triggers. Grand Central Dispatch handles all the load balancing between secondary threads auto-magically. On the downside, it makes handling multi-threading with MongoDB Realm particularly problematic — given rule 3 listed above.

While implementing our Cosync Storage product that bridges the gap between MongoDB Realm and Amazon S3 for image and video assets, we discovered a workaround to this dilemma. In our toolkit, we needed a secondary thread that would read upload requests from Realm, process them, and upload image and video assets up to Amazon S3. After uploading the assets, it would then write an asset object into Realm with the associated URLs of the image cuts of that asset. The Cosync Storage product will be available to MongoDB Realm programmers later this month.

Given rule 3 listed above, the first thing that we needed to do was explicitly create a thread to handle the uploading tasks. To this end, our Upload Manager declares a private member variable to store the thread object.

private var uploadThread: Thread?

The next thing we do is to create an entry point for the thread that creates a runloop thread, so as to handle outside requests from the primary thread or block callbacks.

@objc func uploadThreadEntryPoint(uid: String) {
    autoreleasepool {
        Thread.current.name = "CosyncUploadThread_\(uid)"
        let runLoop = RunLoop.current
        runLoop.add(NSMachPort(), forMode:
                RunLoop.Mode.default)
        runLoop.run()
    }
}

We then create a function to terminate the thread, that is executed within the thread itself.

@objc func uploadThreadExit() {
    Thread.exit()
}

Lastly, we add code in the Upload Manager setup function to actually create the thread and launch the setup code.

if let uid = RealmManager.shared.currentUserId {
    if self.uploadThread==nil {
        self.uploadThread = Thread(target: self, selector:
             #selector(uploadThreadEntryPoint(uid:)), object: uid)
        self.uploadThread!.start()
    }
    if let uploadThread = self.uploadThread {
        perform(#selector(setupBackground), 
            on: uploadThread, 
            with: nil, 
            waitUntilDone: false, 
            modes: [RunLoop.Mode.common.rawValue])
    }
}

The perform(#selector(setupBackground) function will call the setupBackground member function on the uploadThread that was created above.

@objc func setupBackground() -> Void {
    if  let user = RealmManager.shared.app.currentUser,
        let uid = RealmManager.shared.currentUserId,
        let sessionId = AssetManager.shared.sessionId {
        self.userRealm = try! Realm(configuration:
             user.configuration(partitionValue: uid))
        if let realm = self.userRealm {
            let results = realm.objects(CosyncAssetUpload.self)
                .filter("uid == '\(uid)' && sessionId=='\(sessionId)' && status=='initialized'")
            self.notificationToken = results.observe { [self] (changes: RealmCollectionChange) in
                switch changes {
                case .initial:
                    for assetUpload in results {
                        self.uploadAsset(assetUpload: assetUpload)
                    }
                case .update( let results, _, _, _):
                    for assetUpload in results {
                        self.uploadAsset(assetUpload: assetUpload)
                    }
                case .error(let error):
                    // An error occurred while opening the Realm
                    fatalError("\(error)")
                }
            }
        }
    }
}

The @objc attribute is needed because the Thread support for Swift is actually implemented using the ObjectiveC runtime. As noted above, the Realm is opened on the background upload thread using a sync open, and not an async open. This is critical.

Terminating the thread from the primary thread is done by calling the following:

if let uploadThread = self.uploadThread {
    perform(#selector(uploadThreadExit), 
            on: uploadThread, 
            with: nil, 
            waitUntilDone: false, 
            modes: [RunLoop.Mode.common.rawValue])
    self.uploadThread = nil
}

In conclusion, MongoDB Realm does support multi-threading very well, but the important thing is to keep Realm access in a secondary thread confined to that secondary thread. It should be noted that Realm objects cannot be shared between thread — even for reading! In order to enforce confinement to the secondary thread, the programmer should use the Thread object directly rather than rely on Grand Central Dispatch or Operation Queues.

To all my fellow developers, happy Realming.