Migrating from Realm Cloud to MongoDB Realm
I have been a Realm Cloud developer since early 2018, shortly after the Realm company introduced its Realm Cloud upgrade to its native object data base. As a mobile developer, I have been using Realm since 2015 to cache all my local data for both iOS and Android development. In my opinion, Realm has been a necessary adjunct for any developer programming cloud based mobile applications, using simple REST APIs. Prior to the introduction of Realm Cloud, I used Google Firebase/Firestore as a server sync solution in conjunction with Realm, which I used for local object caching and storage. This hybrid solution required a significant amount of bridge code between Firebase and Realm, both for the upload of data to the cloud and for responding to notifications along with download of data from the cloud.
The introduction of Realm Cloud greatly simplified this task and provided a unified object model at the local level, with cloud synchronization done automatically in the background. About two years ago, I became convinced that Realm Cloud was the key enabling technology for collaborative mobile applications and was ushering in a whole new category of computation, similar to the PC, web, and mobile revolutions that proceeded it. Although Google Firebase is a good back end as a service, it is severely deficient with respect to client-side data modeling and does not offer an effective offline first solution for mobile app development. In fact, I was so excited about the potential behind Realm Cloud that I founded a startup Cosync, Inc that provides a JWT authentication service for Realm and am working on an engine for building collaborative cross platform appliances on top of Realm.
In April 2019, Realm was acquired by MongoDB — a publicly traded database company in NYC (Nasdaq ticker MDB) — to complement Atlas (their hosted server product) with a state-of-the-art client-side solution. Over the last year, the MongoDB and Realm software teams have engineered a new product called MongoDB Realm that combines the client-side Realm API with the server-side MongoDB Atlas cluster infrastructure. The company launched the new product on June 10th, 2020, at the MongoDB Live conference. MongoDB Realm effectively replaces the older Realm Cloud offering.
MongoDB Realm is a big upgrade and has taken me several weeks both to learn and wrap my head around. In essence, the entire backend for Realm Cloud has now been replaced by the MongoDB Atlas tool set. The new product has preserved the front-end API to Realm Cloud for the most part, but there are some significant conceptual differences with the old product — specifically with the security model and backend processing. Also, the new MongoDB Realm product is still in Beta; some features have yet to be delivered and the documentation is still somewhat rough around the edges, as compared to the documentation for Realm Cloud. Overall, the upgrade is positive and addresses a number of issues with the old product.
Specifically, Realm Cloud did not offer a hosted solution for backend processing. In the past, a developer would have to host their own Node.JS server that would listen for changes to the Realm instance and process the data accordingly. This provided a significant administrative overhead to client-side development, because of the hosting requirement for the Node.JS backend server. With the new MongoDB Realm product, backend processing is done through Stitch functions that are hosted by the Atlas infrastructure. These functions can act as triggers and be attached to any changes to the MongoDB Atlas collections. The developer does not have to be concerned about hosting a separate server on top of AWS or Azure for backend processing.
The old Realm Cloud product was primarily mobile first, as that is the platform from which the original Realm object data base had evolved from. Developing a complementary web application around a Realm Cloud instance was very challenging to say the least. This was perhaps its greatest deficiency with respect to Google Firebase, which offered a unified web/mobile solution, albeit with a very server centric view of the world. Since the new MongoDB Realm cloud service is built on top of Atlas, web developers can use all of MongoDB web tool set to build a web app that accesses the same database as the mobile application that fully synchronizes with the client-side mobile applications.
The old Realm Cloud product provided two approaches to synchronization: full synchronization and query-based synchronization. With full synchronization, all of the data from a Realm was downloaded to the client, updates as they happened would then be synchronized in the background. With full synchronization, a developer would architect a Realm instance with a number of shared read-only Realms along with a private user realm that would be read/write. This provided the most scalable approach. With Query based synchronization, each client would register a query with the server for the Realm data they were interested in — only the data that matched the query would be downloaded to the client. While this approach limited data traffic, it suffered from not being very scalable. Typically, query-based synchronization would burden the server down with too many queries, and performance would degrade after about thirty clients were simultaneously synchronizing to the same Realm. Query based synchronization was fine only for collaborative software scenarios with a small number of simultaneous users. The new MongoDB Realm product only provides one synchronization approach, but it is a lot more scalable.
When starting development on the new MongoDB Realm product, one must first get acquainted with the MongoDB Atlas tool set upon which it is built. Fortunately, MongoDB offers excellent courses to get a developer quickly up to speed, and they are free. At a minimum, a developer should take the MongoDB Basics class at MongoDB University https://university.mongodb.com. This class shows how to set up an Atlas cluster and how to use Compass to navigate a database. This course takes about two days to complete but is well worth the time investment. It is practically impossible to navigate the MongoDB Realm product without some understanding of MongoDB Atlas. In a second step, I would recommend reading the MongoDB Realm documentation at https://docs.mongodb.com/realm and going through the process of building their tutorial application called Task Tracker.
One of the first conceptual changes with the new MongoDB Realm product is that there is no longer a concept of multiple Realms within an overall application Realm Instance. In the old Realm Cloud product, the developer would first create a Realm Instance to support a collaborative application. The developer would then define a user authentication method — nickname, anonymous, JWT, or username/password — for authenticating all the user who would access the shared instance. Users could either be regular users or have admin privilege. Typically, the only admin user was the user who signed into the Node.js server that performed backend processing on behalf of the application. As far as multiple Realms were concerned, the developer would architect some shared read-only Realms with common data across all users and a number of read/write private user realms for each user. In addition, the developer could add separate Realms for shared chat or team data. The schema for each Realm, could either be defined inline in the client-side code, or defined explicitly by the Node.JS server. My preference was for using the Node.JS schema as the single source of truth, and then export out the client-side schemas from Realm studio for use in the client-side code. I would encourage using the same strategy in the new MongoDB Realm product, where the schemas can be exported from the SDK tab in the MongoDB Realm application web portal. Manually trying to maintain schema coherency across multiple programing languages is a nightmare fraught with the worse kind of errors. This is a task that a developer must absolutely automate.
In the new MongoDB Realm product, the Realm instance concept is replaced by a MongoDB Atlas cluster and a database name within that cluster to sync, along with a partition key. The partition key is what maps Realm to MongoDB Atlas. There is only one partition key per Realm application, and it cannot be changed once defined — the typical name used is ‘_partition’, but the developer can name it anything. A database collection is part of a Realm if it has a partition key property defined on it. It is the string value of the partition key that defines which specific Realm the collection belongs to. Shared Realms can be modeled using a common partition key value like ‘sharedData’, while private user Realms can be modeled with the user id as the partition key value, other shared realms used to model team data could use a GUID as the partition key value.
Perhaps the best to show how to use the new MongoDB Realm by example. In order to familiarize myself with the new MongoDB Realm, I decided to write a simple chat application using SwiftUI for iOS. I have posted the code to this application to a public GitHub repository at https://github.com/Cosync/SuperSimpleChat.git. I called this application Super Simple Chat; it basically allows a user to signup/login with a simple email password and add a chat message to a shared chat thread view. The code is open source and developers are free to use it as they see fit.
First go to https://mongodb.com to create and signup into your developer account if you have not already done so. The first step in writing a MongoDB application consists of creating a MongoDB Atlas database cluster to support it. This step is easy, as MongoDB offers one free cluster per developer to try this out with. When defining a cluster however, it is important to select ‘MongoDB 4.4 — Beta’ as the version under Additional Settings. Earlier versions will not work under MongoDB Realm. Also, you must define the cluster name at this point. I called the cluster ‘SuperSimpleChat’ as shown in the user interface below
It will take a few minutes for MongoDB Atlas to create and deploy the cluster. Once that is complete, you can then move on a create the associated Realm application. Then click on the Realm tab at the top of the web interface. You will be prompted to create your first sample MongoDB Realm application as follows. Since this is for iOS, select the options as shown below.
You will then be prompted to name your Realm application, again enter the text ‘SuperSimpleChat’ as the name. The system will prompt you to link your application to the database cluster ‘SuperSimpleChat’ defined previously, simply use the default value as shown below and click ‘Create Realm Application’ at the lower right.
You will then be asked to Set up Sync in Dev Mode, hit ‘Get Started’. Select ‘SuperSimpleChat’ as the cluster to Sync. Next you must define a database name that will contain the Realm collections to sync, for the purposes of this demo we use ‘SuperSimpleChatDB’ as the database name. Lastly, choose a partition key that Realm will use to identify those collections that need to be synced. We choose a partition key called ‘_partition’ with a field type of string. Remember the partition key cannot be changed after this is set up, only its value can be changed. Hit ‘Turn Dev Mode on’ to move forward.
Next, you will need to enable Email/Password as an authentication provider for your Chat application. Click on the Users tab to the left, and then select Providers. Click Edit under Email/Password. First, toggle the ‘Provider Enabled’ to true. Second, select ‘Automatically Confirm Users’ as your User Confirmation Method. Third, select ‘Run a password reset function’ as your Password Reset Method. Forth, select ‘New Function’ as your Function. The function default will be called ‘resetFunc’ and do not change that. Select ‘Save’ to enable the Email/Password authentication method.
You are now ready to REVIEW & DEPLOY your application. Press the button at the top of the window to do that. The press ‘Deploy’ for your changes to take effect. Your Realm application is now set up on the backend and ready to use by a front-end client.
The source code for the SuperSimpleChat application for iOS is available on the following GitHub public repository https://github.com/Cosync/SuperSimpleChat.git. Please feel free to download that to your Mac.
From the terminal type
> git clone https://github.com/Cosync/SuperSimpleChat.gitGo into the SuperSimpleChat directory and install the pods. Again, from the terminal type
> pod installThen open the workspace for SuperSimpleChat in xCode.
> open SuperSimpleChat.xcworkspaceLastly you need to set the Realm App Id in the SuperSimpleChat so that your client can connect to the MongoDB Realm application at runtime. You can copy this Realm App Id from the MongoDB Realm portal by clicking on the SuperSimpleChat icon in the top left of the UI. Edit the file ‘Constants.swift’ to set this Realm App Id as shown below:
You are now ready to run your app in the Simulator. Go to the Signup tab and register a user with the Chat application as shown below.
After you sign up as a user, you will be logged into the chat application and can start adding chat entries to the main chat window. Any user can log in and add to the collective chat window. You can also logout and subsequently log back in using your signup credentials.
This super simple chat app, while not extremely useful by itself, illustrates how easy it is to build a simple shared chat program using MongoDB Realm. The Realm data model/schema we use is found in the file called models.swift.
1. import Foundation
2. import RealmSwift
3.
4. class UserData: Object {
5. @objc dynamic var _id = ObjectId.generate()
6. @objc dynamic var _partition = ""
7. @objc dynamic var uid = ""
8. @objc dynamic var name = ""
9. override static func primaryKey() -> String? {
10. return "_id"
11. }
12. override static func indexedProperties() -> [String] {
13. return ["uid"]
14. }
15.
16. convenience init(uid: String, partition: String, name: String) {
17. self.init()
18. self._partition = partition
19. self.uid = uid
20. self.name = name
21. }
22. }
23.
24. class ChatEntry: Object {
25. @objc dynamic var _id = ObjectId.generate()
26. @objc dynamic var _partition = ""
27. @objc dynamic var name = ""
28. @objc dynamic var text = ""
29. @objc dynamic var createdAt: Date? = nil
30.
31. override static func primaryKey() -> String? {
32. return "_id"
33. }
34. override static func indexedProperties() -> [String] {
35. return ["createdAt"]
36. }
37.
38. convenience init(partition: String, name: String, text: String) {
39. self.init()
40. self._partition = partition
41. self.name = name
42. self.text = text
43. self.createdAt = Date()
44. }
45. }The application essentially defines two Realms. A user Realm with a partition key whose value is the user id of the user. This Realm stores the UserData structure which records the user’s name. This record is written to MongoDB Atlas at the time the user signs in. The second Realm is the chat realm, whose partition key value is ‘chat’. This realm contains the ChatEntry records for each shared chat entry that is added to the shared chat. A new record is added everytime a user adds a chat entry from the bottom of the Chat view.
The whole UI is written in SwiftUI with only two views; the LoginView that contains a tab control for Login and Signup and the ChatView that contains a List of chat entries that are added to the chat. The whole program is only a few hundred lines of code and shows how easy it is to implement Chat functionality within the MongoDB framework.
I hope this little example proves useful to the growing MongoDB Realm developer community. I am very enthusiastic about this technology and see it as a major steppingstone for developing truly collaborative applications for mobile and desktop platforms. Incidentally, MongoDB Realm is offline first. Even with this simple application, if you turn off connectivity on the client, the chat functionality will still work and re-sync once the application reconnects to the cloud. This is truly an added bonus that MongoDB Realm gives the developer without any extra code on his/her part.
