Architecture: Onsite Notification Service

2 min read

We want to make sure every Bukalapak user received the message they need. So, we develop a notification platform. It’s similar to Facebook notification, personalized notification for every user. This is the onsite notification.

onsite notification

Requirements

  1. Support multiple types of notification
  2. User personalized notification
  3. Accommodate 10 million Bukalapak users
  4. Independent service

Architecture: bit by bit

Database

MySQL. Simple reason, most of our engineers had the most experience in using MySQL. This will alleviate the scaling pain in the long run. We considered other bleeding edge databases like MongoDB, Cassandra, and RethinkDB, but for the sake of stability and maintainability, we chose to be conservative this time.

Database Design

Only 1 table for this service. These are the fields.

No.FieldTypeDescription
1idSERIALyour standard row identification
2user_idINTreference to user, indexed
3typeVARCHAR(10)to differentiate kind of notifications, indexed
4dataTEXTnotification payload
5read_atDATETIMEtime of read, indexed
6created_atDATETIMEstandard timestamp, indexed, just in case

We chose to as denormalize as possible to improve performance and easier future maintenance. We considered splitting into 2 tables to reduce redundancy but joining table reducing performance by quite a number. If in the future we want to scale horizontally by parting the table, one table is easier to split. We used SERIAL() for id field to accommodate more than billions of notifications. Using standard `INT` will make us reach the ceiling in a short time. In fact, we have reached 1/3 road to that 2 billion integer ceiling after 4 months of the release. We need new SSD.

data field will be JSON serialized string. The rest of the field are indexed except the data field as we swore not to query based on the data field.

Application

We want the notification platform to be independent in order to reduce the load of our main monolithic service and make teams move swifter. Yes, this is a step into micro-services architecture.

On to the application, we used Rails because this starts as an experimental feature, we want to develop the app rapidly. Secondly, we tried other lower level language like Go, but after some calculation, we believe choosing Go will not give us a significant performance boost. After all, this application is an API only application with no templating, no authentication, and no complex logic, simple insertion and simple read based on user_id. But, of course, if we need more performance and efficiency, we would happy to try other options. Rust, maybe, since we have rust prophet in our engineering team.

Background Worker

To make other services could insert notification, we use RabbitMQ as the queue service. Thus, we only need to implement worker(background application) that listen to the queue. We decided not to make API for inserting notification because by using queue service, we are not worried to be overwhelmed by tremendous amount of request. If the request is greater than our throughput, the rest of the notification will safely be stored in the queue and will be consumed eventually. Like every good things in this life, it’s best consumed with moderation.

While on the other hand if the same case happens with API, either we need to scale up the application server or the performance will degrade. Though we will implement insertion API for other services that need synchronize request.

Result

All API endpoint got less than 20 ms in average response time. Moreover, the most used endpoint, the one that counts the unread notification, is averaging in 3 ms. We clearly did not expect to be this fast, since ruby is only “fast enough”! The service store about 40 GiB worth of notifications every month and there is no sign of slowing the performance. The only time that slows down the response time is when we insert a lot of(millions) notifications in short time. Phew.

This notification platform is only available in desktop, for now. The mobile web, Android, and iOS will be released in the near future. Let us know if you have any insight!

How Kubernetes can make life easier for Bukalapak’s …

Currently, Bukalapak is transitioning from an monolithic architecture into a microservice-based architecture for its entire software system. This is a massive undertaking that encompass splitting...
Geeas Prisila
1 min read

How I Design My Code

P.S: This post is also available at Medium Programming is becoming mainstream nowadays. I saw people code and build some applications. As a programmer,...
Indra Saputra
1 min read

2 Replies to “Architecture: Onsite Notification Service”

  1. I like your terms -> ((rust prophet)) :))

    Based on this terms, so it means Bukalapak will implement microservice-architecture and migrate all of your ruby(on rails) stacks into microservices? And using Rust ? Wow, it’s interesting 🙂

    1. Ah, yes we are on migrating to microservices, but not only using rust. Rust is one of our option, but the call will be made by each team. We could use another option like Ruby or Go.

Leave a Reply

Your email address will not be published. Required fields are marked *