Rails 7.1 adds support for MessagePack as a Message Serializer

railsJanuary 31, 2024Dotby Alkesh Ghorpade

MessagePack

MessagePack is an efficient binary data format used for exchanging data between different programming languages. It's like JSON, but it aims to be:

  • Faster: MessagePack serializes and deserializes data much quicker than JSON, often resulting in significant performance improvements.

  • Smaller: MessagePack uses efficient encoding for common data types like integers and strings, leading to smaller data payloads than JSON.

  • More flexible: MessagePack supports a broader range of data types than JSON, including binary data and non-UTF-8 encoded strings.

You can use the msgpack-ruby gem, a library that allows you to serialize and deserialize Ruby objects in the MessagePack format. This gem makes it easy to leverage MessagePack's speed, efficiency, and flexibility advantages in your Ruby applications.

This is how the gem can be used in Rails applications.

require "msgpack"

msg = "hello world".to_msgpack
 => "\xABhello world"

MessagePack.unpack(msg)
 => "hello world"

msg = ["a", "b", "c"].to_msgpack
 => "\x93\xA1a\xA1b\xA1c"

MessagePack.unpack(msg)
 => ["a", "b", "c"] 

Before Rails 7.1

Before Rails 7.1, there was no built-in support for serializing data using the MessagePack. You need to install the msgpack-ruby gem and serialize the data.

verifier = ActiveSupport::MessageVerifier.new("secret_key")
 => #<ActiveSupport::MessageVerifier:0x00007fa6ffef7768 @digest="SHA1", @on_rotation=nil, @options={}, @rotations=[], @secret="secret_key", @serializer=Marshal>

data = ["hello world", { user_id: 1 }, { user_id: 2 }.with_indifferent_access, 1.to_f]
 => ["hello world", {:user_id=>1}, {"user_id"=>2}, 1.0]

message = verifier.generate(data)
 => "BAhbCUkiEGhlbGxvIHdvcmxkBjoGRVR7BjoMdXNlcl9pZGkGQzotQWN0aXZlU3VwcG9ydDo6SGFzaFdpdGhJbmRpZmZlcmVudEFjY2Vzc3sGSSIMdXNlcl9pZAY7AEZpB2YGMQ==--bf2f1bfbd0aa77be2b88154d73186c9c7a455e09" 

verifier = ActiveSupport::MessageVerifier.new("secret_key", serializer: MessagePack)
 => #<ActiveSupport::MessageVerifier:0x00007fa6fcfe98b8 @digest="SHA1", @on_rotation=nil, @options={:serializer=>MessagePack}, @rotations=[], @secret="secret_key", @serializer=MessagePack>

message = verifier.generate(data)
 => "lKtoZWxsbyB3b3JsZIGndXNlcl9pZAGBp3VzZXJfaWQCyz/wAAAAAAAA--9de5e87891c7370495f199750063dc2b52d9fe51"

As seen, you need to pass MessagePack as the serializer explicitly. If you were using JSON serializer, it had another issue with the data types of Ruby. MessagePack would return the correct class for the passed messages to the verifier.

# When config.active_support.message_serializer = :json

verifier = ActiveSupport::MessageVerifier.new("secret_key")
data = ["hello world", { user_id: 1 }, { user_id: 2 }.with_indifferent_access, 1.to_f]
message = verifier.generate(data)

verifier.verified(message)
 => ["hello world", { user_id: 1 }, { user_id: 2 }.with_indifferent_access, 1.to_f]

verifier.verified(message).map(&:class)
 => [String, Hash, Hash, Float]

# When config.active_support.message_serializer = :message_pack

verifier.verified(message).map(&:class)
 => [String, Hash, ActiveSupport::HashWithIndifferentAccess, Float]

In Rails 7.1

Rails 7.1 adds ActiveSupport::MessagePack. This helped the Rails team add support for message_pack as a message serializer.

To set the message_pack as the default serializer, add the following line in the config/application.rb file.

# config/application.rb

config.active_support.message_serializer = :message_pack

With this config, you don't need to pass the :serializer option to the ActiveSupport::MessageVerifier instance.

verifier = ActiveSupport::MessageVerifier.new("secret_key")
 => #<ActiveSupport::MessageVerifier:0x00007fa6ffef7768 @digest="SHA1", @on_rotation=nil, @options={}, @rotations=[], @secret="secret_key", @serializer=MessagePack> 

data = ["hello world", { user_id: 1 }, { user_id: 2 }.with_indifferent_access, 1.to_f]
 => ["hello world", {:user_id=>1}, {"user_id"=>2}, 1.0]

message = verifier.generate(data)
 => "lKtoZWxsbyB3b3JsZIGndXNlcl9pZAGBp3VzZXJfaWQCyz/wAAAAAAAA--9de5e87891c7370495f199750063dc2b52d9fe51" 

Note:

The :message_pack serializer is implemented via ActiveSupport::Messages::SerializerWithFallback and can fall back to deserializing with AS::JSON. Additionally, the :marshal, :json, and :json_allow_marshal serializers can fall back to deserializing with AS::MessagePack. This feature also supports :message_pack_allow_marshal as a serializer, which can fall back to deserializing with Marshal as well as AS::JSON.

To know more about this feature, please refer to the following PRs:

Closing Remark

Could your team use some help with topics like this and others covered by ShakaCode's blog and open source? We specialize in optimizing Rails applications, especially those with advanced JavaScript frontends, like React. We can also help you optimize your CI processes with lower costs and faster, more reliable tests. Scraping web data and lowering infrastructure costs are two other areas of specialization. Feel free to reach out to ShakaCode's CEO, Justin Gordon, at [email protected] or schedule an appointment to discuss how ShakaCode can help your project!
Are you looking for a software development partner who can
develop modern, high-performance web apps and sites?
See what we've doneArrow right