tscs37's Personal Blog

I put down some of my personal thoughts, some of my experiments and some of my projects

SSH RPC with MSGPACK

MSGPACK over SSH

For a few projects I needed a way to transmit MSGPACK over an SSH, namely, to make a simple request-response RPC system.

Of course, it is not fancy, but it's also not complex. To establish the connection, the client probes for the server at a well known path (such as /opt/bin/rpc-server) and a flag to indicate it's only testing the connection. An example would be ssh remote-server /opt/bin/rpc-server rpc --check. If the standard output contains a well-known message and the exit code is 0, then RPC is supported and can proceed, otherwise it errors out.

To build the final connection, the binary is called with a new flag to indicate it wants an RPC session. Standard Error is connected to that of the client and logging verbosity is passed along so that the user sees both client and server doing their thing; ssh remote-server /opt/bin/rpc-server rpc --connect --log-level=warn. Standard Input and Output are wired into a simple RPC Framework.

The messages themselves are length prefixed with a 32-bit unsigned integer and can be up to 40 Megabytes long (in my projects, I hit 20MiBs and I wanted to have some headroom). The rest of the message then follows in it's entirety. Once a message is received by the server, it replies with it's answer. By default the first message pair in my projects exchanges a protocol version, so that if I make incompatible changes, the incompatible side can abort and the client and retry with a different version it supports (if that is desired at all)

Obviously this has a bunch of downsides, since it supports no streaming or asynchronous behaviour. All messages are strictly ordered and there is only request-reply workflows. But for my purposes it's good enough.