Fast Pure JavaScript MessagePack Encoder and Decoder

Online demo: http://kawanet.github.io/msgpack-lite/

• Pure JavaScript only (No node-gyp nor gcc required)
• Faster than any other pure JavaScript libraries on node.js v4
• Even faster than node-gyp C++ based msgpack library (90% faster on encoding)
• Streaming encoding and decoding interface is also available. It's more faster.
• Ready for Web browsers including Chrome, Firefox, Safari and even IE8
• Tested on Node.js v0.10, v0.12, v4, v5 and v6 as well as Web browsers

varmsgpack=require("msgpack-lite");// encode from JS Object to MessagePack (Buffer)varbuffer=msgpack.encode({"foo": "bar"});// decode from MessagePack (Buffer) to JS Objectvardata=msgpack.decode(buffer);// =>{"foo": "bar"}// if encode/decode receives an invalid argument an error is thrown

varfs=require("fs");varmsgpack=require("msgpack-lite");varwriteStream=fs.createWriteStream("test.msp");varencodeStream=msgpack.createEncodeStream();encodeStream.pipe(writeStream);// send multiple objects to streamencodeStream.write({foo: "bar"});encodeStream.write({baz: "qux"});// call this once you're done writing to the stream.encodeStream.end();

varfs=require("fs");varmsgpack=require("msgpack-lite");varreadStream=fs.createReadStream("test.msp");vardecodeStream=msgpack.createDecodeStream();// show multiple objects decoded from streamreadStream.pipe(decodeStream).on("data",console.warn);

varmsgpack=require("msgpack-lite");// decode() accepts Buffer instance per defaultmsgpack.decode(Buffer([0x81,0xA3,0x66,0x6F,0x6F,0xA3,0x62,0x61,0x72]));// decode() also accepts Array instancemsgpack.decode([0x81,0xA3,0x66,0x6F,0x6F,0xA3,0x62,0x61,0x72]);// decode() accepts raw Uint8Array instance as wellmsgpack.decode(newUint8Array([0x81,0xA3,0x66,0x6F,0x6F,0xA3,0x62,0x61,0x72]));

A CLI tool bin/msgpack converts data stream from JSON to MessagePack and vice versa.

$ echo'{"foo": "bar"}'| ./bin/msgpack -Jm | od -tx1 0000000 81 a3 66 6f 6f a3 62 61 72 $ echo'{"foo": "bar"}'| ./bin/msgpack -Jm | ./bin/msgpack -Mj{"foo":"bar"}

$ npm install --save msgpack-lite

Run tests on node.js:

$ make test

Run tests on browsers:

$ make test-browser-local open the following url in a browser: http://localhost:4000/__zuul

Browser version msgpack.min.js is also available. 50KB minified, 14KB gziped.

<!--[if lte IE 9]><script src="https://cdnjs.cloudflare.com/ajax/libs/es5-shim/4.1.10/es5-shim.min.js"></script><script src="https://cdnjs.cloudflare.com/ajax/libs/json3/3.3.2/json3.min.js"></script><![endif]--><scriptsrc="https://rawgit.com/kawanet/msgpack-lite/master/dist/msgpack.min.js"></script><script>// encode from JS Object to MessagePack (Uint8Array)varbuffer=msgpack.encode({foo: "bar"});// decode from MessagePack (Uint8Array) to JS Objectvararray=newUint8Array([0x81,0xA3,0x66,0x6F,0x6F,0xA3,0x62,0x61,0x72]);vardata=msgpack.decode(array);</script>

Step #1: write some code at first.

varmsgpack=require("msgpack-lite");varbuffer=msgpack.encode({"foo": "bar"});vardata=msgpack.decode(buffer);console.warn(data);// =>{"foo": "bar"}

Proceed to the next steps if you prefer faster browserify compilation time.

Step #2: add browser property on package.json in your project. This refers the global msgpack object instead of including whole of msgpack-lite source code.

{"dependencies":{"msgpack-lite": "*" }, "browser":{"msgpack-lite": "msgpack-lite/global" } }

Step #3: compile it with browserify and uglifyjs.

browserify src/main.js -o tmp/main.browserify.js -s main uglifyjs tmp/main.browserify.js -m -c -o js/main.min.js cp node_modules/msgpack-lite/dist/msgpack.min.js js/msgpack.min.js

Step #4: load msgpack.min.js before your code.

<scriptsrc="js/msgpack.min.js"></script><scriptsrc="js/main.min.js"></script>

It is tested to have basic compatibility with other Node.js MessagePack modules below:

• https://www.npmjs.com/package/msgpack (1.0.2)
• https://www.npmjs.com/package/msgpack-js (0.3.0)
• https://www.npmjs.com/package/msgpack-js-v5 (0.3.0-v5)
• https://www.npmjs.com/package/msgpack-unpack (2.1.1)
• https://github.com/msgpack/msgpack-javascript (msgpack.codec)
• https://www.npmjs.com/package/msgpack5 (3.3.0)
• https://www.npmjs.com/package/notepack (0.0.2)

A benchmark tool lib/benchmark.js is available to compare encoding/decoding speed (operation per second) with other MessagePack modules. It counts operations of 1KB JSON document in 10 seconds.

$ npm install msgpack msgpack-js msgpack-js-v5 msgpack-unpack msgpack5 notepack $ npm run benchmark 10

Streaming benchmark tool lib/benchmark-stream.js is also available. It counts milliseconds for 1,000,000 operations of 30 bytes fluentd msgpack fragment. This shows streaming encoding and decoding are super faster.

$ npm run benchmark-stream 2

Test environment: msgpack-lite 0.1.14, Node v4.2.3, Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz

The following table shows how JavaScript objects (value) will be mapped to MessagePack formats and vice versa.

Note that both null and undefined are mapped to nil 0xC1 type. This means undefined value will be upgraded to null in other words.

The MessagePack specification allows 128 application-specific extension types. The library uses the following types to make round-trip conversion possible for JavaScript native objects.

Other extension types are mapped to built-in ExtBuffer object.

Register a custom extension type number to serialize/deserialize your own class instances.

varmsgpack=require("msgpack-lite");varcodec=msgpack.createCodec();codec.addExtPacker(0x3F,MyVector,myVectorPacker);codec.addExtUnpacker(0x3F,myVectorUnpacker);vardata=newMyVector(1,2);varencoded=msgpack.encode(data,{codec: codec});vardecoded=msgpack.decode(encoded,{codec: codec});functionMyVector(x,y){this.x=x;this.y=y;}functionmyVectorPacker(vector){vararray=[vector.x,vector.y];returnmsgpack.encode(array);// return Buffer serialized}functionmyVectorUnpacker(buffer){vararray=msgpack.decode(buffer);returnnewMyVector(array[0],array[1]);// return Object deserialized}

The first argument of addExtPacker and addExtUnpacker should be an integer within the range of 0 and 127 (0x0 and 0x7F). myClassPacker is a function that accepts an instance of MyClass, and should return a buffer representing that instance. myClassUnpacker is the opposite: it accepts a buffer and should return an instance of MyClass.

If you pass an array of functions to addExtPacker or addExtUnpacker, the value to be encoded/decoded will pass through each one in order. This allows you to do things like this:

codec.addExtPacker(0x00,Date,[Number,msgpack.encode]);

You can also pass the codec option to msgpack.Decoder(options), msgpack.Encoder(options), msgpack.createEncodeStream(options), and msgpack.createDecodeStream(options).

If you wish to modify the default built-in codec, you can access it at msgpack.codec.preset.

msgpack.createCodec() function accepts some options.

It does NOT have the preset extension types defined when no options given.

varcodec=msgpack.createCodec();

preset: It has the preset extension types described above.

varcodec=msgpack.createCodec({preset: true});

safe: It runs a validation of the value before writing it into buffer. This is the default behavior for some old browsers which do not support ArrayBuffer object.

varcodec=msgpack.createCodec({safe: true});

useraw: It uses raw formats instead of bin and str.

varcodec=msgpack.createCodec({useraw: true});

int64: It decodes msgpack's int64/uint64 formats with int64-buffer object.

varcodec=msgpack.createCodec({int64: true});

binarraybuffer: It ties msgpack's bin format with ArrayBuffer object, instead of Buffer object.

varcodec=msgpack.createCodec({binarraybuffer: true,preset: true});

uint8array: It returns Uint8Array object when encoding, instead of Buffer object.

varcodec=msgpack.createCodec({uint8array: true});

usemap: Uses the global JavaScript Map type, if available, to unpack MessagePack map elements.

varcodec=msgpack.createCodec({usemap: true});

The compatibility mode respects for msgpack's old spec. Set true to useraw.

// default mode handles both str and bin formats individuallymsgpack.encode("Aa");// => <Buffer a2 41 61> (str format)msgpack.encode(newBuffer([0x41,0x61]));// => <Buffer c4 02 41 61> (bin format)msgpack.decode(newBuffer([0xa2,0x41,0x61]));// => 'Aa' (String)msgpack.decode(newBuffer([0xc4,0x02,0x41,0x61]));// => <Buffer 41 61> (Buffer)// compatibility mode handles only raw format both for String and Buffervaroptions={codec: msgpack.createCodec({useraw: true})};msgpack.encode("Aa",options);// => <Buffer a2 41 61> (raw format)msgpack.encode(newBuffer([0x41,0x61]),options);// => <Buffer a2 41 61> (raw format)msgpack.decode(newBuffer([0xa2,0x41,0x61]),options);// => <Buffer 41 61> (Buffer)msgpack.decode(newBuffer([0xa2,0x41,0x61]),options).toString();// => 'Aa' (String)

• https://github.com/kawanet/msgpack-lite

• http://msgpack.org/

The MIT License (MIT)

Copyright (c) 2015-2016 Yusuke Kawasaki

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.