Continuation of discussion: #2000 (reply in thread)

Below are the key points discussed:

1. Key Goals Mentioned:

   • Zero external dependencies.
   • Improved browser-friendliness, particularly better bundling.
   • Support for various carriers, including sockets, websockets, webstreams, websocket streams, and web transport.

2. Transport Plugins:

   • @robertsLando suggests making transport plugins tree-shakable by having users register plugins outside the core MQTT.js.

3. Support for MQTTv5:

   • While @seriousme expresses concerns about MQTTv5's architectural design, @robertsLando confirms that MQTT.js already supports it and will continue to do so.

4. Code Performance and Maintainability:

   • @seriousme prioritizes maintainability over performance but notes the importance of lightweight and widespread adoption of MQTT.
   • @robertsLando emphasizes maintaining or improving current performance levels while balancing with maintainability.

5. Contributions from Opifex:

   • @seriousme offers to contribute code from the Opifex project, such as packet encoding/decoding and a socket-like interface for webstreams.
   • @robertsLando mentions the importance of ensuring compatibility and running tests if integrating these contributions.

6. Next Steps:

   • @seriousme expresses interest in adding MQTTv5 client support and potentially browser support to Opifex.

@seriousme #2000 (comment)

I have put all benchmarking code into https://github.com/seriousme/mqtt-packet/tree/benchmark-publish-packet
The script that creates the report is: https://github.com/seriousme/mqtt-packet/blob/benchmark-publish-packet/benchmarks/generateAll.js

It uses:

• mqtt-packet
• Opifex (https://github.com/seriousme/opifex)
• pubPacket which uses new Uint8Array() to allocate and TextEncoder.encode() to encode strings (https://github.com/seriousme/mqtt-packet/blob/benchmark-publish-packet/benchmarks/raw/pubPacket.js)
• pubPacketBuffer which uses Buffer.allocUnsafe() to allocate and Buffer.write() to encode strings (https://github.com/seriousme/mqtt-packet/blob/benchmark-publish-packet/benchmarks/raw/pubPacketBuffer.js)
  I think that we both agree that for high speed data processing:

@seriousme #2000 (comment)

I think that we both agree that for high speed data processing:

1. memory allocations are expensive operations that should be avoided (hence pubPacketBuf only allocates once)
2. data should be copied as little as possible
3. list with pointers to buffers (either BL or the custom one in Opifex) help in archieving goals 1 and 2

What I learned from the whole experience is that:

1. TextEncoder.encode() is a relatively expensive operation on NodeJS . I expected some impact, but the difference with Buffer.write is seriously big and other people have questions about its performance as well.
2. Uint8Array.subarray() is also quite expensive operation when I expected it to be relatively cheap pointer recalculation only.
3. Manual inlining code can still outrun automatic optimizations by the V8 compiler.

@seriousme

It seems like more people are complaining about TextEncoder/TextDecoder performance: whatwg/encoding#343
I haven't found yet why subarray() is so expensive as at first glance I would only expect some simple pointer recalculation.