ZMQ - the universal connector!
Whatever network problem you might have, ZMQ is the solution:
- Point-to-point
- Point-to-Multipoint
- Multipoint-to-Point
- Multipoint Grid
- Unicast, Multicast
- Push / Pull
- Pub / Sub
- Multipart Messages w/ idempotency
- Realtime
- Streaming Protocols (audio / video / slides ...)
- Low Bandwidth protocols LoRaWan / Bluetooth LE
- Caching / Buffering
- Async, Multithreaded
- Brokerless Architectures
- Shared Nothing Architectures
- Embedded / IoT
- Highest Performance
- Written in pure C
- Language bindings to any programming language
- Best supported and documented
- Widely used (Netflix, Microsoft, Samsung, Bosch, Nordic Semi, Mercedes, VW, BMW, NIO, Huawei, BYD, XPeng ...)
- Bug-free!!!
The ZMQ guide / handbook: https://zguide.zeromq.org/
The ZMQ whitepapers: http://wiki.zeromq.org/area:whitepapers
The ZMQ community: https://zguide.zeromq.org/docs/chapter6/
The ZMQ C socket API: https://zeromq.org/socket-api/
The PyZMQ bindings: https://pyzmq.readthedocs.io/
Sources: https://github.com/booksbyus/zguide/tree/master/examples
Note: Don't even think about using other libraries!!!
ZMQ - my personal experience
- Implementation of non-US streaming crypto protocols (Goodbye NSA!)
- ZMQ as caching VPN protocol (like Matrix protocol, but 100x faster)
- Own audio / video / teams conference servers
- Own Active Directory (like) HA server cluster implementation
- Simultaneous administration of 6000 servers
- Distributed Database and Replication
- Hispeed Database with > 1 billion RPS, also see Implicit S-Tree
- Brokerless (HA) Architectures replacing Paxos, Raft Protocols
- Multipoint Asyncronous Server Replication - MARS
- Gemini Protocol over LoRaWan and for Bluetooth LE public grid in underdeveloped or sparsely populated nordic countries
- Implementation of Low Orbit Satellite protocols
ZMQ - Basics
https://zguide.zeromq.org/docs/chapter1/
ZMQ - Brokerless Architectures
http://wiki.zeromq.org/whitepapers:brokerless
ZMQ - Sockets and patterns
https://zguide.zeromq.org/docs/chapter2/
- How to create and work with ZeroMQ sockets.
- How to send and receive messages on sockets.
- How to build your apps around ZeroMQ’s asynchronous I/O model.
- How to handle multiple sockets in one thread.
- How to handle fatal and nonfatal errors properly.
- How to handle interrupt signals like Ctrl-C.
- How to shut down a ZeroMQ application cleanly.
- How to check a ZeroMQ application for memory leaks.
- How to send and receive multipart messages.
- How to forward messages across networks.
- How to build a simple message queuing broker.
- How to write multithreaded applications with ZeroMQ.
- How to use ZeroMQ to signal between threads.
- How to use ZeroMQ to coordinate a network of nodes.
- How to create and use message envelopes for pub-sub.
- Using the HWM (high-water mark) to protect against memory overflows.
ZMQ - Advanced Request-Reply Patterns
https://zguide.zeromq.org/docs/chapter3/
- How the request-reply mechanisms work
- How to combine REQ, REP, DEALER, and ROUTER sockets
- How ROUTER sockets work, in detail
- The load balancing pattern
- Building a simple load balancing message broker
- Designing a high-level API for ZeroMQ
- Building an asynchronous request-reply server
- A detailed inter-broker routing example
ZMQ - Reliable Request Patterns
https://zguide.zeromq.org/docs/chapter4/
- The Lazy Pirate pattern: reliable request-reply from the client side
- The Simple Pirate pattern: reliable request-reply using load balancing
- The Paranoid Pirate pattern: reliable request-reply with heartbeating
- The Majordomo pattern: service-oriented reliable queuing
- The Titanic pattern: disk-based/disconnected reliable queuing
- The Binary Star pattern: primary-backup server failover
- The Freelance pattern: brokerless reliable request-reply
ZMQ - Advanced Pub-Sub Patterns
https://zguide.zeromq.org/docs/chapter5/
*When to use publish-subscribe
- How to handle too-slow subscribers (the Suicidal Snail pattern)
- How to design high-speed subscribers (the Black Box pattern)
- How to monitor a pub-sub network (the Espresso pattern)
- How to build a shared key-value store (the Clone pattern)
- How to use reactors to simplify complex servers
- How to use the Binary Star pattern to add failover to a server
ZMQ - Live Video Streaming Examples
https://github.com/pcrete/zeromq-video-streaming/blob/master/imagezmq/imagezmq.py
https://github.com/alwaysai/zmq-video-streamer
PyZMQ Example: Server - Clients
https://learning-0mq-with-pyzmq.readthedocs.io/en/latest/pyzmq/patterns/client_server.html
Combine with the QuickSafe crypto idea: https://github.com/p-e-w/quicksafe
XOR-Chiffre
Or (better) use the: https://en.wikipedia.org/wiki/XOR_cipher
xorchiffre.py
Try in: https://bellard.org/jslinux/vm.html?url=alpine-x86.cfg&mem=192
Python code to implement your own Client - Server (or SSHD) you find here:
https://zguide.zeromq.org/docs/chapter2/
Or use:
https://github.com/jquast/telnetlib3/
https://github.com/jquast/telnetlib3/blob/master/telnetlib3/client.py
https://github.com/jquast/telnetlib3/blob/master/telnetlib3/server.py
Replace the crypto function by your own! ;-)
To start your private Python telnet server implementation use "socket activation" from systemd: https://0pointer.de/blog/projects/socket-activation.html
Compare to: https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=OpenSSH
The simplicity of C: https://stevens.netmeister.org/631/apue-code/01/simple-shell.c
What this has to do with Hyrum's law: https://www.hyrumslaw.com/
Have fun!