MCNTP - Multicast News Transport Protocol
MCNTP is used to distribute NetNews articles via IP multicast. In the classical sense news are transferred via nntp on point to point links or sometimes batches of articles over uucp. The following considerations only apply to the distribution over nntp as the transport over uucp is most often used these days only towards hosts that are not always online. Even if the (virtual) net of hosts exchanging news is densely meshed, there is no optimal use of the mesh as articles often cross the same physical link multiple times. This can be prevented by installing news servers at every major node which get the articles only one time and then distribute them only to their direct neighbours. This is hardware and support intensive.
Now with IP multicast the idea comes to mind that articles are fed into the multicast backbone and everyone who wishes to receive articles just subscribes the appropriate multicast group to receive the news articles. Multicast routers only deliver articles to those who want them and articles only cross the physical links once as the routers only duplicate the packets when they need to.
The communication between mcxmit(8) and mcrcv(8) can be compressed (by using libz(3) ) and crypted. Compression can save even more bandwidth, but uses some cpu cycles. Cryption can be used to provide private newsgroups over the public mbone. Note that cryption is heavily dependent on the law as there are countries which don't allow to use encryption technologies to be used.
Before sending, mcxmit signs every article with the private key of the sender. Mcrcv checks the articles with the public key of the sender of the packet. If the signature is not valid, the article is thrown away and a log entry is made.
With MCNTP you have at the provider site some sender processes, mcxmit(8), that bring the articles on the wire and the receiver sites run mcrcv(8) to get the articles of the wire and into their news system. That process is coordinated by a some controlling daemon processes, yawgmoth(8). yawgmoth(8) and mcrcv(8) can limit the senders that they accept messages from by setting up access control lists.
With this setup articles will cross links usually only once resulting in savings in line cost. As the transfer via IP multicast is not reliable, you still have to run the classical nntp connections, but which will only be «ihave» «already got it» streams most of the time especially if you use delayed nntp as for example provided by nntplink(8) jobs that are just run once every five minutes. Mcntp is quite fast in use -- running it in parallel with nntplink(8) shows that more than the half of articles can be transferred with mcntp even if nntplink does not delay the articles.
As you might already have guessed, mcntp need a working IP multicast infrastructure and hosts that have the appropriate extensions to their kernels. This paper won't describe how to achieve this, but your provider might be able to help you out. Also http://www.mbone.com and http://www.mbone.de/ will be of help.
Here comes a very short description of every module of the mcntp project; each one of it has its own man page(s) to look for more information.
mcxmit
Sender program. It receives a article path in the spool, message id
and article age on stdin, gets the article from the file system and
sends it to the appropriate multicast group.
mcrcv
Receiver program. This one tunes into the appropriate multicast
group(s) for the selected news groups, fetches the articles from
there and feeds them to a local nntp server.
yawgmoth
Coordinator. This tool is used to coordinate the relation between
newsgroup names and multicast group numbers. It must run on every
host that also wishes to run
mcxmit(8).
mess_send
Measurement tool. It can be used to find out packet loss rate between
two multicast hosts.
groupdump
Debugging tool. Dumps the relation between newsgroup names and multicast
group numbers as communicated between
yawgmoth(8) processes.
mcchat
Example. Little chat program a bit like
talk(1) that uses multicast
for communication with chat partners. Shows how one can use the
functions of
libmc(3) to program a simple chat program with only a
few lines of code.
acltest
Example and helpful tool.
acltest(1) is a tool to validate given
access control lists. These lists can then be used to limit receiption
of control messages and articles.
keygen
tool to create private and public keys for use with mcntp. This is
also used to manipulate the keyring, where public keys from senders
that you trust, are kept.
rmd
tool that produces RIPE-MD 160 fingerprints of files. RIPE-MD160
has been chosen over MD5 as is is more secure.
libmc
The Library that provides a higher level multicast api than the
normal socket interface.
libmc(3) is needed to get all the programs
work that come with mcntp.
acltest(1), mcxmit(8), groupdump(1), mcrcv(8), yawgmoth(8), nntplink(8), innxmit(8), libmc(3), libz(3), news(5), innd(8)
RFC 977
The RFC describing NNTP (Net News Transfer Protocol) by Phil Lapsley
and Brian Kantor
RFC1036
Standard for Interchange of USENET Messages by M.Horton and Rick
Adams
RFC 1112
Hostextensions for IP multicasting by Steve Deering
INN Paper
The Usenix paper from Rich Salz, where he first presented the INN
system at ftp://ftp.uu.net/networking/news/nntp/inn/inn.usenix.ps.Z
nntp.ps
A overview paper for those that are not familiar with nntp and news
by Tom Podnar <tpodnar@bones.wcupa.edu> at
ftp://bones.wcupa.edu/pub/nntp.ps
Zlib documentation
The data format used by the zlib library is described by RFCs (Request
for Comments) 1950 to 1952 in the files
ftp://ds.internic.net/rfc/rfc1950.txt (zlib format), rfc1951.txt
(deflate format) and rfc1952.txt (gzip format). These documents are
also available in other formats from
ftp://ftp.uu.net/graphics/png/documents/zlib/zdoc-index.html
RIPEMD-160 documentation
The documentation of RIPEMD-160 can be found in the ripe-md160 directory
of the mcntp distribution.
Heiko W.Rupp (hwr@pilhuhn.de)
None known yet. If you find any, then please report them to <mcntpbugs@pilhuhn.de>