Retroshare Forums

  • Component: Forums
  • Development: April. 2008 – Jan. 2010
  • Status: released

This is my first blog as part of the Retroshare (RS) development team. Firstly if you are an RS user, hope you are loving the software, and if you are not, really do give it a try. I am going to describe Forums, one of Retroshare’s most used service.

Retroshare’s Forums is a decentralised message distribution service which bares many similarites to the more familiar web forum. Like web forums it allows users to post a topic and, users to subsequently post messages about the topic (which are called threads). The interesting bit about RS Forums is how topics are published, administered and distributed among peers.

Distribution

The distribution of topics and messages in Forums are done through the synchronisation of Forum data between RS peers.  Once a topic is published by a peer, it is shared to all peers directly connected to the peer. The peers who subscribe to the topic then distribute it to peers they are directly connected to and so on.

Messages for a given topic is shared amongst networked peers in the same way. A depiction of an instance of Forum message distribution is shown in Figure 1. Here the purple circles depict peers who have subscribed to a topic published by Peer A. The purple peer’s propagate the topic and its messages along the network. Yellow peers have received the messages and the topic of peer A but haven’t subscribed.

Forum Evolution

Note that peer G is aware of neither topic or messages as he is not part of the network initially. Once he joins A’s network through peer F he is immediately updated with peer F’s current topics and become’s aware of A’s topic. Again, it is important to note that a peer must be subscribed to a topic in order to forward messages of that topic as well as the topic itself.

Forums Privacy and Authentication Policy

Forums topics are published with a combination of privacy and authentication policy. The two policies have no effect on each other. There are two types of privacy policies available:

  1. Public 
  2. Private

A public topic allows anyone to post to it. A private topic only allows privileged peers chosen by the author to post. This is implemented via public key cryptography which allows messages to be encrypted using a public publish key. The message can only be decrypted by peers who have the private publish key.

There are two types of authentication policy available:

  1. Authenticated (PGP)
  2. Anonymous

Authenticated topics require peers to sign messages, before they can be published, with their private PGP key (synonymous to Retroshare identity). This allows peers with the public PGP key of the publishing peer to be able to authenticate the message, and confirm the message’s author [1]. In Retroshare only direct friends of the peer, and “friends of friends” typically have a peer’s public PGP key. Anonymous Forum topics do not require such a signature.

Self Regulation and Localisation

Popular Forum topics are more likely to be subscribed to, and this enhances their distribution throughout the network. This characteristic of Forums leads to self-regulation of topic availability on the network. A positive by-product of this self regulation is the efficient use of bandwidth by not distributing unpopular or spammy topics.

Localisation is a characteristic seen primarily in authenticated Forum topics. It means that topic messages are only received that originate from a few hops away on one’s Retroshare  network. This serves to keep the discussion relevant to a peer’s circle of friends, and enhance your privacy.  Localisation is caused by peers not distributing messages which they cannot authenticate (i.e they don’t have the author’s PGP Key). Typically, these are messages from authors 3 or more hops away.

Authenticated Forums significantly reduce spam levels. Firstly, people are generally more considerate when they are not anonymous. Additionally, localisation prevent spamming a popular topic from far afield.

Popularity of a topic is calculated as the percentage of direct peers who are subscribed to the topic. This method of calculating popularity arises naturally from Retroshare’s decentralised nature. Consider Figure 2 and the table 1 below that. They consider a single topic which peers A-K all have, but of which A, E, H and K are subscribed to.

Network Topology

It is important to note that peers on the network do not necessarily see the same value for this popularity metric. This can be seen in table 1 which shows the calculated popularity values for the peers shown in figure 2.

Table 1: Popularty of a Topic for a Few of the Peers in Figure 2

Peer # Friends # Subscribers Local popularity
Peers A 6 3 43%
Peers B 1 1 50%
Peers D 5 3 50%
Peers E 4 3 60%
Peers G 3 1 25%
Peers H 5 4 66%
Peers J 2 1 33%
Peers K 4 2 40%

The key points to take from Forum’s localisation and self-regulating properties are:

  • Interesting stuff get shared
  • Rubbish stuff gets dropped
  • Popularity is local to peer

Resistance to Censorship

In keeping with Retroshare’s ideals [2], Forums is highly resistant to censorship. Its decentralised structure makes it extremely hard for any one peer to suppress a message or topic distributed in Retroshare’s network. Secondly with anonymous topics users have the choice not to associate their Retroshare identity with any posts they might make.

Back End: The Cache System

A little technical aside is made on the back-end of  Forums, the Cache system, which powers a few other services in Retroshare. It is quite simple in design and effectively synchronises files (called “caches”) between peers in a network. All peers operating the cache based service have a remote and local cache. The local cache represents a set of cache files created by the peer which are synchronised to other peers. The remote cache represents effectively the local cache of all peers on the peer’s network.

The distribution policy of Forums is enforced here which pushes topics peers subscribe to into their local cache (as well as messages for the topic). The sum effect of this is a certain amount of duplication of Forum data between peers. This duplication enhances the distribution of Forum data through the network as not all peers on the network may be on-line to synchronise their caches. There is a downside to this which is spoken about next in issues with the current Forums.

Issues with Forums

  • Redundancy: There is a huge amount of redundancy of Forum data held by peer. This is not a problem with the Forum concept, but rather its underlying implementation using Retroshare’s Cache system. Cache lacks the ability for more fine grained synchronisation (i.e. you would not want to receive topics or messages you already have).
  • Missing Messages: This occurs with authenticated forums because of not having the public PGP key which is needed to authenticate its author. This disrupts the continuity of a thread.
  • Single PGP identity to publish with: While one may not want to compromise their Retroshare identity with a post, they may want to maintain a handle for say a controversial topic they are discussing. It would be useful for peers to be able to create pseudonyms they can use when posting messages.
  • Spam: While not so much an issue with authenticated Forum topics, it can be an issue with anonymous Forum topics in which spam can travel across the whole RS network to which the spamming peer belongs to.
  • The fact that unpopular topics do not get distributed as widely is a double edged sword. A peer might be interested in such a topic and it may not reach them simply because it is not popular.

Future

The RS development team is currently working on a replacement for the back-end of many of its distributed message services like Forums. It is called GXS (General Exchange Service) which aims to address many of the deficiencies highlighted above. All in all watch this space!

Links:

[1] http://en.wikipedia.org/wiki/Digital_signature

[2] https://retroshareteam.wordpress.com/2012/10/26/ideals-behind-retroshare/

Edit: Fixed table calculations

Advertisements

About chrisparker126

Been part of the RS developement team for about 4 years contributing code sporadically. Worked on Forums, Channels and a lot of back end stuff. Joined to learn "how the internet worked". Learnt much of stuff since then.
This entry was posted in Software components. Bookmark the permalink.

8 Responses to Retroshare Forums

  1. apo says:

    Great post, thank you! Learned quite a bit.

    (Figure 2, peer j sees 1 subscriber IMO. )

  2. chrisparker126 says:

    Hi apo,

    Thanks, you are very right! Actually the calculations were done incorrectly in table 1. I’ve corrected them now.
    apologies for the confusion.
    Popularity formula is (friends+1)/(# of subscribers), +1 is for the peer whom you are calculating popularity for, and # of subscribers in the table includes the peer if he is subscribed.

  3. impedance says:

    whether it possible to adopt email system like you made forums already. as I could understand the email(messaging module) in RS still not provide message storage in cache. I mean that if I send message to offline person, then went offline by myself, the person will not recieve this message until I come online. as I know i2p-bote provide offline storaging of mails. so I think RS also can do this, just take some technologies implemented in forum system, isn’t it?

    • impedance says:

      it could be real alternative to mails services, the so colled serverless mail.

    • Cyril says:

      I’m currently working on it. But the new distant message system is not going to use the cache, since it is a bit too slow to send messages. It’s based on a global routing system, where each peer knows the best next friend where to forward a message to a given key.

  4. SFS says:

    When can we expect a screen sharing feature? We have been following the project closely and LOVE it so far but it can not completely replace some of our other communications platforms as it does not have screen-sharing capability. A whiteboard would be awesome as well, but not nearly as important as screen sharing capabilities,

  5. John Riotto says:

    does your x32 bit qt5 run well on a x64 bit os 7 windows system, all my friends are showing off line even though I know they are on line. Do you know if I am doing something wrong or have a wrong setting. We have a strong network and trying to get retroshare working. Also, none of the chat rooms are showing

    • Cyril says:

      Generally speaking, prefer the Qt4 version over Qt5. Some people have problems with it, which we cannot explain yet.

      You only see the Chat rooms that your friends are participating to. So at start, when no chatrooms have been created, you don’t see any. The way to go is that one person creates a chat room and his friends join, which gives access to friends of friends etc.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s