GhostWhiteCrab
This is a portable, stand-alone GWebCache implemented in C. It is open-source and distributed under a BSD-style license (MIT license). It was developed as a light-weight replacement for CGI GWebCache implementations in PHP, Perl, ASP etc. Unlike those GhostWhiteCrab can easily handle tens of thousands requests per hour consuming little memory and CPU resources. Thus, you can turn your old workstation into a full-blown GWebCache. Although GWebCaches are usually used as bootstrap system for Gnutella, you can use them for any other network as well nowadays. GhostWhiteCrab doesn't handle multiple networks at a time but you can configure it for any network you like.

Additionally, GhostWhiteCrab can be used as an UDP Hostcache (UHC) for the Gnutella network.

Please, read the documentation carefully and make sure you're aware of all consequences before installing a GWebCache or UDP Hostcache. You have been warned.

Download
The BZIP2 compressed sources:
gwc-0.9.4.tar.bz2 [HTTP] [Signature]

Documentation
You'll find the documentation in doc/readme.html. An online copy can be read here. Further information regarding the configuration along with examples is can be found in files examples/crab.conf and examples/uhc.conf.

You can use the following file to bootstrap your GWebCache: bootstrap_urls.txt. Just copy it to good_url_cache. Crab will verify the URLs and then propagate those that work in replies to urlfile requests.

To bootstrap your UDP Hostcache just put at least one IP address and port of a Gnutella peer or other UDP Hostcache into the peer_cache file:

    192.0.2.33:6346    1    NONE
Crab will sent the listed peers an UHC ping in order to acquire peer addresses. This is unnecessary as soon as your UDP Hostcache becomes popular. If you use Crab as GWebCache and UDP Hostcache at the same time, this isn't necessary either.

To check whether your UHC is up and running, you can also use the Gnutella ping tool called Guppy.

IPT
The following is a set of Perl scripts provided by Raphaël Manfredi. With these you can dynamically ban greedy and abusive clients utilizing iptables. The script analyses Crab's log_alert file periodically and adds appropriate iptable rules. This helps reducing bandwidth usage and works as an additional defense against hostile parties. However, you'll have to adjust the paths and port settings, so it's not quite useable out-of-the-box. In any case, have a look the scripts. You can probably adapt them to other firewall/packet filter systems like ipf, pf etc. as well.

ipt-5.0.tar.bz2 [HTTP] [Signature]

Changes in GhostWhiteCrab from 0.9.3 to 0.9.4
[Bugfix] The UDP Hostcache could not decompress GGEP IPP blocks in some cases.
[Bugfix] The logging format used for log_access has been corrected.
[New] Partial IPv6 support. This feature is not yet complete and should be considered experimental.
Changes in GhostWhiteCrab from 0.9.2 to 0.9.3
[Bugfix] The UDP Hostcache failed to answer pings that contained other GGEP IDs before the SCP block.
[New] Crab does now check whether the clock jumps backwards because such a clock could cause trouble internally and cause invalid statistics. If the clock jumps too far backwards, the process will abort.
The default value of the option max_connect_time was changed from 60 to 30 (seconds).
The default value of the option url_check_allow_dupes was changed from true to false.
The documentation was updated and explanations of the several configuration options were added.
A couple of code was rearranged to use generic data structures instead of rather obscure ones.
[New] Experimental pthreads support for the DNS helper. This means instead of a secondary process, Crab will use a thread for DNS lookups. There's no significant advantage but if you curious give it a try. It's disabled by default but can be enabled at compile-time by using the parameter –use-threads for config.sh.
[New] If your getaddrinfo() is broken you can pass –with-gethostbyname to config.sh to enforce the use of gethostname() instead.
Changes in GhostWhiteCrab from 0.9.1 to 0.9.2
Several bugfixes and enhancements in the UHC code. UHC supported is still considered unfinished and experimental.
Confused underscore with hyphen in the case of an unknown vendor.
Restructuring because there was far too much code in main.c.
Changes in GhostWhiteCrab from 0.9 to 0.9.1
Bugfix: There was a bug which can cause an assertion failure after a while due to not properly counting the number of cached peers.
Improved config.sh e.g., checks for potentially necessary libraries and checks for zlib.
More documentation.
Preliminary and experimental UHC (UDP Host Cache) support. See the option support_uhc. At the moment, this can only used to extend leases of cached peers and to reply to SCP pings. The current implementation is naive and simple. Thus, enable only if you want to experiment with it. It is completely disabled by default.
Decreased default timeouts.
Changed default of url_check_allow_dupes from false to true.
Updated the example systrace policy and added comments.
Added option max_cached_peers to configure the amount of peers to cache. The previous hardcoded setting was 8191. The current default is 10000. Note that this value will be rounded to the next prime.
Contact
If you have any problems with the software or suggestions, let me know:
christianbiere at gmx dot de
[encrypted mail is welcome: my OpenPGP key]

Last edited: 2005-09-21