dropbox.com
I wanted to bring up Dropbox.com to your attention. I wanted to share what I thought was a fantastic company that’s basically taken the concept of JungleDisk and moved it one step further. How? Instead of really cheap (Jungledisk), dropbox is free for the first 2 gigs. This, combined with the below usability features, results in rapid growth in user adoption. As of January, the company has over 4 million users. That’s over 8 Petabytes of potential disk space usage if every user was a free 2GB plan.
All of the public infrastructure is hosted at AWS, and can thus scale with the userbase. As Amazon charges anywhere from $0.055 to $0.150 per GB (pricing structure), each free user consumes from $0.11 to $0.30 per month. Even with 6.25% utilization of 8 Petabytes, Dropbox.com pays the $0.105 per GB rate, Given their next paid upgrade is for 50 GB and costs $9.99/month, one paid customer can support the storage fees for up to ($9.99 − 50 clients × $0.105) ÷ $0.21 = 22 clients covered in the cost of a single paid user, at full utilization of each user. However, most users won’t be using their full utilization (think Google mail), and their business model becomes more lucrative.
Starting with dropbox is simple. Download the client, register your account, and a folder is created where you can drag and drop your data. All data in this folder is replicated to dropbox servers, and to all dropbox client nodes that you link to your account.
The application ships with an intuitive user interface (no S3 accounts to configure, as this is done for you), and a rich feature set that includes automatic versioning, automatic syncing, cross-platform compatibility, intuitive web publishing, and a simple pricing model.
The one feature that really stood out however, was how they garner feedback for this actively developed product. Users can navigate to https://www.dropbox.com/votebox, where they can submit suggestions that get voted on and commented upon. This popularity contest likely helps the company focus on what projects need developing next.
All in all, this is an exciting company to be a customer of, and I would recommend y’all take a look at it too, as this appears to be what proper execution of a good idea looks like.
Kudos dropbox.com on a product well done.
Why 100Mbps Does Not Mean 100Mbps Transfer Rates
You will not always see 100Mbps upload/download speeds even with a 100Mbps port. Much of the slowdown occurs because as packet travel distance increases, so does latency, which has a large detrimental effect on large file transfers. For smaller files, like those associated with not-too-graphical web pages, this has less impact. Without getting too technical, this is because file transfer protocols that use TCP require that the recipient respond with confirmations of data received, and this is one reason that file transfers over longer distances are slower, in direct proportion with the increase in response times.
See http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-distance-links/ for a more in-depth discussion on this.
Most download accelerators are able to increase transfer rates by simply employing multiple TCP pipes that dump into the same file. This doesn’t solve the TCP window size problem, but takes advantage of what the uplink is capable of handling. Most modern browsers do this automatically, so download accelerators are really not a necessity any more.
You may wish to optimize your per-TCP connection transfer rates though. To do so, determine your optimal TCP window size based on the expected latency of your most bandwidth intense client-base (see the calculator at the above link). Then, based on that, adjust your TCP/IP stack to adjust below:
To tweak Windows 2008 TCP Window Scaling, please refer to the following:
http://www.minasi.com/newsletters/nws0802.htm
http://www.speedguide.net/read_articles.php?id=2574
Note that Windows 2008 doesn’t allow you to tweak settings like 2003 did. You can make the system adjust it “more aggressively,” but you can’t hard code numbers in.
To tweak Windows 2003 TCP Window Scaling, please refer to the following:
http://articles.techrepublic.com.com/5100-10878_11-5034413.html
You may wish to also try: http://www.speedguide.net/tcpoptimizer.php
To tweak Linux TCP Window Scaling, please refer to the following:
http://www.speedguide.net/read_articles.php?id=121
Note that many other factors come into play for bandwidth calculation. In a hosting environment, your server must compete with other servers in the data center to reach the core routers and from there, must concentrate in various nodes and exchanges to reach a packet’s destination. Along the way, routers must prioritize and queue packets for transmission. We can check the health of this process by performing a traceroute between “slow links.” Network congestion at any one of these nodes can reduce overall transfer rate. On either one of the endpoints, disk I/O, or other system stress may be a bottleneck.
All in all, an 100Mbps, or even an 1000Mbps uplink will not provide transfer rates greater than what the network fabric in between the source and destination can handle, and not greater than what the server / client can negotiate within the TCP pipe.
#18 Feb 2010 – Edited for spelling/grammar.
#24 Mar 2010 – Updated link for 2008 tuning.
Slowloris and You
UPDATE: 20090826 – Corrected typo in “Slowloris and You.” It used to say “Slowlaris and You.” I keep getting slowloris confused with my nickname for “Solaris.” =D
Back in July, http://ha.ckers.org/slowloris/ published an exploit against Apache and other web servers (go to the link for further) that takes advantage of multi-threaded applications. It works by tying up web server threads with partial HTTP requests, then sends TCP handshakes to keep the socket open. In general, multi-threaded web servers such as httpd, apache, and apache2 are vulnerable. IIS and most proxies are not vulnerable
CERT suggested using iptables to rate limit incoming port 80 requests. In general, this should be fine for many applications, though CERT has warned that some large clients behind NAT’s may be affected and thus the hitcount/time ratio should be adjusted according to your needs.
http://www.funtoo.org/en/security/slowloris/ offers tips on mitigating this attack by enabling delayed binding on hardware load balancers.
In short, it appears as though the consensus mitigation method involves connection restrictions in the form of iptables or apache modules (most are of limited value, frankly), or shielding the web servers behind load balancers (such as HA-Proxy).
Google Wave Set for Limited Release in September
Well, it looks like after much ado, Google is prepared to release its new Wave product (a revolutionary communication tool that blends the aspects of instant messaging, document collaboration, and e-mail), in September to a limited number of end-users.
Sign up here, and continue to watch for updates here.
If you don’t have to host your own email server, why do it? In fact, if any web facing application on your site can be hosted by dedicated specialists with no real appreciable loss to availability, confidentiality, integrity, or other factor, don’t fall victim of the do-it-just-because-you-can mentality. The chief services I advise folks to outsource are e-mail (IMAP/POP/SMTP), and DNS. These critical services are often in better hands with the likes of Google Apps, with your registrar, or OpenDNS (not necessarily an endorsement as these names are just on the top of my head). These folks can dedicate a lot more fault-tolerant capacity than most, taking charge of security and systems management. On top of that, they sell it mostly likely because they’re good at it (there are no doubt exceptions, but not with the above two links). With configuration and maintenance out of your scope of responsibility, you don’t have to spend countless hours tracing emails or checking DNS proliferation. Get a quote and do an analysis. If the amount of time you save by outsourcing some services is worth the price you pay to do so, then it’s a no-brainer, really.
Note: Speaking of checking all that stuff: some neat online tools: MX Toolbox, DNSStuff, and IPTools are some handy websites in regards to mail or DNS.
