The Internet's seething Web of content resembles endless bubbles popping to the surface for only a day, then vanishing, a security study from Blue Coat Systems released today indicates. That means there are a huge number of new, unknown and transient sites daily, posing challenges to determine whether they are benign, or should be blocked as dangerous.
Blue Coat analyzed 660 unique hostnames in a 90-day period, finding fully 71% of all the them appeared for only a 24-hour period. "To me, it was pretty surprising to know the Internet is expanding like a universe growing but these sites are only around for a day," said Dr. Hugh Thompson, chief security strategist at Blue Coat about the report "One-Day Wonders: How Malware Hides among the Internet's Short-Lived Websites."
+ Also on NetworkWorld: Cybercrime wave whacks European banks+
The security firm's study found the vast majority of the Web's "one-day wonders" were associated with processes that Google, Amazon and Yahoo appear to utilize for content delivery. "The shared trait among these type of organizations is the use of Content Delivery Networks," the report says. "The speed and reliability CDNs provide are essential to their operation. It appears these organizations use unique subdomains (and sub-sub-domains) to keep track of content in the CDN. This could be to identify a particular user, session or request, and once that user/session/request is finished, the sub-sub-domain isn't used again. A by-product of these CDN architectures is the proliferation of One-Day Wonders."
Other major drivers of one-day wonders appear to rise from web performance optimization, or blogging sites such as Blogspot, Tumblr and Wordpress, not to mention the porn site xvideos.com for video sharing, which according to the Blue Coat report is "considered to be the most popular pornographic website in the world."
The U.S. and China dominated as ranked by IPv4 addresses and URLs generated, accounting for about 40% of the hostnames in this sampling, which were requested by 75 million users. In all of this change, the challenge is determining which are being used for malicious purposes and which are not.
Of the top 50 domains, 22% were identified as malicious, the report says. One in the .info domain, for example, was simply a command-and-control server for a Trojan dialer that had more than 1.3 million sub-domains. There were about a dozen more like this that could be discerned.
"They want to keep the good guys guessing," Thompson points out, noting it's intended to throw off spam and web filters by hiding in this on-going proliferation of one-day wonders.
Blue Coat says that with such rapid building up and tearing down of new sites for unknown purposes, there's a need to not rely on just black lists but to understand context around domains and IP addresses, as well as pattern discovery related to finding a "baseline of transient hostnames."