HitTail, The Long Tail, and Beta Software

HitTail logoI’ll admit it, I am much more interested in creating content and in reading and learning about new material than I am in digging around in the stats and analytic data associated with any of my Web sites. I know that’ll surprise some of you because I talk about the importance of being at least a closet “quant” so you know what’s popular on your site and what it tells you about your audience, but the fact is that I just don’t really enjoy digging into the arcane world of traffic and log file analysis applications. So I don’t.
But this still leaves me sporadically wondering how people are finding my site and what pages are most attractive to them. As a result, when I bumped into a reference for new startup service HitTail.com on my friend Chris Pirillo‘s “braintrust” mailing list, I was intrigued. A third party traffic analysis system that was inspired by Chris Anderson’s thought-provoking The Long Tail and was free?
Hey, what’s not to like?
So I signed up my busy Ask Dave Taylor Tech Support site, let ‘er rip for a week or two and slammed into the “beta” wall…


First off, I have to give you a sense of the traffic level of the Ask Dave Taylor site, because sometimes I watch it and am pretty amazed at how much data is zipping off that server 24×7. The site sees upwards of 25,000 page views every day representing at least 10,000 unique visitors. Looking at hits you get a fairly impressive rolling average of 12,500 hits/hour, 207 hits/minute, or, if you really want to go there, 3-5 hits per second. On a monthly basis, the site has well over 250,000 unique visitors.
But then again, I’m guessing with unique visitor counts because, well, you already know I’m not paying such close attention to these sort of analytics. I mean, really, does it matter whether I see 2,000, 20,000 or 200,000 unique visitors in a given period of time?
Anyway, HitTail isn’t focused on helping me figure out unique visitors to a Web site, it’s focused on helping identify which search phrases are used by visitors and then analyzing them to identify the “long tail” results, rather than just the most popular. Of course, it does show you the most popular too, but that’s rather incidental to its purpose even if it’s likely some of the most interesting to its average user.
However, it’s still in beta, and it’s a real drag to learn that my site is too busy and generates too much data for their system to work and produce its recommendations for optimal “long tail” topics to focus on as the site grows.
When I log in, the traffic display itself works (and it’s a very attractive presentation at that):

HitTail Traffic Report for AskDaveTaylor.com

You can see that the top ten searches only account for 3.4% of the 159,007 unique keywords collected (I disabled HitTail at that point since there’s no point in collecting more data when I can’t use their tool to analyze it) and are presented in red. The basic premise of HitTail is that it’s the other 96% of my traffic that I should be exploring, the green bars in the figure. Indeed, the actual page on HitTail scrolls down and down and down, showing thousands of popular keywords in reverse popularity.
The problem is, if I want to do any data analysis of their information at all, I hit the “beta wall” and am told that the system is “loading data…” until, finally, I just give up. The tab of greatest interest, of course, is Suggestions and when it worked, it was darn interesting!
I emailed the HitTail folk and asked them what was going on and here’s their explanation:

“Yes, we are aware of the problem that some over our most high-search-traffic-volume domains might be experiencing. There is a very large amount of information accumulated for your site and this can cause delays when retrieving your data.

“We are working on fixing this issue right now and will let you know about the progress as soon as possible. We apologize for the inconvenience and thank you for your patience.”

I’m looking forward to them working out the bugs since HitTail definitely seems cool and worth checking out, but meanwhile I am forcibly reminded of just how much beta software we all rely on to run our businesses and work in the brave new “Web 2.0” world.

3 comments on “HitTail, The Long Tail, and Beta Software

  1. We’re all over it!
    You’re in elite company, Dave. There are only about 15 beta testers who are taxing our system, and you’re one of them. And it’s not on the tracking that we’re having the difficulty (we’re collecting away at full speed), it’s on surfing the data once you’ve logged in. HitTail is going to remain free for all sites under a as-of yet undetermined traffic level, and the beta site is showing us just what that level is. Thanks for that 🙂
    On a related none, performance drop-off is quite equitable. Heavy volume sites have no impact on performance for lower-volume sites. We can sign up a nearly unlimited number of small sites, and the heavy sites like AskDaveTaylor don’t slow them down (good news for them, but not for you).
    But don’t despair! Fixing this high volume site issue is our very top priority. It was a pleasant coincidence to see your post today. We plan on accommodating the heavy traffic HitTailers just as well as the long tail of littleun’s.
    Can’t give time-frame, but suffice to say, you’re very much on our minds.

  2. Just a reminder for those coming here from the Yahoo Finance link, HitTail is a site for small to medium traffic sites. Dave’s very popular site only slowed down the user interface for him. Small and medium size sites have very snappy responsiveness.
    None of this is a bad thing. We’d love to support high volume sites, but it actually worked as designed.
    I encourage everyone to give it a try, and yes, we are getting our premium services in order for high volume sites like Dave’s who wish to partake. We like that too.

Leave a Reply

Your email address will not be published.