Have ever you
tagged your blog posts with Technorati tags but then not had them show up on
Technorati's tag pages for yonks? And when they do show up eventually (if you're lucky), they're said to apparently have been posted X hours ago, all of them at once, instead of at the times when you actually posted them?
You'll notice too that typically this may happen even if your posts do get indexed on Technorati (though that too is taking longer and longer after a ping), i.e. even when the same posts come up in a standard Technorati search for words in the post. And it may happen even when other posts of yours, before
and after the problem posts, show up fine on the Technorati tag pages.
Now I'll explain below what I think has been going on, and what to do about it. And just to get an idea of the scale of the problem, if you have ever experienced that problem yourself, why not answer the
poll at the end of this post (and please pass the link to this post on to your fellow bloggers) - so hopefully we'll all get a better picture of how widespread the problem is and how many are actually affected.
Summary
If you've had trouble getting your tagged posts appearing on Technorati's tag pages, and you feel you've done everything you should to
tag them in the right way, then the problem is probably down to Technorati, not you.
First, check this: can you find those posts via a simple search on Technorati (e.g. for an unusual word or phrase in the post)?
If no, then Technorati haven't indexed them at all. Make sure all the posts concerned are displayed on your blog's main page and not just archive pages or post pages (for why, see further below) - by, if necessary, tweaking the number of posts displayed on your main page, then
ping Technorati and check again later, hopefully no more than a day at most.
If yes, then there's a problem either with Technorati not picking up the tags for their tag database, or with some particular tags where the posts are in Technorati's database but they are just not showing on the tag pages. The only thing you can do is
email Technorati support (making sure those posts are on your front page in case they need to crawl your blog again. It would be helpful if Technorati could confirm whether they need to re-crawl your blog to fix the tag page problem or whether they can just adjust their database, but in the absence of solid information from them about this, best have them on your main page just in case... details below.)
The details
Well I've had the problem mentioned above, repeatedly, since I started experimenting with Technorati tags (see my
introduction to Technorati tags,
this post and
this post!). At first I thought it was my fault: maybe I'd been tagging wrong, handling multiple word tags wrong, using too many tags per post, etc. Then suddenly it started working though I hadn't changed what I was doing at my end. Ever since, on and off, the problem has kept recurring. In fact, it's happened again - my recent posts weren't on their tag pages for days, then today I see that they are.
Now I'm a persistent chapess, and I've tried hard on and off to get to the bottom of this problem. If the problem's not at my end, which seems to be the case as it'll suddenly start working though I haven't changed anything, then it must at Technorati's end: it must be something to do Technorati's spidering, indexing or handling of their tag pages. And it is.
Why should I care why it sometimes doesn't work? Well if it's down to my something wrong, like using too many tags so that Technorati's crawler thinks my blog is a spam blog and refuses to index it, then I want to know how to fix it: how many tags is it safe for me to use per post before I am considered an evil spammer, etc.
So, I kept trying for a while but just couldn't get a proper answer out of Technorati. Now I'm a big fan of Technorati, and I could understand it if Technorati didn't want to give too much away - e.g. if they told us exactly how many tags would be considered too many, spammers as well as legit bloggers could make use of that information - but if that's the case I believe that it would just be a courtesy to say to users that they won't give the information, and why, rather than ignore the question.
However I suspect the failure to respond and, in some cases, lack of clarity and comprehensiveness in the response, has been due mainly to scarce resources and pressure of work rather than anything sinister (they've indeed had some turnover in their support staff, Technorati's community manager
Niall Kennedy said, and
are hiring if anyone's interested! In that vein I think
any tech support operation would do well to take a leaf or several from
redryder52/Truckspy's book (just check out the
Blogger forum), in fact why not just hire him for huge sums of money and be done with it? Not that I'm trying to unwarrantedly blow anything up his... I mean, blow his trumpet without good reason... I mean, sing his praises for nothing... psssst! we'll chat about the fee later...).
Technorati issues
Anyway, here are some key points to note about Technorati, based on what I've deduced from the hints in the responses I did get (and I gather I've been much better off than many people who haven't had any response at all):
1. Technorati sometimes doesn't pick up tags in your post for their tag pages
This can happen when Technorati are updating their indexes and your blog gets caught up in it (maybe it also happens in other situations I don't know about). Somehow some blogs get left out during this process. If it's yours, tough luck. It's nothing to do with your tags containing multiple words or digits, it's to do what Technorati have been doing.
The only solution:
email Technorati support to report that your tagged posts aren't appearing on their tag pages, telling them the last date when your posts properly appeared on their tag pages and which posts
aren't appearing, and ask them to fix their database or reindex your pages or both. If they don't respond and your posts are still not on the right tag pages in a few days (be kind, give them a little time as they're currently short staffed), then chase them up, and keep chasing every few days until it's sorted out. That's all we users can do, at our end. (Note that if the problem was due to their updating their indexes, often your tagged posts which were published before or after they were doing that work will be fine - it seems to be just the posts from around the time of that work, and just posts from a few blogs too, that are affected - not all of them. But then you need to get the problem posts sorted.)
Suggestion: you should get on to Technorati as soon as you notice a problem, before too many of your posts go off your front page and so might get missed.
Which leads to the next suggestion - before you contact Technorati support, you might want to make sure that your missed posts are all on your blog's main or home page, not just your post or item pages (in my case for example my main page is just at
http://consumingexperience.blogspot.com/), and that the posts stay there for as long as it takes for Technorati to pick them up properly and show them on the right tag pages. (All this assumes that Technorati need to reindex the missed posts in order to fix the tag pages problem - it could be that they might be able to sort it out, if their spider has crawled those pages and they have the posts in their index, simply by tweaking their databases without re-crawling your blog, but I really don't know the answer to that, so unless Technorati confirm the position I suspect the safest bet is to make those posts available for a re-crawl).
How to control number of posts shown on your main blog page - there will usually be a setting in your blog platform for this. With Blogger, in your Dashboard go to the blog in question then click Settings and Formatting and it's the first tiem. With Wordpress, go to Options, the Reading tab, and it's the first item again.
Doing all this may make that page load more slowly, which is a pain for your readers especially if you use a lot of pics or have long posts like me, but (assuming a re-crawl is necessary) you may find you simply have to do it or your posts may not get indexed by Technorati. This takes me to the second main problem with Technorati.
2. Technorati will only index posts (including the related post or item pages) which are on your main blog page
This is not so much a problem as a "feature", as they say - in that it's due to a deliberate decision by Technorati not to crawl or index anything that's no longer on the main page (maybe for resource reasons? I'd certainly be interested to know why). Contrast this policy with Google's - Google
will go back and re-crawl "old" webpages or blog posts, so changed or added text from later edits will be stored in Google's cache, and be fully searchable. Which is what I personally prefer, as I do edit some old posts to keep related information together in the same place. Google will pick up those changes, Technorati won't.
The combination of 1 and 2 is unfortunate. If Technorati are regularly failing to index tagged pages properly or posts are lost from their index or tag databases, which by many accounts is happening, it compounds the problem if (assuming a re-crawl is needed to fix the problem) they won't index "old" posts that are off the front page after their system is working properly again. Bloggers get the worst of both worlds; tagged posts which should have been added to and retrievable from the tag database may never get added. Technorati really should fix that problem - ideally by re-crawling
old posts including those linked to from the main page, where the posts don't get into the tag database because of a glitch at Technorati such as index updating. In the meantime, and certainly if they don't address this problem, all we bloggers can do is keep lots of posts on our front pages, and keep
emailing Technorati support about the issue.
Sidebar links: will Technorati's spider follow links to individual post pages listed in your sidebar? I don't know. I've asked them that several times and they've never answered that question. I suspect not. Pity, because at least it would pick up recent posts that way, given the way many blogs are set up (eg Blogger always seems to list and link to the 10 most recent posts).
3. Technorati do have a way to tackle spam blogs or link farms - that is, blogs with no real content but just lots of marketing or promotional links
I don't know exactly what their method is and how it works, but I gather from what Technorati have told me that some blogs do get labelled as spam by their software, and it seems to have something to do with the number (or density, I am speculating here?) of tags or perhaps of links on your main blog page.
All I can suggest in the absence of further information about the mechanics is, if you think you've been unfairly caught out by this and your blog isn't being indexed by Technorati (whether on their tag pages, or indeed at all), you should
contact Technorati support about it.
4. Technorati's tag pages don't always correctly display what's in their database behind the scenes
An example is the "nominative determinism" tag which I started. As I write this only one post appears on
that tag page. But there are more posts that have used that tag, see e.g. some of the posts under the Humour heading in my sidebar.
Technorati told me that their database is complete, the tags are in the underlying database, but with a few tags like this one their display code is missing them out. The issue seems to be related to specific tags (they're not all multiple word ones either, there's a problem with the
"Wailing" and
"aptonym" tag pages too, I was told) - sometimes the tag page displays all the posts with that tag, and sometimes they don't. They know about it and they're working on it. But this issue doesn't seem to be generally known. Which brings me to another point.
But even for individual tags which might seem OK, there are problems too. For example, I tag all of my posts with "Improbulus". Most of my recent posts
are on the
Improbulus tag page, but two of them from 9 June (see
my archive) aren't there. See for example my post on rules of thumb, which can be found
on searching Technorati for "cognitive heuristics", but isn't on their
tag page for that word. So those pages did get indexed by Technorati - they're just not in the tags database, or they're not displaying properly from that database, and I haven't a clue which it is. It can't be to with that specific tag, because some posts with that tag are showing both before and after those posts. I have emailed Technorati about them and with any luck they'll sort it out soon, but I'd sure love to know what went wrong there.
Problems with tech support? (generally, not just Technorati!)
Customer service is something I feel strongly about, including tech support. The Americans are meant to be way better than the Brits on this front, but not always.
In all sorts of areas, communicating with users is important: particularly when there are severe and continuing difficulties which seem inexplicable to those not in the know. To mangle a phrase, people abhor a vacuum, and in the absence of information will fill it with frustration (which can ultimately rebound on the company concerned, and not in a good way).
It's about time Technorati introduced (and then should thereafter keep updated) "Known issues" and "Status" pages, and have big links to them from their main page and search and tag pages. That would be less work for Technorati too than having to field numerous emails (and risk alienating users by not answering them). And the first thing on the "Known issues" page should be an explanation about the problems with tag pages: exactly what the problem is (indexing? Adding to the tag database? Retrieving info from it?) and how it's being addressed (and is re-crawling of particular blogs needed to fix it once spotted?).
Companies like Technorati should build up a database of issues and queries not just for a "Known Issues" page but also to facilitate quick standard replies to support requests. It's entirely possible for responses to be rapid, resource-efficient and succinct yet sufficiently informative to keep users happy. A lot of the time, most people are quite tolerant and can live with bugs and issues, particularly with a free service, but only if they are kept informed - being ignored just breeds anger and frustration, and even mere resignation may be enough to induce mass desertion if other blog search engines appear with tagging facilities.
Dave Sifry the Technorati CEO said at the end of
his post introducing Technorati's new beta service that feedback comments were the first thing people at Technorati looked at each day. It's all very well and good to have an eye to the future, visionary thinking is indeed laudable, but what about problems in the here and now? Support requests should be equally important, if not more so, as a source of user feedback - e.g. for clues on bugs or issues, such as with tag pages.
Or is it the case that users with problems would be better off emailing Feedback rather than support, in terms of getting their issues addressed?
It's received wisdom that companies should pay heed to their "difficult" users/customers, the ones who have problems, the ones who ask for help, the ones who complain. For each person who takes the trouble to put finger to keyboard, you can bet there will be many others who've suffered the same problem, but just gone away dissatisfied rather than bother to give any feedback to the company. It all adds up. Reading and responding fully and promptly to support emails is one of the best ways for a company not only to get feedback from its users, but to let them know they're listening, that they care about their users, and that they intend - even if not quite yet, due to limited resources and more pressing priorities - to address their issues and problems.
Now that Technorati have launched their
new beta, let's hope that after they have a bit of a breather (which seems much needed after rolling all that lot out!) they'll turn their attention to the tag and tag page problems.
The poll
So if you have views about the tag pages issue, please take part in this poll. I hope that even those who haven't had problems will take part (there's only one question to answer in that case!) as that will help gauge whether in fact only a small percentage of people have had problems (and they're the only ones I've heard from!). I'm not the best person in the world with HTML layout/design, OK I'm useless at it hence all the spaces, and it should be obvious but note it's meant to go from left to right...
Technorati Tags: Technorati, Technoratitags, Technorati tags, tags, tagging, blog, blogs, blogging, support, tech support, customer service, customer issues, Improbulus, A Consuming Experience, Consuming Experience
