Google Webmaster Central Blog has recently announced that Google started to index web pages hidden behind web forms. "In the past few months we have been exploring some HTML forms to try to discover new web pages and URLs that we otherwise couldn't find and index for users who search on Google. Specifically, when we encounter a <FORM> element on a high-quality site, we might choose to do a small number of queries using the form. For text boxes, our computers automatically choose words from the site that has the form; for select menus, check boxes, and radio buttons on the form, we choose from among the values of the HTML. Having chosen the values for each input, we generate and then try to crawl URLs that correspond to a possible query a user may have made. If we ascertain that the web page resulting from our query is valid, interesting, and includes content not in our index, we may include it in our index much as we would include any other web page." For now, only a small number of websites will be affected by this change and Google will only fill forms that use GET to submit data and don't require personal information.

Many web pages are difficult to find because they're not indexed by search engines and they're only available if you know where to search and what to use as a query. All these web pages create the Invisible Web, which was estimated to include 550 billion documents in 2001. "Traditional search engines create their indices by spidering or crawling surface Web pages. To be discovered, the page must be static and linked to other pages. Traditional search engines can not see or retrieve content in the deep Web -- those pages do not exist until they are created dynamically as the result of a specific search."

Anand Rajaraman found that the new feature is related to a low-profile Google acquisition from 2005.

Between 1995 and 2005, Web search had become the dominant mechanism for finding information. Search engines, however, had a blind spot: the data behind HTML forms. (...) The key problem in indexing the Invisible Web are:

1. Determining which web forms are worth penetrating.
2. If we decide to crawl behind a form, how do we fill in values in the form to get at the data behind it? In the case of fields with checkboxes, radiobuttons, and drop-down menus, the solution is fairly straightforward. In the case of free-text inputs, the problem is quite challenging - we need to understand the semantics of the input box to guess possible valid inputs.

Transformic's technology addressed both problems (1) and (2). It was always clear to us that Google would be a great home for Transformic, and in 2005 Google acquired Transformic. (...) The Transformic team have been been working hard for the past two years perfecting the technology and integrating it into the Google crawler.

It's not clear what are the high-quality sites used by Google for the new feature, but this list includes some good options. Along with Google Book Search, Google Scholar, Google News Archive, this is yet another way to bring to light valuable information.

orkut, Google's social network, added a lot of features in the past year to be more competitive and to become more popular outside Brazil and India. Google Trends shows that the interest for orkut is declining, but it's likely that some people no longer search for "orkut" since Google India and Google Brazil added the service to the navigational bar in December 2007. orkut has around 120 million users, up from 50 million users in April 2007, but Brazil (53.99% of orkut users) and India (16.91%) continue to be the countries where orkut is popular, followed by the US (15.13%).

orkut was one of the few Google services that didn't have a mobile interface, but now you can access orkut mobile at m.orkut.com. The simplified interface shows the most recent scraps, updates from your friends, a link to your profile and a search box for orkut users. The mobile interface lets you write scraps, get birthday reminders and respond to friend requests. You can also use shortcuts to access the most important sections of the page.


There's also a new version of orkut for low bandwidth that displays less pictures. orkut will automatically switch to this new version depending on your connection, but you can opt to use it in the settings.

{ Thank you, Darnell Clayton. }

iGoogle added a new feature that lets you share gadgets with your contacts and allow them to edit the content from your gadgets. For example, if you share the To-Do list gadget with your friends or co-workers, they'll receive an invitation to add the gadget to their iGoogle pages and every time someone makes a change it will be propagated to all the instances of your shared gadget.


For now, the list of collaborative gadgets includes: to-do lists, notebooks, crossword puzzles, birthday reminders, but we'll see more interesting ideas as gadget developers can easily add this new feature.

"It's now possible for multiple instances of a gadget - on the home pages of different users - to access the same user-preference data, the part of the gadget state that is hosted by iGoogle. (...) It's a simple sharing model - last write wins, and a reload is required to see changes made by others."

To share a gadget, click on the small arrow from the title bar and select "Share this gadget". For most gadgets, you can invite your contacts to use them and optionally send your settings, but collaborative gadgets have a different dialog:


{ via Google Blog }

Google Notebook updated the editing toolbar and to make it look more consistent with Google Docs. There are also new exporting options:

* you can export to HTML any notebook, not just public notebooks

* notebooks that include addresses can be visualized on a map and converted into a personalized map

* public notebooks already had feeds, but it's much easier to subscribe to the feeds by clicking on "export as RSS" in the Tools menu (ironically, Google uses the Atom format). This should be useful to track the changes in collaborative notebooks.


Google Notebook integrates with a lot of Google services: you can find a "Note this" option next to Google's search results, there's an option to import some text from public domain books in Google Book Search, Google Bookmarks are added to a special notebook, you can export a notebook into Google Docs to create a document or to save it as PDF, the latest version of Google Toolbar for IE integrates with Notebook, there's an iGoogle gadget and a cool integration with Blogger thanks to the hAtom microformat. Google Notebook is actually a web clipboard, a bridge between web applications and I expect to see options to identify structured content, bookmark videos, events, contact details, code snippets and more.

{ Thanks, AniMatrix. }

YouTube Fast Search is a cool web application that uses YouTube's API to create a way to both play videos and search for new videos at the same time. You can build a playlist from search results and add new videos to the playlist without interrupting it. The interface is similar to the one from MSN Video, but you can also edit the playlist and enlarge the video. Too bad that there's no option to save the playlist to your YouTube account or to import an existing playlist.


YouTube has the option to add videos from search results to a Quicklist and then play all the videos from the list, but you need to switch from the view mode to the search mode.

{ via Google Code }

Viewfinder is an interesting technology that wants to bring photos to a software like Google Earth and display them as part of the satellite imagery.

"Geotagged photos, geographically indexed on a world map, either manually or via GPS, are an increasingly popular phenomenon. However, current implementations treat maps, and particularly 3D models, in fundamentally different modalities than photographs. The result is that photos tend to hover like playing cards, seemingly suspended over the world, remaining 2D objects in a 3D environment, and negating the transformative experience that we think should occur when combining images and a 3D world. (...) It's possible to place a photo in a 3D model in such a way that it appears seamlessly aligned with the model."

Google has already bought Panoramio, a Spanish photo sharing site that selected around 3 million geotagged photos to be added in a Google Earth layer. In October last year, Flickr had more than 42 million geotagged photos. All these photos could be used to compose a more accurate representation of the world. Combining this with projects such as Microsoft's PhotoSynth should result in new exciting ways to explore the world.


{ via BoingBoing }

Update: More about Viewfinder in this Google techtalk.

YouTube updated the design of its video watch pages by adding tabs for actions and showing statistics in a new container. The "Favorite" tab is the only one connected to an immediate action and it should be removed, while the favorite videos could be generated based on ratings. Hunter Walk, product manager at YouTube, says that the sharing options "are now contextual to the logged in user, so for example, if you use Digg a lot but not Facebook, Digg will be elevated to a persistent top-level display instead of Facebook".

The search results page has also been updated and the option to sort results by popularity is back. To restrict your search to one or more categories or to a certain language, check the new advanced search.

YouTube's personalized homepage lets you reorder the sections so you can place the featured videos at the top, but it's still less usable than the classic YouTube homepage.

In what may be the beginning of a new partnership, Yahoo announced that it will display Google ads next to its search results in a limited experiment. "The test will apply only to traffic from yahoo.com in the U.S. and will not include Yahoo!'s extended network of affiliate or premium publisher partners. The test is expected to last up to two weeks and will be limited to no more than 3% of Yahoo! search queries."

Yahoo explores new ways to defend itself against Microsoft's assault. "While there has been some limited interaction between management of our two companies, there has been no meaningful negotiation to conclude an agreement. We understand that you have been meeting to consider and assess your alternatives, including alternative transactions with others in the industry, but we've seen no indication that you have authorized Yahoo! management to negotiate with Microsoft." - an extract from a letter sent by Microsoft to Yahoo's Board last week.

While switching to Google's search ads means admitting the defeat, Yahoo could use this strategy temporarily. Yahoo has a lot of interesting projects in the mobile space, in search and communication, so the future looks better without Microsoft, a company with a different culture.

"If this test — potentially within the three-week window Microsoft set up to launch a proxy fight — pans out and Yahoo also manages to come through with a first-quarter report on Apr. 22 that looks stronger than the current low expectations of analysts, suddenly Yahoo will have some leverage it didn't have before. If the quarter tanks, though, advantage goes back to Microsoft," anticipates Business Week.

Update: Some interesting new developments. WSJ reports that "Yahoo Inc. and Time Warner Inc.'s AOL are closing in on a deal to combine their Internet operations". Google already provides search ads for AOL, so Yahoo's test with Google ads is just "a way for Google to test how well Yahoo monetizes", to quote Danny Sullivan.

iGoogle, previously known as Google's personalized homepage, offers two options to backup your gadgets and feeds at the bottom of the settings page.


There's an option to backup the iGoogle page on Google's servers. While this may seem pointless, it's useful to backup your page before adding gadgets that might create problems.

The second option is to download an XML file that includes all the tabs, layouts and themes, the feeds, gadgets and their settings. The XML file can be imported in any iGoogle page, so this is a good way to migrate the page to a different Google account. You can also download the XML file to change the settings in a text editor and then upload the new version.


{ via Blogoscoped Forum }

Google Presentations (or Presently) finally added a feature that prevented most people from using it: exporting the presentations to a format that could be imported in Microsoft Office or OpenOffice. Now you can save your presentations as PPT and open them in your favorite desktop applications when you're offline or send them to other people. You can still export the presentations as PDF, a great format for printing.


Here's the list of formats supported by the three Google Docs applications:



Some other features that should be available soon in Presently: inserting tables, adding YouTube videos and viewing presentations offline.