Chromatik was an elaborate demo, the result of a long effort on both the back-end and the front-end. It indexes one million images. For each image, a unique color signature was built and indexed. Our current intuitive user interface, exploits this index to help you filter and select images by choosing a combination of colors, luminosity or text.

A large number of people tried and liked the Chromatik demo so much that we received several requests to integrate it into the official Exalead search site. And because the demo ran relatively bug free and smoothly, our friends thought it was a piece of cake. Of course, it was a bit more work than we initially expected. So where are the challenges?

1) The front-end side

A lot of questions needed to be answered:

• How will I adapt the GUI of my application to integrate the new features?
• Are all these new features necessary?
• What is the feedback we’ve received on the different features?
• What is the added value of these features?

The answers to these questions will impact the total amount of space on the GUI we will take for surfacing them.

2) The back-end side

Let’s begin with a little theory:

Theorem of the factor 10 effect:
No matter how good a developer you are, if non-trivial code has been designed and tested with only N elements, it won’t work without modifications when applied to 10 * N elements.

Demonstration: Rather simple: if you don’t believe it, try it yourself…

In this case we wanted a factor 1000, so we knew it would need some adjustments. When you know this theorem, the advantage is that you can anticipate potential problems. And the experience we have accumulated from similar situations at Exalead help us predict most of the bottlenecks.

Example 1: Chromatik needed 300MB RAM, which is quite good for 1M images. But, if you multiply this number by 2000, you have 600GB RAM, which is quite large, even if the final index is distributed over multiple machines.
We therefore decided to reduce the richness of the colors, while maintaining usability, migrate from version 4.6 to version 5.0 of Exalead CloudView, and use a more compressed encoding. In the end, it now only costs 9GB.

Example 2: When you want to analyze two billion images, you need to have a robust code, which means that’s able to handle all sort of images even those which do not have a valid RFC. It’s not that easy, when even the most used library in the world for basic image manipulation can crash on some images as we reported.
The result was that this run spotted some bugs in our code we hadn’t seen before and therefore had to fix.

Example 3: The demo was initially a single machine application. We needed to use the distributed system framework included in the CloudView technology to be able to run the whole process of extracting, crawling, and indexing in only a few weeks. This framework really helped us transform the single machine demo to a fully load-balanced and monitored application. This use case is a little different than our standard www.exalead.com chain, so we discovered and tweaked a few cumbersome points in the code.

The purpose of this integration was to offer a new service to the users of the exalead.com search engine and improve the robustness of the Chromatik technology. We now better understand the impact of different tweaks on color indexing.

Transforming a demo into a real product is not as easy as it seems. I hope this post helps you understand why a lot of companies only show you demos but never real live applications.

At Exalead, we don’t sell demos to our customers; we sell tested and robust solutions. We make sure we work hard to test and uncover all the issues so our customers’ implementations go smoothly.

Press photography is about to undergo fundamental changes in its business model. Up until now, it has been “buy photo content, store content, wait for client, sell photo”. They have seen the signs, or possibly heard Clay Shirky’s message about the death of old business models in the newspaper industry

…there is one possible answer to the question “If the old model is broken, what will work in its place?” The answer is: Nothing will work, but everything might. Now is the time for experiments, lots and lots of experiments…

Press photography is soon to experiment with a new business model: moving from content owner to content producer and distributor, moving from photos to videos.

They plan (I learned this last night from a friend in the photography world) to no longer wait for customers to come buy their photos, but rather to harvest the vast amount of locally produced, real time news images and to use their photojournalistic experience to filter and edit and package to produce their own content: story, video, editing. They plan to sell or give away a packaged widget to (online) news outlets. Activating the widget will send control back to the press photography site, which then streams its story and manages it own advertising revenue from the stream.

What does this shift portend for information retrieval?

Information retrieval began as text based storage, gradually incorporating other formats, including structured textual information from databases.

Then search engines began to index photos and videos. Google added its Image tab to its home page around October 9, 2001, and its Video tab around August 14, 2006. We added our Exalead Video search tab about a year later.

These first video search systems used the same methods as with image search, using the text found around the raw data sources, and the text in file names, and in links pointing to the multimedia file, to serve as index terms.

But as more and more information may only be presented in video form, it will be necessary to index the content of videos, too. We do this in our experimental video search system, VoxaleadNews.

But what I didn’t realize until last night is that systems like Voxalead might not just be some cute add-on to a search engine, but that this might actually be what future search engines are: that information might only come in video, and that things like text and even still images, will become some quaint, outdated media, like engravings are now.

An innovation produced by Exalead’s Exalabs research laboratory, Voxalead News lets you search for keywords inside videos, rather than simply searching limited external information like titles or descriptions. A tremendous timesaver, Voxalead further lets you jump right to the point in the video in which your search term is used! The Voxalead demonstration currently offers search in four languages (English, French, Mandarin Chinese and Arabic) across a select set of news sources.

In addition to captivating news aficionados, this ‘most practical’ innovation is capturing the attention of business professionals in sectors such as Media, Publishing, eDiscovery, Competitive Intelligence, and Social Networking and Content Sharing.

For more information, visit the Multimedia Grand Challenge 2009 Web page. You can also test drive Voxalead News and other Exalead innovations at the Exalabs site.

• The physical addresses of offices or stores

• Wikipedia articles (country/city info, geo-science articles, etc.)

• Photos and videos with content of local interest on “Web 2.0” content-sharing sites

• POI (Point Of Interest) services like tourism portals (attractions, hotels, etc.)

Naturally enough, numerous services have recently appeared allowing Net surfers to view search results on a map, bridging the gap between the ‘real world’ and Web content. No longer satisfied with viewing mappable information presented in an old-fashioned laundry list of search results, users today are hungry to search for information within and through a map. And Exalead is eager to join users in exploring the boundaries of map-based search. We’ve therefore added a new geo-spatial search demo, Sourcier, to our Exalabs site.

Map-Based Search with Exalead
Sourcier was produced in collaboration with France’s leading public earth science institution, BRGM (Bureau of Geological and Mining Research). It provides map-centered search for metropolitan France groundwater resources using publicly available data from the ADES site.

Sourcier enables users to search water resources and water-related points of interest (along with associated scientific data) according to:

• A chemical component, environmental factor, or micro-biological element

• A concentration range

• A time period

• A geographical area, by navigating the map using drawing tools, or specifying an administrative area (region, county or municipality)

Results are displayed directly on the map along with markers for exploring related data in a graphical form.

To Come…
Forthcoming indexation and geo-search experiments using Exalead technology will involve support for more complex geometric objects and distance calculations to support extrapolation to different geometric shapes (for example, using a specified radius to manipulate maximum distance).

The indexation of geometric objects  such as polygons would permit one, for example, to search London parks and reuse their geo-spatial ‘fingerprints’ on the map as search criteria, and to retrieve related Web data like geo-tagged photos or videos.

Sourcier is available at: http://sourcier.labs.exalead.com/

(Note: the service is currently available in French.)