Exalead Blog - The blog of Exalead

Jean Marc brought you a very delightful post about Chromatik last week with a lot of beautiful images. I will now describe in more detail how it was built. As with the DVD you perhaps watched last night, I am afraid there will be fewer big special effects in this blog post than in Jean Marc’s post, but I hope to give you an insightful view of what happened behind the scenes.

Chromatik was an elaborate demo, the result of a long effort on both the back-end and the front-end. It indexes one million images. For each image, a unique color signature was built and indexed. Our current intuitive user interface, exploits this index to help you filter and select images by choosing a combination of colors, luminosity or text.

A large number of people tried and liked the Chromatik demo so much that we received several requests to integrate it into the official Exalead search site. And because the demo ran relatively bug free and smoothly, our friends thought it was a piece of cake. Of course, it was a bit more work than we initially expected. So where are the challenges?

1) The front-end side

A lot of questions needed to be answered:

• How will I adapt the GUI of my application to integrate the new features?
• Are all these new features necessary?
• What is the feedback we’ve received on the different features?
• What is the added value of these features?

The answers to these questions will impact the total amount of space on the GUI we will take for surfacing them.

2) The back-end side

Let’s begin with a little theory:

Theorem of the factor 10 effect:
No matter how good a developer you are, if non-trivial code has been designed and tested with only N elements, it won’t work without modifications when applied to 10 * N elements.

Demonstration: Rather simple: if you don’t believe it, try it yourself…

In this case we wanted a factor 1000, so we knew it would need some adjustments. When you know this theorem, the advantage is that you can anticipate potential problems. And the experience we have accumulated from similar situations at Exalead help us predict most of the bottlenecks.

Example 1: Chromatik needed 300MB RAM, which is quite good for 1M images. But, if you multiply this number by 2000, you have 600GB RAM, which is quite large, even if the final index is distributed over multiple machines.
We therefore decided to reduce the richness of the colors, while maintaining usability, migrate from version 4.6 to version 5.0 of Exalead CloudView, and use a more compressed encoding. In the end, it now only costs 9GB.

Example 2: When you want to analyze two billion images, you need to have a robust code, which means that’s able to handle all sort of images even those which do not have a valid RFC. It’s not that easy, when even the most used library in the world for basic image manipulation can crash on some images as we reported.
The result was that this run spotted some bugs in our code we hadn’t seen before and therefore had to fix.

Example 3: The demo was initially a single machine application. We needed to use the distributed system framework included in the CloudView technology to be able to run the whole process of extracting, crawling, and indexing in only a few weeks. This framework really helped us transform the single machine demo to a fully load-balanced and monitored application. This use case is a little different than our standard www.exalead.com chain, so we discovered and tweaked a few cumbersome points in the code.

The purpose of this integration was to offer a new service to the users of the exalead.com search engine and improve the robustness of the Chromatik technology. We now better understand the impact of different tweaks on color indexing.

Transforming a demo into a real product is not as easy as it seems. I hope this post helps you understand why a lot of companies only show you demos but never real live applications.

At Exalead, we don’t sell demos to our customers; we sell tested and robust solutions. We make sure we work hard to test and uncover all the issues so our customers’ implementations go smoothly.

Many of you have already tested Chromatik, the color-based image search demo from our research laboratory, Exalabs. In line with our commitment to incorporate the best of Exalabs technology into our public search engine, we have now integrated Chromatik functionality into the Exalead Image Search. See the new color palette options on menu of the Image Search results page:

The tool features a palette of 8 colors. Simply click on a color to filter results according to your color of choice:

You can refine your selections even further by selecting additional colors from the palette, helping you locate images with specific combinations of colors:

There’s no limit to the colors you can mix and match!

The last option on the color palette menu lets you narrow your results to black and white images only, a feature available on the prior version of our Image Search:

This is only the beginning! We’ve already begun work on incorporating other Chromatik features into our Exalead Image Search. So be sure to check back soon, and in the meantime, we’d love to receive your feedback on feedback.exalead.com.

More Mappable Data, and Map-Hungry Users
More and more information on the Internet is ‘mappable’:

• The physical addresses of offices or stores

• Wikipedia articles (country/city info, geo-science articles, etc.)

• Photos and videos with content of local interest on “Web 2.0” content-sharing sites

• POI (Point Of Interest) services like tourism portals (attractions, hotels, etc.)

Naturally enough, numerous services have recently appeared allowing Net surfers to view search results on a map, bridging the gap between the ‘real world’ and Web content. No longer satisfied with viewing mappable information presented in an old-fashioned laundry list of search results, users today are hungry to search for information within and through a map. And Exalead is eager to join users in exploring the boundaries of map-based search. We’ve therefore added a new geo-spatial search demo, Sourcier, to our Exalabs site.

Map-Based Search with Exalead
Sourcier was produced in collaboration with France’s leading public earth science institution, BRGM (Bureau of Geological and Mining Research). It provides map-centered search for metropolitan France groundwater resources using publicly available data from the ADES site.

Sourcier enables users to search water resources and water-related points of interest (along with associated scientific data) according to:

• A chemical component, environmental factor, or micro-biological element

• A concentration range

• A time period

• A geographical area, by navigating the map using drawing tools, or specifying an administrative area (region, county or municipality)

Results are displayed directly on the map along with markers for exploring related data in a graphical form.

To Come…
Forthcoming indexation and geo-search experiments using Exalead technology will involve support for more complex geometric objects and distance calculations to support extrapolation to different geometric shapes (for example, using a specified radius to manipulate maximum distance).

The indexation of geometric objects  such as polygons would permit one, for example, to search London parks and reuse their geo-spatial ‘fingerprints’ on the map as search criteria, and to retrieve related Web data like geo-tagged photos or videos.

Sourcier is available at: http://sourcier.labs.exalead.com/

(Note: the service is currently available in French.)

It’s been an interesting week further exploring the topic of Social Networking and Customer Service. Having a background in database, BI and now search technologies, I have been through my fair share of hype cycles, so am working on getting a better sense of the current state of the application and technology space. It is little surprise that Gartner places Social CRM: Customer Service at the peak of the hype cycle for Social Software (2009) with 2 to 5 years lead time to mainstream adoption and the related Social CRM: Community Marketing as on the rise with 5 to 10 years to mainstream adoption.

There is certainly no shortage of debate on the definition, scope and relative viability of social CRM (SCRM). Check out Social CRM: Strategy, Technology or Passing Fad? by Bob Thompson, for a recent opinion and some cross references to others’ thoughts. It’s clear that interest and momentum is high but plenty of room remains for clarification. I, for one, would like to see more in the way of reference use cases across CRM and adjacent functions to better understand processes, data usage and interdependencies. I just can’t imagine adoption hitting stride until companies can see how functional pieces and related data need fit together. All you have to do is take a look at Gartner’s rather crowded Social Software hype cycle to get a sense of potential process and data overlaps and the accompanying urge to tap the breaks a bit. Paul Greenberg, a regular contributor to ZDNet, promises more discussion on related business models and strategies..

We see that companies are beginning to get a sense of the social conversations that involve their products and services. A subset of those companies is having direct conversations with customers through social media channels. For example, one of my good friends does PR for a video game company and is crushed by the need to address misinformation and negative sentiment and to bring some control to the conversation about his clients. We look forward to going beyond the hype – seeing more examples of how companies are really building the processes and technologies around social CRM.

At Exalead, we’re working on Social CRM related technologies. Constellations is an Exalead Labs experiment that looks at connections between people and between web sites. It presents a graph of related Web sites or people.  The experiment is built upon our Web index. The approach to relating people and things is flexible and can be based upon a configurable set of factors.

Well, as some of you have guessed what has been cooked up and others were taken into our confidence… here it is!  The new version of our Exalead search engine, on-line today.

While concerned with promoting innovation in the web search engine sector, Exalead’s teams are proud to present you with this new version that is only the beginning of a new adventure.

Mostly based on a graphic overhaul of the engine, this version certainly provides new features and functions.  Some of these further respond to several suggestions that you have communicated to us via the Feedback space.

Among the new features and functions:

•    Auto-completion of the query, suggesting associated terms from the index that help you formulate your search

•    Associated searches, beneath the search bar, that allows you to re-launch a query in line with the preceding one,

•    Refinement by « languages » and by « country », presented in the form of a pie chart,

•    Display of thousands of images on the same search page and the repositioning of the refinement window, for an optimized presentation of results,

•    Customization tools for searches more adapted to your needs.  By clicking on the “tools” icon on the top right of the window, you can choose search refinement categories that you want to appear.

You will also notice that the sponsored links are displayed in a less intrusive manner than before, for a more comfortable read.

Rendez-vous from today forward on exalead.com/search to test it yourself!
And above all, don’t forget to send us your suggestions for improvement and discussion with other users on feedback.exalead.com!

We have a search engine for videos and images, and there is always room for another to add more variety to each.

When it comes to the Web, Twitter has a lot coming its way and hence we have Tweepz: a search engine to find people on Twitter based on Exalead’s core technologies.

Simply enter the name, bio of a person you are looking for and Tweepz brings out the closest possible matches.

Demonstrating the capability of Exalead’s powerful technology, Tweepz, created by Exalead’s Netherlands Team (Jochem Prins), was conceived in only a couple of days. Congratulation Jochem

Tweepz is currently indexing just over 2.5 million Twitter users (follow @tweepz to be added to the database) and aims to make finding people who interest you an easier task.

Tweepz not only lets you search by geography, biography or name, but also hopes to index the content of Twitter users’ nominated URLs in the future, thus offering a larger scope and capacity to return meaningful search results.

But Tweepz takes it further than that!

You can also filter your results. You can narrow your results by number of followers, the language they speak or extracted entities (key words which show up in their bios) all of these divided into percentile ranges.

RSS feeds for specific queries are available, as is the simple ability to tweet the results you are seeing.

Test Tweepz.com and give us your feedback!

BAAGZ version 1.1 has been released recently.

On the agenda : bug fixes (”error 500″ problems are solved) and a bunch of new funny features.

- Massive upload for images

Stop uploading pictures from your computer one by one. You can now select several files at once and save time. It is a invaluable tool to create image galleries.

- Gallery

You can now add one (or more) image galleries in your baagz. Customize a gallery the way you want by changing picture size, borders for each photo or display order.

New borders have also been added. Do not hesitate to play with them to create nice effects in your baagz.

Tip : try and click several times on a border icon and see what happens

- Wall

To give your visitors the ability comment on your creations, you can add a fully customizable wall of comments anywhere in your baag.

Have fun with those new features and do not hesitate to leave a message to show us the baag you’re especially proud of !

Exalead Image Search has just received a major update: a new user interface and 2 billion images.

The brand new Exalead Image Search stands out for its freshness (you’ll find images from the news), and its user interface that is both simple and efficient.

With Exalead Image Search:

• save images on your hard drive in a single click
• organize your Favorite Images your way
• leverage your wide screen, the number of search results adapts to the biggest resolutions
• refine your search in a click, to search only faces for instance

Exalead Image Search offers an extremely clear view of your search results: only thumbnails and sizes are displayed. Detailed information appears when your mouse hovers a thumbnail.

The right column (a.k.a Zapette) is gone, giving even more space to search results. Refinement features are displayed upon focus of the search input field.

Try Exalead Image Search and drop your feedback.

We have been waiting for it for some months: the latest creation from Exalead’s R&D department is finally going to be released from its beta format.

After a real facelift, BAAGZ, the new community site, allows you to discover and share images, videos and articles on your favorite subjects. All of this, in a completely personalized fashion.

Discover

Passionate about sports, fashion or another topic, BAAGZ is the place where you will find the most interesting and up-to-date information, organized around thematic communities.

Share

Publish your ideas on rich pages, illustrate your topics with images and videos. Allow your friends to enjoy it and the rest of the community.

Participate

Comment on the news, meet others passionate about the same subject, vote for the best content, and express yourself in forums.

Stay connected

Select your favorite communities and receive information in real time on your profile page.

Create a community

BAAGZ gathers more than 10 thematic communities. You didn’t find what you were looking for? Suggest it and invite other members to participate.

Easy to integrate into your existing site and built on robust technologies, the BAAGZ platform allows you to construct your own communities and display your existing content.

Come and test the new application, available as a white label for web portals and and please do not hesitate to give us your feedback !

Exalead one:desktop software uses Exalead search technology to find almost any type of file on your PC – mail, documents, charts, presentations, media files… With its exalead.com look, this software deeply integrates within your customized exalead.com interface.

Now, you can search the Web, your PC, images, Wikipedia and videos from one single entry point FOR FREE! The same way you search on exalead.com, you can refine your search by date, related terms, file type, source, language, file size, sender, recipient…

New with one:desktop 4.6 :* Intuitive navigation of search results with the Exalead web search engine interface
* Rich preview of virtually any kind of document
* A “Search and act” system: users can “act” on a retrieved document directly from the results page
* A Right Column that permits users to manage the results page: choose the categories and the order of display
Do not hesitate to try this exalead one:desktop free edition and read this Quick User Guide and tell us about your user experience !The Press Release is available here.

exalead one:desktop also exists in a professional edition to search on your company’s Intranet and more. Click here for more information