In large mass-production systems like Flickr, automatic thumbnails are unavoidable, and my point is not that they should never be used. Instead, my point is that, on many levels, pictures require more human input than text to make them optimally usable. Pattern recognition — the simple observation that the thumbnail of a picture of a sunset SHOULD CONTAIN THE SUN — is something that the human brain does easily, but this does not come naturally for a computer.

Another sort of problem in automatic production of thumbnails is making a thumbnail by simply reducing the size of the large picture. If the main subject of the picture is relatively small, it is not visible in a small thumbnail.

The picture to the left is from the Hardin Library ContentDM collection. The inset in the upper right shows the thumbnail that’s generated automatically by the system, which does a poor job of showing details of the picture. The lower inset shows a thumbnail made manually, which gives a much more clear view of the central image in the picture.

Cropping of a picture to produce a thumbnail, as done here, takes more subtle human judgement than the case with the Flickr picture in the first example, where the weakness of automatic production is obvious. With cropping, there’s inevitably a trade-off between showing the whole picture in the thumbnail or showing the most important subject of the picture. In cases such as this one from ContentDM, where most all of the detail in the picture will be lost in a small thumbnail, it seems better to focus on a central image that will show up in the thumbnail.

Finally, a few examples from Hardin MD, below, show how we have done cropping to improve the detail in our thumbnails. The thumbnails on the left in each of the three pairs are made by simply reducing the size of the full picture. On the right in each pair are the thumbnails we use, that we have made by cropping the full picture before making the thumbnail.

The biomedical, scientific pictures that we work with in Hardin MD are fairly easy to make thumbnails for, because they generally have a well-defined focus, that’s usually captured well by automatically-generated thumbnails. More artistic, humanities-oriented pictures, such as the ones discussed here from Flickr and ContentDM, however, often have more subtle subjects, that benefit from the human intelligent touch in the production of thumbnails.

In pre-Google days, the task of finding good lists of web links needed the input of smart humans (and Hardin MD was on the cutting edge in doing this). Now, though, Google Web Search gives us all the lists we need.

Pictures are another story — on many levels, pictures require much more human input than text.

The basic, intractable problem with finding pictures is that they have no innate “handle” allowing them to be found. Text serves as its own handle, so it’s easy for Google Web Search to find it. But Google Image Search has a much more difficult task. It still has to rely on some sort of text handle that’s associated with a picture to find it, and is at loss to find pictures not associated with text.

The explosive growth of Hardin MD since 2001 (page views in 2008 are over 50 times larger) has been strongly correlated with the addition of pictures. This time period has also gone along with the growing presence of Google, with its page-rank technology, and this has come to make old-style list-keeping, as had been featured in Hardin MD, less important.

Though Google has accomplished much in the retrieval of text-based pages, it’s made little progress in making pictures more accessible. Google Image Search is the second most-used Google service, but its basic approach has changed little over the years.

The basic problem for image search is that pictures don’t have a natural handle to search for. Because of this it takes much more computer power for the Google spider to find new pictures, and consequently it takes much longer for them to be spidered, compared to text pages (measured in months instead of days).

Beyond the problem of identifying pictures there are other difficult-to-automate problems for image search:
• How to display search results most efficiently to help the user find the what they want — Do you rank results according to picture size, number of related pictures at a site, or some other, more subjective measure of quality?
• What’s the best way to display thumbnail images in search results?
• How much weight should be given to pictures that have associated text that helps interpret the picture?

So — Good news for picture people! — I would suggest that pictures are a growth sector of the information industry, and a human-intensive one. I would predict that text-based librarians will continue to be replaced, as computers become more prominent. But there will continue to be a need for human intelligence working in all areas relating to pictures, from indexing/tagging to designing systems to make them more accessible.