Clive Thompson, in his recent comments on how crowdsourcing has the potential to transform eBooks, refers to a a rudimentary form of crowdsourcing that’s already being studied in print textbooks. The work he’s referring to is by Cathy Marshall, who finds that used-book-buyers place value in the annotations (highlighting and notes) left in the books by previous owners.

Thompson had no link to Marshall’s work, so I got in touch with her, to get more details. As she describes in her papers (listed below), the motivation for her work is to discover effective methods of utilizing readers’ annotations of eBooks. In her early research, done mostly in the 1990′s, she observed students, and interviewed them, as they looked for used textbooks, to see if they favored books because of the nature of the annotations in them.

After confirming that students do, in fact, value annotations, Marshall went on to study the nature of the annotations to find patterns that could be used as models for eBook annotating. In the example below, on the left are four copies of the same page from different reader’s books with their annotations. On the right is the middle paragraph from the page, which all four readers annotated, to determine more precisely what was marked by each reader — which turns out to be the first sentence in all four cases. (The graphics are from the Marshall 2008 paper)

Kudos to Marshall for this ingenious, basic, application of crowdsourcing to books! It’s great that her little-noticed work, started way back in the mid-1990′s, is finally gaining recognition, as eBooks come closer to reality, and producers look for ways to make them more usable.

As Mitch Ratcliffe said in April, creating marginalia (an old name for the annotations of Marshall) is “an art made for the era of crowdsourcing.”

Selected papers of Cathy Marshall on annotations:

Complete list of Marshall’s publications

Eric Rumsey is at @ericrumsey

Just as Google Wave was announced yesterday, I was thinking of writing about the usefulness of the pictures that accompany results in Twitter Search, giving a good immediate overview of search results. I find this especially valuable in searching for Twitter users, to see how connected they are — It’s easy to see at a glance if most of the tweets listed are by the person being searched. So now Google Wave takes the idea a step further, with pictures of the people in an email thread. Below: Left: Twitter Search.  Right: Google Wave (from yesterday’s Google demo)

Facebook, of course, has similar pictures in its status updates. It’s interesting to follow how the use of pictures has progressed — In Facebook the status update pictures are relatively small. In Twitter, they grow larger, and now, in Google Wave, there are multiple pictures. This increasing reliance on pictures is smart. With the brain’s highly-developed facial recognition skills, we’re able to take in a large amount of information very quickly.

Eric Rumsey is at @ericrumsey

Several commentors on my recent article about Salman Rushdie’s imaginative foretelling of the Web have suggested that Rushdie’s vision — of a library made up of the Stream of all Stories ever told — was influenced by Jorge Luis Borges’ story The Library of Babel — which describes the universe as a library containing all books. There certainly is a resemblance, in that both Rushdie and Borges imagine a library of all knowledge. But the nature of the libraries pictured by the two writers is quite different. Borges’ library is very much a print library, made up of physical books. Much of the description of the library (as in the quote below) involves the intricate geometry of the shelves and the exact description of the books in the library.

In Borges’ library, as in the traditional print library, the books sit on the shelf, with no suggestion of their being connected to each other, no sense of movement. Rushdie, on the other hand, imagines a library in which the books flow in a stream — twisting and stretching and weaving in and out of each other. As I’ve discussed in the previous article, Rushdie’s vision resonates with recent discussions of the growing sense of the Web’s Stream-like, flowing nature and also with the coming revolution in libraries, as books are digitized, remaking them into Rushdie’s “fluid form.” The excerpts below give a sense of the different visions of the Library of Rushdie and Borges.

From Rushdie’s Haroun and the Sea of stories:

[The Ocean of the Stream of Stories is] “made up of a thousand thousand thousand and one different currents, each one a different color, weaving in and out of one another like a liquid tapestry of breathtaking complexity.” Rushdie imagines this Ocean as “the biggest library in the universe. And because the stories were held here in fluid form, they retained the ability to change, to become new versions of themselves, to join up with other stories and so become yet other stories; so that unlike a library of books, the Ocean of the Streams of Story was much more than a storeroom of yarns. It was not dead but alive.” [See previous article for complete passage and for picture credit.]

From Borges’ The Library of Babel:

“The universe (which others call the Library) is composed of an indefinite and perhaps infinite number of hexagonal galleries, with vast air shafts between, surrounded by very low railings. From any of the hexagons one can see, interminably, the upper and lower floors. The distribution of the galleries is invariable. Twenty shelves, five long shelves per side, cover all the sides except two; their height, which is the distance from floor to ceiling, scarcely exceeds that of a normal bookcase. … There are five shelves for each of the hexagon’s walls; each shelf contains thirty-five books of uniform format; each book is of four hundred and ten pages; each page, of forty lines, each line, of some eighty letters which are black in color.” [The picture accompanies the Web version of story, and is not credited; it appears on numerous other sites.]

Related articles:

Eric Rumsey is at @ericrumsey

When I first read the passage below in Salman Rushdie’s Haroun and the sea of stories three years ago, it struck me as a remarkable word picture of my experience of the Web. So of course I went right to Google to see if anyone else had made this connection — My searching, surprisingly, has found little since then, so I’ve thought about writing it up, but it didn’t get done. In the last week, I’ve gotten nudges (discussed below) that tell me this is the time. Here’s Rushdie:

Haroun looked into the water and saw that it was made up of a thousand thousand thousand and one different currents, each one a different color, weaving in and out of one another like a liquid tapestry of breathtaking complexity; and [the Water Genie] explained that these were the Streams of Story, that each colored strand represented and contained a single tale. Different parts of the Ocean contained different sorts of stories, and as all the stories that had ever been told and many that were still in the process of being invented could be found here, the Ocean of the Streams of Story was in fact the biggest library in the universe. And because the stories were held here in fluid form, they retained the ability to change, to become new versions of themselves, to join up with other stories and so become yet other stories; so that unlike a library of books, the Ocean of the Streams of Story was much more than a storeroom of yarns. It was not dead but alive.

Pow! Isn’t this a strikingly clear metaphorical description of the Web Stream that we all swim in every day? My first idea of a title for this article was “Did Salman Rushdie predict the Web?” I decided that was a bit too presumptuous  — But not by much — The passage does indeed verge on prediction. It was written in 1990 — Interestingly, the same year that Tim Berners-Lee “invented” the World Wide Web. It’s tempting to imagine the left-brained engineer (Berners-Lee) and the right-brained artist-seer (Rushdie) both envisioning the future Web in their own ways — Berners-Lee in outlining his Web ideas at CERN, and Rushdie in writing Haroun.

How has this passage and its Webishness gone unnoticed for so many years? Haroun is a story on many levels — Rushdie wrote it for his young son, and it’s often put in the category of “childrens’ literature.” I suspect this is the main reason it hasn’t been read often enough by grown-up Web users for someone to have seen Rushdie’s Stream of the Web metaphor. (Note that Haroun is on a prominent list of the 100 Greatest Novels of All Time)

How about the library connections in the passage? As a librarian, it certainly occurs to me that it could be viewed as being especially about libraries, maybe even seen as a threat to the traditional print library (“storeroom of yarns”). But I think, to the contrary, that Rushdie’s passage does the library world a great service, ushering us into the “liquid tapestry” of the digital Ocean, in which the Stream of “the library” will be able to “weave in and out” with the “thousand thousand and one different currents” outside of the traditional library world. Recent discussions of Google Book Search and orphan books show that the world is eagerly anticipating the stories in libraries being put into “fluid form.” And, in fact, library leader Peter Brantley, in commentary on GBS written in January 2009, talking about the coming age of digital books, uses language reminiscent of Rushdie: “We stride into a world where books are narratives in long winding rivers … and seas from which all rivers and rain coalesce.”

Speaking of the library world — As mentioned, there has been a notable lack of anyone else seeing a connection between the Rushdie passage and the Web. But the closest I’ve seen is in a paper co-authored by an engineer (JH Lienhard) and two librarians (JE Myers, TC Wilson), that was written in 1992, Surfing the Sea of Stories: Riding the Information Revolution (Mechanical Engineering 1992 Oct; 114(10): 60-65). This does an excellent job of connecting the Rushdie passage to the coming digital revolution, as it was seen in 1992, and contains the perceptively-done graphic in this article (above). But of course the full-blown Web was not born until 1995, so this view is limited. (The paper is summarized in the transcript of a radio program about it.)

Nudges for writing about this in the last week: First, In his blog article, Is The Stream What Comes After the Web?, Nova Spivack suggests that the metaphor of The Stream may soon replace The Web. The article doesn’t mention Rushdie, but it has elicited much discussion on Twitter, and someone would surely make the connection soon. Spivack does mention Twitter, saying that it and other microblogging systems are “the best example of the Stream,” which is related to the other nudge I’ve gotten, a blog article by Joff Redfern, Twitter is becoming the Ocean of the Stream of Stories. This is short , consisting mainly of the Rushdie quote above, but with its title it would likely be connected to Spivack’s Stream and Rushdie’s Streams of Stories sooner or later. Taken together, I think the articles by Spivack and Redfern indicate that Twitter is bringing to peoples’ minds the “stream-like” nature of the Web — The way big streams (e.g. swine flu 2 weeks ago) weave in and out with the day-to-day small streams of peoples’ lives on the Twitter ocean, with the stories constantly rewriting themselves.

Related articles:

Eric Rumsey is at @ericrumsey

Two recent articles, one by a librarian and one by a publisher, talk of the growing realization on the part of both parties that they increasingly have common interests, as both learn how to deal with the the implications of electronic publishing — Librarian Barbara Fister’s Library Journal cover story Publishers & Librarians: Two cultures one goal and publisher Neil Schlager’s blog article The problem with reference publishing.

Reading these articles has got me thinking about what I’ve been writing about on this blog in the last several months — As shown in the Categories (right sidebar) many of the subjects discussed here have common librarian-publisher threads. And in fact some of these articles have drawn comments from publisher kinds of people as well as librarians (See below). Thinking further, I realize how valuable Twitter has been for connecting to the publisher community, serving as a wide-ranging forum for discussion of current topics. So I’m listing below some of the people I’ve met on Twitter who talk about librarian/publisher issues:

Librarians:
Jose Afonso Furtado (@jafurtado)
Peter Brantley (@naypinya)
Nancy Picchi (@islandlibrarian)
Roy Tennant (@rtennant)
Lorcan Dempsey (@lisld)
Publishers:
Adam Hodgkin (@adamhodgkin)
Mike Shatzkin (@MikeShatzkin)
Tim O’Reilly (@timoreilly) – See more below
Neil Schlager (@neilschlager)
Kat Meyer (@KatMeyer)
Joe Wikert (@jwikert)

In her own category: Kassia Krozier (@booksquare) – Not a librarian or a publisher, but in the center of the discussion!

Tim O’Reilly (@timoreilly) and his group at O’Reilly Publishing have created a unique gathering place for thinking about the future of publishing. The O’Reilly Radar blog has articles by Tim and a group of other writers, some with library connections, notably Peter Brantley (@naypinya). In addition, the annual O’Reilly Tools of Change for Publishing Conference (TOC) in New York has speakers from the library world as well as the publishing world. Writers and speakers for the O’Reilly blog and TOC conference appear regularly on @timoreilly‘s Twitter tweets. Also, during the TOC conference, on-site Twitter reports are extensive. Joe Wikert (@jwikert), from the list above, also works at O’Reilly.

To read Seeing the Picture articles about issues of libraries and publishing, see the categories Publishing and TOC. Articles that have had comments/discussions with publishing people: Copyright in Google Books: Pictures & Text and Jon Orwant on Google Book Search at TOC.

Eric Rumsey is at @ericrumsey

Poking around in Google Similar Images, I’ve found examples that give indications of how the system works. I’ve put several of these together in a Flickr set, from which the example below is taken.

The top image in each of the pairs below (“Full size image”) is a choice from the initial search in GSI (“blackbird” in the example below). Clicking “Similar images” for this choice goes to a set of refined images, represented by the bottom row of images in the pair. The blackbird example here shows some of the strengths and weaknesses of GSI. It often seems to do best with color photographs, but not so well with monocolor pictures. In the first instance, the red spot on the wing and the greenish background likely are clues used by GSI, to good effect. The lack of color clues in the second case is likely a problem for GSI. It also shows pretty clearly that GSI is getting clues from words associated with images, in this case causing it to confuse the blackbird with the US Air Force plane that has the same name.

The importance of color clues for GSI that’s shown in the example above occurs in several additional examples in the Flickr set — B/W line drawings especially cause problems for GSI. Here are some other observations from the Flickr examples:

  • One notable example shows how GSI has a tendency to give too much weight to a word associated with a picture, as in the blackbird example — In a search for “george“, the “similar images” for a non-famous person named George are dominated by pictures of the recently prominent George Bush!
  • GSI does best when the image is focused clearly on one subject; it doesn’t do well when there are multiple subjects, especially when they are unusual subject combinations, that don’t typically occur together.
  • It does poorly with abstract or stylized, non-realistic images.
  • Strongly featured text sometimes “pulls the attention” of GSI away from the “picture content” of the image.

Despite the problems described here, I think GSI is a true advance in the technology of image search. In general, it does a surprisingly good job of detecting similarity. So, kudos to Google engineers!

Ben Lorica’s excellent post at O’Reilly Radar (Waiting for the Billionth Download) is full of details about iPhone apps. A bit overshadowed is the growth in the Books category. I’ve done some clipping and rearranging on one of Lorica’s charts to bring this out — Note that the Rate-of-Change for books in the last 3 months is at least double the rate for all other categories except Travel.

Another notable growth category shown here is Medical — Though its share of all apps is still small, it’s the third fastest-growing app, behind books and travel.

Here’s Lorica’s text on books:

Measured in terms of number of unique apps Books remains the fastest-growing category: during the week ending 4/12/2009, 11% of the apps in the U.S. store were in the Books category. Books has surpassed the Utilities category and may soon overtake the Entertainment category. Over the past week, Kindle for iPhone was the top app in the Books category.

Here’s the whole chart from Lorica’s article, with all 20 categories of apps:

Thanks to Twitter stars @adamhodgkin and @mtamblyn for highlighting Lorica’s article.

A notable feature of the KIndle iPhone app that was announced today is that it has color, unlike Amazon’s Kindle device. The most complete comments I’ve found on this are at CNET.com, written by Nicole Lee (whose interest in comic books gives her good reason to look for color!) Her article is especially valuable because it has a good comparison of screenshots on the iPhone and the Kindle. I’m showing this prominently because it appears at the bottom of the CNET story, and I suspect may be missed by lots of folks. Lee’s comments on color are excerpted below the screenshots.

Here’s the text that accompanies the picture : “Comparing comic books on the Kindle and the Kindle iPhone app. The quality isn’t great since you can’t zoom in (which is a problem for reading text), but the potential is there. (Credit: Nicole Lee/CBS Interactive)”

And here are other comments from Lee’s story on color …

… there is one potential advantage the iPhone has over the Kindle, and that is this: Color. Why would you want color in an ebook? Why, for comics, of course. I’m a big comic book fan, so I went looking for comics in the Amazon Kindle Store to see how it would look on the new Kindle 2 with its 16 shades of gray. There aren’t a lot of choices out there, so I just downloaded a few samples to check them out. The results are not that great, sad to say. Each comic page is considered an image, so it’s a little slower to download. I was fine with the gray shading, but the comic format does not quite fit the size of the Kindle 2′s screen. Smaller format books like manga digests look a little better however. Still, navigating the pages is a pain. There’s no way to zoom in and out of panels, and if I wanted to enlarge the images to full-size, I had to do so for each page. Plus word balloons are almost impossible to read since I can’t zoom in.

I downloaded those same comic samples to the Kindle application on the iPhone. I still couldn’t zoom in, and it’s still hard to read the word balloons. But I was awed and amazed that they arrived in full-blown color. Yes, I couldn’t read any of them, but it gave me a small glimmer of hope that maybe some day there’ll be a way. Until then, I guess I’ll have to live with the individually-sold iVerse comic applications.

It’s interesting, of course, that the first format that shows the value of color e-books is comic books — But, the implications are far-ranging for illustrated books in general, such as medical and science textbooks in which illustrations have an important place.

Yale Image Finder is a search engine for searching medical articles in PubMed Central for images. YIF is notable because it searches for text that is contained in images, many of which are charts and graphs with embedded “text” describing the data being presented. The “text” in these images, as in the example from YIF below, is converted to searchable OCR text.

What especially strikes me about this project is how similar it is to several initiatives from Google — For several years, Google has been working on image-to-text conversion in various of its facets, starting with Google Catalogs (now defunct) and Google Book Search. More recently, in 2008, several patents were published which extend the potential use of this sort of technology to a variety of possibilities, some of which include use in Google Maps street view, labels in museums and stores, and YouTube videos. Also showing Google’s continuing interest in this area is the announcement in Oct, 2008 that scanned PDF documents in Google Web Search are being converted to OCR text format.

Yale Image Finder was first announced in August, 2008, so it’s surprising that I have not been able to find anywhere (including a scholarly description by the developers) that it’s been connected to the initiatives by Google, which seem to be so similar. The same sorts of expressions of awe and amazement that have been expressed about the Google initiatives apply equally well to the Yale project, so I’m excerpting several of these commentaries below, all written in January, 2008, when the latest patents from Google inventors Luc Vincent and Adrian Ulges were published …

Bill Slawski, who has written several articles on Google image-to-text patents – Google on Reading Text in Images from Street Views, Store Shelves, and Museum Interiors :

One of the standard rules of search engine optimization that’s been around for a long time is that “search engines cannot read text that is placed within images.” What if that changed?

Here’s more from Slawski – Googlebot In Aisle Three: How Google Plans To Index The World? :

It’s been an old sawhorse for years that Google couldn’t recognize text that was displayed in images while indexing pages on the Web. These patent filings hint that Google may be able to do much more with images than we can imagine.

Duncan RileyGoogle Lodges Patent For Reading Text In Images And Video :

I may be stating the blatantly obvious when I say that if Google has found a way to index text in static images and video this is a great leap forward in the progression of search technology. This will make every book in the Google Books database really searchable, with the next step being YouTube, Flickr (or Picasa Web) and more. The search capabilities of the future just became seriously advanced.

Of course — sorry to keep harping on it! — as much as recognizing text in pictures would be a great advance, the REAL advance, of recognizing the actual objects in pictures, the philosopher’s stone of image search, still seems far from happening.

Please comment here or Twitter @ericrumsey