The recent controversy about the Google Book Search Settlement seems to have taken up peoples’ Google-watching attention so much that advances in the way GBS actually works have been getting overlooked. Several notable improvements were made during the summer, for example, that got very little recognition. Another change that seems to have gotten little recognition is that Google web searches have begun to include links to books in GBS in the last 1-2 years (as in the example at left). Particularly in searching for historical topics, I’ve been seeing searches recently in which the majority of the first 10 hits are from GBS — A great advance, I think, for historical research. Up to now, my experience has been that history has been a fairly weak subject on the Web — Locked away in books, not on Web pages.

I had occasion to take advantage of the newly accessible books from GBS recently, when I was least expecting it, while having a discussion with my son David, who’s a long-distance runner, about track runners of the past at the University of Iowa. I remembered that one particular runner on the team, Ted Wheeler, ran on the US Olympic team in the 1950’s, and that he later went on to become the coach for the UI track team (I especially knew about him because while he was the coach he married Sheila Creth, the University Librarian at the University of Iowa Libraries, where I work). David knew that Wheeler had been in the Olympics, and thought that he had been an assistant coach at Iowa, rather than the head coach. So … of course I turned to Google to settle the “discussion.” It turned out to be a surprisingly difficult search. I assumed that it would be fairly easy to find records of recent track coaches at a large, Big Ten program like Iowa. But it wasn’t — I tried several search terms without success before — Bingo! — I finally hit upon the combination that turned up the page shown here, establishing that Wheeler was, indeed, the UI track coach from 1978 to 1996 — with the added benefit of a great picture!

The point of this little story: I think integrating GBS links into Google web search is a great advance, and deserves more attention. As I said above, there’s been so much negative press for Google in recent discussions of the Settlement that everything they do is interpreted negatively — I saw a link in the last couple of weeks, that I unfortunately didn’t keep track of, decrying Google’s putting GBS links in Web search results because someone thought Google was trying to unfairly boost their own content. Really?? I think there’s such a treasure in old books that the world will benefit from Google’s making them more accessible. There are questions, certainly, about the algorithm used by Google to determine which books are included in Web search results, and I hope Google will say more about that. But it’s not only Google that’s saying little on the subject — I haven’t seen much discussion at all by anybody on the integration of GBS books in Google web search results –  If anyone can find it, please add a comment or contact me by Twitter or Email.

Eric Rumsey is at: eric-rumsey AttSign uiowa dott edu and on Twitter @ericrumsey

Eric Rumsey is at: eric-rumsey AttSign uiowa dott edu and on Twitter @ericrumsey

I was actually thinking of taking Dempsey’s thought a step further, and proposing the idea of including Google’s word clouds in library catalogs. But when I started looking more closely at GBS word clouds I found problems — The first thing I noticed in the cloud for Origin of Species (below and here at GBS [scroll down to Common terms and phrases]) is that it has the plant-related words “seeds,” “pistil,” and “pollen,” but does not have the word “plant(s).” Hmm, that’s odd — So I searched for “plants” and found that there are in fact 100 occurrences of it in the book. Then I clicked some of the terms in the cloud shown below, and found that the number of results often does not correlate well with the font size of the word (which is what’s supposed to happen in a word cloud) …

Note that the words “admit,” “cause,” and “male,” which are in the smallest font, have more occurrences than other terms with larger fonts — “Asa Gray” and “pistil” in particular.

I tried several books, and found similar results in all of them — The font size of terms in the word cloud does not show much correlation with the number of occurrences of words in the books. In Snippet view books (as at least one of the books in Dempsey’s article is) the problem is not apparent because the number of search results is limited to three links in the book, making it impossible to determine how many occurrences of the term there are.

I suspect that the GBS word cloud problem has not been noticed more because the word clouds are rather “buried” — Not on the default Read (Front cover) page, but inconspicuously down in the middle of the Overview page, probably not seen by the vast majority of users.

We need more documentation about word clouds in GBS — How are they derived? What exactly are they intended to mean? Google has said about other metadata problems that they are working on them, and that they’ll slowly get fixed. Hopefully, that will apply to word clouds also. Maybe Google thinks of word clouds as still being “in beta” — they were, after all, only launched in July — and that’s why they’re giving them a low profile.

Eric Rumsey is at: eric-rumsey AttSign uiowa dott edu and on Twitter @ericrumsey

The Copyright Office has been following the Google Library Project since 2003 with great interest. We first learned about it when Google approached the Library of Congress, seeking to scan all of the Library’s books. At that time, we advised the Library on the copyright issues relevant to mass scanning, and the Library offered Google the more limited ability to scan books that are in the public domain. An agreement did not come to fruition because Google could not accept the terms.

As discussed in my article in June, it seems surprising that the Library of Congress has not taken a more active role in the mass-scanning project that Google is doing. Peters’ words explain why — The copyright mess! If copyright gets fixed, LC might be doing the project instead of Google.

It’s encouraging that Peters has finally been given a platform to talk about the mess. She did talk about it at a Columbia University meeting in March, although it was not widely reported, and was apparently only recorded on a video which was not transcribed (see my transcription of a key passage here). At that conference, she’s reported to have said that Congress had shown no interest in hearing her testimony. Hopefully they’re ready to listen now.

Peters stresses in her testimony yesterday, and in her talk at Columbia, that Congress needs to be the one to fix copyright law. Letting the judiciary branch speak through the Settlement, she says, is making an “end run around the legislative process” — her words in yesterday’s testimony. Brewster Kahle used the same words in April.

With the GBS settlement discussion heating up, it’s becoming increasingly clear to me that the root of the problem is US Copyright law. As Peters suggests, until copyright is fixed, mass-scanning of books is going to be problematic.

Eric Rumsey is on Twitter @ericrumsey

Reading Nunbergs blog post corresponding to the talk is very entertaining in a juvenile sort of way. The poor guy has been trying to use Google Books as a linguistic research corpus, and has discovered to his professed horror that there are all sorts of errors, many of them humorous, in its metadata.

I must now let you in on a closely held secret among library metadata technologists which due to the Google Books metadata fiasco must now be revealed to the general public. There is some crappy data in library catalogs. How much is an interesting question, and my ability to comment on how much is limited by confidentiality obligations. However, I am free to observe that studies have been published on the error rate in OpenURL linking. OpenURL linking usually depends on matching of metadata between a source metadata file and a target metadata file; errors in either file can cause a linking error. Reported error rates are in excess of 1%. In his response to Nunberg blog post, Jon Orwant points out that a one in a million error occurs a million times if you have a trillion metadata items; my guess is that an error rate of one part per million may be overly optimistic by four orders of magnitude when applied to library metadata.

In my post on “collecting too much data”, I wrote that a huge challenge of maintaining a large metadata database is battling entropy as the collection grows. I’ve observed that most people trying to collect metadata go through an early period of thinking it’s easy, and then gradually gain understanding of the real challenges. Google has certainly been no exception to this pattern. When they first started dealing with book metadata, they were oblivious to the difficulties of maintaining a large metadata database. As Orwant’s response to Nunberg shows, they are currently in the phase of understanding the true difficulties of what they need to do. They have most certainly become attuned to the importance of keeping track of the source (provenance) of their metadata, if for no other reason than to have someone to blame for the inevitable metadata stupidities. Much of the “Linked Data” crowd has yet to digest this lesson fully.

Nunberg’s thesis is that Google Books will be the “Last Library” and that it would be a disaster for society if Google does a bad job of it. He does not consider the converse possibility. What if Google manages to do a better job of it than libraries have done? If that happens, all of the library world could be turned upside down. Existing metadata maintenance cooperatives would vanish overnight and libraries around the world would become dependent on Google’s metadata prowess. Google would acquire a legal metadata monopoly through technical merit rather than through class action maneuvering. What if Google, with pseudo-monopoly funding and the smartest engineers anywhere, manages to figure out new ways to separate the bird shit from the valuable metadata in thousands of metadata feeds, thereby revolutionizing the library world without even intending to do so? Is it this even conceivable?

Context: Recent articles by Geoff Nunberg:
Google Books: A Metadata Train Wreck, Language Log blog (Hellman’s comments above refer to this article)
Google’s Book Search: A Disaster for Scholars, Chronicle of Higher Education

Eric Rumsey is on Twitter @ericrumsey

Google, of course, puts the book at the top of the list because its deep metadata indicates that metadata is an important topic, and PageRank likely indicates that other people also value Weinberger’s discussion of the topic.

So, why don’t library catalogs find the book? — The problem is the subject headings assigned by the Library of Congress, and used in most all library catalogs:

Knowledge management.
Information technology — Management.
Information technology — Social aspects.
Personal information management.
Information resources management.
Order.

Even though the book discusses metadata at length and on many pages, it’s not deemed important enough to be a heading — The problem is that the traditional catalog is what Weinberger calls a “second order” resource, being limited to the small number of subject headings that will fit on a card in the (bygone) catalog. Given resources to assign a larger number of subject headings, no doubt metadata would be included.

So … Librarians can’t afford to be smug about metadata — Google has problems (as discussed in Geoff Nunberg articles linked below). But libraries have their own problems. In many ways the traditional library catalog lacks metadata features that have become common in Google, Amazon, and other sites.

Hope for Libraries — WorldCat does find the book with the david weinberger metadata search (#2 in results), because it has additional tags listed in its “Abstract” (scroll down) which include metadata — Sooner or later, maybe libraries will add the WorldCat Abstract to their catalogs to “enrich their metadata.”

Context: Recent articles by Geoff Nunberg:
Google Books: A Metadata Train Wreck, Language Log blog
Google’s Book Search: A Disaster for Scholars, Chronicle of Higher Education

Eric Rumsey is on Twitter @ericrumsey

Context: Recent articles by Geoff Nunberg:
Google Books: A Metadata Train Wreck, Language Log blog
Google’s Book Search: A Disaster for Scholars, Chronicle of Higher Education

Eric Rumsey is on Twitter @ericrumsey

Kelly’s comments parallel the static print libary of Borges and the flowing, connected library of Rushdie:

(Kelly) The common vision of the library’s future (even the e-book future) assumes that books will remain isolated items, independent from one another, just as they are on shelves in your public library. But this vision misses the chief revolution birthed by scanning books: in the universal library, no book will be an island.

Metadata as the real magic (cross-linked … extracted … indexed … analyzed …) — Mike Cane should like this! :

(Kelly) Turning inked letters into electronic dots that can be read on a screen is simply the first essential step in creating this new library. The real magic will come in the second act, as each word in each book is cross-linked, clustered, cited, extracted, indexed, analyzed, annotated, remixed, reassembled and woven deeper into the culture than ever before. In the new world of books, every bit informs another; every page reads all the other pages.

Clive Thompson says the community of digital book readers will transform books by building a web of linked commentaries. Kelly says much the same:

Once a book has been integrated into the new expanded library by means of this linking, its text will no longer be separate from the text in other books. … Books, including fiction, will become a web of names and a community of ideas.

Peter Brantley imagines a world in which digital books become connected as long winding rivers that flow together. Here’s Kelly sounding similar:  …

Search engines are transforming our culture because they harness the power of relationships, which is all links really are. … This tangle of relationships is precisely what gives the Web its immense force. The static world of book knowledge is about to be transformed by the same elevation of relationships, as each page in a book discovers other pages and other books. Once text is digital, books seep out of their bindings and weave themselves together.

Rushdie describes the countless currents in the Stream of Stories “weaving in and out of one another like a liquid tapestry” — Likewise Kelly:

When books are digitized, reading becomes a community activity. … In a curious way, the universal library becomes one very, very, very large single text: the world’s only book. … So what happens when all the books in the world become a single liquid fabric of interconnected words and ideas? …

What Happens When Books Connect? – This is the title for one of the sections of Kelly’s article from which most of the quotes above are taken, and it is really an overriding theme for all of the them — The digitized books of the future will talk easily to each other, which will transform books in the same way the Web has already transformed other aspects of culture.

Eric Rumsey is on Twitter @ericrumsey

All of this hidden information — exploded out, made explicit — turns an ebook from a dumb object into a smart object. … With such exploded data, an eBook becomes a ticket for admission to a vast collection of databased information.

An eBook becomes a local terminal connected to a growing and living cloud of associated information, with meanings and implications no publisher or writer can currently imagine. It lets the reader make those connections. It’s an eBook that can do something. … And this is precisely why Google wants the Book Search settlement to go through: it sees that as the future. Google is staffed by geeks who juggle information with an expertise that print publishers lack. … Google makes information do things.

Print publishing freezes information into a static object — An object that stands alone, disconnected, unable to do anything. … There needs to be another layer slathered over [the Publisher]. The information geeks. The ones who will take the static objects, extract the hidden information, and database it. … They are new publishers for a new age.

This metadata has value. And that value will increase as it ages. As new connections are formed, and new data is added, its value increases exponentially. The metadata value of a publisher could equal, if not surpass, that of the works on which it’s based.

Metadata will become a multi-billion dollar business. … The entire global economy is built on metadata. And it’s accessing that metadata that would justify more than a five-dollar price for an eBook. Consumers would see [that] an investment has been made to turn a text data dump into something active and intelligent. … no longer a flat, linear collection of words. Dimensions have been added that breathe and grow. The eBook price becomes a ticket. People are … buying into an ongoing experience.

Metadata Librarians will Rule the World …

Metadata, of course, is a concept that’s near-and-dear to the hearts of librarians … Which led to a bit of serendipity in thinking about the title of this article. I found in Google that no one else has used the phrase “Metadata will Rule the World.” But in playing around with various combinations I did discover the phrase “Librarians will one day rule the world” — in a 2004 blog post by Robert Wolfe (@metametadata), who works in Metadata services at the MIT library.

Of course, I couldn’t resist the opportunity to use the picture here of Wolfe’s Librarian Trading Card — “That’s right, I’ve got special metadata related powers.” (How do you like that, Mike Cane!)

The last posting on the Metametametadata blog is August, 2006 — Too bad — Robert Wolfe has interesting ideas.

Cane: Metadata turns an eBook into an active, growing, living cloud …

Cane’s contrasting of living eBooks with print publishing’s static books is reminiscent of the language used in the articles I wrote in May on the Stream as the new metaphor of the Web, particularly the article on Salman Rushdie’s Stream library & JL Borges’ Print library — Rushdie’s vision of books twisting and stretching and weaving in and out of each other sounds much like Cane’s vision in the quote above.

Eric Rumsey is @ericrumsey

Google Books titles with color pictures – Botany, Medical Botany

[Examples below link to screenshots in Flickr of Overview : Selected Pages in GBS; links in Flickr go to actual GBS page.]

The Botanical Magazine, Or, Flower-garden Displayed
By William Curtis, vol 9, 1795, Harvard Univ

Curtis’s Botanical Magazine, Or, Flower-garden Displayed
By John Sims, vol 41, 1815, Harvard Univ

The Family Herbal
By John Hill, 1812, Oxford Univ

Flora Medica
By George Spratt, 1830, Oxford Univ

Vegetable Materia Medica of the United States, Or, Medical Botany
By William Paul Crillon Barton, 1818, Oxford Univ

Medicinal Plants (vol 2)
By Robert Bentley, Henry Trimen, David Blair, 1880, Harvard Univ

Medicinal Plants (vol 4)
By Robert Bentley, Henry Trimen, David Blair, 1880, Harvard Univ

Paxton’s Magazine of Botany, and Register of Flowering Plants (vol 1)
By Joseph Paxton, 1836, Oxford Univ

Strasburger’s Text-book of Botany
By Eduard Strasburger, Hans Fitting, Ludwig Jost, William Henry Lang, Heinrich Schenck, George Karsten, 1921, Univ California

Google Books titles with color pictures – Dermatology

Atlas and Epitome of Diseases of the Skin
By Franz Mraček, 1905, Stanford Univ

Atlas Der Hautkrankheiten, Mit Einschluss Der Wichtigsten Venerischen
By Eduard Jacobi, 1906, Stanford Univ

Atlas of Diseases of the Skin
By Franz Mraček, Henry Weightman Stelwagon, 1899, Stanford Univ

Illustrated Skin Diseases
By William Samuel Gottheil, 1902, Harvard Univ

An Introduction to Dermatology
By Norman Purvis Walker, 1906, Stanford Univ

On Diseases of the Skin
By Erasmus Wilson, 1865, Harvard Univ

Portfolio of Dermochromes (vol 2)
By Jerome Kingsbury, Eduard Jacobi, John James Pringle, William Gaynor States, 1913, Harvard Univ

Skin Diseases
By Melford Eugene Douglass, 1900, Univ Michigan

If you know of other areas that have books in Google Books with color pictures, please send comments.