Many possible takes on this picture. What comes to my mind first is the idea of the Attention Economy –The idea that in the days of the traditional library, before the Internet, information was a limited resource. Libraries could afford to work under the assumption that “we’ve got the good stuff, and our users have to to come to us to get it.” There was little motivation to improve overly-complicated search interfaces like the picture on the right above, because users had no choice. In the new environment of the Internet, however, the limiting factor is not information, but attention. The problem of users now is not finding information, but being flooded by too much information. In this environment, users naturally gravitate to the easiest information to find, which, of course, Apple, Google et al are glad to provide.

Another take on this is the high cost of Simplicity –The simple interfaces of Apple and Google are just the tip of the iceberg, built upon the costly labor of armies of engineers. Libraries just can’t afford to compete with this sort of juggernaut. Personally, I consider myself lucky, as a librarian, to be working in a medical library — Medical libraries have a long history of generous federal support, in the interests of the country’s health, which has enabled the creation of tools to streamline access to medical information, from Index Medicus to PubMed. For libraries generally, however, it’s still hard to compete with the resources of dotcom information providers. To end on a hopeful note — It’s encouraging to see that libraries are increasingly realizing the importance of providing Google-like interfaces for their catalogs, to gain back the attention from users that they’ve lost in recent few years.

The picture above, and the title of this post, are adapted from an article by Scott Monty — Thanks!

Eric Rumsey is at: eric-rumsey AttSign uiowa dott edu and on Twitter @ericrumsey

Reading Nunbergs blog post corresponding to the talk is very entertaining in a juvenile sort of way. The poor guy has been trying to use Google Books as a linguistic research corpus, and has discovered to his professed horror that there are all sorts of errors, many of them humorous, in its metadata.

I must now let you in on a closely held secret among library metadata technologists which due to the Google Books metadata fiasco must now be revealed to the general public. There is some crappy data in library catalogs. How much is an interesting question, and my ability to comment on how much is limited by confidentiality obligations. However, I am free to observe that studies have been published on the error rate in OpenURL linking. OpenURL linking usually depends on matching of metadata between a source metadata file and a target metadata file; errors in either file can cause a linking error. Reported error rates are in excess of 1%. In his response to Nunberg blog post, Jon Orwant points out that a one in a million error occurs a million times if you have a trillion metadata items; my guess is that an error rate of one part per million may be overly optimistic by four orders of magnitude when applied to library metadata.

In my post on “collecting too much data”, I wrote that a huge challenge of maintaining a large metadata database is battling entropy as the collection grows. I’ve observed that most people trying to collect metadata go through an early period of thinking it’s easy, and then gradually gain understanding of the real challenges. Google has certainly been no exception to this pattern. When they first started dealing with book metadata, they were oblivious to the difficulties of maintaining a large metadata database. As Orwant’s response to Nunberg shows, they are currently in the phase of understanding the true difficulties of what they need to do. They have most certainly become attuned to the importance of keeping track of the source (provenance) of their metadata, if for no other reason than to have someone to blame for the inevitable metadata stupidities. Much of the “Linked Data” crowd has yet to digest this lesson fully.

Nunberg’s thesis is that Google Books will be the “Last Library” and that it would be a disaster for society if Google does a bad job of it. He does not consider the converse possibility. What if Google manages to do a better job of it than libraries have done? If that happens, all of the library world could be turned upside down. Existing metadata maintenance cooperatives would vanish overnight and libraries around the world would become dependent on Google’s metadata prowess. Google would acquire a legal metadata monopoly through technical merit rather than through class action maneuvering. What if Google, with pseudo-monopoly funding and the smartest engineers anywhere, manages to figure out new ways to separate the bird shit from the valuable metadata in thousands of metadata feeds, thereby revolutionizing the library world without even intending to do so? Is it this even conceivable?

Context: Recent articles by Geoff Nunberg:
Google Books: A Metadata Train Wreck, Language Log blog (Hellman’s comments above refer to this article)
Google’s Book Search: A Disaster for Scholars, Chronicle of Higher Education

Eric Rumsey is on Twitter @ericrumsey

Where to start. The MeSH database is stiff and laboured, with occasional outbreaks of tumid extravagance. My group all agreed that we need clearer, more intuitive visual displays of the thesaurus and subheadings. The creation of a search statement using MeSH headings needs a complete rethink. … Even searching for MeSH headings is difficult and unpredictable. But worse, no one really understands it.  When I teach MeSH, my students glaze over as if I were lecturing on 12-tone music. The way PubMed presents MeSH is fussy and needlessly complex. We need a MeSH mashup.

Hey medical librarians — Let’s help our users discover the buried treasure of MeSH!

#Journalism future? ‘It’s in the data.’ #Metadata, that is – makes the #news last, rather than a perishable commodity http://tr.im/lmetadata
9:52 PM Jul 22nd from web

Montgomery brings an interesting perspective, with feet in the world of librarianship, where metadata has been a focus for a long time, and in journalism, which has only more recently begun to awaken to the value of metadata. Montgomery’s tweet links to a blog article (The Future, it’s in the Data) by journalist Carrie Brown-Smith (@Brizzyc), who interviewed Montgomery. Here’s an excerpt …

(quoting Montgomery) Librarians are precisely who have been leading in adding value and context to information … In all the ink and pixels spilled over the future of journalism I have not heard one mention of this … Information is valuable, and it needs structure, ­ keywording, and taxonomy added so it can be accessed, and repurposed.  All this is then repackaged and sold …

Brown-Smith also reports on a recent provocative article by journalist Dan Conover (@xarker) about the importance of adding data to news stories which could provide “a rich trove that could be mined to discover new connections and relationships.” (quote from Brown-Smith)

Conover’s article is a long and chatty discussion of metadata in journalism, and why news reporters resist adding it. He tells an interesting story of reporting on a house fire with and without metadata, and how coding can increase the future value of the work of reporting. He says that “the structure of [metadata] information is [now] the news organization’s primary product.” Unfortunately, though, he says, journalists hate the idea of adding this structured metadata — Why? …

Metadata coding is viewed as a library (or, in newsroom jargon, “morgue”) function … Journalism is a profession for storytellers, and our newsroom culture celebrates romantic myths that are generally hostile to structure. So I understand my curmudgeonly colleagues when they scoff behind my back at the word “metadata.”

I suspect that journalism is not the only profession that “celebrates romantic myths that are hostile to structure” … In journalism, as in publishing and libraries, discussed in my previous article, we’ve come to the interesting point when it’s the computer-library-coding geeks who will be, in Mike Cane’s words “the new publishers for a new age” … the ones who “make information do things.”

Eric Rumsey is @ericrumsey

All of this hidden information — exploded out, made explicit — turns an ebook from a dumb object into a smart object. … With such exploded data, an eBook becomes a ticket for admission to a vast collection of databased information.

An eBook becomes a local terminal connected to a growing and living cloud of associated information, with meanings and implications no publisher or writer can currently imagine. It lets the reader make those connections. It’s an eBook that can do something. … And this is precisely why Google wants the Book Search settlement to go through: it sees that as the future. Google is staffed by geeks who juggle information with an expertise that print publishers lack. … Google makes information do things.

Print publishing freezes information into a static object — An object that stands alone, disconnected, unable to do anything. … There needs to be another layer slathered over [the Publisher]. The information geeks. The ones who will take the static objects, extract the hidden information, and database it. … They are new publishers for a new age.

This metadata has value. And that value will increase as it ages. As new connections are formed, and new data is added, its value increases exponentially. The metadata value of a publisher could equal, if not surpass, that of the works on which it’s based.

Metadata will become a multi-billion dollar business. … The entire global economy is built on metadata. And it’s accessing that metadata that would justify more than a five-dollar price for an eBook. Consumers would see [that] an investment has been made to turn a text data dump into something active and intelligent. … no longer a flat, linear collection of words. Dimensions have been added that breathe and grow. The eBook price becomes a ticket. People are … buying into an ongoing experience.

Metadata Librarians will Rule the World …

Metadata, of course, is a concept that’s near-and-dear to the hearts of librarians … Which led to a bit of serendipity in thinking about the title of this article. I found in Google that no one else has used the phrase “Metadata will Rule the World.” But in playing around with various combinations I did discover the phrase “Librarians will one day rule the world” — in a 2004 blog post by Robert Wolfe (@metametadata), who works in Metadata services at the MIT library.

Of course, I couldn’t resist the opportunity to use the picture here of Wolfe’s Librarian Trading Card — “That’s right, I’ve got special metadata related powers.” (How do you like that, Mike Cane!)

The last posting on the Metametametadata blog is August, 2006 — Too bad — Robert Wolfe has interesting ideas.

Cane: Metadata turns an eBook into an active, growing, living cloud …

Cane’s contrasting of living eBooks with print publishing’s static books is reminiscent of the language used in the articles I wrote in May on the Stream as the new metaphor of the Web, particularly the article on Salman Rushdie’s Stream library & JL Borges’ Print library — Rushdie’s vision of books twisting and stretching and weaving in and out of each other sounds much like Cane’s vision in the quote above.

Eric Rumsey is @ericrumsey

As the “de facto national library” of the US and “the largest library in the world,” wouldn’t it seem logical that LC be involved in thinking about GBS and the Settlement, which some say will change the way we read more than anything since the printing press?

I’ve been thinking about this idea for several months, but especially after writing an article in May on the apparently woeful state of Information Technology Strategic Planning at LC, as stated in a report by LC’s  Inspector General. Could there be a connection? Is this apparent lack of vision related to LC’s non-engagement with the momentous issues of the Settlement?

I was glad to discover, in doing research for this article, that someone else is thinking at least a bit along the same lines — Peter Eckersly, at the Electronic Frontier Foundation, suggested recently that Google put a copy of all books they scan at the Library of Congress — A fairly modest proposal, but maybe it will at least have the effect of bringing the Library of Congress at long last into the spotlight.

Eric Rumsey is at @ericrumsey

In Borges’ library, as in the traditional print library, the books sit on the shelf, with no suggestion of their being connected to each other, no sense of movement. Rushdie, on the other hand, imagines a library in which the books flow in a stream — twisting and stretching and weaving in and out of each other. As I’ve discussed in the previous article, Rushdie’s vision resonates with recent discussions of the growing sense of the Web’s Stream-like, flowing nature and also with the coming revolution in libraries, as books are digitized, remaking them into Rushdie’s “fluid form.” The excerpts below give a sense of the different visions of the Library of Rushdie and Borges.

From Rushdie’s Haroun and the Sea of stories:

[The Ocean of the Stream of Stories is] “made up of a thousand thousand thousand and one different currents, each one a different color, weaving in and out of one another like a liquid tapestry of breathtaking complexity.” Rushdie imagines this Ocean as “the biggest library in the universe. And because the stories were held here in fluid form, they retained the ability to change, to become new versions of themselves, to join up with other stories and so become yet other stories; so that unlike a library of books, the Ocean of the Streams of Story was much more than a storeroom of yarns. It was not dead but alive.” [See previous article for complete passage and for picture credit.]

From Borges’ The Library of Babel:

“The universe (which others call the Library) is composed of an indefinite and perhaps infinite number of hexagonal galleries, with vast air shafts between, surrounded by very low railings. From any of the hexagons one can see, interminably, the upper and lower floors. The distribution of the galleries is invariable. Twenty shelves, five long shelves per side, cover all the sides except two; their height, which is the distance from floor to ceiling, scarcely exceeds that of a normal bookcase. … There are five shelves for each of the hexagon’s walls; each shelf contains thirty-five books of uniform format; each book is of four hundred and ten pages; each page, of forty lines, each line, of some eighty letters which are black in color.” [The picture accompanies the Web version of story, and is not credited; it appears on numerous other sites.]

Related articles:

• Did Salman Rushdie envision the Web in 1990? – Rushdie’s word-picture of the Ocean of the Stream of Stories.
• Digital books: Narratives in long winding streams, Peter Brantley, 2009
• The Web as a Stream of Stories – Spivack and Brantley flow together into Rushdie’s Stream
• The Stream of Streams has arrived, Nova Spivack

Eric Rumsey is at @ericrumsey

Haroun looked into the water and saw that it was made up of a thousand thousand thousand and one different currents, each one a different color, weaving in and out of one another like a liquid tapestry of breathtaking complexity; and [the Water Genie] explained that these were the Streams of Story, that each colored strand represented and contained a single tale. Different parts of the Ocean contained different sorts of stories, and as all the stories that had ever been told and many that were still in the process of being invented could be found here, the Ocean of the Streams of Story was in fact the biggest library in the universe. And because the stories were held here in fluid form, they retained the ability to change, to become new versions of themselves, to join up with other stories and so become yet other stories; so that unlike a library of books, the Ocean of the Streams of Story was much more than a storeroom of yarns. It was not dead but alive.

Pow! Isn’t this a strikingly clear metaphorical description of the Web Stream that we all swim in every day? My first idea of a title for this article was “Did Salman Rushdie predict the Web?” I decided that was a bit too presumptuous  — But not by much — The passage does indeed verge on prediction. It was written in 1990 — Interestingly, the same year that Tim Berners-Lee “invented” the Web. It’s tempting to imagine the left-brained engineer (Berners-Lee) and the right-brained artist-seer (Rushdie) both envisioning the future Web in their own ways — Berners-Lee in outlining his Web ideas at CERN, and Rushdie in writing Haroun.

How has this passage and its Webishness gone unnoticed for so many years? Haroun is a story on many levels — Rushdie wrote it for his young son, and it’s often put in the category of “childrens’ literature.” I suspect this is the main reason it hasn’t been read often enough by grown-up Web users for someone to have seen Rushdie’s Stream of the Web metaphor. (Note that Haroun is on a prominent list of the 100 Greatest Novels of All Time)

How about the library connections in the passage? As a librarian, it certainly occurs to me that it could be viewed as being especially about libraries, maybe even seen as a threat to the traditional print library (”storeroom of yarns”). But I think, to the contrary, that Rushdie’s passage does the library world a great service, ushering us into the “liquid tapestry” of the digital Ocean, in which the Stream of “the library” will be able to “weave in and out” with the “thousand thousand and one different currents” outside of the traditional library world. Recent discussions of Google Book Search and orphan books show that the world is eagerly anticipating the stories in libraries being put into “fluid form.” And, in fact, library leader Peter Brantley, in commentary on GBS written in January 2009, talking about the coming age of digital books, uses language reminiscent of Rushdie: “We stride into a world where books are narratives in long winding rivers … and seas from which all rivers and rain coalesce.”

Speaking of the library world — As mentioned, there has been a notable lack of anyone else seeing a connection between the Rushdie passage and the Web. But the closest I’ve seen is in a paper co-authored by an engineer (JH Lienhard) and two librarians (JE Myers, TC Wilson), that was written in 1992, Surfing the Sea of Stories: Riding the Information Revolution (Mechanical Engineering 1992 Oct; 114(10): 60-65). This does an excellent job of connecting the Rushdie passage to the coming digital revolution, as it was seen in 1992, and contains the perceptively-done graphic in this article (above). But of course the full-blown Web was not born until 1995, so this view is limited. (The paper is summarized in the transcript of a radio program about it.)

Nudges for writing about this in the last week: First, In his blog article, Is The Stream What Comes After the Web?, Nova Spivack suggests that the metaphor of The Stream may soon replace The Web. The article doesn’t mention Rushdie, but it has elicited much discussion on Twitter, and someone would surely make the connection soon. Spivack does mention Twitter, saying that it and other microblogging systems are “the best example of the Stream,” which is related to the other nudge I’ve gotten, a blog article by Joff Redfern, Twitter is becoming the Ocean of the Stream of Stories. This is short , consisting mainly of the Rushdie quote above, but with its title it would likely be connected to Spivack’s Stream and Rushdie’s Streams of Stories sooner or later. Taken together, I think the articles by Spivack and Redfern indicate that Twitter is bringing to peoples’ minds the “stream-like” nature of the Web — The way big streams (e.g. swine flu 2 weeks ago) weave in and out with the day-to-day small streams of peoples’ lives on the Twitter ocean, with the stories constantly rewriting themselves.

Related articles:

• The Stream of Streams has arrived, 2009 – Nova Spivack says that the new metaphor of the Web is the Stream
• Digital books: Narratives in long winding streams, Peter Brantley, 2009
• The Web as a Stream of Stories – Spivack and Brantley flow together into Rushdie’s Stream
• Rushdie’s Stream library & Borges’ print library

Eric Rumsey is at @ericrumsey

Two recent articles, one by a librarian and one by a publisher, talk of the growing realization on the part of both parties that they increasingly have common interests, as both learn how to deal with the the implications of electronic publishing — Librarian Barbara Fister’s Library Journal cover story Publishers & Librarians: Two cultures one goal and publisher Neil Schlager’s blog article The problem with reference publishing.

Reading these articles has got me thinking about what I’ve been writing about on this blog in the last several months — As shown in the Categories (right sidebar) many of the subjects discussed here have common librarian-publisher threads. And in fact some of these articles have drawn comments from publisher kinds of people as well as librarians (See below). Thinking further, I realize how valuable Twitter has been for connecting to the publisher community, serving as a wide-ranging forum for discussion of current topics. So I’m listing below some of the people I’ve met on Twitter who talk about librarian/publisher issues:

In her own category: Kassia Krozier (@booksquare) – Not a librarian or a publisher, but in the center of the discussion!

Tim O’Reilly (@timoreilly) and his group at O’Reilly Publishing have created a unique gathering place for thinking about the future of publishing. The O’Reilly Radar blog has articles by Tim and a group of other writers, some with library connections, notably Peter Brantley (@naypinya). In addition, the annual O’Reilly Tools of Change for Publishing Conference (TOC) in New York has speakers from the library world as well as the publishing world. Writers and speakers for the O’Reilly blog and TOC conference appear regularly on @timoreilly’s Twitter tweets. Also, during the TOC conference, on-site Twitter reports are extensive. Joe Wikert (@jwikert), from the list above, also works at O’Reilly.

To read Seeing the Picture articles about issues of libraries and publishing, see the categories Publishing and TOC. Articles that have had comments/discussions with publishing people: Copyright in Google Books: Pictures & Text and Jon Orwant on Google Book Search at TOC.

Eric Rumsey is at @ericrumsey

Information Technology Strategic Planning

Report from Karl Schornagel, Inspector General of Library of Congress, presented to James Billington, the Librarian of Congress

[p 1-2] EXECUTIVE SUMMARY

The intent of this review was to assess the effectiveness of information technology (IT) strategic planning at the Library of Congress (Library or LC). To evaluate whether the Office of Strategic Initiatives (OSI) Strategic Plan supports and implements the Library’s Strategic Plan as it pertained to the IT infrastructure, the Library Office of the Inspector General (OIG) contracted with A‐TECH Systems, Inc. … the Strategic Planning process for IT at the Library of Congress is not well integrated with essential planning components, and is not instituted Library‐wide, resulting in the following findings.

1. STRATEGIC PLANNING PROCESS ‐ Strategic Planning for IT is not a unifying force at the Library, does not link directly to the Library Strategic Plan, and does not have a forward‐looking view.
2. IT INVESTMENT PROCESS ‐ Strategic Planning is not linked to the IT investment process, resulting in the duplication of efforts and acquisitions.
3. ORGANIZATIONAL STRUCTURE ‐ The organizational structure of the Information Technology Services (ITS) directorate at the Library does not foster strategic planning and good IT governance.
4. ENTERPRISE ARCHITECTURE ‐ The Library is missing an Enterprise Architecture program that should be coupled with a strategy to provide a roadmap for implementing future technology.
5. CUSTOMER SERVICE – ITS customer service needs improvement.

In our opinion, all of these findings are in large part the result of an unclear sense of how IT planning fits into the Library’s mission and the roles and responsibilities of the employees, as well as a lack of linkage between the IT strategic planning processes at the Library and actual performance. Furthermore, those Library employees charged with IT planning need to adopt a holistic view of planning that incorporates and supports a clear mission view with an insight into customer goals and objectives. Although some steps have been taken towards this effort, the progress is not seen Library‐wide.

We received a formal response to this report on April 15, 2009. Library management agreed with the majority of our findings and recommendations. Although management did not feel the improvements since the LC21 report (in 2000) were adequately addressed, we believe these improvements were sufficiently addressed in the executive summary and the conclusion of this report. Management responses and A‐Tech comments are included in the report after each recommendation. The entire response can be found in Appendix E.

[p 7] FINDING 1 – STRATEGIC PLANNING PROCESS

We found that the strategic planning process is not a unifying force at the Library of Congress and not incorporated into the organization’s culture. Specifically, we found that:

1. The Library’s Strategic Planning process was not inclusive of all internal stakeholders;
2. The Library’s IT Strategic Plan does not align well with the Library’s Strategic Plan;
3. The Library’s digitization efforts are scattered and lacking in specific focus.

[p 8] We do not agree with the decision of the Library’s leadership to make strategic planning a management‐only activity. We suggest that the Library allow line employees to actively participate in the strategic planning process. …

Lack of Buyin to Library’s Strategic Plan Below the Senior Management Level (Section headings in boldface)

In interviewing Library staff, we found that most felt they had not been active participants in the development of the Library’s Strategic Plan or in the IT Strategic Plan. Those interviewees who previously worked at other federal agencies felt that the Library’s processes for IT strategic planning were “immature” by comparison.

[p 9] Misaligned Strategic Plans and Ineffective Planning Process

[p 10] The Library Does Not Have a Focused Digitization Vision
… despite many successes, the strategy for “digitizing” the Library collections seems to lack an overall Library vision. …

[p 14] FINDING 2 – IT INVESTMENT PROCESS

We found that the IT investment process at the Library is not linked to its strategic plan.

1. The Library’s IT planning is not linked to an investment process.
2. There is duplication of costs.
3. There is no consistent Cost‐benefit Analysis (Analysis of Alternatives) done
by ITS.
4. The Library does not transparently track IT costs.

No Comprehensive Library Strategy for IT investments

Despite the MDEP process, we concluded that there is not an overall Library strategy for prioritizing and budgeting for IT investments to include new projects, replacement of existing systems, hardware, software, and services support.

[p 15] No Coordination of IT Costs across Library

[p 17] Inconsistent CostBenefit Analyses

[p 18] Lack of Transparency in Tracking IT Costs

[p 22] FINDING 3 – ORGANIZATIONAL STRUCTURE

The organizational structure of the ITS Directorate at the Library does not foster strategic planning and proper IT governance.

OSI Is Not Optimally Structured

[p 24] RECOMMENDATIONS

The organizational structure of the ITS Directorate needs to be realigned to foster strategic planning and IT governance at the Library.

[p 25] FINDING 4 – ENTERPRISE ARCHITECTURE

The Library lacks an Enterprise Architecture (EA) program. … We found that the Library has not yet implemented an Enterprise Architecture …

[p 28] RECOMMENDATIONS

The Library needs to implement an Enterprise Architecture that could be coupled with a strategy and provide a roadmap for implementing technology in the future.

[p 30] FINDING 5 – CUSTOMER SERVICE

Customer Support Issues

Our review indicated that beyond long‐term strategic planning issues, ITS customers were experiencing significant customer service problems. We believe that this condition is related to the lack of long‐term strategic planning in that ITS does not operate on a long‐term plan to monitor or improve customer service. …

[p 31] The Help Desk is staffed by contractors, whose quality is inconsistent. Help Desk contractors will often install the wrong versions of software and the customers will reinstall the software themselves. Customers have reported that instead of fixing a problem, the Help Desk contractors will frequently replace hard drives or recreate customer accounts.

[p 33] RECOMMENDATIONS

The Library needs to implement a formal process for soliciting customer feedback for recommendations, ideas, and complaints, and implement changes to improve customer service.

[p 35] CONCLUSION

Many recommendations made in this report can be implemented at a low cost and can be accomplished with existing resources. …

The LC21 (2000) report made the following recommendations, which still hold true today:

“…information technology can, should, and must be taken as a strategic asset of the Library as a whole and managed strategically from the very top.“

“…there needs to be serious strategic planning. Concrete projects must be established and undertaken to make real the Library’s ability to select, acquire, preserve, and manage digital content. These initiatives must reach across the whole interlinked set of processes from copyright registration through deposit to reader services.”

[Following p 49] Response, by James Billington. [The body of the document, excerpted above, is native PDF, and can be copied, but Billington's response is apparently scanned PDF, and not able to be copied.] Billington’s main objection to the report is that it doesn’t give LOC credit for having made progress since the last assessment was done in 2000 (LC21: A Digital strategy for the Library of Congress). But in responding to specific points in the new report, he’s mostly in agreement.

Eric Rumsey is at @ericrumsey