Invitation to Service Providers

 Date: Fri, 2 Jan 2004 11:47:28 -0800 (PST)
Subject: California Newspaper Digitization Project

Greetings,
On behalf of the California State Library, the California Preservation Program is reviewing options for digitizing microfilms of California historical newspapers and creating a database searchable by title, volume/issue, date, and full text by key words. We estimate the project to be one million pages of newsprint dating from the mid-19th through the early 20th century.

Our review, to be completed by June 2004, has several goals:

1. Determine the amount of information able to be captured digitally from existing microfilms of historical California newspapers, using a film from Alta California (the earliest California newspaper with statewide scope) as the test case.
2. Document the strengths and limitations of available newspaper search and retrieval software to return meaningful results to researchers' queries.
3. Compare benefits and costs among providers of digitization/database creation services, and compare these to options for partnering with vendors of commercial online newspaper databases available by subscription.
4. Estimate production requirements and costs to create a database of historical California newspapers of sufficient scope and size (estimated at one million pages) to be a significant research resource for the study of California history.
5. Create a publicly accessible website to report the findings of the study, to showcase the benefits of online access to historical newspapers, and to enable providers of services as well as vendors of subscription databases to demonstrate the value of their products and services.

Would you be interested in participating in the review and subsequent production project? Participation in the review would include:

• receiving in January 2004 a test 100 foot roll of 35mm microfilm representative of the quality of printing of Alta California in the later 19th century, as well as of variations in microfilming standards from the 1940s through the 1980s;
• digitizing the film with your capture technology of choice;
• creating a test database using the software of your choice for processing, managing, and retrieving data in response to Web-based queries;
• providing a url to your test website, to which we would link from the project website, thus enabling the review team as well as interested members of the library community to discover the excitement of California historical newspapers online, and to review for themselves the strengths of different approaches to creation of online newspaper databases;
• providing cost estimates for creation of an online database of California historical newspapers (i.e., one million pages) using the technologies and processes you demonstrated above;
• completion of all steps above by April 15, 2004.

Please respond by January 15th if you are interested in participating in this review. The decision to proceed with the subsequent production project depends upon satisfaction with the test database and with costs for completion of the project. If you need additional information, my contact information is below. Thanks very much for your interest.

22 January 04
To: Participants in the California Newspaper Digitization Review
Fr: Re: Test films and more general information

Many thanks for your interest in participating in this review and subsequent production project. In addition to enabling California to decide how to proceed, I anticipate that your participation will spur more interest than ever before throughout the library community in online access to historical newspapers.

Some of you have asked who will be participating. Participants to date are:

• Apex Epublishing Data Conversion Services
• BMI Imaging
• Bytemanagers
• Cold North Wind
• Heritage Microfilm
• iArchives
• Northern Micrographics
• OCLC/Preservation Resources
• RCI Image Systems
• TechBooks

Others of you have asked if we have evaluation criteria; the answer is "not yet" because the evaluation team has not yet been assembled. However, I can say that the evaluation team will include newspaper historians, genealogists, librarians, and researchers. I also can offer the following list of preferences for the end product, which should guide, but not limit, your work to create what you believe will be the best possible product.

• TIFF for the preservation files to maximize probability of future access to the content;
• grayscale and as much image resolution as possible, based on an assumption that the next generation database may need to begin with the original image files.
• ability to retrieve and view responses to searches in context and with the appearance of the original newsprint;
• emphases on both readability and searchability of the text;
• relevant responses to searches (as determined by the evaluation team) rather than large numbers of false hits;
• a very user-friendly web interface because our typical end user (public library user) will not have data base searching skills and experience;
• reasonable response times for data base users with typical public library platforms:
  • ISDN or faster network connections (T1 lines are not uncommon);
  • Windows and Mac operating systems released within the last five years;
  • Internet Explorer and Netscape browsers released within the last three years;
  • Adobe Reader plug-ins for the browsers released within the last three years.

A 100-foot roll of second generation negative 35 mm microfilm, contact duplicated from the preservation masters, is being made now for each of you. If I don't already have a contact name and UPS delivery address from you, please forward the information as soon as possible. I anticipate sending the test rolls the first week of February.

Again, many thanks for your participation in this review.