Handbook for Digital Projects:
A Management Tool for Preservation and Access
A Management Tool for Preservation and Access
VIII
Vendor Relations
Janet Gertz
Columbia University Libraries
Introduction
Quality digital conversion work can be accomplished in-house or through vending out to service providers. Regardless of whether digitization is intended to serve preservation goals, it is a waste of time and money to do a poor job. A digitally converted version of a document must be fully functional. If what is digitized is illegible or so poorly indexed that end users cannot find what they need or read it when they do find it, there is a failure to provide both preservation and access.
In order for digitization to be successful, it is essential that the institution have a clear understanding of its goal for digitization and what kind of final product will serve that goal. Understanding why a project is being undertaken will guide decision making, not only about image quality and user interfaces but about what work should be accomplished in-house and what work may safely be vended out. It is important to:
- Involve all the relevant participants (curators, technical experts, preservation officers) in determining the project goals and making the decisions that will shape it
- Keep a careful record of what decisions are made, and why, to prevent re-inventing the wheel when problems arise
- Document fully how and why work was accomplished both in-house and by the vendor in order to aid future preservation of the digital resources themselves.
There will always be an in-house component to any digitization operation. The institution that holds the materials to be digitized must take responsibility for:
- Selecting materials to be converted
- Determining the purpose of digitization and the nature of the desired product
- Establishing necessary quality levels
- Verifying the quality of the completed work.
There are arguments in favor of working entirely in-house and arguments for employing service bureaus. The difference lies in:
- The degree of immediate control over the work
- The variety of activities that can be performed
- Efficiency
- Economics.
Librarians and archivists are still learning what the parameters of high quality scanning should be. Few guidelines are in place, the vocabulary is not shared industry-wide, and it is not familiar to many librarians and archivists in any case. Institutions must experiment as they go along, and there is a steep learning curve for the institutions as well as for potential vendors. Relatively few scanning vendors work with libraries and archives. Most work routinely for organizations that want quick, cheap scanning and know little about preservation and the importance of high-quality, high-resolution images, and rich metadata. Luckily, the situation is improving and there are now a number of vendors with relevant experience who are willing to share their technical expertise.
Existing best practice and recommendations for preservation-quality resolution and tonality, based on the size of the smallest meaningful element of the document to be scanned, are laid out in earlier chapters. It also has been made clear in those chapters that every change of genre, format, and medium to be digitized introduces a host of decisions on resolution, tonality, metadata, storage media, and user presentation. There are no simple answers, and the interactions among various factors may have unexpected repercussions down the road. It can be difficult to determine which outcome is best for preservation and access purposes, and which set of procedures and technologies will provide that outcome. The lack of clear, nationally accepted standards and specifications could make it difficult to explain to vendors what level of quality is needed in order to achieve results that will be acceptable. Yet, when working with vendors, the institution must be able to state its requirements in clear, quantifiable, and verifiable instructions, and it must be able to recognize whether the product returned by the vendor is what was requested.
Why Digitize In-House
The primary argument for digitizing in-house is that it gives the institution close control over all procedures, handling of materials, and quality of products. There is no need to send valuable or fragile originals off-site and no worry about working with a vendor who turns out to be incompetent, provides something other than what was required, or goes out of business.
Working in-house is a good way to learn the technical side of digitization thoroughly -- useful even when most work in future will be sent to vendors. If it is not clear what sort of product would be best, working in-house provides a way to experiment on a small scale without the need to go through writing technical specifications and contracts. A small in-house pilot can serve as a prelude to vending out the bulk of the work. In-house work on a pilot project to try out a variety of approaches may not be efficient, but it is often a necessary step in the learning process.
Working in-house is probably most successful when:
- A project is relatively small scale and easily handled within any time limits, or can be broken down into small segments;
- The institution has skilled staff, or staff with real interest and incentive to learn, and support from the administration for in-depth training;
- The institution already has appropriate equipment or funding to acquire it. (But remember that equipment and software become obsolete at a frightening pace.)
Why Use Vendors
The primary benefits of working through vendors are financial and technical.
- The institution does not have to devote space to scanning, nor does it need to convert its space (possibly including construction) to suit electrical and other technical requirements.
- The institution does not constantly need to purchase the latest equipment and software. The vendor is responsible for keeping up with the times.
- The institution does not have to deal with hiring, training of sophisticated specialist skills, and management of staff.
- The vendor and not the institution copes with costly equipment breakdowns, downtime, and correction of errors.
- The institution benefits from the vendor's economies of scale and high productivity.
- Finally, the price is stated up front.
Many institutions do not own appropriate scanning equipment, and few institutions have sufficient budgets to keep pace with the latest equipment and software coming on the market. Further, there is often no one on staff with much production-level scanning experience, and it takes time and (unfortunately) learning through failure to build up experience. In theory at least, vendors can be expected to keep up with the latest hardware and software and to have fully trained specialist staff. They also should have a very good idea of what services they can offer and what it costs to provide them. The downside is that the institution is at a distance from the work. Careful quality control assurance by the institution becomes an essential. No one should take for granted that work is being done as specified without verification through detailed inspection of the vendor images and files. Although working with vendors usually entails sending materials -- or film intermediaries -- to a service bureau located somewhere outside the institution, some vendors may be able and willing to bring in equipment to carry out scanning on site. This offers some of the benefits of in-house work, e.g., original materials need not travel, closer oversight of the vendor work. But it also includes some of the deficits, e.g., the need to provide an appropriate work area, day-to-day scheduling issues, security, and insurance.
How to Choose Services and Vendors
Vendors are definitely not all equal. Even when using the same equipment and methods, some vendors produce a much better product than do others. Good vendors are really interested in learning what is important to fulfilling the institution's needs, while others may consider the project a run-of-the-mill job not worth any great effort. Still others want to sell their own proprietary systems rather than to act as a conversion service.
Identifying and selecting a vendor is not a quick or easy process, especially where large and complex projects are involved. Institutions may not need to go through every step described below, but all will need to work through the basics.
- Develop an initial concept of the project and its goals.
- Identify potential vendors.
- Send out an RFI (request for information) to explain the goals of the project clearly and to discover which vendors are interested and have ideas about how to handle it.
- Establish a project methodology and quality requirements.
- Develop a short list of vendors.
- Write an RFP (request for proposal) and send it to the short list along with samples to be scanned.
- Communicate with the vendors while they work on their responses, including site visits and meetings when possible.
- Evaluate and compare the vendors' proposals and select the best.
- Write and sign a contract.
- Work with the vendor during the project.
What are the Project Goals?
The institution must decide what it wants to have done and how. For example, is the aim of digitization a visual index to a manuscript collection, a reserve reading service, detailed reproductions of brittle books, or some combination of access and preservation goals? An institution may know what it ultimately wants to achieve without knowing how to get there. In this case, the institution can describe the desired end product to vendors and ask them to propose how to achieve it, within general guidelines. Other institutions may know in detail the specific requirements for the product in terms of resolution and tonality, file types and storage media, metadata to be recorded, and possibly even the desired type of equipment and software. In these cases, they need only locate a vendor capable of meeting those requirements.
In all cases, the institution must be able to express clearly to the vendor what it needs. This can be an iterative process:
- An RFI giving a very general description of the goals to a relatively large number of vendors
- Responses from some or all of the vendors proposing various possible approaches
- Selection of one or two approaches that seem most appropriate
- An RFP in which the acceptable approaches are spelled out in much more detail and sent to a short list of vendors
- Responses from the vendors with detailed procedural information and bids on prices.
The RFI - Request for Information
The purpose of an RFI is to gain general ideas about possible approaches and to identify potential vendors. If the institution knows what it wants as a final outcome but is not sure of the best methodology/technology/software/metadata, it can use the RFI process to gain an idea of the possibilities. This works especially well when the institution queries vendors who have significant relevant experience.
The RFI consists of:
- A brief description of the proposed project including amounts, timing, and desired outcomes
- A description of any methodology the institution has in mind
- A request to the vendors to comment on the methodology
- An invitation to suggest alternatives to achieve the same outcomes
- A request to the vendors to indicate whether they would be interested in bidding on the project.
For an RFI, it makes sense to contact as many potential vendors as look reasonably likely to be interested in the project. Their responses allow comparison of various approaches, assessment of the quality of the scanning under different circumstances, evaluation of the original assumptions about the product against the actual results, and a period of some review and possible redesign of the project.
There are many ways to identify potential vendors. One of the best is to seek recommendations from knowledgeable colleagues who have vended out work similar to what the institution intends. Reports on ongoing or completed projects may provide ideas. Attend conferences where colleagues discuss their projects and vendors present their services; check the Web sites of such institutions as the Library of Congress, the National Archives, the Research Libraries Group, and other preservation-related organizations.
The Request for Proposal (RFP)
A request for proposal (RFP), in contrast to the RFI, is designed to explain in detail to potential vendors the requirements and specifications for the project, the criteria that will be used to evaluate their proposals, and the specifics of how their bids should be presented. Writing an RFP and evaluating responses can be complex and time-consuming. Examples of documents provided by RLG and the Library of Congress are quite long and complicated (over 200 pages in one instance). Although this may discourage some institutions from even considering the process, reading through the sample documents is well worth the effort. An institution can extract from these exemplars the basic principles needed to construct its own simpler document and can adapt the language to suit its requirements rather than inventing the process from scratch. Still, it is important to allow sufficient time for writing the RFP, for vendors to consider the project carefully, and for evaluating their responses. The whole process can take several months. The good news is that the RFP can translate fairly directly into the eventual contract.
Once the institution has a good idea of the specifics of project methodology, it is time to write an RFP and send it to the short list of the vendors who appear most likely to be able to accomplish the project. The goal is to give vendors full information so that they understand what is desired and can make a reasonable cost estimate for the work. The value of an RFP process is that it elicits explicit responses that can be objectively evaluated to select the best vendor. Further, clearly stated specifications can help the institution avoid the need to accept the lowest bidder if that bidder cannot satisfy the specifications.
The RFP should be clear and explicit, with a specific technical description of the deliverables and how compliance will be evaluated. The RFP should be broad enough to allow for different vendors to propose alternative methodologies where appropriate, but specific enough to ensure that they understand the standards they are required to meet. Divide the RFP into sections that deal with technical requirements, management requirements, pricing, and references. Where possible ask the vendors to provide their responses in a standard format, to facilitate comparison of competing bids.
Contents of an RFP include:
- A description of the project in terms of the ultimate objectives
- A description of the objects to be scanned in as much detail as possible to help the vendor make an intelligent bid
- The quantity and physical nature and dimensions of the materials
- A consideration of the varying sizes of the material to be scanned. Are the materials reasonably uniform throughout? If multiple genres are included, describe each group separately.
- A description of proportions. What proportions are easy or difficult to work with?
- A consideration of the content of materials. Is there an intellectual structure to the materials that must be maintained? Will vendors be able to batch each type of material, or must materials be handled in an order that mixes different sizes and types?
- A consideration of language. What languages does the vendor need to be able to read (text or page numbers) in order to carry out the project?
- Detailed instructions about the preferred methodology, including resolution, tonality, bit depth, file formats, compression, platform, and storage media
- Instructions for producing derivative images as well as the master images
- Definition of the required level of accuracy and how the institution will evaluate it
- Instructions on file naming and metadata
- How to format file names
- Whether pre-existing identification numbers or other information must be keyed in
- What information about processing and equipment must be reported (e.g., kind of scanner used, its settings, color definitions used, date of capture, description of film stock if film intermediaries are scanned)
- Requirements for how the data should be coded in and laid out, and what type of spreadsheet or database to use
- Schedule for weekly/monthly deliveries, deadlines, and turnaround time
- Handling (and lighting levels if that is an issue), security, insurance, and shipping requirements for original materials
- Name of the person at the institution the vendor should contact with questions and to whom the bid should be sent and in how many copies.
- List the hardware and software they would use (Are files and data in proprietary systems acceptable?)
- Specify their quality control procedures
- Describe their production capacity and document that they can accomplish the work at the specified quality within the timeframe
- Explain how delivery of materials and files will be accomplished (vendor pick up, courier, UPS, or other)
- Describe environmental controls in the facility if that is an issue for original materials.
- Provide the name and qualifications of the project manager
- Supply references for similar work done with other libraries, archives, or museums
- Scan a representative sample that represents a fair cross section of the materials, including both easy and difficult items (If the originals are valuable or fragile, the sample should consist of reasonably similar items that are less valuable or are expendable.)
- Respond with a price proposal
- State prices in specified units of measure, for instance per page, per image, or whatever is appropriate
- Costs for data input, cost of storage media, shipping, insurance, and any other additional charges
- Determine if prices are firm for the duration of the project
- Provide suggestions for alternative methods that can accomplish the project at the same level of quality.
|
Communicating with Vendors During the RFP Process |
|
|
Expect questions from the vendors as they work through the RFP. Insightful questions can help refine the project plan. Depending on the circumstances, and especially if vendors will be scanning original materials, invite them to attend a meeting at the institution to see the materials and participate in a question session before they respond. If possible, make site visits to see whether each vendor has the capacity and staffing to handle the work and whether the facilities are clean and well managed. |
Evaluating Responses from Vendors
While writing the RFP, the institution should be building a plan for evaluating the responses and writing up criteria for objective and accurate comparison of the vendors' abilities to meet the specifications and requirements. Some people advocate setting up a numerical rating system, with higher weight given to the more important aspects of the proposal. A rating on a scale of 1-3 might be given to each factor to be evaluated, with the more important factors then multiplied by a weighting factor. Whether or not actual numerical ratings are assigned, the most important factors (sometimes called critical success factors) are the ones necessary for a successful project. For instance, the ability to provide a database for the metadata may be required (critical), while any serviceable software may be acceptable.
Write up criteria and benchmarks for evaluation of image quality and metadata accuracy. Establish how you will perform image quality evaluation; determine what viewing software, monitor, and printer will be used; and decide who will make the evaluation.
Criteria for assessing bids include:
- Quality of vendor products and technical methodology
- Appropriate overall technical approach
- Ability to produce sample work that meets or exceeds the RFP specifications
- Familiarity with existing guidelines and best practices
- Satisfactory quality control procedures
- Identification of unusual items, and judgment in asking the institution for further instructions
- Ability to handle original materials safely and house them securely
- Demonstrated understanding of the scope of work and the requirements
- Clear evidence that the vendor really understands the project
- Ability to answer all RFP questions in the terms requested. (If they can't follow instructions, will they be able to do the work properly?
- Evidence of ability to carry out the whole project
- Size of the organization (Is the company large enough to handle the project or will it need to hire new, inexperienced staff?)
- Ability to accomplish the work within the project timeframe and schedule
- Successful previous experience with similar work
- Staff and facilities
- A bid that is professional in appearance and presentation
- Personnel with appropriate experience and a cooperative, service-oriented, professional attitude
- Up-to-date equipment and clean, well-organized facilities
- Commitment to the work and to the long-term relationship that will be necessary for a successful project
- Financially sound basis for the company
- Reasonable cost proposal.
Carry out the evaluation of the quality and technical adequacy of the responses separately from comparison of the cost proposals. Establish which of the vendors meet the criteria before determining which price is best. Eliminate any vendor who cannot meet the quality and technical criteria, regardless of the price.
Compare the RFPs and samples carefully using the established criteria. Discuss any unclear, unexpected, or unsatisfactory issues with the vendor. Consider providing a second chance if an unsatisfactory result was due to misunderstanding of the requirements.
Call the references and conduct thorough discussions of vendor quality, service attitude, turnaround, and other factors. Before calling, develop a series of questions to make sure no important issues are omitted.
Rank the vendors whose quality, workflow, technology, and facilities best meet the criteria and needs of the project.
Compare the price bids to identify the vendor with the best combination of high success scored in the evaluation of the bids and samples and low price. Beware of any bid that is priced unrealistically low (or high). Vendors are well aware of each other's prices and of their own profit margins. Vendors who propose similar work will generally fall within a range of prices (although companies that are either very small or part of a very large corporation may fall toward one extreme or the other). If any vendor's price is significantly lower, be wary. A very low bid may indicate a vendor who cuts corners or has failed to understand what is really required to achieve the final product. Very low prices can be a hint of low quality unless that vendor is much larger and more experienced than the others, is located somewhere where very low wages are paid, or is proposing a completely different solution. On the other hand, very high price is not necessarily a guarantee of very high quality -- it may simply indicate over-charging.
If the low bidder is not the preferred vendor, determine whether the low bidder can meet the specifications at all based on samples, references, and bid statements. If not, this forms grounds for rejecting the bid. If no one vendor combines all desired factors, discuss possible changes with the preferred vendor(s) to bring them closer to the desiderata.
Depending on the institution and the complexity of the work to be done, anything from a simple letter of agreement to a full contract may be required before vendor work on the project can begin. The specifications laid out in the RFP serve as the body of the agreement, to which are added logistic, legal, and financial details. Needless to say, involving the institution's financial and legal offices early in the process is recommended.
Contracts normally begin with sections covering the legal obligations of the two parties and a description of the work being contracted. Details, such as procedures, can be attached as appendices. The contract should:
- State what the project goal or product is supposed to be
- Describe briefly what the institution is responsible for sending, including original objects, film intermediaries, list of file names or a filenaming scheme
- Describe briefly what the vendor is agreeing to do, for instance, produce digital images, carry out OCR, create metadata files
- State the legal terms covering subcontracting
- Specify terms for accepting the product by defining the minimum acceptable level of accuracy
- Specify how errors will be defined and corrected, what error correction will be cost-free, and what the institution must pay for
- Specify how materials and files are to be transported and handled
- Specify insurance, security, and storage environment while materials are at vendor and in transit
- Name the primary contact on either side, and arrange for periodic visits to the site and visits from the vendor to the institution
- State that the materials and any products produced from them are the property of the institution and may not be used or distributed for any purpose without official written permission from the institution
- Specify frequency of reports and invoices and what information they must contain
- Specify deadlines and penalties for missed deadlines
- Define what will constitute default, how to dissolve the contract amicably on mutually specified grounds, and how to handle arbitration
- State the prices and guarantee that prices will remain firm for the duration or will increase only under specified conditions.
The second part of the contract lays out the technical specifications, including:
- Equipment and software to be used
- Storage media to use
- Specifications to follow for resolution, tonality, file formats, compression, and so forth
- The form in which to enter filenames and metadata
- How the vendor will carry out quality control.
Appendices cover details of:
- In-depth descriptions of the materials
- Schedule, timeline, benchmarks
- Error correction, handling of originals, shipping
- Samples of work forms
- Other useful information.
Once the contract is written to the institution's satisfaction, a common procedure is to send two copies to the vendor. The vendor will probably have changes to suggest. If negotiations result in significant changes, a new version may be needed. If the changes are small and simple, they can be written in and initialed by the vendor and the institution. Once the contract is fully settled, the vendor signs both copies and returns them to the institution. The institution signs both, and returns one copy to the vendor.
Working and Communicating with Vendors
Together with careful planning beforehand, the keys to a successful project are flexibility and constant communication with the vendor during the project. The better the communication between the institution and the vendor, the better the project is likely to go. It is important to assign responsibility for day-to-day communication to one person in the institution who is closely involved with the project, even if that person must refer some questions to others in the institution. In the same way, ask the vendor to name one person to serve as the institution's project contact. Encourage the vendor to communicate in a timely manner by telephone, fax, or email whenever issues arise, and be conscientious about responding quickly. Delays in solving a small problem can hold up the entire project. If possible, visit the vendor during the project and invite the vendor to visit the institution as well.
The essence of good vendor relations is to be fair to the vendor. Stay on schedule, or if unavoidable delays arise, inform the vendor as soon as possible and be prepared to shift the entire project schedule. Vendors schedule the work they take on fairly tightly. If delays at the institution's end push the project out of its assigned window of time, the vendor cannot slow down other institutions' work to recoup the lost time.
Maintain agreed-upon levels of productivity for shipments to the vendor and avoid unannounced changes in the nature or quality of the materials. For instance, if the vendor bids on the basis of a sample of uniformly legible and nonbrittle materials, inclusion of a significant amount of low contrast or brittle items can seriously affect the anticipated workflow and productivity. Significant increase or decrease in what the institution sends also can throw off the vendor's workflow and should be negotiated. Be sure all materials are prepared in the agreed upon manner before they are sent to the vendor. Finally, label all materials clearly and consistently so that the vendor can easily determine what is what.
Working with Vendors: Quality Control and Handling Corrections
Other chapters describe how to carry out quality control on images. Accuracy of vendor-supplied file names and metadata also must be verified, since erroneous metadata or miskeyed file names in essence mean that an image is lost. Calling up image after image and examining them carefully for flaws is a very time-consuming operation, but it is essential to ensuring that the vendor's product meets specifications.
When working with an unfamiliar vendor, it is especially important to carry out thorough quality control as early as possible in the project. Timeliness in returning errors is important, since:
- It prevents the vendor from continuing to replicate errors
- It alerts the vendor to problem procedures or ill-trained staff
- There is normally a cut-off date (often several months) after which the vendor will no longer accept errors for free correction.
If the project is small it may be possible to examine every image, but most projects are too large. In this case, recommended procedures are as follows.
- Set up a manageable first shipment that will be due back at an early date in the project.
- Perform careful quality control on 100% of the images and metadata in the first returned shipment.
- Record all errors in detail on quality control worksheets.
- Evaluate the errors to determine which are the institution's responsibility due to flaws in the institution's own procedures, to information that the institution failed to give the vendor, or to variations in the materials being scanned that the vendor should have been warned of. The institution will need to pay to have these errors corrected and obviously will need to revise procedures to avoid them in future.
- Determine which errors are due to mistakes by the vendor. If the percent of errors is higher than the agreed-upon rate, return the entire shipment with a full explanation of the errors and require the vendor to start over and produce a new batch. As necessary, discuss changes in vendor procedures for image capture and quality control.
- Repeat the 100% inspection. If the error rate is still too high, send it back again to be redone completely. If the vendor cannot get it right by the third try, it may be time to renegotiate the whole project.
- If the vendor's work meets or is lower than the agreed-upon error rate, return only the individual problem cases for corrections.
- Continue 100% inspection for the first two to three shipments.
- Once it is clear that the vendor is regularly returning a product below the acceptable error rate, cut back to a lower percent (often 10%) inspection of every shipment. This does mean that a few errors will go undetected until the day some user tries to access those images.
- If the error rate begins to climb, return to 100% inspection until the problem is identified and solved.
Electronic Imaging Request for Proposal Guidelines, ANSI/AIIM TR27-1996.
Sample RFI and RFP documents also can be found at:
The Library of Congress. The American Memory Project Background Papers and Technical Information site includes three National Digital Library RFPs for scanning and text conversion services. These are long, complex documents with all the bells and whistles required by a federal agency. http://memory.loc.gov/ammem/ftpfiles.html
Research Libraries Group (RLG), Preservation Program Tools for Digital Imaging, provides a series of documents produced by Cornell University's Department of Preservation and Conservation for RLG. They include a worksheet for estimating costs, guidelines for creating an RFI and an RFP, and a model RFI and RFP. Significantly simpler than the Library of Congress examples, the models are very thorough in their coverage of the issues. http://lyra.rlg.org/preserv/RLGtools.html
|
|
Northeast Document Conservation Center
100 Brickstone Square
Andover, MA 01810-1494
Telephone: (978) 470-1010
Fax: (978) 475-6021
http://www.nedcc.org
Last Modified: January 21, 2003
Copyright 2000. Northeast Document Conservation Center