As described previously, this review included the following:

• hiring qualified experts to conduct the testing who understood the communications needs of people who are blind, who could read various forms of Braille, and who had experience in evaluating Braille and audio formats of publications; and
• evaluating a sample of federal institutions to determine whether they provide publications in alternative formats and whether these publications are of good quality.

The methodology selected for this review was similar to that of other studies conducted on behalf of the Commission. A sample of 50 institutions was drawn from the list of federal institutions governed by the Financial Administration Act (FAA), which are therefore subject to the Communications Policy. A short document was selected from the online publications list of each test institution. The consultants then ordered these documents, conducted a comparative analysis of the alternative format and the print versions, assessed the quality of the alternative format and commented on the service provided.

The review was conducted between January 23 and March 10, 2006. The documents were ordered between January 24 and February 15, 2006.
 
Groups Consulted
The Commission and Government Consulting Services consulted with four groups in the inception phase of this review: the Canadian National Institute for the Blind (CNIB), the Alliance for Equality of Blind Canadians (AEBC), the Council of Canadians with Disabilities (CCD) and the Canadian Council of the Blind (CCB). Additional information on these groups is included in Appendix B.

The information gleaned from this consultation helped the project managers better understand the requirements of print-disabled communities when communicating with federal institutions to order alternative formats of publications.

Experts
The consultants hired are blind, so they have expert knowledge and understanding of the communications needs of people who are blind or visually impaired. They are well versed in Braille and have significant experience using assistive technologies. They also have experience in providing expert advice in this area.

These experts, one anglophone and one francophone, carried out the actual tests. Their input and suggestions were useful in designing the testing protocol and finalizing this report.

This approach ensured that issues surrounding complaints of discrimination in service delivery were well understood.

Sample
A sample of 50 federal institutions was drawn from the list of federal institutions governed by the FAA and subject to the Communications Policy. Altogether, 102 institutions are included in the FAA schedules I, I.1 and II, so the sample of 50 institutions constitutes 49 percent of the total group.

Document Selection and Assignment of Format
Once the sample was established, the next step consisted of selecting a short document from among each institution’s online publications. These were typically brochures, forms or reports. The publications selected were short (no more than 15 pages), so that the cost of their translation to an alternative format would not cause undue hardship to the federal institution.

The consultants selected 29 English documents and 21 French documents.

As for the formats, the publications were split evenly between audio and Braille formats (25 documents for each format). The publications in Braille were further split between the Contracted (13) and Uncontracted (12) form.

Testing Protocol
The testing protocol was designed to test whether federal institutions are providing documents in alternative formats and how quickly they respond to requests. To simulate a real-life situation, the consultants did not mention that their request was part of a study. Rather, they indicated that they were ordering the document for their own purposes, using the following script:

I am blind and I require a copy of a document in (selected format).

The consultants provided the title of the document and, when available, the publication number. They also enquired whether there were any costs involved; if this was the case, they were to enquire about the price of the print version. The rationale is that all formats of an information product must have the same price, and if a print publication is free, so too must be all of its alternative format equivalents.

When the document was available at no cost in the assigned format—and this was the case for all available publications—consultants asked for the document to be sent to them as soon as possible and asked about the expected shipment timeframe.

If a document was not available, the consultants were asked to probe for the reason for this lack of availability, to determine whether the document was no longer available in print or if the institution lacked the capacity to produce it in an alternative format. The Communications Policy, as discussed earlier, also stipulates in requirement 1(h) “that prompt and clear explanations are provided when information requested by the public is unavailable.”

After each request, the consultants completed an analysis grid, recording the following:

• the date and time the consultant contacted the federal institution to request the publication in a multiple format;
• the date the order was confirmed, if it was different than the date of the first call;
• the availability of the document;
• the reason that a document was unavailable (when applicable); and
• the expected delivery timeline.

When the consultants received each document, they indicated the date of reception in the analysis grid. They then proceeded with the comparative revision and analysis of the document. The purpose of the analysis grid was to measure the quality of each publication’s alternative format as objectively as possible. To that end, consultants recorded the number of times the text or content of the alternative format differed from that of the print version, and they assessed the mechanics of the alternative format. For instance, they assessed whether Braille contractions were used properly and whether the audio had been recorded at a good pace.

The complete analysis grid is included in Appendix C of this report.

A Definition for “Within a Reasonable Time”
To complete this study, the consultants established a cut-off date for receiving the documents in an alternative format. It would be unreasonable to expect federal institutions to provide documents in an alternative format immediately. Conversely, the print-disabled community should not have to wait for lengthy periods to receive the service to which they are entitled. As a compromise, it was determined that the documents should be delivered within a reasonable amount of time, but what constitutes a “reasonable amount of time” is not clear. There are no rules, standards or guidelines for determining it. The consultants, therefore, concluded that the cut-off date had to be determined in the context of all the circumstances. These circumstances include the following:

• the Communications Policy requires that institutions offer a service that is “timely, courteous, fair, efficient and offered with all due regard for the privacy, safety, convenience, comfort and needs of the public”;
• documents on the Web are immediately accessible to those who are not print-disabled;
• print documents can be sent out within the same business day of the request and can be received within 10 to 15 working days of the date they were ordered;
• sometimes print-disabled people urgently need a particular document; and
• the documents requested did not exceed 15 pages in length.

As a result of these circumstances, the consultants made the conclusions in this study on the basis that “within a reasonable time” means within four weeks of the time a publication in an alternative format was ordered.

Limitations
This review was conducted over a short period—seven weeks. Although this is typically sufficient time to order and receive a traditional print publication, and despite the fact that 22 federal institutions were able to provide an alternative format within a reasonable time, additional time might have helped some federal institutions prepare and send the requested alternative format publication. When this report was being written (March 31, 2006), publications from seven federal institutions still had a “pending” status. In other words, the institution had promised to send the publication in the requested alternative format, but the consultant had not received it between seven and 10 weeks later.

Under the testing protocol, the consultants ordered and reviewed only one document in one language and in one alternative format from each federal institution in the sample. The particular document and format may not have reflected the extent of the services an organization provides or the general quality of the alternative publications an institution releases to the public.

The sample included 49 percent of the federal institutions governed by the Financial Administration Act. If all institutions had been tested, the results might have been different than those described herein.

Testing
The review was conducted between January 23 and March 10, 2006. The 50 publications, as determined in the sample, were ordered between January 24 and February 15, 2006. Both consultants called their assigned federal institutions to order the required publications. They recorded their interactions and observations in the analysis grids, and noted the expected time of delivery or the reason that the publication was unavailable.

In one instance, two departments that had recently been divided from one larger department still shared the same back-office capabilities, including their publications office. When this fact became known, the consultants replaced one of these new departments with another federal institution from the same FAA schedule.

The consultants observed that ordering the publications was frustrating.

• Ordering the publications in an alternative format took up to two weeks in some instances. However, non-print-disabled people can often order a traditional print document in a few minutes or download it immediately from the Web. It often took between three to five calls to confirm the availability of a publication in the requested alternative format.
• Most of the publications officers receiving the requests did not know how to get alternative formats. Consultants were directed to other publications officers, who often had to verify the availability of the document and phone the consultants back.
• Some publications officers did not speak French and this fact made ordering the French documents (in French) very challenging.
• Several publications officers thought that all blind and visually impaired Canadians use computer and adaptive technologies. Some suggested that the consultants visit their website and read the document in HTML format. Some departments are no longer willing to produce an audio cassette and offered a CD in its place. They didn’t see why it was necessary to provide their documents in Braille or on audio cassette. Although technology is helping print-disabled people access information, many Canadians who are blind, deaf-blind or partially sighted do not have the computer equipment or skills to use this technology.
• Two departments use Teleflash, an automated system for ordering frequently requested documents in several alternative formats. This system would be challenging for some individuals to navigate and there did not appear to be the option of speaking with a publications officer.
• The consultants needed to follow up with at least six federal institutions that were not returning their calls.
• The consultants mentioned that if they had been ordering the publications for their own purposes and not in the context of a review, they would have cancelled their request or not followed up with the departments due to the high frustration level.

Most publications officers wanted to be helpful but just didn’t know what to do. Some expressed concern when they were unable to grant the request, offering to do whatever they could to assist. Others just stated that the document was not available in the format requested. The reasons they offered for the unavailability of the publication in an alternative format are shown in the following chart.

Reasons Given by Federal Institutions Regarding
the Unavailability of the Requested Publication in
Alternative Format

Once the publication was ordered and its availability confirmed, production and delivery took anywhere from less than one week to over six weeks, depending on whether the document was available off-the-shelf, whether the department made the audio recording in-house, whether the department contracted out the production work, and whether the alternative format document was sent by Canada Post or courier. Some of the documents had not arrived by the close of the analysis period (seven weeks). One department indicated that the production and delivery would take up to eight months—an unacceptable length of time. For the federal institutions that did comply, the average document delivery time was 13 business days.