Certification Programs


[ Franšais ]
PWGSC NMSO EZ107-120003 (Imaging Hardware) Valid from November 1, 2013 to March 31, 2018

BENCHMARK TESTING AND SCORING METHODOLOGY

INTERTEK technicians will evaluate each Hardware product submitted to the benchmark as follows:

Testing will consist of:

  • Performance
  • Features
  • Usability
  • Quality

    I10.1 Overall Hardware Score Methodology

    An overall score for each Hardware product will be derived by averaging the weighted scores for each of the four major INTERTEK tests: speed, quality, features and usability. The final overall scores are presented on a scale from 1 to 10, with 10 representing the best possible score in a single category.

    Test Weight

    Speed 25%

    Quality 25%

    Features 25%

    Usability 25%

    I10.2 Speed Scoring Methodology

    The Speed of each Hardware product will be tested using a series of predetermined files. INTERTEK uses a number of real world application files in order to give a more realistic indication of the printer capabilities. The files were created using Microsoft office, Adobe Acrobat, and AutoCAD applications. Each file type contains one or more of the following elements: graphics sparse text, dense text, fonts, bit-mapped images. The Microsoft Word document contains text, embedded Excel charts, and graphic images. The Microsoft Excel document contains a combination of simple spreadsheets, and a variety of charts. The PowerPoint file is a basic presentation including text and graphics. The Adobe Acrobat PDF file is similar in form to many PDF files in use today, it contains a combination of text and graphics.

    A Hardware product's raw score on any given test is the time it takes to generate one of the predetermined files. All tests will be performed at least twice to check for relative consistency. All raw scores will be recorded in number of seconds. If the speed times for two runs of the test vary more than 5%, INTERTEK will rerun the test until two scores obtained are within a 5% degree of variance.

    110.3 Quality Scoring Methodology

    INTERTEK rates the quality of all Hardware using the following test categories, where applicable:

    Monochrome objectives

    Colour objectives

    The quality of each Hardware product is evaluated by INTERTEK using the output from several Postscript files. Quality testing will be conducted at the highest resolution and quality settings available for the product. In order to create comparable quality tests for printers which may not have PostScript capability, INTERTEK has developed its own PostScript interpreter, as a part of its' test suite. The quality output contains both subjective and objective scoring. However, the subjective scoring applies only to the colour Hardware.

    Objective tests:

    The monochrome quality tests consist of a series of pages containing such elements as a photographic image, text in a wide range of sizes, and a variety of circles and horizontal and vertical lines. These tests also stress more esoteric features, such as the Hardware's paper-handling ability and how well the Hardware displays reversed (white on black) text and graphics. These tests demonstrate a Hardware product's suitability for graphics-oriented applications. The colour quality test examines the ability of the hardware to print a wide range of hues as well as to blend between the hues. It also examines ink placement problems - the ability of the Hardware to accurately place lines without misregistration or smearing. These tests are self-scoring and absolute in that each score is only a function of the particular Hardware's abilities, it does not compare one Hardware product against another.

    Subjective tests:

    One output consists of a set of five photographs scanned on an HP IICX scanner. They were scanned at 200x200 dpi, 24 bits per pixel. Two other images are composites of photographs scanned at 600x600 dpi. The files are full-page images, a bit larger than 8.5x11 inches. The quality of these pages are judged relative to the other printers in the grouping. When quality output from all Hardware in a category is completed, an INTERTEK technician will group the output into different categories. Although output will be marked on the back to identify the Hardware from which it came, the tester evaluating the quality of the output on the printed side of each sheet will not know what Hardware produced any given sheet of output. The technician will judge each element of the output in a given category on a scale of 1 to 5, with 1 representing the relatively worst output and 5 representing the relatively best output.

    Each quality output file will be generated on both paper provided by the Offeror and transparencies.

    To calculate the overall quality score for each Hardware product, the ratings assigned to each major element will be averaged for each image printed; an average of all of the scores will be calculated to achieve a score for both vendor paper and transparencies. The scaled quality score will be calculated, by multiplying the scaled objective score by 0.8 and adding the result to 0.2 times the scaled subjective score.

    The final score for each Hardware product will be adjusted to a 10-point scale for easy comparison. An overall quality score of 10 thus represents the highest possible score for a Hardware product in a category.

    I10.4 Features Scoring Methodology:

    Prior to the beginning of benchmark testing, Offerors will be issued a features questionnaire for each Hardware product they are submitting. The questionnaire will consist of more than 40 items for which Offerors specify the particular attributes for each hardware product. During the course of benchmark testing, INTERTEK technicians will review the features questionnaire and verify the responses against the Haredware and documentation which will be provided for the benchmark. Individual features for each Hardware product are tabulated in the results section of the report.

    The verified features for each Hardware product will be scored according to the following methodology. A number of items are assigned weights and scored (e.g., auto switching between ports). Weighted items are assigned a weight from 0.5 to 2. Exact weights for each item will be found in the first column of the features table in the data section of the report. A Hardware product receives a score of "1" for each weighted feature it includes. The Hardware receives a score of "0" for each weighted feature it does not include. If an item is listed as optional, the printer receives a score of ".5" for that item. All items for each Hardware product are then multiplied by the corresponding weight and compared to the total possible number of points for the category. The final score for each Hardware product will be adjusted to a 10-point scale for easy comparison. An overall features score of 10 thus represents the highest possible score for a Hardware product on the features questionnaire.

    Many of the items in the questionnaire are included for informational purposes only and are not used in the computation of the features score (e.g., voltages supported).

    I10.5 Usability Scoring Methodology:

    During the course of benchmark testing, INTERTEK technicians will review and score each Hardware product on a number of predefined usability criteria. The INTERTEK technicians will work with the Hardware and documentation to perform some basic tasks such as setting up the Hardware, loading paper and changing some items on the Hardware's setup/control menu. Points will be assigned based on the functionality and ease of use of the Hardware.

    An overall usability score will be derived by averaging the scores for all items. The final score for each Hardware product is adjusted to a 10 point scale for easy comparison. An overall usability score of 10 thus represents the highest possible score for a Hardware product on the usability questionnaire.