To:       MAB Distribution

From:     Gary C. Dixon
          John W. Gintell
          R. C. vonSeeburg

Date:     February 13, 1981

Subject:  Revised MAB on TR Goals


PLEASE REMOVE THIS COVER LETTER BEFORE FILING THIS MAB


This MAB describes changes to MAB-044, which defines  Multics  TR
Processing  Goals.   Most  significant  changes  in definition of
goals include the following:

(1) On page 3 in discussion of response, the  following  sentence
    was removed.

       Response  time  includes  the  time  needed  to  route the
       response back to the submitter.

    The TR System does not include such time.  Perhaps it should,
    but it doesn't.

(2) Removal of concept of installed-in-EXL being a resolution  to
    a problem.  This decision was made some time ago, but the MAB
    was never updated to reflect the policy change.

(3) The TR goal is stated in terms of  completing  processing  of
    75% of TRs within the expected amount of time (times given in
    Table  1).   The  prior  MAB  did  not  state  this 75% goal,
    implying our goal was to process the average  TR  within  the
    expected time.

    The  goal  is  measured as percent_completed_on_time for each
    type and processing stage, weighted by the number of  reports
    completed in that stage.  The exact algorithm is given in the
    MAB.


All  other  changes  are minor wording changes or clarifications.
The intent of this revision is to reflect  the  current  policies
and procedures now in effect.


MULTICS ADMINISTRATIVE BULLETIN                    MAB-044, Rev 1


To:       MAB Distribution

From:     Gary C. Dixon
          John W. Gintell
          R. C. Von Seeburg

Date:     February 4, 1981

Subject:  Goals for Processing Multics Trouble Reports



INTRODUCTION

This bulletin defines performance goals for processing of Trouble Reports (TRs) entered by Multics users through their SiteSAs, and categorized in the TR System as external TRs.

DEFINITION OF TERMS

The criteria for measuring achievement of TR processing goals are defined using the following terminology.

Types of TRs

There are three kinds of trouble reports. 1. Problem Reports: reports of problems in installed system programs, info segments, or system documentation. 2. Questions: questions about operation of the system, or about installed system programs, info segments and documentation. 3. Suggestions: suggestions for changes/improvements to installed system programs, info segments, or system documentation. _________________________________________________________________ Multics Project internal working documentation. Not to be reproduced or distributed outside the Multics Project. _________________________________________________________________

Problem Report Priorities

Problem reports are further divided into Priority groups, according to the significance of the problem. 1. Problem Reports A. Critical Priority, a problem which: - interferes with system operation to an extent that usefulness of the system is significantly limited for most users of the system. (1) _________________________________________________________________ (1) This includes, but is not limited to, situations in which a site is totally down, or is running in a highly crippled state. _________________________________________________________________ - irreparably compromises the security or integrity of data stored in the system. Data storage problems at system priority must affect many system files, or files which are difficult to retrieve or recreate. B. High Priority, a problem which: - causes installed system programs to fail to operate correctly in limited circumstances, with no known bypass for the problem. Such problems can be tolerated for short periods of time. - compromises data security or integrity for a small number of files in a way which can be repaired without significant loss of data. Such problems can be tolerated for short periods of time. C. Normal Priority, a problem which: - causes installed system programs to fail in a minor way which can be bypassed, or ignored. - results from incorrect documentation.

TR Processing Operations

Processing of TRs involves the following steps. 1. Submission occurs when a SiteSA or user enters a problem report, suggestion, or question. A report is entered by using the enter_problem_report (epr), enter_suggestion (es), or enter_question (eq) command on System M. A SiteSA usually reports Critical problems to Multics Software Support (MSS) by telephone to avoid delay in processing. For performance tracking purposes, MSS personnel enter such telephoned TRs into the TR system after the initial telephone conversation is complete. 2. Verification of a report validates the user's statement of problem, or initially appraises suggestions, or answers questions when answer is known. Verification is performed by the TR Administrator or investigator in Phoenix. 3. Routing of a report forwards the report to developers, investigators, and interested parties. Routing is performed by the TR Administrator in Phoenix. 4. Response to a problem report: acknowledges existence of a bug, indicates a bug bypass if any, and states that problem will be fixed in a future release; or it indicates that the problem cannot or will not be fixed (a system limitation). No response is required for questions or suggestions. (2) _________________________________________________________________ (2) The answer to a question is provided when the question report is resolved. Current management policy states that response to, and resolution of, suggestions by the developer is optional. _________________________________________________________________ Response is made by the developer, investigator or TR Administrator, and is forwarded to the submitter by the TR Administrator. A response is entered by using the answer_trouble_report (atr) command. 5. Resolution of a problem report occurs when changes to installed system programs causing the problem are submitted for installation in the Multics System Libraries, or when documentation errors are corrected; or when the nature of a user error causing the problem is pointed out to the submitter; or when the report is unclear or unreproducible and additional information about the problem is required. Resolution of a question report occurs when the question is answered. Software problems are considered resolved when the programs are submitted for installation in System Libraries on System M, because at this point, an emergency fix can be shipped to the site if this is deemed necessary. Bug fixes are not normally distributed between releases however, and the time from submission of installation of software on System M until distribution of the bug fix is NOT included in the resolution time. Documentation is considered fixed when the corrected manual is sent to the Brighton Print Shop. A trouble report resolution is entered by using the answer_trouble_report (atr) command. Developers are responsible to entering notification that a problem fix has been submitted for installation into the System Libraries. The Manager of Multics Technical Documentation at CISL and the Project Leader for the Multics Documentation Project in Phoenix are responsible for entering final resolution notice when manuals containing corrections are sent to the Brighton Print Shop.

TR PROCESSING PERFORMANCE GOAL

The goal for TR processing performance states that 75% of all TR processing operations will be completed within a specified amount of time, where times are measured from submission of the TR into the TR system to completion of the TR processing operation. The procedures for TR processing state the order in which TRs are to be processed, and the expected amount of time needed to perform each TR processing operation.

Expected TR Processing Times

The expected amount of time required to perform each TR processing operation is shown in the table below. Times are expressed as the number of calendar weeks which should elapse from time of submission to the time of operation completion, averaged over all TRs whose operations have completed during the reporting period. Because the times are averages, they do not state the processing performance expected for a particular TR. They instead address our overall TR processing performance. Table 1: Average Calendar Weeks from Submission to Completion of TR Processing Operation OPERATION Problem Problem COMPLETED Critical High Query Normal Suggestion ________________________________________________________________ | | Verification | 0.2 1 1 2 4 | | | Response | 1 2 4 13 -- | | | Resolution | 4 13 4 52 -- | |________________________________________________| Verification and routing operations are performed as a single processing operation, and are therefore shown as a single processing goal in the table above. Also, as the table indicates, no response is required for questions. No response or resolution is required for suggestion reports since current management policies state that response and resolution of suggestions is optional.

Formula For Computing Performance To Goal

_____ _____ \ \ # ops completed total_ops = > > since 07/01/80 /____ /____ for problems (critical, for verification, high & normal) and response and questions resolution ops _____ _____ \ \ # ops %_on_time %_on_time = > > completed * for op /____ /____ since 07/01/80 for problems (critical, for verification, high & normal) and response and questions resolution ops %_performance = %_on_time / total_ops Our goal is: %_performance >= 75%

Order of Processing TRs

TRs should be processed in the following order of importance: 1. Problem Reports, Critical Priority 2. Problem Reports, High Priority 3. Questions 4. Problem Reports, Normal Priority 5. Suggestions Type of report and priority of problem reports are initially specified by the submitter of the report, but can be adjusted by investigators, TR Administrator (who routes TRs) and developers as the report becomes better understood.

REPORTING OF PERFORMANCE

TR Processing performance will be reported to the Director of Software Development on a quarterly basis. It will be reported weekly to the Manager of Multics Software Development in status reports. Reporting will begin with statistics for reports submitted during the third quarter, 1979.