MULTICS ADMINISTRATIVE BULLETIN MAB-069 To: MAB Distribution From: Gary C. Dixon Date: August 13, 1985 Subject: Multics Configuration Management: Guidelines for Auditing Software ABSTRACT This MAB is one of a series of MABs on Multics Configuration Management. Refer to MAB-070 for an overall statement of Multics configuration management policies. Refer to MAB-066 for an overview of Multics software development procedures. The particular configuration management policy which applies to this MCR is stated as follows. POLICY: All Multics software changes must be audited before being installed in the System Libraries. The auditor is responsible for reviewing and approving the changes, and for ensuring that the developer corrects any problems found during the audit. An auditing checklist showing results of each audit session must be attached when submitting changes for installation. The auditor has final control of the changed software just prior to its submission to ensure that all code submitted for installation has been properly audited. To implement the above policy, this MAB gives basic procedures and guidelines for auditing of Multics software modules. Software auditing is a fairly complex activity. Only minimum procedures and guidelines are described in this MAB. The developer and auditor can mutually decide to extend these procedures or to apply additional guidelines for a particular auditing task. _________________________________________________________________ Multics Project internal working documentation. Not to be reproduced or distributed outside the Multics Project. - i - MAB-069 Audit Guidelines CONTENTS Page 1: Overview . . . . . . . . . . . . . . . . . . . 1 2: Procedures for Auditing Software . . . . . . . 3 2.1: Procedure for Auditing Planned Changes . . . 4 2.1.1: Select an Auditor . . . . . . . . . . . . . 4 2.1.2: Provide Auditing Materials . . . . . . . . 5 2.1.3: Copy Auditing Materials . . . . . . . . . . 6 2.1.4: Prepare to Audit . . . . . . . . . . . . . 7 2.1.5: Auditor's Review . . . . . . . . . . . . . 9 2.1.5.1: The Auditing Checklist . . . . . . . . . 9 2.1.5.1.1: Listing Modules on the Checklist . . . 9 2.1.5.1.2: Items to Audit . . . . . . . . . . . . 10 2.1.5.1.3: Reporting Failures . . . . . . . . . . 10 2.1.5.2: Type of Review Comments . . . . . . . . . 10 2.1.6: Discuss Auditing Results . . . . . . . . . 12 2.1.7: Resolve Items which Failed Audit . . . . . 12 2.1.8: Submit Passed Changes for Installation . . 14 2.2: Procedures for Auditing Emergency Changes . . 16 2.2.1: Select an Auditor . . . . . . . . . . . . . 16 2.2.2: Provide Auditing Materials . . . . . . . . 16 2.2.3: Copy Auditing Materials . . . . . . . . . . 16 2.2.4: Prepare to Audit . . . . . . . . . . . . . 16 2.2.5: Auditor's Review . . . . . . . . . . . . . 16 2.2.6: Discuss Auditing Comments . . . . . . . . . 17 2.2.7: Resolve Items Which Failed Audit . . . . . 17 2.2.8: Submit Passed Changes for Installation . . 17 2.3: Procedures for Auditing Critical Fix Changes 19 2.3.1: Select an Auditor . . . . . . . . . . . . . 19 2.3.2: Provide Auditing Materials . . . . . . . . 19 2.3.3: Copy Auditing Materials . . . . . . . . . . 20 2.3.4: Prepare to Audit . . . . . . . . . . . . . 20 2.3.5: Auditor's Review . . . . . . . . . . . . . 20 2.3.6: Discuss Auditing Comments . . . . . . . . . 20 2.3.7: Resolve Items Which Failed Audit . . . . . 20 2.3.8: Submit Passed Changes for Installation . . 20 3: Guidelines for Auditing All Software . . . . . 22 3.1: Change Request and Documentation . . . . . . 22 3.2: Installation Form . . . . . . . . . . . . . . 22 3.3: Program Modules . . . . . . . . . . . . . . . 24 3.3.1: Approved Functionality . . . . . . . . . . 24 3.3.2: Correct Operation . . . . . . . . . . . . . 24 ii Audit Guidelines MAB-069 CONTENTS (cont) Page 3.3.3: Code Understandable and Maintainable . . . 25 3.3.4: Coding Standards Observed . . . . . . . . . 26 3.3.5: Satisfactory Performance . . . . . . . . . 29 3.3.6: Fulfills Requirements . . . . . . . . . . . 29 3.4: Bind Files and Bound Object Segments . . . . 30 3.5: Include Files . . . . . . . . . . . . . . . . 30 3.6: Info Files . . . . . . . . . . . . . . . . . 31 3.7: Test Plan and Test Cases . . . . . . . . . . 31 Appendix A: Guidelines for Auditing Trusted Software . . . . . . . . . . . . . . . . . . . . . 32 A.1: Gate Target Checks . . . . . . . . . . . . . 32 A.1.1: Parameter Reference Patterns . . . . . . . 32 A.1.2: Parameter Domain Checks . . . . . . . . . . 32 A.1.3: Naming Conventions for Gate Parameters . . 33 A.1.4: Software Validation Level Checks . . . . . 33 A.1.5: Circumvention of AIM Checks . . . . . . . . 33 A.2: Trusted Server Checks . . . . . . . . . . . . 33 A.2.1: Parameter Reference Patterns . . . . . . . 33 A.2.2: Parameter Domain Checking . . . . . . . . . 34 A.2.3: Access Checked on Behalf of Submitter . . . 34 A.2.4: Pathnames in do_subtree Exec_coms . . . . . 34 A.3: Checks for All Trusted Software . . . . . . . 34 A.3.1: Security Auditing of Operations . . . . . . 34 A.3.2: System Error Message Documentation . . . . 35 A.3.3: Source and Include File Comments . . . . . 35 A.3.4: Multics Design Documents . . . . . . . . . 35 Appendix B: Guidelines for Auditing B2 Functional Test Software . . . . . . . . . . . . . . . . . . 36 B.1: General Standards . . . . . . . . . . . . . . 36 B.2: Test Program Standards . . . . . . . . . . . 37 B.3: Utility Program Standards . . . . . . . . . . 38 B.4: Test Program Documentation . . . . . . . . . 38 Appendix C: Auditing Checklist . . . . . . . . . . 40 C.1: Single Session Audit . . . . . . . . . . . . 40 C.2: Multiple Session Audit . . . . . . . . . . . 41 - iii - Audit Guidelines MAB-069 1: OVERVIEW This is one of a series of MABs on Multics Configuration Management. MAB-070 gives an overview of the entire configuration management process, and outlines policies for three types of software changes which are controlled by configuration management: o Planned changes to pre-release software and documentation. o Emergency changes to pre-release software and documentation. o Critical fix changes to released software and documentation. Each type of software change requires that the new or changed software modules undergo a peer review to ensure that they are ready to be installed. While there are many similarities in the review procedures for the three types of changes, the differences are significant enough to warrant separate description of the procedures for each change type. Section 2 of this MAB describes the procedures used for each type of software change. The most frequently performed review procedure is for planned changes to pre-release software. This auditing procedure is summarized below. PROCEDURE: The auditing process for planned changes to pre-release software is performed by the following steps: 1) Developer selects an auditor. 2) Developer provides auditing materials to the auditor. 3) Auditor copies the materials to a protected place in his hierarchy. 4) Auditor produces compilation listings and compare_ascii output files showing the software changes. 5) Auditor uses an auditing checklist to review and comment upon the software changes. 6) Auditor and developer discuss the auditing comments, and decide if changes and further auditing are necessary. 7) Developer resolves items which failed the audit, and returns revised modules to the auditor for reaudit. - 1 - MAB-069 Audit Guidelines 8) When all items pass the audit, auditor completes MSCR (giving pathname of audited software), attaches auditing checklist(s), and submits MSCR package for further approval. Section 3 of this MAB gives guidelines for items to check during an audit. These guidelines apply to audits for all three types of software changes. They can be summarized as follows. GUIDELINES: the following major items are checked during an audit: 1) Review change request and documentation to learn about the change. 2) Audit MSCR form to ensure correct modules are listed. 3) Audit program modules for: approved functionality, correct operation, maintainability of code, adherence to coding standards, performance characteristics, and fulfillment of requirements. 4) Audit bind files for adherence to standards. 5) Audit include files for adherence to standards. 6) Audit info files for content and format. 7) Audit test plan, test cases and test case documentation, and test results for thoroughness and correctness of testing. 8) Audit Multics Design Document (MDD) changes associated with the new or changed software. Appendices A and B provide additional guidelines for auditing trusted software modules (gate, gate targets, inner ring modules and software for trusted server processes), and for auditing functional test cases and testing utilities. - 2 - Audit Guidelines MAB-069 2: PROCEDURES FOR AUDITING SOFTWARE Auditing of software is a software quality process in which the software developer and an independent auditor form a consensus that a software module is ready to be installed. These two people must come to a mutual agreement that the software is operating correctly, with performance and functionality adequate for its intended purpose. Often, compromise is necessary to reach a consensus. The most frequent type of change is a planned change to pre-release software or documentation. It follows that auditing planned changes is the most frequently performed auditing procedure. Therefore, the procedure for auditing planned changes is described in Section 2.1 in significant detail. The procedure for auditing emergency changes to pre-release software is described in Section 2.2. Since this procedure is quite similar to that for auditing planned changes, many of the steps in section 2.2 refer the reader to procedures for auditing planned changes. The same is true for the procedure for auditing critical fixes to released software, described in Section 2.3. - 3 - MAB-069 Audit Guidelines 2.1: Procedure for Auditing Planned Changes Auditing of planned changes to pre-release software modules is described by the next eight subsections. Auditing is one step in the larger process of Multics software development. This process for developing planned software changes is described in MAB-066. The process for installing planned changes into the System Libraries is described in MAB-056. 2.1.1: SELECT AN AUDITOR PROCEDURE: The developer selects a person to audit his software changes as part of the software development plan for that code. Auditing requires a substantial amount of time (often up to 20% of the time required to develop the code itself). Also, the auditor has a significant responsibility for assuring the quality of the software changes and for performing software review and submission in a timely fashion. Thus, it is important to include time for auditing in the product development plan, and to select an auditor at the start of product development, well in advance of the point at which auditing must be performed. The auditor should be a knowledgeable colleague who understands the problem to be solved, and who understands the programming environment in which the code will operate (eg, the hardcore supervisor environment, or user ring command level, or ring one administrative ring). When working on a large subsystem, the auditor should NOT be a co-author who worked on the code being audited. A co-author would be too involved with subsystem development to provide an objective assessment. Someone who worked on a different part of the same subsystem is marginally acceptable as auditor. Such persons must be wary of letting their intimate knowledge of the subsystem interfere with an objective appraisal. Someone who has not worked on your version of the subsystem is the best choice for auditor. Auditing is more an art than a science, and some people have "the knack" for auditing. You should choose an auditor who can do a good job, and whose opinions on coding practices you respect. Pick someone who will take the time needed to provide a thorough audit of the code. A thorough audit benefits the developer by reducing the likelihood of installing bugs; and it benefits the Multics System by producing high-quality software for installation. - 4 - Audit Guidelines MAB-069 2.1.2: PROVIDE AUDITING MATERIALS PROCEDURE: The developer provides the auditor with the materials needed to perform the audit. The following auditing materials must be provided: o an approved Multics Change Request (MCR) describing the problem being solved, giving reasons for solving the problem and a summary of the software changes and new or deleted modules. This MCR often includes documentation for manuals, info segments, and Software Release Bulletin (SRB) notices. It may also reference Multics Technical Bulletins (MTBs) which further describe the change. o a Multics System Change Request (MSCR) form for a planned software change, describing the module(s) involved in the change. For details, refer to MAB-056. The form should list all segments being installed (source modules, include, info and bind files, etc), along with any special instructions for installing the changes.(1) o a description of the test plan, test programs and test results used by the developer to validate that the program is working properly. o the pathname of, and access to, the actual modules to be audited, to allow the auditor to copy them into a protected location within his hierarchy. Access to the modules being submitted should be set as follows: Change Directories Change Segments ----------------------------- ----------------------------- sma DEVELOPER.PROJECT.* re DEVELOPER.PROJECT.* s AUDITOR.PROJECT.* re AUDITOR.PROJECT.* sma *.SysDaemon.* re *.SysDaemon.* null *.*.* null *.*.* One or more change directories may be referenced by a given MSCR form, depending upon how many MSCR detail sheets are included and whether the developer has grouped changes for multiple detail _________________________________________________________________ (1) Even tools and administrative commands should have info segments describing their operation. Info segments must be part of the software changes being submitted (rather than depending upon the Documentation Unit to install them at a later date). They can then be audited for proper technical content, and can provide initial documentation when the change is installed on System M, CISL and ACTC systems. - 5 - MAB-069 Audit Guidelines sheets into a single directory or several directories. Access on each such directory should be set as shown above. In general, specific terms for the auditor and Security Coordinator are not needed, since those individuals will be registered on either the Multics or SysMaint projects. But they are listed separately in the ACL above as a reminder of the exceptional case in which the auditor or Security Coordinator designee isn't registered on Multics or SysMaint. 2.1.3: COPY AUDITING MATERIALS PROCEDURE: The auditor copies all materials to be installed from the developer's hierarchy into his own storage system hierarchy. Access must be set on the copied modules to prevent the developer from modifying the modules. The purpose of this step is to ensure that the materials which are installed are exactly those materials which have been audited. Copying the materials into protected storage helps the developer avoid the temptation of making "last minute changes" without the auditor's knowledge. The developer supplies the pathname of his installation directory to the auditor. The auditor copies all of the modules from the developer's directory into an installation directory in the auditor's hierarchy. This can usually be done by one or more copy_dir commands. Access on the directory should allow only the auditor to change the directory; access on the segments should allow only the auditor to modify the segments. Recommended ACLs are: Change Directories Change Segments ----------------------------- ----------------------------- s DEVELOPER.PROJECT.* re DEVELOPER.PROJECT.* sma AUDITOR.PROJECT.* re AUDITOR.PROJECT.* s SEC_COORD.PROJECT.* re SEC_COORD.PROJECT.* s MANAGER.PROJECT.* re MANAGER.PROJECT.* s *.SysMaint.* re *.SysMaint.* sma *.SysDaemon.* re *.SysDaemon.* null *.*.* null *.*.* In particular, the SysMaint project must be given access to the changed software in the auditor's directory so that the installer can access the modules for installation. - 6 - Audit Guidelines MAB-069 2.1.4: PREPARE TO AUDIT PROCEDURE: The auditor produces listings for use in the audit. The following steps should be performed: o Run the history_comment display operation to display the new history comments in all source, bind and include files. The proper command is: hcom display prog.pl1 incomplete Return modules which are missing new history comments (or which are missing the approval field from new history comments), to the developer for correction. o Compile each new or changed source module to produce a compilation listing. PL/I programs should be compiled with -optimize and -map control arguments, and optionally with -table and -prefix size,strz,strg,subrg.(1) ALM programs should be compiled with -list. Executable objects from these compilations are used for testing the bind file. Also, if the auditor desires to run the test suite against the changed modules, then the tests should be run against the object segments produced by this compilation. o Bind all bound segments with the -list control argument to produce bind listings, showing the results of binding new or changed bound segments. _________________________________________________________________ (1) NOTES: -table and -prefix are recommended when the auditor intends to run the code to test its functionality. By making a symbol table available, and enabling PL/I size, stringsize, stringrange and subscriptrange checking, changed programs can be tested and examined more easily and thoroughly. The pl1 command -single_symbol_list control argument should NOT be used for pl1 compilations, because it makes it harder to locate undeclared variables and declared but unreferenced variables. - 7 - MAB-069 Audit Guidelines o For changed source, include, bind or info files, use the compare_ascii or compare_pl1 command to show the actual changes from installed versions. Comparison output can be attached to the end of the compilation or bind listing, or it can be placed in a separate file. Because compare_ascii output is required, it is important to avoid reformatting of existing modules when making small changes. For larger changes where the developer felt reformatting was needed, the auditor should reformat the system library copy of the original source and compare that with the changed module, to reduce the amount of reformatting-type changes appearing in the cpa output. o Dprint all compilation listings, bind listings and compare_ascii output for use during the audit. o Dprint all info segments associated with the new or changed modules. Info segments must be provided for modules which are externally visible and supported for use by users, including info segments for tools, operator commands, subsystem requests, etc. - 8 - Audit Guidelines MAB-069 2.1.5: AUDITOR'S REVIEW PROCEDURE: The auditor uses an auditing checklist to ensure that all important aspects of the new or changed module are properly reviewed. One checklist should be used for each auditing session needed to review the submission. During the review, the auditor notes all auditing issues (questions, comments or problems with the changes, etc) by writing such comments on the listings near the software or documentation at issue. Alternately, he can note such comments on a separate sheet of paper. The checklist and notes are needed to help convey results of the audit to the developer. 2.1.5.1: The Auditing Checklist The auditing checklist serves a variety of purposes. It helps the auditor remember all important aspects which should be audited. For large submissions, it helps the auditor keep track of his progress during the auditing. It helps to organize the auditing results so that problems and comments can be conveyed back to the developer. And it serves as a permanent record of the audit which acts as input to those who must approve the submission (the Security Coordinator and responsible Manager), or who review the submission at a later date. A sample auditing checklist is shown in Appendix C of this MAB. 2.1.5.1.1: Listing Modules on the Checklist The checklist is organized for easy reporting of the most commonly encountered auditing results (that all modules passed audit, or that a few modules have problems). For small submissions audited during a single review session, only modules names which fail the audit need to be listed on the form, along with their reason for failure. If no modules fail, the auditor can simply check a box to indicate that all modules have passed the audit. For larger submissions that require several auditing sessions, a separate checklist should be used to record the results of each auditing session. Each form lists names of both passing and failing modules audited during that session, so the auditor can accurately track which modules have been audited. The auditor checks a box to indicate that all modules reviewed during the session passed audit, or identifies which modules failed along with reasons for failure. - 9 - MAB-069 Audit Guidelines Modules are named on the checklist in one of two module name sections. Associated with each module name is a letter which is used as a shorthand way to identify the module when reporting that a module failed a particular auditing item. 2.1.5.1.2: Items to Audit The checklist is organized as a series of reminder items, with one item for each of the major auditing guidelines in this MAB (Section 3, and Appendices A and B). In this way, the guidelines serve as descriptions for each of the reminder items. 2.1.5.1.3: Reporting Failures When a module fails to meet auditing requirements for a specific auditing item, its module letter is listed to the left of that item. This quickly identifies which modules have problems, and what types of problems they have. There is also an OTHER COMMENTS section on the checklist in which major problem areas are summarized. When the auditor and developer discuss the failures, they may decide to install the module without correcting a minor failure. The module letter identifying a particular module failure can be circled to indicate that the auditing requirement has been WAIVED for that failure. The exact nature of the failure and reasons for WAIVING the requirement must be described in the OTHER COMMENTS section of the auditing checklist. Before an MSCR can be submitted, all modules must pass all audit items; or modules must pass most items with all other failing items marked as WAIVED. 2.1.5.2: Type of Review Comments When making auditing comments, the auditor should clearly distinguish between the following types of comments: o questions about the code or documentation. o problems which must be corrected before the auditor will approve the installation. o problems which the auditor desires to have corrected, but which can be negotiated. o suggestions for possible changes, which the developer can make at his own discretion. - 10 - Audit Guidelines MAB-069 The following hints on making comments may help the auditor to avoid confrontations with the developer over touchy points. o Be specific. General statements like "this module is hard to read", or "this algorithm is bad" do not help the developer to understand what aspects of the module could be improved. Instead, comment on particular lines of code or documentation, and then if necessary, refer to several of these comments when making an overall comment about the general acceptability of the code. o Be positive. When possible, point how out to correct a problem as well as identifying the problem itself. Don't say what is wrong and leave the developer hanging. Indicate how it can be made right. o Emphasize technical issues. Base comments upon technical standards and conventions, rather than on personal preferences. o Avoid issues of style (such as program formatting). Don't ask the developer to change a program to conform to your programming style. However, do require the developer to use a uniform programming style throughout an entire module. In particular, changes to an existing module must conform with the style of that module, because switching formatting styles in a given module usually makes it harder to read. o Don't criticize the approved, external interfaces. The change review board has already approved the external interfaces and detailed design proposal, as specified in the MCR and MTBs. However, do comment when the approved design does not seem to solve the original problem, or when it turns out to be difficult to use, etc. - 11 - MAB-069 Audit Guidelines 2.1.6: DISCUSS AUDITING RESULTS PROCEDURE: The developer and auditor discuss the auditing results. They decide if further changes are needed to resolve issues found during the audit. The developer and auditor must reach a consensus on what changes (if any) are needed to resolve problems found during the audit. At times, compromises are necessary, trading off small issues to leave time to resolve more important problems. All important issues (those affecting system reliability, integrity, security, proper functioning of the interface, etc) must be corrected. Small issues which will not be resolved should be marked as WAIVED on the auditing checklist, and reasons for waiving should be listed in the comments section of the checklist. 2.1.7: RESOLVE ITEMS WHICH FAILED AUDIT PROCEDURE: If the auditing checklist shows that one or more modules FAILED certain aspects of the audit, the developer corrects these problems and gives the revised code to the auditor for further review. If one or more auditing items FAILED review and the necessary changes are minor and straightforward, the following steps occur: o Auditor gives auditing checklist and listings with comments to the developer. o Developer changes his copy of the modules as mutually agreed, based upon checklist and comments. The developer reruns test cases at this point to insure that no new problems were introduced by the revision. o Developer returns the revised modules to the auditor for final review, along with the original auditing checklist, and listings with comments. o Auditor copies the revised software into his directory, and protects it from modification. o Auditor uses compare_ascii to compare the revised modules with those originally audited. The auditor may also rerun test cases at this point to insure that no new problems were introduced by the revision. - 12 - Audit Guidelines MAB-069 o Auditor prepares a new checklist covering the revised modules, reviewing the items which failed the previous audit, and giving other aspects of the revised modules a final once-over. If some issues were not properly resolved or new issues were introduced by the revision, the auditor marks these items FAILED on the new checklist. o Auditor discusses any failures with the developer, and the steps in this section are then repeated. o When all checklist items are PASSED or WAIVED, auditor proceeds with submitting the changes for installation. Note that when significant changes are necessary to one or more modules, the procedure above should be followed, but the significantly changed modules should be completely reaudited using a new auditing checklist. - 13 - MAB-069 Audit Guidelines 2.1.8: SUBMIT PASSED CHANGES FOR INSTALLATION PROCEDURE: When the auditing checklist shows all items either PASSED or WAIVED, the auditor submits the changes for installation. The auditor performs the following steps to submit the changes: o If the auditor's installation directory does not reside on System M, transfer the installation directories and their contents to System M. If a transfer is necessary, be sure to set the ACL on the change directories and segments as described in Section 2.1.3 above. o Sign the MSCR form authorizing the installation. o Put the pathname of the auditor's installation directory on the MSCR form as the submission directory. o Attach all auditing checklists to the MSCR package, including early checklists showing failures which were subsequently corrected by the developer. At this point, the MSCR package consists of: the MSCR, copies of associated MCRs, info segment dprints, dprints of the SRB Notice and other associated documentation, and the auditing checklists. o Run the history_comment update operation to fill in the audit field for all audited source, bind and included files. The proper command is: hcom add_field prog.pl1 -audit o On System M, run the history_comment check operation against all source, bind and include files, to ensure that properly completed history comments have been added. For existing modules, the proper command to use is: hcom ck prog.pl1 -vdt hcom_mdc_validate_ -orig [lpn prog.pl1] For new program modules, use: hcom ck prog.pl1 -vdt hcom_mdc_validate_ If the check fails for any module, correct the problem with the assistance of the developer. Usually, such problems arise because of typographical errors in MCR numbers in the approve field, etc. A module which is missing its history comments would already have been corrected during the "Prepare to Audit" step. - 14 - Audit Guidelines MAB-069 o For changes to the Trusted Computing Base (TCB), its functional test cases, test utilities and test case document, and for changes to Multics Design Documents (MDDs), submit the MSCR package to the Security Coordinator, to obtain his approval for the installation.(1) o For other changes, submit the MSCR package to the documentation unit for approval.(2) _________________________________________________________________ (1) MAB-070 defines what software is part of the TCB. MAB-071 defines the responsibilities of the Security Coordinator. For TCB submissions, the Security Coordinator will: perform a secondary audit of the security aspects of the MSCR package; review results of Functional Tests; sign the MSCR; and submit the MSCR package to the documentation unit for approval. (2) The documentation unit will review the MSCR package and make sure documentation changes needed for the change get applied to the various manuals. Info segments will be validated for format. SRB Notices will be incorporated into the System Release Bulletin. When the documentation unit approves the installation, they pass it to the Manager responsible for the area of the system being changed. The Manager will: review the MSCR package; sign the MSCR; and submit the approved MSCR package to the System Integration group for installation. Refer to MAB-056 for details. - 15 - MAB-069 Audit Guidelines 2.2: Procedures for Auditing Emergency Changes Auditing of emergency changes to pre-release software modules is described by the next eight sections. Auditing is one step in the larger process of developing and installing emergency changes for the System Libraries. This larger process is described in MAB-057. 2.2.1: SELECT AN AUDITOR Select an auditor just as described for planned changes in Section 2.1.1. Note that emergency changes are generally spur-of-the-moment operations and it is more difficult to get an auditor on short notice. Arrange for an auditor as soon as you discover than an emergency change will be needed. 2.2.2: PROVIDE AUDITING MATERIALS The developer provides the auditor with the following materials needed to perform the audit of an emergency change: o a Multics System Change Request (MSCR) form for an emergency software change, briefly describing the problem being solved, giving the proposed method of solution, and listing the modules to be changed, and a summary of how the changes were test. o the pathname of, and access to, the modules to be audited. Access should be set as described in Section 2.1.2. 2.2.3: COPY AUDITING MATERIALS The auditor copies modules to be audited into a directory in his own hierarchy. If the MSCR form is online, the online form should be copied as well for rapid communication of the change to the installer. The auditor must protect these directories and segments, using the ACLs suggested in Section 2.1.3 above. 2.2.4: PREPARE TO AUDIT Prepare to audit the changes, as described for planned changes in Section 2.1.4. - 16 - Audit Guidelines MAB-069 2.2.5: AUDITOR'S REVIEW Review the change, using an auditing checklist, as described for planned changes in Section 2.1.5. 2.2.6: DISCUSS AUDITING COMMENTS Discuss auditing results with the developer, as described for planned changes in Section 2.1.6. 2.2.7: RESOLVE ITEMS WHICH FAILED AUDIT Correct the failing items, as described for planned changes in Section 2.1.7. 2.2.8: SUBMIT PASSED CHANGES FOR INSTALLATION The auditor performs the following steps to submit for installation an emergency change which has passed audit: o If the auditor's installation directory does not reside on System M, transfer the installation directories and their contents to System M. If a transfer is necessary, be sure to set the ACL on the change directories and segments as described in Section 2.1.3 above. o Sign the MSCR form authorizing the installation. o Put the pathname of the auditor's installation directory on the MSCR form as the submission directory. o Attach all auditing checklists to the MSCR package. At this point, the MSCR package consists of: the MSCR describing the problem and its solution, and the auditing checklists. o Run the history_comment update operation to fill in the audit field for all audited source, bind and included files. The proper command is: hcom add_field prog.pl1 -audit o On System M, run the history_comment check operation against all source, bind and include files, to ensure that properly completed history comments have been added. For existing modules, the proper command to use is: hcom ck prog.pl1 -vdt hcom_mdc_validate_ -orig [lpn prog.pl1] - 17 - MAB-069 Audit Guidelines For new program modules, use: hcom ck prog.pl1 -vdt hcom_mdc_validate_ If the check fails for any module, correct the problem with the assistance of the developer. Usually, such problems arise because of typographical errors in MCR numbers in the approve field, etc. A module which is missing its history comments would already have been corrected during the "Prepare to Audit" step. o For emergency changes to the TCB, functional test cases, test utilities and test case documentation, and for changes to Multics Design Documents (MDDs), the auditor submits the MSCR package to the Security Coordinator. o For other emergency changes, the auditor submits the MSCR package to the Unit Manager responsible for the software area being changed.(1) _________________________________________________________________ (1) Note that emergency changes to pre-release software do not require Documentation Unit approval. - 18 - Audit Guidelines MAB-069 2.3: Procedures for Auditing Critical Fix Changes Auditing of critical fixes to released software modules is described by the next eight sections. Auditing is one step in the larger process of developing and installing critical fixes into the Critical Fix Library. This larger process is described in MAB-063. 2.3.1: SELECT AN AUDITOR Select an auditor just as described for planned changes in Section 2.1.1. Note that emergency changes are generally spur-of-the-moment operations and it is more difficult to get an auditor on short notice. Arrange for an auditor as soon as you discover than an emergency change will be needed. 2.3.2: PROVIDE AUDITING MATERIALS The developer provides the auditor with the following materials needed to perform the audit of an emergency change: o the pathname of, and access to, the modules to be audited. o The pathname of, and access to, a Critical Fix Announcement segment, containing the following items: o symptoms of the problem, o description of the problem, o numbers of TRs and error list entries associated with the problem, o releases in which the problem exists, o releases to which the fix applies, o names of modules being fixed, and names of their containing bound segments, o review procedures to be used (either TCB or nonTCB), o fix type (either nonsecurity fix, or data security fix), and o level of testing of the fix, along with a brief description of the types of tests which were run. For examples of the type and format of information needed, look at some of the later transactions in the Critical_Fixes forum (>udd>sm>fixes>Critical_Fixes or fixes, for short). The developer should set access to these materials as described in Section 2.1.2. - 19 - MAB-069 Audit Guidelines 2.3.3: COPY AUDITING MATERIALS The auditor copies modules to be audited into a directory in his own hierarchy. The auditor must protect these directories and segments, using the ACLs suggested in Section 2.1.3 above. 2.3.4: PREPARE TO AUDIT Prepare to audit the changes, as described for planned changes in Section 2.1.4. 2.3.5: AUDITOR'S REVIEW Review the change, using an auditing checklist, as described for planned changes in Section 2.1.5. Since there is no MSCR form assocaited with a critical fix, you should list all modules which are audited on the auditing checklist, and check the MULTIPLE SESSION Type of Audit. 2.3.6: DISCUSS AUDITING COMMENTS Discuss auditing results with the developer, as described for planned changes in Section 2.1.6. 2.3.7: RESOLVE ITEMS WHICH FAILED AUDIT Correct the failing items, as described for planned changes in Section 2.1.7. 2.3.8: SUBMIT PASSED CHANGES FOR INSTALLATION The auditor performs the following steps to submit for installation a critical fix which has passed audit: o If the auditor's installation directory does not reside on System M, transfer the installation directories and their contents to System M. If a transfer is necessary, be sure to set the ACL on the change directories and segments as described in Section 2.1.3 above. - 20 - Audit Guidelines MAB-069 o Run the history_comment update operation to fill in the audit field for all audited source, bind and included files. The proper command is: hcom add_field prog.pl1 -audit This step signifies that the software has passed audit, and that the auditor approves installation of the critical fix. o The auditor notifies the Critical Fix Coordinator by electronic mail that the fix is ready to be installed and announced. This mail consists of: o The pathname of the auditor's copy of the fixed modules. o The pathname of the Critical Fix Announcement segment, which the auditor copied into his audit hierarchy. o A copy of the completed auditing checklist, showing the results of the audit. If a hardcopy auditing checklist was used, then the hardcopy checklist must be delivered to the Critical Fix Coordinator prior to installation of the critical fix. If an online version of the checklist was used, then send the pathname of the online checklist to the Critical Fix Coordinator via electronic mail. o For all TCB changes, the auditor must send a copy of the notification mail to the Multics Security Coordinator, as well as to the Critical Fix Coordinator. The Security Coordinator must send mail to the Critical Fix Coordinator authorizing the changes to the TCB, and indicating whether the fix is a data security critical fix or nonsecurity fix. - 21 - MAB-069 Audit Guidelines 3: GUIDELINES FOR AUDITING ALL SOFTWARE The following are general guidelines for reviewing the audit materials. These guidelines apply when auditing all types of software modules. In addition to the guidelines in this section, special guidelines should be applied when auditing gates and inner ring software modules. These are described in Appendix A. Similarly, Appendix B describes additional guidelines to be applied when auditing the functional test cases and testing utilities for Trusted Computing Base (TCB) modules. Most of the guidelines below are derived from the programming standards in MAB-068. Refer to that MAB for additional details. 3.1: Change Request and Documentation Examine MCRs, MTBs, info segments and documentation, to determine: o what problem is being solved; o reasons for solving the problem; o how the problem is to be solved. This information forms a functional specification. The auditor compares this specification with the new or changed modules to determine if the approved functionality is correctly implemented, and if it actually solves the intended problem. The auditor also checks the info segments and documentation to insure that they reflect the interface actually implemented by the software. He should also consider whether the documentation is appropriate for the intended audience. The modules can diverge in small ways from the specification, so long as documented user interfaces are implemented as approved. The auditor and developer must decide when variances from the specification are large enough to require formal review (eg, a new or revised MCR). Finally, the auditor should check MTBs related to the software to be sure they have been updated to reflect the actual implementation. - 22 - Audit Guidelines MAB-069 3.2: Installation Form The MSCR form lists all source segments, include file, info files and bind files being changed as part of the installation. Files are grouped by bound segments on the MSCR form, with include and info files supporting a source program listed with the bound segment containing the source module. The form indicates whether a modules is to be added, replaced, deleted or recompiled (because one of its include files was modified). Examine the MSCR. Make sure that the MSCR is complete and legible, and that: o all listed segments are supplied as part of auditing materials. o all segments supplied with the auditing materials appear on the MSCR. o all fields on front of MSCR are filled in appropriately. o all pertinent items are checked off on the MSCR checklist. o all unbound segments have the proper names on the copy of the segment submitted for installation. - 23 - MAB-069 Audit Guidelines 3.3: Program Modules Examine programs to check for the following items. 3.3.1: APPROVED FUNCTIONALITY Do the changes implement the user interface and features approved in the MCRs? Unapproved features or interface changes are not allowed. 3.3.2: CORRECT OPERATION Do the changes function correctly as implemented? You can determine this: (A) by reading the code, building a conceptual model of its interfaces and operation, and comparing this model with the functional specification in the MCR and associated documentation; or (B) by running test cases against the new code, and comparing actual operation and results against the specification. When examining the implementation, watch out for: o Proper checking of input values. Do commands properly diagnose argument values outside the documented range of acceptable values? Do subroutines which accept structures validate the structure version number?(1) o Proper establishment of cleanup handlers. Are all variables referenced by a cleanup handler initialized BEFORE the handler is established? Are there windows in which items which should be cleaned up are used before the cleanup handler is established? Does the handler reset all appropriate static variables, free all based storage, release all temporary segments, close/detach/destroy all I/O switches, etc? o Proper program exit. Are all variables properly cleaned up at normal exit? Are all output parameters set properly? _________________________________________________________________ (1) In general, subroutines are supposed to assume they have been called correctly, and should not try to validate their input arguments. The exceptions are that subroutines should always validate version numbers of input structures; and subroutines which interface between the user environment and the Trusted Computing Base (TCB) are required to validate all input arguments. - 24 - Audit Guidelines MAB-069 o Unplanned side effects. Do the changes adversely affect existing code, module interfaces and databases? o New limitations. Does the way the change was implemented introduce limitations on the operation of the program? Are these limitation documented as comments in the code, and in user documentation when appropriate? 3.3.3: CODE UNDERSTANDABLE AND MAINTAINABLE Is the code easy to read, to understand, to maintain and extend? Having code which is structured simply is usually more important than any performance gains that might be achieved by a more complex coding scheme. Do the modifications made to an existing program make the program less understandable? Points to look for are: o Proper comments. These include: history comments summarizing module changes; interface specifications for internal procedures not documented elsewhere; module overview descriptions explaining algorithms, structure and methods of operation, etc; block comments describing major sections of code; detailed comments when needed to explain particular statements. o Proper formatting. Does formatting properly reflect the flow-of-control statements in the code? Is a standard style of formatting used throughout the program? o Proper modularization. Are modules short enough to understand? Is the program over-modularized? o Understandable flow-of-control. Are there too many control paths through the program? o Data hiding. - Are local variables declared in the internal procedure in which they are referenced, rather than declaring them all in the main procedure? - Are procedures structured to hide internal data values, forcing their manipulation though explicit subroutine calls? For example, if a large subsystem manipulates a list of highly structured data items, it is best to have a centralized procedure which adds, deletes, modifies and retrieves particular items for other subsystem modules. The alternative of having each subsystem module code in its own manipulation routines is error prone, and makes it difficult to restructure the data. - 25 - MAB-069 Audit Guidelines o Meaningful variable names. 1-letter names (eg, i, j, p, q) do not convey enough meaning. o Code should not use "magic numbers". - Program statements should use lbound/hbound builtin functions instead of coding literal number array bounds into do statements, etc. - Programs statements should refer to variable extents using the currentsize, size, length and maxlength builtin functions. - Programs should use system-provided constants (eg, sys_info$max_seg_size), rather than hard-coding such values as literal constants into the program. - Most other literal constants should be replaced by named constants or builtin function references. For example: if substr(var,1,length("item:")) = "item:" then ... is preferred over: if substr(var,1,5) = "item:" then ... because it isn't obvious to the reader where the literal 5 comes from in the second example. o Structured programming techniques. Are goto statements and switches used appropriately? 3.3.4: CODING STANDARDS OBSERVED Does the implementation violate any coding standards? Does it follow current coding conventions? Coding standards are required items, and are listed in MAB-068. Coding conventions are MCR Board policies, and common coding practices which have not been codified as standards, but which are expected in all Multics programs. All programs are expected to meet coding standards and conventions. Some items to watch for are: o Protection notices. All new source programs must have a protection notice added at the beginning of the program. The developer or auditor must use the add_pnotice command to add the proper protection notice. Include, info and bind files should not have pnotices. - 26 - Audit Guidelines MAB-069 o History comments. All source programs must have history comments immediately following the protection notice. The comments must be in a form acceptable to the history_comment command, and must summarize the basic function of a new module, or changes made to an existing module. o Upgrading Existing Programs. When existing programs are modified, a requirement of the modifications is to upgrade the existing code to meeting current coding standards and conventions. However, the developer and auditor may agree to waive this requirement when upgrading would require an excessive amount of programming effort beyond that needed to implement the approved changes. o Use of Obsolete Procedures. Existing programs are not allowed to use obsolete procedures (those in >system_library_obsolete) or those which have been replaced by more modern interfaces (eg, decode_clock_value_ was replaced by date_time_$from_clock). o Use of Priced Software Products. The operation of programs must not depend upon routines which are part of (another) PSP module, unless that dependency has been explicitly approved as part of the MCR. o Proper declarations. - Are variables declared with proper data types? For example, the precision and scale of numeric variables should match their intended use (ie, a variable which can have values of 0, 1, 2 should be declared "fixed bin(2) unsigned"). Based variables should be used within the rules of the PL/I overlay defining. - Are structures properly declared? Padding between declared elements of a structure must itself be explicitly declared as must-be-zero (mbz) fields. Version elements must be included in all user-visible structures. Version should usually be character string elements, whose values are set/checked using named constants declared in the structure's include file. - Is the controlled storage class used? For performance reasons, linked based variables should be used instead. - Does module use external static variables whose names do not include a dollar-sign ($)? In general, such names are not permitted, because they are misused for passing data between procedures. Exceptions are the XXX_severity_ variables which commands use to set a - 27 - MAB-069 Audit Guidelines return code to be accessed by the severity active function. - Does module use external static variables whose names do include a dollar-sign ($)? If so, the variables should be read-only constants. Writable data items of this type are not permitted, because they are misused for passing data between procedures. - Are all parameters, variables, constants and builtin functions explicitly declared? The section of a PL/I compilation listing entitled "NAMES DECLARED BY CONTEXT OR IMPLICATION" should always be empty. - Are there any unused declarations? The compilation listing section entitled "NAMES DECLARED BY DECLARE STATEMENT AND NEVER REFERENCED" should contain only variables declared in include files. Make sure that at least one declaration from the include file is actually referenced. - Do all declarations for named constants use the options(constant) attribute? - Is the initial attribute used for automatic variables? Explicit assignment of values to automatic variables is more efficient in subroutines with multiple entrypoints, and makes the code easier to understand. o Structure elements properly qualified. All references to elements of a structure must be qualified by at least the level 1 structure name to make the location of the element within the structure obvious to someone reading the code. o Proper command argument validation. Do commands, active functions, and subsystem requests validate all arguments before performing any action? No action should be performed if any arguments are incorrect. o No allocations in user free area. Coding standards do not permit system programs to allocate storage in the user free area. All PL/I allocate and free statements must include an "in (area)" option to reference storage in another area (eg, system free area, area in a temporary segment, etc). o Proper use of temporary segments. Don't use temporary segments if automatic storage could be used instead (eg, storage allocated by a begin block). Don't use temporary storage if more storage is needed than will fit into an average size (1024K) process directory. - 28 - Audit Guidelines MAB-069 o Balanced paired actions. Typical paired actions are: allocate and free; initiate_file_ and terminate_file_; attach/open_file and close_file/detach/destroy_iocb; get_temp_segment_ and release_temp_segment_; define_area_ and release_area_. If a program performs the first half of an action pair, it is usually required to perform the second half as well. o Compilation errors. No compilation errors or warnings of any sort are allowed. 3.3.5: SATISFACTORY PERFORMANCE Do the changes perform within reasonable expectations for CPU time usage, page faults, response time, etc? Will the changes significantly impact overall system performance, as measured by standard benchmarks? 3.3.6: FULFILLS REQUIREMENTS Do the changes resolve the original problem, as stated in the MCR, trouble reports, Product Functional Specification (PFS), etc? Are the changes reasonably easy for their intended audience to use? - 29 - MAB-069 Audit Guidelines 3.4: Bind Files and Bound Object Segments Examine bind files to ensure that: o a history comment is included in the bind file. o proper names are added to bound segments. o proper entrypoints are retained with proper synonyms. o names in Addname statement are alphabetized by long command or subroutine name, with short names appearing on the same line as their corresponding long name. This puts names on the segment in an order which appears sensible to users. o objectname statements appear in alphabetic order, for ease of reference. For new bind files, ensure that the file includes Objectname, Order, Addname and "Global: delete" statements. These are required by coding standards. 3.5: Include Files Examine include files to ensure that: o they begin and end with proper delimiter comments, which identify the name of the include file. o they include a beginning comment giving the purpose of the include file. o they include a history comment, summarizing changes to the include file. o each include file is self-contained. All variables referenced in the file must be declared in that file, or in another include file included by that file. Use the peruse_crossref command to ensure that all programs which reference a changed include file are included in the MSCR. - 30 - Audit Guidelines MAB-069 3.6: Info Files Examine info segments for proper content and format. The content should match the user interfaces of the modules being installed. The format should conform to standards for info segment format. For details, type: help info_seg A quick test of info segment format is to issue the following commands: help xxx -brief Make sure all control arguments are listed in the brief output, along with the Syntax section. Items in the Arguments section are listed individually in -brief output if they also appear in the Syntax line. help xxx -title Make sure all section titles are listed. If some are left out, the usual cause is not having two blank lines preceding the section title, or having space or horizontal tab characters on one of the blank lines. validate_info_seg xxx Check for errors reported by this info segment format checker. 3.7: Test Plan and Test Cases Examine the test plan to ensure that it tests a reasonable subset of the paths through the new code. Examine the test cases to ensure that they do in fact test items listed in the test plan. Examine the test results to ensure that all tests produced proper results. - 31 - MAB-069 Audit Guidelines APPENDIX A: GUIDELINES FOR AUDITING TRUSTED SOFTWARE MAB-068, Section 13 describes standards for coding trusted software (gates, targets of gates, inner ring modules, and software controlling trusted server processes). This appendix briefly summarizes the aspects of those standards which should be checked when auditing software modules of this type. Refer to MAB-068 for details. A.1: Gate Target Checks In Multics, a gate is a simple ALM transfer vector whose ring brackets cause the ring of execution to change when the gate is called. The gate transfers to a program entrypoint called the gate target, which is responsible for validating gate arguments, performing some operation, and returning results to the calling ring. All gate target entrypoints must adhere to the following standards. A.1.1: PARAMETER REFERENCE PATTERNS The gate target must read each input parameter only once, and write each output parameter only once. Typical modes of violation are: o Reading a parameter more than once. o Writing a parameter and then reading it. o Writing an output parameter several times (other than for initialization to a default value, followed by assignment of the real result). o Passing a parameter to another subroutine. o Using parameters in expressions. o Using packed pointer parameters. o Improper reference to elements of structure parameters, or to structures identified by a pointer parameter. A.1.2: PARAMETER DOMAIN CHECKS The gate target must check that all input parameter values are within the domain of acceptable values for that parameter. - 32 - Audit Guidelines MAB-069 A.1.3: NAMING CONVENTIONS FOR GATE PARAMETERS Gate target parameters must be named according to one of the standard parameter naming conventions (see MAB-068). Also, they must named consistently with other parameters of the module. A.1.4: SOFTWARE VALIDATION LEVEL CHECKS The gate target must properly set the validation level. It must restore the validation level upon return, with a cleanup handler to reset it during crawlouts. It must properly use the caller's validation level when making software enforced access checks. A.1.5: CIRCUMVENTION OF AIM CHECKS The gate target must make proper use of system privileges to implement multiclass objects. It must use sys_info$XXXX_privilege constants when changing privileges. A.2: Trusted Server Checks A trusted server is a process which performs operations on behalf of other users. It reads user requests from a message segment, and uses special privileges (eg, access to gates, control of I/O devices, etc) not granted to normal users in performing some operation for the user. An example is an I/O Daemon Driver, which processes dprint requests on the user's behalf. All software modules running in trusted server processes must adhere to the following standards. A.2.1: PARAMETER REFERENCE PATTERNS The user interface to a trusted server process is in the program which reads user requests from the message segment. These requests are the input parameters to the server. Generally, no output parameters are returned to the requestor. The trusted server module which processes user requests must reference its parameters much as gate targets do. The trusted server module must read request messages out of the message segment only once. If the message points to additional data, it must access that data only once. It must either copy the data into the address space of the trusted server, or it must not base access decisions on uncopied data elements (eg, the segment being dprinted does not contain data used by the I/O Daemon driver in making access control decisions; therefore the segment need not be copied into server process storage). - 33 - MAB-069 Audit Guidelines A.2.2: PARAMETER DOMAIN CHECKING The server module must check that information in the request message falls within the domain of acceptable values. Trusted servers must validate their input parameters. A.2.3: ACCESS CHECKED ON BEHALF OF SUBMITTER When accessing objects, the server module must check the submitter's rights to access the object, as well as the access rights of the server process. hcs_$get_user_access_modes should be used when making discretionary access control decisions. Mandatory AIM checks must be made as well, with separate checking for system privileges which may be enabled. A.2.4: PATHNAMES IN DO_SUBTREE EXEC_COMS For a server process that runs an exec_com to walk a subtree, executing a command line for each entry in the subtree (eg, Salvager.SysDaemon runs the admin.ec "x repair" function to salvage directories in the hierarchy), subtree pathnames in such exec_coms must be requoted. Otherwise, pathnames containing command processor special characters could cause the trusted server to execute unauthorized commands. A.3: Checks for All Trusted Software All trusted software (gate targets, inner ring modules, and trusted server software) must adhere to certain standards. A.3.1: SECURITY AUDITING OF OPERATIONS TCB modules which manage objects or manipulate security parameters must call the access_audit_ subroutine to generate an audit trail. The term "object" refers to any segment, directory, or other resource managed by the TCB. Audit trail entries should be generated at the following times: o when an object is created or destroyed. o when making a decision to grant or deny access to an object. o when making an object accessible or inaccessible to a given process (eg, changing its ACL). o when changing security parameters. - 34 - Audit Guidelines MAB-069 MAB-068, Section 13 summarizes the type of information which must be provided to the access_audit_ subroutine. A.3.2: SYSTEM ERROR MESSAGE DOCUMENTATION Programs calling any of the following subroutines must include error message documentation: syserr$*, admin_gate_$syserr_*, hphcs_$syserr_* sys_log_$* (except for the sys_log_ entrypoints containing "command"). Programs calling access_audit_$*, access_audit_r1_$*, access_audit_gate_$* or as_access_audit_$* need not include explicit error message documentation for these calls, even though these routines do generate log messages. Because the access audit messages are generated in a strictly enforced format, generalized documentation for security audit messages appears in the System Administrative Procedures manual (Order No. AK50). This documentation must be updated whenever: new entries are added to the access_operations_ table; new detailed operation codes are added; or when the possible values for the added_info field of messages is changed. A.3.3: SOURCE AND INCLUDE FILE COMMENTS All Multics software must have sufficient comments to make the code understandable. TCB modules must adhere to the following additional requirements: internal entrypoints must be completely described in comments; include files describing TCB data structures must have comments which explain each structure element (not merely restate its name) and the values which it can have. A.3.4: MULTICS DESIGN DOCUMENTS All Multics Design Documents (MDDs) associated with TCB modules must follow MDD documentation standards described in MAB-068, Section 14. These standards define both the expected content of an MDD, and the format in which MDD information must be presented. Refer to MAB-068 for details. - 35 - MAB-069 Audit Guidelines APPENDIX B: GUIDELINES FOR AUDITING B2 FUNCTIONAL TEST SOFTWARE MAB-068, Section 15 describes standards for B2 functional tests and testing utility programs. This appendix briefly summarizes the aspects of those standards which should be checked when auditing function test programs and testing utilities. Refer to MAB-068 for more details. B.1: General Standards The following standards apply to both test programs and the testing utilities: o Protect all modules with a trade_secret protection notice. o Use the required format_pl1 source format, which is: format: style4,indattr o Group declarations in classes which begin with a comment. The classes are: parameter, automatic, based, entry, external, static and "miscellaneous". It is best for identifiers declared in each class to explicitly include the class attribute (except for miscellaneous) in their declaration. This is required for parameters. Within a class, declarations should be ordered alphabetically. o Prefix the parameter names for external entrypoints with "P_". Make names for labels and constants upper case. o Place include files at the end of the program. o Implement setup and cleanup functions as internal procedures. o Report all errors via sub_err_. o Begin all internal procedures on a new page. It is best for internal procedure names to be in upper case. - 36 - Audit Guidelines MAB-069 B.2: Test Program Standards The following standards apply to programs written to test gate entries: o Testing of Discretionary Access Control (DAC, which includes ACLs and ring brackets) and Mandatory Access Control (MAC, which includes AIM) is done in separate programs for each gate, gate entrypoint, and DAC/MAC combination. DAC and MAC cannot be tested in the same program. o Test program names follow standard naming conventions: {GATE}_{ENTRY}_[dac|mac].pl1 For example: hcs_fs_get_mode_dac.pl1. o Test program has a standard test program comment header. Test program has a standard history comment. o Test program uses SETUP, RUN and CLEAN_UP internal procedures to test each access control case. These procedures are called by the tu_$map_over_XXX programs to test the various cases of access control, or by an internal table of test cases. o Test program coded such that operation of individual test cases does not depend upon results of previous test cases. o All utility calls done during SETUP (eg, register a resource, mount a logical volume, etc) are undone by the CLEAN_UP procedure of the test program. o Test program properly references the sectest_args structure. o Test program properly checks that the expected results were obtained. Result items include input parameters (which must not have been modified), proper return information and error codes, and correct security audit messages. - 37 - MAB-069 Audit Guidelines B.3: Utility Program Standards The following standards apply to testing utility programs: o Utility program is referenced via the tu_.alm transfer vector, and the target of the vector is a utility program with a standard format name: tu_${GROUP}_{FUNCTION} => tu_{GROUP}_{FUNCTION}.pl1 Example: tu_$rcp_register transfers to tu_rcp_register.pl1. o Utility program has proper calling sequence, with first parameter being a pointer to sectest_args structure, and last three parameters for utilities which manipulate objects being pointers to security, name and nonsecurity property structures. o Utility program with entrypoints which must run in Sectest_Server.Daemon properly calls st_process_daemon_request_ to execute its code within the daemon process. o Each type of manipulated object has its own utility modules: tu_create_OBJECT, tu_delete_OBJECT, tu_get_properties_OBJECT and tu_set_properties_OBJECT. The utility should employ suitable defaults at object creation time, rather than requiring caller to specify all properties. o Some utilities may have a status code parameter to report nonfatal errors to their calling test program, rather than using the sub_err_ mechanism. o Utility program properly undoes any changes in its CLEAN_UP handler, should its operation fail for any reason. In addition, temporary changes made during utility operations should be undone at normal exit by a FINISH procedure. o Utility programs which are not part of a GROUP of functions have entrypoints declared in sectest_tu_dcls.incl.pl1. Utilities which are part of a GROUP (eg, RCP utilities in modules named tu_rcp_FUNCTION.pl1) have entrypoints declared in a group-related include file named: sectest_tu_GROUP_dcls.incl.pl1 For example, sectest_tu_rcp_dcls.incl.pl1. - 38 - Audit Guidelines MAB-069 B.4: Test Program Documentation Test programs must be documented according to standards which have been agreed upon by MDC's B2 implementation team. These standards are described in MDD-004. Testing utility programs must be documented in MDD-004, according to currently accepted Multics documentation standards. It is expected that these standards will be described in MDD-004, Appendix C. Until that documentation is complete, refer to the examples of testing utility documentation in MDD-004, Section 14. Also, consult with the Security Coordinator for details. - 39 - MAB-069 Audit Guidelines APPENDIX C: AUDITING CHECKLIST A sample auditing checklist is shown at the end of this MAB. The form can be used: for a small submission in which all changes are auditing during a single auditing session; or for larger submissions audited during several auditing sessions, perhaps by several different auditors. Separate instructions are given below for using the checklist with each type of audit. C.1: Single Session Audit Note that Single Session audits cannot be used when auditing critical fixes; because there is no MSCR form associated with a critical fix, the names of all modules which are audited must be listed on the auditing checklist. Instead use the Multiple Session audit procedure for critical fixes. When all modules of a submission are audited during a single auditing session, fill out the checklist using the following steps. o In the Type of Audit box, check "Single session". o Write the current date and your person_id on the form. o Review auditing materials against pertinent checklist items. The item number is an MAB section number. If you do not understand a particular checklist item, refer to the MAB section describing that item for details. o When a module fails some checklist item, write its name in one of the MODULE NAME sections, and put the letter associated with its MODULE NAME slot next to the failed checklist item. Make notes in the listings to describe the exact nature of each failure. o General comments and auditing results may be placed on the checklist under OTHER COMMENTS. o When auditing is complete, if all modules passed all checklist items, there will be no names written under the MODULE NAMES, and no failing module letters next to checklist items. In this case, check the PASSED box at the top of the audit checklist as a positive indication that everything passed. o When discussing audit results, if auditor and developer decide to waive a particular failing item for a given module, circle the module's letter next to the failing item. Indicate under OTHER COMMENTS the exact nature of the failure, and why the failure was waived. - 40 - Audit Guidelines MAB-069 o If some modules failed, the developer must correct these failures and then return the modules to the auditor. The auditor follows the procedures under "Multiple Session Audit" on a new checklist to reaudit the failing items. If significant changes were made to a module to correct the failure, reaudit all checklist items for that module rather than just the failing items. C.2: Multiple Session Audit Multiple auditing sessions are needed under the following circumstances. o The submission is too large to be audited by one auditor during a single session. o The submission is too large to be audited by a single auditor. The auditing effort must be split between several auditors. o Some modules failed certain audit items during their original audit. Fixes to these problems must be reaudited during a second auditing session. o A critical fix is being audited. When multiple auditing sessions are required, a separate auditing checklist should be used by each auditor to record the results of each auditing session. Use the checklist according to the steps below. o In the Type of Audit box, check "Multiple session". o Write the current date and your person_id on the form. o Write the names of modules to be audited during this session under MODULE NAMES. Names of all audited modules must be listed, even if the module passes all audit items. o Review auditing materials against pertinent checklist items. o When a module fails some checklist item, put the letter associated with its MODULE NAME slot next to the failed checklist item. Make notes on the failure in the module listing. o General comments and auditing results may be placed on the checklist under OTHER COMMENTS. - 41 - MAB-069 Audit Guidelines o When auditing is complete, if all modules audited during the session passed all checklist items, there will be no failing module letters next to checklist items. In this case, check the PASSED box at the top of the audit checklist as a positive indication that everything audited during the session passed. o When discussing audit results, if auditor and developer decide to waive a particular failing item for a given module, circle the module's letter next to the failing item. Indicate under OTHER COMMENTS the exact nature of the failure, and why the failure was waived. o If some modules failed, the developer must correct these failures and then return the modules to the auditor. The auditor follows the procedures under "Multiple Session Audit" on a new checklist to reaudit the failing items. If significant changes were made to a module to correct the failure, reaudit all checklist items for the module rather than just the failing items. - 42 - ________________Type_of_Audit___________________ AUDITING CHECKLIST | CHECK ONE: | | [ ] SINGLE SESSION: checklist applies to all | [ ] Check here if all modules | modules on MSCR form. Only failing | covered by this checklist | modules are listed in MODULE NAMES. | PASSED audit. | | | [ ] MULTIPLE SESSION: checklist applies ONLY | | to modules listed in MODULE NAMES. Both | Date: _____________________ | passing and failing modules are listed. | |____ Use this method for a critical fix. _____| Auditor: _____________________ MODULE NAMES (for MSCR; bind, include, and info files; test plan, test cases): A. ___________________________________ E. ___________________________________ B. ___________________________________ F. ___________________________________ C. ___________________________________ G. ___________________________________ D. ___________________________________ H. ___________________________________ CHECKLIST for MSCR, and bind, include and info files, and test plan: - Identify failing modules by letter from list above. - Circle letters of failing modules whose failures are WAIVED. - Item numbers are section numbers in MAB-069. Refer there for details. Checklist Items FAILING Module Letters ------------------------------------------------- ---------------------------- 3.2 MSCR is complete, legible. MSCR checklist ____________________________ complete. Unbound segs have proper names. 3.4 Bind file has history comment, and proper ____________________________ Objectname, Addname, Order, retain, synonym and "Global: delete" stmts. 3.5 Include file has BEGIN/END comments and ____________________________ history comment; is self-contained. All modules using include file are on MSCR. 3.6 Info file has proper content and format. ____________________________ Results of "help -brief", "help -title" and validate_info_seg are OK. 3.7 Test plan OK. Test cases follow plan. ____________________________ Test results positive. OTHER COMMENTS: _______________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ _______________________________________________________________________________ Auditing Checklist: version 1. Refer to MAB-069 for details. MODULE NAMES (for source modules): AUDITING CHECKLIST (page 2) J. ___________________________________ P. ___________________________________ K. ___________________________________ R. ___________________________________ L. ___________________________________ S. ___________________________________ M. ___________________________________ T. ___________________________________ N. ___________________________________ U. ___________________________________ CHECKLIST for source modules: - Identify failing modules by letter from list above. - Circle letters of failing modules whose failures are WAIVED. - Item numbers are section or appendix numbers in MAB-069. Checklist Items FAILING Module Letters ------------------------------------------------- ---------------------------- 3.3.1 Module implements only approved functions. ____________________________ 3.3.2 Module operates correctly: input args, ____________________________ cleanup handlers, pgm exit and output args OK. Side effects, and limitations OK. 3.3.3 Module understandable and maintainable: ____________________________ comments, formatting, modularization, data hiding, flow-of-control, variable names, "magic numbers", structured programming OK 3.3.4 Coding standards: pnotice, history comment ____________________________ present; existing pgm upgraded; no use of obsolete or PSP pgm; dcls OK; structure element names qualified; paired actions balanced, no compile errors, etc. 3.3.5 Performance is satisfactory. ____________________________ 3.3.6 Meets requirements, solves original prob. ____________________________ For Trusted Software Modules: A.1 Gate target correctly references parms, ____________________________ checks parm domains, uses standard parm naming rules, uses validation level ok, correctly implements multiclass objects. A.2 Trusted server request processor correctly ____________________________ references parms, checks parm domains, checks access on behalf of submitter, requotes pathnames in subtree walking ecs. A.3 Trusted software module follows security ____________________________ auditing standards, has system error message documentation, has proper source and include file comments, and proper MDD documentation. For B2 Functional Test Modules: B Test programs and utilities follow ____________________________ standards, and are properly documented.