Merge pull request #81 from ARM-software/master

merging changes from master for v2.4.

README.md

@@ -3,24 +3,25 @@
 
 
 ## Server Base System Architecture
-**Server Base System Architecture** (SBSA) specification specifies a hardware system architecture based on the ARM 64-bit architecture. Server system software such as operating systems, hypervisors, and firmware rely on this. It addresses processing element features and key aspects of system architecture.
+**Server Base System Architecture** (SBSA) specification specifies a hardware system architecture based on the Arm 64-bit architecture. Server system software such as operating systems, hypervisors, and firmware rely on this. It addresses processing element features and key aspects of system architecture.
 
 For more information, download the [SBSA specification](http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.den0029b/index.html)
 
 
 ## SBSA - Architecture Compliance Suite
 
 SBSA **Architecture Compliance Suite** (ACS) is a collection of self-checking, portable C-based tests.
-This suite includes a set of examples of the invariant behaviours that are provided by the [SBSA](http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.den0029b/index.html) specification, so that implementers can verify if these behaviours have been interpreted correctly.
+This suite includes a set of examples of the invariant behaviors that are provided by the [SBSA](http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.den0029b/index.html) specification, so that implementers can verify if these behaviours have been interpreted correctly.
 Most of the tests are executed from UEFI Shell by executing the SBSA UEFI shell application.
 A few tests are executed by running the SBSA ACS Linux application which in turn depends on the SBSA ACS Linux kernel module.
 
 
 ## Release details
- - Code Quality: REL v2.3
+ - Code Quality: REL v2.4
  - The tests are written for version 5.0 of the SBSA specification.
+ - PCIe RCiEP tests for Appendix E of SBSA 6.0 specification are also included.
  - The compliance suite is not a substitute for design verification.
- - To review the SBSA ACS logs, ARM licensees can contact ARM directly through their partner managers.
+ - To review the SBSA ACS logs, Arm licensees can contact Arm directly through their partner managers.
  - To know about the gaps in the test coverage, see [Testcase checklist](docs/testcase-checklist.md).
  - Refer to changelog.txt to know about the changes.
 
@@ -49,7 +50,7 @@ Prebuilt images for each release are available in the prebuilt_images folder of
 ### Prerequisites
     Before starting the ACS build, ensure that the following requirements are met.
 
-- Any mainstream Linux based OS distribution.
+- Any mainstream Linux based OS distribution running on a x86 or aarch64 machine.
 - git clone the UDK2018 branch of [EDK2 tree](https://github.com/tianocore/edk2).
 - Install GCC 5.3 or later toolchain for Linux from [here](https://releases.linaro.org/components/toolchain/binaries/).
 - Install the build prerequisite packages to build EDK2. 
@@ -66,7 +67,7 @@ To start the ACS build, perform the following steps:
 
 ### Linux build environment
 If the build environment is Linux, perform the following steps:
-1.  export GCC49_AARCH64_PREFIX= GCC5.3 toolchain path pointing to /bin/aarch64-linux-gnu-
+1.  export GCC49_AARCH64_PREFIX= GCC5.3 toolchain path pointing to /bin/aarch64-linux-gnu- in case of x86 machine. For aarch64 build it should point to /usr/bin/
 2.  source edksetup.sh
 3.  make -C BaseTools/Source/C
 4.  source AppPkg/Applications/sbsa-acs/tools/scripts/avsbuild.sh
@@ -80,17 +81,18 @@ If the build environment is Windows, perform the following steps:
    build -a AARCH64 -t GCC49 -p ShellPkg/ShellPkg.dsc -m
    AppPkg/Applications/sbsa-acs/uefi_app/SbsaAvs.inf
 
+**Note:** To build the ACS with NIST Statistical Test Suite, see [SBSA_NIST_User_Guide](docs/Arm_SBSA_NIST_User_Guide.md)
+
 ### Build output
 
-The EFI executable file is generated at 
-edk2_path /Build/Shell/DEBUG_GCC49/AARCH64/Sbsa.efi
+The EFI executable file is generated at <edk2_path>/Build/Shell/DEBUG_GCC49/AARCH64/Sbsa.efi
 
 
 ## Test suite execution
 
 The execution of the compliance suite varies depending on the test environment. These steps assume that the test suite is invoked through the ACS UEFI shell application.
 
-For details about the SBSA ACS UEFI Shell application, see [SBSA ACS USER Guide](docs/SBSA_ACS_User_Guide.pdf)
+For details about the SBSA ACS UEFI Shell application, see [SBSA ACS USER Guide](docs/Arm_SBSA_Architecture_Compliance_User_Guide.pdf)
 
 ### Post-Silicon
 
@@ -102,7 +104,7 @@ On a system where a USB port is available and functional, perform the following
 4. To determine the file system number of the plugged in USB drive, execute 'map -r' command. 
 5. Type 'fsx' where 'x' is replaced by the number determined in step 4.
 6. To start the compliance tests, run the executable Sbsa.efi with the appropriate parameters. 
-   For details on the parameters, refer to [SBSA ACS User Guide](docs/SBSA_ACS_User_Guide.pdf)
+   For details on the parameters, refer to [SBSA ACS User Guide](docs/Arm_SBSA_Architecture_Compliance_User_Guide.pdf)
 7. Copy the UART console output to a log file for analysis and certification.
 
 
@@ -119,7 +121,7 @@ On an emulation environment with secondary storage, perform the following steps:
 4. To determine the file system number of the secondary storage, execute 'map -r' command. 
 5. Type 'fsx' where 'x' is replaced by the number determined in step 4.
 6. To start the compliance tests, run the executable Sbsa.efi with the appropriate parameters. 
-   For details on the parameters, see [SBSA ACS User Guide](docs/SBSA_ACS_User_Guide.pdf)
+   For details on the parameters, see [SBSA ACS User Guide](docs/Arm_SBSA_Architecture_Compliance_User_Guide.pdf)
 7. Copy the UART console output to a log file for analysis and certification.
 
 
@@ -130,12 +132,17 @@ On an Emulation platform where secondary storage is not available, perform the f
 1. Add the path to 'Sbsa.efi' file in the UEFI FD file.
 2. Build UEFI image including the UEFI Shell.
 3. Boot the system to UEFI shell.
-4. Run the executable 'Sbsa.efi' to start the compliance tests. For details about the parameters, see [SBSA ACS User Guide](docs/SBSA_ACS_User_Guide.pdf).
+4. Run the executable 'Sbsa.efi' to start the compliance tests. For details about the parameters,
+   see [SBSA ACS User Guide](docs/Arm_SBSA_Architecture_Compliance_User_Guide.pdf).
 5. Copy the UART console output to a log file for analysis and certification.
 
 
 ## Linux OS-based tests
-Certain PCIe and IOMMU tests require Linux operating system with kernel version 4.10 or above. The procedure to build and run these tests is described in [SBSA ACS User Guide](docs/SBSA_ACS_User_Guide.pdf).
+Certain PCIe and IOMMU tests require Linux operating system with kernel version 4.10 or above.
+The procedure to build and run these tests is described in [SBSA ACS User Guide](docs/Arm_SBSA_Architecture_Compliance_User_Guide.pdf).
+
+## Security implication
+Arm Enterprise ACS test suite may run at higher privilege level. An attacker may utilize these tests as a means to elevate privilege which can potentially reveal the platform security assets. To prevent the leakage of secure information, it is strongly recommended that the ACS test suite is run only on development platforms. If it is run on production systems, the system should be scrubbed after running the test suite.
 
 
 ## License
@@ -146,5 +153,9 @@ SBSA ACS is distributed under Apache v2.0 License.
 
  - For feedback, use the GitHub Issue Tracker that is associated with this repository.
  - For support, please send an email to "[email protected]" with details.
- - ARM licensees can contact ARM directly through their partner managers.
- - ARM welcomes code contributions through GitHub pull requests. See GitHub documentation on how to raise pull requests.
+ - Arm licensees may contact Arm directly through their partner managers.
+ - Arm welcomes code contributions through GitHub pull requests. See GitHub documentation on how to raise pull requests.
+
+--------------
+
+*Copyright (c) 2018-2020, Arm Limited and Contributors. All rights reserved.*

docs/Arm_SBSA_NIST_User_Guide.md

@@ -0,0 +1,177 @@
+# National Institute of Standards and Technology: Statistical Test Suite
+
+**Need for Randomness?**
+
+Randomness relates to many areas of computer science, in particular with cryptography. Well-designed cryptographic primitives like hash functions, stream ciphers should produce pseudorandom data. The outputs of such generators may be used in many cryptographic applications like the generation of key material. Generators suitable for use in cryptographic applications need to meet stronger requirements than for other applications. In particular, their outputs must be unpredictable in the absence of knowledge of the inputs.
+
+**Statistical Test Suites**
+
+Randomness testing plays a fundamental role in cryptography. Randomness is tested using test suites consisting of many tests of randomness each focusing on different feature. These tests can be used as first steps in determining whether or not a generator is suitable for a particular cryptographic application.
+
+**NIST with SBSA**
+
+There are five well-known statistical test suites -- NIST STS, Diehard, TestU01, ENT and CryptX. Only the first three test suites are commonly used for the randomness analysis since CryptX is a commercial software and ENT provides only basic randomness testing. Since NIST STS has a special position by being published as an official document, it is often used in the preparation of formal certifications or approvals.
+
+**Building NIST STS with SBSA ACS**
+
+To build NIST statistical test suite with SBSA ACS, NIST STS 2.1.2 package is required. This package is obtained from <https://csrc.nist.gov/CSRC/media/Projects/Random-Bit-Generation/documents/sts-2_1_2.zip>  and is downloaded automatically as part of the build process.
+
+This is an updated version of [NIST Statistical Test Suite (STS)](http://csrc.nist.gov/groups/ST/toolkit/rng/documentation_software.html) tool for randomness testing. The reason for the update is, the original source code provided with NIST does not compile cleanly in UEFI because it does not provide erf() and erfc() functions in the standard math library and (harcoded the inputs -- needs to be rephrased). Implementation of these functions has been added as part of SBSA val and a patch file is created.
+
+**Tool Requirement**
+
+Current release require the below tool:
+
+1. unzip
+
+**Build steps**
+
+To start the ACS build with NIST STS, perform the following steps:
+
+1.  Add the following to the [LibraryClasses.common] section in ShellPkg/ShellPkg.dsc
+```
+    UefiRuntimeLib|MdePkg/Library/UefiRuntimeLib/UefiRuntimeLib.inf
+    !ifdef $(ENABLE_NIST)
+        SbsaNistLib|AppPkg/Applications/sbsa-acs/test_pool/nist_sts/SbsaNistLib.inf
+        SbsaValNistLib|AppPkg/Applications/sbsa-acs/val/SbsaValNistLib.inf
+        SbsaPalNistLib|AppPkg/Applications/sbsa-acs/platform/pal_uefi/SbsaPalNistLib.inf
+    !else
+        SbsaValLib|AppPkg/Applications/sbsa-acs/val/SbsaValLib.inf
+        SbsaPalLib|AppPkg/Applications/sbsa-acs/platform/pal_uefi/SbsaPalLib.inf
+    !endif
+```
+2.  Add the following in the [components] section of ShellPkg/ShellPkg.dsc
+```
+    !ifdef $(ENABLE_NIST)
+        AppPkg/Applications/sbsa-acs/uefi_app/SbsaAvsNist.inf
+    !else
+        AppPkg/Applications/sbsa-acs/uefi_app/SbsaAvs.inf
+    !endif
+```
+3.  Modify CC Flags in the [BuildOptions] section of ShellPkg/ShellPkg.dsc
+```
+    !ifdef $(ENABLE_NIST)
+        *_*_*_CC_FLAGS = -DENABLE_NIST
+    !else
+        *_*_*_CC_FLAGS =
+    !endif
+
+    !include StdLib/StdLib.inc
+```
+4.  Modify the following in the StdLib/LibC/Main/Main.c
+```
+    -extern int main( int, char**);
+    +extern int ShellAppMainsbsa( int, char**);
+```
+5.  Modify the following in ShellAppMain() of StdLib/LibC/Main/Main.c
+```
+    -ExitVal = (INTN)main( (int)Argc, gMD->NArgV);
+    +ExitVal = (INTN)ShellAppMainsbsa( (int)Argc, gMD->NArgV);
+```
+6. Comment the map[] variable in StdLib/LibC/Main/Arm/flt_rounds.c to avoid -werror=unused-variable
+```
+    +#if 0
+    static const int map[] = {
+      1,  /* round to nearest */
+      2,  /* round to positive infinity */
+      3,  /* round to negative infinity */
+      0   /* round to zero */
+    };
+    +#endif
+```
+
+
+To build the SBSA test suite with NIST STS, execute the following commands:
+***Linux build environment***
+```
+source AppPkg/Applications/sbsa-acs/tools/scripts/avsbuild.sh NIST
+```
+
+***Windows build environment***
+```
+build -a AARCH64 -t GCC49 -p ShellPkg/ShellPkg.dsc -m AppPkg/Applications/sbsa-acs/uefi_app/SbsaAvs.inf -D ENABLE_NIST
+```
+
+**Directory structure of SBSA ACS**
+
+The following figure shows the source code directory for SBSA ACS
+
+    sbsa<br/>
+    ├── docs<br/>
+    ├── linux_app<br/>
+    ├── patches<br/>
+    │   └── nist_sbsa_sts.diff  ────────> Patch to compile NIST STS with SBSA ACS<br/>
+    │<br/>
+    ├── platform<br/>
+    │   ├── pal_baremetal<br/>
+    │   ├── pal_linux<br/>
+    │   ├── pal_uefi<br/>
+    │   └── secure_sw<br/>
+    │       └── arm-tf<br/>
+    │<br/>
+    ├── test_pool<br/>
+    │   ├── exerciser<br/>
+    │   ├── gic<br/>
+    │   ├── io_virt<br/>
+    │   ├── pcie<br/>
+    │   ├── pe<br/>
+    │   ├── peripherals<br/>
+    │   ├──  power_wakeup<br/>
+    │   ├── secure<br/>
+    │   ├── timer_wd<br/>
+    │   └── nist_sts<br/>
+    │       ├── test_n001.c     ────────>  NIST entry point to STS<br/>
+    │       └── sts-2.1.2<br/>
+    │           └── sts-2.1.2   ────────>  NIST STS package<br/>
+    │<br/>
+    ├── tools<br/>
+    │   └── scripts<br/>
+    ├── uefi_app<br/>
+    └── val<br/>
+        ├── include<br/>
+        └── src<br/>
+            └── avs_nist.c      ────────>  erf and erfc() implementations<br/>
+
+**Running NIST STS**
+
+Run the UEFI Shell application with the "-nist" as an argument argument
+
+    uefi shell> sbsa.efi -nist
+
+**Interpreting the results**
+
+Final analysis report is generated when statistical testing is complete. The report contains a summary of empirical results which is displayed on the console. A test is unsuccessful when P-value < 0.01 and then the sequence under test should be considered as non-random. Example result as below
+
+    ------------------------------------------------------------------------------
+    RESULTS FOR THE UNIFORMITY OF P-VALUES AND THE PROPORTION OF PASSING SEQUENCES
+    ------------------------------------------------------------------------------
+    ------------------------------------------------------------------------------
+     C1  C2  C3  C4  C5  C6  C7  C8  C9 C10  P-VALUE  PROPORTION  STATISTICAL TEST
+    ------------------------------------------------------------------------------
+      4   1   1   2   1   1   0   0   0   0  0.122325     10/10      OverlappingTemplate
+     10   0   0   0   0   0   0   0   0   0  0.000000 *    0/10   *  Universal
+      7   2   1   0   0   0   0   0   0   0  0.000001 *    8/10      ApproximateEntropy
+      3   4   1   0   2   0   0   0   0   0  0.017912      9/10      Serial
+      4   0   1   0   2   1   0   1   1   0  0.122325     10/10      Serial
+      1   1   1   1   0   2   0   3   0   1  0.534146     10/10      LinearComplexity
+
+
+    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+    The minimum pass rate for each statistical test with the exception of the
+    random excursion (variant) test is approximately = 8 for a
+    sample size = 10 binary sequences.
+
+    The minimum pass rate for the random excursion (variant) test is undefined.
+
+    For further guidelines construct a probability table using the MAPLE program
+    provided in the addendum section of the documentation.
+    - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+
+
+For more details on NIST STS, see: <https://doi.org/10.6028/NIST.SP.800-22r1a>
+
+**Note**: For SBSA level 6 compliance, passing the NIST statistical test suite is not mandatory.
+
+--------------
+
+*Copyright (c) 2020, Arm Limited and Contributors. All rights reserved.*