Subversion Repositories DashDisplay

Rev

Rev 56 | Details | Compare with Previous | Last modification | View Log | RSS feed

Rev Author Line No. Line
56 mjames 1 
HowTo DSP_Lib_TestSuite     16.12.2016
2 
=======================================
3 
 
4 
This file describes the folder structure, content, prerequisites and instructions to validate the
5 
build of the CMSIS-DSP library. This is done by processing input data sets using the DSP Library
6 
functions executing on a target simulator or hardware. The output data sets are then compared
7 
with the reference data set produced by unoptimized DSP functions and a Signal to Noise Ratio (SNR)
8 
is computed. If the SNR is below a defined threshold the test is considered "passed".
9 
 
10 
 
11 
Folder structure
12 
----------------
13 
	.\DSP_Lib_TestSuite                                       Batch files for building the reference libraries and running the tests.
14 
	.\DSP_Lib_TestSuite\Common
15 
	.\DSP_Lib_TestSuite\Common\inc                            DSP_Lib test include files
16 
	.\DSP_Lib_TestSuite\Common\JTest                          JTEST Test Framework + INI files for uVision
17 
	.\DSP_Lib_TestSuite\Common\platform                       ARM/GCC device startup/system files
18 
	.\DSP_Lib_TestSuite\Common\src                            DSP_Lib test source files
19 
	.\DSP_Lib_TestSuite\DspLibTest_FVP                        ARM/GCC DSP_Lib test projects for Fixed Virtual Platforms
20 
	.\DSP_Lib_TestSuite\DspLibTest_MPS2                       ARM/GCC DSP_Lib test projects for MPS2
21 
	.\DSP_Lib_TestSuite\DspLibTest_Simulator                  ARM/GCC DSP_Lib test projects for uVision simulator
22 
	.\DSP_Lib_TestSuite\RefLibs                               ARM/GCC DSP_Lib reference libraries (and projects)
23 
 
24 
 
25 
 
26 
Prerequisites
27 
--------------
28 
 - Python (running on Windows). Tested with ActivePython 2.7.8.10.
29 
 - Keil MDK-ARM (tested with MDK-ARM 5.22: http://www2.keil.com/mdk5)
30 
 - ULINKpro debug adapter (http://www2.keil.com/mdk5/ulink)
31 
 - MPS2 (Cortex-M Prototyping System:https://www.arm.com/products/tools/development-boards/versatile-express/cortex-m-prototyping-system.php)
32 
 - CMSIS 5.0.0 (https://github.com/ARM-software/CMSIS_5/releases/tag/5.0.0)
33 
 
34 
 
35 
Setup
36 
------
37 
 - Copy DSP_Lib_TestSuite to the CMSIS installation/pack folder.
38 
      ...
39 
      .\Keil_v5\ARM\PACK\ARM\CMSIS\DSP_Lib
40 
      .\Keil_v5\ARM\PACK\ARM\CMSIS\DSP_Lib_TestSuite                <- location of DSP_Lib_TestSuite
41 
      .\Keil_v5\ARM\PACK\ARM\CMSIS\Include
42 
      ...
43 
 
44 
 - remove 'read-only' tag from folder ./CMSIS/Lib
45 
   (required for rebuild of the DSP_Lib libraries)
46 
 
47 
 - open a Windows command window in folder .\CMSIS\DSP_Lib_TestSuite.
48 
 
49 
 
50 
 
51 
How to run the tests
52 
---------------------
53 
 
54 
a) build the DSP_Lib libraries:
55 
 - batch file: buildDspLibs.bat
56 
   Note: only require if the DSP_Lib source code got updated or the desired configuration is missing
57 
   buildDspLibs.bat overwrites the prebuild libraries in .\CMSIS\Lib.
58 
   Log files of the build process are generated in folder .\CMSIS\DSP_Lib/[ARM|GCC]
59 
 - run:  buildDspLibs.bat in a Windows command window in folder ./CMSIS/DSP_Lib_TestSuite
60 
         buildDspLibs ARM        -> builds the ARMCC libraries
61 
         buildDspLibs GCC        -> builds the GCC libraries
62 
 
63 
b) build the reference libraries:
64 
 - batch file: buildRefLibs.bat
65 
 
66 
   Log files of the build process are generated in folder .\CMSIS\DSP_Lib_TestSuite\RefLibs/[ARM|GCC]
67 
 - run: buildRefLibs.bat in a Windows command window in folder .\CMSIS\DSP_Lib_TestSuite
68 
        buildRefLibs ARM        -> builds the ARMCC reference libraries
69 
        buildRefLibs GCC        -> builds the GCC reference libraries
70 
 
71 
c) running an individual test using uVision (MDK-ARM):
72 
 - batch file: runTest.bat
73 
 - run:  runTest.bat in a Windows command window in folder .\CMSIS\DSP_Lib_TestSuite
74 
         runTest                                -> prints usage information
75 
    e.g. runTest ARM cortexM4lf Simulator       -> runs the test for toolchain ARM, Cortex-M4 littel endian with FPU, uVision Simulator.
76 
 
77 
   Tests running on MPS2 requires additional steps to setup. See section 'MPS2'.
78 
 
79 
d) parsing the test output log file
80 
 - script: parseLog.py
81 
 - run:  parseLog.py python script in a Windows command window in folder .\CMSIS\DSP_Lib_TestSuite
82 
   command line options should match the invocation of the runTest executed before.
83 
   e.g: runTest ARM cortexM4lf Simulator  ->  python parseLog.py ARM cortexM4lf Simulator
84 
 
85 
 - check the test log
86 
   depending on your test parameters change into the required folder
87 
      .\DSP_Lib_TestSuite\DspLibTest_[FVP|MPS2|Simulator]\[ARM|GCC]\Logs
88 
   the folder will contain the following files (e.g. for a 'runTest') :
89 
       DspLibTest_Simulator.log                    raw result of the last test run.
90 
       DspLibTest_Simulator_cortexM4lf.log         raw result of a cortexM4lf test run
91 
       DspLibTest_Simulator_cortexM4lf_build.log   build result of cortexM4lf test
92 
       DspLibTest_Simulator_cortexM4lf_parsed.log  parsed log of raw result of a cortexM4lf test run
93 
       DspLibTest_Simulator_cortexM4lf_time.log    log how long the test took (some tests e.g. M0 take really a long time!).
94 
   'runTest' produces files of the format:     DspLibTest_<test>_<core>...
95 
 
96 
 
97 
Differences between the tests for FVP, MPS2, Simulator
98 
------------------------------------------------------
99 
 - all tests are identical except for:
100 
    'Simulator' uses uVision with uVision simulator and generates also code coverage information
101 
         can be used for little/big endian tests
102 
         ! do not use 'Simulator' for M7 with FPU      -> no uVision simulation available.
103 
         ! do not use 'Simulator' for ARMv8-M devices  -> no uVision simulation available.
104 
    'MPS2' uses uVision with ULINKpro debugger and MPS2. No code coverage information is generated.
105 
         can be used for little endian only (because of the lack of MPS2 FPGA images).
106 
    'FVP' uses uVision with Models debugger. No code coverage information is generated.
107 
         can be used for little/big endian tests.
108 
         ! config files must be prepared.
109 
         ! uVision target for big endianess are not yet prepared.
110 
 
111 
 
112 
Setup 'MPS2'
113 
-------------
114 
 - load the appropriate FPGA image to the MPS2 board matching the CPU of the test builds prior to running the test
115 
 - check if ULINKpro can connect with the configured debug connection (JTAG or SWD) as this must
116 
   match the protocol implemented in the FPGA image.
117 
 
118 
 
119 
How to select tests for "run all tests"
120 
----------------------------------------
121 
 - edit .\CMSIS\DSP_Lib_TestSuite\Common\src\all_tests.c
122 
   comment out all unwanted test groups.
123 
   e.g.  //    JTEST_GROUP_CALL(complex_math_tests);
124 
 
125 
 - edit .\CMSIS\DSP_Lib_TestSuite\Common\src\<test group>/<test group>_group.c
126 
   comment out all unwanted sub test groups.
127 
   e.g. file .\DSP_Lib_TestSuite\Common\src\basic_math_tests\basic_math_test_group.c ->  //    JTEST_GROUP_CALL(abs_tests);
128 
 
129 
 - edit .\CMSIS\DSP_Lib_TestSuite\Common\src\<test group>/<test>_tests.c
130 
   comment out all unwanted tests.
131 
   e.g. file .\DSP_Lib_TestSuite\Common\src\basic_math_tests\abs_tests.c  ->  //    JTEST_TEST_CALL(arm_abs_f32_test);
132 
 
133 
 
134 
Notes
135 
-----
136 
 - How to use ARM Clang (ARM Compiler 6):
137 
   in uVision 'Options for Target' tab you can select which compiler to use
138 
   by default uVision uses ARMCC V5 for Cortex-M devices and ARMCLANG V6 only for ARMv8M.
139 
   Only ARMv8M cores have been tested using ARMCLANG
140 
 
141 
 - test data used for the tests is used as provided by DSP Concepts.
142 
 
143 
 - some tests run for a very long time before they finish. This is expected
144