Grading Methodology for Run-Time Errors

A score of 0 is given when the error was not detected.

A score of 1 is given for error messages with the correct error name.

A score of 2 is given for error messages with the correct error name and line number where the error occurred but not the file name where the error occurred.

A score of 3 is given for error messages with the correct error name, line number and the name of the file where the error occurred.

A score of 4 is given for error messages which contain the information for a score of 3 but less information than needed for a score of 5.

A score of 5 is given in all cases when the error message contains all the information needed for the quick fixing of the error.


Results

You can sort the table below by clicking on the table headers with one click sorting from best to worst and another click to sort from worst to best.

UPC results

OS/Compiler/ Runtime Tool Out of Bounds Shared Memory Access Using Indices Out of Bounds Shared Memory Access through Pointers Out of Bounds Shared Memory Access in UPC Functions Argument Errors in UPC Functions Wrong Order of UPC Statements and Function Calls Uninitialized Variables Deadlock Race Condition Memory Related Errors Undefined UPC Operations Warnings (Uninitialized Shared Variables) Average
UNICOS/mp/Cray UPC 1.30 1.04 0.91 0.38 0.84 0.08 0.00 0.01 0.18 0.19 0.27 0.47
Linux/Berkeley UPC 0.10 0.00 0.02 0.17 0.90 0.24 0.33 0.00 0.37 0.39 0.00 0.23
Linux/HP UPC 0.03 0.00 0.02 0.00 0.53 0.57 0.36 0.00 0.16 0.15 0.00 0.17
Linux/GCC UPC 0.20 0.21 0.01 0.00 0.89 0.25 0.27 0.00 0.37 0.41 0.00 0.24
Linux/BUPC/UPC-CHECK 0.20 0.00 0.53 4.89 0.95 0.30 5.00 0.05 0.34 2.35 1.34 1.57

.

Note 1. Section "Non Portable Usage of UPC" includes tests which may or may not deadlock, and tests whose behavior is undefined by UPC Specifications V.1.2.

Note 2. HP UPC does not support UPC IO. These tests are skipped, and the scores are calculated for the reduced set of tests.

Note 3. GCC UPC does not support Collective Utilities and UPC IO. These tests are skipped, and the scores are calculated for the reduced set of tests.

.

Computer info:
Opteron
Linux 2.6.18-194.26.1.el5xen
Iowa State University
Compiler:
Berkeley Unified Parallel C : Version 2.12.1
Compiler options used:
-g
Environment variables:
None
Computer info:
Opteron
Linux 2.6.9-42.ELsmp
Iowa State University
Compiler:
HP UPC V3.1-107 for Linux
Compiler options used:
-g -diag
Environment variables:
UPCRTS_DISP_ALLDEBUG=1
Computer info:
Opteron
Linux 2.6.18-53.1.4.el5 #1 SMP
Iowa State University
Compiler:
gcc version 4.0.3 20070912 upc-4.0.3.4 (GCC UPC)
Compiler options used:
-g
Environment variables:
None
Computer info:
Opteron
Linux 2.6.18-194.26.1.el5xen
Iowa State University
Tool:
UPC-CHECK
Compiler:
Berkeley Unified Parallel C : Version 2.12.1
Compiler options used:
-pthreads --nightly
Environment variables:
None
Computer info:
Cray X1E
UNICOS/mp 3.1.52
Compiler:
Cray C++ : Version 6.0.0.6
Compiler options used:
-g -hssp -hupc -hbounds
Environment variables:
None