Table Comparision of RTED Test Results

Grading Methodology for Run-Time Errors

A score of 0 is given when the error was not detected.

A score of 1 is given for error messages with the correct error name.

A score of 2 is given for error messages with the correct error name and line number where the error occurred but not the file name where the error occurred.

A score of 3 is given for error messages with the correct error name, line number and the name of the file where the error occurred.

A score of 4 is given for error messages which contain the information for a score of 3 but less information than needed for a score of 5.

A score of 5 is given in all cases when the error message contains all the information needed for the quick fixing of the error.

Results

You can sort the tables below by clicking on the table headers with one click sorting from best to worst and another click to sort from worst to best.

FORTRAN results

OS/OpenMP Compiler/ Runtime Tool	Deadlock	Race Conditions	Environment and Clause Errors	Wrong Order of OpenMP Directives	Uninitialized Shared and Private Variables	Wrong Usage of OpenMP Runtime Library Routines	Implementation Dependent Errors	Average
IBM AIX/ IBM XLF	0.24	0.00	0.27	0.52	0.02	0.00	0.77	0.26
Unicos/ PGI pgf90	0.00	0.00	0.40	0.00	0.02	0.04	0.70	0.17
Unicos/ GNU gfortran	0.03	0.00	0.40	0.19	0.01	0.00	0.74	0.20
Unicos/ Pathscale pathf90	0.00	0.00	0.21	0.00	0.17	0.19	0.37	0.13
Linux/ Pathscale pathf90	0.00	0.00	0.21	0.00	0.14	0.19	0.37	0.13
UNICOS/ Cray Fortran	0.02	0.02	0.00	0.00	0.03	0.31	1.20	0.23
UNICOS/ Cray Fortran	0.02	0.00	0.00	0.00	0.00	0.31	1.25	0.23
IRIX64/ MIPSpro Fortran	0.18	0.00	0.15	0.39	0.19	0.29	0.53	0.25
Solaris/i86/ Sun Fortran95	0.00	0.00	0.00	0.00	0.04	0.00	0.09	0.02
Solaris/i86/ Sun Fortran95/ Thread Analyzer	0.00	2.67	0.00	0.00	0.04	0.00	0.10	0.40
Linux/ Intel Fortran 9.0	0.00	0.01	0.00	0.00	0.03	0.24	0.65	0.13
Linux/ Intel Fortran 9.0/ Thread Checker	0.00	1.09	0.40	0.00	0.30	0.40	0.77	0.42

C results

OS/OpenMP Compiler/ Runtime Tool	Deadlock	Race Conditions	Environment and Clause Errors	Wrong Order of OpenMP Directives	Uninitialized Shared and Private Variables	Wrong Usage of OpenMP Runtime Library Routines	Implementation Dependent Errors	Average
IBM AIX/ IBM XLC	0.24	0.00	0.15	0.39	0.04	0.00	0.82	0.23
Unicos/ PGI pgcc	0.00	0.00	0.43	0.05	0.05	0.04	0.76	0.19
Unicos/ GNU gcc	0.03	0.00	0.36	0.18	0.02	0.00	0.72	0.19
Unicos/ Pathscale pathcc	0.00	0.00	0.31	0.11	0.23	0.22	0.39	0.18
Linux/ Pathscale pathcc	0.00	0.00	0.31	0.13	0.31	0.22	0.39	0.19
UNICOS/ Cray C++	0.03	0.00	0.00	0.23	0.07	0.31	1.43	0.30
UNICOS/ Cray C++	0.03	0.00	0.00	0.00	0.00	0.31	1.43	0.25
IRIX64/ MIPSpro C	0.24	0.00	0.21	0.50	0.30	0.27	0.54	0.29
Solaris/i86/ Sun C	0.00	0.00	0.00	0.11	0.06	0.00	0.00	0.02
Solaris/i86/ Sun C/ Thread Analyzer	0.00	2.54	0.00	0.13	0.05	0.00	0.00	0.39
Linux/ Intel C++ 9.0	0.00	0.00	0.00	0.08	0.05	0.09	0.73	0.14
Linux/ Intel C++ 9.0/ Thread Checker	0.03	1.55	0.12	0.00	0.38	0.13	0.80	0.43

C++ results

OS/OpenMP Compiler/ Runtime Tool	Deadlock	Race Conditions	Environment and Clause Errors	Wrong Order of OpenMP Directives	Uninitialized Shared and Private Variables	Wrong Usage of OpenMP Runtime Library Routines	Implementation Dependent Errors	Average
IBM AIX/ IBM XLC	0.24	0.00	0.23	0.33	0.05	0.00	1.22	0.30
Unicos/ PGI pgCC	0.00	0.00	0.00	0.04	0.02	0.04	1.35	0.21
Unicos/ GNU g++	0.03	0.00	0.36	0.40	0.02	0.00	1.08	0.27
Unicos/ Pathscale pathCC	0.00	0.00	0.31	0.19	0.21	0.22	0.55	0.21
Linux/ Pathscale pathCC	0.00	0.00	0.31	0.31	0.29	0.22	0.55	0.24
UNICOS/ Cray C++	0.03	0.01	0.00	0.28	0.10	0.31	2.40	0.45
UNICOS/ Cray C++	0.03	0.00	0.00	0.05	0.00	0.31	2.40	0.40
IRIX64/ MIPSpro C++	0.24	0.00	0.19	0.42	0.31	0.27	0.81	0.32
Solaris/i86/ Sun C++	0.00	0.00	0.00	0.12	0.08	0.00	0.00	0.03
Solaris/i86/ Sun C++/ Thread Analyzer	0.00	2.55	0.00	0.15	0.08	0.00	0.00	0.40
Linux/ Intel C++ 9.0	0.00	0.00	0.00	0.15	0.07	0.09	1.08	0.20
Linux/ Intel C++ 9.0/ Thread Checker	0.03	1.56	0.12	0.10	0.52	0.13	1.15	0.52

Computer info:

DataStar

IBM p-Series

University of California, San Diego

MPI:

Parallel Environment for AIX 5L V4.1

Compiler:

IBM(R) XL Fortran Enterprise Edition V9.1o

Compiler options used:

-qsuffix=f=f90 -qcheck -qsource

-g -qsigtrap=xl__trce

-qflttrap=enable:zerodivide:invalid:overflow

Environment variables:

MP_EUIDEVELOP=yes

Computer info:

Cray XT4 (Opteron)

Unicos/LC

Compiler:

pgf90 7.0-6 64-bit target on x86-64 Linux

Compiler options used:

-mp -Mconcur=nonuma -O0 -C -g -Ktrap=fp -Mbounds -Mchkfpstk -Mchkstk -Mchkptr

Computer info:

Cray XT4 (Opteron)

Unicos/LC

Compiler:

gfortran V. 4.2.0

Compiler options used:

-fopenmp -frange-check -O0 -fbounds-check -g -ffpe-trap=invalid,zero,overflow -fdiagnostics-show-location=every-line -fcray-pointer

Computer info:

Cray XT4 (Opteron)

Unicos/LC

Compiler:

pathf90 V. 3.0

Compiler options used:

-mp -O0 -g -trapuv -TENV:X=0 -Wdiv-by-zero -C

Computer info:

Lightning - Linux AMD Opteron Cluster

Linux 2.6.9-34.ELsmp

Compiler:

pathf90 V. 2.4

Compiler options used:

-mp -O0 -g -trapuv -TENV:X=0 -Wdiv-by-zero -C

Computer info:

Cray X1

UNICOS/mp 3.1.24

Compiler:

Cray Fortran : Version 5.6.0.0.28

Compiler options used:

-O1 -O task1 -m2 -rl -Rabcs

Computer info:

Cray X2

UNICOS/mp 3.1.24

Compiler:

Cray Fortran : Version 6.0.0.1.2

Compiler options used:

-O1 -O task1 -m2 -rl -Rabcs

Computer info:

SGI Origin 2000

IRIX64 6.5

Compiler:

MIPSpro Compilers f90 : Version 7.4.2m

Compiler options used:

-O0 -g -DEBUG:div_check=3:subscript_check=ON:

conform_check=ON:varargs_prototypes=ON:

varargs_interface_check=ON:

verbose_runtime=ON:fullwarn=ON

Computer info:

Opteron

Solaris 10 i86pc

Iowa State University

Compiler:

Sun Fortran 95 8.3 2007/02/08

Compiler options used:

-xopenmp=noopt -g

Computer info:

Opteron

Solaris 10 i86pc

Iowa State University

Compiler:

Sun Fortran 95 8.3 2007/02/08

Compiler options used:

-xinstrument=datarace -xopenmp=noopt -g

Runtime Tool:

Sun Studio 12 - Thread Analyzer

Computer info:

Intel Pentium 4

Linux 2.6.9 ELsmp

Iowa State University

Compiler:

Intel(R) Fortran Compiler Version 9.0

Compiler options used:

-openmp -g -O0

Computer info:

Intel Pentium 4

Linux 2.6.9 ELsmp

Iowa State University

Compiler:

Intel(R) Fortran Compiler Version 9.0

Compiler options used:

-openmp -g -O0

Runtime Tool:

Intel Thread Checker

Computer info:

DataStar

IBM p-Series

University of California, San Diego

San Diego Supercomputer Center

Compiler:

IBM(R) XL C/C++ Enterprise Edition V7.0

Compiler options used:

-qheapdebug -qinfo -qcheck -g -qsource

-qflttrap=enable:zerodivide:invalid:overflow

Environment variables:

MP_EUIDEVELOP=yes

Computer info:

Cray XT4 (Opteron)

Unicos/LC

Compiler:

pgcc 7.0-6 64-bit target on x86-64 Linux

Compiler options used:

-mp=nonuma -c99 -O0 -C -g -Mdepchk -Ktrap=fp -Mbounds -Mchkfpstk -Mchkstk -Msignextend

Computer info:

Cray XT4 (Opteron)

Unicos/LC

Compiler:

pathcc V. 3.0

Compiler options used:

-mp -O0 -g -trapuv -TENV:X=0 -Wchar-subscripts -Wdiv-by-zero -std=c99

Computer info:

Cray XT4 (Opteron)

Unicos/LC

Compiler:

gcc V. 4.2.0

Compiler options used:

-fopenmp -std=c99 -O0 -fbounds-check -g -ftrapping-math -fsignaling-nans -fcx-limited-range

Computer info:

Lightning - Linux AMD Opteron Cluster

Linux 2.6.9-34.ELsmp

Compiler:

pathcc V. 2.4

Compiler options used:

-mp -O0 -g -trapuv -TENV:X=0 -Wdiv-by-zero -C

Computer info:

Cray X1

UNICOS/mp 3.1.24

Compiler:

Cray C++ : Version 5.6.0.0.89

Compiler options used:

-O1 -h omp -h task1 -h bounds -h matherror=abort -h c99 -h exceptions

Computer info:

Cray X2

UNICOS/mp 3.1.24

Compiler:

Cray C++ : Version 6.0.0.1.4

Compiler options used:

-O1 -h omp -h task1 -h bounds -h matherror=abort -h c99 -h exceptions

Computer info:

SGI Origin 2000

IRIX64 6.5

Compiler:

MIPSpro Compilers cc: Version 7.4.2m

Compiler options used:

-O0 -g

Computer info:

Opteron

Solaris 10 i86pc

Iowa State University

Compiler:

Sun C 5.9 2007/02/08

Compiler options used:

-xopenmp=noopt -g

Computer info:

Opteron

Solaris 10 i86pc

Iowa State University

Compiler:

Sun C 5.9 2007/02/08

Compiler options used:

-xinstrument=datarace -xopenmp=noopt -g

Runtime Tool:

Sun Studio 12 - Thread Analyzer

Computer info:

Intel Pentium 4

Linux 2.6.9 ELsmp

Iowa State University

Compiler:

Intel(R) C++ Compiler Version 9.0

Compiler options used:

-openmp -g -O0

Computer info:

Intel Pentium 4

Linux 2.6.9 ELsmp

Iowa State University

Compiler:

Intel(R) C++ Compiler Version 9.0

Compiler options used:

-openmp -g -O0

Runtime Tool:

Intel Thread Checker

Computer info:

DataStar

IBM p-Series

University of California, San Diego

San Diego Supercomputer Center

Compiler:

IBM(R) XL C/C++ Enterprise Edition V7.0

Compiler options used:

-cpp -qheapdebug -qinfo -qrtti

-qcheck -g -qsource

-qflttrap=enable:zerodivide:invalid:overflow

Environment variables:

MP_EUIDEVELOP=yes

Computer info:

Cray XT4 (Opteron)

Unicos/LC

Compiler:

pgCC 7.0-6 64-bit target on x86-64 Linux

Compiler options used:

-mp=nonuma -O0 -C -g -Mdepchk -Ktrap=fp -Mbounds -Mchkfpstk -Mchkstk -Msignextend --exceptions

Computer info:

Cray XT4 (Opteron)

Unicos/LC

Compiler:

pathCC V. 3.0

Compiler options used:

-mp -O0 -g -trapuv -TENV:X=0 -Wchar-subscripts -Wdiv-by-zero

Computer info:

Cray XT4 (Opteron)

Unicos/LC

Compiler:

g++ V. 4.2.0

Compiler options used:

-fopenmp -O0 -fbounds-check -g -ftrapping-math -fsignaling-nans -fcx-limited-range

Computer info:

Lightning - Linux AMD Opteron Cluster

Linux 2.6.9-34.ELsmp

Compiler:

pathCC V. 2.4

Compiler options used:

-mp -O0 -g -trapuv -TENV:X=0 -Wdiv-by-zero -C

Computer info:

Cray X1

UNICOS/mp 3.1.24

Compiler:

Cray C++ : Version 5.6.0.0.89

Compiler options used:

-O1 -h omp -h task1 -h bounds -h matherror=abort -h exceptions -h scalar1

Computer info:

Cray X2

UNICOS/mp 3.1.24

Compiler:

Cray C++ : Version 6.0.0.1.4

Compiler options used:

-O1 -h omp -h task1 -h bounds -h matherror=abort -h exceptions -h scalar1

Computer info:

SGI Origin 2000

IRIX64 6.5

Compiler:

MIPSpro Compilers CC: Version 7.4.2m

Compiler options used:

-O0 -g

Computer info:

Opteron

Solaris 10 i86pc

Iowa State University

Compiler:

Sun C++ 5.9 2007/02/08

Compiler options used:

-xopenmp=noopt -g

Computer info:

Opteron

Solaris 10 i86pc

Iowa State University

Compiler:

Sun C++ 5.9 2007/02/08

Compiler options used:

-xinstrument=datarace -xopenmp=noopt -g

Runtime Tool:

Sun Studio 12 - Thread Analyzer

Computer info:

Intel Pentium 4

Linux 2.6.9 ELsmp

Iowa State University

Compiler:

Intel(R) C++ Compiler Version 9.0

Compiler options used:

-openmp -g -O0

Computer info:

Intel Pentium 4

Linux 2.6.9 ELsmp

Iowa State University

Compiler:

Intel(R) C++ Compiler Version 9.0

Compiler options used:

-openmp -g -O0

Runtime Tool:

Intel Thread Checker