A score of 0 is given when the error was not detected.
A score of 1 is given for error messages with the correct error name.
A score of 2 is given for error messages with the correct error name and line number where the error occurred but not the file name where the error occurred.
A score of 3 is given for error messages with the correct error name, line number and the name of the file where the error occurred.
A score of 4 is given for error messages which contain the information for a score of 3 but less information than needed for a score of 5.
A score of 5 is given in all cases when the error message contains all the information needed for the quick fixing of the error.
You can sort the tables below by clicking on the table headers with one click sorting from best to worst and another click to sort from worst to best.
FORTRAN results
| OS/Compiler/ Runtime Tool | Array Index out of Bound | Uninitialized Variables | Subprogram Call Errors | Pointer Errors | Floating Point Errors | String Errors | Allocation Deallocation Errors | Memory Leaks | Input Output Errors | Fortran95 Specific Errors | Array Conformance Errors | Average |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| IBM AIX/IBM XLF | 2.64 | 0.03 | 0.00 | 3.00 | 2.86 | 2.33 | 0.89 | 0.00 | 1.00 | 2.88 | 1.00 | 1.51 |
| Linux/NAGWare Fortran 95 | 3.90 | 3.27 | 2.50 | 4.00 | 0.71 | 2.50 | 2.52 | 0.00 | 1.78 | 3.82 | 1.00 | 2.36 |
| SunOS/i86pc/Sun Fortran 95 | 3.00 | 0.02 | 0.00 | 3.03 | 0.00 | 3.00 | 2.67 | 0.00 | 2.17 | 3.00 | 3.00 | 1.81 |
| SunOS/i86pc/Sun Fortran 95/bcheck | 3.10 | 0.87 | 1.00 | 3.09 | 0.29 | 3.00 | 2.67 | 1.00 | 2.17 | 3.00 | 3.00 | 2.11 |
| SunOS/sparc/Sun Fortran 95 | 3.00 | 0.02 | 0.00 | 3.03 | 0.00 | 3.00 | 2.67 | 0.00 | 2.17 | 2.82 | 3.00 | 1.79 |
| SunOS/sparc/Sun Fortran 95/bcheck | 3.05 | 0.98 | 1.00 | 3.03 | 0.29 | 3.00 | 2.78 | 1.00 | 2.17 | 2.88 | 3.00 | 2.11 |
| Linux/Intel Fortran 7.1 | 1.86 | 0.44 | 1.67 | 2.00 | 0.00 | 2.00 | 2.00 | 0.00 | 1.44 | 1.94 | 0.42 | 1.25 |
| Linux/Intel Fortran 9.0 | 3.45 | 0.12 | 0.17 | 3.56 | 0.57 | 0.00 | 1.54 | 0.00 | 2.06 | 2.82 | 0.42 | 1.34 |
| Cray XT4/ PGI pgf90 | 3.85 | 0.15 | 0.00 | 4.00 | 0.00 | 0.00 | 0.39 | 0.00 | 2.44 | 3.82 | 0.42 | 1.37 |
| Cray X1 UNICOS/mp/Cray Fortran | 3.51 | 0.40 | 2.00 | 4.00 | 0.71 | 4.00 | 2.94 | 0.00 | 2.89 | 3.94 | 3.00 | 2.49 |
| Cray X2/ Cray Fortran | 3.76 | 0.40 | 2.00 | 4.00 | 0.00 | 4.00 | 0.89 | 0.00 | 1.00 | 3.94 | 3.00 | 2.07 |
| Cray XT4/PathScale Fortran | 1.89 | 0.00 | 0.00 | 2.96 | 0.00 | 0.00 | 0.67 | 0.00 | 0.61 | 0.71 | 0.00 | 0.62 |
| Linux/GNU Fortran | 2.82 | 0.01 | 0.00 | 3.00 | 0.00 | 0.00 | 1.04 | 0.00 | 2.33 | 2.88 | 1.00 | 1.19 |
| OS/Compiler/ Runtime Tool | Array Index out of Bound | Uninitialized Variables | Subprogram Call Errors | Pointer Errors | Floating Point Errors | String Errors | Allocation Deallocation Errors | Memory Leaks | Input Output Errors | C99 Specific Errors | Average |
|---|---|---|---|---|---|---|---|---|---|---|---|
| IBM AIX/IBM XLC | 0.04 | 0.00 | 0.00 | 0.04 | 0.00 | 0.18 | 0.64 | 0.20 | 0.00 | 0.10 | 0.12 |
| SunOS/i86pc/Sun C | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| SunOS/i86pc/Sun C/bcheck | 0.02 | 0.00 | 0.00 | 0.06 | 0.00 | 0.00 | 0.16 | 1.13 | 0.00 | 0.00 | 0.14 |
| SunOS/sparc/Sun C | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| SunOS/sparc/Sun C/bcheck | 1.19 | 0.61 | 0.95 | 1.38 | 0.00 | 0.75 | 2.68 | 1.93 | 0.30 | 1.00 | 1.08 |
| Linux/Intel C++ 7.1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| Linux/Intel C++ 9.0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| Cray XT4/ PGI pgcc | 1.94 | 0.00 | 0.00 | 0.02 | 0.00 | 0.05 | 0.44 | 0.07 | 0.20 | 1.54 | 0.43 |
| Cray X1 UNICOS/mp/ Cray C++ | 2.63 | 0.22 | 0.32 | 0.03 | 0.67 | 0.00 | 0.08 | 0.27 | 0.20 | 0.25 | 0.47 |
| Cray X2/ Cray C++ | 2.64 | 0.01 | 0.00 | 0.02 | 0.00 | 0.05 | 0.60 | 0.33 | 0.20 | 0.00 | 0.39 |
| Linux/Insure++ | 3.53 | 3.66 | 0.32 | 3.16 | 0.17 | 3.00 | 3.80 | 3.80 | 3.20 | 2.85 | 2.75 |
| Cray XT4/ PathScale C | 0.01 | 0.00 | 0.00 | 0.02 | 0.00 | 0.05 | 0.44 | 0.07 | 0.20 | 0.04 | 0.08 |
| Linux/GNU C | 0.01 | 0.00 | 0.00 | 0.02 | 0.00 | 0.05 | 0.44 | 0.07 | 0.20 | 0.04 | 0.08 |
| OS/Compiler/ Runtime Tool | Array Index out of Bound | Uninitialized Variables | Pointer Errors | Floating Point Errors | String Errors | Allocation Deallocation Errors | Memory Leaks | Input Output Errors | Average |
|---|---|---|---|---|---|---|---|---|---|
| IBM AIX/IBM XLC | 0.04 | 0.00 | 0.10 | 0.00 | 0.18 | 0.38 | 0.10 | 0.00 | 0.10 |
| SunOS/i86pc/Sun C++ | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| SunOS/i86pc/Sun C++/bcheck | 0.02 | 0.00 | 0.02 | 0.00 | 0.00 | 0.25 | 1.17 | 0.00 | 0.18 |
| SunOS/sparc/Sun C++ | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| SunOS/sparc/Sun C++/bcheck | 1.26 | 1.00 | 1.56 | 0.71 | 1.30 | 1.92 | 1.57 | 1.00 | 1.29 |
| Linux/Intel C++ 7.1 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| Linux/Intel C++ 9.0 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
| Cray XT4/ PGI pgCC | 1.76 | 0.00 | 0.03 | 0.00 | 0.05 | 0.31 | 0.05 | 0.07 | 0.28 |
| UNICOS/mp/Cray C++ | 2.91 | 0.34 | 0.04 | 0.65 | 0.00 | 0.05 | 0.14 | 0.07 | 0.53 |
| Cray X2/ Cray C++ | 2.89 | 0.00 | 0.23 | 0.00 | 0.05 | 0.38 | 0.17 | 0.07 | 0.47 |
| Linux/Insure++ | 3.44 | 3.57 | 2.80 | 0.17 | 3.00 | 3.80 | 3.80 | 3.20 | 2.97 |
| Cray XT4/ PathScale C++ | 0.01 | 0.00 | 0.03 | 0.00 | 0.05 | 0.28 | 0.05 | 0.07 | 0.06 |
| Linux/GNU C++ | 0.01 | 0.00 | 0.03 | 0.00 | 0.05 | 0.31 | 0.05 | 0.07 | 0.06 |
.
.
.
.
.
.