| 1 | <html lang="en">
|
|---|
| 2 | <head>
|
|---|
| 3 | <title>Installing GCC: Testing</title>
|
|---|
| 4 | <meta http-equiv="Content-Type" content="text/html">
|
|---|
| 5 | <meta name="description" content="Installing GCC: Testing">
|
|---|
| 6 | <meta name="generator" content="makeinfo 4.6">
|
|---|
| 7 | <!--
|
|---|
| 8 | Copyright © 1988, 1989, 1992, 1993, 1994, 1995, 1996, 1997, 1998,
|
|---|
| 9 | 1999, 2000, 2001, 2002, 2003 Free Software Foundation, Inc.
|
|---|
| 10 | <br><p>
|
|---|
| 11 | <p>Permission is granted to copy, distribute and/or modify this document
|
|---|
| 12 | under the terms of the GNU Free Documentation License, Version 1.2 or
|
|---|
| 13 | any later version published by the Free Software Foundation; with no
|
|---|
| 14 | Invariant Sections, the Front-Cover texts being (a) (see below), and
|
|---|
| 15 | with the Back-Cover Texts being (b) (see below). A copy of the
|
|---|
| 16 | license is included in the section entitled "<a href="./gfdl.html">GNU Free Documentation License</a>".
|
|---|
| 17 |
|
|---|
| 18 | <p>(a) The FSF's Front-Cover Text is:
|
|---|
| 19 |
|
|---|
| 20 | <p>A GNU Manual
|
|---|
| 21 |
|
|---|
| 22 | <p>(b) The FSF's Back-Cover Text is:
|
|---|
| 23 |
|
|---|
| 24 | <p>You have freedom to copy and modify this GNU Manual, like GNU
|
|---|
| 25 | software. Copies published by the Free Software Foundation raise
|
|---|
| 26 | funds for GNU development.-->
|
|---|
| 27 | <meta http-equiv="Content-Style-Type" content="text/css">
|
|---|
| 28 | <style type="text/css"><!--
|
|---|
| 29 | pre.display { font-family:inherit }
|
|---|
| 30 | pre.format { font-family:inherit }
|
|---|
| 31 | pre.smalldisplay { font-family:inherit; font-size:smaller }
|
|---|
| 32 | pre.smallformat { font-family:inherit; font-size:smaller }
|
|---|
| 33 | pre.smallexample { font-size:smaller }
|
|---|
| 34 | pre.smalllisp { font-size:smaller }
|
|---|
| 35 | --></style>
|
|---|
| 36 | </head>
|
|---|
| 37 | <body>
|
|---|
| 38 | <h1 class="settitle">Installing GCC: Testing</h1>
|
|---|
| 39 | Before you install GCC, we encourage you to run the testsuites and to
|
|---|
| 40 | compare your results with results from a similar configuration that have
|
|---|
| 41 | been submitted to the
|
|---|
| 42 | <a href="http://gcc.gnu.org/ml/gcc-testresults/">gcc-testresults mailing list</a>.
|
|---|
| 43 | Some of these archived results are linked from the build status lists
|
|---|
| 44 | at <a href="http://gcc.gnu.org/buildstat.html">http://gcc.gnu.org/buildstat.html</a>, although not everyone who
|
|---|
| 45 | reports a successful build runs the testsuites and submits the results.
|
|---|
| 46 | This step is optional and may require you to download additional software,
|
|---|
| 47 | but it can give you confidence in your new GCC installation or point out
|
|---|
| 48 | problems before you install and start using your new GCC.
|
|---|
| 49 |
|
|---|
| 50 | <p>First, you must have <a href="download.html">downloaded the testsuites</a>.
|
|---|
| 51 | These are part of the full distribution, but if you downloaded the
|
|---|
| 52 | "core" compiler plus any front ends, you must download the testsuites
|
|---|
| 53 | separately.
|
|---|
| 54 |
|
|---|
| 55 | <p>Second, you must have the testing tools installed. This includes
|
|---|
| 56 | <a href="http://www.gnu.org/software/dejagnu/">DejaGnu</a> 1.4.1 or 1.4.3
|
|---|
| 57 | and later, Tcl, and Expect; the DejaGnu site has links to these.
|
|---|
| 58 |
|
|---|
| 59 | <p>If the directories where <code>runtest</code> and <code>expect</code> were
|
|---|
| 60 | installed are not in the <code>PATH</code>, you may need to set the following
|
|---|
| 61 | environment variables appropriately, as in the following example (which
|
|---|
| 62 | assumes that DejaGnu has been installed under <code>/usr/local</code>):
|
|---|
| 63 |
|
|---|
| 64 | <pre class="example"> TCL_LIBRARY = /usr/local/share/tcl8.0
|
|---|
| 65 | DEJAGNULIBS = /usr/local/share/dejagnu
|
|---|
| 66 | </pre>
|
|---|
| 67 |
|
|---|
| 68 | <p>(On systems such as Cygwin, these paths are required to be actual
|
|---|
| 69 | paths, not mounts or links; presumably this is due to some lack of
|
|---|
| 70 | portability in the DejaGnu code.)
|
|---|
| 71 |
|
|---|
| 72 | <p>Finally, you can run the testsuite (which may take a long time):
|
|---|
| 73 | <pre class="example"> cd <var>objdir</var>; make -k check
|
|---|
| 74 | </pre>
|
|---|
| 75 |
|
|---|
| 76 | <p>This will test various components of GCC, such as compiler
|
|---|
| 77 | front ends and runtime libraries. While running the testsuite, DejaGnu
|
|---|
| 78 | might emit some harmless messages resembling
|
|---|
| 79 | <code>WARNING: Couldn't find the global config file.</code> or
|
|---|
| 80 | <code>WARNING: Couldn't find tool init file</code> that can be ignored.
|
|---|
| 81 |
|
|---|
| 82 | <h3 class="section"><a name="TOC0"></a>How can I run the test suite on selected tests?</h3>
|
|---|
| 83 |
|
|---|
| 84 | <p>In order to run sets of tests selectively, there are targets
|
|---|
| 85 | <code>make check-gcc</code> and <code>make check-g++</code>
|
|---|
| 86 | in the <code>gcc</code> subdirectory of the object directory. You can also
|
|---|
| 87 | just run <code>make check</code> in a subdirectory of the object directory.
|
|---|
| 88 |
|
|---|
| 89 | <p>A more selective way to just run all <code>gcc</code> execute tests in the
|
|---|
| 90 | testsuite is to use
|
|---|
| 91 |
|
|---|
| 92 | <pre class="example"> make check-gcc RUNTESTFLAGS="execute.exp <var>other-options</var>"
|
|---|
| 93 | </pre>
|
|---|
| 94 |
|
|---|
| 95 | <p>Likewise, in order to run only the <code>g++</code> "old-deja" tests in
|
|---|
| 96 | the testsuite with filenames matching <code>9805*</code>, you would use
|
|---|
| 97 |
|
|---|
| 98 | <pre class="example"> make check-g++ RUNTESTFLAGS="old-deja.exp=9805* <var>other-options</var>"
|
|---|
| 99 | </pre>
|
|---|
| 100 |
|
|---|
| 101 | <p>The <code>*.exp</code> files are located in the testsuite directories of the GCC
|
|---|
| 102 | source, the most important ones being <code>compile.exp</code>,
|
|---|
| 103 | <code>execute.exp</code>, <code>dg.exp</code> and <code>old-deja.exp</code>.
|
|---|
| 104 | To get a list of the possible <code>*.exp</code> files, pipe the
|
|---|
| 105 | output of <code>make check</code> into a file and look at the
|
|---|
| 106 | <code>Running ... .exp</code> lines.
|
|---|
| 107 |
|
|---|
| 108 | <p>The Java runtime tests can be executed via <code>make check</code>
|
|---|
| 109 | in the <code></code><var>target</var><code>/libjava/testsuite</code> directory in
|
|---|
| 110 | the build tree.
|
|---|
| 111 |
|
|---|
| 112 | <h3 class="section"><a name="TOC1"></a>Additional testing for Java Class Libraries</h3>
|
|---|
| 113 |
|
|---|
| 114 | <p>The <a href="http://sources.redhat.com/mauve/">Mauve Project</a> provides
|
|---|
| 115 | a suite of tests for the Java Class Libraries. This suite can be run
|
|---|
| 116 | as part of libgcj testing by placing the Mauve tree within the libjava
|
|---|
| 117 | testsuite at <code>libjava/testsuite/libjava.mauve/mauve</code>, or by
|
|---|
| 118 | specifying the location of that tree when invoking <code>make</code>, as in
|
|---|
| 119 | <code>make MAUVEDIR=~/mauve check</code>.
|
|---|
| 120 |
|
|---|
| 121 | <a href="http://www-124.ibm.com/developerworks/oss/cvs/jikes/~checkout~/jacks/jacks.html">Jacks</a>
|
|---|
| 122 | is a free test suite that tests Java compiler front ends. This suite
|
|---|
| 123 | can be run as part of libgcj testing by placing the Jacks tree within
|
|---|
| 124 | the libjava testsuite at <code>libjava/testsuite/libjava.jacks/jacks</code>.
|
|---|
| 125 |
|
|---|
| 126 | <h3 class="section"><a name="TOC2"></a>How to interpret test results</h3>
|
|---|
| 127 |
|
|---|
| 128 | <p>The result of running the testsuite are various <code>*.sum</code> and <code>*.log</code>
|
|---|
| 129 | files in the testsuite subdirectories. The <code>*.log</code> files contain a
|
|---|
| 130 | detailed log of the compiler invocations and the corresponding
|
|---|
| 131 | results, the <code>*.sum</code> files summarize the results. These summaries
|
|---|
| 132 | contain status codes for all tests:
|
|---|
| 133 |
|
|---|
| 134 | <ul>
|
|---|
| 135 | <li>PASS: the test passed as expected
|
|---|
| 136 | <li>XPASS: the test unexpectedly passed
|
|---|
| 137 | <li>FAIL: the test unexpectedly failed
|
|---|
| 138 | <li>XFAIL: the test failed as expected
|
|---|
| 139 | <li>UNSUPPORTED: the test is not supported on this platform
|
|---|
| 140 | <li>ERROR: the testsuite detected an error
|
|---|
| 141 | <li>WARNING: the testsuite detected a possible problem
|
|---|
| 142 | </ul>
|
|---|
| 143 |
|
|---|
| 144 | <p>It is normal for some tests to report unexpected failures. At the
|
|---|
| 145 | current time our testing harness does not allow fine grained control
|
|---|
| 146 | over whether or not a test is expected to fail. We expect to fix this
|
|---|
| 147 | problem in future releases.
|
|---|
| 148 |
|
|---|
| 149 | <h3 class="section"><a name="TOC3"></a>Submitting test results</h3>
|
|---|
| 150 |
|
|---|
| 151 | <p>If you want to report the results to the GCC project, use the
|
|---|
| 152 | <code>contrib/test_summary</code> shell script. Start it in the <var>objdir</var> with
|
|---|
| 153 |
|
|---|
| 154 | <pre class="example"> <var>srcdir</var>/contrib/test_summary -p your_commentary.txt \
|
|---|
| 155 | -m gcc-testresults@gcc.gnu.org |sh
|
|---|
| 156 | </pre>
|
|---|
| 157 |
|
|---|
| 158 | <p>This script uses the <code>Mail</code> program to send the results, so
|
|---|
| 159 | make sure it is in your <code>PATH</code>. The file <code>your_commentary.txt</code> is
|
|---|
| 160 | prepended to the testsuite summary and should contain any special
|
|---|
| 161 | remarks you have on your results or your build environment. Please
|
|---|
| 162 | do not edit the testsuite result block or the subject line, as these
|
|---|
| 163 | messages may be automatically processed.
|
|---|
| 164 |
|
|---|
| 165 | <hr />
|
|---|
| 166 | <p>
|
|---|
| 167 | <a href="./index.html">Return to the GCC Installation page</a>
|
|---|
| 168 |
|
|---|
| 169 | </body></html>
|
|---|
| 170 |
|
|---|
