source: trunk/src/gcc/INSTALL/test.html@ 1042

Last change on this file since 1042 was 2, checked in by bird, 23 years ago

Initial revision
  • Property cvs2svn:cvs-rev set to 1.1
  • Property svn:eol-style set to native
  • Property svn:executable set to *
File size: 6.2 KB
Line 
1<html lang="en"><head>
2<title>Installing GCC: Testing</title>
3<meta http-equiv="Content-Type" content="text/html">
4<meta name=description content="Installing GCC: Testing">
5<meta name=generator content="makeinfo 4.0">
6<link href="http://texinfo.org/" rel=generator-home>
7</head><body>
8
9<p>B<p>efore you install GCC, we encourage you to run the testsuites and to
10compare your results with results from a similar configuration that have
11been submitted to the
12<a href="http://gcc.gnu.org/ml/gcc-testresults/">gcc-testresults mailing list</a>.
13This step is optional and may require you to download additional software,
14but it can give you confidence in your new GCC installation or point out
15problems before you install and start using your new GCC.
16
17<p>First, you must have <a href="download.html">downloaded the testsuites</a>.
18These are part of the full distribution, but if you downloaded the
19"core" compiler plus any front ends, you must download the testsuites
20separately.
21
22<p>Second, you must have the testing tools installed. This includes
23a <a href="http://www.gnu.org/software/dejagnu/">current version of DejaGnu</a>;
24dejagnu 1.3 is not sufficient.
25It also includes Tcl and Expect; the DejaGnu site has links to these.
26
27<p>Now you may need specific preparations:
28
29<ul>
30
31<li>The following environment variables may need to be set appropriately, as in
32the following example (which assumes that DejaGnu has been installed
33under <code>/usr/local</code>):
34
35<pre> TCL_LIBRARY = /usr/local/share/tcl8.0
36 DEJAGNULIBS = /usr/local/share/dejagnu
37</pre>
38
39<p>On systems such as Cygwin, these paths are required to be actual
40paths, not mounts or links; presumably this is due to some lack of
41portability in the DejaGnu code.
42
43<p>If the directories where <code>runtest</code> and <code>expect</code> were
44installed are in the <code>PATH</code>, it should not be necessary to set these
45environment variables.
46
47</ul>
48
49<p>Finally, you can run the testsuite (which may take a long time):
50<pre> cd <var>objdir</var>; make -k check
51</pre>
52
53<p>The testing process will try to test as many components in the GCC
54distribution as possible, including the C, C++, Objective-C and Fortran
55compilers as well as the C++ and Java runtime libraries.
56
57<p>While running the testsuite, DejaGnu might emit messages resembling
58<code>WARNING: Couldn't find the global config file.</code> or
59<code>WARNING: Couldn't find tool init file</code>.
60These messages are harmless and do not affect the validity of the tests.
61
62<h2><a name="TOC0">How can I run the test suite on selected tests?</h2>
63
64<p>As a first possibility to cut down the number of tests that are run it is
65possible to use <code>make check-gcc</code> or <code>make check-g++</code>
66in the <code>gcc</code> subdirectory of the object directory. To further cut down the
67tests the following is possible:
68
69<pre> make check-gcc RUNTESTFLAGS="execute.exp <var>other-options</var>"
70</pre>
71
72<p>This will run all <code>gcc</code> execute tests in the testsuite.
73
74<pre> make check-g++ RUNTESTFLAGS="old-deja.exp=9805* <var>other-options</var>"
75</pre>
76
77<p>This will run the <code>g++</code> "old-deja" tests in the testsuite where the filename
78matches <code>9805*</code>.
79
80<p>The <code>*.exp</code> files are located in the testsuite directories of the GCC
81source, the most important ones being <code>compile.exp</code>,
82<code>execute.exp</code>, <code>dg.exp</code> and <code>old-deja.exp</code>.
83To get a list of the possible <code>*.exp</code> files, pipe the
84output of <code>make check</code> into a file and look at the
85<code>Running <small>...</small> .exp</code> lines.
86
87<p>To run only the tests for a library, run <code>make check</code> from the
88the library's testsuite in a subdirectory of the object directory:
89<code>libstdc++-v3/testsuite</code> or <code>libcgj/testsuite</code>.
90
91<h2><a name="TOC1">Additional testing for Java Class Libraries</h2>
92
93<p>The <a href="http://sources.redhat.com/mauve/">Mauve Project</a> provides
94a suite of tests for the Java Class Libraries. This suite can be run
95as part of libgcj testing by specifying the location of the Mauve tree
96when invoking <code>make</code>, as in <code>make MAUVEDIR=~/mauve check</code>.
97
98<h2><a name="TOC2">How to interpret test results</h2>
99
100<p>After the testsuite has run you'll find various <code>*.sum</code> and <code>*.log</code>
101files in the testsuite subdirectories. The <code>*.log</code> files contain a
102detailed log of the compiler invocations and the corresponding
103results, the <code>*.sum</code> files summarize the results. These summaries list
104all the tests that have been run with a corresponding status code:
105
106<ul>
107<li>PASS: the test passed as expected
108<li>XPASS: the test unexpectedly passed
109<li>FAIL: the test unexpectedly failed
110<li>XFAIL: the test failed as expected
111<li>UNSUPPORTED: the test is not supported on this platform
112<li>ERROR: the testsuite detected an error
113<li>WARNING: the testsuite detected a possible problem
114</ul>
115
116<p>It is normal for some tests to report unexpected failures. At the
117current time our testing harness does not allow fine grained control
118over whether or not a test is expected to fail. We expect to fix this
119problem in future releases.
120
121<h2><a name="TOC3">Submitting test results</h2>
122
123<p>If you want to report the results to the GCC project, use the
124<code>contrib/test_summary</code> shell script. Start it in the <var>objdir</var> with
125
126<pre> <var>srcdir</var>/contrib/test_summary -p your_commentary.txt \
127 -m gcc-testresults@gcc.gnu.org |sh
128</pre>
129
130<p>This script uses the <code>Mail</code> program to send the results, so
131make sure it is in your <code>PATH</code>. The file <code>your_commentary.txt</code> is
132prepended to the testsuite summary and should contain any special
133remarks you have on your results or your build environment. Please
134do not edit the testsuite result block or the subject line, as these
135messages are automatically parsed and presented at the
136<a href="http://gcc.gnu.org/testresults/">GCC testresults</a> web
137page. Here you can also gather information on how specific tests
138behave on different platforms and compare them with your results. A
139few failing testcases are possible even on released versions and you
140should look here first if you think your results are unreasonable.
141
142<hr />
143<p>
144<a href="./index.html">Return to the GCC Installation page</a>
145
146</body></html>
147
Note: See TracBrowser for help on using the repository browser.