running clang compiler test suite from within macports infrastructure?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

running clang compiler test suite from within macports infrastructure?

Kenneth F. Cunningham
I suspect only Jeremy or LarryV could answer this, but is it possible to run the llvm / clang compiler test suite from within the macports infrastructure?

The clang build on macports is very heavily modified with patches and configuration settings - it seems overwhelming to try to duplicate that as a separate download in the usual way this might have been imagined.

I can turn the llvm & clang tests on with the portfile configure switches easily enough -- and build the port without destrooting or installing it, I guess, to leave everything in place -- but then I suspect I'd be poking around for days trying to figure out how to make this work reliably.

Given how many years clang / llvm has been part of macports, I suspect this has been sorted out -- has anyone already already been through this exercise?

I have clang-3.8 running really quite reliably on ppc now - only exceptions are not working, and not many ports use exceptions. But I'd feel much better if I could run the test suite and see where we really are with it.

Ken
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: running clang compiler test suite from within macports infrastructure?

Jack Howarth
Ken,
     Note that you have to explicitly configure the cmake build to compile the test suite by passing...

-DLIBCXX_INCLUDE_TESTS=ON


(if you want to test the matching libcxx anyway). Also in fink, we also pass -DLLVM_CONFIG_PATH=%b/../build/last/bin/llvm-config to LIBCXX_CMAKE_OPTIONS placeholder in our shell scripts. If you look at the files at...


https://sourceforge.net/p/fink/package-submissions/4933/


you will see the current state of our 3-stage bootstrap with stage2/stage3 file comparison. The general process that we use is...


1) build stage1 against the system libcxx and clang compiler

2) build stage1.5 libcxx with the stage1 compiler

3) build the stage2 compiler with the stage1 compiler against the stage1.5 libcxx

4) build the stage2.5 libcxx with the stage2 compiler

5) build the stage3 compiler against the stage2 compiler with the stage2.5 libcxx

6) perform an exhaustive stage2/stage3 object file comparison to detect any non-determinism that has crept in to the compiler.


While a bit difficult to follow and maintain, I really like this build approach as upstream llvm.org seems to be rather lax about doing proper 3-stage bootstraps with stage2/stage3 file comparison ala FSF gcc.


I've run into a couple of instances where upstream trunk had gone non-deterministic and the majority of the llvm developers were completely oblivious to it due to the absence of proper bootstrap testing on the buildbots.

           Jack


On Thu, Jun 29, 2017 at 8:33 PM, Ken Cunningham <[hidden email]> wrote:
I suspect only Jeremy or LarryV could answer this, but is it possible to run the llvm / clang compiler test suite from within the macports infrastructure?

The clang build on macports is very heavily modified with patches and configuration settings - it seems overwhelming to try to duplicate that as a separate download in the usual way this might have been imagined.

I can turn the llvm & clang tests on with the portfile configure switches easily enough -- and build the port without destrooting or installing it, I guess, to leave everything in place -- but then I suspect I'd be poking around for days trying to figure out how to make this work reliably.

Given how many years clang / llvm has been part of macports, I suspect this has been sorted out -- has anyone already already been through this exercise?

I have clang-3.8 running really quite reliably on ppc now - only exceptions are not working, and not many ports use exceptions. But I'd feel much better if I could run the test suite and see where we really are with it.

Ken

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: running clang compiler test suite from within macports infrastructure?

Kenneth F. Cunningham
Thanks, Jack.

There appear to be different ways to test clang. I thought the testing would all happen in during or related to the build, but this does not appear to be exactly so.

At any rate, there is a "test-suite" of 400+ tests for each version of clang, which is one way to test it. It is unrelated to the clang-3.8 build, it appears. Just for fun, there are at least three different ways to run it :>

I went through the steps for one of the ways to test, and here is what worked:


Install lit:

From the source directory of clang-3.8 on macports:
clang-3.8/work/llvm-3.8.1.src/utils/lit
sudo python setup.py install


install and build the test-suite matching your version of clang. For clang381, this is:
svn co http://llvm.org/svn/llvm-project/test-suite/tags/RELEASE_381/final/ test-suite
mkdir test-suite-build
cd test-suite-build
cmake ../test-suite
make

Then run the tests:
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/lit -v -j 1 . -o results.json


Some time later, you will get final results

Ken
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: running clang compiler test suite from within macports infrastructure?

Kenneth F. Cunningham
In reply to this post by Kenneth F. Cunningham
This test suite proved very very helpful!

Clang-3.8 on PPC produced only 13 errors out of 500 tests.

10 of those were on the same issue, 32bit exceptions.

The other three included on a weird math error in fast-fourier-transfroms, and two errors in the logic of handling certain va_args (which I thought we had fixed).

Gives me encouragement to continue on with clang-3.8 on PPC.

Ken




On 2017-06-29, at 5:33 PM, Ken Cunningham wrote:

> I suspect only Jeremy or LarryV could answer this, but is it possible to run the llvm / clang compiler test suite from within the macports infrastructure?
>
> The clang build on macports is very heavily modified with patches and configuration settings - it seems overwhelming to try to duplicate that as a separate download in the usual way this might have been imagined.
>
> I can turn the llvm & clang tests on with the portfile configure switches easily enough -- and build the port without destrooting or installing it, I guess, to leave everything in place -- but then I suspect I'd be poking around for days trying to figure out how to make this work reliably.
>
> Given how many years clang / llvm has been part of macports, I suspect this has been sorted out -- has anyone already already been through this exercise?
>
> I have clang-3.8 running really quite reliably on ppc now - only exceptions are not working, and not many ports use exceptions. But I'd feel much better if I could run the test suite and see where we really are with it.
>
> Ken

Loading...