[3/3] configure: Allow detecting and using LLVM lld-link as linker for windows

Message ID 1469390253-2666-3-git-send-email-martin@martin.st
State Committed
Headers show

Commit Message

Martin Storsjö July 24, 2016, 7:57 p.m.
---
For ARM, this linker currently produces broken binaries when
the binaries end up larger than around 6 MB (i.e. fate tests for
single features work, but loading the linked avconv.exe fails).
---
 configure | 12 ++++++++++++
 1 file changed, 12 insertions(+)

Comments

Diego Biurrun July 25, 2016, 9:13 a.m. | #1
On Sun, Jul 24, 2016 at 10:57:33PM +0300, Martin Storsjö wrote:
> ---
> For ARM, this linker currently produces broken binaries when
> the binaries end up larger than around 6 MB (i.e. fate tests for
> single features work, but loading the linked avconv.exe fails).

Did you try a bug report?

> ---
>  configure | 12 ++++++++++++
>  1 file changed, 12 insertions(+)

OK

> --- a/configure
> +++ b/configure
> @@ -3268,6 +3268,18 @@ probe_cc(){
>          _cflags_speed='-O4'
>          _cflags_size='-O2'
>          _flags_filter=cparser_flags
> +    elif $_cc -? 2>/dev/null | grep -q 'LLVM.*Linker'; then
> +        # lld can emulate multiple different linkers; in ms link.exe mode,
> +        # the -? parameter gives the help output which contains an identifyable
> +        # string, while it gives an error in other modes.
> +        _type=lld-link
> +        # The link.exe mode doesn't have a switch for getting the version,
> +        # but we can force it back to gnu mode and get the version from there.
> +        _ident=$($_cc -flavor gnu --version)
> +        _ld_o='-out:$@'
> +        _flags_filter=msvc_flags
> +        _ld_lib='lib%.a'
> +        _ld_path='-libpath:'
>      fi

Please move this entry up above cparser so that the three msvc-like
entries which share some of the values they set are grouped together.

Diego
Martin Storsjö July 25, 2016, 9:15 a.m. | #2
On Mon, 25 Jul 2016, Diego Biurrun wrote:

> On Sun, Jul 24, 2016 at 10:57:33PM +0300, Martin Storsjö wrote:
>> ---
>> For ARM, this linker currently produces broken binaries when
>> the binaries end up larger than around 6 MB (i.e. fate tests for
>> single features work, but loading the linked avconv.exe fails).
>
> Did you try a bug report?

Not yet, but I certainly will do.

>> ---
>>  configure | 12 ++++++++++++
>>  1 file changed, 12 insertions(+)
>
> OK
>
>> --- a/configure
>> +++ b/configure
>> @@ -3268,6 +3268,18 @@ probe_cc(){
>>          _cflags_speed='-O4'
>>          _cflags_size='-O2'
>>          _flags_filter=cparser_flags
>> +    elif $_cc -? 2>/dev/null | grep -q 'LLVM.*Linker'; then
>> +        # lld can emulate multiple different linkers; in ms link.exe mode,
>> +        # the -? parameter gives the help output which contains an identifyable
>> +        # string, while it gives an error in other modes.
>> +        _type=lld-link
>> +        # The link.exe mode doesn't have a switch for getting the version,
>> +        # but we can force it back to gnu mode and get the version from there.
>> +        _ident=$($_cc -flavor gnu --version)
>> +        _ld_o='-out:$@'
>> +        _flags_filter=msvc_flags
>> +        _ld_lib='lib%.a'
>> +        _ld_path='-libpath:'
>>      fi
>
> Please move this entry up above cparser so that the three msvc-like
> entries which share some of the values they set are grouped together.

Sure, I can do that.

// Martin
Martin Storsjö Aug. 9, 2016, 11:26 a.m. | #3
On Sun, 24 Jul 2016, Martin Storsjö wrote:

> ---
> For ARM, this linker currently produces broken binaries when
> the binaries end up larger than around 6 MB (i.e. fate tests for
> single features work, but loading the linked avconv.exe fails).

FWIW, this has been fixed now, so with the latest SVN trunk versions of 
clang/llvm/lld, I can now build win/arm binaries that work, without ever 
invoking wine on the build host.

https://fate.libav.org/armv7-win32-clang-4.0/20160809112350

// Martin
Luca Barbato Aug. 9, 2016, 1:22 p.m. | #4
On 09/08/16 13:26, Martin Storsjö wrote:
> On Sun, 24 Jul 2016, Martin Storsjö wrote:
> 
>> ---
>> For ARM, this linker currently produces broken binaries when
>> the binaries end up larger than around 6 MB (i.e. fate tests for
>> single features work, but loading the linked avconv.exe fails).
> 
> FWIW, this has been fixed now, so with the latest SVN trunk versions of
> clang/llvm/lld, I can now build win/arm binaries that work, without ever
> invoking wine on the build host.
> 
> https://fate.libav.org/armv7-win32-clang-4.0/20160809112350
> 

Could you please teach me how :) ?

lu
Martin Storsjö Aug. 9, 2016, 8:24 p.m. | #5
On Tue, 9 Aug 2016, Luca Barbato wrote:

> On 09/08/16 13:26, Martin Storsjö wrote:
>> On Sun, 24 Jul 2016, Martin Storsjö wrote:
>>
>>> ---
>>> For ARM, this linker currently produces broken binaries when
>>> the binaries end up larger than around 6 MB (i.e. fate tests for
>>> single features work, but loading the linked avconv.exe fails).
>>
>> FWIW, this has been fixed now, so with the latest SVN trunk versions of
>> clang/llvm/lld, I can now build win/arm binaries that work, without ever
>> invoking wine on the build host.
>>
>> https://fate.libav.org/armv7-win32-clang-4.0/20160809112350
>>
>
> Could you please teach me how :) ?

It should be something like this:

- Build a recent trunk version (the last fixes were merged a few days 
ago) - I'm using these versions:
  - llvm 380a420a69bfe1198143f0cef0c0c30f51e7d5b3
  - clang 20ac821dee568d0293fc7efee414aa1b961b5142
  - lld 003757eef4ea1d5c1769e4077dd11a3d3db03034
- Have MSVC and a windows SDK available; I've tested it with 2013 and 2015 
(2012 also works but gives a lot of warnings, iirc)
- Set the env variables INCLUDE and LIB to point to the directories (just 
like for normal MSVC, but with unix style paths instead of windows style)
- Build some files from compiler-rt (almost any version should be fine 
here) and package them up into a static lib. I've built them with a make 
snippet like this:

vpath %.c path/to/llvm/projects/compiler-rt/lib/builtins

%.o: %.c
         clang -target armv7-win32-msvc -c $< -o $@ -DWINAPI_FAMILY=WINAPI_FAMILY_PHONE_APP -O3

libclang-arm-win.lib: moddi3.o umoddi3.o udivmoddi4.o divsi3.o udivsi3.o umodsi3.o udivdi3.o modsi3.o divdi3.o
         llvm-ar rc $@ $+

Put the built library somewhere in the lib path somewhere.

(This should probably be upstreamed somewhere - I haven't gotten to that 
yet. But I've also heard that clang in msvc isn't supposed to need these, 
so it might also be going away, but for now, they're needed at least.)

- Configure libav with something like this:
  --arch=arm --cpu=armv7-a --target-os=win32
  --extra-cflags='-DWINAPI_FAMILY=WINAPI_FAMILY_PHONE_APP'
  --enable-cross-compile --target-exec='ssh device wine'
  --extra-ldflags='msvcrt.lib oldnames.lib libclang-arm-win.lib
   -nodefaultlib:kernel32.lib -nodefaultlib:ole32.lib WindowsPhoneCore.lib'
  --cc='clang -target armv7-win32-msvc' --ar=llvm-ar --nm=llvm-nm
  --as='clang -target armv7-win32-gnu' --ld=lld-link

Normally, the "msvcrt.lib oldnames.lib" parts would be injected 
automatically (if using normal MSVC), when using -MD. I think clang does 
the same if you use the clang-cl driver frontend, but I haven't used that, 
just passing them manually to the linker instead. The last extra ldflags, 
"-nodefaultlib:kernel32.lib -nodefaultlib:ole32.lib WindowsPhoneCore.lib" 
are only necessary if building with the MSVC 2013 phone SDK, not necessary 
on 2012 phone, and must be left out with 2015.

// Martin
Diego Biurrun Aug. 10, 2016, 6:34 a.m. | #6
On Tue, Aug 09, 2016 at 11:24:09PM +0300, Martin Storsjö wrote:
> 
> - Configure libav with something like this:
>   --arch=arm --cpu=armv7-a --target-os=win32
>   --extra-cflags='-DWINAPI_FAMILY=WINAPI_FAMILY_PHONE_APP'
>   --enable-cross-compile --target-exec='ssh device wine'
>   --extra-ldflags='msvcrt.lib oldnames.lib libclang-arm-win.lib
>    -nodefaultlib:kernel32.lib -nodefaultlib:ole32.lib WindowsPhoneCore.lib'
>   --cc='clang -target armv7-win32-msvc' --ar=llvm-ar --nm=llvm-nm
>   --as='clang -target armv7-win32-gnu' --ld=lld-link

Not --extra-libs instead of --extra-ldflags?

Diego
Martin Storsjö Aug. 10, 2016, 10:05 a.m. | #7
On Wed, 10 Aug 2016, Diego Biurrun wrote:

> On Tue, Aug 09, 2016 at 11:24:09PM +0300, Martin Storsjö wrote:
>> 
>> - Configure libav with something like this:
>>   --arch=arm --cpu=armv7-a --target-os=win32
>>   --extra-cflags='-DWINAPI_FAMILY=WINAPI_FAMILY_PHONE_APP'
>>   --enable-cross-compile --target-exec='ssh device wine'
>>   --extra-ldflags='msvcrt.lib oldnames.lib libclang-arm-win.lib
>>    -nodefaultlib:kernel32.lib -nodefaultlib:ole32.lib WindowsPhoneCore.lib'
>>   --cc='clang -target armv7-win32-msvc' --ar=llvm-ar --nm=llvm-nm
>>   --as='clang -target armv7-win32-gnu' --ld=lld-link
>
> Not --extra-libs instead of --extra-ldflags?

I guess that would work as well. I actually don't remember the details on 
in which way they differ.

// Martin
Diego Biurrun Aug. 10, 2016, 10:11 a.m. | #8
On Wed, Aug 10, 2016 at 01:05:41PM +0300, Martin Storsjö wrote:
> On Wed, 10 Aug 2016, Diego Biurrun wrote:
> 
> > On Tue, Aug 09, 2016 at 11:24:09PM +0300, Martin Storsjö wrote:
> >> 
> >> - Configure libav with something like this:
> >>   --arch=arm --cpu=armv7-a --target-os=win32
> >>   --extra-cflags='-DWINAPI_FAMILY=WINAPI_FAMILY_PHONE_APP'
> >>   --enable-cross-compile --target-exec='ssh device wine'
> >>   --extra-ldflags='msvcrt.lib oldnames.lib libclang-arm-win.lib
> >>    -nodefaultlib:kernel32.lib -nodefaultlib:ole32.lib WindowsPhoneCore.lib'
> >>   --cc='clang -target armv7-win32-msvc' --ar=llvm-ar --nm=llvm-nm
> >>   --as='clang -target armv7-win32-gnu' --ld=lld-link
> >
> > Not --extra-libs instead of --extra-ldflags?
> 
> I guess that would work as well. I actually don't remember the details on 
> in which way they differ.

--extra-ldflags should be for -L flags and similar flags that modify
the behavior of the linker while --extra-libs is for actual libraries
that should be added to the linker command line. The latter should be
more correct than the latter. If it doesn't work, it's a bug somewhere.
That it works with --extra-ldflags may partially be good luck.

Diego
Martin Storsjö Nov. 1, 2016, 7:31 a.m. | #9
On Tue, 9 Aug 2016, Martin Storsjö wrote:

> It should be something like this:
>
> - Build a recent trunk version (the last fixes were merged a few days ago) - 
> I'm using these versions:
> - llvm 380a420a69bfe1198143f0cef0c0c30f51e7d5b3
> - clang 20ac821dee568d0293fc7efee414aa1b961b5142
> - lld 003757eef4ea1d5c1769e4077dd11a3d3db03034

For the record, the current versions of today (SVN rev 285649) also work 
well:
- llvm f185e91e2bc8a41efad3da70d7a40fb34e2ea3f5
- clang e30072cab95f2538755e30383349e91e20750eca
- lld f5fb6495913ce3c38a18207c568beb6d40f653c4

This version gets rid of the need to manually build compiler-rt and add it 
via --extra-libs or --extra-ldflags; now it relies only on what the msvc 
libs provide.

I've updated my fate instance for clang/llvm 4.0 to use these versions 
now.

// Martin
Luca Barbato Nov. 1, 2016, 10:33 a.m. | #10
On 01/11/2016 08:31, Martin Storsjö wrote:
> On Tue, 9 Aug 2016, Martin Storsjö wrote:
> 
>> It should be something like this:
>>
>> - Build a recent trunk version (the last fixes were merged a few days
>> ago) - I'm using these versions:
>> - llvm 380a420a69bfe1198143f0cef0c0c30f51e7d5b3
>> - clang 20ac821dee568d0293fc7efee414aa1b961b5142
>> - lld 003757eef4ea1d5c1769e4077dd11a3d3db03034
> 
> For the record, the current versions of today (SVN rev 285649) also work
> well:
> - llvm f185e91e2bc8a41efad3da70d7a40fb34e2ea3f5
> - clang e30072cab95f2538755e30383349e91e20750eca
> - lld f5fb6495913ce3c38a18207c568beb6d40f653c4
> 
> This version gets rid of the need to manually build compiler-rt and add
> it via --extra-libs or --extra-ldflags; now it relies only on what the
> msvc libs provide.
> 
> I've updated my fate instance for clang/llvm 4.0 to use these versions now.
> 

Sounds nice.

Patch

diff --git a/configure b/configure
index 78951d4..face48c 100755
--- a/configure
+++ b/configure
@@ -3268,6 +3268,18 @@  probe_cc(){
         _cflags_speed='-O4'
         _cflags_size='-O2'
         _flags_filter=cparser_flags
+    elif $_cc -? 2>/dev/null | grep -q 'LLVM.*Linker'; then
+        # lld can emulate multiple different linkers; in ms link.exe mode,
+        # the -? parameter gives the help output which contains an identifyable
+        # string, while it gives an error in other modes.
+        _type=lld-link
+        # The link.exe mode doesn't have a switch for getting the version,
+        # but we can force it back to gnu mode and get the version from there.
+        _ident=$($_cc -flavor gnu --version)
+        _ld_o='-out:$@'
+        _flags_filter=msvc_flags
+        _ld_lib='lib%.a'
+        _ld_path='-libpath:'
     fi
 
     eval ${pfx}_type=\$_type