Build Machine(s) & Java

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Build Machine(s) & Java

Mohamed Issa
When the Bazel port was committed, a number of build jobs failed. A link to each one is listed below.


In all the stdio logs, I saw the following:

--->  Fetching distfiles for bazel
DEBUG: elevating privileges for fetch: euid changed to 0, egid changed to 0.
DEBUG: dropping privileges: euid changed to 502, egid changed to 501.
DEBUG: Executing proc-pre-org.macports.fetch-fetch-0
Error: Failed to fetch bazel: No Java runtime present, requesting install.
2017-07-02 23:43:19.319 javac[76407:707] JLRequestRuntimeInstall: Error calling: CFMessagePortCreateRemote
child process exited abnormally
DEBUG: Error code: NONE
DEBUG: Backtrace: No Java runtime present, requesting install.
2017-07-02 23:43:19.319 javac[76407:707] JLRequestRuntimeInstall: Error calling: CFMessagePortCreateRemote
child process exited abnormally
    while executing
"$pre $targetname"
Error: See /opt/local/var/macports/logs/_opt_bblocal_var_buildworker_ports_build_ports_devel_bazel/bazel/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.
Error: Processing of port bazel failed
Build of 'bazel' failed.
./mpbb/mpbb: error: `install-port' failed to run successfully
program finished with exit code 1

I'm guessing this was triggered by the pre-fetch phase of the port installation because there is a call to "javac" there via the "auto_execok" Tcl command.
Do the build machines have Java installed? As it stands right now, any changes to the Bazel port (or others like it) will cause build failures. So I'm wondering what can be done to avoid that.

Regards,
Mohamed
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Build Machine(s) & Java

Ryan Schmidt-24

On Jul 3, 2017, at 13:29, Mohamed Issa wrote:

Do the build machines have Java installed? As it stands right now, any changes to the Bazel port (or others like it) will cause build failures. So I'm wondering what can be done to avoid that.

I haven't installed any separate Java on the build machines. Only what came with macOS, if any, is there.

There are some open tickets about this, some of which may date back to our previous build infrastructure.

Should I install Java, or a different Java, on them? If so, which one(s)? Because I know Apple distributed some versions of Java, and then there are others distributed by Oracle now.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Build Machine(s) & Java

Chris Jones

Unless I am mis-remembering Apple stopped shipping any java version with
OSX a while back. So unless you manually install something the build
bots will not have any java version available. versions now should be
downloaded direct from Oracle.

https://support.apple.com/en-gb/HT204036

I believe if you just run `java` in a terminal, it will provide a pop up
that wll take you to the correct download location for that OSX release.

On 05/07/17 01:14, Ryan Schmidt wrote:

>
> On Jul 3, 2017, at 13:29, Mohamed Issa wrote:
>
>> Do the build machines have Java installed? As it stands right now, any
>> changes to the Bazel port (or others like it) will cause build
>> failures. So I'm wondering what can be done to avoid that.
>
> I haven't installed any separate Java on the build machines. Only what
> came with macOS, if any, is there.
>
> There are some open tickets about this, some of which may date back to
> our previous build infrastructure.
>
> Should I install Java, or a different Java, on them? If so, which
> one(s)? Because I know Apple distributed some versions of Java, and then
> there are others distributed by Oracle now.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Build Machine(s) & Java

Ryan Schmidt-24

On Jul 5, 2017, at 04:00, Chris Jones wrote:

> On 05/07/17 01:14, Ryan Schmidt wrote:
>> On Jul 3, 2017, at 13:29, Mohamed Issa wrote:
>>> Do the build machines have Java installed? As it stands right now, any changes to the Bazel port (or others like it) will cause build failures. So I'm wondering what can be done to avoid that.
>> I haven't installed any separate Java on the build machines. Only what came with macOS, if any, is there.
>> There are some open tickets about this, some of which may date back to our previous build infrastructure.
>> Should I install Java, or a different Java, on them? If so, which one(s)? Because I know Apple distributed some versions of Java, and then there are others distributed by Oracle now.
>
> Unless I am mis-remembering Apple stopped shipping any java version with OSX a while back. So unless you manually install something the build bots will not have any java version available.

Right. But for at least some of those macOS versions where Apple does not include Java anymore, a legacy version can still be downloaded and installed from Apple's site:

https://support.apple.com/kb/DL1572?viewlocale=en_US&locale=en_US

Should that go on those buildbot workers, or should the Oracle version be used?

What about on those macOS versions where the now-legacy Java was included -- should that be used, or should it be upgraded to the Oracle version?

Or can they both be installed? And if so, should we be doing that?

I seem to recall that there are differences between the Apple and Oracle distributions w.r.t. where things are installed. Is that important? Does our java portgroup work with both versions?

What happens if our buildbot has one Java installed but the user has a different Java installed? Does everything still work if the user gets a binary that was built on our servers?


> versions now should be downloaded direct from Oracle.
>
> https://support.apple.com/en-gb/HT204036
>
> I believe if you just run `java` in a terminal, it will provide a pop up that wll take you to the correct download location for that OSX release.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Build Machine(s) & Java

Chris Jones
Hi,

On 05/07/17 13:38, Ryan Schmidt wrote:

>
> On Jul 5, 2017, at 04:00, Chris Jones wrote:
>
>> On 05/07/17 01:14, Ryan Schmidt wrote:
>>> On Jul 3, 2017, at 13:29, Mohamed Issa wrote:
>>>> Do the build machines have Java installed? As it stands right now, any changes to the Bazel port (or others like it) will cause build failures. So I'm wondering what can be done to avoid that.
>>> I haven't installed any separate Java on the build machines. Only what came with macOS, if any, is there.
>>> There are some open tickets about this, some of which may date back to our previous build infrastructure.
>>> Should I install Java, or a different Java, on them? If so, which one(s)? Because I know Apple distributed some versions of Java, and then there are others distributed by Oracle now.
>>
>> Unless I am mis-remembering Apple stopped shipping any java version with OSX a while back. So unless you manually install something the build bots will not have any java version available.
>
> Right. But for at least some of those macOS versions where Apple does not include Java anymore, a legacy version can still be downloaded and installed from Apple's site:
>
> https://support.apple.com/kb/DL1572?viewlocale=en_US&locale=en_US
>
> Should that go on those buildbot workers, or should the Oracle version be used?
>
> What about on those macOS versions where the now-legacy Java was included -- should that be used, or should it be upgraded to the Oracle version?
>
> Or can they both be installed? And if so, should we be doing that?
>
> I seem to recall that there are differences between the Apple and Oracle distributions w.r.t. where things are installed. Is that important? Does our java portgroup work with both versions?
>
> What happens if our buildbot has one Java installed but the user has a different Java installed? Does everything still work if the user gets a binary that was built on our servers?

All good questions. I don't have answers. Personally I avoid having java
installed at all on my machines (probably should have said this before)
as its a can of worms I'm not keen to open. It wouldn't be completely
unreasonable to me to have a policy not to install java at all on the
buildbots, and for ports that require it (how many are there b.t.w. ?)
flag those ports as ones not to build on the build bots at all, and
require the user to build themselves with whatever java version they prefer.

Chris
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Build Machine(s) & Java

Ryan Schmidt-24

On Jul 5, 2017, at 07:46, Chris Jones wrote:

> On 05/07/17 13:38, Ryan Schmidt wrote:
>> On Jul 5, 2017, at 04:00, Chris Jones wrote:
>>> On 05/07/17 01:14, Ryan Schmidt wrote:
>>>> On Jul 3, 2017, at 13:29, Mohamed Issa wrote:
>>>>> Do the build machines have Java installed? As it stands right now, any changes to the Bazel port (or others like it) will cause build failures. So I'm wondering what can be done to avoid that.
>>>> I haven't installed any separate Java on the build machines. Only what came with macOS, if any, is there.
>>>> There are some open tickets about this, some of which may date back to our previous build infrastructure.
>>>> Should I install Java, or a different Java, on them? If so, which one(s)? Because I know Apple distributed some versions of Java, and then there are others distributed by Oracle now.
>>>
>>> Unless I am mis-remembering Apple stopped shipping any java version with OSX a while back. So unless you manually install something the build bots will not have any java version available.
>> Right. But for at least some of those macOS versions where Apple does not include Java anymore, a legacy version can still be downloaded and installed from Apple's site:
>> https://support.apple.com/kb/DL1572?viewlocale=en_US&locale=en_US
>> Should that go on those buildbot workers, or should the Oracle version be used?
>> What about on those macOS versions where the now-legacy Java was included -- should that be used, or should it be upgraded to the Oracle version?
>> Or can they both be installed? And if so, should we be doing that?
>> I seem to recall that there are differences between the Apple and Oracle distributions w.r.t. where things are installed. Is that important? Does our java portgroup work with both versions?
>> What happens if our buildbot has one Java installed but the user has a different Java installed? Does everything still work if the user gets a binary that was built on our servers?
>
> All good questions. I don't have answers. Personally I avoid having java installed at all on my machines (probably should have said this before) as its a can of worms I'm not keen to open.

Me too, for the most part.

> It wouldn't be completely unreasonable to me to have a policy not to install java at all on the buildbots, and for ports that require it (how many are there b.t.w. ?) flag those ports as ones not to build on the build bots at all, and require the user to build themselves with whatever java version they prefer.

That's basically what currently happens. The buildbots will fail to build those ports, so binaries won't be available, so they'll be built on user machines.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Build Machine(s) & Java

Mohamed Issa
> On Jul 5, 2017, at 07:46, Chris Jones wrote:
>  
> > On 05/07/17 13:38, Ryan Schmidt wrote:
> >> On Jul 5, 2017, at 04:00, Chris Jones wrote:
> >>> On 05/07/17 01:14, Ryan Schmidt wrote:
> >>>> On Jul 3, 2017, at 13:29, Mohamed Issa wrote:
> >>>>> Do the build machines have Java installed? As it stands right now, any changes to the Bazel port (or others like it) will cause build failures. So I'm wondering what can be done to avoid that.
> >>>> I haven't installed any separate Java on the build machines. Only what came with macOS, if any, is there.
> >>>> There are some open tickets about this, some of which may date back to our previous build infrastructure.
> >>>> Should I install Java, or a different Java, on them? If so, which one(s)? Because I know Apple distributed some versions of Java, and then there are others distributed by Oracle now.
> >>>
> >>> Unless I am mis-remembering Apple stopped shipping any java version with OSX a while back. So unless you manually install something the build bots will not have any java version available.
> >> Right. But for at least some of those macOS versions where Apple does not include Java anymore, a legacy version can still be downloaded and installed from Apple's site:
> >> https://support.apple.com/kb/DL1572?viewlocale=en_US&locale=en_US 
> >> Should that go on those buildbot workers, or should the Oracle version be used?
> >> What about on those macOS versions where the now-legacy Java was included -- should that be used, or should it be upgraded to the Oracle version?
> >> Or can they both be installed? And if so, should we be doing that?
> >> I seem to recall that there are differences between the Apple and Oracle distributions w.r.t. where things are installed. Is that important? Does our java portgroup work with both versions?
> >> What happens if our buildbot has one Java installed but the user has a different Java installed? Does everything still work if the user gets a binary that was built on our servers?
> >
> > All good questions. I don't have answers. Personally I avoid having java installed at all on my machines (probably should have said this before) as its a can of worms I'm not keen to open.
>  
> Me too, for the most part.
>  
> > It wouldn't be completely unreasonable to me to have a policy not to install java at all on the buildbots, and for ports that require it (how many are there b.t.w. ?) flag those ports as ones not to build on the build bots at all, and require the user to build themselves with whatever java version they prefer.
>  
> That's basically what currently happens. The buildbots will fail to build those ports, so binaries won't be available, so they'll be built on user machines.

The build jobs are still marked as failures though. To me it would be more accurate to categorize them as "skipped".

I can appreciate the wisdom of not manually installing particular Java implementations on the build bots. Perhaps the cleanest solution would be to include a file dependency in all Java related ports that need it.
If a manually installed version of Java (Oracle, Apple legacy, etc.) isn't detected, then a default open source Java port (i.e. OpenJDK) is installed as a dependency. That way the build machines act like user machines without Java installed and the job proceeds normally.
Unfortunately, I don't see an OpenJDK (or equivalent) port currently in the repository. So it seems there would be significant amount of work required to create such a port.

Loading...