[MacPorts] #56300: Travis build failure of xorg-libXau

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

[MacPorts] #56300: Travis build failure of xorg-libXau

MacPorts
#56300: Travis build failure of xorg-libXau
----------------------------+---------------------
 Reporter:  ryandesign      |      Owner:  admin@…
     Type:  defect          |     Status:  new
 Priority:  Normal          |  Milestone:
Component:  server/hosting  |    Version:
 Keywords:                  |       Port:
----------------------------+---------------------
 In the attached log, why is Travis trying to build xorg-libXau from
 source? It is distributable and an archive is available.

 Why is it skipping all of the phases before the configure phase?

 {{{
 DEBUG: Skipping completed org.macports.archivefetch (xorg-libXau)
 DEBUG: Privilege de-escalation not attempted as not running as root.
 DEBUG: Skipping completed org.macports.fetch (xorg-libXau)
 DEBUG: Privilege de-escalation not attempted as not running as root.
 DEBUG: Skipping completed org.macports.checksum (xorg-libXau)
 DEBUG: Privilege de-escalation not attempted as not running as root.
 DEBUG: Skipping completed org.macports.extract (xorg-libXau)
 DEBUG: Privilege de-escalation not attempted as not running as root.
 DEBUG: Skipping completed org.macports.patch (xorg-libXau)
 DEBUG: Privilege de-escalation not attempted as not running as root.
 }}}

 Travis is supposed to build each port in a clean environment. None of the
 phases of any port should be marked as completed at that point.

 Why, having decided to build xorg-libXau from source for whatever reason,
 is xorg-libXau's dependency pkg-config not present and active, which
 results in its configure script failing?

--
Ticket URL: <https://trac.macports.org/ticket/56300>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #56300: Travis build failure of xorg-libXau

MacPorts
#56300: Travis build failure of xorg-libXau
-----------------------------+---------------------
  Reporter:  ryandesign      |      Owner:  admin@…
      Type:  defect          |     Status:  new
  Priority:  Normal          |  Milestone:
 Component:  server/hosting  |    Version:
Resolution:                  |   Keywords:
      Port:                  |
-----------------------------+---------------------
Changes (by ryandesign):

 * Attachment "07aabf1d09a4.txt" added.


--
Ticket URL: <https://trac.macports.org/ticket/56300>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #56300: Travis build failure of xorg-libXau

MacPorts
In reply to this post by MacPorts
#56300: Travis build failure of xorg-libXau
-----------------------------+---------------------
  Reporter:  ryandesign      |      Owner:  admin@…
      Type:  defect          |     Status:  new
  Priority:  Normal          |  Milestone:
 Component:  server/hosting  |    Version:
Resolution:                  |   Keywords:
      Port:                  |
-----------------------------+---------------------

Comment (by raimue):

 This was not the first build failure for xorg-libXau, as the port failed
 to fetch before. But the log was also lost, probably both caused by a
 temporary network problem: https://travis-ci.org/macports/macports-
 ports/jobs/365900385#L945

 I noticed the `runner` is not using `mpbb list-subports`, but
 [https://github.com/macports/mpbot-
 github/blob/c68ab523c6c357239a48734aba8498a762e755ef/ci/portutil.go#L18
 some other method]. Therefore the list of ports to build is not sorted in
 dependency order. I am not sure if this could be the cause, but it should
 be fixed anyway.

--
Ticket URL: <https://trac.macports.org/ticket/56300#comment:1>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #56300: Travis build failure of xorg-libXau

MacPorts
In reply to this post by MacPorts
#56300: Travis build failure of xorg-libXau
-----------------------------+---------------------
  Reporter:  ryandesign      |      Owner:  admin@…
      Type:  defect          |     Status:  new
  Priority:  Normal          |  Milestone:
 Component:  server/hosting  |    Version:
Resolution:                  |   Keywords:
      Port:                  |
-----------------------------+---------------------

Comment (by jmroot):

 Backing up a step, if this is supposed to be a clean environment, how are
 a number of the dependencies already installed?

--
Ticket URL: <https://trac.macports.org/ticket/56300#comment:2>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #56300: Travis build failure of xorg-libXau

MacPorts
In reply to this post by MacPorts
#56300: Travis build failure of xorg-libXau
-----------------------------+---------------------
  Reporter:  ryandesign      |      Owner:  admin@…
      Type:  defect          |     Status:  new
  Priority:  Normal          |  Milestone:
 Component:  server/hosting  |    Version:
Resolution:                  |   Keywords:
      Port:                  |
-----------------------------+---------------------

Comment (by raimue):

 See the link to the Travis build for the whole log of port install
 operations. After each subport, its build log is uploaded to
 paste.macports.org. The attached log is only the build log for one of the
 ports (rb-gstreamer), but other ports were installed before.

--
Ticket URL: <https://trac.macports.org/ticket/56300#comment:3>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #56300: Travis build failure of xorg-libXau

MacPorts
In reply to this post by MacPorts
#56300: Travis build failure of xorg-libXau
-----------------------------+---------------------
  Reporter:  ryandesign      |      Owner:  admin@…
      Type:  defect          |     Status:  new
  Priority:  Normal          |  Milestone:
 Component:  server/hosting  |    Version:
Resolution:                  |   Keywords:
      Port:                  |
-----------------------------+---------------------

Comment (by mojca):

 See also #56110. We first need a minor change in `list-subports` to let
 Travis use it in a proper way.

--
Ticket URL: <https://trac.macports.org/ticket/56300#comment:4>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #56300: Travis build failure of xorg-libXau

MacPorts
In reply to this post by MacPorts
#56300: Travis build failure of xorg-libXau
-----------------------------+---------------------
  Reporter:  ryandesign      |      Owner:  admin@…
      Type:  defect          |     Status:  new
  Priority:  Normal          |  Milestone:
 Component:  server/hosting  |    Version:
Resolution:                  |   Keywords:
      Port:                  |
-----------------------------+---------------------

Comment (by ryandesign):

 Same problem with libidn2. Why is it trying to build libidn2 from source,
 when it is distributable and an archive exists? Why is the program
 `autoreconf` not found, when libidn2 declares a dependency on the autoconf
 port?

--
Ticket URL: <https://trac.macports.org/ticket/56300#comment:5>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #56300: Travis build failure of xorg-libXau

MacPorts
In reply to this post by MacPorts
#56300: Travis build failure of xorg-libXau
-----------------------------+---------------------
  Reporter:  ryandesign      |      Owner:  admin@…
      Type:  defect          |     Status:  new
  Priority:  Normal          |  Milestone:
 Component:  server/hosting  |    Version:
Resolution:                  |   Keywords:
      Port:                  |
-----------------------------+---------------------
Changes (by ryandesign):

 * Attachment "5014375de250.txt.bz2" added.

 libidn2 failure

--
Ticket URL: <https://trac.macports.org/ticket/56300>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #56300: Travis build failure of xorg-libXau

MacPorts
In reply to this post by MacPorts
#56300: Travis build failure of xorg-libXau
-----------------------------+---------------------
  Reporter:  ryandesign      |      Owner:  admin@…
      Type:  defect          |     Status:  new
  Priority:  Normal          |  Milestone:
 Component:  server/hosting  |    Version:
Resolution:                  |   Keywords:
      Port:                  |
-----------------------------+---------------------

Comment (by ryandesign):

 Of the 11 builds attempted for [https://github.com/macports/macports-
 ports/pull/1668#issuecomment-384523824 this PR], 7 failed, 6 because of
 this problem.

--
Ticket URL: <https://trac.macports.org/ticket/56300#comment:6>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #56300: Travis build failure of xorg-libXau

MacPorts
In reply to this post by MacPorts
#56300: Travis build failure of xorg-libXau
-----------------------------+---------------------
  Reporter:  ryandesign      |      Owner:  admin@…
      Type:  defect          |     Status:  new
  Priority:  Normal          |  Milestone:
 Component:  server/hosting  |    Version:
Resolution:                  |   Keywords:
      Port:                  |
-----------------------------+---------------------

Comment (by raimue):

 Unfortunately, the log of the first try of installing texinfo was once
 again [https://travis-ci.org/macports/macports-ports/jobs/371382427#L530
 lost]. Could there be a correlation that the network outage causes it to
 fail like this later? I assume the port would be left in an unclean
 state...

 @l2dy\\
 If uploading the log file fails due to a network error, could we just back
 off for a bit and then try again?

--
Ticket URL: <https://trac.macports.org/ticket/56300#comment:7>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #56300: Travis build failure of xorg-libXau

MacPorts
In reply to this post by MacPorts
#56300: Travis build failure of xorg-libXau
-----------------------------+---------------------
  Reporter:  ryandesign      |      Owner:  admin@…
      Type:  defect          |     Status:  new
  Priority:  Normal          |  Milestone:
 Component:  server/hosting  |    Version:
Resolution:                  |   Keywords:
      Port:                  |
-----------------------------+---------------------

Comment (by l2dy):

 Replying to [comment:7 raimue]:
 > @l2dy\\
 > If uploading the log file fails due to a network error, could we just
 back off for a bit and then try again?

 Then we risk build timeout. How many times to retry? Should we specify a
 time limit for the HTTP requests?

--
Ticket URL: <https://trac.macports.org/ticket/56300#comment:8>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #56300: Travis build failure of xorg-libXau

MacPorts
In reply to this post by MacPorts
#56300: Travis build failure of xorg-libXau
-----------------------------+---------------------
  Reporter:  ryandesign      |      Owner:  admin@…
      Type:  defect          |     Status:  new
  Priority:  Normal          |  Milestone:
 Component:  server/hosting  |    Version:
Resolution:                  |   Keywords:
      Port:                  |
-----------------------------+---------------------

Comment (by raimue):

 It is not about a timeout in the HTTP request, it looks like DNS failed.

 If the upload failed, just wait for 30 seconds and then try again. Do that
 three times. As you can see in the logs, the next fetch worked again, so
 the network problem must have been quite short. That would not add that
 much to risk a build timeout.

--
Ticket URL: <https://trac.macports.org/ticket/56300#comment:9>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #56300: Travis build failure of xorg-libXau

MacPorts
In reply to this post by MacPorts
#56300: Travis build failure of xorg-libXau
-----------------------------+---------------------
  Reporter:  ryandesign      |      Owner:  admin@…
      Type:  defect          |     Status:  closed
  Priority:  Normal          |  Milestone:
 Component:  server/hosting  |    Version:
Resolution:  wontfix         |   Keywords:
      Port:                  |
-----------------------------+---------------------
Changes (by l2dy):

 * status:  new => closed
 * resolution:   => wontfix


Comment:

 Replying to [comment:9 raimue]:
 > It is not about a timeout in the HTTP request, it looks like DNS failed.
 >
 > If the upload failed, just wait for 30 seconds and then try again. Do
 that three times. As you can see in the logs, the next fetch worked again,
 so the network problem must have been quite short. That would not add that
 much to risk a build timeout.

 I've deployed a new version that uploads logs with
 [https://github.com/hashicorp/go-retryablehttp a retryable HTTP client]
 from HashiCorp, see https://github.com/macports/mpbot-
 github/commits/develop.

 Closing this as it's probably a temporary network issue on Travis, and we
 can do nothing about that. The `mpbb list-subports` issue should be
 tracked in #56110.

--
Ticket URL: <https://trac.macports.org/ticket/56300#comment:10>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #56300: Travis build failure of xorg-libXau

MacPorts
In reply to this post by MacPorts
#56300: Travis build failure of xorg-libXau
-----------------------------+---------------------
  Reporter:  ryandesign      |      Owner:  admin@…
      Type:  defect          |     Status:  closed
  Priority:  Normal          |  Milestone:
 Component:  server/hosting  |    Version:
Resolution:  wontfix         |   Keywords:
      Port:                  |
-----------------------------+---------------------

Comment (by ryandesign):

 You "wontfix" the problem that Travis isn't using available binary
 archives, builds are skipping phases they shouldn't skip, dependencies
 that shouldn't already be installed are, and dependencies that are
 declared aren't installed, which results in error conditions that should
 be impossible? What Travis is doing for us is almost completely batty.

--
Ticket URL: <https://trac.macports.org/ticket/56300#comment:11>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #56300: Travis build failure of xorg-libXau

MacPorts
In reply to this post by MacPorts
#56300: Travis build failure of xorg-libXau
-----------------------------+----------------------
  Reporter:  ryandesign      |      Owner:  admin@…
      Type:  defect          |     Status:  reopened
  Priority:  Normal          |  Milestone:
 Component:  server/hosting  |    Version:
Resolution:                  |   Keywords:
      Port:                  |
-----------------------------+----------------------
Changes (by l2dy):

 * status:  closed => reopened
 * resolution:  wontfix =>


Comment:

 Replying to [comment:11 ryandesign]:
 > You "wontfix" the problem that Travis isn't using available binary
 archives, builds are skipping phases they shouldn't skip, dependencies
 that shouldn't already be installed are, and dependencies that are
 declared aren't installed, which results in error conditions that should
 be impossible? What Travis is doing for us is almost completely batty.

 On second thought, cleaning `/opt/local/var/macports/build/*` after each
 build may fix some issues. And it's also probably the cause of skipped
 phases: build failed when installing dependencies of a previous port due
 to connectivity issue, and then another port also has a dependency on this
 port which hasn't been cleaned. Reopening then.

 We could report this to Travis, but Travis depends on MacStadium for these
 VMs so I'm not sure if there can be a permanent fix soon.

 I've been busy recently and didn't reread this ticket carefully. Sorry
 about that.

--
Ticket URL: <https://trac.macports.org/ticket/56300#comment:12>
MacPorts <https://www.macports.org/>
Ports system for macOS
Reply | Threaded
Open this post in threaded view
|

Re: [MacPorts] #56300: Travis build failure of xorg-libXau

MacPorts
In reply to this post by MacPorts
#56300: Travis build failure of xorg-libXau
-----------------------------+----------------------
  Reporter:  ryandesign      |      Owner:  admin@…
      Type:  defect          |     Status:  reopened
  Priority:  Normal          |  Milestone:
 Component:  server/hosting  |    Version:
Resolution:                  |   Keywords:
      Port:                  |
-----------------------------+----------------------

Comment (by l2dy):

 Replying to [comment:12 l2dy]:
 > Replying to [comment:11 ryandesign]:
 > On second thought, cleaning `/opt/local/var/macports/build/*` after each
 build may fix some issues. And it's also probably the cause of skipped
 phases: build failed when installing dependencies of a previous port due
 to connectivity issue, and then another port also has a dependency on this
 port which hasn't been cleaned. Reopening then.

 I now recall that I deliberately skipped the clean phase to save some
 precious build time (If a port failed, why should I spend the time on
 cleaning it and then re-extract and rebuild it again?), but I didn't
 consider network failure by then. So "skipping phases they shouldn't skip"
 was somewhat working as intended.

 Idea: `find /opt/local/var/macports/build -type d -mindepth 4 -maxdepth 4
 -print0 | sudo xargs -0 rm -rf` would be ideal if network is stable and
 port is really failing (saves more time by making the next phase fail
 immediately). Use `mpbb cleanup` for unstable network, which also does
 other cleanups including deactivate ports.

 >> dependencies that shouldn't already be installed are

 Idea: executing `port -fp deactivate active and rleaves` after `mpbb
 install-dependencies` may solve this.

 >> dependencies that are declared aren't installed

 Maybe `mpbb`'s fault? I'm just calling it to installing the
 dependencies...

 Ultimately, if MacStadium's network is stable and CI builds can get
 undistributable binary archives, all these probably would not happen. I
 think when I closed the issue I was thinking that the ticket summary is
 for a specific port and the issue is caused by a network problem (which I
 consider temporary), so let's track actual issues in different tickets and
 close this one.

--
Ticket URL: <https://trac.macports.org/ticket/56300#comment:13>
MacPorts <https://www.macports.org/>
Ports system for macOS