Issues with clock_gettime(CLOCK_REALTIME, &wait) pre macOS 10.13

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Issues with clock_gettime(CLOCK_REALTIME, &wait) pre macOS 10.13

Christopher Jones
Hi,

I've stumbled into the same issue twice in recent days, with two
different ports, which is the use of

clock_gettime(CLOCK_REALTIME, &wait);

which is only available in macOS 10.12 or newer. See for instance the
issue I found yesterday in xrootd.

https://github.com/xrootd/xrootd/issues/846

I am still waiting to see what upstream say, but I am hopeful they will
consider it a bug. ( It would seem quite extreme to reduce the supported
OSX releases from 10.7+ to 10.12+ in a minor patch revision...)

But I was wondering, is this something anyone else has stumbled over,
and do we have a way of fixing this particular issue in the older OSes ?

Chris
Reply | Threaded
Open this post in threaded view
|

Re: Issues with clock_gettime(CLOCK_REALTIME, &wait) pre macOS 10.13

Ken Cunningham

On 2018-10-23, at 1:57 AM, Chris Jones wrote:

> Hi,
>
> I've stumbled into the same issue twice in recent days, with two different ports, which is the use of
>
> clock_gettime(CLOCK_REALTIME, &wait);
>
> which is only available in macOS 10.12 or newer. See for instance the issue I found yesterday in xrootd.
>
> https://github.com/xrootd/xrootd/issues/846
>
> I am still waiting to see what upstream say, but I am hopeful they will consider it a bug. ( It would seem quite extreme to reduce the supported OSX releases from 10.7+ to 10.12+ in a minor patch revision...)
>
> But I was wondering, is this something anyone else has stumbled over, and do we have a way of fixing this particular issue in the older OSes ?
>
> Chris


We have worked around this in a number of ports so far, and turned off some functionality in others.

I use the_silver_searcher to find stuff like this, for example in the macports-ports repo:

ag -i clock_gettime .


Best,

Ken
Reply | Threaded
Open this post in threaded view
|

Re: Issues with clock_gettime(CLOCK_REALTIME, &wait) pre macOS 10.13

Michael Dickens-4
OSX sometimes doesn't provide useful features, or has issues. I love the idea of the `snowleopardfixes` port to provide and fix for Snow Leopard. I'd support expanding the concept to other missing features such as this `clock_gettime` for ports installing on OSX that doesn't provide this functionality (ditto for `posix_memalign`). Doing this would greatly simplify some port patches as well as provide better older-OS compatibility; also keeps these fixes in 1 place for easier maintenance and referral. Seems like a reasonable solution to me! - MLD

On Tue, Oct 23, 2018, at 10:15 AM, Ken Cunningham wrote:

> We have worked around this in a number of ports so far, and turned off
> some functionality in others.
>
> I use the_silver_searcher to find stuff like this, for example in the
> macports-ports repo:
>
> ag -i clock_gettime .
>
>
> Best,
>
> Ken
Reply | Threaded
Open this post in threaded view
|

Re: Issues with clock_gettime(CLOCK_REALTIME, &wait) pre macOS 10.13

Christopher Jones
In reply to this post by Ken Cunningham

> We have worked around this in a number of ports so far, and turned off some functionality in others.
>
> I use the_silver_searcher to find stuff like this, for example in the macports-ports repo:
>
> ag -i clock_gettime .

Sorry, I don't follow the reference to the_silver_searcher or what the
'ag' command above is supposed to be (I don't have it).

Anyway, I guess I was not precise enough. Yes, I already have noted a
number of ports in the repo have patches to work around this. What I was
hoping for was a way to fix this *without* have to patch or touch the
port at all. Something like updating snowleopardfixes to add whatever is
needed so that ports automatically find the required implementation and
use it, transparently.

Chris
Reply | Threaded
Open this post in threaded view
|

Re: Issues with clock_gettime(CLOCK_REALTIME, &wait) pre macOS 10.13

Ken Cunningham

On 2018-10-23, at 8:18 AM, Chris Jones wrote:

>
>> We have worked around this in a number of ports so far, and turned off some functionality in others.
>> I use the_silver_searcher to find stuff like this, for example in the macports-ports repo:
>> ag -i clock_gettime .
>
> Sorry, I don't follow the reference to the_silver_searcher or what the 'ag' command above is supposed to be (I don't have it).
>
> Anyway, I guess I was not precise enough. Yes, I already have noted a number of ports in the repo have patches to work around this. What I was hoping for was a way to fix this *without* have to patch or touch the port at all. Something like updating snowleopardfixes to add whatever is needed so that ports automatically find the required implementation and use it, transparently.
>
> Chris

the_silver_searcher is one great tool. I probably use it every day in one way or another. it's a port, you can install it. Highly recommended by me.

I think extending snowleopard_fixes is a fine idea.

I have been pondering a good way to make it add the header definition properly ( ? use specific headers such as string.h injected ahead of the system search directory and "include_next", perhaps?) and also possibly to make each definition selectable, or at least OS-version groups blocked off with guards.


BTW, here is a nice clock_gettime() replacement that Jeremy wrote up:

<https://github.com/macports/macports-ports/blob/master/graphics/cogl/files/patch-clock_gettime.diff>

Ken
Reply | Threaded
Open this post in threaded view
|

Re: Issues with clock_gettime(CLOCK_REALTIME, &wait) pre macOS 10.13

Christopher Jones
Hi,

On 23/10/18 16:35, Ken Cunningham wrote:

>
> On 2018-10-23, at 8:18 AM, Chris Jones wrote:
>
>>
>>> We have worked around this in a number of ports so far, and turned off some functionality in others.
>>> I use the_silver_searcher to find stuff like this, for example in the macports-ports repo:
>>> ag -i clock_gettime .
>>
>> Sorry, I don't follow the reference to the_silver_searcher or what the 'ag' command above is supposed to be (I don't have it).
>>
>> Anyway, I guess I was not precise enough. Yes, I already have noted a number of ports in the repo have patches to work around this. What I was hoping for was a way to fix this *without* have to patch or touch the port at all. Something like updating snowleopardfixes to add whatever is needed so that ports automatically find the required implementation and use it, transparently.
>>
>> Chris
>
> the_silver_searcher is one great tool. I probably use it every day in one way or another. it's a port, you can install it. Highly recommended by me.

Ah I see. Missed it was a port.

As my port tree is a full git clone, then I can use 'git grep' to search
for anything in it, and I bet that beats anything else ;)

>
> I think extending snowleopard_fixes is a fine idea.
>
> I have been pondering a good way to make it add the header definition properly ( ? use specific headers such as string.h injected ahead of the system search directory and "include_next", perhaps?) and also possibly to make each definition selectable, or at least OS-version groups blocked off with guards.

Yeah, this was my idea...

>
>
> BTW, here is a nice clock_gettime() replacement that Jeremy wrote up:
>
> <https://github.com/macports/macports-ports/blob/master/graphics/cogl/files/patch-clock_gettime.diff>


Thanks. That is a bit better than what I cooked up for xrootd. I might
replace it ;)

Chris

>
> Ken
>
Reply | Threaded
Open this post in threaded view
|

Re: Issues with clock_gettime(CLOCK_REALTIME, &wait) pre macOS 10.13

Christopher Jones

>>
>> BTW, here is a nice clock_gettime() replacement that Jeremy wrote up:
>>
>> <https://github.com/macports/macports-ports/blob/master/graphics/cogl/files/patch-clock_gettime.diff>
>>
>
>
> Thanks. That is a bit better than what I cooked up for xrootd. I might
> replace it ;)

Actually, I take that back. I am not convinced the implementation in the
above link is giving results consistent with the method it is supposed
to be provide. See the attached example test code...

test1 is the output from  clock_gettime(CLOCK_REALTIME, &ts);, whether
that be from the system ( 10.12+) or from the link above (older).

test2 is my own workaround.

One 10.12 I get

  > ./mac1013.exe
time1 1540311943653502000
time2 1540311943653580000
time1 1540311943653606000
time2 1540311943653619000

so the two give the same times (note time of course moves on between
each call, but the two clearly agree).

on 10.9 though I get

  > ./mac109.exe
time1 1156992536990
time2 1540311478974454000
time1 1156992628574
time2 1540311478974495000

see how the values of time2 agree, but time1 (now from Jeremy's
implementation) clearly has a completely different scale.

Not sure why, but for sure its not quite a drop in replacement...

Chris



time.cpp (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Issues with clock_gettime(CLOCK_REALTIME, &wait) pre macOS 10.13

Christopher Jones
In reply to this post by Ken Cunningham
Hi,

> I have been pondering a good way to make it add the header definition properly ( ? use specific headers such as string.h injected ahead of the system search directory and "include_next", perhaps?) and also possibly to make each definition selectable, or at least OS-version groups blocked off with guards.

I’ve been experimenting in the direction of the above, and I think there is some milage to this approach.

I’ve attached a couple files to this mail.

time.cpp - A basic test application

time.h - A wrapping header that uses include_next to wrap time.h adding an implementation of clock_gettime for OSX10.11 and older.

I *think* I have made it reasonably complete, in that if I compare what I get on macOS10.14 (so using the system implementation) to that on OSX10.10 I see

 > ./mac1014.exe
CLOCK_REALTIME  1540330516865744000
CLOCK_REALTIME  1540330516865769000
CLOCK_REALTIME  1540330516865773000
CLOCK_REALTIME  1540330516865776000
CLOCK_REALTIME  1540330516865779000
CLOCK_REALTIME  1540330516865782000
CLOCK_REALTIME  1540330516865785000
CLOCK_REALTIME  1540330516865789000
CLOCK_REALTIME  1540330516865792000
CLOCK_MONOTONIC 692301875594000
CLOCK_MONOTONIC 692301875598000
CLOCK_MONOTONIC 692301875601000
CLOCK_MONOTONIC 692301875606000
CLOCK_MONOTONIC 692301875609000
CLOCK_MONOTONIC 692301875612000
CLOCK_MONOTONIC 692301875615000
CLOCK_MONOTONIC 692301875619000
CLOCK_MONOTONIC 692301875622000

 > ./mac1010.exe
CLOCK_REALTIME  1540330771661752000
CLOCK_REALTIME  1540330771662759000
CLOCK_REALTIME  1540330771662770000
CLOCK_REALTIME  1540330771662776000
CLOCK_REALTIME  1540330771662783000
CLOCK_REALTIME  1540330771662789000
CLOCK_REALTIME  1540330771662795000
CLOCK_REALTIME  1540330771662801000
CLOCK_REALTIME  1540330771662807000
CLOCK_MONOTONIC 3731777139000
CLOCK_MONOTONIC 3731777145000
CLOCK_MONOTONIC 3731777151000
CLOCK_MONOTONIC 3731777157000
CLOCK_MONOTONIC 3731777162000
CLOCK_MONOTONIC 3731777168000
CLOCK_MONOTONIC 3731777174000
CLOCK_MONOTONIC 3731777180000
CLOCK_MONOTONIC 3731777186000


The REALTIME clock is supposed to give values roughly the same in both cases, as these are supposed to be absolute wall clock times since the UNIX epoch.

MONOTONIC is allowed to have different values, as its w.r.t. different points for each machine.

Compilation is just a case of

 > ls *.h *.cpp
time.cpp time.h
 > clang++ -I. time.cpp -o mac1010.exe
 >

If you remove the -I. then on 10.10 it correctly fails.

 > clang++ time.cpp -o mac1010.exe
time.cpp:26:45: error: use of undeclared identifier 'CLOCK_REALTIME'
    std::cout << "CLOCK_REALTIME  " << time(CLOCK_REALTIME) << std::endl;
                                            ^
time.cpp:31:45: error: use of undeclared identifier 'CLOCK_MONOTONIC'
    std::cout << "CLOCK_MONOTONIC " << time(CLOCK_MONOTONIC) << std::endl;
                                            ^
2 errors generated.

so the include_next seems to work OK.

I have tried with a number of gcc and clang compilers, and it seems fine with them all.

So I *think* we could consider including the wrapper time.h in the snowleopardfixes port, and then as long as builds use -I${prefix}/include then should pick it up without having to patch the code in any way, which is my aim here...

Chris


time.h (869 bytes) Download Attachment
time.cpp (676 bytes) Download Attachment
smime.p7s (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Issues with clock_gettime(CLOCK_REALTIME, &wait) pre macOS 10.13

Ken Cunningham
Yep, that is exactly what I was thinking as well.

I was pondering a specific include folder like

${prefix}/include/snowleopardfixes

for want of a better title for it, where these could be placed. Then we could just add that folder to the beginning of the system header search queue. (It would in fact be automatic and require no extra include folder spec at all if we used /usr/local/include, but then we lose some control that way. That is exactly why homebrew is both popular and harder to control.)

Maybe we could do all the missing function replacements that way, and not even need a linked in dylib. That would let us control visibility with system blocks and help avoid name collisions when linking.


Ken




On 2018-10-23, at 2:47 PM, Christopher Jones wrote:

> Hi,
>
>> I have been pondering a good way to make it add the header definition properly ( ? use specific headers such as string.h injected ahead of the system search directory and "include_next", perhaps?) and also possibly to make each definition selectable, or at least OS-version groups blocked off with guards.
>
> I’ve been experimenting in the direction of the above, and I think there is some milage to this approach.
>
> I’ve attached a couple files to this mail.
>
> time.cpp - A basic test application
>
> time.h - A wrapping header that uses include_next to wrap time.h adding an implementation of clock_gettime for OSX10.11 and older.
>
> I *think* I have made it reasonably complete, in that if I compare what I get on macOS10.14 (so using the system implementation) to that on OSX10.10 I see
>
>> ./mac1014.exe
> CLOCK_REALTIME  1540330516865744000
> CLOCK_REALTIME  1540330516865769000
> CLOCK_REALTIME  1540330516865773000
> CLOCK_REALTIME  1540330516865776000
> CLOCK_REALTIME  1540330516865779000
> CLOCK_REALTIME  1540330516865782000
> CLOCK_REALTIME  1540330516865785000
> CLOCK_REALTIME  1540330516865789000
> CLOCK_REALTIME  1540330516865792000
> CLOCK_MONOTONIC 692301875594000
> CLOCK_MONOTONIC 692301875598000
> CLOCK_MONOTONIC 692301875601000
> CLOCK_MONOTONIC 692301875606000
> CLOCK_MONOTONIC 692301875609000
> CLOCK_MONOTONIC 692301875612000
> CLOCK_MONOTONIC 692301875615000
> CLOCK_MONOTONIC 692301875619000
> CLOCK_MONOTONIC 692301875622000
>
>> ./mac1010.exe
> CLOCK_REALTIME  1540330771661752000
> CLOCK_REALTIME  1540330771662759000
> CLOCK_REALTIME  1540330771662770000
> CLOCK_REALTIME  1540330771662776000
> CLOCK_REALTIME  1540330771662783000
> CLOCK_REALTIME  1540330771662789000
> CLOCK_REALTIME  1540330771662795000
> CLOCK_REALTIME  1540330771662801000
> CLOCK_REALTIME  1540330771662807000
> CLOCK_MONOTONIC 3731777139000
> CLOCK_MONOTONIC 3731777145000
> CLOCK_MONOTONIC 3731777151000
> CLOCK_MONOTONIC 3731777157000
> CLOCK_MONOTONIC 3731777162000
> CLOCK_MONOTONIC 3731777168000
> CLOCK_MONOTONIC 3731777174000
> CLOCK_MONOTONIC 3731777180000
> CLOCK_MONOTONIC 3731777186000
>
>
> The REALTIME clock is supposed to give values roughly the same in both cases, as these are supposed to be absolute wall clock times since the UNIX epoch.
>
> MONOTONIC is allowed to have different values, as its w.r.t. different points for each machine.
>
> Compilation is just a case of
>
>> ls *.h *.cpp
> time.cpp time.h
>> clang++ -I. time.cpp -o mac1010.exe
>>
>
> If you remove the -I. then on 10.10 it correctly fails.
>
>> clang++ time.cpp -o mac1010.exe
> time.cpp:26:45: error: use of undeclared identifier 'CLOCK_REALTIME'
>    std::cout << "CLOCK_REALTIME  " << time(CLOCK_REALTIME) << std::endl;
>                                            ^
> time.cpp:31:45: error: use of undeclared identifier 'CLOCK_MONOTONIC'
>    std::cout << "CLOCK_MONOTONIC " << time(CLOCK_MONOTONIC) << std::endl;
>                                            ^
> 2 errors generated.
>
> so the include_next seems to work OK.
>
> I have tried with a number of gcc and clang compilers, and it seems fine with them all.
>
> So I *think* we could consider including the wrapper time.h in the snowleopardfixes port, and then as long as builds use -I${prefix}/include then should pick it up without having to patch the code in any way, which is my aim here...
>
> Chris
>
> <time.h><time.cpp>

Reply | Threaded
Open this post in threaded view
|

Re: Issues with clock_gettime(CLOCK_REALTIME, &wait) pre macOS 10.13

Christopher Jones
Hi,

> On 23 Oct 2018, at 11:26 pm, Ken Cunningham <[hidden email]> wrote:
>
> Yep, that is exactly what I was thinking as well.
>
> I was pondering a specific include folder like
>
> ${prefix}/include/snowleopardfixes
>
> for want of a better title for it, where these could be placed. Then we could just add that folder to the beginning of the system header search queue. (It would in fact be automatic and require no extra include folder spec at all if we used /usr/local/include, but then we lose some control that way. That is exactly why homebrew is both popular and harder to control.)

I see no reason to not just place them directly in the main MacPorts include prefix, i.e. normally /opt/local/include. Placing them anywhere else requires that specific folder to be included as well, which is a complication I dont see the need for. As long as these headers are written properly, they should be completely safe and transparent to use. Either they just include the primary header in cases where it is complete, or they add the missing methods in cases where they are required, but only where they are required.

>
> Maybe we could do all the missing function replacements that way, and not even need a linked in dylib. That would let us control visibility with system blocks and help avoid name collisions when linking.

I was thinking that as well. If the need for libraries can be avoid it makes things much simpler to handle. I actually suspect all of what you currently have in snowleopardfixes could be reimplemented in this way, and then largely avoid the need for the additional options you then have to add to all compilations in the portgroup.

Also because of the above I now think it is probably better to not add this to the existing snowleopardfixes port and instead start a new one for this. For one thing the name is misleading once it used for more systems ;) but, more seriously what I am imagining is a port that could simply be depended on as required, which just installs these wrapper headers and then which are hopefully just picked up by the builds through automatically, because they should already be including the MP prefix include dir. all the additional compilation of the portgroup should I think just go away.

Chris

>
>
> Ken
>
>
>
>
>> On 2018-10-23, at 2:47 PM, Christopher Jones wrote:
>>
>> Hi,
>>
>>> I have been pondering a good way to make it add the header definition properly ( ? use specific headers such as string.h injected ahead of the system search directory and "include_next", perhaps?) and also possibly to make each definition selectable, or at least OS-version groups blocked off with guards.
>>
>> I’ve been experimenting in the direction of the above, and I think there is some milage to this approach.
>>
>> I’ve attached a couple files to this mail.
>>
>> time.cpp - A basic test application
>>
>> time.h - A wrapping header that uses include_next to wrap time.h adding an implementation of clock_gettime for OSX10.11 and older.
>>
>> I *think* I have made it reasonably complete, in that if I compare what I get on macOS10.14 (so using the system implementation) to that on OSX10.10 I see
>>
>>> ./mac1014.exe
>> CLOCK_REALTIME  1540330516865744000
>> CLOCK_REALTIME  1540330516865769000
>> CLOCK_REALTIME  1540330516865773000
>> CLOCK_REALTIME  1540330516865776000
>> CLOCK_REALTIME  1540330516865779000
>> CLOCK_REALTIME  1540330516865782000
>> CLOCK_REALTIME  1540330516865785000
>> CLOCK_REALTIME  1540330516865789000
>> CLOCK_REALTIME  1540330516865792000
>> CLOCK_MONOTONIC 692301875594000
>> CLOCK_MONOTONIC 692301875598000
>> CLOCK_MONOTONIC 692301875601000
>> CLOCK_MONOTONIC 692301875606000
>> CLOCK_MONOTONIC 692301875609000
>> CLOCK_MONOTONIC 692301875612000
>> CLOCK_MONOTONIC 692301875615000
>> CLOCK_MONOTONIC 692301875619000
>> CLOCK_MONOTONIC 692301875622000
>>
>>> ./mac1010.exe
>> CLOCK_REALTIME  1540330771661752000
>> CLOCK_REALTIME  1540330771662759000
>> CLOCK_REALTIME  1540330771662770000
>> CLOCK_REALTIME  1540330771662776000
>> CLOCK_REALTIME  1540330771662783000
>> CLOCK_REALTIME  1540330771662789000
>> CLOCK_REALTIME  1540330771662795000
>> CLOCK_REALTIME  1540330771662801000
>> CLOCK_REALTIME  1540330771662807000
>> CLOCK_MONOTONIC 3731777139000
>> CLOCK_MONOTONIC 3731777145000
>> CLOCK_MONOTONIC 3731777151000
>> CLOCK_MONOTONIC 3731777157000
>> CLOCK_MONOTONIC 3731777162000
>> CLOCK_MONOTONIC 3731777168000
>> CLOCK_MONOTONIC 3731777174000
>> CLOCK_MONOTONIC 3731777180000
>> CLOCK_MONOTONIC 3731777186000
>>
>>
>> The REALTIME clock is supposed to give values roughly the same in both cases, as these are supposed to be absolute wall clock times since the UNIX epoch.
>>
>> MONOTONIC is allowed to have different values, as its w.r.t. different points for each machine.
>>
>> Compilation is just a case of
>>
>>> ls *.h *.cpp
>> time.cpp    time.h
>>> clang++ -I. time.cpp -o mac1010.exe
>>>
>>
>> If you remove the -I. then on 10.10 it correctly fails.
>>
>>> clang++ time.cpp -o mac1010.exe
>> time.cpp:26:45: error: use of undeclared identifier 'CLOCK_REALTIME'
>>   std::cout << "CLOCK_REALTIME  " << time(CLOCK_REALTIME) << std::endl;
>>                                           ^
>> time.cpp:31:45: error: use of undeclared identifier 'CLOCK_MONOTONIC'
>>   std::cout << "CLOCK_MONOTONIC " << time(CLOCK_MONOTONIC) << std::endl;
>>                                           ^
>> 2 errors generated.
>>
>> so the include_next seems to work OK.
>>
>> I have tried with a number of gcc and clang compilers, and it seems fine with them all.
>>
>> So I *think* we could consider including the wrapper time.h in the snowleopardfixes port, and then as long as builds use -I${prefix}/include then should pick it up without having to patch the code in any way, which is my aim here...
>>
>> Chris
>>
>> <time.h><time.cpp>
>

Reply | Threaded
Open this post in threaded view
|

Re: Issues with clock_gettime(CLOCK_REALTIME, &wait) pre macOS 10.13

Ken Cunningham

On 2018-10-23, at 4:23 PM, Chris Jones wrote:
>
> I see no reason to not just place them directly in the main MacPorts include prefix, i.e. normally /opt/local/include. Placing them anywhere else requires that specific folder to be included as well, which is a complication I dont see the need for. As long as these headers are written properly, they should be completely safe and transparent to use. Either they just include the primary header in cases where it is complete, or they add the missing methods in cases where they are required, but only where they are required.
>

I have always liked this idea, and I think we should explore it through to completion. It could ease the burden of supporting older OS versions greatly.

Given that once this "port of headers" is installed it would affect every build whether listed as a dep or not, it would have to be assumed to be installed always, ie a MacPorts default installation.

We would also have to make sure it doesn't interfere with bootstrapping and building compilers like gcc, that have their own mechanism of "fixincludes" that we don't want to mess up.

Ken
Reply | Threaded
Open this post in threaded view
|

Re: Issues with clock_gettime(CLOCK_REALTIME, &wait) pre macOS 10.13

Christopher Jones
Hi,

On 24/10/18 01:44, Ken Cunningham wrote:
>
> On 2018-10-23, at 4:23 PM, Chris Jones wrote:
>>
>> I see no reason to not just place them directly in the main MacPorts include prefix, i.e. normally /opt/local/include. Placing them anywhere else requires that specific folder to be included as well, which is a complication I dont see the need for. As long as these headers are written properly, they should be completely safe and transparent to use. Either they just include the primary header in cases where it is complete, or they add the missing methods in cases where they are required, but only where they are required.
>>
>
> I have always liked this idea, and I think we should explore it through to completion. It could ease the burden of supporting older OS versions greatly.

I am interested in working on this, as I think it would ease some issues
in a few ports I have stumbled over recently. Let me try and put
something to test together.

>
> Given that once this "port of headers" is installed it would affect every build whether listed as a dep or not, it would have to be assumed to be installed always, ie a MacPorts default installation.

Correct. It would only work as I imagine if having these headers
installed and available is completely transparent.

>
> We would also have to make sure it doesn't interfere with bootstrapping and building compilers like gcc, that have their own mechanism of "fixincludes" that we don't want to mess up.

We would certainly need to carefully test things, and the compilers
would be an obvious thing to make sure are OK.

Again though, I would hope for it to be transparent.... But to be proved.

Chris

>
> Ken
>
Reply | Threaded
Open this post in threaded view
|

Re: Issues with clock_gettime(CLOCK_REALTIME, &wait) pre macOS 10.13

Christopher Jones
Hi,

> I am interested in working on this, as I think it would ease some issues
> in a few ports I have stumbled over recently. Let me try and put
> something to test together.

I have started to put something together for this. I have a little new
project in github

https://github.com/cjones051073/macports-legacy-support

and then a new port that installs the headers from this repo.

<https://github.com/cjones051073/macports-ports/commit/f1b25a6dea9e895ae6d1d14411099b6cd43153ea>

I have run a first quick test, building xrootd, and the signs are
positive in that as I hoped it fixed the issues I was seeing with xrootd
completely transparently, without having to patch the port or code in
anyway. Obviously much more testing is needed though, so please if you
are interested (Ken?) checkout my ports branch above and give it a try
yourself.

Also, if you have any suggestions for additional header fixes to add,
let me know (or make a PR against the project...). ( Longer term, if
this idea has legs, I think the git repo should move from my account to
the main MacPorts area, but this can wait. )

Chris
Reply | Threaded
Open this post in threaded view
|

Re: Issues with clock_gettime(CLOCK_REALTIME, &wait) pre macOS 10.13

Christopher Jones

FWIW I have tested the basic idea back as far as OSX 10.6, and it works
fine there with the system gcc 4.2.1

MacVM106 /Volumes/VMware Shared Folders/Projects/Test > g++ -I. time.cpp
-o mac106.exe
MacVM106 /Volumes/VMware Shared Folders/Projects/Test > ./mac106.exe
0 CLOCK_REALTIME  1540375323758611000
0 CLOCK_REALTIME  1540375323758750000
0 CLOCK_REALTIME  1540375323758772000
0 CLOCK_REALTIME  1540375323758791000
0 CLOCK_REALTIME  1540375323758808000
0 CLOCK_REALTIME  1540375323758826000
0 CLOCK_REALTIME  1540375323758844000
0 CLOCK_REALTIME  1540375323758861000
0 CLOCK_REALTIME  1540375323758879000
0 CLOCK_MONOTONIC 1329176161010324
0 CLOCK_MONOTONIC 1329176161027572
0 CLOCK_MONOTONIC 1329176161044815
0 CLOCK_MONOTONIC 1329176161061733
0 CLOCK_MONOTONIC 1329176161078708
0 CLOCK_MONOTONIC 1329176161095875
0 CLOCK_MONOTONIC 1329176161112940
0 CLOCK_MONOTONIC 1329176161130299
0 CLOCK_MONOTONIC 1329176161147827
MacVM106 /Volumes/VMware Shared Folders/Projects/Test > g++ --version
i686-apple-darwin10-g++-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5666) (dot 3)
Copyright (C) 2007 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


so the `include_next` directive is fine that far back...

OSX 10.6 is the oldest I can test, but it would be interesting to see if
it also works on 10.5 (PPC)....

Chris

On 24/10/18 10:17, Chris Jones wrote:

> Hi,
>
>> I am interested in working on this, as I think it would ease some
>> issues in a few ports I have stumbled over recently. Let me try and
>> put something to test together.
>
> I have started to put something together for this. I have a little new
> project in github
>
> https://github.com/cjones051073/macports-legacy-support
>
> and then a new port that installs the headers from this repo.
>
> <https://github.com/cjones051073/macports-ports/commit/f1b25a6dea9e895ae6d1d14411099b6cd43153ea>
>
>
> I have run a first quick test, building xrootd, and the signs are
> positive in that as I hoped it fixed the issues I was seeing with xrootd
> completely transparently, without having to patch the port or code in
> anyway. Obviously much more testing is needed though, so please if you
> are interested (Ken?) checkout my ports branch above and give it a try
> yourself.
>
> Also, if you have any suggestions for additional header fixes to add,
> let me know (or make a PR against the project...). ( Longer term, if
> this idea has legs, I think the git repo should move from my account to
> the main MacPorts area, but this can wait. )
>
> Chris
Reply | Threaded
Open this post in threaded view
|

Re: Issues with clock_gettime(CLOCK_REALTIME, &wait) pre macOS 10.13

Fred Wright
In reply to this post by Christopher Jones

On Tue, 23 Oct 2018, Chris Jones wrote:

> I've stumbled into the same issue twice in recent days, with two different
> ports, which is the use of
>
> clock_gettime(CLOCK_REALTIME, &wait);
>
> which is only available in macOS 10.12 or newer. See for instance the issue I
> found yesterday in xrootd.
>
> https://github.com/xrootd/xrootd/issues/846
>
> I am still waiting to see what upstream say, but I am hopeful they will
> consider it a bug. ( It would seem quite extreme to reduce the supported OSX
> releases from 10.7+ to 10.12+ in a minor patch revision...)
>
> But I was wondering, is this something anyone else has stumbled over, and do
> we have a way of fixing this particular issue in the older OSes ?

Yes, in both GPSD and ntpsec.  Rather than trying to figure out which of
the many messages in this thread to answer directly, I'll just throw out
what I know about this issue.

There are three "global" timescales potentially provided by
clock_gettime():  CLOCK_REALTIME, CLOCK_MONOTONIC, and
CLOCK_MONOTONIC_RAW.  Only the first is eligible for clock_settime().

CLOCK_REALTIME is the same "Unix" timescale as the original time() and
later gettimeofday(), but with (ostensibly) nanosecond resolution.  It's
subject to both slewing and step adjustments, as needed to synchronize its
value to some time source.

CLOCK_MONOTONIC was created to avoid problems (including crashes)
sometimes caused by the backward step adjustments that may be applied to
CLOCK_REALTIME.  Although the official documentation is woefully
underspecified, it's typically implemented as a variation on
CLOCK_REALTIME that excludes all step adjustments (including the initial
one that "sets the clock"), but includes all slewing.  This makes it
continuous as well as monotonic, but its rate accuracy is corrupted by the
slewing adjustments.  In practice, it's almost never what you really want.

CLOCK_MONOTONIC_RAW is also woefully underspecified, but is usually just
the raw hardware time source scaled to standard units based on the
assumed clock rate, but not steered at all.  Since even the cheapest
crystals are typically rated at +/- 100ppm or better, and since the
slewing adjustments applied to CLOCK_MONOTONIC can easily be much larger
than that, CLOCK_MONOTONIC_RAW is usually a more accurate timescale for
rates, durations, and delays than CLOCK_MONOTONIC.


There are basically two fallback options for CLOCK_REALTIME:
gettimeofday() and the Mach-specific clock_get_time() based on
CALENDAR_CLOCK.  The latter ostensibly has nanosecond resolution, but in
reality it's only the *representation* that has nanosecond resolution,
while the values are all multiples of 1000 nanoseconds.  In addition, it's
more than an order of magnitude slower than gettimeofday() even in the
best case, and the commonly circulated example of its use is even slower,
as well as having a "port leak" bug.  Thus, it's best to simply use
gettimeofday() with a microseconds->nanoseconds as a fallback.  This
approach also works for substituting settimeofday() for
clock_settime(CLOCK_REALTIME, ...).  IMO, the microsecond->nanosecond
conversion should be done without "rounding", but the
nanosecond->microsecond conversions should round by adding 500ns prior to
the floored division by 1000.  The unrounded conversion is consistent with
both clock_get_time() and the "official" clock_gettime() in 10.12+, which
still only has microsecond actual resolution.

For CLOCK_MONOTONIC, I believe the only functionally correct fallback is
to use clock_get_time() with SYSTEM_CLOCK.  As noted above, programs
shouldn't really be using CLOCK_MONOTONIC anyway, but it's necessary to
include it for compatibility with programs too dumb to know that.  The
problem with clock_get_time() is that it requires messing with Mach ports.
The most efficient way to do this is to obtain a SYSTEM_CLOCK port once
initially, and the reuse it on each call.  Even with this, it takes over
700ns on a 3.46GHz Mac Pro, as compared to ~40ns for gettimeofday(), but
that's the price of correctness.  Since something intended to be a drop-in
replacement for clock_gettime() can't rely on initialization or cleanup
functions, the best it can do is to allocate the Mach port on first call,
and then rely on exit cleanup to eventually deallocate it.

For CLOCK_MONOTONIC_RAW, the straightforward approach is to use
mach_absolute_time() with the proper scaling.  As long as the scale
factors are constant, they can be obtained once initially and then cached
for later use.  Unlike the clock_get_time() case, cleanup isn't even an
issue.  However, I've seen some mention of the possibility that the rate
of mach_absolute_time() may not be constant.  I'm not aware of any cases
where this actually happens, and perhaps it's only theoretical, but if it
did actually happen, it would complicate things significantly.  In order
to convert a variable-rate clock to standard units, it's necessary to know
not only the current scale factor, but also the last time that the factor
changed and what the correspondence was at that time.  Since that
information isn't provided, either the scale is actually constant or the
API is deficient.  Hopefully it's the former.


My clock_gettime() replacement for ntpsec is defined directly in a header
file as an inline function.  Although it currently only supports
CLOCK_REALTIME, it does include a switch() on the clock_id for
extensibility.  A significant advantage of the inline approach is that
whenever the clock_id is a compile-time constant (as is almost always the
case in real use cases), the optimizer can completely remove the switch()
and degenerate into just the inline code needed (quite simple for
CLOCK_REALTIME) in the relevant case.  And of course it also avoids
adding new link-time dependencies.

Fred Wright