Quantcast

manpages is mdoc(7)

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

manpages is mdoc(7)

Jan Stary
Currently, the port-* manpages are written
using the legacy man(7) language which uses
low-level roff constructs to described presentational details.

I propose to rewrite them into the _semantic_ markup of mdoc(7) language.
Both have been around for decades and are well supported by groff.

As an example, please see a proposed rewrite of port-cat.1
(and compare it to the current version).

I there any interest in this?

        Jan


.Dd March 30, 2017
.Dt PORT-CAT 1
.Os
.Sh NAME
.Nm port-cat
.Nd print the contents of a Portfile
.Sh SYNOPSIS
.Nm
.Op Fl D Ar portdir
.Ic cat
.Op Ar portname | pseudo-portname | port-expressions | port-url
.Sh DESCRIPTION
Print the
.Pa Portfile
of the given port(s).
Note that the latest version in your local ports tree is printed.
.Sh SEE ALSO
.Xr port 1 ,
.Xr port-selfupdate
.Sh AUTHORS
.An Clemens Lang Aq mt [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: manpages is mdoc(7)

Jan Stary
Err, the intended subject was "manpages in mdoc(7)" :-)
But yes, manpages is mdoc(7), imho.

On Mar 30 19:08:44, [hidden email] wrote:

> Currently, the port-* manpages are written
> using the legacy man(7) language which uses
> low-level roff constructs to described presentational details.
>
> I propose to rewrite them into the _semantic_ markup of mdoc(7) language.
> Both have been around for decades and are well supported by groff.
>
> As an example, please see a proposed rewrite of port-cat.1
> (and compare it to the current version).
>
> I there any interest in this?
>
> Jan
>
>
> .Dd March 30, 2017
> .Dt PORT-CAT 1
> .Os
> .Sh NAME
> .Nm port-cat
> .Nd print the contents of a Portfile
> .Sh SYNOPSIS
> .Nm
> .Op Fl D Ar portdir
> .Ic cat
> .Op Ar portname | pseudo-portname | port-expressions | port-url
> .Sh DESCRIPTION
> Print the
> .Pa Portfile
> of the given port(s).
> Note that the latest version in your local ports tree is printed.
> .Sh SEE ALSO
> .Xr port 1 ,
> .Xr port-selfupdate
> .Sh AUTHORS
> .An Clemens Lang Aq mt [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: manpages is mdoc(7)

Rainer Müller-4
In reply to this post by Jan Stary
On 03/30/2017 07:08 PM, Jan Stary wrote:
> Currently, the port-* manpages are written
> using the legacy man(7) language which uses
> low-level roff constructs to described presentational details.
>
> I propose to rewrite them into the _semantic_ markup of mdoc(7) language.
> Both have been around for decades and are well supported by groff.
>
> As an example, please see a proposed rewrite of port-cat.1
> (and compare it to the current version).

Our man pages are actually written in AsciiDoc and then converted to the roff
output. We only maintain the generated roff output in the repository to solve
the problem of installing base without any external dependencies.

Changing the output format would require adjusting the corresponding DocBook XSL
stylesheets, which is not trivial and out of our scope.

To edit the man pages, first get the required dependencies:

$ sudo port install asciidoc docbook-xsl

1) Modify port-*.1.txt
2) Run 'make' to generate port-*.1
3) Check result 'man ./port-*.1' and repeat 1)-2) as necessary

To finalize your changes:

4) git commit -m 'Fix ...' port-*.1.txt
5) make
6) git commit -m 'Regenerate man pages' port-*.1

This separate commit after regenerating is required because the build rules use
the git author date of the latest commit to set the date of the man page.

Rainer
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: manpages is mdoc(7)

Clemens Lang-2
In reply to this post by Jan Stary
Hey,

On Thu, Mar 30, 2017 at 07:08:44PM +0200, Jan Stary wrote:

> Currently, the port-* manpages are written using the legacy man(7)
> language which uses low-level roff constructs to described
> presentational details.
>
> I propose to rewrite them into the _semantic_ markup of mdoc(7)
> language. Both have been around for decades and are well supported by
> groff.
>
> As an example, please see a proposed rewrite of port-cat.1
> (and compare it to the current version).
>
> I there any interest in this?

There's in fact so much interest in getting rid of the legacy format
that the current version of most of the port-* manpages are generated
from asciidoc versions that are right next to them.

See for example
 https://github.com/macports/macports-base/blob/master/doc/port-activate.1.txt
 https://github.com/macports/macports-base/blob/master/doc/port-activate.1

Not all of the manpages have been converted yet, though. If you want to
help out, asciidoc versions of portfile(7) and porthier(7) would be very
welcome.

--
Clemens
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: manpages is mdoc(7)

Jan Stary
In reply to this post by Rainer Müller-4
On Mar 30 19:08:44, [hidden email] wrote:
> I propose to rewrite them into the _semantic_ markup of mdoc(7) language.

On Mar 30 20:59:31, [hidden email] wrote:
> Our man pages are actually written in AsciiDoc and then converted to the roff
> output. We only maintain the generated roff output in the repository to solve
> the problem of installing base without any external dependencies.
>
> Changing the output format would require
> adjusting the corresponding DocBook XSL stylesheets [...]

Just forget it. My intentions when proposing the mdoc(7) rewrite
were the opposite of having asciidoc and docbook and xml/xsl ...

        Jan

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: manpages is mdoc(7)

Rainer Müller-4
On 03/30/2017 09:26 PM, Jan Stary wrote:
> Just forget it. My intentions when proposing the mdoc(7) rewrite
> were the opposite of having asciidoc and docbook and xml/xsl ...

While the dependencies seem quite heavy, languages like man(7)/mdoc(7) just fell
out of fashion and rarely anyone still understands it or is willing to learn it...

The intention of using AsciiDoc is to make contributions easier, and it has
proven its fitness for this task in other open source projects such as Git.

Rainer
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: manpages is mdoc(7)

Jan Stary
On Mar 30 21:47:38, [hidden email] wrote:
> On 03/30/2017 09:26 PM, Jan Stary wrote:
> > Just forget it. My intentions when proposing the mdoc(7) rewrite
> > were the opposite of having asciidoc and docbook and xml/xsl ...
>
> While the dependencies seem quite heavy,

(to say the least)

> languages like man(7)/mdoc(7) just fell out of fashion

The vast majority of UNIX manpages is written in these languages.

> and rarely anyone still understands it or is willing to learn it...
> The intention of using AsciiDoc is to make contributions easier, and it has
> proven its fitness for this task in other open source projects such as Git.

I doubt very much that people cannot read or write mdoc(7).
As for contributions, does that mean people contributed
those asciidoc rewriter to macports?

Anyway, the mdoc(7) rewrite is apparently not happening.

        Jas


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: manpages is mdoc(7)

Ryan Schmidt-24

On Mar 30, 2017, at 15:11, Jan Stary wrote:

> On Mar 30 21:47:38, [hidden email] wrote:
>> On 03/30/2017 09:26 PM, Jan Stary wrote:
>>> Just forget it. My intentions when proposing the mdoc(7) rewrite
>>> were the opposite of having asciidoc and docbook and xml/xsl ...
>>
>> While the dependencies seem quite heavy,
>
> (to say the least)
>
>> languages like man(7)/mdoc(7) just fell out of fashion
>
> The vast majority of UNIX manpages is written in these languages.
>
>> and rarely anyone still understands it or is willing to learn it...
>> The intention of using AsciiDoc is to make contributions easier, and it has
>> proven its fitness for this task in other open source projects such as Git.
>
> I doubt very much that people cannot read or write mdoc(7).
> As for contributions, does that mean people contributed
> those asciidoc rewriter to macports?

Clemens contributed most of the asciidoc rewrites of the manpages.

> Anyway, the mdoc(7) rewrite is apparently not happening.

Right; the asciidoc rewrite of the manpages was just released to the public in MacPorts 2.4 so there's probably no interest in changing it again so soon.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: manpages is mdoc(7)

Jan Stary
On Mar 31 23:07:06, [hidden email] wrote:
> > Anyway, the mdoc(7) rewrite is apparently not happening.
>
> Right; the asciidoc rewrite of the manpages was just released to the public in MacPorts 2.4 so there's probably no interest in changing it again so soon.

But the age of the asciidoc rewrite has nothing to do with it, right?

I'm not hoping to change the course here,
but what were the manpages written in before this?
porthier.7 is in mdoc(7), with .Dd June 1, 2007
- were all the base manpages im mdoc(7) before?

Now that they are in asciidoc,

 * the actual man(7)page needs to be generated
 * the generating requires horrendous xsl transformations
 * both the asciidoc source and the generated man(7) need to be in the repo
 * the result is this:

'\" t
.TH "PORT\-CAT" "1" "2016\-11\-06" "MacPorts 2\&.4\&.99" "MacPorts Manual"
.\" -----------------------------------------------------------------
.\" * Define some portability stuff
.\" -----------------------------------------------------------------
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.\" http://bugs.debian.org/507673
.\" http://lists.gnu.org/archive/html/groff/2009-02/msg00013.html
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
[...]


That's right: let's start in each and every manpage
with a workaround to a 2009 bug in docbook-xsl.

I just can't believe how this can be viewed as better
than simply having mdoc(7) manpages which work by themselves,
have been around and supported for decades, and are a clear
and concise description of the semantic.

        Jan

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: manpages is mdoc(7)

Rainer Müller-4
On 2017-04-01 10:28, Jan Stary wrote:
> But the age of the asciidoc rewrite has nothing to do with it, right?
>
> I'm not hoping to change the course here,
> but what were the manpages written in before this?
> porthier.7 is in mdoc(7), with .Dd June 1, 2007
> - were all the base manpages im mdoc(7) before?

Back when the NewHelpSystem [1] was started, the man pages were in roff
format and rarely received updates. No idea if it was mdoc(7), because
for someone not knowing the syntax at all, it looks equally bad.

> Now that they are in asciidoc,
>
>  * the actual man(7)page needs to be generated
>  * the generating requires horrendous xsl transformations

Why are generated man pages a problem? I fully agree that XSL is
horrendous, but we do not maintain this XSL, it is provided by DocBook.

Many open source projects generate their man pages from a high-level
markup language. I am only aware of the various *BSD systems that keep
writing roff directly.

>  * both the asciidoc source and the generated man(7) need to be in the repo
>  * the result is this:

[...]

> That's right: let's start in each and every manpage
> with a workaround to a 2009 bug in docbook-xsl.

Who cares? Nobody looks at the roff input...

Do you also look at generated HTML in your browser and complain about
all the hacks and workarounds that are necessary for certain browsers?

Rainer

[1] https://trac.macports.org/wiki/NewHelpSystem
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: manpages is mdoc(7)

Michael_google gmail_Gersten
In reply to this post by Rainer Müller-4
My ... (if 2 bits is 25 cents, then 0.2 bits is 2.5 cents, ... ok, call it inflation) 0.2 bits on this subject:

In the past, I attempted to write stuff with man page macros. I could not find any actual docs on the macros or how to use them.

I found that there was no consistent style or rules from reverse engineering the shipped system man pages.

After a lot of attempts to decipher rules, find what seemed to be non-existent docs, etc., I came away with the following:

As long as it looked close enough, that was good enough.

If you are going to talk about re-writing something in a new language, it isn't enough to say that it is documented; point out the docs.

If you are going to say that "Oh, this language is better because it makes semantic separation, and you can make it look appropriate yourself", make sure you are not falling into the stupidity of the HTML to CSS trap -- going from "Here's a simple, uniform, consistent way to say bold or underline" to "Here's a site dependent way to format, that will differ from site to site, that isn't even easy for an end user to adjust or change, that tries to make your monitor look like the dev's monitor even if the two monitors are completely different in all aspects, that changes the rules for scrolling, for letting people see stuff, that leaks privacy data, that while it can say "this is a paragraph" or "this is a link" or "this is someone's name" or "this is a video", etc., in reality everyone wants to declare different things in different ways, and the end user only gets one override style sheet that if used will replace everything else by everyone making it completely useless in the end".

Yes, I'm bitter about that.

Seriously: Being able to say "This is to be emphasized" or "This is to be stressed" or "This is to be ignored" is not really much different than "This is to be bold" or "this is to be italic" or "this is to be strikethrough"; or having a display translate strikethrough into greyed-out because it can't do strikethrough.

But being able to arbitrarily declare "this is a new class of thing", and then require a rule for how to display thing before it displays correctly because lacking that CSS it now displays wrong?

Man page rules allowed specifying the important stuff -- standard section headers, command name, etc. The language was badly documented, and not really consistently templated. Gnu switched to a "info" language. Apple just made PDF's from their own internal stuff and didn't even bother with man pages. Etc.

> Do you also look at generated HTML in your browser and complain about all the hacks and workarounds that are necessary for certain browsers?

I've had to go into source mode / firebug / Stylish / other tools and *FIX* things to work with my browser.
So yes.

===

The TL;DR: Based on what has happened historically, it really does look like the -man macros for roff don't provide enough function and are too hard to use for what is provided. There's 14 competing standards, and a 15th won't do better (ref XKCD). Use something that is usable if displayed raw, that has conversions into other formats (such as -man), and doesn't fall into the "everything behaves differently" trap.

An ascii-based rewrite sounds like it fits that.
A UTF-based rewrite might help, but it also opens up the horrors of a page that cannot be displayed on an ascii screen at all using nothing but foreign alphabet stuff.
And an extended ascii rewrite (providing standard accents, tilde-n, etc) might be a "good enough" compromise.

Heck, programming languages went through the sequence of "very limited" to "can do anything and everything" to "no gotos, this set of control flows". Maybe we should be doing something similar with formatting rules by now?

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: manpages is mdoc(7)

Jan Stary
In reply to this post by Rainer Müller-4
On Apr 01 14:24:42, [hidden email] wrote:
> > I'm not hoping to change the course here,
> > but what were the manpages written in before this?
> > porthier.7 is in mdoc(7), with .Dd June 1, 2007
> > - were all the base manpages im mdoc(7) before?
>
> Back when the NewHelpSystem [1] was started, the man pages were in roff
> format and rarely received updates. No idea if it was mdoc(7), because
> for someone not knowing the syntax at all, it looks equally bad.

Let me give a very short xample trying to illustrate the difference.
This is a line from the current port-load(1).

  \fBport\fR [\fB\-D\fR \fIportdir\fR]

What it says, in low-level roff typesetting commands, is this:

  Switch to bold and type "port", then witch back to roman.
  Type a left bracket, switch to bold, type "-D' and switch back to roman.
  hen switch to italics, type "portdir", swith back to roman,
  and type a right bracket.

An equivaent in mdoc(7) is this:

  .Nm
  .Op Fl D Ar portdir

which says, describing clearly the semantics:

  The utility takes an optional flag 'D' with a 'portdir' argument.

That's not "equally bad", that fundamentally better.
That's why I offered to rewrite the manpages into mdoc(7).

> > Now that they are in asciidoc,
> >  * the actual man(7)page needs to be generated
> >  * the generating requires horrendous xsl transformations
>
> Why are generated man pages a problem?

It's not necessarily a problem, but it is
a disadvantage with respect to having a simple manpage.1

> I fully agree that XSL is
> horrendous, but we do not maintain this XSL, it is provided by DocBook.

How is "it's horrendous, but we did not write it"
a reason to use horrendous software?

> Many open source projects generate their man pages from a high-level
> markup language. I am only aware of the various *BSD systems that keep
> writing roff directly.

They do _not_ use roff. Thats the whole point.
There is not a single roff command in the entirety
of e.g. the OpenBSD base system.

> >  * both the asciidoc source and the generated man(7) need to be in the repo
> >  * the result is this:
>
> [...]
>
> > That's right: let's start in each and every manpage
> > with a workaround to a 2009 bug in docbook-xsl.
>
> Who cares? Nobody looks at the roff input...

I care. The manpage formatter cares,
and has to _parse_ the roff input .

> Do you also look at generated HTML in your browser and complain about
> all the hacks and workarounds that are necessary for certain browsers?

Of course not. But that does not make shitty html page equal
to a good html page _even_if_they_looked_almost_the_same in the
browser (to stick to your analogy which does not really work).

        Jan

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: manpages is mdoc(7)

Jan Stary
> On Apr 01 14:24:42, [hidden email] wrote:
> > Many open source projects generate their man pages from a high-level
> > markup language. I am only aware of the various *BSD systems that keep
> > writing roff directly.

Just to clarify:

* no-one is advocating "writing roff directly".
* mdoc(7) is not roff

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: manpages is mdoc(7)

Jan Stary
In reply to this post by Michael_google gmail_Gersten
On Apr 01 09:24:22, [hidden email] wrote:
> In the past, I attempted to write stuff with man page macros.
> I could not find any actual docs on the macros or how to use them.
> I found that there was no consistent style or rules
> from reverse engineering the shipped system man pages.

Whether you mean the man(7) macros or the mdoc(7) macros,
they have been around and clearly documented for decades,
on each and every UNIX system. To quote http://man.openbsd.org/mdoc

  The mdoc language first appeared as a troff macro package in 4.4BSD.
  It was later significantly updated by Werner Lemberg and Ruslan Ermilov
  in groff-1.17.

(Seriously, are you pretending to not know that?)

> After a lot of attempts to decipher rules,
> find what seemed to be non-existent docs, etc.,
> I came away with the following:
> As long as it looked close enough, that was good enough.

That's the impression I got so far in the reactions.

So let's use the horrendous docbook-xsl to produce
low-level roff of abysmal quality - it still looks close enough.

> If you are going to talk about re-writing something in a new language,
> it isn't enough to say that it is documented; point out the docs.

The new language is as old as 4.4BSD,
and the documentation for it is exactly as old.

> If you are going to say that "Oh, this language is better
> because it makes semantic separation, and you can make it
> look appropriate yourself", make sure you are not falling
> into the stupidity of the HTML to CSS trap
> -- going from "Here's a simple, uniform, consistent way to say
> bold or underline" to "Here's a site dependent way to format,
> that will differ from site to site, that isn't even easy for
> an end user to adjust or change, that tries to make your monitor
> look like the dev's monitor even if the two monitors are completely
> different in all aspects, that changes the rules for scrolling,
> for letting people see stuff, that leaks privacy data,
> that while it can say "this is a paragraph" or "this is a link"
> or "this is someone's name" or "this is a video", etc.,
> in reality everyone wants to declare different things in different ways,
> and the end user only gets one override style sheet that
> if used will replace everything else by everyone
> making it completely useless in the end".

Could you please elaborate on each of those points, specifically?
How is mdoc(7) "site dependent"? What "rules for scrolling"? etc.
How does it "leak privacy data"? (And what what "privacy data"
are in the manpages?) etc

> Yes, I'm bitter about that.

Apparently.

> Seriously: Being able to say "This is to be emphasized"
> or "This is to be stressed" or "This is to be ignored"
> is not really much different than "This is to be bold"
> or "this is to be italic" or "this is to be strikethrough"
> or having a display translate strikethrough into greyed-out
> because it can't do strikethrough.

That's a strawman attack. mdoc(7) doesn't do _any_ of that.
On the contrary, it has annotations for "this is an optional argument",
"this is a variable name" etc - precisely the things you need to say
in a manpage for a piece of software.

> Man page rules allowed specifying the important stuff -- standard section headers, command name, etc. The language was badly documented,

No it wasn't, and isn't.

> and not really consistently templated. Gnu switched to a "info" language.
> Apple just made PDF's from their own internal stuff and didn't even bother
> with man pages.

What's that nine thousand files doing
in my OSX's /usr/share/man then?

> The TL;DR: Based on what has happened historically, it really does look like the -man macros for roff don't provide enough function and are too hard to use for what is provided.

That's why I suggest to use mdoc(7), as opposed to man(7),
to be used for the manpages.

> Use something that is usable if displayed raw,
> that has conversions into other formats (such as -man),

From the mdoc(7) source, the following are easily generated
using either gorff or mandoc:

ascii
html
man
markdown
pdf
ps

> and doesn't fall into the "everything behaves differently" trap.

I don't know what you mean by that,
and how mdoc(7) "falls into that trap".

> An ascii-based rewrite sounds like it fits that.

mdoc(7) does not fit that by definition, if you want
something that is usable if displayed raw,
which rules out any kind of markup.

        Jan

Loading...