ls(1): Change -g and make -n imply -l to implement POSIX

[minsoochoo0122_proton.me]

Summary

Bugzilla bug report in 2004

According to SUSv4 and POSIX, -g, -n, and -o options are required for ls.
This has not been implemented for the compatibility for 4.3 BSD, but I think there is no need to insist it anymore.

f_flags is deleted from case 'o' because file flags are not displayed in the -o option of POSIX.

Test Plan

cd /usr/src/bin/ls
make && make install

[minsoochoo0122_proton.me]

bin/ls/ls.c

378–380

I fixed this by editing usage

378–380

I modified getopt_long on the 280th line.

[minsoochoo0122_proton.me]

• update ls man page

[minsoochoo0122_proton.me]

• change -o to -O in files in src tree

[minsoochoo0122_proton.me]

In D34747#790015, @pauamma_gundo.com wrote:

As disgorged by grep -rH -E 'ls +-[a-zA-A0-9]*o' /usr/src, the following files in src also need changes:
/usr/src/share/skel/dot.shrc:alias ll='ls -laFo'
/usr/src/sys/contrib/zstd/tests/gzip/init.sh: perms=\ls -dgo "$d" 2>/dev/null\ &&
/usr/src/bin/ls/tests/ls_tests.sh: atf_set "descr" "Verify that the output from ls -o prints out the chflag values or '-' if none are set"
/usr/src/bin/ls/tests/ls_tests.sh: -s exit:0 ls -lo a.file
/usr/src/bin/ls/tests/ls_tests.sh: -s exit:0 ls -lo b.file
/usr/src/bin/chflags/chflags.1:You can use "ls -lo" to see the flags of existing files.

Nothing obvious in the ports tree, but that doesn't include actual source code, so I think an exp-run is in order. There's one ls -lo in the handbook (I haven't checked the rest of the doc tree). And of course, that doesn't and can't account for all the unported apps and homegrown ops or devops shell programs out there. With that in mind, I suggest a staged change instead, with introducing -O and adding a deprecation message to -o as the first stage.

Thank you for your help. I edited most of the files, but I cannot find "/usr/src/sys/contrib/zstd/tests/gzip/init.sh".

[minsoochoo0122_proton.me]

• present a deprecation message to -o option

[pauamma_gundo.com]

In D34747#790074, @fel1x.mintchoco.development_gmail.com wrote:

Thank you for your help. I edited most of the files, but I cannot find "/usr/src/sys/contrib/zstd/tests/gzip/init.sh".

Once I checked 14-current source as of 12ish hours ago instead of 12.2 source (I thought i was, but I must have gotten confused), I noticed that file is gone indeed, but I found 2 more:

sys/contrib/openzfs/tests/zfs-tests/tests/functional/acl/off/dosmode.ksh; sys/contrib/openzfs/tests/zfs-tests/tests/functional/acl/off/dosmode.ksh: ls -lo $path | awk '{ gsub(",", "\n", $5); print $5 }' | \
tools/test/stress2/misc/fts2.sh; sprintf(help, "ls -lo %s", p->fts_path);

[minsoochoo0122_proton.me]

• change -o to -O in files in src tree

[pauamma_gundo.com]

LGTM as far as the manual page and the use of -o/-O in src. I'll leave deciding whether an exp-run is needed to others.

[jrtc27]

I'm concerned about changing the interface in a backwards-incompatible manner for something as core as ls but will leave that to others

bin/ls/ls.c

439

Well it doesn't behave the same as -O, so suggesting that seems wrong. Printing warnings that can't be disabled is also often not well received, and it should definitely not go to stdout.

This line is also way too long for the column limit.

bin/ls/ls.h

57

f_sgroup does not show the name of the group, it _disables_ showing the name of the group, which is the opposite of the other variables here, and the opposite of what your comment says

bin/ls/print.c

238

You're missing a space after the comma

[minsoochoo0122_proton.me]

• fix spacing syntax

[minsoochoo0122_proton.me]

• better comment

[minsoochoo0122_proton.me]

bin/ls/ls.c

439

Then how about adding a warning message in man page, not through stdout?

[pauamma_gundo.com]

bin/ls/ls.c

439

Is the manual page visible enough? (But if the deprecation message stays in source,, yes please to stderr.)

[minsoochoo0122_proton.me]

• print warning message to stderr

[pauamma_gundo.com]

Manual page and UX both LGTM. Still needs src review.

[pstef]

bin/ls/ls.c

439

Recently some systems started warning about fgrep being deprecated (this is GNU's doing) and to me that's infuriating. So I vote for moving this information to the manual page only.

[minsoochoo0122_proton.me]

bin/ls/ls.c

439

Thank you for your suggestion. I like this idea. I removed warning message in stderr and moved it to the man page.

[minsoochoo0122_proton.me]

I reflected all the suggestions and it seems that the reviewers don't have further opinion on this. I'll accept this revision just for now, but if this revision needs any modification, please I would like to hear your opinions.

[carlavilla]

@emaste @jhb can you please take a look and give the approval from src?

[imp]

I think this is a bad idea. It's one of the cases where in the past we've made a conscious decision to deviate from POSIX and I see little to no value to our users here. These are little used options there's not a lot of demand for and at the same time, it breaks compatibility with old code. On the whole, I think that situation would suggest we do not want this.

[minsoochoo0122_proton.me]

In D34747#898852, @imp wrote:

I think this is a bad idea. It's one of the cases where in the past we've made a conscious decision to deviate from POSIX and I see little to no value to our users here. These are little used options there's not a lot of demand for and at the same time, it breaks compatibility with old code. On the whole, I think that situation would suggest we do not want this.

I don't understand what you mean.

FreeBSD is an Open Source, standards-compliant Unix-like operating system

source: FreeBSD Handbook

If this revision is not accepted because FreeBSD has decided not to follow POSIX, then why do you call it "standards-compliant operating system"?

[rpokala]

bin/ls/ls.1

350 ↗

(On Diff #112480)

I think it would be clearer if the second sentence looked more like this:

This behavior changed in
.Fx 14 ,
to conform to
.St -p1003.1-2008 ;
for the previous behavior of displaying file flags, use
.Pq Fl O .

[jhb]

In D34747#898912, @fel1x.mintchoco.development_gmail.com wrote:

In D34747#898852, @imp wrote:

I think this is a bad idea. It's one of the cases where in the past we've made a conscious decision to deviate from POSIX and I see little to no value to our users here. These are little used options there's not a lot of demand for and at the same time, it breaks compatibility with old code. On the whole, I think that situation would suggest we do not want this.

I don't understand what you mean.

Breaking people's existing scripts (and with a tool as core as ls people will have scripts that invoke and and parse its output) is a cost. The amount of the cost is relative, but in this case it's a widely used tool (ls) and it's a feature that is also used as it has no standard alternative (there is no way in POSIX to list file flags, so any existing scripts fetching file flags via ls are using this). Those both make the cost higher than in other cases of changing behavior.

FreeBSD is an Open Source, standards-compliant Unix-like operating system

source: FreeBSD Handbook

If this revision is not accepted because FreeBSD has decided not to follow POSIX, then why do you call it "standards-compliant operating system"?

Because FreeBSD's usage here (and BSD in general) pre-dates the standard. All of the UNIX-like OS's have extensions to and deviations from POSIX in part because POSIX itself was originally an attempt to harmonize differences between the BSD and SYSV "forks" of UNIX. FWIW, both NetBSD and OpenBSD implement the compromise I was going to suggest which is to implement -g and -n per POSIX, but leave -o as the custom BSD option that lists file flags.

[imp]

In D34747#900964, @jhb wrote:

Because FreeBSD's usage here (and BSD in general) pre-dates the standard. All of the UNIX-like OS's have extensions to and deviations from POSIX in part because POSIX itself was originally an attempt to harmonize differences between the BSD and SYSV "forks" of UNIX. FWIW, both NetBSD and OpenBSD implement the compromise I was going to suggest which is to implement -g and -n per POSIX, but leave -o as the custom BSD option that lists file flags.

I like the NetBSD/OpenBSD solution. It breaks nothing, increases standards conformance and lets us have one BSD standard for this.

I'd like to also point out that the -o thing likely is keyed off matching either FreeBSD or *BSD in scripts, so people already cope with the varying standards for this.

[minsoochoo0122_proton.me]

In D34747#903407, @imp wrote:

In D34747#900964, @jhb wrote:

Because FreeBSD's usage here (and BSD in general) pre-dates the standard. All of the UNIX-like OS's have extensions to and deviations from POSIX in part because POSIX itself was originally an attempt to harmonize differences between the BSD and SYSV "forks" of UNIX. FWIW, both NetBSD and OpenBSD implement the compromise I was going to suggest which is to implement -g and -n per POSIX, but leave -o as the custom BSD option that lists file flags.

I like the NetBSD/OpenBSD solution. It breaks nothing, increases standards conformance and lets us have one BSD standard for this.

I'd like to also point out that the -o thing likely is keyed off matching either FreeBSD or *BSD in scripts, so people already cope with the varying standards for this.

I want to try this:

# if __POSIX_VISIBLE >= 200809

Can this macro work in this situation?

[jrtc27]

In D34747#910651, @fel1x.mintchoco.development_gmail.com wrote:

In D34747#903407, @imp wrote:

In D34747#900964, @jhb wrote:

Because FreeBSD's usage here (and BSD in general) pre-dates the standard. All of the UNIX-like OS's have extensions to and deviations from POSIX in part because POSIX itself was originally an attempt to harmonize differences between the BSD and SYSV "forks" of UNIX. FWIW, both NetBSD and OpenBSD implement the compromise I was going to suggest which is to implement -g and -n per POSIX, but leave -o as the custom BSD option that lists file flags.

I like the NetBSD/OpenBSD solution. It breaks nothing, increases standards conformance and lets us have one BSD standard for this.

I'd like to also point out that the -o thing likely is keyed off matching either FreeBSD or *BSD in scripts, so people already cope with the varying standards for this.

I want to try this:

# if __POSIX_VISIBLE >= 200809

Can this macro work in this situation?

No, for multiple reasons.

[imp]

In D34747#910651, @fel1x.mintchoco.development_gmail.com wrote:

In D34747#903407, @imp wrote:

In D34747#900964, @jhb wrote:

Because FreeBSD's usage here (and BSD in general) pre-dates the standard. All of the UNIX-like OS's have extensions to and deviations from POSIX in part because POSIX itself was originally an attempt to harmonize differences between the BSD and SYSV "forks" of UNIX. FWIW, both NetBSD and OpenBSD implement the compromise I was going to suggest which is to implement -g and -n per POSIX, but leave -o as the custom BSD option that lists file flags.

I like the NetBSD/OpenBSD solution. It breaks nothing, increases standards conformance and lets us have one BSD standard for this.

I'd like to also point out that the -o thing likely is keyed off matching either FreeBSD or *BSD in scripts, so people already cope with the varying standards for this.

I want to try this:

# if __POSIX_VISIBLE >= 200809

Can this macro work in this situation?

That only controls what posix C functions are visible. It is not a conformance level relating to how commands work. Also, even if it were the consensus of the review is that the incompatible change is a hard no.

[minsoochoo0122_proton.me]

In D34747#910653, @imp wrote:

In D34747#910651, @fel1x.mintchoco.development_gmail.com wrote:

In D34747#903407, @imp wrote:

In D34747#900964, @jhb wrote:

Because FreeBSD's usage here (and BSD in general) pre-dates the standard. All of the UNIX-like OS's have extensions to and deviations from POSIX in part because POSIX itself was originally an attempt to harmonize differences between the BSD and SYSV "forks" of UNIX. FWIW, both NetBSD and OpenBSD implement the compromise I was going to suggest which is to implement -g and -n per POSIX, but leave -o as the custom BSD option that lists file flags.

I like the NetBSD/OpenBSD solution. It breaks nothing, increases standards conformance and lets us have one BSD standard for this.

I'd like to also point out that the -o thing likely is keyed off matching either FreeBSD or *BSD in scripts, so people already cope with the varying standards for this.

I want to try this:

# if __POSIX_VISIBLE >= 200809

Can this macro work in this situation?

That only controls what posix C functions are visible. It is not a conformance level relating to how commands work. Also, even if it were the consensus of the review is that the incompatible change is a hard no.

Okay, I will work on reverting -o option and update diff file.

[minsoochoo0122_proton.me]

Updated patch file

[jhb]

bin/ls/print.c

241

Nothing in the new patch ever sets f_sgroup now, so I think you can remove that variable and make this printf unconditional?

[pauamma_gundo.com]

English in manual page still LGTM.

[jhb]

The log message should be updated to reflect the effective changes now which I think is roughly "change -g to implement POSIX" and "make -n imply -l".

[minsoochoo0122_proton.me]

Update commit message to ls(1): Change -g and make -n imply -l to implement POSIX

[jhb]

Can you confirm your author details, my guess is "Minsoo Choo <[e-mail]>"?

[minsoochoo0122_proton.me]

In D34747#934900, @jhb wrote:

Can you confirm your author details, my guess is "Minsoo Choo <[e-mail]>"?

Hi,

I'm switched my email address to [e-mail].

[jhb]

Ah, this adds at least one test failure for -g which I will fix before pushing.