makefs: add msdosfs support

[emaste]

Summary

Rebased patch from Siva Mahadevan. Unlike previous work this duplicates the msdosfs kernel files in usr.sbin/makefs/msdos as requested in D11197.

I would prefer that we share common files between kernel and userland, but in the interest of expediency am fine with pursuing this approach for now, perhaps followed by careful deduplication.

[emaste]

usr.sbin/makefs/ffs/buf.h

48–50 ↗

(On Diff #45825)

Changes in this file seem to be whitespace/style; if we're going to include them, commit them separately.

usr.sbin/makefs/makefs.8

38

Bump .Dd on commit

242

included in although this could probably be clarified, and out to be committed (referencing only ffs) independently first.

usr.sbin/makefs/makefs.c

78

alpha order

usr.sbin/makefs/msdos.h

46–47

alpha order

usr.sbin/makefs/msdos/Makefile.inc

5

I think this path should not be included

usr.sbin/makefs/msdos/denode.h

1

Copy of sys/fs/msdosfs/denode.h with #ifdef KERNEL block removed. This one is likely easily shared.

usr.sbin/makefs/msdos/direntry.h

1

Copy of sys/fs/msdosfs/denode.h with #ifdef KERNEL block changed.

usr.sbin/makefs/msdos/fat.h

1

sys/fs/msdosfs/fat.h modulo _KERNEL

usr.sbin/makefs/msdos/msdosfs_conv.c

2

rather different

usr.sbin/makefs/msdos/msdosfs_fat.c

1

modified sys/fs/msdosfs/msdosfs_fat.c

[emaste]

• Exclude buf.h changes
• Rebase on newly copied sys/fs/msdosfs files

[emaste]

usr.sbin/makefs/msdos/msdosfs_conv.c

2

Our kernel version of this file sys/fs/msdosfs/msdosfs_conv.c corresponds to NetBSD sys/msdosfs/msdosfs_conv.c 1.25
https://github.com/NetBSD/src/commit/9ee2678cd5ee833080978bd635bb22fbf0ef7839

I'm not sure where the version of this file in this review comes from (specifically, I can't seem to find a copy of this with a 1994 Copyright).

For reference, versions from other BSDs:

NetBSD (sys/fs/msdosfs/msdosfs_conv.c)

/*      $NetBSD: msdosfs_conv.c,v 1.17 2016/06/30 09:34:01 nonaka Exp $ */

/*-
 * Copyright (C) 1995, 1997 Wolfgang Solfrank.
 * Copyright (C) 1995, 1997 TooLs GmbH.
 * All rights reserved.
 * Original code by Paul Popelka ([e-mail]) (see below).

OpenBSD (sys/msdosfs/msdosfs_conv.c)

/*      $OpenBSD: msdosfs_conv.c,v 1.19 2015/10/23 10:45:31 krw Exp $   */
/*      $NetBSD: msdosfs_conv.c,v 1.24 1997/10/17 11:23:54 ws Exp $     */

/*-
 * Copyright (C) 1995, 1997 Wolfgang Solfrank.
 * Copyright (C) 1995, 1997 TooLs GmbH.
 * All rights reserved.
 * Original code by Paul Popelka ([e-mail]) (see below).

OpenBSD (usr.sbin/makefs/msdos/msdosfs_conv.c)

/*      $OpenBSD: msdosfs_conv.c,v 1.1 2016/10/18 17:05:30 natano Exp $ */
/*      $NetBSD: msdosfs_conv.c,v 1.24 1997/10/17 11:23:54 ws Exp $     */

/*-
 * Copyright (C) 1995, 1997 Wolfgang Solfrank.
 * Copyright (C) 1995, 1997 TooLs GmbH.
 * All rights reserved.
 * Original code by Paul Popelka ([e-mail]) (see below).

FreeBSD (sys/fs/msdosfs/msdosfs_conv.c)

/* $FreeBSD$ */
/*      $NetBSD: msdosfs_conv.c,v 1.25 1997/11/17 15:36:40 ws Exp $     */

/*-
 * SPDX-License-Identifier: BSD-4-Clause
 *
 * Copyright (C) 1995, 1997 Wolfgang Solfrank.
 * Copyright (C) 1995, 1997 TooLs GmbH.
 * All rights reserved.
 * Original code by Paul Popelka ([e-mail]) (see below).

[emaste]

usr.sbin/makefs/msdos/msdosfs_conv.c

195–196

This change was introduced in NetBSD msdosfs_conv.c 1.17, so this file appears to be a modified copy of the latest available in NetBSD.

Compared to that version, this has:

• some functions reordered
• unix2winfn added from somewhere (this may explain the modified copyright information)
• use of C99 types (uint16_t instead of u_int16_t)
• miscellaneous header changes
• dos2unix[] array removed
• size_t instead of int for sizes
• some UTF-related changes

[cem]

I'm not really a fan of this duplicating code approach. Despite the pain that building code in both the kernel and userspace can occasionally cause, I think it's worth it, at least here. How much work does msdosfs really see in 2018? kib's example from the other revision is almost two years old. While I'm sure there have been more recent examples, I don't think the degree of pain is so high as to overcome the alternative cost of maintaining duplicate code.

[emaste]

Updated patch that shares kernel headers, based on part of D11197

[emaste]

It's possible to build makefs FAT support now:

1. apply D16526 (header modifications)
2. copy msdosfs_fat.c and msdosfs_lookup.c from sys/fs/msdosfs to usr.sbin/makefs/msdos
3. apply D16438

I propose that we proceed with this approach, and subsequently update the kernel msdosfs support and deduplicate.

[cem]

This is a pretty big patch. At least some of this appears to be refactoring. Can that be broken out and committed separately? Any other small pieces possible to break out would help, too.

It's still unclear to me why we're copying files from the kernel.

I'm not sure if we lost Unicode support with this change or not, but msdos with LFN supports 255 UCS-2 character filenames (i.e., BMP only Unicode).

usr.sbin/makefs/msdos.c

83

Not UINT64?

95–96

why'd we lose this?

105–106

This rename seems spurious

139

gratuitous rename

[emaste]

usr.sbin/makefs/msdos.c

139

hrm, not sure how Siva ended up with this change; in any case it's not in NetBSD so will revert
http://cvsweb.netbsd.org/bsdweb.cgi/src/usr.sbin/makefs/msdos.c?annotate=1.20&only_with_tag=MAIN

[emaste]

Undo renames pointed out by @cem

[emaste]

usr.sbin/makefs/msdos.c

83

.maximum is a long long, not sure what the idea was before

95–96

not sure

[cem]

usr.sbin/makefs/msdos.c

83

Is there any reason for it to be a signed type? Not that it matters too much for this use.

[emaste]

rebase

[emaste]

usr.sbin/makefs/msdos.c

95–96

It is defined in NetBSD's sys/fs/msdosfs/msdosfsmount.h

[emaste]

rebase and remove extraneous changes

[emaste]

usr.sbin/makefs/makefs.8

230–242

Done in rS351232, need to rebase this review.

usr.sbin/makefs/makefs.h

186–187

these should probably be in alpha order (as a separate commit)

usr.sbin/makefs/msdos.c

105–106

Indeed, I'm not sure if @smahadevan_freebsdfoundation.org perhaps ended up with an older version of the NetBSD code in this change? In any case, this should be undone, assuming the current NetBSD code has msdos_opt.

[imp]

one last nit.
Still object to copying, but since no progress on de-dupe has been made and there's been a 2 year lag, I accept it's the last bad choice to move this forward.

usr.sbin/makefs/Makefile

11

nit: sort alphabetically, one per line.

[cem]

I still don't like the duplication, but otherwise can't object. (My approval is contingent on Warner's remarks.)