Add Semantic Line Break checking with vale

[bofh]

Summary

In our doc guidelines we use "One Sentence Per Line" or more technically Semantic Line Breaking. Use vale to implement this.

[salvadore]

I have tested it successfully on this quarter status reports.

Two suggestions for improvements:

• I had two false positives due to abbreviations (Inc. and e.g.);
• it is a bit annoying that it always stops analyzing files at the first error, it would be nicer if all wrong lines were reported at once. Maybe changing level: error to something lower would be a good idea.

[salvadore]

Maybe also remove max: 1.

[salvadore]

In D39560#900629, @salvadore wrote:

Maybe also remove max: 1.

No, sorry: this must stay. I thought it refered to how many times the rule was allowed to fail before stopping checking the file, but I realized that it counts the number of sentences instead.

[bofh]

In D39560#900625, @salvadore wrote:

I have tested it successfully on this quarter status reports.

Two suggestions for improvements:

• I had two false positives due to abbreviations (Inc. and e.g.);

Ah yes this is fixable. I will need to add another rule.

• it is a bit annoying that it always stops analyzing files at the first error, it would be nicer if all wrong lines were reported at once. Maybe changing level: error to something lower would be a good idea.

Oh damn yeah. I never meant to set it to error but warning. I will update the diff.

[bofh]

In D39560#900668, @salvadore wrote:

In D39560#900629, @salvadore wrote:

Maybe also remove max: 1.

No, sorry: this must stay. I thought it refered to how many times the rule was allowed to fail before stopping checking the file, but I realized that it counts the number of sentences instead.

Yes this counts the number of sentences.

[grahamperrin]

In D39560#900625, @salvadore wrote:

I have tested it successfully on this quarter status reports. …

Can you recall, were there any multi-sentence list items (bullet points)?

Thinking back a few months … my perceptions of ambiguity might have differed from other people's perceptions.

[salvadore]

In D39560#901131, @grahamperrin wrote:

In D39560#900625, @salvadore wrote:

I have tested it successfully on this quarter status reports. …

Can you recall, were there any multi-sentence list items (bullet points)?

Thinking back a few months … my perceptions of ambiguity might have differed from other people's perceptions.

Yes, we have some in this quarter FreeBSD Foundation status report, which I am fixing in https://reviews.freebsd.org/D39567#change-bq5lJMuUrbA0.

[grahamperrin]

In D39560#901262, @salvadore wrote:

… fixing in https://reviews.freebsd.org/D39567#change-bq5lJMuUrbA0.

Thanks. Running vale here (with diff 120268), I see that it catches line 100, but not the list items.

[salvadore]

In D39560#901276, @grahamperrin wrote:

In D39560#901262, @salvadore wrote:

… fixing in https://reviews.freebsd.org/D39567#change-bq5lJMuUrbA0.

Thanks. Running vale here (with diff 120268), I see that it catches line 100, but not the list items.

Have you tried switching from level: error to level: warning as suggested in the above comments?

[grahamperrin]

In D39560#901277, @salvadore wrote:

Have you tried switching from level: error to level: warning as suggested in the above comments?

Phabricator hid the comments, thanks for the hint.

No difference:

% vale website/content/en/status/report-2023-01-2023-03/freebsd-foundation.adoc | grep sentence
 110:98   warning  Only use one sentence per       FreeBSD.SemanticLineBreak 
% grep level .vale/styles/FreeBSD/SemanticLineBreak.yml
level: warning
%

I guess, bullet points (or list items or whatever) are exempt from paragraph scope, even when the points are integral to what the human sees as a paragraph.

From https://github.com/errata-ai/vale/discussions/512#discussioncomment-4112643:

You could also use script for more fine-grained control.

I'm not troubled by the bullet point example, just curious.

[salvadore]

Since I remember I had spotted those multisentences list items through vale, I have tested it again. I confirm what @grahamperrin reports: even with level: warning it only reports line 110. I add that the same happens with level: suggestion.

However, if line 110 is fixed, then vale reports line 89. When line 89 is fixed, it reports line 91 and so on. So, for some reason, vale applies the rule only once.

[grahamperrin]

In D39560#901295, @salvadore wrote:

… if line 110 is fixed, then vale reports line 89. When line 89 is fixed, it reports line 91 and so on. So, for some reason, vale applies the rule only once.

% vale documentation/content/en/books/handbook/cutting-edge/_index.adoc | grep Semantic
 801:69   warning  Only use one sentence per       FreeBSD.SemanticLineBreak 
%

This fails to catch at least two occurrences earlier in the file. https://github.com/freebsd/freebsd-doc/pull/164/commits/6fd0575ff225da6cb748f248b01d0c280ab29302:

• lines 250 and 254.

[grahamperrin]

Toying with scopes:

% vale documentation/content/en/books/handbook/cutting-edge/_index.adoc | grep Semantic
 250:90   warning  Only use one sentence per       FreeBSD.SemanticLineBreak 
% grep scope .vale/styles/FreeBSD/SemanticLineBreak.yml
scope: summary
% nano /usr/doc/.vale/styles/FreeBSD/SemanticLineBreak.yml
% grep scope .vale/styles/FreeBSD/SemanticLineBreak.yml
scope: paragraph
% vale documentation/content/en/books/handbook/cutting-edge/_index.adoc | grep Semantic
 801:69   warning  Only use one sentence per       FreeBSD.SemanticLineBreak 
%

[bofh]

Modify warning level to suggestion

[bofh]

There is another scope called list which might be helpful for bullet points.

[bofh]

Handle semantic line breaking in lists.

[bofh]

In D39560#901295, @salvadore wrote:

Since I remember I had spotted those multisentences list items through vale, I have tested it again. I confirm what @grahamperrin reports: even with level: warning it only reports line 110. I add that the same happens with level: suggestion.

However, if line 110 is fixed, then vale reports line 89. When line 89 is fixed, it reports line 91 and so on. So, for some reason, vale applies the rule only once.

Unless you have already committed 89/90/110 now it should display all the suggestion. :D

[salvadore]

I have checked out the repository to a version before the commit and tested. I confirm that it now works as expected, showing lines 89, 90, 91 and 110 all at once, thanks @bofh.

[grahamperrin]

Thanks, I have not yet tested diff 120428.

Also, please see the suggestion at https://github.com/errata-ai/vale/discussions/512#discussioncomment-5643950.

[bofh]

@salvadore Somewhere mentioned about some exceptions with Inc. and e.g. which will contain . but not apparently at the end of line. This patch handles this exception.

@grahamperrin I have seen that thread as it was created by me. However the way vale operates is based on rules naming preference so in case a line has multiple errors it will show only one and will generate another once the previous is fixed. So I believe we are good with this.