libfetch: don't include fragments in HTTP requests

[gahr]

Summary

Fragments are reserved for client-side processing, see
https://www.rfc-editor.org/rfc/rfc9110.html#section-7.1

Also, some servers don't like to receive HTTP requests with fragments.

$ fetch 'https://dropbox.com/a/b'
fetch: https://dropbox.com/a/b: Not Found

$ fetch 'https://dropbox.com/a/b#'
fetch: https://dropbox.com/a/b#: Bad Request

This is a real-world scenario, where some download link from dropbox
(eventually) redirects to an URL with a fragment:

$ fetch -v 'https://www.dropbox.com/sh/<some>/<thing>?dl=1' 2>&1 | grep requesting
requesting https://www.dropbox.com/sh/<some>/<thing>?dl=1
requesting https://www.dropbox.com/scl/fo/<foo>/<bar>?rlkey=<baz>&dl=1
requesting https://<boo>.dl.dropboxusercontent.com/zip_download_get/<some-long-strig>#

See how the last redirect ends with a #.

Currently, libfetch includes the ending fragment and makes it impossible
to download the file.

Test Plan

manually tested

[des]

This belongs in the URL parser, not in the HTTP client.

[gahr]

In D46318#1056275, @des wrote:

This belongs in the URL parser, not in the HTTP client.

I was unsure whether the URL parser should terminate the doc part at the first fragment or not. The reason I did it in the http module is that I don't know if any other protocols supported by libfetch might use the fragment for anything. E.g., should fetch file:///path/to/file#foo fetch a file named 'file#foo'?

[des]

In D46318#1056278, @gahr wrote:

In D46318#1056275, @des wrote:

This belongs in the URL parser, not in the HTTP client.

I was unsure whether the URL parser should terminate the doc part at the first fragment or not. The reason I did it in the http module is that I don't know if any other protocols supported by libfetch might use the fragment for anything. E.g., should fetch file:///path/to/file#foo fetch a file named 'file#foo'?

The URL parser knows which protocol it's dealing with.

[gahr]

Handle fragments while parsing the URL

[gahr]

Update diff