Whatwg protocol setter only seems to support special schemes

[sebakerckhof]

Currently the new whatwg url class doesn't seem to work with any other scheme except so called 'special schemes' in the spec:
https://url.spec.whatwg.org/

E.g. this works:

const { URL } = require('url');
const url = new URL('http://foo.bar');
url.protocol = 'ftp';
url.toString(); // => ftp://foo.bar

But this doesn't work:

const { URL } = require('url');
const url = new URL('http://foo.bar');
url.protocol = 's3';
url.toString(); // http://foo.bar => should be s3://foo.bar

Even schemes defined here: https://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml don't work.

Not that this does work:

const { URL } = require('url');
const url = new URL('s3://foo.bar');
url.protocol; // = s3:

[mscdex]

/cc @nodejs/url

[TimothyGu]

By spec you are not allowed to change a special-scheme URL to a non-special-scheme, or vice versa. See scheme state step 2.1.1-2. However you can change from one special scheme to another (as you have found out), or from one non-special scheme to another:

const { URL } = require('url');
const url = new URL('s3://foo.bar');
url.protocol = 's4';
url.toString(); // => s4://foo.bar

The spec change was made in whatwg/url@5533c8d. See there for more details about it.

[sam-github]

Isn't this a bug in node's URL docs, or do they mention this special case?

[jasnell]

Not a bug as much as not documented. The docs can be expanded.

[gibfahn]

Reopening as a request to document this then.

[broofa]

@TimothyGu Are you sure that's how the spec is to be interpreted? My reading of https://url.spec.whatwg.org/#scheme-state, indicates that the parse method should only return in cases where state override is set. Otherwise it should just go ahead and set the scheme regardless of it being special/unspecial scheme. Am I reading that wrong? And what exactly is the state override argument for? 'Pretty abstract name, that.

It's worth noting that Chrome does not have this issue/behavior. It sets the scheme regardless

In practice, I would argue that silently failing to do what is very obviously the desired action here (change the protocol) is arguably the worst behavior possible. At a minimum, there should be a console warning of some sort, probably even an thrown exception. I know this is probably more appropriate for discussion on the WhatWG forum, but this just wasted an hour and a half of my time and I'm not interested in throwing any more of my time into that particular sink hole. ☹️

[TimothyGu]

the parse method should only return in cases where state override is set

Yes, and:

The protocol attribute’s setter must basic URL parse the given value, followed by U+003A (:), with context object’s url as url and scheme start state as state override.

It's worth noting that Chrome does not have this issue/behavior.

Then it is Chrome's bug for not implementing whatwg/url@5533c8d.

In practice, I would argue that silently failing to do what is very obviously the desired action here (change the protocol) is arguably the worst behavior possible.

While I'm not exactly a huge fan of silently ignoring errors, we are unfortunately in a bind here, as that is what browsers as well as the URL Standard are doing for all other attributes (for example, setting url.port to something out of range like 65536 will reset the port to 0 in Chrome). On the other hand, the URL Standard has the responsibility to maintain compatibility between URL and HTMLAnchorElement, as IIRC URL is intended to be a DOM-independent drop-in replacement of HTMLAnchorElement.

[broofa]

What about this?

The url and state override arguments are only for use by various APIs. [HTML]

Since Node isn't HTML, doesn't this mean "no state override"?

[TimothyGu]

You have to take the entire quote into context:

The encoding override argument is a legacy concept only relevant for HTML. The url and state override arguments are only for use by various APIs. [HTML]

It's my belief that the [HTML] link only refers to the first sentence, while by "various APIs" they mean the URL interface (and the getters/setters of its attributes) that is visible in JavaScript.

[jasnell]

ping @domenic ... thoughts on this?