Support protocols different from http / https

[Gozala]

I have originally reported elm/browser#20 but as far as I can tell reason is not in Browser.application but rather in the fact that following code explicitly deals with just http / https protocols

url/src/Url.elm

Lines 111 to 120 in 384b1dc

	fromString : String -> Maybe Url
	fromString str =
	if String.startsWith "http://" str then
	chompAfterProtocol Http (String.dropLeft 7 str)
	else if String.startsWith "https://" str then
	chompAfterProtocol Https (String.dropLeft 8 str)
	else
	Nothing

[Gozala]

Reading the source I see that supporting just Https | Http was deliberate choice. Reported error

What is the root? The root of your file system?

makes me think that is related to the file:// protocol where Browser.Navigation would have questionable semantics.

I'd like to still suggest to make this module compatible with arbitrary protocols and instead reject URLs with no authority like file:// from the Browser instead. That would allow Elm apps to be usable in other protocols that have notion of root in form of authority.

Give that currently protocols are tags it does not seems like this could be accomplished without non-breaking change. Maybe Protocol could be extended to:

type Protocol = Http | Https | Custom String

To make this change less disruptive ? Alternative could be just adding known protocols to the list but I don't think that would be a good idea.

It is also worth pointing out that electron apps often introduce custom app specific protocols, we at Mozilla also working with network protocol authors to allow adding them to Firefox via WebExtensions it would be unfortunate if Elm would not be an option for those use cases due to this limitation.

[evancz]

Can you give examples of the other protocols you care about? I just need more information. I cannot design without any examples.

[Gozala]

Can you give examples of the other protocols you care about? I just need more information. I cannot design without any examples.

In the context of libdweb we're collaborating with following projects that are bringing corresponding protocols and are the ones I care the most.

• https://datproject.org/
  • dat://{pub_key}/path/to/thing
• http://ipfs.io/
  • ipfs://${tree_hash}/path/to/thing
  • ipns://${owner_pub_key}/path/to/thing (ipns is like branch in git, just pointer to specific ipfs url)
• https://github.com/ssbc/scuttlebot
  • ssb://${pub_key}

There are also few other projects that we may collaborate with but I don't feel comfortable discussing them publicly (yet).

I think these are the things to consider in terms of design. Most of the custom protocols are semantically similar, which is being reinforced by the API that we (and others projects) provide for protocol implementers, which are:

• Implementation is bound to a specific scheme that obeys general rules for scheme component.
• Root of the content is determined by an authority which is case insensitive and is mandatory, meaning no iop:/// is allowed. Same character set restrictions apply as with http / https.
• No ports or credentials are allowed in authority component (might change in the future, but not the case now).
• URIs are normalized to URLs as ipfs:BoomBom -> ipfs://boombom, ipfs:/hah -> ipfs://hah. In other words it is guaranteed that location of the document will be normalized to a following form: scheme://lower_case_authroity/maybe/path
• Origin of the location is ${scheme}://${authority} which further implies:
  • That pushState / replaceState is not allowed to alter authority or protocol
  • It's not possible to escape authority like foo://bar/ with relative URLs like ../../../boom/thing (they'll be resolved to foo://bar/boom/thing)

Let me know if there is anything else that I can address.

[Gozala]

What I meant to imply that while I do have two or three protocols that I personally care about, I don't think that would necessarily provide required context. Which is why my last comment focused more on the semantics imposed on custom protocols as I think that would better inform design constraints. But if that assumption is incorrect please let me know and I'm happy to provide specific details per protocol basis.

[Mouvedia]

Will this fix ws and wss schemes?

Can you give examples of the other protocols you care about?

The app scheme comes to mind.

[w0rm]

I've been creating html files of my presentations, e.g. file:///Users/w0rm/Work/elm-slice-show/example/index.html and used hash based navigation between slides. Now I have to start a web server just to see my slides. It makes it harder to access the content.

[sarasfox]

Any Idea When the file:// could be working? I new to elm but I am willing to help.

[m4lvin]

It would also be nice to support data:,Hello%2C%20World!

[aforemny]

I have been using file: URLs and hash based navigation in electron apps, too.

[Gozala]

It might be worth considering if browser native URL could be worth leveraging for URL parsing / resolution as it would automatically support every protocol that underlying runtime does. But as I’m mostly guessing why current implementation is the way it very well could be intentionally not doing it.

[lestersm]

I'm working with electron apps and just got here with the same issue. Is there any way we can enable file: ?

[ream88]

This has bitten me as well while trying to update our electron based app to 0.19.

[xla]

@evancz Another use-case which currently doesn't work is hash-based routing in web extensions. The protocols would: chrome-extension:// and moz-extension://.

[alythobani]

Built a .html file just like @w0rm above and got the same error because of the file:// prefix.

Uncaught Error: Browser.application programs cannot handle URLs like this

Worked in Elm 0.18 with Navigation.programWithFlags but not in Elm 0.19 with Browser.application.

[TroyJoachim]

I would also like to see support for the Windows style file:/// because it's used in Electron.

[Mouvedia]

Here are the schemes out of the registered ones currently requested in this issue:

• file:// and file:///
• ws:// and wss://
• chrome-extension://
• data:

[glinesbdev]

I'd like to add mobile frameworks like Capacitor: capacitor://localhost.
This is used for custom URL schemes for iOS projects.

[Erudition]

There's also the Safe Network, safe://
and tor onion://
all of which could host elm apps

[supermario]

I have run into this issue in trying to upgrade to 0.19 in our Cordova wrapped Elm app.

In my case related to https://github.com/ionic-team/cordova-plugin-ionic-webview which uses ionic:// when the app is running on-device/emulator on iOS.

This change from 0.18 blocks our upgrade and I cannot seem to see any way around it.

[shamansir]

We really don't like the hacks at all, but in our case we have to do it, since we use Elm 0.19 and we need to provide user with the self-hosted serverless version of our app.

So here's our hack for the WebPack bundle (webpack.config.json), in its glory, it uses replace-in-file-webpack-plugin:

    plugins: [
      new ReplaceInFileWebpackPlugin([{
          files: ['<your-bundle-name>.bundle.js'],
          rules: [
            {
              search: /var .=.\.fragment/,
              replace: function(match) {
                const varLetter = match[4];
                const fragmentLetter = match[6];
                return 'var ' + varLetter + '=' + fragmentLetter + '?' + fragmentLetter + '.fragment:{}';
              }
            },
            {
              search: 'case 1:throw new Error("Browser.application programs cannot handle URLs like this:\\n\\n    "+document.location.href+"\\n\\nWhat is the root? The root of your file system? Try looking at this program with `elm reactor` or some other server.");case 2:',
              replace: 'case 2:'
            },
            {
              search: /return (\w+)\((\w+)\.location\.href\)\.(\w+)\s*\|\|\s*\w+\(1\)/,
              replace: function(match, x, y, z) {
                const href = y + '.location.href';
                const toLocalhost = '\'http://localhost:8080/\'+' + href + '.substring(' + href + '.indexOf(\'index.html\'))';
                return 'return ' + x + '(' + href + ').' + z + '||' + x + '(' + toLocalhost + ').' + z;
              }
            }
          ]
      }])

[makeros]

Hi.
Regarding Elm + Navigation + Electron and to avoid the

What is the root? The root of your file system?

there could be another solution that i found recently.

To get this example working in electron:

import Browser
import Browser.Navigation as Nav
import Html exposing (..)
import Html.Attributes exposing (..)
import Url

-- MAIN

main : Program () Model Msg
main =
  Browser.application
    { init = init
    , view = view
    , update = update
    , subscriptions = subscriptions
    , onUrlChange = UrlChanged
    , onUrlRequest = LinkClicked
    }

-- MODEL

type alias Model =
  { key : Nav.Key
  , url : Url.Url
  }

init : () -> Url.Url -> Nav.Key -> ( Model, Cmd Msg )
init flags url key =
  ( Model key url, Cmd.none )

-- UPDATE

type Msg
  = LinkClicked Browser.UrlRequest
  | UrlChanged Url.Url

update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
  case msg of
    LinkClicked urlRequest ->
      case urlRequest of
        Browser.Internal url ->
          ( model, Nav.pushUrl model.key (Url.toString url) )

        Browser.External href ->
          ( model, Nav.load href )

    UrlChanged url ->
      ( { model | url = url }
      , Cmd.none
      )


-- SUBSCRIPTIONS

subscriptions : Model -> Sub Msg
subscriptions _ =
  Sub.none

-- VIEW

view : Model -> Browser.Document Msg
view model =
  { title = "URL Interceptor"
  , body =
      [ text "The current URL is: "
      , b [] [ text (Url.toString model.url) ]
      , ul []
          [ viewLink "/home123"
          , viewLink "/profile"
          , viewLink "/reviews/the-century-of-the-self"
          , viewLink "/reviews/public-opinion"
          , viewLink "/reviews/shah-of-shahs"
          ]
      ]
  }

viewLink : String -> Html msg
viewLink path =
  li [] [ a [ href path ] [ text path ] ]

i used the electron.protocol.interceptStringProtocol like this:

const {
  app, BrowserWindow, protocol
} = require('electron');

const fs = require('fs')
const path = require('path')
const chokidar = require('chokidar')

let mainWindow

function createApp () {
  protocol.interceptStringProtocol('http', function (req, callback) {
    const filePath = path.join('.', req.url.split('http://-/')[1])
    const data = fs.readFileSync(filePath, 'utf8')

    callback({
      mimeType: req.headers['Accept'].split(',')[0],
      data
    });
  }, (error) => {
    if (error) {
      throw Error('failed to register protocol handler for HTTP')
    }

    createWindow();
  })
}

function createWindow() {
  mainWindow = new BrowserWindow({
    width: 800,
    height: 600,
    webPreferences: {
      nodeIntegration: true
    }
  })
  mainWindow.loadURL('http://-/index.html')

  mainWindow.on('closed', function() {
    mainWindow = null
  })

  chokidar.watch(['app/main.js', 'index.html'])
    .on('change', () => {
      if (mainWindow) {
        mainWindow.reload()
      }
    })
}

app.on('ready', createApp)

app.on('window-all-closed', function() {
  if (process.platform !== 'darwin') {
    app.quit()
  }
})

app.on('activate', function() {
  if (mainWindow === null) {
    createWindow()
  }
})

[wolfadex]

To add another datapoint, BeakerBrowser just hit v1.0.0-beta and uses the protocol hyper://. This protocol is now part of the IANA URL registry too https://twitter.com/pfrazee/status/1261347702073446400?s=20

[BendingBender]

I have run into this issue in trying to upgrade to 0.19 in our Cordova wrapped Elm app.

In my case related to https://github.com/ionic-team/cordova-plugin-ionic-webview which uses ionic:// when the app is running on-device/emulator on iOS.

This change from 0.18 blocks our upgrade and I cannot seem to see any way around it.

To support this even more, the implementation for cordova-ios has changed recently (v6.0.0) and it doesn't support serving the app from http/https scheme any more. The default scheme they've picked is app:// but this is configurable. However, configuring http/https doesn't work, it is simply ignored.

And as already described, you can't simply write an Elm app for cordova-android either, the default implemenation uses the file:// scheme which is also not supported by this package.

[ronanyeah]

I'm currently looking into using Capacitor to publish an Elm app and looks like I'll also be blocked by this.

[Strepto]

I'm also looking into serving a local Elm app on iOS using capacitor. I have developed it fine for Android, but porting it to iOS is slowed by this issue.

It's not possible to serve local assets (on iOS) using http(s) scheme, https scheme is reserved by the WKWebView for remote urls.

[mradke]

@Strepto @ronanyeah I had the same problem with capacitor and came up with a workaround for a specific usecase: https://discourse.elm-lang.org/t/handling-custom-protocol-in-url-with-elm-was-forking-elm-url-to-enable-custom-url-protocol/7856/6 (beware: the "solution" is ugly...)

That being said: at least iOS, MacOS and Android also have the ability to follow "app links" which may reference content in the application (e. g. a signup view). While not strictly necessary it would probably be convenient to simply hand the URL to the webview and letting Elm do the parsing.

[philer]

I'm trying to use Browser.application with electron and am also blocked by this since electron uses file:// urls.

[phenax]

+1 bump. Need this for Browser.application with electron using custom app:// scheme.

[tlentz]

+1 bump. Need this for Browser.application with tauri using custom tauri:// scheme.