Kitten

Index › 17. htmx, the htmx WebSocket extension, and socket routes

While the fetching a fediverse timeline is fun and all, wouldn’t it be cooler if you could stream it? Let’s do just that using a WebSocket on the server and htmx to enhance the base fetchiverse example.

💡 Mastodon removed unauthenticated access to the public streaming API so we’re going to consume a curated stream of posts from a little Kitten app called Streamiverse from streamiverse.small-web.org.

💡 Kitten, via its first-class support for htmx, encourages a Hypermedia-Driven Application architecture for web applications where application state is represented in hypermedia. Basically, this means we send HTML between the client and server instead of using data formats like JSON and state is managed on the server.

💡 If you want to learn htmx (and about hypermedia in general), there is now a book called Hypermedia Systems by the folks who authored and maintain htmx.

The only exception to this is when it comes to protecting the identity and privacy of the person who owns a Kitten site/app. Identity and authentication, as we will see later, are handled entirely in the client (in the browser) via public-key cryptography as we expect that the client runs on a device (a computer, phone, etc.) that is entirely within the control of the person.

index.page.js

Let’s start with the code for our new index page:

import Post from './Post.component.js'

export default async function route () {
  const response = await fetch('https://streamiverse.small-web.org/public/')
  const posts = await response.json()
  
  return kitten.html`
    <page htmx htmx-websocket>

    <h1>Aral’s Public Fediverse Timeline</h1>
    <ul id='posts' hx-ext='ws' ws-connect='/updates.socket'>
      ${posts.map(post => kitten.html`<${Post} post=${post} />`)}
    </ul>

    <style>
      body { font-family: sans-serif; font-size: 1.25em; padding-left: 1.5em; padding-right: 1.5em; }
      h1 { font-size: 2.5em; text-align: center; }
      ul { padding: 0; }
    </style>
  `
}

Pay special attention to the unordered list tag’s attributes:

<ul id='posts' hx-ext='ws' ws-connect='/updates.socket'>
  ${posts.map(post => kitten.html`<${Post} post=${post} />`)}
</ul>

Specifically:

• It has an id. We will be using this id from our socket code to tell client where to add new posts as they stream in.

• It uses the htmx WebSockets extension (hx-ext='ws') and tells it to connect to a WebSocket route at the path /updates.socket (ws-connect='/updates.socket').

Finally, we have to tell Kitten, explicitly, that we are using htmx and its WebSocket extension on the page so it knows to include script tags in the head of the rendered page to load in those libraries:

<page htmx htmx-websocket>

💡 Kitten has built-in support for htmx, the htmx WebSockets extension, and Alpine.js. It exposes the HTMX, HTMX_WEBSOCKET, and ALPINEJS constants for you globally to use when returning a libraries array from your page routes.

Since htmx is a progressive enhancement on HTML, if you forget to include it, your page will render and display without any errors, it just won’t have any client-side interactivity.

💡 Note that Kitten also adds syntactic sugar on top of htmx, like its Streaming HTML workflow or aliases like trigger for hx-trigger and swap for hx-swap that do away with the hx- prefix to simplify authoring. (As these are syntactic sugar, they get precompiled down to standard htmx. And, of course, you’re welcome to use standard htmx to begin with if you’d rather.)

Believe it or not, that’s all the code you need on the client to set up and manage a WebSocket connection.

😻 When I said Kitten loves you, I meant Kitten loves you.

Post.component

Notice how we’ve refactored the fetchiverse example so that we now have a Post component. We’ve also added the simple Avatar and Content components to the same file in the name of locality of behaviour.

export default function Post ({ post }) {
  return kitten.html`
    <li class='Post component'>
      <${Avatar} post=${post} />
      <${Content} post=${post} />
    </li>
    <style>
      .Post {
        display: flex; align-items: flex-start; column-gap: 1em; padding: 1em;
        margin-bottom: 1em; background-color: #ccc; border-radius: 1em;
      }
    </style>
  `
}

// Private components (can only be used by Post).

const Avatar = ({ post }) => kitten.html`
  <a class='Avatar component' href='${post.account.url}'>
    <img src='${post.account.avatar}' alt='${post.account.username}’s avatar' />
  </a>
  <style>
    .Avatar img {
      width: 8em;
      border-radius: 1em;
    }
  </style>
`

const Content = ({ post }) => kitten.html`
  <div class='Content component'>
    ${kitten.safelyAddHtml(post.content)}
    ${post.media_attachments.map(media => (
      media.type === 'image' && kitten.html`<img src='${media.url}' alt='${media.description}'>`
    ))}
  </div>
  <style>
    .Content { flex: 1; }
    .Content p:first-of-type { margin-top: 0; }
    .Content p { line-height: 1.5; }
    .Content a:not(.Avatar) {
      text-decoration: none; background-color: rgb(139, 218, 255);
      border-radius: 0.25em; padding: 0.25em; color: black;
    }
    .Content img { max-width: 100%; }
    /* Make sure posts don’t overflow their containers. */
    .Content a {
      word-break: break-all;
    }
  </style>
`

A post is a natural unit for a component in our example as we receive individual post updates from the Mastodon API. Since we send HTML over the wire, our web socket will have to create Post instances. And we also have to create Post instances in our original GET route in the index page that sends over the initial timeline. By having Post as a component in its own module, we can import and use it from both places.

💡 None of the code in the Post, Avatar, or Content components has otherwise changed from the fetchiverse example.

Finally, let’s look at the big new thing that makes this version stream: the socket route.

updates.socket

🐈 In Kitten, you declare WebSocket routes in .socket.js files.

🪤 You will have noticed that Kitten usually strips the extensions from your routes. If you have a page called /hello.page, for example, you can access it from https://localhost/hello. The exception is WebSocket routes, which keep their extensions. So the path to the updates.socket route is https://localhost/updates.socket.

💡 The main reason for this is due to the built-in redirection Kitten performs to forward URIs that don’t contain a trailing slash to ones that do (e.g., https://localhost/hello will get a 308 forward to https://localhost/hello/). This works for HTTP routes but is not guaranteed to work for WebSocket routes (because clients are not obligated to follow redirects during the handshake/protocol upgrade stage… don’t ask me why not.) So, to avoid the situation where you could have to refer to your WebSocket route as, for example, ‘/chat/’ and where it would not work if you forgot the trailing slash – which would be very easy to do – Kitten decrees that the file extension is kept for socket routes, thereby bypassing the issue altogether (while also improving semantics and further differentiating WebSocket routes from HTTP routes at a glance).

The function signature of your socket route is similar to the structure of regular HTTP routes but, in that it expects a parameter object that has a reference to the HTTP request. However, instead of a reference to the HTTP response – which, in WebSocket routes is managed for you and which you should not interfere with manually – you get a reference to the socket instance that you can use to communicate with the current connection to the page (as well as to all connected pages via the broadcast and all methods on it).

💡 In the example below, since we are not using the request for anything, we simply do not declare it in the function signature.

import Post from './Post.component.js'

let stream = null

export default function socket ({socket}) {
  // Lazily start listening for timeline updates on first request.
  if (stream === null) {
    console.info('  🐘 Listening for Mastodon updates…')
    stream = new kitten.WebSocket('wss://streamiverse.small-web.org/stream.socket')
    // stream = new kitten.WebSocket('wss://mastodon.ar.al/api/v1/streaming?stream=public')

    stream.addEventListener('message', event => {
      const message = JSON.parse(event.data)
      if (message.event === 'update') {
        const post = JSON.parse(message.payload)

        console.info(`  🐘 Got an update from ${post.account.username}!`)

        const update = kitten.html`
          <div swap-target="afterbegin:#posts">
            <${Post} post=${post} />
          </div>
        `

        socket.all(update)
      }
    })
  }
}

The WebSocket route itself creates a WebSocket connection to consume Aral’s public Mastodon feed from Aral’s Mastodon server. It also adds a message listener that gets called each time there’s a message from the Mastodon server. Since we only care about new posts in this example, we only handle update messages.

💡 As you can see here, WebSocket connections do not have to be client-to-server, they can be server-to-server also. To help you create these sorts of connections, Kitten exposes a reference to the WebSocket module it uses internally (ws) at kitten.WebSocket. This is what we’re using in the code above.

The messages sent by Mastodon are in JavaScript Object Notation (JSON) format so the first thing we do is to parse them into a plain JavaScript object. And in that object we find the payload property that contains the post itself.

Now comes the the htmx magic:

const update = kitten.html`
  <div swap-target="afterbegin:#posts">
    <${Post} post=${post} />
  </div>
`

The swap-target attribute is Kitten’s syntactic sugar for HTMX’s hx-swap-oob attribute (which stands for swap out-of-band). It states that this post should be added to the top of the list of posts on the page (remember that our posts list had the id of posts).

Finally, after we’ve created our Post snippet, we send it to all connected WebSocket clients using the special .all() method on the socket object:

socket.all(update)

And that’s all there is to it!

Run the example using kitten and visit https://localhost to see Aral’s public fediverse timeline streaming from his Mastodon server.

💡 Notice that we’re sending HTML over the wire using the WebSocket. This is how htmx works. It makes it possible for us to create dynamic functionality like a streaming fediverse timeline without writing any custom client-side JavaScript.

In this tutorial, you learned how to use a WebSocket to push data from the server to the client. But the WebSocket protocol is full-duplex (which is a fancy way of saying ‘two-way’). In the next tutorial, let’s see how we can also push data from the client to the server by creating a simple WebSocket echo server.

Next tutorial: Two-way WebSocket communication