Kitten: Tutorials

Kitten gives you a simple-to-use event-based HTML over WebSocket implementation called Streaming HTML (because you’re streaming HTML updates to the client) that you can use to build web apps.

Topics covered

Using Kitten’s Streaming HTML flow.
Review of Kitten fundamentals (the page tag, fragments and components, etc.).
Streaming HTML in pure htmx.
Streaming HTML with pure client-side JavaScript.
Streaming HTML in plain old Node.js.

Streaming HTML

Streaming HTML is a new workflow developed in Kitten that makes creating data-driven interactive web applications easy to author without writing client-side JavaScript by following a hypermedia-based approach.

If that makes it sounds scary, don’t worry, it’s not.

By handling a lot of the tedious bits for you magically, it is actually simpler than other more traditional approaches.

So let’s jump in and see how Streaming HTML works by implementing the ubiquitous counter example.

💡 Don’t worry about the “magic”, either. Once you see how Streaming HTML works, we will spend the rest of the tutorial breaking it down and demystifying the magic all the way to showing you how you can implement Streaming HTML without even using Kitten. You don’t have to read the whole tutorial to understand how to use Streaming HTML but doing so will give you a very comprehensive idea of how everything works internally.

O counter! My counter!

Create a directory for the example and enter it:
```
mkdir counter
cd counter
```

Create a file called index.page.js and add the following content to it:

// Initialise the database table if it doesn’t already exist.
if (kitten.db.counter === undefined)
  kitten.db.counter = { count: 0 }

// Default route that renders the page on GET requests.
export default () => kitten.html`
  <page css>
  <h1>Counter</h1>

  <${Count} />

  <button
    name='update' connect data='{value: -1}'
    aria-label='decrement'
  >-</button>

  <button
    name='update' connect data='{value: 1}'
    aria-label='increment'
  >+</button>
`

// The Count fragment.
const Count = () => kitten.html`
  <div
    id='counter'
    aria-live='assertive'
    morph
    style='font-size: 3em; margin: 0.25em 0;'
  >
    ${kitten.db.counter.count}
  </div>
`

// Handle the «update» event from the client.
export function onUpdate (data) {
  kitten.db.counter.count += data.value
  this.send(kitten.html`<${Count} />`)
}

Run Kitten:
```
kitten
```

💡 In the video, you see a slighty older and more verbose way of adding an event handler. Going forward, you should prefer the method shown in the code example here.

Once Kitten is running, hit https://localhost, and you should see a counter at zero and two buttons.

Press the increment and decrement buttons and you should see the count update accordingly.

Press CtrlC in the terminal to stop the server and then run kitten again.

Refresh the page to see that the count has persisted.

What just happened?

In a few lines of very liberally-spaced code, you have built a very simple Streaming HTML web application in Kitten that:

Is fully accessible (turn on your screen reader and have a play).
Persists data to a database.
Triggers events on the server in response to button presses and sends custom data from the client to the server.
Sends an updated Count component back to the client which automatically gets morphed into place, maintaining state.
Uses a basic semantic CSS library to style itself.
Uses WebSockets, htmx, and Water behind the scenes to achieve its magic.

In a nutshell, Kitten gives you a simple-to-use event-based HTML over WebSocket implementation called Streaming HTML (because you’re streaming HTML updates to the client) that you can use to build web apps.

HTML over WebSocket is not unique to Kitten – the approach is formalised with different implementations in a number of popular frameworks and application servers. And the general idea of hypermedia-based development actually predates the World Wide Web and HTML.

What is unique, however, is just how simple Kitten’s implementation is to understand, learn, and use.

That simplicity comes from the amount of control Kitten has over the whole experience. Kitten is not just a framework. Nor is it just a server. It’s both. This means we can simplify the authoring experience using file system-based routing combined with automatic WebSocket handling, a built-in in-process native JavaScript database, a simple high-level declarative API, and built-in support for libraries like htmx.

Kitten’s Streaming HTML flow – and Kitten’s development process in general – stays as close to pure HTML, CSS, and JavaScript as possible and progressively enhances these core Web technologies with features to make authoring web applications as easy as possible.

Where next? (Choose your own adventure.)

If you’re happy to simply use the Streaming HTML workflow, you can stop reading here. You already know how.

If you’d like to practice what you learned by building a basic collaborative pixel drawing app, continue on to the next section to watch a video tutorial.

And, if you want to venture under covers to the really understand how Streaming HTML is implemented in Kitten, continue by reading the Let’s break it down section.

Draw Together

Draw Together is a fun little collaborative pixel drawing app where anyone on the page can toggle the pixels in a 20×20 grid between black and white.

You can play it right here in the embed below.

Learn how to build Draw Together yourself by viewing its source code and watching the video tutorial below.

Now, let’s get back to our counter example and break it down piece by piece so you can see how Streaming HTML works under the hood.

Let’s break it down

OK, so now we have a high-level understanding of how Streaming HTML works, let’s go through our initial counter example and dissect it to see exactly how everything works by peeling away the layers of magic one by one.

Let’s begin with what happens when you start the server.

During its boot process, Kitten recurses through your project’s folder and maps the file types it knows about to routes based on their location in the directory hierarchy.

In our simple example, there is only one file – a page file. Since it’s located in the root of our project folder and named index, the created route is /.

Pages, like other route types in Kitten, are identified by file extension (in this case, .page.js) and are expected to export a default function that renders the web page in response to a regular GET request.

Initial page render

export default () => kitten.html`
  <page css>
  <h1>Counter</h1>

  <${Count} />

  <button
    name='update' connect data='{value: -1}'
    aria-label='decrement'
  >-</button>

  <button
    name='update' connect data='{value: 1}'
    aria-label='increment'
  >+</button>
`

This renders a heading, the current count, and two buttons onto the page and sprinkles a bit of magic semantic CSS styling.

Notice a few things about this code:

We are not returning a complete HTML document. Specifically, there is no <html> tag, or <head>, or <body>. Kitten creates the outer shell of the HTML page for us and we have ways of adding elements to different parts of that page and changing things in the head, like the title, etc.
There isn’t a single root element. Pages can, and usually do, contain multiple elements and don’t have to be wrapped up in a single root tag. (Unlike components and fragments, which we shall see later, which do.)
A page can contain a mix of plain old regular HTML tags (h1, button), custom Kitten tags (page), and custom components and fragments (Count).
The value being returned is a custom Kitten tagged template string that’s available from the global kitten.html reference. This template string is what allows us to extend HTML with custom tags and components/fragments.
This is just a regular JavaScript tagged template string and we use string interpolation to inject values into it.
Notice, especially, how we handle components and fragments: They are written as tags but the name is interpolated. In our example, we use ${Count}, which is a reference to the function that renders the fragment.
Since HTML templates in Kitten are plain old JavaScript, we don’t need any special tooling for Kitten to make use of the language intelligence that’s already in your editor. So you can, say, easily jump to the definition of the Count fragment from within the markup or get warned by your editor if you misspell the name of a component, etc.

Next, let’s take a look at the Kitten-specific aspects of this template, starting with the first tag.

Deconstructing the page

The first piece of magic on the page is the simplest, a <page> that has a css attribute specified:

<page css>

The page tag is transpiled by Kitten into HTML. In this case, since the css attribute is specified, it results in the following stylesheet reference in the head of the page:

<link rel="stylesheet" href="/🐱/library/water-2.css">

This, in turn, loads in the Water semantic CSS library that gives our page some basic styles based on the HTML elements we used.

🐱 Go ahead and delete the line with the <page> tag and see how it affects the display of the page, then undo the change. Kitten will automatically update your page in the browser whenever you save your file.

Kitten has first-class support for certain libraries, Water being one of them, that it serves from the reserved /🐱/library/ namespace. Instead of manually including these libraries, you can just use the <page> tag like we did here.

Most of the magic in this example, as we will see later, relies on a different library called htmx and its WebSocket and idiomorph extensions.

Components and fragments

Next in our page, we have a heading, followed by the Count fragment, included in the page using:

export default () => kitten.html`
  …
  <${Count} />
  …
`

This results in the Count function being called to render the fragment as HTML:

const Count = () => kitten.html`
  <div
    id='counter'
    aria-live='assertive'
    morph
    style='font-size: 3em; margin: 0.25em 0;'
  >
    ${kitten.db.counter.count}
  </div>
`

🐱 Kitten encourages you to split your code into components and fragments^[1]. This becomes even more important in a streaming HTML workflow where you initially render the whole page and then send back bits of the page to be morphed into place to update the page. Breaking up your content into components and fragments enables you to remove redundancy in your code.

This fragment creates a div that displays the current count of the counter, which it gets from Kitten’s magic default database.

Kitten’s magic database

Kitten comes with a very simple, in-process JavaScript database called – drumroll – JavaScript Database (JSDB).

It even creates a default one for you to use at kitten.db.

🐱 You’re not limited to using the default database that Kitten makes available. You can create your own and even use multiple databases, etc., using database app modules. You can also implement type safety in your apps, including for your database structures.)

In JSDB you store and access data in JavaScript arrays and objects and you work with them exactly as you would with any other JavaScript array or object. The only difference is that the changes you make are automatically persisted to an append-only JavaScript transaction log in a format called JavaScript Data Format (JSDF).

It’s a common pattern in JSDB to check whether an array or object (the equivalent of a table in a traditional database) exists and, if not, to initialise it.

This is what we do at the very start of the file that contains the page route, creating the counter object with its count property set to zero if it doesn’t already exist:

if (kitten.db.counter === undefined)
  kitten.db.counter = { count: 0 }

Once you are sure a value exists in your database, you can access it using regular JavaScript property look-up syntax (because it is just a regular JavaScript object).

This is what we do in the Count component:

const Count = () => kitten.html`
  …
  ${kitten.db.counter.count}
  …
`

So the first time the page renders, the count will display as zero.

After the Count fragment, we have the last two elements on the page: two buttons, one to decrement the count and the other to increment it.

But what is the magic that allows us to connect those buttons to the server, mutate the count, and persist the value?

Let’s look at that next.

A magic connection

At the heart of Kitten’s Streaming HTML workflow is a cross-tier eventing system that maps events on the client to handlers on the server.

Take a look at the two buttons declared in our page to see how it works:

<button
  name='update' connect data='{value: -1}'
  aria-label='decrement'
>-</button>

<button
  name='update' connect data='{value: 1}'
  aria-label='increment'
>+</button>

Both of the buttons are marked with the connect attribute and have the same name, update.

The presence of the connect attribute triggers Kitten’s automatic mapping of element names on the client to event handlers on the server. This means that performing the default action on either of those buttons (in this case, a “click”) results in an event handler called onUpdate getting called on the server (if you exported a function with that name from your page route).

So the naming convention is that a connected DOM element with its name set to something will be mapped to an exported event handler function called onSomething.

💡 If the element doesn’t have a name attribute, Kitten will fall back to using its id. If neither are available, Kitten cannot route the event and you will see an error in the console.

Additionally, the contents of the magic data attribute are also sent to the event handler.

In this example, we use the value property of the data object to differentiate between which increment and decrement buttons.

💡 Sometimes, it’s useful to separate the name of an HTML element from the name of the event.
For example, <details> elements are grouped if they have the same name and the browser will enforce the rule that only one element in a group may be open at a given time.
If you want to handle, say, toggle events on multiple elements but you also want people to be able to open more than one element at the same time, name your elements starting with the shared event name, add a colon (:), and finally provide a unique suffix. e.g.,
<details name='toggleDetail:1'>…</details>
<details name='toggleDetail:2'>…</details>
The colon and anything beyond it is ignored when the event name is being calculated. So, in the example here, clicking/tapping either of the detail disclosure elements would result in the toggleDetail event on the server.

The event handler in question is the only other bit of code in our pithy example:

export function onUpdate (data) {
  kitten.db.counter.count += data.value
  this.send(kitten.html`<${Count} />`)
}

In it, we simply update the persisted count and use the send() method on the live kitten.Page instance that our event handler automatically gets bound to by Kitten to stream an updated render of the Count component.

If you remember, the Count component had one last magic attribute on it called morph:

<div
  id='counter'
  aria-live='assertive'
  morph
  style='font-size: 3em; margin: 0.25em 0;'
>

This makes Kitten intelligently morph the streamed HTML into the DOM, replacing the element that matches the provided id.

Notice that unlike web apps that you may be familiar with, we are not sending data to the client, we are sending hypermedia in the form of HTML.

Streaming HTML is a modern event-based full-duplex approach to building hypermedia-based applications.

Its greatest advantage is its simplicity, which arises from keeping state on one tier (the server) instead of on two (the client and the server). In essence, it is the opposite of the Single-Page Application (SPA) model, embracing the architecture of the Web instead of attempting to turn it on its head. In fact, you can create whole Web apps without writing a single line of client-side JavaScript yourself.

And with that, we now know what Streaming HTML is and what each part of the code does.

Now, let’s go back to the start and review the process as we start to understand how things work at a deeper level.

💡 In the video, you see a slighty older and more verbose way of adding an event handler:
export function onConnect ({ page }) {
  page.on('update', data => {
    kitten.db.counter.count += data.value
    page.send(kitten.html`<${Count} />`)
  })
}
The onConnect() handler – like the `onDisconnect() handler – is an automatic/intrinsic event handler called by Kitten when the page first connects its WebSocket and when it disconnects before being unloaded. It’s a good place to carry out initialisation and deinitialisation tasks.
In the video, which was recorded a little while ago, we use this page reference to set up an event handler for the update event.
This is because in the initial Kitten component model, the event handlers were not bound to the live page object that Kitten keeps in memory so we had to rely on the page reference we received in the parameter object, as you see above.
Today, with Kitten’s improved component model, we could just as easily have written this as:
  export function onConnect () {
    this.on('update', data => {
      kitten.db.counter.count += data.value
      this.send(kitten.html`<${Count} />`)
    })
  }
But, as you see in the original code listing, above, even that is no longer necessary, as Kitten’s improved component model includes automatic mapping of event names to event handlers using the naming convention where a connected DOM element with its name set to something will be mapped to an exported event handler function called onSomething.
In fact, if you run the above example, it will work but you will get a deprecation notice:
[Deprecated] on(): Instead of on('update', handler), export onUpdate() from your page.
So please refactor your code to use the new method as support for this deprecated method will eventually be removed from Kitten (likely before we hit API version 1).

High-level flow

Let’s go step-by-step, starting from when we launch Kitten to when the counter is updated:

Kitten parses the page and sees that at least one event handler (in this case, onUpdate) was exported from the page so it creates a default WebSocket route for the page and wires it up so that it can automatically map events on the client to handlers on the server.
When the person presses the increment button, it sends a message to the default WebSocket route. Since the button’s name is update, Kitten calls the onUpdate() event handler, passing a copy of any data that was sent along.
In this case, the data is {value: 1}. It is an object that has a value property set to 1. So we add the value to the count we are keeping in our database and send a new Count fragment render back.

At this point, you might be wondering about several things:

How exactly does Kitten wire up the client so that the WebSocket connection is made and that messages are sent to the server when we click the buttons?
How does Kitten update the page on the client with new HTML fragments as they are sent from the server?

The answer to both of those questions is ‘through the magic of htmx’.

So what is htmx?

Let’s find out!

Peeking behind the curtain: htmx

Earlier, I wrote that most of the magic in this example relies on a library called htmx and its WebSocket and idiomorph extensions. Let’s now dive a little deeper into the internals of Kitten and take a look at how Kitten transpiles your code to use this library and its extenions.

In our example, whenever either the increment or decrement button gets pressed on the client, the update event handler gets called on the server, whereupon it updates the counter accordingly and sends a new Count fragment back to the client that gets morphed into place in the DOM.

There are three things working in tandem to make this happen, all of which sprinkle htmx code into your page behind the scenes.

First, whenever Kitten sees that one of your pages has exported at least one event handler function (a function with a name written in camelCase, starting with on), it:

Adds the htmx library, as well as its WebSocket and idiomorph extensions, to the page.
It creates a special default WebSocket route for your page. In this case, since our page is the index page and is accessed from the / route, it creates a socket that is accessed from /default.socket. In that socket route, it adds an event listener for the message event and maps any HTMX-Trigger-Name headers it sees in the request to exported event handlers defined on the page.
When the page is hit, it creates a kitten.Page instance to act as a live representation of the page in memory. This instance has a send() method that can be used to stream responses back to the client. We haven’t used them in this example but it also has everyone() and everyoneElse() methods that can be used to stream responses back not just to the person on the current page but to every person that has the page open (or to every person but the current one).

Second, it goes through your code and, whenever it sees a form, it adds the necessary htmx WebSocket extension code so form submits will automatically trigger serialisation of form values. (We don’t make use of this in this example, preferring to forego a form altogether and directly connect the buttons instead.)

Finally, it applies some syntactic sugar to attribute names by replacing:

connect with ws-send
morph with hx-swap-oob='morph'
data=<data> with hx-vals='js:<data>'

These little conveniences make authoring easier without you having to remember the more verbose htmx attributes. You can, of course, use the htmx attributes instead, as well as any other htmx attribute, because it is just htmx under the hood.

Progressive enhancement

Kitten’s design adheres to the philosophy of progressive enhancement.

At its very core, Kitten is a web server. It will happily serve any static HTML you throw at it from 1993.

However, if you want to, you can go beyond that. You can use dynamic pages, as we have done here, to server render responses, use a database, etc.

Similarly, Kitten has first-class support for the htmx library and some of its extensions, as well as other libraries like Alpine.js.

The idea is that you can build your web apps using plain old HTML, CSS, and JavaScript and then layer additional functionality on top using Kitten’s Streaming HTML features, htmx, Alpine.js, etc. You can even use its unique features to make peer-to-peer Small Web apps.

So Kitten’s implementation of Streaming HTML is based on core Web technologies and progressively enhanced using authoring improvements, htmx, and a sprinkling of syntactic sugar (collectively, what we refer to as ‘magic’).

All this to say, you can do everything we did in the original example by using htmx and creating your WebSocket manually.

Let’s see what that would look like next.

Goodbye, magic! (Part 1: Goodbye, syntactic sugar; hello, htmx)

Right, let’s peel away a layer of the magic and stop making use of Kitten’s automatic event mapping and syntactic sugar and use plain htmx instead, starting with the index page:

index.page.js

import Count from './Count.fragment.js'

export default function () {
  return kitten.html`
    <page htmx htmx-websocket htmx-idiomorph css>
    <main
      hx-ext='ws'
      ws-connect='wss://${kitten.domain}:${kitten.port}/count.socket'
    >
      <h1>Counter</h1>
      <${Count} />
      <button
        name='update' ws-send hx-vals='js:{value: -1}'
        aria-label='decrement'
      >-</button>
      <button
        name='update' ws-send hx-vals='js:{value: 1}'
        aria-label='increment'>
      +</button>
    </main>
  `
}

Notice, what’s different here from the previous version:

The Count fragment now lives in its own file (with the extension .fragment.js) that we import into the page. This is because we now have to create the WebSocket route ourselves, in a separate file, and it will need to use the Count fragment too when sending back new versions of it to the page. Previously, our event handler was housed in the same file as our page so our fragment was too.
We have to manually let Kitten know that we want the htmx library and its two extensions loaded in, just like we had to do with the Water CSS library (the css attribute is an alias for water; you can use either. Kitten tries to be as forgiving as possible during authoring).
We wrap our counter in a main tag so we have some place to initialise the htmx ws (WebSocket) extension. We also have to write out the connection string to our socket route manually. As we’ll see later, our socket route is called count.socket. While writing the connection string, we make use of the Kitten globals kitten.domain and kitten.port to ensure that the connection string will work regardless of whether we are running the app locally in development or from its own domain in production.
Instead of Kitten’s syntactic sugar, we now use the regular htmx attributes ws-send and hx-vals in our buttons.

Next, let’s take a look at the Count fragment.

Count.fragment.js

if (kitten.db.counter === undefined)
  kitten.db.counter = { count: 0 }

export default function Count () {
  return kitten.html`
    <div
      id='counter'
      aria-live='assertive'
      hx-swap-oob='morph'
      style='font-size: 3em; margin: 0.25em 0;'
    >
      ${kitten.db.counter.count}
    </div>
  `
}

Here, apart from being housed in its own file so it can be used from both the page and the socket routes, the only thing that’s different is that we’re using the htmx attribute hx-swap-oob (htmx swap out-of-band) instead of Kitten’s syntactic sugar morph attribute.

We also make sure the database is initialised before we access the counter in the component.

We’re carrying out the initialisation here and not in the socket (see below) because we know that the page needs to be rendered (and accessed) before the socket route is lazily loaded. While this is fine in a simple example like this one, it is brittle and requires knowledge of Kitten’s internals. In a larger application, a more solid and maintainable approach would be to use a database app module to initialise your database and add type safety to it while you’re at it.

🐱 A design goal of Kitten is to be easy to play with. Want to spin up a quick experiment or teach someone the basics of web development? Kitten should make that simple to do. Having magic globals like the kitten.html tagged template you saw earlier help with that.
However, for larger or longer-term projects where maintainability becomes an important consideration, you might want to make use of more advanced features like type checking.
The two goals are not at odds with each other.
Kitten exposes global objects and beautiful defaults that make it easy to get started and, at the same time, layers on top more advanced features that make it easy to build larger and longer-term projects.

Finally, having seen the page and the Count component, let’s now see what the WebSocket route – which was previously being created for us internally by Kitten – looks like.

count.socket.js

import Count from './Count.fragment.js'

export default function socket ({ socket }) {
  socket.addEventListener('message', event => {
    const data = JSON.parse(event.data)

    if (data.HEADERS === undefined) {
      console.warn('No headers found in htmx WebSocket data, cannot route call.', event.data)
      return
    }

    const eventName = data.HEADERS['HX-Trigger-Name']
    
    switch (eventName) {
      case 'update':
        kitten.db.counter.count += data.value
        socket.send(kitten.html`<${Count} />`)
      break

      default:
        console.warn(`Unexpected event: ${eventName}`)
    }
  })
}

Our manually-created socket route is functionally equivalent to our onUpdate() handler in the original version. However, it is quite a bit more complicated because we have to manually implement, at a slightly lower level, what magic mapping that Kitten previously handled for us.

Socket routes in Kitten are passed a parameter object that includes a socket reference to the WebSocket instance. It can also include a reference to the request that originated the initial connection.^[2]

The socket object is a ws WebSocket instance with a couple of additional methods – like all() and broadcast(), mixed in by Kitten.^[3]

On this socket instance, we listen for the message event and, when we receive a message, we manually:

Deserialise the event data.
Check that htmx headers are present before continuing and bail with a warning otherwise.
Look for the HX-Trigger-Name header and, if the trigger is an event we know how to handle (in this case, update), carry out the updating of the counter that we previously did in the onUpdate() handler.

For comparison, this was the onUpdate() handler from the original version where Kitten essentially does the same things for us behind the scenes and routes the update event to our handler:

export function onUpdate (data) {
  kitten.db.counter.count += data.value
  this.send(kitten.html`<${Count} />`)
}

If you run our new – plain htmx – version of the app, you should see exactly the same counter, behaving exactly the same as before.

While the plain htmx version is more verbose, it is important to understand that in both instances we are using htmx. In the original version, Kitten is doing most of the work for us and in the latter we’re doing everything ourselves.

Kitten merely progressively enhances htmx just like htmx progressively enhances plain old HTML. You can always use any htmx functionality and, if you want, ignore Kitten’s magic features.

Goodbye, magic! (Part 2: goodbye, htmx; hello, plain old client-side JavaScript)

So we just stipped away the magic that Kitten layers on top of htmx to see how we would implement the Streaming HTML flow using plain htmx.

Now, it’s time to remove yet another layer of magic and strip away htmx also (because htmx is just a bit of clever client-side JavaScript that someone else has written for you).

We can do what htmx does manually by writing a bit of client-side JavaScript (and in the process see that while htmx is an excellent tool, it’s not magic either).

Let’s start with the index page, where we’ll strip out all htmx-specific attributes and instead render a bit of client-side JavaScript that we’ll write ourselves.

Our goal is not to reproduce htmx but to implement an equivalent version of the tiny subset of its features that we are using in this example. Specifically, we need to write a generic routine that expects a snippet of html encapsulated in a single root element that has an ID and replaces the element that’s currently on the page with that ID with the contents of the new one.

index.page.js

import Count from './Count.fragment.js'

export default function () {
  return kitten.html`
    <page css>
    <h1>Counter</h1>
    <${Count} />
    <button onclick='update(-1)' aria-label='decrement'>-</button>
    <button onclick='update(1)' aria-label='increment'>+</button>
    <script>
      ${[clientSideJS.render()]}
    </script>
  `
}

/**
  This is the client-side JavaScript we render into the page.
  It’s encapsulated in a function so we get syntax
  highlighting, etc. in our editor.
*/
function clientSideJS () {
  const socketUrl = `wss://${window.location.host}/count.socket`
  const ws = new WebSocket(socketUrl)
  ws.addEventListener('message', event => {
    const updatedElement = event.data

    // Get the ID of the new element.
    const template = document.createElement('template')
    template.innerHTML = updatedElement
    const idOfElementToUpdate = template.content.firstElementChild.id

    // Swap the element with the new version.
    const elementToUpdate = document.getElementById(idOfElementToUpdate)
    elementToUpdate.outerHTML = updatedElement
  })

  function update (value) {
    ws.send(JSON.stringify({event: 'update', value}))
  }
}
clientSideJS.render = () => clientSideJS.toString().split('\n').slice(1, -1).join('\n')

Here’s how the page differs from the htmx version:

The htmx and htmx-websocket attributes are gone. Since htmx is no longer automatically creating our socket connection for us, we do it manually in our client-side JavaScript.
The htmx-idiomorph extension is also gone. Since htmx is not automatically carrying out the DOM replacement of the updated HTML fragments we send it, we do that manually also in our client-side JavaScript.
We do so by first creating a template element and populating its inner HTML with our HTML string. Then, we query the resulting document fragment for the id of its top-level element. Finally, we use the getElementById() DOM look-up method on the resulting document fragment to get the current version of the element and replace it by setting its outerHTML to the updated HTML fragment we received from the server.
Finally, since we no longer have htmx to send the HX-Trigger-Name value so we can differentiate between event types, we add an event property to the object we send back to the server via the WebSocket.

Documenting the (overly) clever bits

There are two bits of the code where we’re doing things that might be confusing.

First, when we interpolate the result of the clientSideJS.render() call into our template, we surround it with square brackets, thereby submitting the value wrapped in an array:

export default function () {
  return kitten.html`
    …
    <script>
      ${[clientSideJS.render()]}
    </script>
  `
}

This is Kitten shorthand for circumventing Kitten’s built in string sanitisation. (We know that the string is safe because we created it.)

🐱 Needless to say, only use this trick with trusted content, never with content you receive from a third-party. By default, Kitten will sanitise any string you interpolate into a kitten.html string. So the default is secure. If you want to safely interpolate third-party HTML content into your pages, wrap the content in a call to kitten.safelyAddHtml() which will sanitise your html using the sanitize-html library.

The other bit that might look odd to you is how we’re adding the render() function to the clientSideJS() function:

function clientSideJS () {
  //…
}
clientSideJS.render = () => clientSideJS.toString().split('\n').slice(1, -1).join('\n')

You might be wondering why we wrote our client-side JavaScript code in a function on our server to begin with instead of just including it directly in the template.

We did so to make use of the language intelligence in our editor.

💡 Of course, we could also have taken one extra HTTP hit and just put the client-side JavaScript into a plain old .js file and and imported it client-side.

Given how little code there is in this example, we could have just popped it into the template string. But this provides a better authoring experience and is more maintainable.

Of course what we need is a string representation of this code – sans the function signature and the closing curly bracket – to embed in our template.

Again, we could have just added that logic straight into our template:

export default function () {
  return kitten.html`
    …
    <script>
      ${[clientSideJS.toString().split('\n').slice(1, -1).join('\n')]}
    </script>
  `
}

That does the same thing but it doesn’t really roll off the tongue.

I feel that templates should be as literate, readable, and close to natural language as possible and that any complex stuff we might have to do should be done elsewhere. And since in JavaScript nearly everything is an object, including functions, why not add the function to render the inner code of a function onto the function itself?^[4]

OK, enough JavaScript geekery.

Next, let’s take a look at how the WebSocket route has changed.

count.socket.js

import Count from './Count.fragment.js'

export default function socket ({ socket }) {
  socket.addEventListener('message', event => {
    const data = JSON.parse(event.data)

    if (data.event === undefined) {
      console.warn('No event found in message, cannot route call.', event.data)
      return
    }

    switch (data.event) {
      case 'update':
        kitten.db.counter.count += data.value
        socket.send(kitten.html`<${Count} />`)
      break

      default:
        console.warn(`Unexpected event: ${eventName}`)
    }
  })
}

The general structure of our WebSocket route remains largely unchanged with the following exception: instead of using htmx’s HX-Trigger-Name header, we look for the event property we’re now sending back as part of the data and using that to determine which event to handle. (Again, in our simple example, there is only one event type but we’ve used a switch statement anyway so you can see how you could support other events in the future by adding additional case blocks to it.)

Finally, the Count fragment remains unchanged.

Here it is, again, for reference:

if (kitten.db.counter === undefined) kitten.db.counter = { count: 0 }

export default function Count () {
  return kitten.html`
    <div
      id='counter'
      aria-live='assertive'
      hx-swap-oob='morph'
      style='font-size: 3em; margin: 0.25em 0;'
    >
      ${kitten.db.counter.count}
    </div>
  `
}

Goodbye, magic! (Part 3: goodbye, Kitten; hello plain old Node.js)

So we just saw that Kitten’s Streaming HTML workflow can be created by writing some plain old client-side JavaScript instead of using the htmx library (which, of course, is just plain old client-side JavaScript that someone else wrote for you).

But we are still using a lot of Kitten magic, including its file system-based routing with its convenient WebSocket routes, its first-class support for JavaScript Database (JSDB), etc.

What would the Streaming HTML counter example look like if we removed Kitten altogether and created it in plain Node.js?

🐱 Kitten itself uses Node.js as its runtime. It installs a specific version of Node.js – separate from any others you may have installed in your system – for its own use during the installation process.

Streaming HTML, plain Node.js version

To follow along with this final, plain Node.js version of the Streaming HTML example, make sure you have a recent version of Node.js installed. (Kitten is regularly updated to use the latest LTS version so that should suffice for you too.)

First off, since this is a Node.js project, let’s initialise our package file using npm so we can add three Node module dependencies that we previously made use of without knowing via Kitten.

Create a new folder for the project and switch to it.
```
mkdir count-node
cd count-node
```
Initialise your package file and install the required dependencies – the ws WebSocket library as well as Small Technology Foundation’s https and JSDB libraries.
Of the Small Technology Foundation modules, the former is an extension of the standard Node.js https library that manages TLS certificates for you automatically both locally during development and via Let’s Encrypt in production and the latter is our in-process JavaScript database.
```
npm init --yes
npm i ws @small-tech/https @small-tech/jsdb 
```
Tell Node we will be using ES Modules (because, hello, it’s 2024) by adding "type": "module" to the package.json file (do you get the feeling I just love having to do this every time I start a new Node.js project?)
Either do so manually or use the following one-line to make yourself feel like one of those hackers in the movies:^[5]
```
sed -i '0,/,/s/,/,\n  "type": "module",/' package.json
```

Create the application.

// Import dependencies.
import path from 'node:path'
import { parse } from 'node:url'
import { WebSocketServer } from 'ws'
import JSDB from '@small-tech/jsdb'
import https from '@small-tech/https'

// Find the conventional place to put data on the file system.
// This is where we’ll store our database.
const dataHome = process.env.XDG_DATA_HOME || path.join(process.env.HOME, '.local', 'share')
const dataDirectory = path.join(dataHome, 'streaming-html-counter')
const databaseFilePath = path.join(dataDirectory, 'db')

/** JavaScript database (JSDB). */
const db = JSDB.open(databaseFilePath)

// Initialise count.
if (db.counter === undefined) db.counter = { count: 0 }

/**
  A WebSocket server without its own http server
  (we use our own https server).
*/
const webSocketServer = new WebSocketServer({ noServer: true })
webSocketServer.on('connection', ws => {
  ws.on('error', console.error)

  ws.on('message', message => {
    const data = JSON.parse(message.toString('utf-8'))

    if (data.event === undefined) {
      console.warn('No event found in message, cannot route call.', message)
      return
    }

    switch (data.event) {
      case 'update':
        db.counter.count += data.value
        ws.send(Count())
      break

      default:
        console.warn(`Unexpected event: ${eventName}`)
    }
  })
})

/**
  An HTTPS server instance that automatically
  handles TLS certificates.
*/
const httpsServer = https.createServer((request, response) => {
  const urlPath = parse(request.url).pathname

  switch (urlPath) {
    case '/':
      response.end(renderIndexPage())
    break

    default:
      response.statusCode = 404
      response.end(`Page not found: ${urlPath}`)
    break
  }
})

// Handle WebSocket upgrade requests.
httpsServer.on('upgrade', (request, socket, head) => {
  const urlPath = parse(request.url).pathname

  switch (urlPath) {
    case '/count.socket':
      webSocketServer.handleUpgrade(request, socket, head, ws => {
        webSocketServer.emit('connection', ws, request)
      })
    break

    default:
      console.warn('No WebSocket route exists at', urlPath)
      socket.destroy()
  }
})

// Start the server.
httpsServer.listen(443, () => {
  console.info(' 🎉 Server running at https://localhost.')
})

// TO get syntax highlighting in editors that support it.
const html = String.raw
const css = String.raw

/**
  Renders the index page HTML.
*/
function renderIndexPage() {
  return html`
    <!doctype html>
    <html lang='en'>
      <head>
        <title>Counter</title>
        <style>
          ${styles}
        </style>
      </head>
      <body>
        <h1>Counter</h1>
        ${Count()}
        <button onclick='update(-1)' aria-label='decrement'>-</button>
        <button onclick='update(1)' aria-label='increment'>+</button>
        <script>
          ${clientSideJS.render()}
        </script>
      </body>
    </html>
  `
}

/** The Count fragment. */
function Count () {
  return html`
    <div
      id='counter'
      aria-live='assertive'
      style='font-size: 3em; margin: 0.25em 0;'
    >
      ${db.counter.count}
    </div>
  `
}

/**
  This is the client-side JavaScript we render into the page.
  It’s encapsulated in a function so we get syntax highlighting,
  etc. in our editors.
*/
function clientSideJS () {
  const socketUrl = `wss://${window.location.host}/count.socket`
  const ws = new WebSocket(socketUrl)
  ws.addEventListener('message', event => {
    const updatedElement = event.data

    // Get the ID of the new element.
    const template = document.createElement('template')
    template.innerHTML = updatedElement
    const idOfElementToUpdate = template.content.firstElementChild.id

    // Swap the element with the new version.
    const elementToUpdate = document.getElementById(idOfElementToUpdate)
    elementToUpdate.outerHTML = updatedElement
  })

  function update (value) {
    ws.send(JSON.stringify({value}))
  }
}
clientSideJS.render = () => clientSideJS.toString().split('\n').slice(1, -1).join('\n')

/**
  Subset of relevant styles pulled out from Water.css.
 (https://watercss.kognise.dev/)
*/
const styles = css`
  :root
  {
    --background-body: #fff;
    --selection: #9e9e9e;
    --text-main: #363636;
    --text-bright: #000;
    --text-muted: #70777f;
    --links: #0076d1;
    --focus: #0096bfab;
    --form-text: #1d1d1d;
    --button-base: #d0cfcf;
    --button-hover: #9b9b9b;
    --animation-duration: 0.1s;
  }

  @media (prefers-color-scheme: dark) {
    :root {
      --background-body: #202b38;
      --selection: #1c76c5;
      --text-main: #dbdbdb;
      --text-bright: #fff;
      --focus: #0096bfab;
      --form-text: #fff;
      --button-base: #0c151c;
      --button-hover: #040a0f;
    }
  }

  ::selection {
    background-color: #9e9e9e;
    background-color: var(--selection);
    color: #000;
    color: var(--text-bright);
  }

  body {
    font-family: system-ui, -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen', 'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue', 'Segoe UI Emoji', 'Apple Color Emoji', 'Noto Color Emoji', sans-serif;
    line-height: 1.4;
    text-rendering: optimizeLegibility;
    color: var(--text-main);
    background: var(--background-body);
    margin: 20px auto;
    padding: 0 10px;
    max-width: 800px;
  }

  h1 {
    font-size: 2.2em;
    font-weight: 600;
    margin-bottom: 12px;
    margin-top: 24px;
  }

  button {
    font-size: inherit;
    font-family: inherit;
    color: var(--form-text);
  	background-color: var(--button-base);
    padding: 10px;
  	padding-right: 30px;
  	padding-left: 30px;
    margin-right: 6px;
    border: none;
    border-radius: 5px;
    outline: none;
    cursor: pointer;
    -webkit-appearance: none;

  	transition: background-color var(--animation-duration) linear, border-color var(--animation-duration) linear, color var(--animation-duration) linear, box-shadow var(--animation-duration) linear, transform var(--animation-duration) ease;
  }

  button:focus {
    box-shadow: 0 0 0 2px var(--focus);
  }

  button:hover {
    background: var(--button-hover);
  }

  button:active {
    transform: translateY(2px);
  }
`

So that is considerably longer (although almost half of it is, of course, CSS). And while we haven’t recreated Kitten with a generic file system-based router, etc., we have still designed the routing so new routes can easily be added to the project. Similarly, while our client-side DOM manipulation is very basic compared to everything htmx can do, it is still generic enough to replace any element it gets based on its ID.

I hope this gives you a solid idea of how the Streaming HTML flow works, how it is implemented in Kitten, how it can be implemented using htmx, and even in plain JavaScript.

Maybe this will even inspire you to port it to other frameworks and languages.

Next tutorial: End-to-end encrypted peer-to-peer Small Web apps

A component or fragment in Kitten is just a function that returns an HTML element.
In Kitten, your components and fragments can take properties (or ‘props’) and return HTML using the special kitten.html JavaScript tagged template string.
In fact, the only difference between a component and a fragment is their indented usage. If an element is intended to be used in multiple places on a page, it is called a component and, for example, does not contain an id. If, on the other hand, an element is meant to be used only once on the page, it is called a fragment and can contain an id.
You can, of course, pass an id attribute – or any other standard HTML attribute – to any component when instantiating it. When creating your components, you just have to make sure you pass these standard props to your component. ↩︎
You would use the request reference, if, for example, you wanted to access session data which would be available at request.session if your request parameter was named request. In our example, since we’re not using the first argument, we prefix our parameter with an underscore to silence warnings about unused arguments in our editor. ↩︎
For example, see the Kitten Chat sample application for use of the all() method. ↩︎
In fact, if we wanted to get really fancy, we could have bound the render() function to the clientSideJS() function so we could have referred to the latter from the former using this:
```
function clientSideJS () {
  //…
}

clientSideJS.render = (
  function () {
    return this
      .toString()
      .split('\n')
      .slice(1, -1)
      .join('\n')
  }
).bind(clientSideJS)
```
Notice that we cannot use a closure – also known as an arrow function expression in JavaScript – because the this reference in closures is derived from the lexical scope and cannot be changed at run-time. Lexical scope is just fancy computer science speak for ‘where it appears in the source code’. In this case, it means that since we’d be defining the closure at the top-level of the script, it would not have a this reference. ↩︎
It basically says “in the range of the start of the file to the first comma, replace any commas you find…” – yes, that’s the first comma, I know, but it’s sed, things were different back then – “…with a comma followed by the line we want to insert.” ↩︎

Index › 19. Streaming HTML

Topics covered

Streaming HTML

O counter! My counter!

What just happened?

Where next? (Choose your own adventure.)

Draw Together

Let’s break it down

Initial page render

Deconstructing the page

Components and fragments

Kitten’s magic database

A magic connection

High-level flow

Peeking behind the curtain: htmx

Progressive enhancement

Goodbye, magic! (Part 1: Goodbye, syntactic sugar; hello, htmx)

index.page.js

Count.fragment.js

count.socket.js

Goodbye, magic! (Part 2: goodbye, htmx; hello, plain old client-side JavaScript)

index.page.js

Documenting the (overly) clever bits

count.socket.js

Goodbye, magic! (Part 3: goodbye, Kitten; hello plain old Node.js)

Streaming HTML, plain Node.js version

Like this? Fund us!