neighbourhoodie-nnh-logo

CouchDB 3.4.1 New Feature QuickJS posted Monday, September 23, 2024 by The Neighbourhoodie CouchDB Team

QuickJS is a JavaScript engine that can do the same work as CouchDB’s existing JavaScript engine SpiderMonkey, but with different tradeoffs that have advantages for CouchDB users.

Introduction

First and foremost: SpiderMonkey is still here and is still the default JavaScript engine in CouchDB. So when you upgrade to CouchDB 3.4.1, nothing changes for you.

But: you can start using QuickJS alongside SpiderMonkey for individual design documents by setting "language": "javascript_quickjs". This will cause all JavaScript in this design document to be evaluated by QuickJS.

Why QuickJS

To learn why QuickJS is an exciting new option for CouchDB, it is easiest to see how it compares to SpiderMonkey. As with the Nouveau Search rewrite, there are multiple areas to cover:

  1. Maintainability 1: SpiderMonkey is the JavaScript engine built for Firefox. Even though it is available as a standalone library, it is not treated as one and if Firefox needs substantial API changes, SpiderMonkey follows suit. This makes it hard for the CouchDB development team to keep up with newer versions of SpiderMonkey. QuickJS in contrast is a true standalone library that has a slow changing API.
  2. Maintainability 2: SpiderMonkey is 700+ files of C/C++/Rust code, includes an arcane Python/Autotools build system and is generally very large™. QuickJS in comparison is just a handful of C files that are straightforward to read, build and use. In fact, QuickJS is so tiny in comparison to SpiderMonkey that we are bundling its source with CouchDB so you don’t have to install it as a separate dependency! — This is useful because RedHat 9 dropped SpiderMonkey as an available package.
  3. Security: It is easier to isolate JavaScript operations from separate databases from each other with sandboxing. This enhances security when folks with different security contexts are accessing the same CouchDB server.
  4. Performance 1: It is early days, but the CouchDB team is already seeing the following improvements of QuickJS over SpiderMonkey:
    1. 4x faster than SpiderMonkey 1.8.5
    2. 5x faster than SpiderMonkey 91
    3. 6x reduced memory usage per couchjs process (5MB vs 30MB)
      1. Neighbourhoodie has even observed 10x on arm64 macOS (3,3MB vs 33MB)
  5. Performance 2: QuickJS allows the ahead-of-time compilation of JavaScript to bytecode. This makes it faster to launch new couchjs processes with QuickJS.

What do I need to know?

QuickJS is a smaller and faster alternative to SpiderMonkey that ships with CouchDB 3.4.1 as an option.

SpiderMonkey remains supported and the default.

You can safely try QuickJS out on a design document by design document basis without fully committing to it.

CouchDB 3.4.1 also ships with a scanner plugin that takes all your design docs and runs them through both SpiderMonkey and QuickJS and records any differences in the CouchDB log. If you enable the scanner and your log shows no recorded differences after it scanned all your design docs, you can safely upgrade everything to QuickJS and reap the performance benefits.

There are two documented incompatibilities between SpiderMonkey 91 (the latest that CouchDB supports) and QuickJS:

  1. The $1…$9 fields on the RegExp object are a deprecated JavaScript feature that is not available in QuickJS.
  2. The toString() function on a Date object in QuickJS does not include the timezone e.g. (CEST), just the offset. SpiderMonkey includes the timezone abbreviation as a string.

Safe upgrades with the Scanner Plugin

To demonstrate how the scanner plugin can be used to detect whether your views are relying on one of these incompatibilities, let’s first create a design document with the following map function:

function (doc) {
	if (doc.text) {
		emit(doc._id, (new Date(doc.text)).toString())
	}
}

And then create a bunch of regular documents with a text property:

{
  "_id": "04d92f0418cc0a279a924a871f0007fc",
  "_rev": "1-1182d01f9ab032a4deea82f1df05d107",
  "text": "foo"
}
{
  "_id": "04d92f0418cc0a279a924a871f002049",
  "_rev": "1-8bcbb8604cf7b75c5f5c46a956c768d1",
  "text": "1234"

}

Now when you start CouchDB, the scanner will start running in the background and you will see the following output in your CouchDB logs:

[warning] 2024-09-22T11:19:52.803036Z couchdb@localhost <0.541.0> -------- couch_quickjs_scanner_plugin s:1727003992-6302447d8f12 starting.
[warning] 2024-09-22T11:19:53.018434Z couchdb@localhost <0.541.0> -------- couch_quickjs_scanner_plugin s:1727003992-6302447d8f12 db:app/00000000-7fffffff ddoc:_design/test view validation failed {map_doc,<<"04d92f0418cc0a279a924a871f002049">>,[[[<<"04d92f0418cc0a279a924a871f002049">>,<<"Sun Jan 01 1234 00:53:00 GMT+0053">>]]],[[[<<"04d92f0418cc0a279a924a871f002049">>,<<"Sun Jan 01 1234 01:00:00 GMT+0100 (CET)">>]]]}
[notice] 2024-09-22T11:19:53.429922Z couchdb@localhost <0.536.0> -------- couch_scanner_server : couch_scanner_plugin_ddoc_features finished

These are three lines, one for starting the scanner plugin, one for our design doc and one for the scanner plugin stopping.

By default, the scanner plugin runs when you start CouchDB, but you can also configure it to start at a later time using the [couch_quickjs_scanner_plugin] after configuration setting and changing it either to a unix timestamp (ex. 1712345678) or a date/time setting: YYYY-MM-DD, YYYY-MM-DDTHH, YYYY-MM-DDTHH:MM.

And you can configure it to not only start a certain point but to periodically repeat by setting [couch_quickjs_scanner_plugin] repeat where the default is restart for no repeats after the first run or $num_$timeunit (ex.: 1000_sec, 60_min, 12_hours, 24_hour, 2_days, 3_weeks, 1_month) or $weekday (ex.: mon, tuesday, Wed, etc)

The second line of the output above contains the information we are looking for, if a bit complex, so let’s break it down:

The database shard that includes the issue: db:app/00000000-7fffffff

The design document with the issue: ddoc:_design/test

The issue text: view validation failed

From here we know that that view emits data that looks different when run through SpiderMonkey and QuickJS. Now let’s see which of our documents caused the issue and what the differences are (with some extra punctuation elided):

[
<<"04d92f0418cc0a279a924a871f002049">>,
<<"Sun Jan 01 1234 00:53:00 GMT+0053">>
],
[
<<"04d92f0418cc0a279a924a871f002049">>,
<<"Sun Jan 01 1234 01:00:00 GMT+0100 (CET)">>
]

The first group is the QuickJS output and the second group is the SpiderMonkey output. You can see that in the SpiderMonkey output, the (CET) is present when it is missing in the QuickJS output.

There are a number of other configuration options for the QuickJS scanner plugin and for other plugins in general. These include options for skipping databases, design documents or documents.

General scanner options include rates for how fast to process databases, design documents and documents. Follow the link for a full picture.

What should I do?

Do start migrating design documents to QuickJS, ideally with the help of the scanner plugin, at your earliest convenience.

If you have any questions, do get in touch.

« Back