lib,src,test,doc: add node:sqlite module

[cjihrig]

#53264 has been open for over a month with no objections, so I am opening this PR with an initial node:sqlite module. There is other functionality that could potentially be exposed in the future, but I believe this is enough for an experimental MVP.

Fixes: #53264

Summary: Node.js now includes a built-in sqlite module (require('node:sqlite')) that becomes available when using the --experimental-sqlite flag

The following example shows the basic usage of the node:sqlite module to open
an in-memory database, write data to the database, and then read the data back.

import{DatabaseSync}from'node:sqlite';constdatabase=newDatabaseSync(':memory:');// Execute SQL statements from strings.database.exec(` CREATE TABLE data( key INTEGER PRIMARY KEY, value TEXT ) STRICT`);// Create a prepared statement to insert data into the database.constinsert=database.prepare('INSERT INTO data (key, value) VALUES (?, ?)');// Execute the prepared statement with bound values.insert.run(1,'hello');insert.run(2,'world');// Create a prepared statement to read data from the database.constquery=database.prepare('SELECT * FROM data ORDER BY key');// Execute the prepared statement and log the result set.console.log(query.all());// Prints: [{key: 1, value: 'hello' },{key: 2, value: 'world' } ]

[nodejs-github-bot]

Review requested:

• @nodejs/gyp
• @nodejs/loaders
• @nodejs/startup

[targos]

I can't review the C++ code, but API and tests LGTM for a MVP.

[benjamingr]

Yay, happy to see this come to fruition. Let's land as experimental and iterate 🎉

[benjamingr]

It would be nifty to also support Symbol.dispose here but that can be in a follow up PR.

[benjamingr]

This is a lot more feature complete and ready than I was expecting, good job Colin!

[avivkeller]

This is adding a new feature, so shouldn't be semver-minor? (and with this kind of change, notable-change as well?)

[ShogunPanda]

Love it! Amazing job @cjihrig

[mcollina]

lgtm

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/60156/

[Qard]

Should we be concerned at all about the performance impact of people using these sync APIs within requests and blocking the event loop? With sync being the only option here I feel like the risk is increased.

If the concern is simplicity, I feel like only async would be better than only sync. Especially given top-level await. What reason is there to favour sync over async here?

I won't block on that concern, and in fact I think the work done so far is great. I would be happy landing what's here now. I just wanted to share my concern with going for sync first rather than async first.

[benjamingr]

Should we be concerned at all about the performance impact of people using these sync APIs within requests and blocking the event loop? With sync being the only option here I feel like the risk is increased.

SQLite runs in process and is often CPU bound, we do have async APIs for some CPU intensive stuff (like in crypto) but I suspect for most people this API is the better one performance wise (that said, in some cases an async API is preferable and we should expose that as well)

[thomasdao]

I'm really grateful that Sqlite is natively supported in NodeJS and want to thank @cjihrig and everyone involved for working on this feature. Thanks for choosing Sqlite - it's fast and much more reliable than most alternatives. This feature would be super useful for Electron users - currently most Sqlite bindings don't work well for Electron apps, and having native support for Sqlite would make it easier to interact with the database.

[louwers]

This is awesome! Is support for official SQLite extensions something that is being considered? In particular I am interested in the Session Extension.

[cjihrig]

@louwers it hasn't been discussed AFAIK, but there is a PR for it, so it's not out of the question: #53900.

EDIT: Oh, that one is built in already, we would just need to support enabling it.

[louwers]

@cjihrig Actually the Session Extension is not a runtime loadable extension. It needs to be compiled into SQLite.

To be able to use it with Node.js you would need to wrap its C API: https://www.sqlite.org/session/intro.html

[TheOneTheOnlyJJ]

To be able to use it with Node.js you would need to wrap its C API: https://www.sqlite.org/session/intro.html

+1 for this.

Supporting all of the sqlite3 amalgamation extensions should be a strong goal for Node.js (through wrapping the C APIs of each).
While at first glance they may not look relevant, these extensions are very powerful & vital for the projects that make use of them.

Maybe a new issue concerning this topic specifically should be opened?

[louwers]

@cjihrig@TheOneTheOnlyJJ I opened a PR so we can have some discussion about the API (among other things): #54181

[webda2l]

@cjihrig Thanks for your work!

Regarding this case, I am a bit skeptical because better-sqlite3 allows for more named parameters to be passed than those actually used in the SQL. This can be useful when you build your SQL with conditions in a literal string, for example.