Implement Node-API in bun.js (napi)

[Jarred-Sumner]

Node-API is Node.js' native addon API. Bun implements Node-API so that native addons built for Node.js work in Bun without modification or recompilation.

If a native addon crashes or misbehaves in Bun, please open a new issue with a reproduction rather than commenting here.

Status: API surface complete

Every napi_* and node_api_* function is implemented and exported. Bun runs Node.js' own js-native-api and node-api test suites in CI on every commit (see test/napi/node-napi-tests/).

Suite	Pass	Todo	%
js-native-api (engine-agnostic core)	53	0	100%
node-api (Node.js-runtime-specific)	21	23	47.7%
Combined	74	23	76.3%

_{Counted on Linux. macOS/Windows each have one extra platform-specific todo (73/97, 75.3%).}

The node-api todos are concentrated in async_hooks integration, uv_* loop/threadpool semantics, worker-terminate finalization, and fatal-error reporting — see the todoIf entries in test/napi/node-napi-tests/test/node-api/*/do.test.ts for the live list.

Remaining work is behavioral parity in those areas and edge-case compatibility with specific packages, not missing API surface.

Implementation

Where	What
src/bun.js/bindings/napi.cpp	Most napi_* functions, JSC integration
src/napi/napi.zig	Threadsafe functions, async work, event loop, handle scopes
src/bun.js/bindings/v8/	V8 C++ API shim (for addons that bypass N-API)
src/symbols.txt	Exported symbol list

Known gaps & caveats

• async_hooks integration — napi_async_init, napi_async_destroy, napi_make_callback, napi_open_callback_scope, and napi_close_callback_scope all work, but do not emit async_hooks lifecycle events. We don't plan to implement this. Node.js itself marks async_hooks as Experimental and discourages its use: "Please migrate away from this API, if you can. We do not recommend using the createHook, AsyncHook, and executionAsyncResource APIs as they have usability issues, safety risks, and performance implications." Bun implements the stable AsyncLocalStorage API instead, which is what Node recommends migrating to.
• libuv — a subset of uv_* symbols is exported for addons that link against libuv directly. On Linux/macOS, Bun does not run on libuv, so uv_default_loop() is not the runtime's event loop — addons must use napi_get_uv_event_loop (N-API addons using uv_default_loop for uv_async don’t fire callbacks; using napi_get_uv_event_loop fixes it #23192). We don't currently plan to implement all of libuv inside Bun, but we may revisit if enough bug reports show it's needed in practice — tracked in Implement libuv functions #18546.
• Worker termination — finalizer and buffer-callback ordering during Worker.terminate() doesn't fully match Node. Tracked as part of Worker & worker_threads stability tracking issue #15964 alongside other worker_threads stability work.
• napi_adjust_external_memory reports to JSC's GC heuristic but does not behave identically to V8.

Full function list — 156/156 implemented

napi_*

• napi_acquire_threadsafe_function
• napi_add_async_cleanup_hook
• napi_add_env_cleanup_hook
• napi_add_finalizer
• napi_adjust_external_memory
• napi_async_destroy
• napi_async_init
• napi_call_function
• napi_call_threadsafe_function
• napi_cancel_async_work
• napi_check_object_type_tag
• napi_close_callback_scope
• napi_close_escapable_handle_scope
• napi_close_handle_scope
• napi_coerce_to_bool
• napi_coerce_to_number
• napi_coerce_to_object
• napi_coerce_to_string
• napi_create_array
• napi_create_array_with_length
• napi_create_arraybuffer
• napi_create_async_work
• napi_create_bigint_int64
• napi_create_bigint_uint64
• napi_create_bigint_words
• napi_create_buffer
• napi_create_buffer_copy
• napi_create_dataview
• napi_create_date
• napi_create_double
• napi_create_error
• napi_create_external
• napi_create_external_arraybuffer
• napi_create_external_buffer
• napi_create_function
• napi_create_int32
• napi_create_int64
• napi_create_object
• napi_create_promise
• napi_create_range_error
• napi_create_reference
• napi_create_string_latin1
• napi_create_string_utf16
• napi_create_string_utf8
• napi_create_symbol
• napi_create_threadsafe_function
• napi_create_type_error
• napi_create_typedarray
• napi_create_uint32
• napi_define_class
• napi_define_properties
• napi_delete_async_work
• napi_delete_element
• napi_delete_property
• napi_delete_reference
• napi_detach_arraybuffer
• napi_escape_handle
• napi_fatal_error
• napi_fatal_exception
• napi_get_all_property_names
• napi_get_and_clear_last_exception
• napi_get_array_length
• napi_get_arraybuffer_info
• napi_get_boolean
• napi_get_buffer_info
• napi_get_cb_info
• napi_get_dataview_info
• napi_get_date_value
• napi_get_element
• napi_get_global
• napi_get_instance_data
• napi_get_last_error_info
• napi_get_named_property
• napi_get_new_target
• napi_get_node_version
• napi_get_null
• napi_get_property
• napi_get_property_names
• napi_get_prototype
• napi_get_reference_value
• napi_get_threadsafe_function_context
• napi_get_typedarray_info
• napi_get_undefined
• napi_get_uv_event_loop
• napi_get_value_bigint_int64
• napi_get_value_bigint_uint64
• napi_get_value_bigint_words
• napi_get_value_bool
• napi_get_value_double
• napi_get_value_external
• napi_get_value_int32
• napi_get_value_int64
• napi_get_value_string_latin1
• napi_get_value_string_utf16
• napi_get_value_string_utf8
• napi_get_value_uint32
• napi_get_version
• napi_has_element
• napi_has_named_property
• napi_has_own_property
• napi_has_property
• napi_instanceof
• napi_is_array
• napi_is_arraybuffer
• napi_is_buffer
• napi_is_dataview
• napi_is_date
• napi_is_detached_arraybuffer
• napi_is_error
• napi_is_exception_pending
• napi_is_promise
• napi_is_typedarray
• napi_make_callback
• napi_module_register
• napi_new_instance
• napi_object_freeze
• napi_object_seal
• napi_open_callback_scope
• napi_open_escapable_handle_scope
• napi_open_handle_scope
• napi_queue_async_work
• napi_ref_threadsafe_function
• napi_reference_ref
• napi_reference_unref
• napi_reject_deferred
• napi_release_threadsafe_function
• napi_remove_async_cleanup_hook
• napi_remove_env_cleanup_hook
• napi_remove_wrap
• napi_resolve_deferred
• napi_run_script
• napi_set_element
• napi_set_instance_data
• napi_set_named_property
• napi_set_property
• napi_strict_equals
• napi_throw
• napi_throw_error
• napi_throw_range_error
• napi_throw_type_error
• napi_type_tag_object
• napi_typeof
• napi_unref_threadsafe_function
• napi_unwrap
• napi_wrap

node_api_*

• node_api_create_buffer_from_arraybuffer
• node_api_create_external_string_latin1
• node_api_create_external_string_utf16
• node_api_create_property_key_latin1
• node_api_create_property_key_utf16
• node_api_create_property_key_utf8
• node_api_create_syntax_error
• node_api_get_module_file_name
• node_api_post_finalizer
• node_api_symbol_for
• node_api_throw_syntax_error

Related tracking

• V8 C++ API compatibility: Support V8 C++ APIs for "nan" addons and other packages to work #4290
• libuv functions: Implement libuv functions #18546
• Worker / worker_threads stability: Worker & worker_threads stability tracking issue #15964
• Open napi-labeled bugs: issue list

[kriszyp]

I made a test repo that exercises a number of things that are currently having problems with the bun napi interface:
https://github.com/kriszyp/bun-test-apw
Clone that, npm install it, and then you can compare node index.js to bun index.js:

• Occasional segfaults in calls through napi_call_threadsafe_function
• napi_release_threadsafe_function not triggering thread_finalize_cb
• NAPI_MODULE macro fails with symbol 'napi_register_module_v1' not found in native module (easy workaround, since NAPI_MODULE_INIT works fine)
• If you get through those, I think the Callback().Call() ends up triggering escapable scopes, but that's not that important.

You will have uncomment NAPI_MODULE from apw.cpp (and comment NODE_MODULE_INIT) when you want to test that, figured the other things were more important.

Hope that helps, thanks for the great work!

[kriszyp]

I also wanted to check to see if this is roughly the right approach loading a module for napi + ffi usage. I think the hybrid strategy is good: use napi for general purpose stuff like building objects, async work, and use ffi for "hot" functions that need to be fast. Here is an example from my code where I am loading the platform's binary with the napi loading package, and then, if running in bun, loading it through the FFI loader. But, I think this raises a couple points:

• I assume calling dlopen twice (once through the napi loader, once through the ffi loader) is cached at some level of the OS and doesn't result in duplicate binary footprints in memory
• require.resolve doesn't seem to exist in bun, and this seems challenging for loading a platform-specific package with a binary, which is direction that we are going with node-gyp-build: https://github.com/prebuild/node-gyp-build/pull/45/files#diff-e727e4bdf3657fd1d798edcd6b099d6e092f8573cba266154583a746bba0f346R50

Anyway, here is my loading sequence right now:

import { dirname, join, default as pathModule } from 'path';
import { fileURLToPath } from 'url';
import loadNAPI from 'node-gyp-build'; // loads the correct napi binary, using the current OS/arch

let dirName = (typeof __dirname == 'string' ? __dirname : // for bun, which doesn't have fileURLToPath
	dirname(fileURLToPath(import.meta.url))).replace(/dist$/, ''); // for node, which doesn't have __dirname in ESM

export let nativeAddon = loadNAPI(dirName);
if (process.isBun) {
	const { dlopen, FFIType } = require('bun:ffi');
	// load the same napi module as above, but use bun's ffi
	// ideally we should use require.resolve to find a package, but here just hoping it is a sibling package in node_modules
	let libPath = join(dirName, '..', 'lmdb-linux-' + process.arch, 'node.napi.node');
	let lmdbLib = dlopen(libPath, {
		getByBinary: { args: [FFIType.f64, FFIType.u32], returns: FFIType.u32},
		iterate: { args: [FFIType.f64], returns: FFIType.i32},
		position: { args: [FFIType.f64, FFIType.u32, FFIType.u32, FFIType.u32, FFIType.f64], returns: FFIType.i32},
		write: { args: [FFIType.f64, FFIType.f64], returns: FFIType.i32},
		resetTxn: { args: [FFIType.f64], returns: FFIType.u8},
	});
	Object.assign(nativeAddon, lmdbLib.symbols);
}

[Jarred-Sumner]

let dirName = (typeof __dirname == 'string' ? __dirname : // for bun, which doesn't have fileURLToPath

For bun you can also do import.meta.dir and/or import.meta.path, which at least saves you from the typeof check

I assume calling dlopen twice (once through the napi loader, once through the ffi loader) is cached at some level of the OS and doesn't result in duplicate binary footprints in memory

This is a good point. I don't know either, but worth checking.

require.resolve doesn't seem to exist in bun, and this seems challenging for loading a platform-specific package with a binary, which is direction that we are going with node-gyp-build: https://github.com/prebuild/node-gyp-build/pull/45/files#diff-e727e4bdf3657fd1d798edcd6b099d6e092f8573cba266154583a746bba0f346R50

This is an oversight – require.resolve should exist but does not currently. There are at least 5 other ways you can resolve without loading modules though:

• Bun.resolve(path, fromPath) (async)
• Bun.resolveSync(path, fromPath)
• import.meta.resolve(path) (async)
• import.meta.resolveSync(path) (sync)
• Loader.resolveSync (lower-level, uses JSC builtin ModuleLoader and may be disabled at some point)

[kriszyp]

assume calling dlopen twice (once through the napi loader

My evidence is that when I set the value of a static C variable from a NAPI call, that value was accessible/correct from an FFI call, so I think working correctly, but I am not claiming this as absolute proof.

This is an oversight – require.resolve should exist but does not currently

Ok, so you plan on adding require.resolve? I think in node-gyp-build it would either need to go through createRequire(libraryPath), or use the two arg version (require.resolve(id, { path: [ libraryPath ] }). node-gyp-build is CJS module, so unfortunately import.meta.resolve can't be used there (and can't even be mentioned, if (false) import.meta.resolve('test') will even fail in Node in CJS mode).

Anyway, thanks for the progress here, I know these are probably annoying details of the napi/node machinery, but I think node-gyp-build is pretty commonly used.

[Jarred-Sumner]

Ok, so you plan on adding require.resolve? I think in node-gyp-build it would either need to go through createRequire(libraryPath), or use the two arg version (require.resolve(id, { path: [ libraryPath ] }). node-gyp-build is CJS module, so unfortunately import.meta.resolve can't be used there (and can't even be mentioned, if (false) import.meta.resolve('test') will even fail in Node in CJS mode).

Support for both of these was added in Bun v0.0.83 (released today)

[devongovett]

FYI, I tried to run parcel-css with Bun, but got the following error (logging with DYLD_PRINT_APIS). Not sure if this is useful. I tried to debug with lldb but wasn't able to due to macOS code signing. I guess I'd have to compile bun myself to debug further.

dyld[50777]: dlopen("/Users/devongovett/dev/parcel-css/parcel-css.darwin-arm64.node", 0x00000001)
dyld[50777]:       dlopen(parcel-css.darwin-arm64.node) => 0x2089957e0
dyld[50777]: dlsym(0x2089957e0, "napi_register_module_v1")
dyld[50777]:      dlsym("napi_register_module_v1") => 0x110ab41b0
dyld[50777]: dlsym(0xfffffffffffffffe, "getentropy")
dyld[50777]:      dlsym("getentropy") => 0x183fd8f5c

Example:

const parcelCss  = require('./parcel-css.darwin-arm64.node');
parcelCss.transform({
  filename: 'test.css',
  code: Buffer.from('.foo { color: red }'),
  minify: true
});

[kjvalencik]

If I attempt to load a native module that uses Node-API, I get a crash on missing symbols. Additionally, I don't see any of the Node-API symbols in the bun binary (0.1.1, macOS x86_64).

Is there something additional that needs to be done to load these symbols or was this possibly optimized out in the latest release?

$ symbols $(which bun) | grep napi

Edit: It appears to be only the macOS release where they are missing. The symbols are present in linux.

[Pruxis]

What do you mean by async_hooks resource tracking will not be supported, it will just ignore them.

As bluebird uses async_hooks and requires it explicitly like this for example:

var AsyncResource = util.isNode && util.nodeSupportsAsyncResource ?
    require("async_hooks").AsyncResource : null;

Which then results into:

Could not resolve: "async_hooks". Maybe you need to "bun install"?

[Jarred-Sumner]

async_hooks will need to be polyfilled but initially just disabled so it is a no-op rather than failure on require() Also in bun at least, you should try to avoid Bluebird. It’s a many X slowdown

…

On Sun, Jul 10, 2022 at 10:13 AM Laurens Lavaert ***@***.***> wrote: What do you mean by async_hooks resource tracking will not be supported, it will just ignore them. As bluebird uses async_hooks and requires it explicitly like this for example: var AsyncResource = util.isNode && util.nodeSupportsAsyncResource ? require("async_hooks").AsyncResource : null; Which then results into: Could not resolve: "async_hooks". Maybe you need to "bun install"? — Reply to this email directly, view it on GitHub <#158 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFNGSZC646YQEGC6NGCNRTVTMAFNANCNFSM5VDV6N3Q> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[droukd]

Also in bun at least, you should try to avoid Bluebird. It’s a many X slowdown

What would you say is recommended to use (instead)? Par example,

(async () => { for (let i=0; i<800000; i++) { await functionCall(i); } })();

...would work, but only one execution at a time.
(And without limitation program will produce errors.)
(Promise.map([...Array(800000).keys()], functionCall, {concurrency: 8});)

P. S. http://bluebirdjs.com/docs/benchmarks.html claims to be faster (in node) than async-await.
P. S. If I am not mistaken, another problem rises with big arrays: bun took 7 GB (and counting) of RAM for 800 000 keys. Speaking of slowdowns.

[artokun]

PrismaJS also requires async_hook

[harryrabin]

Hey folks! Is there any progress on napi_create_external?