How to use a dynamic library written in Rust with Node.js

Let’s take a look at a hypothetical problem: we have an existing project written in Node.js and we want to optimize the performance of a function that performs heavy calculations. The decision was made to do this by using Rust, a low-level programming language that handles the same task much faster.

Different approaches can be used for implementation (dynamic libraries, C++ Node.js add-ons, etc.). In this article, I intend to use FFI (Foreign Function Interface) and a C-compatible library written in Rust. This method is not the most efficient in terms of performance but nevertheless, it can give a good gain of optimization.

However, the example given here serves only to demonstrate how you can organize interaction between Node.js and Rust — it has nothing to do with the task described above. Nevertheless, the techniques used here are the key to its implementation!

Task 1: Swapping Strings

Share strings between the Rust library and Node.js to illustrate the interaction.

Task 2: Structures

Use structures to share data between the library and Node.js

Node.js

Note: the current solution works only with Node.js v16.18

Let’s write a Node.js script that accesses the dynamic library and calls its functions with passing data back and forth.

To do it we have to install the following dependencies:

• ffi-napi — an extension for loading and calling dynamic libraries
• ref -napi— an additional dependency to help us work with C-like types in JavaScript
• ref-struct-napi — the struct type implementation for Node.js

Let’s take a look at the index.js:

Despite the fact, the code is quite easy to read some lines still require explanation.

* 7 In order to share data with a dynamic library, we must arm ourselves with special types this library can work with. We cannot use the JS-array type within the library so instead, we will work with pointers (Pointer). To do this, we need the ref (C ++ addon) module (which, among other things, allows us to create our own types) as well as modules that implement the structure type and the array type. In this line, we create a C-like array type of 2 elements that will contain values ​​of type int64.

* 10 Since the library’s multiply function will return a struct, which is not represented in JavaScript, we need the ref-struct module that allows us to work with this data type and, in particular, convert it to a JavaScript object. At this point in the script, we create the OutputType structure (like Rust or C) with three type fields: int64, OutputArrayType, CString. The last type represents the so-called null-terminated string (NULL-terminated C-style strings).

* 24 To initialize access to the library’s functions we use the ffi.Library call which has the following signature:

ffi.Library ( path_to_library , { function_name : [ return_type , [ arg_type_1 , arg_type_2 , ...], ...]);

The return type for the multiply function is the OutputType structure we created earlier. The function argument is another structure.

You may have noticed that the return type and the argument type of the hello function is a simple string. This is the same as ref.types.CString so there is no problem here - I did it on purpose, for demonstration. The ref module allows you to use “string” and “number”. See the module reference for details.

* 35 We call the library’s hello function like a normal JavaScript object method.

* 41 We create a structure with data. Note that I intentionally omit the result property.

* 46 Accessing the property of the structure, just like we do with regular JS objects.

* 47 Converting to a regular array.

* 51 Since we cannot work with the structures in Node.js using its native capabilities we will convert it to an object. (Note that we can always access the structure fields directly using the JavaScript object property access notation).

Rust

Now let’s look at the complete listing of the library code:

Let’s start with the `hello` function.

* 1, * 3–5 We need some dependencies to work with the C-types.

* 7, * 56 The annotation tells the Rust compiler not to mangle the name of this function.

During the creation of the Rust library, the function name is changed in the compiled file (a technique called mangling). In order for the function to be called from other languages, we should disable this behavior.

* 8 I added this annotation to prevent the clippy linter from throwing the unsafe function error.

* 10 The function is declared as public (can be called outside of this module), and the extern keyword indicates that we are going to work with FFI, call it from another language. The “C” indicates which Application Binary Interface (ABI) to use. The ABI defines how to call a function at the linking level. In this case, it is C.

The function takes and returns values ​​of type * const c_char

* const T — is the “pointer” type (raw pointer).
c_char — is an equivalent of the char type in the C language.

* 14 Since we receive a pointer to the function input, and we want to work with a string, we need to execute an unsafe (for a number of reasons, for example, due to the lack of a validity guarantee) from_ptr method, which takes a pointer and returns a representation of the borrowed C-string
(& ‘a CStr). This structure allows us to use some useful methods. Since the function is unsafe, we wrap it in the unsafe * 11 block

* 17 Using the to_str() method to get a slice of the string

* 22 Return a new string as a pointer of type c_char

The `multiply` function and related structures

* 25, * 43 The annotation #[repr(C)] before the description of the structure Output tells the compiler that this type must be C-compatible (c- representation)

* 29, * 38 We return text data as a pointer

`Cargo.toml` for the library

To compile the library we will need to add the following lines to the Cargo.toml file:

[dependencies]
libc = “*”

[lib]
name = “_node_rust”
path = “src / lib.rs”
crate-type = [“dylib”]

These directives tell the compiler that we want to compile a dynamic library; we specify the path to the source file and the name.

Now you can compile (cargo build — release) library and run node index.js

As you can see from the example, we can quite conveniently use the dynamic library functions in Node.js, transmit data to it, and return results. This can be used to solve the problem I mentioned at the very beginning. So if you like JavaScript and Rust, it’s good to keep that in mind.

But the best way to understand the principle is to try it out. Follow
this link to find a repository of the described example and installation instructions. Good luck!

Links:

[1]. Foreign Function Interface
[2]. Article: Rust Inside Other Languages
[3]. Pointers in C programming language
[4]. Name mangling (Wikipedia)
[5]. Type: std / pointer
[6]. Type: std / c_char
[7]. Type layout: c-representation