Node js C++ Addon Overload Resolution - Refactor & Add C++ Type Converter

Remember node-overload-resolution ?

I've been working on it for a while to see how I can support automatic async support.

I always believed if you can work a little harder to save a lot of time later, its most likely going to pay off, one of the biggest motivation to write the node-overload-resolution project was to easily convert C++ libraries to Node js without changing the API too much.

One of the major roadblocks in the last version of node-overload-resolution to this goal is that v8 objects are not accessible through any other thread other than node js main thread since node doesn't release the locker on v8 vm. To solve this issue I thought and implemented a way to parse and store the v8 objects as their projected C++ objects. in essence copying v8::String to std::string, v8::Number to double and so on.

Some issues I've encountered with this approach is how to store v8 Objects, C++ Class wrappers and Arrays, but if I can actually tell the resolution module which C++ types each v8 type translates to, maybe it could work.

I've been working on this premise and implemented a converter and value holder on top of the working overload resolution module and currently it supports Number, Function, String, Buffer, a typed struct and ObjectWrap.

Currently there are issues with Function, its implemented as or::Callback, but the actual function is not called yet.

Other issues is that I couldn't find a translation for Promise, Proxy and RegExp.

I think this module is on the correct path at the moment, it should be relatively easy to support async function calls by adding a check if the matched function have an additional function callback parameter, caching the parameter value as C++ type, pushing the function call to libuv threadpool and executing the return value conversion and callbacks activation when the thread finishes execution.

You can find the code on the native_types branch.

The code is concentrated in 3 main components, the value_holder, the value_converter and the generic_value_holder. 

The value_holder is a derived template, storing the Value inside a template type.

template<typename T>
class value_holder : public value_holder_base {
public:
    T Value;
    value_holder() : value_holder_base() {}
    value_holder(T val) : value_holder_base(), Value(val) {}
    virtual ~value_holder() {}
};

The value_converter is a derived template with template specialization for each major type handled, the specialization is both for primitives (including v8 basic types) and for derived classes for IStructuredObject and ObjectWrap classes, allowing a more specific behavior for structures and C++ classes, such as parsing/creating new v8 objects as well as ObjectWrap::Wrap and ObjectWrap::Unwrap.

for example, this template specialization is for all derived classes of ObjectWrap, it wraps/unwraps to/from v8::Object:

template<typename T>
class value_converter<T*, typename std::enable_if<std::is_base_of<ObjectWrap, T>::value>::type> : publicvalue_converter_base {
public:

    virtual T* convert(v8::Local<v8::Value> from) {
        return or::ObjectWrap::Unwrap<T>(from.As<v8::Object>());
    }


    virtual v8::Local<v8::Value> convert(T* from) {
        return from->Wrap();
    }

    virtual v8::Local<v8::Value> convert(std::shared_ptr<value_holder_base> from) {
        auto from_value = std::dynamic_pointer_cast<value_holder<T*>>(from);
        return from_value->Value->Wrap();
    }

    virtual std::shared_ptr<value_holder_base> read(v8::Local<v8::Value> val) {
        auto parsed_value = std::make_shared<value_holder<T*>>();
        parsed_value->Value = convert(val);
        return parsed_value;
    }

};

lastly the generic_value_holder stores a pair of value_holder and value_converter and by that it can act as some sort of non-convertible variant that can return a v8 object from intended C++ types.

class generic_value_holder {
private:
    std::shared_ptr< or ::value_converter_base> _prefetcher;
    std::shared_ptr< or ::value_holder_base> _value;
public:
    void Set(std::shared_ptr< or ::value_converter_base> value_converter, std::shared_ptr< or ::value_holder_base> value) {
        _prefetcher = value_converter;
        _value = value;
    }

    template<typename T>
    void Set(T returnValue) {
        //store value_converter type
        auto returnPrefetcher = std::make_shared < or ::value_converter<T>>();

        //store value inside a valueholder
        auto valueHolder = std::make_shared < or ::value_holder<T>>();
        valueHolder->Value = returnValue;

        Set(returnPrefetcher, valueHolder);
    }

    v8::Local<v8::Value> Get() {
        return _prefetcher->convert(_value);
    }
};

Update 2016-12-14:
The automatic async has been partially implemented, at this moment, the tests are working as far as I can see. what happens is that the overload resolution is checking the last argument passed to the function, if its not part of the function parameters and its type is a function, its assumed to be an async request. so the overload resolution engine stores all the function arguments in memory as C++ objects, calls the function and post process the return value and lastly calls the callback function supplied.

for example, assuming you have a function test(arg1 : string, arg2 : number), if a match is found that contains test("a",1), it will execute the function synchronously. But if a match is found as test("a",1,function(err,val){}), it will be executed asynchronously (note that there is no way to know which parameters are defined for function(err,val), so the example above is for clarity sake.

The functions implementation have to use info.at<T>(index) to read the parameters and have to use info.SetReturnValue<T>(value) to return a value, otherwise the implementation will fail because node js keeps the vm locked and there's no access to v8 objects through normal info[index] mechanism, if the implementer insists on using info[index], an access violation will occur and the application will crash when executing as async.

TODO:
- Finish the callback implementation.
- Implement Async call detection and execution - partial.
- This() access from inside a class member function
- cleanup

Read More

Tags: async, node js, node-overload-resolution, templates, v8

Dror Gluska December 13, 2016  No comment

Async in Node.js Addons with NAN

Async in node js is implemented as threadpool jobs. The way node js works is that V8 is executed in the main thread, when V8 needs to call a c++ function it can either execute it in the main thread or it can schedule it in the threadpool.

Node js uses libuv asynchronous I/O library. by default it uses 4 threads in the threadpool, but it can be changed with the UV_THREADPOOL_SIZE environment variable. (ref)

The way NAN async works is, first you define a c++ class (which inherits from AsyncWorker or that accepts a callback and c++ primitive data types as parameters. then you define the Execute function which does the actual work and lastly you define the HandleOKCallback function which will do the actual callback.

The first step is to create a Nan::Callback instance from the v8::Function parameter, which is a standard Javascript error-first callback, which later can be converted to Promises.

// Asynchronous access to the `Estimate()` function
NAN_METHOD(CalculateAsync) {
  int points = To<int>(info[0]).FromJust();
  Callback *callback = new Callback(info[1].As<Function>());

  AsyncQueueWorker(new PiWorker(callback, points));
}

If you need to keep data between the call and the callback you can use SaveToPersistent and GetFromPersistent, this data is not available inside the async execution but its only used between the Callback instantiation and the javascript callback after execution.

From v8 Embedder's Guide:

Persistent handles provide a reference to a heap-allocated JavaScript Object, just like a local handle. There are two flavors, which differ in the lifetime management of the reference they handle. Use a persistent handle when you need to keep a reference to an object for more than one function call, or when handle lifetimes do not correspond to C++ scopes. Google Chrome, for example, uses persistent handles to refer to Document Object Model (DOM) nodes. A persistent handle can be made weak, using PersistentBase::SetWeak, to trigger a callback from the garbage collector when the only references to an object are from weak persistent handles.

If you need to use data for the execution you must use native c/c++ types and you must do it while creating the class instance, either in the constructor or before calling AsyncQueueWorker (more on this later).

In our example, PiWorker needs the number of points to calculate an estimation of PI, since we don't have access to any of the V8 values that we got in the function arguments (e.g. info[0]), we'll need to convert the value to C type and then use it in the Async Worker Execute.

Now we have created an instance of PiWorker, which inherits from AsyncWorker, we passed the number of points AND the callback which was passed to our CalculateAsync function:

addon.calculateAsync(100, function(err,result){
  if (err){
    console.log("error executing pi calculations", err);
    return;
  }

  console.log("result", result);
});

We can schedule the PiWorker on the threadpool, 

// Asynchronous access to the `Estimate()` function
NAN_METHOD(CalculateAsync) {
  int points = To<int>(info[0]).FromJust();
  Callback *callback = new Callback(info[1].As<Function>());

  AsyncQueueWorker(new PiWorker(callback, points));
}

actually, when calling AsyncQueueWorker it schedules the Execute for execution and WorkComplete  to execute (on the main thread) when the execution is done.

inline void AsyncExecute (uv_work_t* req) {
  AsyncWorker *worker = static_cast<AsyncWorker*>(req->data);
  worker->Execute();
}

inline void AsyncExecuteComplete (uv_work_t* req) {
  AsyncWorker* worker = static_cast<AsyncWorker*>(req->data);
  worker->WorkComplete();
  worker->Destroy();
}

inline void AsyncQueueWorker (AsyncWorker* worker) {
  uv_queue_work(
      uv_default_loop()
    , &worker->request
    , AsyncExecute
    , reinterpret_cast<uv_after_work_cb>(AsyncExecuteComplete)
  );
}

The Execute method will be executed in one of the threads of the threadpool and when its done, it will execute either the HandleOKCallback or the HandleErrorCallback depending if the SetErrorMessage was called or not.

virtual void WorkComplete() {
  HandleScope scope;

  if (errmsg_ == NULL)
    HandleOKCallback();
  else
    HandleErrorCallback();
  delete callback;
  callback = NULL;
}

So writing Async Callbacks is pretty simple once you understand the program flow.

Source Code:
https://github.com/nodejs/nan/tree/master/examples/async_pi_estimate

Read More

Tags: async, NAN, node js

Dror Gluska November 30, 2016  No comment