Node js C++ Addon Overload Resolution - Update

Since the last update on December 13th, multiple enhancements have been made, So far I'm happy!

Major changes and improvements:

• The major performance issue in the tests have been found, when many overloads are added, the engine goes over all of them to find the most suitable one, this takes time and CPU. The first part that needs optimization is probably the determineType function.
• make_param added overloads and template type specialization for multiple types, which makes the overload creation simpler and more robust.
• Automatic async function implementation is now simple and straight forward, as long as only c++ typed templated functions are used for This<T> and at<T> (replaces info[]) and SetReturnValue<T>, the function should be able to run async, just note that async functions are executed in libuv's threadpool, which means you'll have to handle locks manually or use thread safe function calls. With locks comes great responsibility, node js libuv threadpool default size is 4 threads, too many waiting threads will either kill or slow down node js significantly.
• Accessing This, info[] and GetReturnValue will now throw an exception if attempted execution in async, no more crashes if these calls were made by mistake, just a normal v8 exception.
• Partial logging for what happens in the engine, TRACE for function calls, DEBUG for diagnostics, WARN for something that's not as designed and ERROR for usage mistakes.

Read More

Tags:

Dror Gluska December 24, 2016  2 comments

Node js Addon Logger Module

One of the issues I'm attempting to solve while programming node js addons is logging. I've looked for solutions for some time but there's no way to communicate to v8 the addon's internal logs. so I'm left with a few options which are not completely satisfactory. 

Dump the logs to console
I can dump the logs to the console but how do I redirect them to a file? if I redirect the entire stderr or stdout, I'll have irrelevant log entries in my log and trying to split them by module name could be useless work I'll need to do later.

Dump the logs to file
Dumping the logs to a log file is probably the easiest solution, I could use something like log4cpp or Pantheios, but importing them into a node addon could add a whole other set of problems, like how to set the appender at runtime and how do I manage modules, but Pantheios is claimed to be one of the fastest frameworks by their own benchmarks.

Eventually I've decided to try to POC my own with node js domain in mind, its not the best, but its good enough (tm), at least for now.

The basic requirements are:
- must support multithreaded logging
- must be easy on the v8 engine, e.g don't freeze the VM.
- must be able to filter out unneeded log levels, so debug/trace in debug sessons and info and above for production.
- optional - communicate with v8 the log messages

So I went ahead and planned it, its not too complicated, we have only two components at the moment, a queue for holding the debug messages between flushes and a notifier to v8 that there are pending log messages to be processed.

At the moment the queue is a quick write of locking std::queue implemented with std::lock_guard<std::mutex> with an atomic counter to keep track of the number of messages in the queue.

the notifier is implemented as libuv uv_async_send api, which basically schedules an async to be executed on the main node js thread.

on top of it I've added the following APIs exposed to node:

RegisterLogger - to register log callbacks, this way node js can use log4js and do everything in one place, not very efficient, but again, good enough for now.

Flush - which forces the callback to execute and flush any pending log messages
log_level property - to control what to log, so if we set this on Error but a Debug message comes in, the logger will ignore it.

batch_length property - how many log messages to process in one main thread event loop iteration, reducing it will make the log slower but the vm will be more responsive, increasing it may cause the vm to freeze when many log messages are being returned back to v8.

and of curse Log - which can be used for testing purposes or anything else, just don't use it for actual logging as the round trip from v8 -> c++ -> v8 is a waste of perfectly good cpu and memory resources.

Issues:
- Performance, which could be improved by using a better lock-free queue or perhaps even using a queue per thread as node js is essentially threadpooled. perhaps the callback can accept an array of log messages saving the round trip time between v8 and c++.

- Separation of logs, currently all log messages go in the same queue, so when your callback is getting called, its getting all the messages. 

- One callback only, no support for multiple callbacks

- Queue is only increasing in size, its not a memory leak, but if you don't flush your callback every once in a while, it might look like one. std::queue does not implement shrink_to_fit, so the only way to flush it is to create a new one and swap them.

- Log messages might get lost when the vm crashes, since all logging is done in memory, if the vm crashes without the logs being purged, the messages will be lost to the gods of memory dumps.

This module is a start, I'll probably improve it as the needs grow/change.

You can find the project here:
https://github.com/drorgl/node-addon-tracer

What's next? Perhaps an instrumentation module... :-)

Read More

Tags: addon, logging, node js

Dror Gluska December 19, 2016  1 comment

Node js C++ Addon Overload Resolution - Refactor & Add C++ Type Converter

Remember node-overload-resolution ?

I've been working on it for a while to see how I can support automatic async support.

I always believed if you can work a little harder to save a lot of time later, its most likely going to pay off, one of the biggest motivation to write the node-overload-resolution project was to easily convert C++ libraries to Node js without changing the API too much.

One of the major roadblocks in the last version of node-overload-resolution to this goal is that v8 objects are not accessible through any other thread other than node js main thread since node doesn't release the locker on v8 vm. To solve this issue I thought and implemented a way to parse and store the v8 objects as their projected C++ objects. in essence copying v8::String to std::string, v8::Number to double and so on.

Some issues I've encountered with this approach is how to store v8 Objects, C++ Class wrappers and Arrays, but if I can actually tell the resolution module which C++ types each v8 type translates to, maybe it could work.

I've been working on this premise and implemented a converter and value holder on top of the working overload resolution module and currently it supports Number, Function, String, Buffer, a typed struct and ObjectWrap.

Currently there are issues with Function, its implemented as or::Callback, but the actual function is not called yet.

Other issues is that I couldn't find a translation for Promise, Proxy and RegExp.

I think this module is on the correct path at the moment, it should be relatively easy to support async function calls by adding a check if the matched function have an additional function callback parameter, caching the parameter value as C++ type, pushing the function call to libuv threadpool and executing the return value conversion and callbacks activation when the thread finishes execution.

You can find the code on the native_types branch.

The code is concentrated in 3 main components, the value_holder, the value_converter and the generic_value_holder. 

The value_holder is a derived template, storing the Value inside a template type.

template<typename T>
class value_holder : public value_holder_base {
public:
    T Value;
    value_holder() : value_holder_base() {}
    value_holder(T val) : value_holder_base(), Value(val) {}
    virtual ~value_holder() {}
};

The value_converter is a derived template with template specialization for each major type handled, the specialization is both for primitives (including v8 basic types) and for derived classes for IStructuredObject and ObjectWrap classes, allowing a more specific behavior for structures and C++ classes, such as parsing/creating new v8 objects as well as ObjectWrap::Wrap and ObjectWrap::Unwrap.

for example, this template specialization is for all derived classes of ObjectWrap, it wraps/unwraps to/from v8::Object:

template<typename T>
class value_converter<T*, typename std::enable_if<std::is_base_of<ObjectWrap, T>::value>::type> : publicvalue_converter_base {
public:

    virtual T* convert(v8::Local<v8::Value> from) {
        return or::ObjectWrap::Unwrap<T>(from.As<v8::Object>());
    }


    virtual v8::Local<v8::Value> convert(T* from) {
        return from->Wrap();
    }

    virtual v8::Local<v8::Value> convert(std::shared_ptr<value_holder_base> from) {
        auto from_value = std::dynamic_pointer_cast<value_holder<T*>>(from);
        return from_value->Value->Wrap();
    }

    virtual std::shared_ptr<value_holder_base> read(v8::Local<v8::Value> val) {
        auto parsed_value = std::make_shared<value_holder<T*>>();
        parsed_value->Value = convert(val);
        return parsed_value;
    }

};

lastly the generic_value_holder stores a pair of value_holder and value_converter and by that it can act as some sort of non-convertible variant that can return a v8 object from intended C++ types.

class generic_value_holder {
private:
    std::shared_ptr< or ::value_converter_base> _prefetcher;
    std::shared_ptr< or ::value_holder_base> _value;
public:
    void Set(std::shared_ptr< or ::value_converter_base> value_converter, std::shared_ptr< or ::value_holder_base> value) {
        _prefetcher = value_converter;
        _value = value;
    }

    template<typename T>
    void Set(T returnValue) {
        //store value_converter type
        auto returnPrefetcher = std::make_shared < or ::value_converter<T>>();

        //store value inside a valueholder
        auto valueHolder = std::make_shared < or ::value_holder<T>>();
        valueHolder->Value = returnValue;

        Set(returnPrefetcher, valueHolder);
    }

    v8::Local<v8::Value> Get() {
        return _prefetcher->convert(_value);
    }
};

Update 2016-12-14:
The automatic async has been partially implemented, at this moment, the tests are working as far as I can see. what happens is that the overload resolution is checking the last argument passed to the function, if its not part of the function parameters and its type is a function, its assumed to be an async request. so the overload resolution engine stores all the function arguments in memory as C++ objects, calls the function and post process the return value and lastly calls the callback function supplied.

for example, assuming you have a function test(arg1 : string, arg2 : number), if a match is found that contains test("a",1), it will execute the function synchronously. But if a match is found as test("a",1,function(err,val){}), it will be executed asynchronously (note that there is no way to know which parameters are defined for function(err,val), so the example above is for clarity sake.

The functions implementation have to use info.at<T>(index) to read the parameters and have to use info.SetReturnValue<T>(value) to return a value, otherwise the implementation will fail because node js keeps the vm locked and there's no access to v8 objects through normal info[index] mechanism, if the implementer insists on using info[index], an access violation will occur and the application will crash when executing as async.

TODO:
- Finish the callback implementation.
- Implement Async call detection and execution - partial.
- This() access from inside a class member function
- cleanup

Read More

Tags: async, node js, node-overload-resolution, templates, v8

Dror Gluska December 13, 2016  No comment