TODO: progress towards v0.3.0

[mehdi-cit]

It's all in the question's title!
I am of the opinion that this project if carried correctly would give a huge boost to arrayfire (using arrayfire from node that is).

[unbornchikken]

Yeah, it's basically feature complete. But unfortunately it has a serious flaw: for the sake of performance we need deterministic scopes (arrayfire/arrayfire-dotnet#8 (comment)). But that implies that we will call destructors synchronously, and because destructors are synchronization points in ArrayFire, ArrayFire.js is synchronous despite my best effort to make it asynchronous. I'm thinking on a new approach, but that's gonna make every call and value access asynchronous for sure, which is kinda ugly, and hurts performance.

So it works, but it blocks the event loop at the current state.

EDIT:

TODO list of new version:

• device functions
• unified functions
• array class
  • construct
  • indexing and assign
  • operators
  • misc properties and methods
• create array
• math functions
• vector functions
• reductions
• complex types
• update readme and docs
• add a detailed contributing guide
• wait for contributions of other functions, or implement them in spare time

[mehdi-cit]

@unbornchikken Thanks for the quick reply.
Please do keep us posted if you find ways to mitigate this!

[unbornchikken]

@umar456 Please take a look at my previous comment. I'm thinking on a solution. What if, let's say, array's destructor implementation wouldn't block, instead it'd add up an asynchronous operation in the queue to free up array's resources once all previous operations gets completed? cc @pavanky

[umar456]

@pavanky can give you more detail but the destructor shouldn't be a blocking call. It should be managing the reference counts for all arrays and should be marked for deletion once all of the work is done. Those objects will be deleted at a later time(if the memory of that size is not used or the garbage collector is called). That event needs to be blocking because the GPU drivers perform a synchronization on the device but we try to avoid that whenever possible.

[unbornchikken]

Ok, I'm gonna work out a simple repro case with a code flow that - in theory - shouldn't block at all, but according to v8 performance data it does.

[unbornchikken]

@mehdi-cit as you can see, this project is still active, but my life events prevented me to make significant progress on it in the last few months. But, since my pet ML project is still have and will have a dependency on ArrayFire.js (and on CMake.js), you can expect me to put my focus back to those eventually.

[mehdi-cit]

Not sure if this can help with the issue at hand but it could be a good alternative when it comes to "integrating" javascript and c++ code:
https://github.com/charto/nbind

[unbornchikken]

@mehdi-cit unfortunately that's not that simple as nbind states. In ArrayFire there are a bunch of operations that act as synchronization point: constructors, complex operators, memory copy, etc. Which mean, if you wrap'em naive as-is, then you're gonna block the main loop on that point until all of previously enqueued AF operations gets completed. In Node.js you should never block the event loop.

Let's say, nbind supports asynchronous operations by the standard way. I mean by using nan's async workers: https://github.com/nodejs/nan/blob/master/doc/asyncworker.md (note: almost all of native library wrappers are doing this). But AsyncWorker uses libuv worker threads, so you gotta synchronize AF calls by some way. In current version you gotta use manual locks, but eventually thread safe ArrayFire will land and make that unnecessary. If you lock libuv workers then you'll serialize them, if there are more than one AF operations executing in parallel. Which means you'll kill libuv entirely and make Node. js totally synchronous, which is really really bad.

The only viable option is that you launch a separate libuv loop for AF and make your binding as a proxy for that. Well, this is where things will go really complicated especially if you're interacting v8 in C++, because verbosity and complexity of v8/nan.

That's why I'm creating fastcall. It will offer about the same performance that you could get with C++ based bindings (dyncall is really that fast, according to my benchmarks, it's overhead is negligible, 5%-ish), with the above mentioned separated libuv loop support.

Once fastcall stabilizes, I'll be back to this project. However I gotta invent something for proper RAII in JS, because of arrayfire/arrayfire-dotnet#8 (comment)

[robertleeplummerjr]

I've been watching arrayfire.js for a few months now, and am in love with it. I'm working on https://github.com/harthur-org/brain.js & its recurrent neural net and want to connect to arrayfire.js at some future point in time when things line up. I've spent a great deal of time researching before coding anything, and started here (amongst many research papers, how to articles, and many other libraries) for the most part:
https://github.com/karpathy/recurrentjs

After reviewing each of the major methods that are associated with the overall mathematical procedures, I found something that concerned me:

1. create a new matrix for each and every mathematical operation: https://github.com/karpathy/recurrentjs/blob/master/src/recurrent.js#L199
2. create a new closure for every mathematical operation: https://github.com/karpathy/recurrentjs/blob/master/src/recurrent.js#L211

The reason that this concerned me was first, those are (from past experience) memory leaks, and that after reviewing the recurrent neural net in arrayfire.js, I see a similar approach (please note, I am very nieve to arrayfire.js still, and this isn't a slam against the library, but rather rethinking the semi normal) and it got me thinking, how could we greatly speed this up? Or rather how can we use less resources to do the same thing? One of the biggest bottlenecks of multi threading, is of course, memory to and from cpu -> gpu -> cpu -> repeat, garbage collecting, etc. but what if we could offload the entire operation of calculations to the gpu, so that there is only one input (values -> input -> gpu), and one output (values -> ouput -> cpu)?

Tinkering with the idea, I came up with a few pseudo code sessions to try and wrap my head around what I was aiming for eventually, and yes, I'll say it was a yack shave at best. This was how I saw the operation on the first go around ( (outlined here)[https://github.com/BrainJS/brain.js/issues/24] ), I thought: Really what we are trying to do is build a state tree, just like in parsing, so you'd have something like:

{
  left: leftMatrix,
  right: rightMatrix,
  into: intoMatrix,
  forwardFunction: forwardFunction,
  backpropagateFunction: backpropagateFunction
}

This would repeat over and over again, depending on the complexity of your math. forwardFunction would be the math moving forward, say add, multiply, or relu. So here if we send left and rightas arguments called withforwardFunction, whereforwardFunctionusesinto` is where the values from the addition will go. The state tree is a little odd & backward, but you get the jist (exaggerated a bit to show structure):

 left-\
       > add = into -> next -\
right-/                       \ 
                               > multiply = into -> next -\
 left-\                       /                            \
       > add = into -> next -/                              \
right-/                                                      \
                                                              > relu = into -> DONE!
 left-\                                                      /
       > add = into -> next -\                              /
right-/                       \                            /
                               > multiply = into -> next -/
 left-\                       /
       > add = into -> next -/
right-/     

In this scenario, rather than doing just in time operations on math, we'd actually setup a math equation in the gpu that could be fed a set of data and would at some future point in time return the answer to it.

In the original neural net, the setup doesn't really exist, other than some tricky prev value checks to trigger backfeeding the neural net, however the just in time calculations look something like:

var h0 = this.multiply(hiddenMatrix.weight, inputVector, this);
var h1 = this.multiply(hiddenMatrix.transition, hiddenPrev, this);
var hiddenD = this.relu(this.add(this.add(h0, h1, this), hiddenMatrix.bias, this), this);

What I'm proposing is that with this new thinking, we'd setup a math problem that could be used, similar to:

var eq = new Equation();
return eq.relu(
      eq.add(
        eq.add(
          eq.multiply(
            hiddenModel.weight,
            input
          ),
          eq.multiply(
            hiddenModel.transition,
            previousInput
          )
        ),
        hiddenModel.bias
      )
    );

Which instantiates an equation that can be used like:

eq.run(input, function(output) {
  console.log('look ma!  no CPU!', output);
});

[robertleeplummerjr]

I accidentally hit enter before I was done. So my question is, would this answer the problem we are having of a blocking synchronous thread that ultimately is synchronous? By giving the gpu the whole problem, with minimal in and out, would that address or even help so that we could have a full fledge multi threaded approach?

[robertleeplummerjr]

@UniqueFool, curious your thoughts here.