Server-Side, Embeddable Numerical Analysis Software

Server-Side, Embeddable Numerical Analysis Software

Post by Erik van Z » Wed, 13 Dec 2006 22:25:24


Folks,

I know this is a scilab-centered group, but here goes anyway:

I work for a company that processes high-volume realtime market data. To
extend our services I want to be able to run realtime calculations on
this data to compute derived data.

To this end I'm looking for a programmable, extensible expression
evaluation engine that compares to what analysists currently use on
their desktop (e.g. Matlab, Stata). I'm not a statistician and I'm
hoping some of you could guide me in the right direction based on your
experiences.

Our environment differs from desktop usage in that we need to be able to
evaluate many expressions in parallel in response to real-time quote
updates. We could have thousands of custom expressions of which we need
to evaluate hundreds per second. Expressions would typically be "small".

Performance is a key element as expressions are used to enrich real-time
data. To satify these requirements I'm looking for a package with the
following features:

- High-Level Programming/Scripting Language
- JIT/Pre-Compilation (compile a script once, evaluate many times)
- Library/API Based (no integrated GUI/CLI)
- Embeddable In-Process in a server-side component (to avoid IPC)
- C/C++ Runtime Engine
- Ability evaluate parallel expressions (no shared global variables)
- Extensive Toolbox (statistics, economics, etc)
- Extensible (via custom C/C++/Java function implementations)
- Lively, Supporting Community

I've studied R, S-PLUS, Ox, Octave, GRETL and several somewhat more
obscure packages but I find that most packages are excellent on the
desktop where parsing overhead, parallel evaluation, and an API are less
important.

The most interesting package IMHO is R (also because it's free), but it
seems to lack isolated parallel evaluation support (requires multiple
engine instances, which leads to expensive IPC).

I hope somebody recognizes my requirements and can help me in isolating
those packages that suit the (embedded, high-performance, parallel)
environment best.

kind regards,
Erik van Zijst
--
QOTD:
I love your outfit, does it come in your size?