How to implement cell-automata/neural-net stuff ?

How to implement cell-automata/neural-net stuff ?

Post by news » Wed, 09 May 2007 21:48:40


We are trying to prototype some ideas to simulate the COSA system
as described at:
http://www.yqcomputer.com/

I'm guessing that knowledge of how neural-networks are simulated
would help ?

On the one hand they seem to be screaming for an OOP
implementation, where each of the duplicated neuron is an object ?

OTOH using OOP hides the communication between neurons, which I'm
not comfortable with, and having arrays, which are scanned once per
cycle, [all changes occur in clocked cycles] seems natural.

The basic method[s] used by spreadsheets seem appropriate ?

Here's a description of some of the 'objects':

Although lacking OOP experience, this sounds like:
- Sensors & effectors are descendants of cell-objects
- a class attribute of cell-objects is 'synapse'
with value=DestinationCell.

Personally I'd prefer to have tables/arrays where the
stepping is updated explicetly by accessing [running through]
the appropriate arrays.

Thanks for any ideas or advice,

== Chris Glur.
 
 
 

How to implement cell-automata/neural-net stuff ?

Post by bob the bu » Thu, 10 May 2007 00:16:17


Hmm, i made a neural network in excel (long time ago), it didnt make
me happy. If you want to build a network from scratch then c++ or java
would be a better choice. And maybe you can program a more traditional
ANN first, to get the hang of it. If you decide to use c++ then you
could use Dev-c++ for example (http://www. *** shed.net/dev/
devcpp.html) (its free).

 
 
 

How to implement cell-automata/neural-net stuff ?

Post by Steve » Thu, 10 May 2007 05:51:31

n May 8, 5:48 am, XXXX@XXXXX.COM wrote:

I read a few of the articles at the site and found them interesting
but beyond my ability to judge the merits of the arguments.
It is most interesting to see someone undertaking such an
ambitious and significantly different approach.


Seems like it would. That does seem to be the underlying model
they are basing their approach on. However, as you will
discover as you browse here and the web about NN's, there are
many different specialized types of NN's. Some (e.g. backprop) are
specialized
to perform a specific task accurately; these would seem the
least appropriate for the kind of thing they're doing.
Unsupervised recurrent NN's have feedback loops, are more along
the lines of a "free-running system", and there are
modifications that try to simulate parallel processing by
selecting randomly which cell(s) will do their calculations, rather
than running through an array in sequence.

See this group's FAQ document for a very good overview
of ANN's and introduction to the field, an
introduction to the terminology, and links. It's a good place to get
started.


For a large project (and even a small one), I believe OOP is
the only way to go. It helps keep you organized, keeps your
code clean, and helps makes your code understandable and maintainable.
There's now a variety of OOP languages, but most are offshoots of C++.
If you learn any of them, you'll know the fundamental concepts of all
of them. There are free compilers available for most of them.
What is not consistent and portable among the languages are
the libraries (Microsoft MFC or .NET, Borland VCL, Java whatever,
etc.),
so the operating system you want to use and the variety of functions
(e.g. graphics) you need will be major considerations in the choice.
There's a pretty steep learning curve for the libraries, much harder
than for the languages themselves, so choose with care.

After looking at various ones, I still prefer C++.
MS Visual Studio C++ 2005 Express is available as a free download.
I've been using it for a week or so, and like it. It will probably
become my new primary development language/IDE.

If it is of any interest, there is a free open source C++ neural
network program on my website at
http://25yearsofprogramming.com/ai/neuralnet/index.htm
The network classes are on the related page
http://25yearsofprogramming.com/ai/neuralnet/networkcpp.htm

I'm currently working on the (now functioning)
version for STL and MS Visual Studio C++ 2005 (.NET), and hope to have
it uploaded shortly.

What I thought might be of interest to you is the classes
(Node, Synapse, Network). The full program creates a backprop network,
definitely not what you'd want, but the classes should be easy to
press
into service for any type of network. The code is
heavily commented and should be easy to modify.

If those don't suit your purpose, you might browse around
http://www.sourceforge.net. However, most of the ANN programs I looked
at there are either quite specialized or have source code that is
pretty difficult to figure out by inspection.


Not sure what you mean by that.


I doubt this. For such an ambitious venture, I suspect a spreadsheet
approach will quickly turn out to be more limiting than is tolerable.

 
 
 

How to implement cell-automata/neural-net stuff ?

Post by Kent Paul » Thu, 10 May 2007 10:02:01


If you'd _read_ the site, you'd see how counterproductive this idea
is. What make COSA "work" is precisely that it _is_ the operating
system, thoroughly debugged, and protected from user code twinking,
even at the applications level. _Simulating_ it opens it to twinking
and
to introduction of new bugs. If you want COSA, pick a machine to
dedicate to the enterprise, and run COSA on it. Otherwise, forego the
main benefit COSA offers. If you do that, why on earth bother with
COSA?

IMO

xanthian.
 
 
 

How to implement cell-automata/neural-net stuff ?

Post by news » Fri, 11 May 2007 00:18:24

n May 8, 5:48 am, XXXX@XXXXX.COM wrote:

SteveW wrote:

The way 'objects communicate' [call methods] is hidden by the
language implementation. But I suspect I'm thinking at the wrong
level of granularity ?


Well, it's so far removed from my experience that even initial
modelling with match-sticks could advance my understanding.

Thanks, I'm looking at your references.
xanthian wrote:

No ! COSA is only a concept. I'm in contact with the 'founder'
who's looking for collaborators to implement it. You can join too.

Thanks,

== Chris Glur.

 
 
 

How to implement cell-automata/neural-net stuff ?

Post by Steve » Fri, 11 May 2007 05:44:25

> > > OTOH using OOP hides the communication between neurons,

Yes, I think so, if I understand your question correctly now.
The "communication" between neurons just consists of
numbers (output levels and connection weights) stored
in variables and retrieved for calculations when they're
needed. So, for example, when a neuron needs to
calculate its inputs, it just looks at the other
neurons that input to it, reads their
outputs, and sums them.

Thus, the variables are available for inspection whenever needed.
For example, in my program it would be possible to
add a feature where positioning the mouse over a node
would cause a popup report of the node's current
inputs, their connection weights, and activation state.
I doubt I'll get around to implementing that,
but it's do-able.
 
 
 

How to implement cell-automata/neural-net stuff ?

Post by John Ladas » Sat, 12 May 2007 04:45:31


I wrote my own neural net implementation about a year ago, in Python.
It's slow, but it works, and I can see clearly into its inner
workings.

The reason that I chose to write my own is that I was studying neural
nets on paper, and discovered that many useful functions (for me)
needed four or even five layers of neurons to implement them. At the
same time, that I wanted to implement these long, tricky strings, I
didn't want to waste layers of neurons on functions which do not
require so many layers. Thus I implemented a "layerless" neural net.
Each node receives a number. The highest-numbered nodes are an output
layer of sorts -- but every node in the net, including the output
nodes, is capable of receiving imput from any lower-numbered node. In
the low-numbered nodes, it's essentially a free-for-all.

It would have been a huge waste of computing resources (and
programming time) for me to model each neuron as its own object. I
can think of a few reasons why you might want each neuron to be an
object, but none of them applied to me:

1) You have multiple types of neurons, with different topologies,
weighting, or summation functions. All of my neurons are homogeneous,
and nearly every neural net I've ever read about is the same way.

2) You need neurons to possess a persistent internal state -- for
memory, perhaps. In my system, if memory is required, it will arise
in feedback loops.

3) Your search/optimization algorithm involves a lot of neuron
transposition -- i.e., you're changing the topology of the network.
Although I can imagine that shuffing neurons around might be useful in
some circumstances, I've never seen it done.


+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-+
| Ladasky Home Solar, Inc.: blowing sunshine up your |
| power grid since March 24, 2005. Fiat lux! |
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-+
| Uptime Downtime kWh generated kWh consumed |
| 772.5 days 13 hours 13485 14875 |
+-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-+
 
 
 

How to implement cell-automata/neural-net stuff ?

Post by Tim Tyle » Sat, 12 May 2007 18:29:14


>> neuron is an object ?


[snip]

Encapsulation and type safety would appear on many such lists.
--
__________
|im |yler http://www.yqcomputer.com/ @XXXXX.COM Remove lock to reply.
 
 
 

How to implement cell-automata/neural-net stuff ?

Post by Josip Alma » Fri, 18 May 2007 21:50:36


...

Not really; you got detailed answer already.

OTOH I did implement 'neurons' with objects, in java, Observer, and
wrote a fuzzy essay to explain it to my OOP coleques,
http://www.yqcomputer.com/

In my tests, I had 100000 'neurons' per node with response time ~100 ms.
With soft MRU cache (*) I got about 2M 'neurons' per node; 1000
'neurons' in memory, but memory requirements grew to 2G for references.
Response time remained about the same due to good cache hit rate.
So it works for me, but in general case, hardly.
It *could* also work for you, say your + 'neurons' don't have many
'synapses'...

But note these quotes I never forget to put around 'neurons'... it's not
ANN, it's *inspired* by NNs. Very basic difference is that my 'neurons'
are carefully crafted state machines with much more complex behaviour
and much less connections compared to (most of) ANNs, not to mention NNs.

Also note that due to that synchronous requirement it's not easy to use
either Observers or Listeners. It's possible, see that Checkpoint trick
in the essay.

HTH, questions, drop me a mail, I'll be glad to help.

WRT COSA, I have to say I'm ambivalent. I do tend to agree that
event-driven programming tends to produce less errors than say state
machines; OTOH there's lot of buggy electronics too. Especially in PC
world, like, I plug in woodoo and my scsi controler hangs. The way I see
it, it's due to open interfaces. And in both hardware and software,
protocols specify communication but not peer states, and you simply
can't avoid states.
OK to make long story short - proof of concept is required, either to
prove or disaprove.
Therefore - good luck guys.

Regards...

(*) like this:
http://www.yqcomputer.com/
 
 
 

How to implement cell-automata/neural-net stuff ?

Post by Vend » Tue, 22 May 2007 05:17:26


The question is: why you would?


Hint: you can put objects into arrays (or lists).


Don't use OOP if you aren't familiar with it.
 
 
 

How to implement cell-automata/neural-net stuff ?

Post by A.L. » Tue, 22 May 2007 05:35:35


It looks like reinventing the wheel: it is very similar to Hoare's CSP
(Communicating Sequential Processes) where concurrently running
processes ("cells") comunicate through synchronous channels. Hoare's
book is available free on Internet. This concept was implemented in
the language Occam that was "high level assembly language" of platform
known as Transputer (now extinct). CSP has been implemented in Java
and implementation is known as JCSP. There are two implementations;
the best known is from University of Kent

http://www.yqcomputer.com/

A.L.
 
 
 

How to implement cell-automata/neural-net stuff ?

Post by Travele » Tue, 22 May 2007 10:21:06

n Sun, 20 May 2007 15:35:35 -0500, A.L. < XXXX@XXXXX.COM > wrote:


Well, the concept of synchronously communicating concurrent cells is
nothing new and I doubt that Hoare would lay claim to having inventing
the idea. Cells in a neural network do essentially the same thing.
However, I don't think that Hoare's primary goal was to build a
software environment for the construction of bug-free programs. It
seems that he was more concerned with creating a mathematical
formalism for parallel computation. Indeed, Occam was developed for
INMOS's Transputer, a computer that was ahead of its time.

The main problem with Occam (I am biased) is that it is a computer
language. In my opinion, the linguistic approach is ill-suited for
depicting communicating concurrent entities. The graphical approach is
much better if only because it presents the user with a easily
understood visual depiction of the communicating pathways between
objects. The other problem with Occam is that Hoare apparently did not
fully appreciate the advantage of using synchronous reactive entities
as a means to solve the software reliability crisis.

Although COSA is inherently concurrent and is thus well-suited to
parallel processing, parallelism is not its main strength. My primary
goal in designing COSA was to solve a vexing problem within the
software industry: unreliability. Unreliability imposes an upper limit
on the complexity of software systems. As an example, we could
conceivably be riding in self-driving vehicles right now (which would
prevent tens of thousands of accident-related deaths every year) but
concerns over safety and reliability will not allow it. In addition,
the cost of developing safety-critical software rises exponentially
with the level of complexity. The reason is that complex software is
much harder to test and debug.

It occurred to me that the only way to solve the problem was to devise
a software execution model and development environment that
facilitated and enforced deterministic control over the timing of all
events and operations: nothing is allowed to happen before or after
its determined time. The advantage of this approach is that it makes
it possible to build a software "sentinel" that automates the job of
discovering and enforcing the laws (constraints) that govern the
temporal behavior of a program. Additions and modifications to the
program are not allowed to break exisitng protocols thereby insuring
solid consistency throuhout the development cycle. The beauty of this
approach is that the number of constraints increases with complexity.
This enables the creation of bug-free software of arbitrary
complexity. Counterintuitively enough to a programmer, complexity
ceases to be the enemy of software reliability and becomes a welcome
and trusted friend.

There are other important advantages such as the automatic resolution
of data dependencies which go a long way toward creating bug-free
software. The idea that programs can be described exclusively with
actors (sensors and effectors) and their environment (data) simplifies
the development process and makes programs more comprehensible. It can
all be found at the URL below. It's nice to see Project COSA being
discussed on usenet.

Louis Savain

Why Software Is Bad and What We Can Do to Fix It:
http://www.rebelscience.org/Cosas/Reliability.htm
 
 
 

How to implement cell-automata/neural-net stuff ?

Post by A.L. » Tue, 22 May 2007 11:12:27

On Sun, 20 May 2007 21:21:06 -0400, Traveler < XXXX@XXXXX.COM >




Generally, I don't see anything in your project that would contribute
to the above claims, especially, quote: "environment that facilitated
and enforced deterministic control over the timing of all events and
operations: nothing is allowed to happen before or after its
determined time...". For me, this is still CPS-like environment where
you have all problems associated with concurrency, and that is not
free of such problems as, for example, deadlock. I don't see also any
"magic" that would allow, quote: "enables the creation of bug-free
software of arbitrary complexity..". You must be much more specific to
convince the public.

A.L.

P.S. Maybe you should check the following book: "Flow-Based
Programming: A New Approach to Application Development" by J. Paul
Morrison, Van Nostrand, 1994
 
 
 

How to implement cell-automata/neural-net stuff ?

Post by Vend » Tue, 22 May 2007 19:26:08


I don't know much about CPS, but it seems to me that this COSA is a
kind of concurrent imperative language. Since everything is done
concurrently, it seems that the programmer has to explicitely take
care of timing constraints. This is a great complication, in
particular it severely impair the ability to make modular programs,
since the timing of a module, which is implementation dependent, must
be leaked to the module user.
 
 
 

How to implement cell-automata/neural-net stuff ?

Post by Travele » Tue, 22 May 2007 22:15:58


I once had a brief exchange on usenet with either Hoare or someone
intimately familiar with Occam and he agreed with me that parallel
synchronous systems are inherently more reliable than algorithmic
systems. What Hoare and others fail to grasp is that the secret of
reliability can be summed up with one word, timing. I'll rephrase what
I wrote previously to make it even simpler to understand. Once you
gain control over the timing of events (this is automatic and
transparent in a synchronous system), then it is easy to create a
mechanism that automates the discovery and enforcement of the temporal
expectations. A temp *** expectation is a simple constraint such as
event A always follows event B or event A and B are always
simultaneous.

It is not magic, as you call it, but it is definitely a silver bullet.
Best of all, it's extremely simple. Sorry you are having such a hard
time seeing how it works. I would not know how else to explain it. It
could be because I have been thinking about it for so long, I now take
certain things for granted.


Well,I don't see why I should do that since COSA is not a flow-based
model, although it shares some similarities. COSA is a synchronous
reactive software model in the tradition of synchronous reactive
languages like Esterel, Signal, etc...

Louis Savain

Why Software Is Bad and What We Can Do to Fix It:
http://www.yqcomputer.com/