XXIIVV — neural nets

In neural nets, synthetic neurons are the base computational unit.

This is model of the neuron that doesn't capture all of its properties and behavior but just enough to capture the way it performs computation.

A McCulloch-Pitts neuron uses arity saturation, excitatory and inhibitory fibers.

A neuron has a given number of incoming and outgoing connections to other neurons, it gives out an output that stimulates or inhibits other neurons when the sum of its inputs goes over a threshold arity value. A neuron with an arity of zero will always fire.

A neuron connected by two excitatory fibers, one inhibitory fiber, with an arity of two.

A Notation

Typically, neural networks are programmed with flow diagrams, I found those to be prone to clutter. Instead, this documentation will use a rule-based textual representation, which allows networks to be created more rapidly and precisely.

A neuron*. neuron/0

In a rule, a neuron is specified with a star suffix, an excitatory connection is separated by a colon, an inhibitory connection is separated by a semi-colon, a rule is terminated by a comma, rules can be chained, everything else is ignored.

I have matches*, and some kindling*:
	I can make a fire*: fire/2
	which in turn emits light*. light/1

But if it's raining*;
	I cannot make a fire*.

I have matches* and kindling*. matches/0, kindling/0

Programming with Neural Nets

In the example above, for each step of the evaluation, the matches/0 and kindling/0 will send excitatory signals to the fire/2 neuron. But, let's say we want a neuron to fire once and turn off, we can connect it back onto itself with an inhibitory fiber:

hello-world*; hello-world*.

Print hello-world*, and turn itself off.

A program will most likely involve starting a program with an initial signal/1. To do so, we can fork the signal from the self-deactivating init/1 neuron toward the rest of our program:

Network	Evaluation
init; init: signal. The signal will fire once: and activate the program. init my program.	00 init/1 01 signal/1 02 program/1

Loops

Loops can be created by connecting neurons in a circle. In the following program, two such circles are created, in one, 3 neurons are connected in a circle, and whenever the f2/1 neuron fires, a neuron fizz/1 is awakened. In the second loop of 5 neurons, whenever the b4/1 neuron fires, a neuron buzz/1 is awakened.

Network	Evaluation
init; init: f0* b0. The fizz network. f0: f1: f2: f0* fizz. The buzz network. b0: b1: b2: b3: b4: b0* buzz. init my fizzbuzz program.	08 f2/1 b3/1 09 f0/1 b4/1 fizz/1 10 f1/1 b0/1 buzz/1 11 f2/1 b1/1 12 f0/1 b2/1 fizz/1 13 f1/1 b3/1 14 f2/1 b4/1 15 f0/1 b0/1 fizz/1 buzz/1 16 f1/1 b1/1 17 f2/1 b2/1 18 f0/1 b3/1 fizz/1 19 f1/1 b4/1 ..

Network

Evaluation

init*; init*: f0* b0*.

The fizz network.
f0*: f1*: f2*: f0* fizz*.

The buzz network.
b0*: b1*: b2*: b3*: b4*: b0* buzz*.

init* my fizzbuzz program.

08 f2/1 b3/1 
09 f0/1 b4/1 fizz/1
10 f1/1 b0/1 buzz/1 
11 f2/1 b1/1 
12 f0/1 b2/1 fizz/1
13 f1/1 b3/1 
14 f2/1 b4/1 
15 f0/1 b0/1 fizz/1 buzz/1 
16 f1/1 b1/1 
17 f2/1 b2/1 
18 f0/1 b3/1 fizz/1
19 f1/1 b4/1 
..

Logic

Binary logic is implemented by a combination of inhibitory and excitatory fibers, the system welcomes the implementation of ternary logic gates as well. Here are the implementation of a few logic gates:

a*; a*.
b*; b*.

a*   ; NOT/0*.
a* b*: AND*.
a* b*: OR/1*.
a* b*; NOR/0*.

Activate both a* & b*.

Memory

A neuron/1 can store a single bit of information by using an excitatory feedback fiber connected onto itself, and a excitatory start fiber and inhibitory stop fiber. Delaying a signal is the same as remembering it for a length of time, making neurons the ideal memory storage.

bit*: bit*.

controls:
- start*: bit.
- stop*; bit.

Connecting these memory neurons in a sequence, we can create a capacitor that will accumulate pulses until it reaches its storage limit, output a signal, and start over.

init*; init*: t0*.

Timer.
t0*: t1*: t2*: t4*: t0*.

Fork every fourth t0*:
	to input*.

Connect input*:
	to m0/1* m1/2* m2/2* m3/2*.

Connect neurons on themselves and reset.
	m0*:m0* m1*.
	m1*:m1* m2*.
	m2*:m2* m3*.
	m3*:m3* reset* bang*.

reset* will turn off all counting neurons;
	m0* m1* m2* m3*.

Let's init* our program.

Arithmetic

TODO.
Come back soon.

Implementation

An implementation of the full symbolic runtime is about 150 lines of C89:

cc neur.c -o neur view raw

By virtue of the similarities this system shares with other rewriting systems, by being programmed by specifying rules and an initial state, resembles a rewriting language enough that I thought it should classified as such.