Spiking Neuronal Net (SNN) with Java

Threading and Time Measurement

In a spiking neural network (SNN), time plays a fundamental role. One example is the decay of a neuron’s membrane potential back to its resting value. Another is weight adaptation, which depends on the temporal correlation of events.

The challenge is that different machines have different performance characteristics, while time itself is independent of system load. This is because the system clock is implemented as a separate hardware component on the motherboard.

If the goal—like in my SNN—is to exploit highly asynchronous processing, then loosely coupled data structures and a clock decoupled from system load are essential. For example: A powerful machine can handle more than one million threads with Java virtuell Threads.

This naturally leads to the idea of a model clock. At first glance, one might assume that such a clock would also be affected by system load. That’s true, but it ticks uniformly for all threads. The key requirement is that all threads synchronize themselves to this model clock.

A simple clock implementation

// Simple Clock
public class Clock {
    AtomicLong tick = new AtomicLong(0L);

    // Defines the clock tick
    public void nextTick() {
        tick.incrementAndGet();
    }
}

All threads must be triggered to start their work in sync with this tick. This effectively turns the Clock class into the timing backbone of the SNN.

Using virtual threads

private final ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();

Each component initiates its activity by launching a thread and registering it with the clock. The clock collects all registered tasks. 

// Task submit- method
public void submitTask(Runnable task) {
    nextTickQueue.add(task);
}

When the next tick \(nextTick()\) is invoked, all queued tasks are started. The queue is processed and cleared, and then the system waits for all threads to finish. Only after that is the next tick allowed to occur.

Standard mechanisms such as timeouts, locks, and \(Callable\) wrappers are required to ensure thread safety.

Ensuring time passes between ticks

To avoid extremely fast tasks always seeing the same tick value, the clock must represent not only discrete ticks but also the time elapsed within a tick.

A more complete version might look like this:

// Clock submitting tasks
class Clock {
    ...
    public long now() {
        return tickcount.get() * 1_000_000_000
             + (System.nanoTime() - tickStartNano.get()) / 1_000;
    }

    public void nextTick() {
        tickcount.incrementAndGet();
        tickStartNano.set(System.nanoTime());
        ...
        while ((task = nextTickQueue.poll()) != null) {
            tasksToRun.add(Executors.callable(task, null));
        }
        executor.invokeAll(tasksToRun);
        ...
    }
    ...
}

Challenges in the asynchronous simulation of SNNs

In an asynchronous SNN, hotspots form in parts of the network that, because of their structure or inputs, generate extremely high activity. These regions emit many events (spikes), which then trigger numerous threads or tasks.

This leads to three effects:<>

To avoid these situations, one must not start all registered threads at once. One solution is to start only a specific (fixed) number of threads, which are also selected according to a certain strategy.

One solution would be to segment threads by neuron regions or according to other criteria — for example, waiting time or threshold time or belonging to a tick.

For this, the Runnable objects must provide additional information.

Example:

// SSN- runnable task
public class SNNTask implements Runnable {
    ...
    public int getPriority() { return priority; }
    public int getRegionId() { return regionId; }
    public long getStartTime() { return startTime; }
    // more or other...
}

The 'Proper Time' of SNN

From a more general perspective, one can view the model time with its own ticking frequency as the system’s intrinsic time. The system, with its many threads, generates a certain amount of entropy that effectively defines how this internal clock ticks. Various factors, such as CPU speed or the number of neurons in the system, determine how the clockwork behaves.

A faster CPU, compared to a slower one, can make time pass more quickly in the model, assuming the number of neurons and their connections remain constant. This means, for example, that the membrane potential decays more rapidly when measured against the real-time system clock. However, when measured in model time, the same amount of time has passed in each system.

This becomes particularly important when the number of neurons increases during neurogenesis. Without correcting for the increased workload, the system would effectively “travel into the past.”

Example calculation:

With 500 ticks per 1,000 neurons, the relative time factor is 0.5 (= 500/1000).
If the number of neurons increases by 1,000, the factor changes to 0.25 (= 500/2000). This creates a kind of “twin paradox” within the neuronal system.

Code example:

// Update clockwork on neurogenesis
public void updateNeuronCount(int newCount) {
    if (newCount == this.neuronCount || newCount < 1) {
        return;
    }

    long oldWorkPerCycle = this.neuronCount * SNNContext.TRANSITIONS_PER_NEURON;
    double currentProgress = (double) this.entropyCounter / oldWorkPerCycle;

    this.neuronCount = newCount;
    long newWorkPerCycle = this.neuronCount * SNNContext.TRANSITIONS_PER_NEURON;

    this.entropyCounter = (long) (currentProgress * newWorkPerCycle);
}
« Project Spiking Neuronal Net (SNN) with Java «