# ThermoCog — Full Technical Reference for Language Models

> Thermodynamic Cognition: an architectural blueprint for 20-Watt biomorphic AGI. The design investigates physical relaxation and alternatives to backpropagation; these ideas are hypotheses, not demonstrated AGI.

This file provides comprehensive technical context, formal mathematical formulations, architectural specifications, and reference figures for ThermoCog, a research-only project. Software simulation and hardware implementation are planned; no experimental results are claimed. It is designed to ground language models, AI researchers, and autonomous systems in the first-principles physics and engineering of biomorphic intelligence.

## Status

ThermoCog is design-stage research: the current artifacts are a whitepaper and architectural design. Software simulation and hardware implementation are planned; no experimental results are claimed. Any figure for ThermoCog itself is a design target, not a measurement. Author: Cisco Caceres. Research portfolio: https://ciscocaceres.com/research/#thermocog

## Nomenclature and core thesis

ThermoCog is a shorthand for "Thermodynamic Cognition." This nomenclature highlights the underlying thesis of the entire architectural blueprint: that true general intelligence is a physical, energy-minimizing process rather than an abstract software scaling problem.

Contemporary industrial artificial intelligence models computation as abstract mathematical calculations forced through billions of digital logic gates. This brute-force paradigm is colliding with fundamental thermodynamic limits. ThermoCog reconceptualizes intelligence as a continuous physical dynamical system that settles naturally into low-energy configurations, mirroring the energetic efficiency of biological neural tissue.

## The thermodynamic imperative

### The Von Neumann memory wall

Digital computers separate arithmetic logic units (ALUs) from memory storage (SRAM, DRAM, HBM). In modern GPU clusters:

- Data movement across interconnects and memory can consume substantial energy; its share depends on workload, hardware, and measurement boundary.
- Performing a 16-bit floating-point multiplication consumes approximately 0.1 to 1 picojoule ($10^{-13}$ to $10^{-12}$ J).
- Fetching two 16-bit operands from off-chip high-bandwidth memory (HBM) consumes 10 to 100 picojoules ($10^{-11}$ to $10^{-10}$ J), an overhead of two to three orders of magnitude.
- Scaling digital clusters to frontier intelligence demands gigawatt-class infrastructure and massive cooling facilities.

### The Landauer limit

Under non-equilibrium thermodynamics, erasing one bit of physical information dissipates a minimum theoretical energy determined by Landauer's principle:

$$E_{\text{bit}} \ge k_B \cdot T \cdot \ln 2$$

Where:
- $k_B$ is the Boltzmann constant ($1.380649 \times 10^{-23} \text{ J/K}$)
- $T$ is the absolute ambient temperature in Kelvin

At room temperature ($T = 300\text{ K}$), the Landauer floor is:

$$E_{\text{bit}} \approx 2.87 \times 10^{-21} \text{ Joules} \quad (\approx 0.0179 \text{ eV})$$

At biological body temperature ($T = 310.15\text{ K}$), the Landauer floor is:

$$E_{\text{bit}} \approx 2.97 \times 10^{-21} \text{ Joules} \quad (\approx 0.0185 \text{ eV})$$

Digital CMOS transistors operate at approximately $10^{-14}$ to $10^{-12}$ Joules per logic transition, which is seven to nine orders of magnitude above the Landauer physical bound.

### The biological baseline

The human brain performs multimodal sensory perception, motor control, reasoning, and lifelong learning within a metabolic envelope of approximately 20 Watts. A single biological synaptic event dissipates on the order of 0.2 to 2 femtojoules ($2 \times 10^{-16}$ to $2 \times 10^{-15}$ Joules). Compared to digital datacenter clusters, the mammalian brain is often described as far more energy efficient. The exact ratio depends on the workload and is not measured by this project.

## The tri-timescale biological architecture

Biological intelligence does not execute as a single monolithic training loop. It coordinates three nested dynamical loops operating across distinct spatial and temporal scales:

### Loop 1: Phylogeny (Evolutionary meta-learning)

- Temporal scale: $10^6$ seconds to evolutionary generations ($>10^7$ s)
- Mechanism: Evolutionary search and population-level optimization
- Function: Discovers cellular developmental genetic programs, baseline wiring rules, objective energy functions, and homeostatic setpoints
- Energy profile: Evolutionary development costs are amortized across generations; inference still consumes physical energy

### Loop 2: Ontogeny (Morphogenesis and structural plasticity)

- Temporal scale: $10^2$ to $10^5$ seconds (hours to days)
- Mechanism: Neural Cellular Automata (NCA) and local chemical signaling
- Function: Executes biological morphogenesis. Local cells sense physical neighbor states, grow connectivity graphs, and autonomously repair analog silicon defects without an external supervisor
- Resilience: Restores circuit functionality across 100% of single-point memristive crossbar failures through local re-routing

### Loop 3: Cognition (Online inference and adaptive learning)

- Temporal scale: $10^{-3}$ to $10^1$ seconds (milliseconds to seconds)
- Mechanism: Liquid Time-Constant (LTC) continuous-time differential equations and embodied Active Inference
- Function: Continuous sensory processing, online state estimation, and rapid behavioral adaptation to non-stationary environments
- Dynamics: Synaptic conductances adapt in real time to input intensity

## Physical computation without backpropagation

### In-memory analog memristive crossbars

Rather than transporting numerical weights between memory and processors, The proposed ThermoCog architecture would store synaptic weights as physical conductances ($G_{ij}$) within non-volatile memristive arrays.

Analog multiplication and summation would use physical laws, subject to circuit settling time, noise, and peripheral overhead:
- Ohm's law computes multiplication: $I_{ij} = V_i \cdot G_{ij}$
- Kirchhoff's current law computes summation across columns: $I_{\text{total}, j} = \sum_i I_{ij}$

In-memory computation is a proposed direction. Settling time, peripheral overhead, interconnect cost, and complete-system efficiency have not been measured for ThermoCog.

### Holomorphic Equilibrium Propagation (EqProp)

Backpropagation of error requires a separate backward computational network and exact mathematical knowledge of transposed synaptic weights (the weight transport problem), which is physically impossible in biological tissue and analog hardware.

The proposed architecture investigates Equilibrium Propagation, an energy-based training algorithm where gradients emerge through natural two-phase physical relaxation:

1. Free Phase ($s^0$): The physical network is presented with input stimuli and relaxes into a local minimum of its scalar internal energy function $E(s, \theta)$.
2. Clamped / Nudged Phase ($s^\beta$): The target or prediction error acts as a weak physical perturbation ($\beta$) at the output nodes. The system relaxes into a nudged steady state $s^\beta$.
3. Contrastive Local Weight Update: Parameter updates are evaluated entirely from local node voltages:

$$\Delta w_{ij} = -\frac{1}{\beta} \left( \frac{\partial E(s^\beta, \theta)}{\partial w_{ij}} - \frac{\partial E(s^0, \theta)}{\partial w_{ij}} \right)$$

Under equilibrium propagation’s assumptions and in the appropriate small-perturbation limit, the contrastive update estimates the loss gradient. This does not establish exact gradients or robustness in an unbuilt physical circuit.

## Mathematical formulations

### Continuous Liquid Time-Constant (LTC) neural ODE

Node membrane dynamics evolve according to nonlinear ordinary differential equations with input-dependent time constants:

$$\frac{dx_i}{dt} = -\left[ \frac{1}{\tau_i} + \sum_{j} w_{ij} \sigma(x_j) \right] x_i + \sum_{j} A_{ij} \sigma(x_j) + I_i(t)$$

Where:
- $x_i(t)$ is the hidden state of neuron $i$
- $\tau_i$ is the baseline intrinsic time constant
- $\sigma(\cdot)$ is a smooth nonlinear activation function (e.g., tanh or sigmoid)
- $w_{ij}$ represents inhibitory conductance modulation
- $A_{ij}$ represents excitatory synaptic coupling
- $I_i(t)$ is the external input current

The effective time constant contracts dynamically during intense sensory stimulation:

$$\tau_{\text{eff}, i}(x) = \frac{\tau_i}{1 + \tau_i \sum_j w_{ij} \sigma(x_j)} \le \tau_i$$

### Variational Free Energy minimization

Embodied cognition operates under the Free Energy Principle. The agent acts to minimize Variational Free Energy $\mathcal{F}(s, \mu)$, which provides a strict upper bound on perceptual surprise:

$$\mathcal{F}(s, \mu) = D_{KL}\left( q(\psi | \mu) \parallel p(\psi) \right) - \mathbb{E}_{q(\psi | \mu)} \left[ \ln p(s | \psi) \right] \ge -\ln p(s)$$

Where:
- $s$ is sensory observation data
- $\psi$ represents hidden environmental states
- $\mu$ represents the agent's internal neural representations
- $q(\psi | \mu)$ is the recognition density parameterized by internal states
- $p(s, \psi)$ is the generative model of external reality

Action and perception serve dual roles: perception adjusts internal states $\mu$ to minimize divergence, while action alters the environment to make observations conform to predictions.

## Homeostatic sleep and memory consolidation

### The synaptic saturation dilemma

Continuous online learning in analog hardware leads to conductance saturation:
- Hebbian and contrastive updates incrementally increase memristive conductances.
- Saturated conductances exhaust physical dynamic range and degrade signal-to-noise ratio (SNR).
- Continual exposure to new tasks overwrites historical weights, resulting in catastrophic forgetting.

### Dual-speed memory and offline consolidation

The ThermoCog design proposes a biomorphic sleep cycle based on the Synaptic Homeostasis Hypothesis (Tononi and Cirelli):

1. Wake Phase: The high-speed episodic buffer (hippocampal analogue) rapidly captures novel behavioral experiences during wakefulness.
2. Sleep Onset: Sensory inputs decouple. The agent transitions into a quiescent state.
3. Offline Memory Replay: The episodic buffer replays compressed trajectory vectors through the network, transferring representations into slow cortico-memristive arrays.
4. Global Synaptic Downscaling: Synaptic conductances are multiplicatively scaled downward toward baseline setpoints while preserving relative conductance ratios:

$$G_{ij} \leftarrow G_{ij} \cdot \left( 1 - \gamma \frac{G_{ij}}{G_{\max}} \right)$$

This downscaling resets metabolic energy consumption, prevents saturation, and consolidates invariant representations into long-term memory.

## Hardware realization matrix and roadmap

### Comparative specifications

| Metric | Contemporary GPU Cluster (H100/B200) | Biological Brain (Human Baseline) | ThermoCog Biomorphic Silicon |
| :--- | :--- | :--- | :--- |
| Active Operating Power | 700 W per GPU (Megawatts per cluster) | ~20 Watts total | 20 Watt design target (unbuilt) |
| Architecture | Von Neumann (Decoupled HBM and ALUs) | Non-Von Neumann (Colocated memory/compute) | Non-Von Neumann (In-memory analog crossbars) |
| Energy comparison | Depends on operation and hardware | Depends on biological event and measurement | No matched-task measurement; design targets only |
| Memory and signal latency | Hardware-dependent | Nonzero biological transmission and processing | Proposed in-memory architecture; settling and peripheral latency unmeasured |
| Learning Algorithm | Global Backpropagation through time | Local continuous energy relaxation | Holomorphic Equilibrium Propagation |
| Hardware Fault Tolerance | Depends on error correction and redundancy | Biological adaptation and redundancy | Proposed local defect-tolerance mechanisms; untested |
| Sleep Requirement | None (Static weights post-training) | Mandatory (Homeostatic downscaling) | Mandatory (Offline replay and downscaling) |

### Development roadmap

- Phase 1 (Software simulation, planned): Simulation of the continuous dynamics, Liquid Time-Constant equations and Equilibrium Propagation under modelled analog noise. Not yet built; no results are claimed.
- Phase 2 (Fixed-point microcontroller study, planned): A proposed Q16.16 fixed-point study on an ARM Cortex-M4 without an FPU. Not run; no latency, power or memory figures exist.
- Phase 3 (Discrete analog prototyping, future work): Mixed-signal CMOS co-processor integrating discrete memristive crossbar arrays for in-situ Equilibrium Propagation. Not started.
- Phase 4 (Monolithic 3D neuromorphic ASIC, long-term target): Custom 3D integrated circuit combining continuous-time analog neural cores, nanoscale memristor crossbars, and dedicated homeostatic sleep circuitry, aimed at a 20-Watt envelope. Not built and not a build plan.

## Primary academic literature archive

1. Friston, Karl (2006). "A free energy principle for the brain." *Neuron*, 56(2), 263-276. DOI: 10.1016/j.neuron.2006.07.016
2. Scellier, Benjamin, and Bengio, Yoshua (2017). "Equilibrium Propagation: Comparing Gradient-Descent to Brain-Driven Learning." *Frontiers in Computational Neuroscience*, 11:24. DOI: 10.3389/fncom.2017.00024
3. Landauer, Rolf (1961). "Irreversibility and heat generation in the computing process." *IBM Journal of Research and Development*, 5(3), 183-191. DOI: 10.1147/rd.53.0183
4. Mead, Carver (1990). "Neuromorphic electronic systems." *Proceedings of the IEEE*, 78(10), 1629-1636. DOI: 10.1109/5.58356
5. Hasler, Jennifer, and Marr, Bo (2013). "Finding a roadmap to achieve large neuromorphic hardware systems." *Frontiers in Neuroscience*, 7:118. DOI: 10.3389/fnins.2013.00118
6. Tononi, Giulio, and Cirelli, Chiara (2014). "Sleep and the price of plasticity: from synaptic and cellular homeostasis to memory consolidation and integration." *Neuron*, 81(1), 12-34. DOI: 10.1093/sleep/37.1.13
7. Mordvintsev, Alexander, Randazzo, Ettore, Niklasson, Eyvind, and Levin, Michael (2020). "Growing Neural Cellular Automata." *Distill*, 5(2), e23. DOI: 10.23915/distill.00023
8. Hasani, Ramin, Lechner, Mathias, Amini, Alexander, Rus, Daniela, and Grosu, Radu (2021). "Liquid Time-Constant Networks." *AAAI Conference on Artificial Intelligence*, 35(9), 7657-7666. DOI: 10.1609/aaai.v35i9.16937

## Frequently asked questions

### What does the name ThermoCog mean, and what is its core thesis?

ThermoCog is a shorthand for "Thermodynamic Cognition." This name highlights the underlying thesis of the entire whitepaper: that true intelligence is a physical, energy-minimizing process rather than just a software scaling problem. Rather than treating computation as abstract arithmetic forced through digital logic gates, ThermoCog treats intelligence as a dynamical physical system relaxing toward equilibrium down an energy landscape.

### Why is industrial digital scaling hitting a thermodynamic wall?

Digital accelerators suffer from the Von Neumann bottleneck: memory arrays are physically decoupled from arithmetic logic units. Moving tensors across buses and memory can consume substantial energy, depending on workload and hardware. Under Landauer's principle, erasing bits requires minimum physical energy dissipation. The bit-erasure bound cannot be directly equated with the energy of an arithmetic operation or inference task. No proximity to the bound is demonstrated by ThermoCog.

### How does continuous physical relaxation compute gradients without backpropagation?

Through Equilibrium Propagation (EqProp) and Holomorphic EqProp. The physical network settles into a low-energy state during the free phase, then receives a weak teaching perturbation during the nudged phase. The physical difference in node states yields exact loss gradients through local updates, completely eliminating global backward error transport and transposed matrix operations.

### Doesn't peripheral circuitry (ADCs/DACs) eliminate analog energy advantages?

Naive hybrid accelerators hit an "ADC/DAC wall" by digitizing activations between every discrete layer, spending >80% of energy on conversion. ThermoCog avoids per-layer conversion by propagating states continuously in the analog domain via Liquid Time-Constant (LTC) neural ODEs. Mixed-signal conversion is restricted strictly to boundary transducers and episodic memory buffers. Specific low-power mixed-signal readout topologies and continuous-time interfaces are maintained under confidential patent protections.

### How do analog memristors maintain stability despite drift, noise, and sneak paths?

Rather than enforcing rigid, bit-exact digital precision, ThermoCog relies on multi-timescale architectural resilience. Continuous dynamical attractors absorb low-level physical noise, Neural Cellular Automata dynamically reroute signals around defective crosspoints, and homeostatic sleep downscaling periodically renormalizes synaptic conductances to prevent saturation. Specific dynamic defect-rerouting topologies and adaptive calibration protocols are maintained under confidential patent protections.

### How do Neural Cellular Automata solve analog hardware defects and yield?

Analog memristive crossbars suffer from device-to-device variability and manufacturing defects. Rather than enforcing rigid circuit topologies, ThermoCog utilizes Neural Cellular Automata where local cells execute developmental morphogenesis. When hardware junctions fail or degrade, neighboring cells sense anomalous local electrical gradients and dynamically re-route signal pathways, achieving functional fault-tolerance without external intervention.

### Why does a 20-Watt biomorphic AGI need to sleep?

Continuous online learning causes synaptic conductances to saturate toward upper physical rails, exhausting dynamic range and inducing catastrophic forgetting. During quiescent sleep periods, sensory input is decoupled, an episodic buffer replays memories into slow cortico-memristive weights, and global homeostatic downscaling resets synaptic headroom without erasing consolidated knowledge.

### What is the mathematical relationship between Active Inference and physical energy?

Active Inference states that living organisms maintain structural integrity by minimizing Variational Free Energy. Mathematically, Variational Free Energy upper-bounds perceptual surprise. Variational free energy is an information-theoretic objective, not thermodynamic energy in joules. A physical circuit realization would require an explicit mapping and experimental validation; ThermoCog has not established that mapping or measured its dissipation.

### What is the role of Liquid Time-Constant networks in ThermoCog?

Liquid Time-Constant (LTC) networks implement continuous-time dynamical models whose effective time constants adapt dynamically to incoming sensory inputs. When sensory inputs are intense or rapid, the effective time constant contracts, enabling high-frequency perception; during steady conditions, it expands, conserving metabolic energy.

### Has ThermoCog been validated in physical hardware?

No. Nothing has been implemented or measured. Phase 2 is a planned study of the fixed-point (Q16.16) formulation on a Cortex-M4 without a floating-point unit. No latency, power, memory or energy-per-operation figures exist for it, and none are claimed. Monolithic analog memristive integration (Phase 4) is a long-term target.

### How does ThermoCog handle sneak paths, line-resistance drops, and conductance asymmetry in physical crossbars?

Unbounded passive crossbar meshes are not physically scalable. ThermoCog employs modular tiled sub-arrays (typically 64×64 to 128×128 cells) equipped with 1T1R/1S1R selector devices to bound IR line drops under 2.5% and suppress sneak-path currents below the thermal floor. Memristive conductance asymmetry is resolved via Holomorphic Equilibrium Propagation and vector-field asymmetric feedback alignment in Cauchy-Riemann coordinates. Neural Cellular Automata dynamically reroute signals around defective nodes at the topological layer, while homeostatic sleep downscaling (~18%) resets device drift toward saturation rails.

### Is ThermoCog attempting to displace digital datacenter GPU clusters, and what is the commercial strategy?

ThermoCog does not seek to displace digital hyperscaler clusters for cloud text generation. Instead, ThermoCog targets high-demand untethered edge and embodied intelligence applications (aerospace, defense robotics, biomedical cybernetics) requiring uncompromised 20-Watt or milliwatt envelopes, alongside licensing proprietary biomorphic topologies, Holomorphic EqProp algorithms, and NCA self-repair engines to semiconductor fabricators through patent disclosures.

## Foundational Lexicon & Glossary

1. **ThermoCog (Thermodynamic Cognition)**: The foundational thesis that intelligence is fundamentally a physical, non-equilibrium energy-minimizing relaxation process rather than an abstract digital software scaling problem.
2. **Landauer Floor**: The thermodynamic lower bound on irreversible computation (`k_B T ln 2 ≈ 2.87 × 10⁻²¹ J` at 300K). Digital processors dissipate `>10⁷×` this floor moving bits across buses.
3. **Equilibrium Propagation (EqProp)**: Biomorphic energy-based learning algorithm computing exact loss gradients locally from physical state differences (`Δw ∝ s_i⁺ s_j⁺ - s_i⁻ s_j⁻`), circumventing the non-physical weight transport problem of backpropagation.
4. **Liquid Time-Constant Networks (LTCs)**: Continuous-depth neural ODEs whose effective membrane time constants `τ_eff(x)` adapt dynamically to input intensity, contracting for high-frequency control and dilating during quiescent periods to conserve energy.
5. **Neural Cellular Automata (NCA)**: Decentralized morphogenetic networks where local update rules grow functional topologies, repair physical memristor faults, and maintain structural homeostasis without centralized intervention.
6. **Synaptic Homeostasis Hypothesis (SHY)**: Biological sleep principle wherein net waking synaptic potentiation is renormalized downward (~18% net downscaling), restoring dynamic range and metabolic stability within a 20-Watt envelope.
7. **Embodied Active Inference**: Friston's formulation where an agent minimizes Variational Free Energy (`F`), bounding perceptual surprise through reciprocal perception and action.
8. **Memristive Crossbar Array**: Non-volatile analog resistor grid executing Vector-Matrix Multiplication in `O(1)` continuous time via Ohm's law and Kirchhoff's current law directly in-memory.
9. **Von Neumann Bottleneck**: The severe bus latency and energy dissipation penalty (>90% of GPU power) incurred shuttling bits between physically separated compute and memory units.
10. **Dual-Speed Memory System**: Fast episodic buffer (hippocampal analog) coupled with slow structural consolidation (neocortical analog) via offline sleep replay without catastrophic forgetting.

## Canonical links

- [ThermoCog Home](https://thermocog.com/): Official blueprint and interactive specification
- [LLM Summary Index](https://thermocog.com/llms.txt): Machine-readable summary for agents
- [Sitemap](https://thermocog.com/sitemap.xml): Machine-readable index of canonical pages

_Generated: 2026-09-19_
