A package timeout became a hardware report.
Wang Miao hit an infinite loop while packaging Normaliz for Debian on LoongArch. The code processed points in parallel and used OpenMP atomic increments for shared counters. Every point could be marked done while the total counter remained short. The gap changed between runs.[1]
The investigation first checked the source logic and OpenMP implementation. Disassembly showed the compiler had emitted the expected LoongArch amadd.d instruction. Extra std::atomic counters also disagreed. A simple atomic-add loop still refused to fail.[1]
The broken counter needed the right neighboring memory traffic before it would tell the truth.
The trigger was wider than the suspicious line.
The reported minimal case paired relaxed atomic operations with interleaved vectorized memory reads on different physical cores. A vectorized memcpy supplied the missing condition. The authors later found a lower-probability scalar-read trigger for particular address relationships.[1]
The report says the failure affected atomic add, compare-and-swap, max, and swap tests under the same conditions. Operations with a data barrier did not fail in its published trial table. That result belongs to the tested LA664 systems and test setup. It is not a general benchmark for LoongArch or atomic operations.[1]
The contracts above the chip were clear.
OpenMP 5.1 defines an atomic construct with clauses that choose the operation and memory ordering behavior.[2] GCC documents atomic built-ins that map C++ memory-model operations to target instructions and fences.[3]
The LoongArch reference manual says the complete read-modify-write process for AM* instructions is atomic. It also describes AMADD as reading the old value, adding the register value, and writing the result.[4] The reported behavior violated the hardware contract that the compiler and application relied on.
A minimal reproducer is a map of hidden conditions.
- Keep the invariant. Record the expected count, observed count, and the rule that says they must match.
- Pin the generated instruction. Save compiler version, flags, disassembly, and barrier form.
- Pin placement. Record processor model, core IDs, sibling relationships, and thread affinity.
- Keep nearby traffic. Preserve copy size, vector width, addresses, alignment, and operations between atomic accesses.
- Keep the machine receipt. Save kernel, firmware, microcode, trial count, failure count, and the fix state.
Removing an innocent memcpy would have made the reproducer smaller and false. Good reduction removes noise without deleting the condition that wakes the bug.
The firmware result still needs a host receipt.
The authors reported the issue to Loongson on August 26. They received test firmware on September 9. Their tests on 3A6000 and 3C6000/S systems stopped reproducing the failure after bit 13 of an internal register was set. They reported no single-core loss and a small multi-core cost. Loongson told them a firmware release was expected before October 1.[1]
That is a first-party investigation report about test firmware. Before closing a production incident, identify the public firmware build, install it on the affected host, rerun the pinned reproducer, and keep the before-and-after counts.