Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions ci/testcases/debug.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -107,3 +107,13 @@ tests:
cores: 4
l2cache: true
args: -n128
- id: simx_functional-3
via: blackbox
drivers:
- simx
app: sgemm
configs: -DSIMX_FUNCTIONAL -DSIMX_MT=4 -DSIMX_QUANTUM=100
shape:
cores: 4
l2cache: true
args: -n128
9 changes: 9 additions & 0 deletions docs/debugging.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,15 @@ before switching to a timed build for cycle-level analysis. Cycle counts in
a functional run are non-physical: never use them for performance
comparisons, `perf_gate`, or `model_parity`.

For the fastest multi-threaded functional runs, add `-DSIMX_QUANTUM=<Q>`:
domains run `Q` cycles between barriers and a cross-domain call waits for
the next quantum boundary. Results stay bit-identical across thread
counts for a given `Q`, but changing `Q` changes the cycle count. `Q=100` is
a good start; the timed kernel rejects `Q > 1`.

// Functional, 16 host threads, barrier every 100 cycles
$ CONFIGS="-DSIMX_FUNCTIONAL -DSIMX_MT=16 -DSIMX_QUANTUM=100" ./ci/blackbox.sh --driver=simx --app=raycast --cores=16 --l2cache

## RTL Debugging

To debug the processor RTL, you need to use VLSIM or RTLSIM driver. VLSIM simulates the full processor including the AFU command processor (using `/rtl/afu/opae/vortex_afu.sv` as top module). RTLSIM simulates the Vortex processor only (using `/rtl/Vortex.v` as top module).
Expand Down
30 changes: 24 additions & 6 deletions docs/designs/simobject.md
Original file line number Diff line number Diff line change
Expand Up @@ -327,10 +327,10 @@ must complete in-cycle). Default to `delay=1` for normal flow.

### Inflight counter and `idle()`

`SimChannelBase::inflight_count()` is a process-global counter
incremented on `reserve()` and decremented on queue pop. Useful for
deadlock detection (a tick that drops to zero traffic and stays there
when work is expected) and for end-of-simulation drain assertions.
Every endpoint counts the packets sent toward it but not yet consumed:
incremented on `reserve()`, decremented on queue pop. The count lives in the
endpoint's execution domain (one cache line per domain, no atomics), because
only that domain's context reserves toward or pops from its endpoints.

Host-side quiescence is a framework service: `SimPlatform::idle()` is true
when no packet is in flight **and** no deferred cross-domain delivery is
Expand Down Expand Up @@ -413,8 +413,8 @@ packet delivery.
2. Calls `do_reset()` on every active-reset object.
3. Resets `cycles_ = 0`.

Inflight-count is *not* reset by the platform — clear it externally if
your test depends on it. Module `on_reset()` should clear all internal
The in-flight counts are *not* reset by the platform, and `idle()` is their
only reader. Module `on_reset()` should clear all internal
queues and counters; channels are reset as a side effect (storage is
reconstructed at construction; reset doesn't recreate them).

Expand Down Expand Up @@ -470,6 +470,15 @@ owner, cross-domain deliveries are merged in a canonical `(due, src, seq)`
order, and each cycle is fenced by barriers. A contributor who follows the
Cardinal Rule gets deterministic parallelism for free.

Cross-domain calls travel through per-(target, source) mailbox lanes, each
written by one source and drained by one target, so no lock or shared counter
sits on the path. A lane is triple-buffered by epoch (a cycle, or a quantum in
the functional kernel): a source writes the current epoch, and the target
drains the previous one after the barrier that closed it. A call therefore runs
at the first epoch boundary at or after its due cycle, in `(due, src, seq)`
order; with one-cycle epochs this is exactly the order of a single shared
inbox.

What makes a module MT-safe is not new code — it is the *absence* of illegal
code, enforced by lint and asserts:

Expand Down Expand Up @@ -505,6 +514,15 @@ the same executor with timing removed, for full-speed architectural runs
- The MT executor, domains, canonical ordering, and topology validation are
retained — `-DSIMX_MT` composes and cycles remain bit-identical across
thread counts.
- `-DSIMX_QUANTUM=<Q>` (resolved in `sim/simx/types.h`, handed to
`SimPlatform::set_quantum()`) lets domains run `Q` cycles between
barriers; a cross-domain call runs at the first quantum boundary at or
after its due cycle.
Results are bit-identical for every thread count at a given `Q`, but `Q`
itself changes the (non-physical) cycle count. A larger `Q` amortizes
synchronization but delays every crossing — an L1 miss into the uncore
domain waits for the boundary — so `Q` around 100 is a good start. The
timed kernel keeps per-cycle lockstep and rejects `Q > 1`.
- Cycle counts are monotonic but **non-physical**: a functional build must
never feed `perf_gate`, `model_parity`, or baseline regeneration.

Expand Down
3 changes: 2 additions & 1 deletion docs/designs/simx_simulator_architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,8 @@ it serves as the RTL oracle for cycle-parity debugging.
With `SIMX_MT > 1` (a build define set via `CONFIGS`) the engine
executes execution domains (one per socket, uncore = domain 0) on
parallel workers in cycle lockstep, with results bit-identical to
serial execution.
serial execution. The functional kernel can instead synchronize every
`SIMX_QUANTUM` cycles (see [SimObject](simobject.md)).
- **`SimEventLink<Msg>`** — a one-way, typed control-plane link (barrier
arrive/resume, FWD arm/done, doorbells): both ends are declared members,
the receiver binds a member-function handler, elaboration wires
Expand Down
11 changes: 8 additions & 3 deletions docs/install_vortex.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,20 +43,25 @@
Useful configure options: `--xlen={32,64}` selects the ISA width,
`--prefix=<path>` the install location, and `--profile=<name>` builds the
machine described by `profiles/<name>.json` by default (see
[Hardware Profiles](simulation.md#hardware-profiles)). Two SimX build knobs are plain
[Hardware Profiles](simulation.md#hardware-profiles)). Three SimX build knobs are plain
config defines, passed like any `VX_CFG_*` option through `CONFIGS`:
`-DSIMX_MT=<T>` runs the simulator on `T` host threads (default serial;
cycle results are bit-identical for every thread count), and
cycle results are bit-identical for every thread count),
`-DSIMX_FUNCTIONAL` builds the **functional** simulation kernel (timing
disabled, for full-speed architectural runs such as conformance suites —
functional cycle counts are non-physical and must never be used for
performance comparisons):
performance comparisons), and `-DSIMX_QUANTUM=<Q>` (functional kernel
only) synchronizes the threads every `Q` cycles instead of every cycle:

```
$ CONFIGS="-DSIMX_MT=4" ./ci/blackbox.sh --driver=simx --app=sgemm --cores=4 --l2cache
$ CONFIGS="-DSIMX_FUNCTIONAL" ./ci/blackbox.sh --driver=simx --app=sgemm
$ CONFIGS="-DSIMX_FUNCTIONAL -DSIMX_MT=16 -DSIMX_QUANTUM=100" ./ci/blackbox.sh --driver=simx --app=sgemm --cores=16 --l2cache
```

Multi-threading pays off only with several sockets (one execution domain
each); a 1-core build has nothing to run in parallel.

`make install` lays out the Vortex SDK under `$VORTEX_PATH` (default
`<build>/install`, override with `../configure --prefix=...`):

Expand Down
114 changes: 114 additions & 0 deletions sim/common/fixed_vector.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
// Copyright © 2019-2026
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#pragma once

#include <array>
#include <cstddef>
#include <cstdint>
#include <initializer_list>
#include <util.h>

namespace vortex {

// A vector with inline storage whose capacity is a compile-time bound (a
// VX_CFG_* thread or operand count): the std::vector subset the simulator
// uses, without a heap allocation per object or per copy. The size is still
// dynamic up to the capacity; exceeding it is an assertion failure.
template <typename T, uint32_t N>
class FixedVector {
public:
using value_type = T;
using size_type = uint32_t;
using iterator = T*;
using const_iterator = const T*;
using reference = T&;
using const_reference = const T&;

FixedVector() : size_(0) {}

explicit FixedVector(size_type n) : size_(0) {
this->assign(n, T{});
}

FixedVector(size_type n, const T& value) : size_(0) {
this->assign(n, value);
}

FixedVector(std::initializer_list<T> init) : size_(0) {
__assert(init.size() <= N, "FixedVector: capacity " << N << " exceeded");
for (auto& v : init) {
data_[size_++] = v;
}
}

static constexpr size_type capacity() { return N; }

size_type size() const { return size_; }
bool empty() const { return size_ == 0; }

T* data() { return data_.data(); }
const T* data() const { return data_.data(); }

iterator begin() { return data_.data(); }
iterator end() { return data_.data() + size_; }
const_iterator begin() const { return data_.data(); }
const_iterator end() const { return data_.data() + size_; }

reference operator[](size_type i) { return data_[i]; }
const_reference operator[](size_type i) const { return data_[i]; }

reference at(size_type i) {
__assert(i < size_, "FixedVector: index " << i << " out of range " << size_);
return data_[i];
}
const_reference at(size_type i) const {
__assert(i < size_, "FixedVector: index " << i << " out of range " << size_);
return data_[i];
}

reference front() { return data_[0]; }
const_reference front() const { return data_[0]; }
reference back() { return data_[size_ - 1]; }
const_reference back() const { return data_[size_ - 1]; }

void assign(size_type n, const T& value) {
__assert(n <= N, "FixedVector: capacity " << N << " exceeded");
for (size_type i = 0; i < n; ++i) {
data_[i] = value;
}
size_ = n;
}

void resize(size_type n, const T& value = T{}) {
__assert(n <= N, "FixedVector: capacity " << N << " exceeded");
for (size_type i = size_; i < n; ++i) {
data_[i] = value;
}
size_ = n;
}

void push_back(const T& value) {
__assert(size_ < N, "FixedVector: capacity " << N << " exceeded");
data_[size_++] = value;
}

void pop_back() { --size_; }
void clear() { size_ = 0; }

private:
std::array<T, N> data_;
size_type size_;
};

} // namespace vortex
Loading
Loading