Catalog: parallelize all-sky tile prefetch
This commit is contained in:
1 parent
d9f2c83b9f
commit
457c0728b4
10 files changed
+371
-55
No files matched your search
@@ -46,7 +46,44 @@ make ENABLE_PNG=1 SPACETIME=minkowski all
|
||||
```
|
||||
|
||||
The renderer lazily loads only tiles touched by source triangles in the frame.
|
||||
It prefetches them serially, so OpenMP splat workers read immutable tile data.
|
||||
It first collects and de-duplicates the frame's touched tile IDs on the main
|
||||
thread. Newly needed CSVs are then read by a bounded OpenMP loader group
|
||||
(`--catalog-load-workers N`, default `4`); each worker owns distinct tile
|
||||
files. The main thread commits their results to the cache only after that group
|
||||
finishes, so OpenMP splat workers always read immutable tile data and never
|
||||
race catalog I/O. A tile already loaded or previously found absent is not read
|
||||
again during the same renderer run. The cache deliberately remains live for
|
||||
the entire movie: final numerical-BBH renders commonly sweep across the full
|
||||
sky, so tile eviction would trade the normal case for repeated filesystem I/O.
|
||||
|
||||
Each frame reports the number of requested/newly loaded/unavailable tiles,
|
||||
newly loaded stars, and CSV-load wall time. The loader count is deliberately
|
||||
independent of the splat worker count: start at the default and benchmark on
|
||||
the target storage before raising it.
|
||||
|
||||
Warm-cache reference measurement, 2026-08-27, on the available partial
|
||||
all-sky directory (18 CSV tiles, 12,917 stars, 128x72, 2-degree field):
|
||||
|
||||
| Loader workers | CSV load time |
|
||||
| ---: | ---: |
|
||||
| 1 | 5 ms |
|
||||
| 2 | 3 ms |
|
||||
| 4 | 2 ms |
|
||||
| 8 | 3 ms |
|
||||
|
||||
This confirms the bounded-loader path and motivates the default of four; it is
|
||||
not a cold-cache throughput claim for the final full catalog. Reproduce it
|
||||
with the rotating observer fixture:
|
||||
|
||||
```sh
|
||||
./build/minkowski_sky --all-sky-catalog assets/2mass/processed/all_sky \
|
||||
--observer-track tests/data/minkowski_rotation_2s.csv \
|
||||
--frames-dir output/imgs/catalog_prefetch_bench/w4 --frames-prefix frame \
|
||||
--start-time 0 --duration 0 --fps 1 --width 128 --height 72 \
|
||||
--coarse-cell-pixels 8 --fov-deg 2 --exposure 1e13 \
|
||||
--catalog-load-workers 4
|
||||
```
|
||||
|
||||
Missing tiles are skipped until the downloader has produced them. Fully
|
||||
contained tiles skip per-star source-triangle containment; partial tiles retain
|
||||
that exact check. For 2MASS apparent-solid-angle amplitudes, start with
|
||||
|
||||
Reference in new issue
Block a user