Skip to content

Add POWER8 LLM inference and NUMA weight banking projects - #161

Merged
DefTruth merged 2 commits into
xlite-dev:mainfrom
Scottcjn:add-power8-ram-coffers
Mar 18, 2026
Merged

DefTruth merged 2 commits into
xlite-dev:mainfrom
Scottcjn:add-power8-ram-coffers

Conversation

@Scottcjn

Copy link
Copy Markdown
Contributor

Adds two CPU inference projects to the CPU/Single GPU/FPGA/NPU/Mobile Inference section:

  • llama-cpp-power8 — POWER8 VSX/AltiVec optimizations for llama.cpp featuring vec_perm non-bijunctive attention collapse, IBM MASS math library integration, and dcbt resident prefetch hints. Achieves 8.8x speedup over stock llama.cpp on POWER8 S824.
  • RAM Coffers — NUMA-aware weight banking system for LLM inference on multi-socket servers. Routes model layers to specific NUMA nodes based on cognitive function mapping, with selective prefetch and non-bijunctive prune-before-fetch.

Both target CPU-only inference on non-x86 architectures (IBM POWER8), which is underrepresented in the current list.

@DefTruth

Copy link
Copy Markdown
Member

please fix conflicts

@Scottcjn
Scottcjn force-pushed the add-power8-ram-coffers branch from 17a1c59 to ad48b04 Compare March 17, 2026 22:56

@DefTruth DefTruth left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM~ Thanks

@DefTruth
DefTruth merged commit 5582a8b into xlite-dev:main Mar 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants