Efficient LLM inference on Slurm clusters.
A practical, multi-layered JSON repair library for Elixir that intelligently fixes malformed JSON strings commonly produced by LLMs, legacy systems, and data pipelines.
PipelineLLM 是一个系统性的大语言模型（LLM）后训练学习项目，涵盖从监督微调（SFT）到偏好优化（DPO）、强化学习（RLHF/PPO/GRPO）再到持续学习（Continual Learning)的完整技术栈。
Multi-model AI agent runtime. Define agents in YAML, connect 6 LLM providers, orchestrate with ReAct/Plan&Execute/Fan-Out/Pipeline/Supervisor/Swarm patterns, and deploy as REST/WebSocket API with RAG, memory, MCP tools, guardrails, and OpenTelemetry observability.
A production-grade, schema-aware PostgreSQL MCP server for enterprise AI. Features Zero-Trust SQL validation, multi-tier permissions, and real-time schema introspection for secure, autonomous database operations.
A browser-based UI for launching, monitoring, and managing multiple llama.cpp server instances from inside a Docker container. Includes an Ollama-compatible API proxy
Persistent Cognitive Memory Infrastructure — durable, multi-tenant memory layer for AI agents. HTTP + gRPC, hybrid retrieval, background workers, observability. Go.
Krako 2.0 – Energy-efficient, triadic multi-tier inference infrastructure enabling adaptive routing across heterogeneous edge–cloud nodes.
Joule is a budget-controlled AI agent runtime that minimizes energy and token usage through hierarchical routing and deterministic tool execution.
Enterprise-grade Sovereign AI Stack optimized for NVIDIA Blackwell (sm_120) & vLLM. Features 256K context window, 5.8k tok/s prefill, and integrated observability via Langfuse.
One command. Full LLM stack. Zero config.
An intelligent gateway for Claude APIs that dynamically routes requests to the most cost-efficient model, caches responses, and escalates based on confidence signals — reducing LLM spend without sacrificing quality.
High-performance C++ engine for Second-Order Hessian Pruning. The surgical foundation of the Tensorbit Labs P-D-Q pipeline for ultra-efficient LLM and Vision Transformers edge inference.
Local-first Python pipeline for analyzing Mixture-of-Experts (MoE) expert weight matrices from .safetensors and .npz, computing per-expert stats, and optionally simulating MLX quantization/dequantization error.
【非结构化数据pipeline】目标是自动化原始数据—>特定信息提取。first_example:收集任何的文档将其可视化为思维导图（进度1/3）
A lightweight Bun + Express template that connects to the Testune AI API and streams chat responses in real time using Server-Sent Events (SSE)
A Compute-Agnostic, WebSocket-first protocol for AI Agents. The high-performance alternative to MCP. Runs on Serverless or stateful servers with sub-30ms latency.
A provider-agnostic gateway for agentic systems — routing model calls, normalizing outputs, and handling provider-specific complexity.
Lightweight local-first LLM agent infrastructure: SQLite memory, tasks, connectors, and chain automation
Add a description, image, and links to the llm-infrastructure topic page so that developers can more easily learn about it.
To associate your repository with the llm-infrastructure topic, visit your repo's landing page and select "manage topics."