Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

A curated list of reinforcement learning with human feedback resources (continually updated)

Open-source pre-training implementation of Google's LaMDA in PyTorch. Adding RLHF similar to ChatGPT.

Let's build better datasets, together!

[CVPR 2024] Code for the paper "Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model"

BeaverTails is a collection of datasets designed to facilitate research on safety alignment in large language models (LLMs).

The ParroT framework to enhance and regulate the Translation Abilities during Chat based on open-sourced LLMs (e.g., LLaMA-7b, Bloomz-7b1-mt) and human written translation and evaluation data.

Implementation of Reinforcement Learning from Human Feedback (RLHF)

Product analytics for AI Assistants

The Prism Alignment Project

[ECCV2024] Towards Reliable Advertising Image Generation Using Human Feedback

Dataset Viber is your chill repo for data collection, annotation and vibe checks.

Code for the paper "Aligning LLM Agents by Learning Latent Preference from User Edits".

[ICML 2024] Code for the paper "Confronting Reward Overoptimization for Diffusion Models: A Perspective of Inductive and Primacy Biases"

[ NeurIPS 2023 ] Official Codebase for "Aligning Synthetic Medical Images with Clinical Knowledge using Human Feedback"

Pause your AI agent. Ask a human. Resume with their answer. Open source human-in-the-loop (HITL) library for production LLM agents: Slack, email, and web dashboard. Typed Pydantic and Zod responses. Durable Temporal and LangGraph adapters. AI verifier. Audit trail. Self-hosted, Apache 2.0. Python and TypeScript.

Documentation at

Reinforcement Learning from Human Feedback with 🤗 TRL

Break out of the AI training bubble

MCP server for human-in-the-loop surveys, A/B preference tests, ratings, and rankings. Get real human feedback inside Claude Code, Claude Desktop, Cursor, Windsurf, and any MCP client — powered by Datapoint AI.