Welcome to Tianshou!

Tianshou (天授) is a reinforcement learning platform based on pure PyTorch. Unlike existing reinforcement learning libraries, which are mainly based on TensorFlow, have many nested classes, unfriendly API, or slow-speed, Tianshou provides a fast-speed framework and pythonic API for building the deep reinforcement learning agent. The supported interface algorithms include:

• DQNPolicy Deep Q-Network

• DQNPolicy Double DQN

• DQNPolicy Dueling DQN

• C51Policy Categorical DQN

• QRDQNPolicy Quantile Regression DQN

• IQNPolicy Implicit Quantile Network

• FQFPolicy Fully-parameterized Quantile Function

• PGPolicy Policy Gradient

• NPGPolicy Natural Policy Gradient

• A2CPolicy Advantage Actor-Critic

• TRPOPolicy Trust Region Policy Optimization

• PPOPolicy Proximal Policy Optimization

• DDPGPolicy Deep Deterministic Policy Gradient

• TD3Policy Twin Delayed DDPG

• SACPolicy Soft Actor-Critic

• DiscreteSACPolicy Discrete Soft Actor-Critic

• ImitationPolicy Imitation Learning

• DiscreteBCQPolicy Discrete Batch-Constrained deep Q-Learning

• DiscreteCQLPolicy Discrete Conservative Q-Learning

• DiscreteCRRPolicy Critic Regularized Regression

• PSRLPolicy Posterior Sampling Reinforcement Learning

• PrioritizedReplayBuffer Prioritized Experience Replay

• compute_episodic_return() Generalized Advantage Estimator

Here is Tianshou’s other features:

• Elegant framework, using only ~3000 lines of code

• State-of-the-art MuJoCo benchmark

• Support parallel environment simulation (synchronous or asynchronous) for all algorithms: Parallel Sampling

• Support recurrent state representation in actor network and critic network (RNN-style training for POMDP): RNN-style Training

• Support any type of environment state/action (e.g. a dict, a self-defined class, …): User-defined Environment and Different State Representation

• Support Customize Training Process

• Support n-step returns estimation compute_nstep_return() and prioritized experience replay PrioritizedReplayBuffer for all Q-learning based algorithms; GAE, nstep and PER are very fast thanks to numba jit function and vectorized numpy operation

• Support Multi-Agent RL

• Comprehensive unit tests, including functional checking, RL pipeline checking, documentation checking, PEP8 code-style checking, and type checking

中文文档位于 https://tianshou.readthedocs.io/zh/latest/

Installation

Tianshou is currently hosted on PyPI and conda-forge. It requires Python >= 3.6.

You can simply install Tianshou from PyPI with the following command:

$ pip install tianshou

If you use Anaconda or Miniconda, you can install Tianshou from conda-forge through the following command:

$ conda -c conda-forge install tianshou

You can also install with the newest version through GitHub:

$ pip install git+https://github.com/thu-ml/tianshou.git@master --upgrade

After installation, open your python console and type

import tianshou
print(tianshou.__version__)

If no error occurs, you have successfully installed Tianshou.

Tianshou is still under development, you can also check out the documents in stable version through tianshou.readthedocs.io/en/stable/.

Tutorials

• Deep Q Network
• Basic concepts in Tianshou
• Understand Batch
• Multi-Agent RL
• Benchmark
• Cheat Sheet

API Docs

• tianshou.data
• tianshou.env
• tianshou.policy
• tianshou.trainer
• tianshou.exploration
• tianshou.utils

Community

• Contributing to Tianshou
• Contributor

Indices and tables

• Index

• Module Index

• Search Page