Welcome to Tianshou!

Tianshou (天授) is a reinforcement learning platform based on pure PyTorch. Unlike existing reinforcement learning libraries, which are mainly based on TensorFlow, have many nested classes, unfriendly API, or slow-speed, Tianshou provides a fast-speed framework and pythonic API for building the deep reinforcement learning agent. The supported interface algorithms include:

• PGPolicy Policy Gradient

• DQNPolicy Deep Q-Network

• DQNPolicy Double DQN with n-step returns

• A2CPolicy Advantage Actor-Critic

• DDPGPolicy Deep Deterministic Policy Gradient

• PPOPolicy Proximal Policy Optimization

• TD3Policy Twin Delayed DDPG

• SACPolicy Soft Actor-Critic

• ImitationPolicy Imitation Learning

• PrioritizedReplayBuffer Prioritized Experience Replay

• compute_episodic_return() Generalized Advantage Estimator

Here is Tianshou’s other features:

• Elegant framework, using only ~2000 lines of code

• Support parallel environment sampling for all algorithms: Parallel Sampling

• Support recurrent state representation in actor network and critic network (RNN-style training for POMDP): RNN-style Training

• Support any type of environment state (e.g. a dict, a self-defined class, …): User-defined Environment and Different State Representation

• Support customized training process: Customize Training Process

• Support n-step returns estimation compute_nstep_return() for all Q-learning based algorithms

中文文档位于 https://tianshou.readthedocs.io/zh/latest/

Installation

Tianshou is currently hosted on PyPI. You can simply install Tianshou with the following command (with Python >= 3.6):

pip3 install tianshou

You can also install with the newest version through GitHub:

pip3 install git+https://github.com/thu-ml/tianshou.git@master

If you use Anaconda or Miniconda, you can install Tianshou through the following command lines:

# create a new virtualenv and install pip, change the env name if you like
conda create -n myenv pip
# activate the environment
conda activate myenv
# install tianshou
pip install tianshou

After installation, open your python console and type

import tianshou as ts
print(ts.__version__)

If no error occurs, you have successfully installed Tianshou.

Tianshou is still under development, you can also check out the documents in stable version through tianshou.readthedocs.io/en/stable/.

Tutorials

• Deep Q Network
• Basic concepts in Tianshou
• Train a model-free RL agent within 30s
• Cheat Sheet

API Docs

• tianshou.data
• tianshou.env
• tianshou.policy
• tianshou.trainer
• tianshou.exploration
• tianshou.utils

Community

• Contributing to Tianshou
• Contributor

Indices and tables

• Index

• Module Index

• Search Page