Skip to content

Commit b999098

Browse files
authored
Update BasicConcepts.md
1 parent 7e82eeb commit b999098

File tree

1 file changed

+18
-13
lines changed

1 file changed

+18
-13
lines changed

Documents/BasicConcepts.md

Lines changed: 18 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -6,28 +6,39 @@ This repo provides or plans to provide following tools for game AI and machine l
66
### Core Functionalities
77
1. Training with Reinforcement Learning
88
* Using PPO(Proximal Policy Optimization) algorithm
9-
* You can provide heruistic(already know better behaviours) to speed up the training.
10-
* PPO with Neural Evolution(work in progess)
119

12-
2. MAES[(Matrix Adaptation Evolution Strategy)](https://en.wikipedia.org/wiki/CMA-ES)
10+
2. CMA-ES[(Covariance Matrix Adaptation - Evolution Strategy)](https://en.wikipedia.org/wiki/CMA-ES)
1311
* A genetic algorithm to find the best solution.
1412
* Can be used with or without ML-Agents.
1513
3. Training with Supervise Learning
1614
* It is called imitation learning in ML-Agents. Here it is called supervised learning or mimic(I am too lazy to unify the name). (basically you show the correct action under different circumstances and let the neural network remember it.)
17-
* The correct action data can be collected from human playing or others, such as MAES.
15+
* The correct action data can be collected from human playing or others, such as CMAES.
1816

1917
4. Neural Evolution(work in progess)
2018
* Evolve the neural network's weights using MAES instead of gradien descent.
19+
* Currently only works for not very deep neural network.
2120
### Other tools
2221
1. GAN(Generative adversarial network)
2322
* Including [Traning with Prediction to Stableize](https://www.semanticscholar.org/paper/Stabilizing-Adversarial-Nets-With-Prediction-Yadav-Shah/ec25504486d8751e00e613ca6fa64b256e3581c8).
2423
* [Improved Training of Wasserstein GANs with gradient penalty](http://papers.nips.cc/paper/7159-improved-training-of-wasserstein-gans.pdf)(future plan).
2524
2. Heat map rendering tool
2625
3. [Artistic Style Transfer](https://arxiv.org/abs/1705.08086)(future plan)
2726
4. In editor progress visualization tool(future plan)
28-
27+
### Features Not Have
28+
1. Curriculum Training
29+
* It is pretty easy to implement curriculum learning by yourself because there is not communication between programs.
30+
2. Recurrent Neural Network
31+
* Keras Sharp currently does not have recurrent neural network because it is not in the c library of c Tensorflow. I might implement it in the future, if I have time and want to, or CNTK can be used as backend instead of Tensorflow.
32+
3. Training on mobile device
33+
* Not unless tensorflow start to include training on mobile build, or CNTK can be used as backend and it support mobile device.
34+
However, you can still use inference on mobile device.
35+
4. Curiosity Module.
36+
* I don't have time to implement it yet.
37+
2938
## Concepts
30-
You can fisrt go through the [vverview of Unity ML-Agents](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/ML-Agents-Overview.md), without python related stuff.
39+
You don't have to read the folowing if you just want to use this repository as it is.
40+
41+
You can fisrt go through the [Overview of Unity ML-Agents](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/ML-Agents-Overview.md), without python related stuff.
3142

3243
Assume that you are somehow familiar with Unity ML-Agents, then following will be some brief explanation of concepts/key conponents that are used in this repo.
3344
(To be added)
@@ -48,10 +59,4 @@ We defined some UnityNetwork scriptable objects, where you can easily define a n
4859
The models implemented by us usually need a network scriptable object that implement certain interface. We have already made the simple version of those network for you. However, you can also easily make your own customized network.
4960

5061

51-
## Features Not Gonna Have
52-
1. Curriculum Training
53-
* It is pretty easy to implement curriculum learning by yourself because there is not communication between programs.
54-
2. Recurrent Neural Network
55-
* Keras Sharp currently does not have recurrent neural network because it is not in the c library of c Tensorflow. I might implement it in the future, if I have time and want to, or CNTK can be used as backend instead of Tensorflow.
56-
3. Training on mobile device
57-
* Not unless tensorflow start to include training on mobile build, or CNTK can be used as backend and it support mobile device. However, you can still use inference on mobile device.
62+

0 commit comments

Comments
 (0)