Match3 example #4515

chriselion · 2020-09-25T23:09:10Z

Proposed change(s)

Work in progress match-3 example.

The abstract board, sensor, and actuator are in the com.unity.ml-agents.extension package. An example with basic mechanics is in the Match3 scene (and corresponding scripts).

TODO

optional color shuffling (as discussed in this paper)

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

Types of change(s)

New feature

Checklist

Added tests that prove my fix is effective or that my feature works
Updated the changelog (if applicable)
Updated the documentation (if applicable)

Other comments

* WIP special pieces * add special piece types to obs (need more from master first) * update sensors and models * docstrings

* handle no valid moves more gracefully * rename callback

com.unity.ml-agents.extensions/Runtime/Match3/Match3ActuatorComponent.cs

vincentpierre

I need to review more but looks good so far.

Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Agent.cs

Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Board.cs

Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Drawer.cs

vincentpierre · 2020-11-02T17:07:50Z

com.unity.ml-agents.extensions/Runtime/Match3/AbstractBoard.cs

+        }
+
+        /// <summary>
+        /// Check if the "half" of a move is matches 3 or more.


Suggested change

/// Check if the "half" of a move is matches 3 or more.

/// Check if the "half" of a move matches 3 or more.

What is a half move ?

Updated:

/// Check if one of the cells that is swapped during a move matches 3 or more. /// Since these checks are similar for each cell, we consider the Move as two "half moves".

vincentpierre · 2020-11-02T17:13:18Z

com.unity.ml-agents.extensions/Runtime/Match3/Move.cs

+        public Direction Direction;
+
+        /// <summary>
+        /// Construct a Move from its index and the board size.


A little more details on this method. Replace a with the since there is a 1 to 1 mapping between index and move and explain that the index corresponds to the index in the list of all potential moves on a board of size (maxRows, maxCols).

I rearranged some of the comments and moved them off of the MoveIndex member and into the class summary. I think between that and the comments on the moveIndex argument, it's fairly clear now.

com.unity.ml-agents.extensions/Runtime/Match3/Move.cs

docs/Learning-Environment-Examples.md

Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Agent.cs

vincentpierre · 2020-11-03T17:48:03Z

config/ppo/Match3.yaml

+    time_horizon: 1000
+    summary_freq: 10000
+    threaded: true
+  Match3SimpleHeuristic:


Very strange to have a section for a heuristic. If possible, I would remove, if not possible (so the curves can be observed, I would remove all hyperparameters except for max_steps and set it to 0 or 1.

vincentpierre · 2020-11-03T18:02:38Z

Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Agent.cs

+        public int MaxMoves = 500;
+
+        [Tooltip("If selected, will pick the move that makes the most points. Otherwise, will pick a random valid move.")]
+        public bool UseSmartHeuristic = false;


Rename UseSmartHeuristic to SmartHeuristic. The word Use makes it sound like Heuristic will be used, but this is dependent on the Actuator setting "Force Heuristic".
Ideally, the UseSmartHeuristic checkbox should be next to Force Heuristic in the Actuator.
Also, I think it might confuse users that we now have 3 different places for the Heuristic:
Behavior Parameters (behavior type); Match3Agent(UseSmartHeuristic); and Actuator (ForceHeurisitic)
Not sure what can be done though...

Changed to an enum for Greedy vs RandomValidMove, and updated some GameObject and behavior names.

Ideally, the UseSmartHeuristic checkbox should be next to Force Heuristic in the Actuator.

Maybe for this specific case, but Greedy vs RandomValidMove is specific to the Match3Agent

vincentpierre · 2020-11-03T18:08:33Z

com.unity.ml-agents.extensions/Documentation~/Match3.md

@@ -0,0 +1,67 @@
+# Match-3 Game Support


This document is not linked anywhere (not that I could see). I think we can link it from the examples page or from the com.unity.ml-agents.extensions.md page in the contents section

Added links from the extensions readme

Chris Elion added 29 commits August 23, 2020 17:00

first pass - matching and gravity work

1f010e6

valid moves WIP

07ca4d9

Merge remote-tracking branch 'origin/master' into match3-example

aad55ef

Merge remote-tracking branch 'origin/master' into match3-example

6059340

move enumeration, simple checking

4447cca

move detection

bb8015c

fix up/down directions

84e3bca

actuator and heuristic

18e8987

sensor

4494c9c

reset if no moves

436ae95

add masking, retrain

7c2c633

cleanup

a85ef3c

Merge remote-tracking branch 'origin/master' into match3-example

8f4c280

config update

b67c532

optionally force heuristic mode

de3c999

Merge remote-tracking branch 'origin/master' into match3-example

3501d6a

compressed observations, reset board if no moves

1b4d9a5

TFModel meta files

5a4e6ea

WIP abstract

f9708cc

random move heuristic

812e8ba

rename Move fields, migrate IsValid logic

66ec74e

comments

c595b22

reduce dependency on Match3Agent

a16b8b0

abstractBoard as monobehavior

2473f20

move actuator to extensions

df6d9d8

move sensor to extensions

3d0cda8

update components in scene

8d490f3

undo accidental checkins

c74dfd2

drawing cleanup, custom maxMoves logic

478cbdf

chriselion requested a review from vincentpierre September 26, 2020 00:10

Chris Elion added 9 commits October 8, 2020 11:37

fix flip for compressed obs

d35db80

save dimensions on sensor

8943ffd

save dimension on actuator

f212120

Merge remote-tracking branch 'origin/master' into match3-example

d02830f

Match3 example - special pieces (#4575)

1a6444d

* WIP special pieces * add special piece types to obs (need more from master first) * update sensors and models * docstrings

more interesting scoring, smarter heuristic

a41abe1

more steps

5a51e4f

longer trained models

ef29e3a

handle no valid moves more gracefully (#4598)

5d91ccb

* handle no valid moves more gracefully * rename callback

chriselion commented Oct 23, 2020

View reviewed changes

com.unity.ml-agents.extensions/Runtime/Match3/Match3ActuatorComponent.cs Show resolved Hide resolved

chriselion commented Oct 23, 2020

View reviewed changes

com.unity.ml-agents.extensions/Runtime/Match3/Match3ActuatorComponent.cs Outdated Show resolved Hide resolved

Chris Elion added 5 commits October 23, 2020 15:38

cleanup heuristic logic

d842556

more sensor and actuator tests

4840c80

update heuristic configs

1c5a911

docstrings and add to environment docs

dfbd2d5

sensor and actuator names

bcd051e

chriselion changed the title ~~[WIP] Match3 example~~ Match3 example Oct 30, 2020

chriselion requested a review from vincentpierre October 30, 2020 21:42

compare compressed obs against saved PNGs

8a7df81

vincentpierre reviewed Nov 2, 2020

View reviewed changes

vincentpierre self-requested a review November 2, 2020 17:16

Chris Elion added 3 commits November 2, 2020 10:01

Merge remote-tracking branch 'origin/master' into match3-example

7ecca9a

review feedback, cleanup TODOs, comments

f5b69ac

docstrings, heuristic benchmarks

f2c7ce6

chriselion commented Nov 2, 2020

View reviewed changes

Project/Assets/ML-Agents/Examples/Match3/Scripts/Match3Agent.cs Show resolved Hide resolved

vincentpierre approved these changes Nov 3, 2020

View reviewed changes

rename heuristic logic in match3 agent, doc links

492fee8

chriselion merged commit f7ef326 into master Nov 5, 2020

chriselion deleted the match3-example branch May 25, 2021 18:13

github-actions bot locked as resolved and limited conversation to collaborators May 25, 2022

	/// Check if the "half" of a move is matches 3 or more.
	/// Check if the "half" of a move matches 3 or more.

Match3 example #4515

Match3 example #4515

Uh oh!

Conversation

chriselion commented Sep 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed change(s)

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

Types of change(s)

Checklist

Other comments

Uh oh!

Uh oh!

Uh oh!

vincentpierre left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chriselion commented Sep 25, 2020 •

edited

Loading