RayPerceptionSensorComponent3D does not function as expected when the agent's scale is not 1.

[WanNJ]

All the agents in the example scenes have a scale of 1. However, if changed the scale to other numbers, the Debug gizmo would also scale, resulting in the wrong rays drawn in the editor.

I haven't checked whether the observed values are correct yet. Presumably, it may have the same issue.

• ML-Agents v0.13.0
• TensorFlow v2.0
• Environment_: Change any of the agent's scale to not equal to 1.

To reproduce this bug, just change the scale of the agent of WallJump example and run using the trained model. You will see that the rays drawn in the editor is wrong.

[chriselion]

I'll look into this tomorrow.

Can you explain more what are you seeing and what do you expect? I would expect that when the scale changes, the length of the rays also increase.

[WanNJ]

I'll look into this tomorrow.

Can you explain more what are you seeing and what do you expect? I would expect that when the scale changes, the length of the rays also increase.

The expected: the red sphere drawn representing the hit point should be the actual hitpoint. However, when the scale changes, the red sphere is drawn to somewhere else, scaling with the scale.

I don't know if the observation feed into the RL component has the same problem or not. The below screenshot shows this bug when I change the training area's scale to 1.5.

[chriselion]

Thanks, the screenshot helps a lot. Looking into it now, I get the feeling the scale is being applied twice somehow.

[chriselion]

OK, I think I know what's going on. I believe this is just a visual bug, but it's definitely something we should fix.
When we do the raycasting here:

ml-agents/com.unity.ml-agents/Runtime/Sensor/RayPerceptionSensor.cs

Lines 219 to 228 in 4ba8b97

	castHit = Physics.SphereCast(startPositionWorld, castRadius, rayDirection, out rayHit,
	rayLength, layerMask);
	}
	else
	{
	castHit = Physics.Raycast(startPositionWorld, rayDirection, out rayHit,
	rayLength, layerMask);
	}
	hitFraction = castHit ? rayHit.distance / rayLength : 1.0f;

we cast up to rayLength units, and save the hit fraction. This fraction is based on rayLength units.

When doing the gizmo drawing here

ml-agents/com.unity.ml-agents/Runtime/Sensor/RayPerceptionSensorComponentBase.cs

Lines 127 to 128 in 4ba8b97

	var rayDirection = endPositionWorld - startPositionWorld;
	rayDirection *= rayInfo.hitFraction;

We try to use the hit fraction to interpolate between the start and end points. Since these start and end points are affected by the scale, they won't necessarily be rayLength units apart. So we'd need to scale the hit fraction (for display only) by rayLength / rayDirection.magnitude.

I think a cleaner fix would be to use the scaled ray length (rayDirection.magnitude) instead of rayLength for raycasting and hit fraction determination in PerceiveStatic. This means that the effective length of the rays would change with the scale; the downside is that this would break existing trained models [edit: would break existing models with non-1 scale].

I'll have a PR for this tomorrow or early next week. In the meantime, I'm pretty confident that training with a RayPerceptionSensor and scale will still work.

[WanNJ]

Thanks, Chris. Ml-agents is such an awesome project because of developers like you.

[chriselion]

This was fixed in #3334

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.