Add missing exit() when shutting down

Summary:
We weren't exiting the device mesh properly
when invoking the local mesh.

(I may have to remove exit() in the testing code
will wait for the unit tests to run).

Reviewed By: andrewjcg, moonli

Differential Revision: D75143906

fbshipit-source-id: 2ebca9055e59d696d8b7243bcc80698c439bca3e

Author: vidhyav

python/monarch/rust_local_mesh.py

@@ -1279,7 +1279,15 @@ def local_meshes(
         system_factory=system_factory,
     )
     with bootstrap:
-        yield dms
+        maybe_error = None
+        try:
+            yield dms
+        except Exception as e:
+            maybe_error = e
+            raise
+        finally:
+            for dm in dms:
+                dm.exit(maybe_error)
 
 
 def local_meshes_and_bootstraps(