Merge pull request #31 from jimbobbennett/master

Updating to latest requests and miniMQTT

Author: brentru

.gitignore

@@ -14,3 +14,4 @@ bundles
 dist
 **/*.egg-info
 .vscode/settings.json
+.venv

.pylintrc

@@ -55,7 +55,7 @@ confidence=
 # no Warning level messages displayed, use"--disable=all --enable=classes
 # --disable=W"
 # disable=import-error,print-statement,parameter-unpacking,unpacking-in-except,old-raise-syntax,backtick,long-suffix,old-ne-operator,old-octal-literal,import-star-module-level,raw-checker-failed,bad-inline-option,locally-disabled,locally-enabled,file-ignored,suppressed-message,useless-suppression,deprecated-pragma,apply-builtin,basestring-builtin,buffer-builtin,cmp-builtin,coerce-builtin,execfile-builtin,file-builtin,long-builtin,raw_input-builtin,reduce-builtin,standarderror-builtin,unicode-builtin,xrange-builtin,coerce-method,delslice-method,getslice-method,setslice-method,no-absolute-import,old-division,dict-iter-method,dict-view-method,next-method-called,metaclass-assignment,indexing-exception,raising-string,reload-builtin,oct-method,hex-method,nonzero-method,cmp-method,input-builtin,round-builtin,intern-builtin,unichr-builtin,map-builtin-not-iterating,zip-builtin-not-iterating,range-builtin-not-iterating,filter-builtin-not-iterating,using-cmp-argument,eq-without-hash,div-method,idiv-method,rdiv-method,exception-message-attribute,invalid-str-codec,sys-max-int,bad-python3-import,deprecated-string-function,deprecated-str-translate-call
-disable=print-statement,parameter-unpacking,unpacking-in-except,old-raise-syntax,backtick,long-suffix,old-ne-operator,old-octal-literal,import-star-module-level,raw-checker-failed,bad-inline-option,locally-disabled,locally-enabled,file-ignored,suppressed-message,useless-suppression,deprecated-pragma,apply-builtin,basestring-builtin,buffer-builtin,cmp-builtin,coerce-builtin,execfile-builtin,file-builtin,long-builtin,raw_input-builtin,reduce-builtin,standarderror-builtin,unicode-builtin,xrange-builtin,coerce-method,delslice-method,getslice-method,setslice-method,no-absolute-import,old-division,dict-iter-method,dict-view-method,next-method-called,metaclass-assignment,indexing-exception,raising-string,reload-builtin,oct-method,hex-method,nonzero-method,cmp-method,input-builtin,round-builtin,intern-builtin,unichr-builtin,map-builtin-not-iterating,zip-builtin-not-iterating,range-builtin-not-iterating,filter-builtin-not-iterating,using-cmp-argument,eq-without-hash,div-method,idiv-method,rdiv-method,exception-message-attribute,invalid-str-codec,sys-max-int,bad-python3-import,deprecated-string-function,deprecated-str-translate-call,import-error,bad-continuation
+disable=print-statement,parameter-unpacking,unpacking-in-except,old-raise-syntax,backtick,long-suffix,old-ne-operator,old-octal-literal,import-star-module-level,raw-checker-failed,bad-inline-option,locally-disabled,locally-enabled,file-ignored,suppressed-message,useless-suppression,deprecated-pragma,apply-builtin,basestring-builtin,buffer-builtin,cmp-builtin,coerce-builtin,execfile-builtin,file-builtin,long-builtin,raw_input-builtin,reduce-builtin,standarderror-builtin,unicode-builtin,xrange-builtin,coerce-method,delslice-method,getslice-method,setslice-method,no-absolute-import,old-division,dict-iter-method,dict-view-method,next-method-called,metaclass-assignment,indexing-exception,raising-string,reload-builtin,oct-method,hex-method,nonzero-method,cmp-method,input-builtin,round-builtin,intern-builtin,unichr-builtin,map-builtin-not-iterating,zip-builtin-not-iterating,range-builtin-not-iterating,filter-builtin-not-iterating,using-cmp-argument,eq-without-hash,div-method,idiv-method,rdiv-method,exception-message-attribute,invalid-str-codec,sys-max-int,bad-python3-import,deprecated-string-function,deprecated-str-translate-call,import-error,bad-continuation,similarities
 
 # Enable the message, report, category or checker with the given id(s). You can
 # either give multiple identifier separated by comma (,) or put this option

README.rst

@@ -13,7 +13,7 @@ Adafruit_CircuitPython_AzureIoT
     :target: https://github.com/adafruit/Adafruit_CircuitPython_AzureIoT/actions/
     :alt: Build Status
 
-A CircuitPython device library for `Microsoft Azure IoT Services <https://azure.microsoft.com/overview/iot/?WT.mc_id=AdafruitCircuitPythonAzureIoT-github-jabenn>`_ from a CircuitPython device. This library only supports key-base authentication, it currently doesn't support X.509 certificates.
+A CircuitPython device library for `Microsoft Azure IoT Services <https://azure.microsoft.com/overview/iot/?WT.mc_id=academic-3168-jabenn>`_ from a CircuitPython device. This library only supports key-base authentication, it currently doesn't support X.509 certificates.
 
 Installing from PyPI
 =====================
@@ -55,7 +55,7 @@ This is easily achieved by downloading
 Usage Example
 =============
 
-This library supports both `Azure IoT Hub <https://azure.microsoft.com/services/iot-hub/?WT.mc_id=AdafruitCircuitPythonAzureIoT-github-jabenn>`_ and `Azure IoT Central <https://azure.microsoft.com/services/iot-central/?WT.mc_id=AdafruitCircuitPythonAzureIoT-github-jabenn>`__.
+This library supports both `Azure IoT Hub <https://azure.microsoft.com/services/iot-hub/?WT.mc_id=academic-3168-jabenn>`_ and `Azure IoT Central <https://azure.microsoft.com/services/iot-central/?WT.mc_id=academic-3168-jabenn>`__.
 
 To create an Azure IoT Hub instance or an Azure IoT Central app, you will need an Azure subscription. If you don't have an Azure subscription, you can sign up for free:
 
@@ -169,9 +169,9 @@ Azure IoT Central
 
 To use Azure IoT Central, you will need to create an Azure IoT Central app, create a device template and register a device against the template.
 
-- Head to `Azure IoT Central <https://apps.azureiotcentral.com/?WT.mc_id=AdafruitCircuitPythonAzureIoT-github-jabenn>`__
-- Follow the instructions in the `Microsoft Docs <https://docs.microsoft.com/azure/iot-central/core/quick-deploy-iot-central?WT.mc_id=AdafruitCircuitPythonAzureIoT-github-jabenn>`__ to create an application. Every tier is free for up to 2 devices.
-- Follow the instructions in the `Microsoft Docs <https://docs.microsoft.com/azure/iot-central/core/quick-create-simulated-device?WT.mc_id=AdafruitCircuitPythonAzureIoT-github-jabenn>`__ to create a device template.
+- Head to `Azure IoT Central <https://apps.azureiotcentral.com/?WT.mc_id=academic-3168-jabenn>`__
+- Follow the instructions in the `Microsoft Docs <https://docs.microsoft.com/azure/iot-central/core/quick-deploy-iot-central?WT.mc_id=academic-3168-jabenn>`__ to create an application. Every tier is free for up to 2 devices.
+- Follow the instructions in the `Microsoft Docs <https://docs.microsoft.com/azure/iot-central/core/quick-create-simulated-device?WT.mc_id=academic-3168-jabenn>`__ to create a device template.
 - Create a device based off the template, and select **Connect** to get the device connection details. Store the ID Scope, Device ID and either the Primary or secondary Key in your ``secrets.py`` file.
 
 .. image:: iot-central-connect-button.png
@@ -254,8 +254,8 @@ Learning more about Azure IoT services
 
 If you want to learn more about setting up or using Azure IoT Services, check out the following resources:
 
-- `Azure IoT documentation on Microsoft Docs <https://docs.microsoft.com/azure/iot-fundamentals/?WT.mc_id=AdafruitCircuitPythonAzureIoT-github-jabenn>`_
-- `IoT learning paths and modules on Microsoft Learn <https://docs.microsoft.com/learn/browse/?term=iot&WT.mc_id=AdafruitCircuitPythonAzureIoT-github-jabenn>`_ - Free, online, self-guided hands on learning with Azure IoT services
+- `Azure IoT documentation on Microsoft Docs <https://docs.microsoft.com/azure/iot-fundamentals/?WT.mc_id=academic-3168-jabenn>`_
+- `IoT learning paths and modules on Microsoft Learn <https://docs.microsoft.com/learn/browse/?term=iot&WT.mc_id=academic-3168-jabenn>`_ - Free, online, self-guided hands on learning with Azure IoT services
 
 Contributing
 ============

adafruit_azureiot/constants.py

@@ -13,10 +13,10 @@
 """
 
 # The version of the IoT Central MQTT API this code is built against
-IOTC_API_VERSION = "2016-11-14"
+IOTC_API_VERSION = "2019-10-01"
 
 # The version of the Azure Device Provisioning Service this code is built against
-DPS_API_VERSION = "2018-11-01"
+DPS_API_VERSION = "2019-03-31"
 
 # The Azure Device Provisioning service endpoint that this library uses to provision IoT Central devices
 DPS_END_POINT = "global.azure-devices-provisioning.net"

adafruit_azureiot/device_registration.py

@@ -13,19 +13,15 @@
 * Author(s): Jim Bennett, Elena Horton
 """
 
-import gc
 import json
 import time
-import adafruit_requests as requests
 import adafruit_logging as logging
 from adafruit_logging import Logger
+import adafruit_minimqtt.adafruit_minimqtt as minimqtt
 from . import constants
 from .quote import quote
 from .keys import compute_derived_symmetric_key
 
-# Azure HTTP error status codes
-AZURE_HTTP_ERROR_CODES = [400, 401, 404, 403, 412, 429, 500]
-
 
 class DeviceRegistrationError(Exception):
     """
@@ -43,23 +39,15 @@ class DeviceRegistration:
     to IoT Central over MQTT
     """
 
-    _loop_interval = 2
-
-    @staticmethod
-    def _parse_http_status(status_code: int, status_reason: str) -> None:
-        """Parses status code, throws error based on Azure IoT Common Error Codes.
-        :param int status_code: HTTP status code.
-        :param str status_reason: Description of HTTP status.
-        :raises DeviceRegistrationError: if the status code is an error code
-        """
-        for error in AZURE_HTTP_ERROR_CODES:
-            if error == status_code:
-                raise DeviceRegistrationError(
-                    "Error {0}: {1}".format(status_code, status_reason)
-                )
-
+    # pylint: disable=R0913
     def __init__(
-        self, socket, id_scope: str, device_id: str, key: str, logger: Logger = None
+        self,
+        socket,
+        iface,
+        id_scope: str,
+        device_id: str,
+        key: str,
+        logger: Logger = None,
     ):
         """Creates an instance of the device registration service
         :param socket: The network socket
@@ -73,105 +61,100 @@ def __init__(
         self._key = key
         self._logger = logger if logger is not None else logging.getLogger("log")
 
-        requests.set_socket(socket)
+        self._mqtt = None
+        self._auth_response_received = False
+        self._operation_id = None
+        self._hostname = None
 
-    def _loop_assign(self, operation_id, headers) -> str:
-        uri = "https://%s/%s/registrations/%s/operations/%s?api-version=%s" % (
-            constants.DPS_END_POINT,
-            self._id_scope,
-            self._device_id,
-            operation_id,
-            constants.DPS_API_VERSION,
+        self._socket = socket
+        self._iface = iface
+
+    # pylint: disable=W0613
+    # pylint: disable=C0103
+    def _on_connect(self, client, userdata, _, rc) -> None:
+        self._logger.info(
+            f"- device_registration :: _on_connect :: rc = {str(rc)}, userdata = {str(userdata)}"
         )
-        self._logger.info("- iotc :: _loop_assign :: " + uri)
 
-        response = self._run_get_request_with_retry(uri, headers)
+        self._auth_response_received = True
+
+    # pylint: disable=W0613
+    def _handle_dps_update(self, client, topic: str, msg: str) -> None:
+        self._logger.info(f"Received registration results on topic {topic} - {msg}")
+        message = json.loads(msg)
+
+        if topic.startswith("$dps/registrations/res/202"):
+            # Get the retry after and wait for that before responding
+            parts = str.split(topic, "retry-after=")
+            waittime = int(parts[1])
+
+            self._logger.debug(f"Retrying after {waittime}s")
 
-        try:
-            data = response.json()
-        except ValueError as error:
-            err = "ERROR: " + str(error) + " => " + str(response)
-            self._logger.error(err)
-            raise DeviceRegistrationError(err) from error
+            time.sleep(waittime)
+            self._operation_id = message["operationId"]
+        elif topic.startswith("$dps/registrations/res/200"):
+            self._hostname = message["registrationState"]["assignedHub"]
 
-        loop_try = 0
+    def _connect_to_mqtt(self) -> None:
+        self._mqtt.on_connect = self._on_connect
 
-        if data is not None and "status" in data:
-            if data["status"] == "assigning":
-                time.sleep(self._loop_interval)
-                if loop_try < 20:
-                    loop_try = loop_try + 1
-                    return self._loop_assign(operation_id, headers)
+        self._mqtt.connect()
 
-                err = "ERROR: Unable to provision the device."
-                self._logger.error(err)
-                raise DeviceRegistrationError(err)
+        self._logger.info(
+            " - device_registration :: connect :: created mqtt client. connecting.."
+        )
+        while not self._auth_response_received:
+            self._mqtt.loop()
+
+        self._logger.info(
+            f" - device_registration :: connect :: on_connect must be fired. Connected ? {str(self._mqtt.is_connected())}"
+        )
+
+        if not self._mqtt.is_connected():
+            raise DeviceRegistrationError("Cannot connect to MQTT")
 
-            if data["status"] == "assigned":
-                state = data["registrationState"]
-                return state["assignedHub"]
-        else:
-            data = str(data)
+    def _start_registration(self) -> None:
+        self._mqtt.add_topic_callback(
+            "$dps/registrations/res/#", self._handle_dps_update
+        )
+        self._mqtt.subscribe("$dps/registrations/res/#")
+
+        message = json.dumps({"registrationId": self._device_id})
 
-        err = "DPS L => " + str(data)
-        self._logger.error(err)
-        raise DeviceRegistrationError(err)
+        self._mqtt.publish(
+            f"$dps/registrations/PUT/iotdps-register/?$rid={self._device_id}", message
+        )
 
-    def _run_put_request_with_retry(self, url, body, headers):
         retry = 0
-        response = None
-
-        while True:
-            gc.collect()
-            try:
-                self._logger.debug("Trying to send...")
-                response = requests.put(url, json=body, headers=headers)
-                self._logger.debug("Sent!")
-                break
-            except RuntimeError as runtime_error:
-                self._logger.info(
-                    "Could not send data, retrying after 0.5 seconds: "
-                    + str(runtime_error)
-                )
-                retry = retry + 1
-
-                if retry >= 10:
-                    self._logger.error("Failed to send data")
-                    raise
-
-                time.sleep(0.5)
-                continue
-
-        gc.collect()
-        return response
-
-    def _run_get_request_with_retry(self, url, headers):
+
+        while self._operation_id is None and retry < 10:
+            time.sleep(1)
+            retry = retry + 1
+            self._mqtt.loop()
+
+        if self._operation_id is None:
+            raise DeviceRegistrationError(
+                "Cannot register device - no response from broker for registration result"
+            )
+
+    def _wait_for_operation(self) -> None:
+        message = json.dumps({"operationId": self._operation_id})
+        self._mqtt.publish(
+            f"$dps/registrations/GET/iotdps-get-operationstatus/?$rid={self._device_id}&operationId={self._operation_id}",
+            message,
+        )
+
         retry = 0
-        response = None
-
-        while True:
-            gc.collect()
-            try:
-                self._logger.debug("Trying to send...")
-                response = requests.get(url, headers=headers)
-                self._logger.debug("Sent!")
-                break
-            except RuntimeError as runtime_error:
-                self._logger.info(
-                    "Could not send data, retrying after 0.5 seconds: "
-                    + str(runtime_error)
-                )
-                retry = retry + 1
-
-                if retry >= 10:
-                    self._logger.error("Failed to send data")
-                    raise
-
-                time.sleep(0.5)
-                continue
-
-        gc.collect()
-        return response
+
+        while self._hostname is None and retry < 10:
+            time.sleep(1)
+            retry = retry + 1
+            self._mqtt.loop()
+
+        if self._hostname is None:
+            raise DeviceRegistrationError(
+                "Cannot register device - no response from broker for operation status"
+            )
 
     def register_device(self, expiry: int) -> str:
         """
@@ -183,65 +166,35 @@ def register_device(self, expiry: int) -> str:
         :raises DeviceRegistrationError: if the device cannot be registered successfully
         :raises RuntimeError: if the internet connection is not responding or is unable to connect
         """
+
+        username = f"{self._id_scope}/registrations/{self._device_id}/api-version={constants.DPS_API_VERSION}"
+
         # pylint: disable=C0103
         sr = self._id_scope + "%2Fregistrations%2F" + self._device_id
         sig_no_encode = compute_derived_symmetric_key(
             self._key, sr + "\n" + str(expiry)
         )
         sig_encoded = quote(sig_no_encode, "~()*!.'")
-        auth_string = (
-            "SharedAccessSignature sr="
-            + sr
-            + "&sig="
-            + sig_encoded
-            + "&se="
-            + str(expiry)
-            + "&skn=registration"
+        auth_string = f"SharedAccessSignature sr={sr}&sig={sig_encoded}&se={str(expiry)}&skn=registration"
+
+        minimqtt.set_socket(self._socket, self._iface)
+
+        self._mqtt = minimqtt.MQTT(
+            broker=constants.DPS_END_POINT,
+            username=username,
+            password=auth_string,
+            port=8883,
+            keep_alive=120,
+            is_ssl=True,
+            client_id=self._device_id,
         )
 
-        headers = {
-            "content-type": "application/json; charset=utf-8",
-            "user-agent": "iot-central-client/1.0",
-            "Accept": "*/*",
-        }
-
-        if auth_string is not None:
-            headers["authorization"] = auth_string
-
-        body = {"registrationId": self._device_id}
+        self._mqtt.enable_logger(logging, self._logger.getEffectiveLevel())
 
-        uri = "https://%s/%s/registrations/%s/register?api-version=%s" % (
-            constants.DPS_END_POINT,
-            self._id_scope,
-            self._device_id,
-            constants.DPS_API_VERSION,
-        )
-
-        self._logger.info("Connecting...")
-        self._logger.info("URL: " + uri)
-        self._logger.info("body: " + json.dumps(body))
-
-        response = self._run_put_request_with_retry(uri, body, headers)
-
-        data = None
-        try:
-            data = response.json()
-        except ValueError as error:
-            err = (
-                "ERROR: non JSON is received from "
-                + constants.DPS_END_POINT
-                + " => "
-                + str(response)
-                + " .. message : "
-                + str(error)
-            )
-            self._logger.error(err)
-            raise DeviceRegistrationError(err) from error
+        self._connect_to_mqtt()
+        self._start_registration()
+        self._wait_for_operation()
 
-        if "errorCode" in data:
-            err = "DPS => " + str(data)
-            self._logger.error(err)
-            raise DeviceRegistrationError(err)
+        self._mqtt.disconnect()
 
-        time.sleep(1)
-        return self._loop_assign(data["operationId"], headers)
+        return str(self._hostname)