feat: Add support for using static discovery documents (#1109)

* feat: Add support for static discovery documents

* Auto generated docs should use static artifacts

Author: parthea

describe.py

@@ -37,6 +37,7 @@
 from googleapiclient.discovery import build
 from googleapiclient.discovery import build_from_document
 from googleapiclient.discovery import UnknownApiNameOrVersion
+from googleapiclient.discovery_cache import get_static_doc
 from googleapiclient.http import build_http
 from googleapiclient.errors import HttpError
 
@@ -395,19 +396,14 @@ def document_api(name, version, uri):
   """
     try:
         service = build(name, version)
+        content = get_static_doc(name, version)
     except UnknownApiNameOrVersion as e:
         print("Warning: {} {} found but could not be built.".format(name, version))
         return
     except HttpError as e:
         print("Warning: {} {} returned {}.".format(name, version, e))
         return
 
-    http = build_http()
-    response, content = http.request(
-        uri or uritemplate.expand(
-            FLAGS.discovery_uri_template, {"api": name, "apiVersion": version}
-        )
-    )
     discovery = json.loads(content)
 
     version = safe_version(version)

docs/start.md

@@ -19,24 +19,24 @@ It is important to understand the basics of how API authentication and authoriza
 These API calls do not access any private user data. Your application must authenticate itself as an application belonging to your Google Cloud project. This is needed to measure project usage for accounting purposes.
 
 **API key**: To authenticate your application, use an [API key](https://cloud.google.com/docs/authentication/api-keys) for your Google Cloud Console project. Every simple access call your application makes must include this key.
-    
+
 > **Warning**: Keep your API key private. If someone obtains your key, they could use it to consume your quota or incur charges against your Google Cloud project.
-    
+
 ### 2. Authorized API access (OAuth 2.0)
 
 These API calls access private user data. Before you can call them, the user that has access to the private data must grant your application access. Therefore, your application must be authenticated, the user must grant access for your application, and the user must be authenticated in order to grant that access. All of this is accomplished with [OAuth 2.0](https://developers.google.com/identity/protocols/OAuth2) and libraries written for it.
 
 *   **Scope**: Each API defines one or more scopes that declare a set of operations permitted. For example, an API might have read-only and read-write scopes. When your application requests access to user data, the request must include one or more scopes. The user needs to approve the scope of access your application is requesting. A list of accessible OAuth 2.0 scopes can be [found here](https://developers.google.com/identity/protocols/oauth2/scopes).
 *   **Refresh and access tokens**: When a user grants your application access, the OAuth 2.0 authorization server provides your application with refresh and access tokens. These tokens are only valid for the scope requested. Your application uses access tokens to authorize API calls. Access tokens expire, but refresh tokens do not. Your application can use a refresh token to acquire a new access token.
-    
+
     > **Warning**: Keep refresh and access tokens private. If someone obtains your tokens, they could use them to access private user data.
-    
+
 *   **Client ID and client secret**: These strings uniquely identify your application and are used to acquire tokens. They are created for your Google Cloud project on the [API Access pane](https://console.developers.google.com/apis/credentials) of the Google Cloud. There are several types of client IDs, so be sure to get the correct type for your application:
-    
+
     *   Web application client IDs
     *   Installed application client IDs
     *   [Service Account](https://developers.google.com/identity/protocols/OAuth2ServiceAccount) client IDs
-    
+
     > **Warning**: Keep your client secret private. If someone obtains your client secret, they could use it to consume your quota, incur charges against your Google Cloud project, and request access to user data.
 
 ## Building and calling a service
@@ -45,7 +45,7 @@ This section describes how to build an API-specific service object, make calls t
 
 ### Build the service object
 
-Whether you are using simple or authorized API access, you use the [build()](http://googleapis.github.io/google-api-python-client/docs/epy/googleapiclient.discovery-module.html#build) function to create a service object. It takes an API name and API version as arguments. You can see the list of all API versions on the [Supported APIs](dyn/index.md) page. The service object is constructed with methods specific to the given API. 
+Whether you are using simple or authorized API access, you use the [build()](http://googleapis.github.io/google-api-python-client/docs/epy/googleapiclient.discovery-module.html#build) function to create a service object. It takes an API name and API version as arguments. You can see the list of all API versions on the [Supported APIs](dyn/index.md) page. When `build()` is called, a service object will attempt to be constructed with methods specific to the given API.
 
 `httplib2`, the underlying transport library, makes all connections persistent by default. Use the service object with a context manager or call `close` to avoid leaving sockets open.
 
@@ -65,6 +65,8 @@ with build('drive', 'v3') as service:
     # ...
 ```
 
+**Note**: Under the hood, the `build()` function retrieves a discovery artifact in order to construct the service object.  If the `cache_discovery` argument of `build()` is set to `True`, the library will attempt to retrieve the discovery artifact from the legacy cache which is only supported with `oauth2client<4.0`. If the artifact is not available in the legacy cache and the `static_discovery` argument of `build()` is set to `True`, which is the default, the library will use the service definition shipped in the library. If always using the latest version of a service definition is more important than reliability, users should set `static_discovery=False` to retrieve the service definition from the internet.
+
 ### Collections
 
 Each API service provides access to one or more resources. A set of resources of the same type is called a collection. The names of these collections are specific to the API. The service object is constructed with a function for every collection defined by the API. If the given API has a collection named `stamps`, you create the collection object like this:

googleapiclient/discovery.py

@@ -193,6 +193,7 @@ def build(
     adc_cert_path=None,
     adc_key_path=None,
     num_retries=1,
+    static_discovery=True,
 ):
     """Construct a Resource for interacting with an API.
 
@@ -246,6 +247,8 @@ def build(
       https://google.aip.dev/auth/4114
     num_retries: Integer, number of times to retry discovery with
       randomized exponential backoff in case of intermittent/connection issues.
+    static_discovery: Boolean, whether or not to use the static discovery docs
+      included in the library.
 
   Returns:
     A Resource object with methods for interacting with the service.
@@ -271,9 +274,12 @@ def build(
                 requested_url,
                 discovery_http,
                 cache_discovery,
+                serviceName,
+                version,
                 cache,
                 developerKey,
                 num_retries=num_retries,
+                static_discovery=static_discovery,
             )
             service = build_from_document(
                 content,
@@ -330,7 +336,15 @@ def _discovery_service_uri_options(discoveryServiceUrl, version):
 
 
 def _retrieve_discovery_doc(
-    url, http, cache_discovery, cache=None, developerKey=None, num_retries=1
+    url,
+    http,
+    cache_discovery,
+    serviceName,
+    version,
+    cache=None,
+    developerKey=None,
+    num_retries=1,
+    static_discovery=True
 ):
     """Retrieves the discovery_doc from cache or the internet.
 
@@ -339,26 +353,39 @@ def _retrieve_discovery_doc(
     http: httplib2.Http, An instance of httplib2.Http or something that acts
       like it through which HTTP requests will be made.
     cache_discovery: Boolean, whether or not to cache the discovery doc.
+    serviceName: string, name of the service.
+    version: string, the version of the service.
     cache: googleapiclient.discovery_cache.base.Cache, an optional cache
       object for the discovery documents.
     developerKey: string, Key for controlling API usage, generated
       from the API Console.
     num_retries: Integer, number of times to retry discovery with
       randomized exponential backoff in case of intermittent/connection issues.
+    static_discovery: Boolean, whether or not to use the static discovery docs
+      included in the library.
 
   Returns:
     A unicode string representation of the discovery document.
   """
-    if cache_discovery:
-        from . import discovery_cache
+    from . import discovery_cache
 
+    if cache_discovery:
         if cache is None:
             cache = discovery_cache.autodetect()
         if cache:
             content = cache.get(url)
             if content:
                 return content
 
+    # When `static_discovery=True`, use static discovery artifacts included
+    # with the library
+    if static_discovery:
+        content = discovery_cache.get_static_doc(serviceName, version)
+        if content:
+            return content
+        else:
+            raise UnknownApiNameOrVersion("name: %s  version: %s" % (serviceName, version))
+
     actual_url = url
     # REMOTE_ADDR is defined by the CGI spec [RFC3875] as the environment
     # variable that contains the network address of the client sending the

googleapiclient/discovery_cache/__init__.py

@@ -23,7 +23,8 @@
 LOGGER = logging.getLogger(__name__)
 
 DISCOVERY_DOC_MAX_AGE = 60 * 60 * 24  # 1 day
-
+DISCOVERY_DOC_DIR = os.path.join(os.path.dirname(
+            os.path.realpath(__file__)), 'documents')
 
 def autodetect():
     """Detects an appropriate cache module and returns it.
@@ -48,3 +49,29 @@ def autodetect():
         LOGGER.info("file_cache is only supported with oauth2client<4.0.0",
             exc_info=False)
         return None
+
+def get_static_doc(serviceName, version):
+    """Retrieves the discovery document from the directory defined in
+    DISCOVERY_DOC_DIR corresponding to the serviceName and version provided.
+
+    Args:
+        serviceName: string, name of the service.
+        version: string, the version of the service.
+
+    Returns:
+        A string containing the contents of the JSON discovery document,
+        otherwise None if the JSON discovery document was not found.
+    """
+
+    content = None
+    doc_name = "{}.{}.json".format(serviceName, version)
+
+    try:
+        with open(os.path.join(DISCOVERY_DOC_DIR, doc_name), 'r') as f:
+            content = f.read()
+    except FileNotFoundError:
+        # File does not exist. Nothing to do here.
+        pass
+
+    return content
+