docs: explain language insights and config mechanisms (#280)

This PR updates the documentation as a follow-up to
https://github.com/sourcegraph/sourcegraph/pull/62011 where we
introduced new environment variables and improved the performance of
language stats insights.

Author: bahrmichael

docs/code_insights/explanations/administration_and_security_of_code_insights.mdx

@@ -59,3 +59,39 @@ The following setting(s) apply to adding new data to a previously backfilled Cod
 
 The following setting(s) apply to both backfilling data and adding new data
 - `insights.query.worker.rateLimit` - Maximum number of Code Insights queries initiated per second on a worker node.
+
+## Language Stats Performance Configuration (>= May 2024 Release)
+
+To create language stats for a repository the Sourcegraph instance analyzes all files from the target repository. Depending
+on the repository's size this can take from a few seconds to multiple minutes. If you need to analyze a repository that's
+larger than 10GB feel free to reach out to Sourcegraph support.
+
+If a query completes the result is stored in an API-level cache and subsequent queries complete within one second. If the query
+does not complete in time, an internal cache is stays partially populated. This cache will be reused on subsequent queries which
+then have less work to do and may resolve faster.
+
+With the May 2024 Release we increased the default number of concurrent requests to the gitserver from 1 to 4, and raised
+the timeout for each language stats query from 3 minutes to 5 minutes.
+
+### Concurrent Requests
+
+This concurrent requests to the gitserver are configurable through the `GET_INVENTORY_GIT_SERVER_CONCURRENCY` environment variable ([#62011](https://github.com/sourcegraph/sourcegraph/pull/62011)).
+We recommend increasing this carefully, as an increase in concurrency may cause the gitserver to become overloaded and slow down responses.
+
+Example:
+
+```
+GET_INVENTORY_GIT_SERVER_CONCURRENCY=4
+```
+
+To understand how this configuration impacts your language stats queries you can use [tracing](/admin/observability/tracing).
+
+### Language Stats Timeout
+
+The timeout in minutes for language stats queries is configurable through the `GET_INVENTORY_TIMEOUT` environment variable ([#62011](https://github.com/sourcegraph/sourcegraph/pull/62011)).
+
+Example:
+
+```
+GET_INVENTORY_TIMEOUT=5
+```

docs/code_insights/explanations/current_limitations_of_code_insights.mdx

@@ -24,10 +24,21 @@ To accurately return historical data for insights running over many repositories
 
 The number of insights you have does not affect the overall speed at which they run: it will take the same total time to run all of them whether or not you let each one finish before creating the next one. As of version 4.4.0 Insights prioritize completing the backfills for the insights that will complete the fastest.  In general this means that insights over many repositories will pause to allow insights over a few repositories to complete.
 
+## Creating language insights for a very large repository
+
+> NOTE: This applies to Sourcegraph versions greater than `5.3`
+
+Similar to [insights in general](#creating-insights-over-very-large-repositories), creating a language insight over a very large repository can be slow.
+
+Language insights become faster as the internal cache populates, but depending on your Sourcegraph instance and repository size this may take a few attempts.
+
+By default the dashboard attempts three queries that take up to 5 minutes. It will automatically retry until the three attempts are exhausted.
+
+Apart from waiting and retrying you may also reach out to your Sourcegraph administrator to [increase the number of concurrent queries or increase the timeout for the query](/code_insights/explanations/administration_and_security_of_code_insights).
 
 ## Creating insights over very large repositories
 
-> NOTE: The feature applies on Sourcegraph version graeter than `3.42`
+> NOTE: The feature applies on Sourcegraph version greater than `3.42`
 
 In some cases, depending on the size of the Sourcegraph instance and the size of the repo, you may see odd behavior or timeout errors if you try to create a code insight running over a single large repository. In this case, it's best to try:
 

docs/code_insights/how-tos/Troubleshooting.mdx

@@ -23,16 +23,16 @@ scale may be responsible.
 3. (admin-only) Check the queries currently in background processing using the GraphQL query
 
    ``` gql 
-      query seriesStatus {
+     query seriesStatus {
       insightSeriesQueryStatus {
-      seriesId
-      query
-      enabled
-      completed
-      errored
-      processing
-      failed
-      queued
+       seriesId
+       query
+       enabled
+       completed
+       errored
+       processing
+       failed
+       queued
       }
     }
    ```

docs/code_insights/language_insight_quickstart.mdx

@@ -24,7 +24,7 @@ If you are more interested tracking the historical or future result count of an
 
 ### 3. Once on the "Set up new language usage insight" form fields page, enter the repository you want to analyze.
 
-Enter repositories in the repository URL format, like `github.com/Sourcegraph/Sourcegraph`.
+Enter repositories in the repository URL format, like `github.com/sourcegraph/docs`. We recommend a small repository so that you get a quick result.
 
 The form field will validate that you've entered the repository correctly.
 

docs/versioned/5.2/code_insights/how-tos/Troubleshooting.mdx

@@ -23,16 +23,16 @@ scale may be responsible.
 3. (admin-only) Check the queries currently in background processing using the GraphQL query
 
    ``` gql 
-      query seriesStatus {
+     query seriesStatus {
       insightSeriesQueryStatus {
-      seriesId
-      query
-      enabled
-      completed
-      errored
-      processing
-      failed
-      queued
+       seriesId
+       query
+       enabled
+       completed
+       errored
+       processing
+       failed
+       queued
       }
     }
    ```