Merge remote-tracking branch 'intel_llvm/sycl' into public_vklochkov_mem_leaks3

Author: v-klochkov

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,29 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: bug
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Please describe the steps to reproduce the behavior:
+
+1. Include code snippet as short as possible
+2. Specify the command which should be used to compile the program
+3. Specify the comment which should be used to launch the program
+4. Indicate what is wrong and what was expected
+
+**Environment (please complete the following information):**
+
+- OS: [e.g Windows/Linux]
+- Target device and vendor: [e.g. Intel GPU]
+- DPC++ version: [e.g. commit hash or output of `clang++ --version`]
+- Dependencies version: [e.g. low-level runtime versions (like NEO 20.04)]
+
+**Additional context**
+Add any other context about the problem here.

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,4 @@
+contact_links:
+  - name: Ask community a question
+    url: https://github.com/intel/llvm/discussions/categories/q-a
+    about: Please use Q&A Discussions category instead of Issues to ask questions

.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,23 @@
+---
+name: Feature request
+about: Suggest an idea/improvement for this project
+title: ''
+labels: enhancement
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe**
+A clear and concise description of what the problem is.
+Include reproducer or code/pseudo-code example.
+Include specific environment details where problem occurs.
+
+**Describe the solution you would like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you have considered**
+A clear and concise description of any alternative solutions or features you
+have considered.
+
+**Additional context**
+Add any other context about the feature request here.

.github/workflows/linux_post_commit.yml

@@ -29,10 +29,10 @@ jobs:
                  wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
                  sudo add-apt-repository "deb http://apt.llvm.org/bionic/   llvm-toolchain-bionic main"
                  sudo apt-get update
-                 sudo apt-get install -y clang-12
+                 sudo apt-get install -y clang-13
                  export ARGS="--shared-libs"
-                 export CC="clang-12"
-                 export CXX="clang++-12"
+                 export CC="clang-13"
+                 export CXX="clang++-13"
                  ;;
              NoAssertions)
                  export ARGS="--no-assertions"

CONTRIBUTING.md

@@ -54,6 +54,63 @@ see [ABI Policy Guide](sycl/doc/ABIPolicyGuide.md) for more information.
 - Run regression tests -
 [instructions](sycl/doc/GetStartedGuide.md#test-dpc-toolchain).
 
+### Tests development
+
+Every product change should be accompanied with corresponding test modification
+(adding new test(s), extending, removing or modifying existing test(s)).
+
+There are 3 types of tests which are used for DPC++ toolchain validation:
+* DPC++ in-tree LIT tests including [check-llvm](../../llvm/test),
+[check-clang](../../clang/test), [check-llvm-spirv](../../llvm-spirv/test) and
+[check-sycl](../../sycl/test) targets stored in this repository. These tests
+should not have hardware (e.g. GPU, FPGA, etc.) or external software
+dependencies (e.g. OpenCL, Level Zero, CUDA runtimes). All tests not following
+this approach should be moved to DPC++ end-to-end or SYCL-CTS tests.
+However, the tests for a feature under active development requiring atomic
+change for tests and product can be put to
+[sycl/test/on-device](../../sycl/test/on-device) temporarily. It is developer
+responsibility to move the tests to DPC++ E2E test suite or SYCL-CTS once
+the feature is stabilized.
+
+    **Guidelines for adding DPC++ in-tree LIT tests (DPC++ Clang FE tests)**:
+    - Use `sycl::` namespace instead of `cl::sycl::`
+
+    - Include sycl mock headers as system headers.
+    Example: `-internal-isystem %S/Inputs`
+    `#include "sycl.hpp"`
+
+    - Use SYCL functions for invoking kernels from the mock header `(single_task, parallel_for, parallel_for_work_group)`
+    Example:
+    ```bash
+    `#include "Inputs/sycl.hpp"`
+    sycl::queue q;
+    q.submit([&](cl::sycl::handler &h) {
+    h.single_task( { //code });
+    });
+    ```
+
+    - Add a helpful comment describing what the test does at the beginning and other comments throughout the test as necessary.
+
+    - Try to follow descriptive naming convention for variables, functions as much as possible.
+    Please refer [LLVM naming convention](https://llvm.org/docs/CodingStandards.html#name-types-functions-variables-and-enumerators-properly)
+
+* DPC++ end-to-end (E2E) tests which are extension to
+[LLVM\* test suite](https://github.com/intel/llvm-test-suite/tree/intel/SYCL).
+A test which requires full stack including backend runtimes (e.g. OpenCL,
+Level Zero or CUDA) should be put to DPC++ E2E test suite following
+[CONTRIBUTING](https://github.com/intel/llvm-test-suite/blob/intel/CONTRIBUTING.md).
+
+* SYCL-CTS are official 
+[Khronos\* SYCL\* conformance tests](https://github.com/KhronosGroup/SYCL-CTS).
+They verify SYCL specification compatibility. All implementation details or
+extensions are out of scope for the tests. If SYCL specification has changed
+(SYCL CTS tests conflict with recent version of SYCL specification) or change
+is required in the way the tests are built with DPC++ compiler (defined in
+[FindIntel_SYCL](https://github.com/KhronosGroup/SYCL-CTS/blob/SYCL-1.2.1/master/cmake/FindIntel_SYCL.cmake))
+pull request should be created under
+[KhronosGroup/SYCL-CTS](https://github.com/KhronosGroup/SYCL-CTS) with required
+patch.
+
 ### Commit message
 
 - When writing your commit message, please make sure to follow

buildbot/dependency.conf

@@ -4,11 +4,11 @@ ocl_cpu_rt_ver=2020.11.11.0.04
 # https://github.com/intel/llvm/releases/download/2020-WW45/win-oclcpuexp-2020.11.11.0.04_rel.zip
 ocl_cpu_rt_ver_win=2020.11.11.0.04
 # Same GPU driver supports Level Zero and OpenCL
-# https://github.com/intel/compute-runtime/releases/tag/21.03.18857
-ocl_gpu_rt_ver=21.03.18857
+# https://github.com/intel/compute-runtime/releases/tag/21.04.18912
+ocl_gpu_rt_ver=21.04.18912
 # Same GPU driver supports Level Zero and OpenCL
-# https://downloadmirror.intel.com/30066/a08/igfx_win10_100.9030.zip
-ocl_gpu_rt_ver_win=27.20.100.9030
+# https://downloadmirror.intel.com/30148/a08/igfx_win10_100.9168.zip
+ocl_gpu_rt_ver_win=27.20.100.9168
 intel_sycl_ver=build
 
 # TBB binaries can be built from sources following instructions under
@@ -25,13 +25,13 @@ ocl_fpga_emu_ver=2020.11.11.0.04
 ocl_fpga_emu_ver_win=2020.11.11.0.04
 fpga_ver=20201021_000005
 fpga_ver_win=20201022_000005
-ocloc_ver_win=27.20.100.8935
+ocloc_ver_win=27.20.100.9168
 
 [DRIVER VERSIONS]
 cpu_driver_lin=2020.11.11.0.04
 cpu_driver_win=2020.11.11.0.04
-gpu_driver_lin=21.03.18857
-gpu_driver_win=27.20.100.9030
+gpu_driver_lin=21.04.18912
+gpu_driver_win=27.20.100.9168
 fpga_driver_lin=2020.11.11.0.04
 fpga_driver_win=2020.11.11.0.04
 # NVidia CUDA driver

clang-tools-extra/clang-query/Query.cpp

@@ -156,8 +156,7 @@ bool MatchQuery::run(llvm::raw_ostream &OS, QuerySession &QS) const {
         if (QS.DetailedASTOutput) {
           OS << "Binding for \"" << BI->first << "\":\n";
           const ASTContext &Ctx = AST->getASTContext();
-          const SourceManager &SM = Ctx.getSourceManager();
-          ASTDumper Dumper(OS, Ctx, SM.getDiagnostics().getShowColors());
+          ASTDumper Dumper(OS, Ctx, AST->getDiagnostics().getShowColors());
           Dumper.SetTraversalKind(QS.TK);
           Dumper.Visit(BI->second);
           OS << "\n";

clang-tools-extra/clang-query/tool/ClangQuery.cpp

@@ -49,6 +49,14 @@ using namespace llvm;
 static cl::extrahelp CommonHelp(CommonOptionsParser::HelpMessage);
 static cl::OptionCategory ClangQueryCategory("clang-query options");
 
+static cl::opt<bool>
+    UseColor("use-color",
+             cl::desc(
+                 R"(Use colors in detailed AST output. If not set, colors
+will be used if the terminal connected to
+standard output supports colors.)"),
+             cl::init(false), cl::cat(ClangQueryCategory));
+
 static cl::list<std::string> Commands("c", cl::desc("Specify command to run"),
                                       cl::value_desc("command"),
                                       cl::cat(ClangQueryCategory));
@@ -109,6 +117,19 @@ int main(int argc, const char **argv) {
 
   ClangTool Tool(OptionsParser->getCompilations(),
                  OptionsParser->getSourcePathList());
+
+  if (UseColor.getNumOccurrences() > 0) {
+    ArgumentsAdjuster colorAdjustor = [](const CommandLineArguments &Args, StringRef /*unused*/) {
+      CommandLineArguments AdjustedArgs = Args;
+      if (UseColor)
+        AdjustedArgs.push_back("-fdiagnostics-color");
+      else
+        AdjustedArgs.push_back("-fno-diagnostics-color");
+      return AdjustedArgs;
+    };
+    Tool.appendArgumentsAdjuster(colorAdjustor);
+  }
+
   std::vector<std::unique_ptr<ASTUnit>> ASTs;
   int ASTStatus = 0;
   switch (Tool.buildASTs(ASTs)) {

clang-tools-extra/clangd/ClangdLSPServer.cpp

@@ -258,6 +258,15 @@ class ClangdLSPServer::MessageHandler : public Transport::MessageHandler {
     };
   }
 
+  template <typename Result>
+  void bind(const char *Method,
+            void (ClangdLSPServer::*Handler)(Callback<Result>)) {
+    Calls[Method] = [Handler, this](llvm::json::Value RawParams,
+                                    ReplyOnce Reply) {
+      (Server.*Handler)(std::move(Reply));
+    };
+  }
+
   // Bind a reply callback to a request. The callback will be invoked when
   // clangd receives the reply from the LSP client.
   // Return a call id of the request.
@@ -301,6 +310,12 @@ class ClangdLSPServer::MessageHandler : public Transport::MessageHandler {
     };
   }
 
+  void bind(const char *Method, void (ClangdLSPServer::*Handler)()) {
+    Notifications[Method] = [Handler, this](llvm::json::Value RawParams) {
+      (Server.*Handler)();
+    };
+  }
+
 private:
   // Function object to reply to an LSP call.
   // Each instance must be called exactly once, otherwise:
@@ -443,6 +458,14 @@ class ClangdLSPServer::MessageHandler : public Transport::MessageHandler {
 };
 constexpr int ClangdLSPServer::MessageHandler::MaxReplayCallbacks;
 
+template <>
+void ClangdLSPServer::MessageHandler::bind<NoParams>(
+    const char *Method, void (ClangdLSPServer::*Handler)(const NoParams &)) {
+  Notifications[Method] = [Handler, this](llvm::json::Value RawParams) {
+    (Server.*Handler)(NoParams{});
+  };
+}
+
 // call(), notify(), and reply() wrap the Transport, adding logging and locking.
 void ClangdLSPServer::callRaw(StringRef Method, llvm::json::Value Params,
                               Callback<llvm::json::Value> CB) {
@@ -487,6 +510,11 @@ void ClangdLSPServer::onInitialize(const InitializeParams &Params,
         "semanticTokens request, choosing the latter (no notifications).");
     Opts.TheiaSemanticHighlighting = false;
   }
+  if (Opts.TheiaSemanticHighlighting) {
+    log("Using legacy semanticHighlights notification, which will be removed "
+        "in clangd 13. Clients should use the standard semanticTokens "
+        "request instead.");
+  }
 
   if (Params.rootUri && *Params.rootUri)
     Opts.WorkspaceRoot = std::string(Params.rootUri->file());
@@ -500,6 +528,7 @@ void ClangdLSPServer::onInitialize(const InitializeParams &Params,
   if (Opts.UseDirBasedCDB) {
     DirectoryBasedGlobalCompilationDatabase::Options CDBOpts(TFS);
     CDBOpts.CompileCommandsDir = Opts.CompileCommandsDir;
+    CDBOpts.ContextProvider = Opts.ContextProvider;
     BaseCDB =
         std::make_unique<DirectoryBasedGlobalCompilationDatabase>(CDBOpts);
     BaseCDB = getQueryDriverDatabase(llvm::makeArrayRef(Opts.QueryDriverGlobs),
@@ -647,17 +676,15 @@ void ClangdLSPServer::onInitialize(const InitializeParams &Params,
 
 void ClangdLSPServer::onInitialized(const InitializedParams &Params) {}
 
-void ClangdLSPServer::onShutdown(const ShutdownParams &Params,
-                                 Callback<std::nullptr_t> Reply) {
+void ClangdLSPServer::onShutdown(Callback<std::nullptr_t> Reply) {
   // Do essentially nothing, just say we're ready to exit.
   ShutdownRequestReceived = true;
   Reply(nullptr);
 }
 
 // sync is a clangd extension: it blocks until all background work completes.
 // It blocks the calling thread, so no messages are processed until it returns!
-void ClangdLSPServer::onSync(const NoParams &Params,
-                             Callback<std::nullptr_t> Reply) {
+void ClangdLSPServer::onSync(Callback<std::nullptr_t> Reply) {
   if (Server->blockUntilIdleForTest(/*TimeoutSeconds=*/60))
     Reply(nullptr);
   else
@@ -1445,8 +1472,7 @@ void ClangdLSPServer::onSemanticTokensDelta(
       });
 }
 
-void ClangdLSPServer::onMemoryUsage(const NoParams &,
-                                    Callback<MemoryTree> Reply) {
+void ClangdLSPServer::onMemoryUsage(Callback<MemoryTree> Reply) {
   llvm::BumpPtrAllocator DetailAlloc;
   MemoryTree MT(&DetailAlloc);
   profile(MT);

clang-tools-extra/clangd/ClangdLSPServer.h

@@ -93,8 +93,8 @@ class ClangdLSPServer : private ClangdServer::Callbacks {
   // Calls have signature void(const Params&, Callback<Response>).
   void onInitialize(const InitializeParams &, Callback<llvm::json::Value>);
   void onInitialized(const InitializedParams &);
-  void onShutdown(const ShutdownParams &, Callback<std::nullptr_t>);
-  void onSync(const NoParams &, Callback<std::nullptr_t>);
+  void onShutdown(Callback<std::nullptr_t>);
+  void onSync(Callback<std::nullptr_t>);
   void onDocumentDidOpen(const DidOpenTextDocumentParams &);
   void onDocumentDidChange(const DidChangeTextDocumentParams &);
   void onDocumentDidClose(const DidCloseTextDocumentParams &);
@@ -161,7 +161,7 @@ class ClangdLSPServer : private ClangdServer::Callbacks {
                              Callback<SemanticTokensOrDelta>);
   /// This is a clangd extension. Provides a json tree representing memory usage
   /// hierarchy.
-  void onMemoryUsage(const NoParams &, Callback<MemoryTree>);
+  void onMemoryUsage(Callback<MemoryTree>);
 
   std::vector<Fix> getFixes(StringRef File, const clangd::Diagnostic &D);
 

clang-tools-extra/clangd/ClangdServer.cpp

@@ -407,7 +407,7 @@ void ClangdServer::formatOnType(PathRef File, llvm::StringRef Code,
       Result.push_back(replacementToEdit(Code, R));
     return CB(Result);
   };
-  WorkScheduler.run("FormatOnType", File, std::move(Action));
+  WorkScheduler.runQuick("FormatOnType", File, std::move(Action));
 }
 
 void ClangdServer::prepareRename(PathRef File, Position Pos,
@@ -635,7 +635,7 @@ void ClangdServer::formatCode(PathRef File, llvm::StringRef Code,
         tooling::calculateRangesAfterReplacements(IncludeReplaces, Ranges),
         File)));
   };
-  WorkScheduler.run("Format", File, std::move(Action));
+  WorkScheduler.runQuick("Format", File, std::move(Action));
 }
 
 void ClangdServer::findDocumentHighlights(

clang-tools-extra/clangd/CodeComplete.cpp

@@ -1123,7 +1123,9 @@ bool semaCodeComplete(std::unique_ptr<CodeCompleteConsumer> Consumer,
   // skip all includes in this case; these completions are really simple.
   PreambleBounds PreambleRegion =
       ComputePreambleBounds(*CI->getLangOpts(), *ContentsBuffer, 0);
-  bool CompletingInPreamble = PreambleRegion.Size > Input.Offset;
+  bool CompletingInPreamble = Input.Offset < PreambleRegion.Size ||
+                              (!PreambleRegion.PreambleEndsAtStartOfLine &&
+                               Input.Offset == PreambleRegion.Size);
   if (Input.Patch)
     Input.Patch->apply(*CI);
   // NOTE: we must call BeginSourceFile after prepareCompilerInstance. Otherwise