Nds/#422

.github/workflows/ci.yml

@@ -255,7 +255,7 @@ jobs:
     timeout-minutes: 30
     strategy:
       matrix:
-        feature: [AcrApi, DicomDimseScp, DicomDimseScu, DicomWebExport, DicomWebStow, HealthLevel7, Fhir]
+        feature: [AcrApi, DicomDimseScp, DicomDimseScu, DicomWebExport, DicomWebStow, HealthLevel7, Fhir, RemoteAppExecutionPlugIn]
         database: [ef, mongodb]
       fail-fast: false
     env:

.licenserc.yaml

@@ -31,6 +31,7 @@ header:
     - 'src/coverlet.runsettings'
     - 'src/Monai.Deploy.InformaticsGateway.sln'
     - 'src/Database/EntityFramework/Migrations/**'
+    - 'src/Plug-ins/RemoteAppExecution/Migrations/**'
     - 'demos/**/.env/**'
     - 'demos/**/*.txt'
     - 'doc/dependency_decisions.yml'

doc/dependency_decisions.yml

@@ -868,7 +868,7 @@
 - - :approve
   - Polly
   - :who: mocsharp
-    :why: New BSD License (https://github.com/App-vNext/Polly/raw/main/LICENSE.txt)
+    :why: New BSD License (https://raw.githubusercontent.com/App-vNext/Polly/main/LICENSE)
     :versions:
     - 7.2.4
     :when: 2022-08-16 23:06:27.913122244 Z
@@ -2324,28 +2324,28 @@
 - - :approve
   - MongoDB.Bson
   - :who: mocsharp
-    :why: Apache-2.0 (https://github.com/mongodb/mongo-csharp-driver/raw/master/License.txt)
+    :why: Apache-2.0 (https://raw.githubusercontent.com/mongodb/mongo-csharp-driver/master/LICENSE.md)
     :versions:
     - 2.21.0
     :when: 2022-11-16 23:38:53.891380809 Z
 - - :approve
   - MongoDB.Driver
   - :who: mocsharp
-    :why: Apache-2.0 (https://github.com/mongodb/mongo-csharp-driver/raw/master/License.txt)
+    :why: Apache-2.0 (https://raw.githubusercontent.com/mongodb/mongo-csharp-driver/master/LICENSE.md)
     :versions:
     - 2.21.0
     :when: 2022-11-16 23:38:54.213853364 Z
 - - :approve
   - MongoDB.Driver.Core
   - :who: mocsharp
-    :why: Apache-2.0 (https://github.com/mongodb/mongo-csharp-driver/raw/master/License.txt)
+    :why: Apache-2.0 (https://raw.githubusercontent.com/mongodb/mongo-csharp-driver/master/LICENSE.md)
     :versions:
     - 2.21.0
     :when: 2022-11-16 23:38:54.553730219 Z
 - - :approve
   - MongoDB.Libmongocrypt
   - :who: mocsharp
-    :why: Apache-2.0 (https://github.com/mongodb/mongo-csharp-driver/raw/master/License.txt)
+    :why: Apache-2.0 (https://raw.githubusercontent.com/mongodb/mongo-csharp-driver/master/LICENSE.md)
     :versions:
     - 1.8.0
     :when: 2022-11-16 23:38:54.863359236 Z

docs/changelog.md

@@ -23,6 +23,7 @@
 
 - gh-435 Fix CLI to read log dir path from NLog config file.
 - New Virtual Application Entity support for DICOMWeb STOW-RS APIs to enable dynamic endpoints.
+- New data [plug-ins](./plug-ins/overview.md) feature to manipulate incoming outgoing data.
 
 
 ## 0.3.21

docs/compliance/third-party-licenses.md

@@ -41669,4 +41669,3 @@ Data pulled from spdx/license-list-data on February 9, 2023.
 ```
 
 </details>
-

docs/plug-ins/overview.md

@@ -0,0 +1,136 @@
+<!--
+  ~ Copyright 2023 MONAI Consortium
+  ~
+  ~ Licensed under the Apache License, Version 2.0 (the "License");
+  ~ you may not use this file except in compliance with the License.
+  ~ You may obtain a copy of the License at
+  ~
+  ~ http://www.apache.org/licenses/LICENSE-2.0
+  ~
+  ~ Unless required by applicable law or agreed to in writing, software
+  ~ distributed under the License is distributed on an "AS IS" BASIS,
+  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  ~ See the License for the specific language governing permissions and
+  ~ limitations under the License.
+-->
+
+# Data Plug-ins
+
+Data plug-ins enable manipulation of incoming data before they are saved to the storage service or outgoing data right before they are exported.
+
+## Using Data Plug-ins
+
+The Informatics Gateway allows you to configure data plug-ins in the following services:
+
+-   (DIMSE) MONAI Deploy DICOM Listener: configure each listening AE Title with zero or more data plug-ins via the [CLI](../setup/cli.md) or via the [Configuration API](../api/rest/config.md).
+-   (DIMSE) DICOM Export: configure the `PluginAssemblies` with one or more data plug-ins in the [ExportRequestEvent](https://github.com/Project-MONAI/monai-deploy-messaging/blob/main/src/Messaging/Events/ExportRequestEvent.cs#L85).
+-   (DICOMWeb) STOW-RS:
+    -   The Virtual AE endpoints (`/dicomweb/vae/...`) can be configured similarly to the DICOM listener by using the [DICOMWeb STOW API](../api/rest/dicomweb-stow.md##post-dicomwebvaeaetworkflow-idstudiesstudy-instance-uid).
+    -   For the default `/dicomweb/...` endpoints, set zero or more plug-ins under `InformaticsGateway>dicomWeb>plug-ins` in the `appsettings.json` [configuration](../setup/schema.md) file.
+-   (DICOMWeb) Export: configure the `PluginAssemblies` with one or more data plug-ins in the [ExportRequestEvent](https://github.com/Project-MONAI/monai-deploy-messaging/blob/main/src/Messaging/Events/ExportRequestEvent.cs#L85).
+
+> [!Note]
+> When one or more plug-ins are defined, the plug-ins are executed in the order as they are listed.
+
+## Available Plug-ins
+
+The following plug-ins are available:
+
+| Name                                 | Description                                                                                                   | Fully Qualified Assembly Name                                                                                                              |
+| ------------------------------------ | ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
+| [DicomDeidentifier](./remote-app.md) | A plug-in that de-identifies a set of configurable DICOM tags with random data before DICOM data is exported. | `Monai.Deploy.InformaticsGateway.PlugIns.RemoteAppExecution.DicomDeidentifier, Monai.Deploy.InformaticsGateway.PlugIns.RemoteAppExecution` |
+| [DicomReidentifier](./remote-app.md) | A plug-in to be used together with the `DicomDeidentifier` plug-in to restore the original DICOM metadata.    | `Monai.Deploy.InformaticsGateway.PlugIns.RemoteAppExecution.DicomReidentifier, Monai.Deploy.InformaticsGateway.PlugIns.RemoteAppExecution` |
+
+
+## Writing Your Plug-ins
+
+To write an input data plug-in, implement the [IInputDataPlugin](xref:Monai.Deploy.InformaticsGateway.Api.PlugIns.IInputDataPlugIn) interface and 
+put the [dynamic link library](https://learn.microsoft.com/en-us/troubleshoot/windows-client/deployment/dynamic-link-library) (DLL) in 
+the `plug-ins/` directories. Similarly, for output data plug-ins, implement the [IOutputDataPlugin](xref:Monai.Deploy.InformaticsGateway.Api.PlugIns.IOutputDataPlugIn) interface.
+
+Refer to the [Configuration API](../api/rest/config.md) page to retrieve available [input](../api/rest/config.md#get-configaeplug-ins) and [output](../api/rest/config.md#get-configdestinationplug-ins) data plug-ins.
+
+
+### Database Extensions
+
+If a plug-in requires to persist data to the database, extend the [DatabaseRegistrationBase](xref:Monai.Deploy.InformaticsGateway.Database.Api.DatabaseRegistrationBase) class to register your database context and repositories.
+
+Refer to the _Monai.Deploy.InformaticsGateway.PlugIns.RemoteAppExecution_ plug-in as a reference.
+
+> [!Important]
+> The Informatics Gateway requires all plug-ins to extend both Entity Framework (sqlite) and MongoDB databases.
+
+#### Entity Framework
+
+Implement the [IDatabaseMigrationManagerForPlugIns](xref:Monai.Deploy.InformaticsGateway.Database.Api.IDatabaseMigrationManagerForPlugIns) interface to register your Entity Framework (EF) database context. A `connectionString` is provided to the `Configure(...)`
+function when you extend the [DatabaseRegistrationBase](xref:Monai.Deploy.InformaticsGateway.Database.Api.DatabaseRegistrationBase) class and allows you to create your [code-first](https://learn.microsoft.com/en-us/ef/ef6/modeling/code-first/workflows/new-database)
+EF database context and generate your migration code using [dotnet ef](https://learn.microsoft.com/en-us/ef/core/cli/dotnet) CLI tool.
+
+In the following example, we extend [DatabaseRegistrationBase](xref:Monai.Deploy.InformaticsGateway.Database.Api.DatabaseRegistrationBase) class to register a new EF database context named `RemoteAppExecutionDbContext`.
+
+In the method, we first register the database context, then register our Migration Manager by implementing the [IDatabaseMigrationManagerForPlugIns](xref:Monai.Deploy.InformaticsGateway.Database.Api.IDatabaseMigrationManagerForPlugIns) interface.
+Lastly, we register our repository for the `RemoteAppExecutions` table.
+
+```csharp
+public class DatabaseRegistrar : DatabaseRegistrationBase
+{
+    public override IServiceCollection Configure(IServiceCollection services, DatabaseType databaseType, string? connectionString)
+    {
+        Guard.Against.Null(services, nameof(services));
+
+        switch (databaseType)
+        {
+            case DatabaseType.EntityFramework:
+                Guard.Against.Null(connectionString, nameof(connectionString));
+                services.AddDbContext<EntityFramework.RemoteAppExecutionDbContext>(options => options.UseSqlite(connectionString), ServiceLifetime.Transient);
+                services.AddScoped<IDatabaseMigrationManagerForPlugIns, EntityFramework.MigrationManager>();
+
+                services.AddScoped(typeof(IRemoteAppExecutionRepository), typeof(EntityFramework.RemoteAppExecutionRepository));
+                break;
+            ...
+        }
+
+        return services;
+    }
+}
+```
+
+#### MongoDB
+
+Similar to the [Entity Framework](#entity-framework) section above, For MongoDB, we first register our Migration Manager by implementing the [IDatabaseMigrationManagerForPlugIns](xref:Monai.Deploy.InformaticsGateway.Database.Api.IDatabaseMigrationManagerForPlugIns) interface,
+and then we register our repository for the `RemoteAppExecutions` collection.
+
+```csharp
+public class DatabaseRegistrar : DatabaseRegistrationBase
+{
+    public override IServiceCollection Configure(IServiceCollection services, DatabaseType databaseType, string? connectionString)
+    {
+        Guard.Against.Null(services, nameof(services));
+
+        switch (databaseType)
+        {
+            case DatabaseType.MongoDb:
+                services.AddScoped<IDatabaseMigrationManagerForPlugIns, MongoDb.MigrationManager>();
+
+                services.AddScoped(typeof(IRemoteAppExecutionRepository), typeof(MongoDb.RemoteAppExecutionRepository));
+                break;
+            ...
+        }
+
+        return services;
+    }
+}
+```
+
+In the `MigrationManager`, we configure the `RemoteAppExecutions` collection.
+
+```csharp
+public class MigrationManager : IDatabaseMigrationManagerForPlugIns
+{
+    public IHost Migrate(IHost host)
+    {
+        RemoteAppExecutionConfiguration.Configure();
+        return host;
+    }
+}
+```

docs/plug-ins/remote-app.md

@@ -0,0 +1,61 @@
+<!--
+  ~ Copyright 2023 MONAI Consortium
+  ~
+  ~ Licensed under the Apache License, Version 2.0 (the "License");
+  ~ you may not use this file except in compliance with the License.
+  ~ You may obtain a copy of the License at
+  ~
+  ~ http://www.apache.org/licenses/LICENSE-2.0
+  ~
+  ~ Unless required by applicable law or agreed to in writing, software
+  ~ distributed under the License is distributed on an "AS IS" BASIS,
+  ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  ~ See the License for the specific language governing permissions and
+  ~ limitations under the License.
+-->
+
+# Remote App Execution Plug-ins
+
+The **Remote App Execution Plug-ins** allow the users to configure a set of DICOM metadata to be replaced with dummy data before
+being exported using the `DicomDeidentifier` plug-in. The original data is stored in the database so when the data returns
+via DICOM DIMSE or DICOMWeb, the data can be restored using the `DicomReidentifier` plug-in.
+
+## Supported Data Types
+
+-   DICOM
+
+## Configuration
+
+One or more DICOM tags may be configured in the `appsettings.json` file. For example, the following snippet will replace
+`AccessionNumber`, `StudyDescription`, and `SeriesDescription`.
+
+```json
+{
+    "InformaticsGateway": {
+        "plugins": {
+            "remoteApp": {
+                "ReplaceTags": "AccessionNumber, StudyDescription, SeriesDescription"
+            }
+        }
+    }
+}
+```
+
+Refer to [NEMA](https://dicom.nema.org/medical/dicom/current/output/chtml/part06/chapter_6.html) for a complete list of DICOM tags
+and use the value from the **Keyword** column.
+
+> [!Note]
+> `StudyInstanceUID`, `SeriesInstanceUID` and `SOPInstanceUID` are always replaced and tracked to ensure the same
+> studies and series get the same UIDs.
+
+> [!Important]
+> Only top-level DICOM metadata can be replaced at this time.
+
+## Fully Qualified Assembly Names
+
+The following plug-ins are available:
+
+| Name                | Fully Qualified Assembly Name                                                                                                              |
+| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
+| `DicomDeidentifier` | `Monai.Deploy.InformaticsGateway.PlugIns.RemoteAppExecution.DicomDeidentifier, Monai.Deploy.InformaticsGateway.PlugIns.RemoteAppExecution` |
+| `DicomReidentifier` | `Monai.Deploy.InformaticsGateway.PlugIns.RemoteAppExecution.DicomReidentifier, Monai.Deploy.InformaticsGateway.PlugIns.RemoteAppExecution` |

docs/plug-ins/toc.yml

@@ -0,0 +1,18 @@
+# Copyright 2021-2022 MONAI Consortium
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+- name: Data Plug-ins
+  href: overview.md
+- name: Remote App Execution Plug-ins
+  href: remote-app.md