Skip to content

SRS updates for handling incoming data #112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 8, 2022
Merged

SRS updates for handling incoming data #112

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified guidelines/diagrams/mig-acr.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
21 changes: 13 additions & 8 deletions guidelines/diagrams/mig-acr.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,17 @@
# SPDX-License-Identifier: Apache License 2.0

# Created at https://sequencediagram.org/
title MONAI Informatics Gateway - SCP
title MONAI Informatics Gateway - ACR API

fontawesome f0f8 "Hospital\nDevices" as HIS

participantgroup #00CC66 Informatics\nGateway
participant "ACR API" as ACR
participant "Inference Request\nQueue" as IRQ
participant "Data Retrieval Service" as DRS
participant "Payload Assembler" as PA
participant "Payload Notification" as PN
fontawesome f0a0 "Temporary Storage" as TS
participant "Payload Assembler Service" as PA
participant "Payload Notification Service" as PN
participant "Object Upload Service" as OUS
end

participantgroup #FF9966 Virtual\nStorage
Expand All @@ -38,12 +38,17 @@ DRS->DRS: Process request
activate DRS
DRS->HIS: Query & retrieve via DICOMweb
DRS<--HIS: Response
DRS->TS: Save DICOM
DRS->DRS: Extract DICOM headers
DRS->DB: Save instance info with DICOM headers
DRS->OUS: Queue DICOM+JSON for upload
activate OUS
OUS->DISK: Upload to temporary bucket path
deactivate OUS
DRS->HIS: Query & retrieve via FHIR
DRS<--HIS: Response
DRS->TS: Save FHIR
DRS->OUS: Queue FHIR for upload
activate OUS
OUS->DISK: Upload to temporary bucket path
deactivate OUS
DRS->DB: Save resource info with headers

DRS->PA: Notify data arrival
Expand All @@ -58,6 +63,6 @@ PA->PN: Queue Payload ready message
activate PN
PA<--PN: Response
deactivate PA
PN->DISK: Upload files associated with the payload
PN->DISK: Move files associated with the payload to payload directory
PN->PS: Publish new workflow request
PN<--PS: Response
Binary file modified guidelines/diagrams/mig-scp.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
16 changes: 8 additions & 8 deletions guidelines/diagrams/mig-scp.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ fontawesome f0f8 "Hospital\nDICOM Devices" as DICOM

participantgroup #00CC66 Informatics\nGateway
participant "DICOM SCP" as SCP
participant "Payload Assembler" as PA
participant "Payload Notification" as PN
fontawesome f0a0 "Temporary Storage" as TS
participant "Payload Assembler Service" as PA
participant "Payload Notification Service" as PN
participant "Object Upload Service" as OUS
end

participantgroup #FF9966 Virtual\nStorage
Expand Down Expand Up @@ -44,11 +44,11 @@ activate DICOM
SCP->DB: Query for \nconfigured\nsources
SCP<--DB: Response
SCP->SCP: Validate
SCP->TS: Save to disk
SCP->SCP: Extract DICOM headers
activate SCP
SCP->DB: Save instance info with DICOM headers
deactivate SCP
SCP->OUS: Queue for upload
activate OUS
OUS->DISK: Upload to temporary bucket path
deactivate OUS
SCP->PA: Notify data arrival
activate PA
PA->PA: Queue instance
Expand All @@ -64,7 +64,7 @@ PA->PN: Queue Payload ready message
activate PN
PA<--PN: Response
deactivate PA
PN->DISK: Upload files associated with the payload
PN->DISK: Move files associated with the payload to payload directory
PN->PS: Publish new workflow request
PN<--PS: Response
end
Binary file modified guidelines/diagrams/storage.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
19 changes: 9 additions & 10 deletions guidelines/srs.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ MIG Storage SCP service can be configured to accept all incoming association req

#### Retry Logic

(REQ-FNC-01) The system would retry the following actions upon failure.
(REQ-FNC-01) The system would retry the following actions upon failure.

| Action | Retry Delay | Maximum Retries |
| ------------ | ----------------------- | --------------- |
Expand Down Expand Up @@ -174,7 +174,7 @@ Not applicable.

#### Retry Logic

(REQ-FNC-01) The system would retry the following actions upon failure.
(REQ-FNC-01) The system would retry the following actions upon failure.

| Action | Retry Delay | Maximum Retries |
| ------ | ----------------------- | --------------- |
Expand Down Expand Up @@ -297,10 +297,9 @@ The following APIs are supported to interact with the ACR-DSI API:

![Storage](diagrams/storage.png)

MIG stores all received & retrieved medical records first in the temporary storage before the dataset is assembled (grouped) by the `Payload Assembler` based on the AE Title's configuration. Once data is assembled into a `Payload` object, MIG uploads all the files to the virtual storage device specified in the configuration file.

MIG ships with [MinIO](https://min.io/) as the default storage service provider given that it supports local, on-prem and cloud storage solutions. However, the default storage service can be easily replaced by implementing the [Monai.Deploy.InformaticsGateway.Api.Storage.IStorageService](https://github.com/Project-MONAI/monai-deploy-informatics-gateway/blob/main/src/Api/Storage/IStorageService.cs) interface. This enables users to extend MIG to meet their needs.
MIG uploads all received & retrieved medical records to the storage service before the dataset is assembled (grouped) by the `Payload Assembler` based on the AE Title's configuration. Once data is assembled into a `Payload` object, MIG issues **move** commands using the storage API.

MIG ships with [MinIO](https://min.io/) as the default storage service provider given that it supports local, on-prem and cloud storage solutions. However, the default storage service can easily be replaced by writing your own plug-in. See [MONAI Deploy Storage](https://github.com/Project-MONAI/monai-deploy-storage) project for more information. The plug-in architecture enables users to extend MIG by writing their own storage plug-ins.

#### Retry Logic

Expand All @@ -318,7 +317,7 @@ Informatics Gateway and the Workflow Manager follow the publisher-subscriber pat

![Message Broker](diagrams/message-broker.png)

MIG publishes and subscribes events to/from other subsystems using the `IMessageBrokerPublisherService` interface and `IMessageBrokerSubscriberService` interface, respectively. The default message broker service used is provided by [RabbitMQ](https://www.rabbitmq.com/).
MIG publishes and subscribes events to/from other subsystems using the `IMessageBrokerPublisherService` interface and `IMessageBrokerSubscriberService` interface, respectively. The default message broker service used is provided by [RabbitMQ](https://www.rabbitmq.com/).

With RabbitMQ, IG publishes message to the exchange (`InformaticsGateway>messaging>publisherSettings>exchange`) with different *routing keys* based on the event type listed in the following sections.

Expand All @@ -329,7 +328,7 @@ Each event is associated with a specific type and is serialized to JSON before s
- Workflow Request event
- Routing Key: `md.workflow.request`
- Class name: `WorkflowRequestMessage`

| Property | Type | Description |
| ------------- | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Bucket | string | Name of the bucket/directory name, which can be used to locate the payload on the shared storage. |
Expand All @@ -356,7 +355,7 @@ Each event is associated with a specific type and is serialized to JSON before s
- Export Complete event
- Routing Key: `md.export.complete`
- Class name: `ExportCompleteMessage`

| Property | Type | Description |
| ------------ | ------------- | ------------------------------------------------------------- |
| WorkflowId | string (UUID) | A UUID generated by the Workflow Manager. |
Expand All @@ -371,11 +370,11 @@ Each event is associated with a specific type and is serialized to JSON before s
- DIMSE SCU Export Service: `md.export.request.monaiscu`
- DICOMweb Export Service: `md.export.request.monaidicomweb`
- Class name: `ExportRequestMessage`

| Property | Type | Description |
| ------------- | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| WorkflowId | string (UUID) | A UUID generated by the Workflow Manager. |
| ExportTaskId | string (UUID) | A UUID generated by the Workflow Manager for the export task. |
| Files | string[] | A list of files to be exported. |
| Destination | string | Export destination. For DICOM C-STORE, a named DICOM destination. For ACR request, use the Transaction ID in the original request. |
| CorrelationId | string | For DIMSE, the correlation ID is the UUID associated with the first DICOM association received. For an ACR inference request, use the Transaction ID in the original request. |
| CorrelationId | string | For DIMSE, the correlation ID is the UUID associated with the first DICOM association received. For an ACR inference request, use the Transaction ID in the original request. |