Skip to content

Commit 566ba8b

Browse files
mocsharpmocsharp
authored
Stops accepting/retrieving data when disk space is low. (#194)
* gh-188 Stops accepting/retreiving data when disk space is low. - Allows users to configure watermark & reserve space similar to 0.1. - gh-188 Stop accepting DICOMweb, HL7 & FHIR when disk space is low Signed-off-by: Victor Chang <[email protected]>
1 parent 609f939 commit 566ba8b

File tree

56 files changed

+732
-74
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

56 files changed

+732
-74
lines changed

‎doc/dependency_decisions.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@
4646
- :who: mocsharp
4747
:why: MIT (https://github.com/Cysharp/ConsoleAppFramework/raw/master/LICENSE)
4848
:versions:
49-
- 4.2.2
49+
- 4.2.3
5050
:when: 2022-08-16 23:05:31.110052610 Z
5151
- - :approve
5252
- Crayon

‎docs/api/rest/dicomweb-stow.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -56,8 +56,9 @@ Response Content Type: `JSON`
5656
| 202 | [DicomDataset](https://github.com/fo-dicom/fo-dicom/blob/development/FO-DICOM.Core/DicomDataset.cs) | All instances are received and stored with warnings (e.g. for a mismatched StudyInstanceUID. |
5757
| 204 | `none` | No data is provided. |
5858
| 400 | [Problem details](https://datatracker.ietf.org/doc/html/rfc7807) | Request contains invalid values. |
59-
| 415 | `none` | Unsupported media type |
60-
| 500 | [Problem details](https://datatracker.ietf.org/doc/html/rfc7807) | Server error |
59+
| 415 | `none` | Unsupported media typ. |
60+
| 500 | [Problem details](https://datatracker.ietf.org/doc/html/rfc7807) | Server error. |
61+
| 507 | [Problem details](https://datatracker.ietf.org/doc/html/rfc7807) | Insufficient storage. |
6162

6263
---
6364

@@ -97,3 +98,4 @@ Response Content Type: `JSON`
9798
| 400 | [Problem details](https://datatracker.ietf.org/doc/html/rfc7807) | Request contains invalid values. |
9899
| 415 | `none` | Unsupported media type |
99100
| 500 | [Problem details](https://datatracker.ietf.org/doc/html/rfc7807) | Server error |
101+
| 507 | [Problem details](https://datatracker.ietf.org/doc/html/rfc7807) | Insufficient storage. |

‎docs/api/rest/fhir.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -55,9 +55,10 @@ Depending on the `Accept` header or the original document, the response supports
5555

5656
If the `Accept` header is missing or a none supported value exists, the service will return the same type as the posted document.
5757

58-
| Code | Data Type | Description |
59-
| ---- | ------------------------------------------------------------- | --------------------------------------------------------------------- |
60-
| 201 | Original JSON or XML document. | Resource created & stored successfully. |
61-
| 400 | [OperationOutcome](http://hl7.org/fhir/operationoutcome.html) | Unable to parse the resource or mismatching resource type specified.. |
62-
| 415 | `none` | Unsupported media type |
63-
| 500 | [OperationOutcome](http://hl7.org/fhir/operationoutcome.html) | Server error. |
58+
| Code | Data Type | Description |
59+
| ---- | --------------------------------------------------------------- | --------------------------------------------------------------------- |
60+
| 201 | Original JSON or XML document. | Resource created & stored successfully. |
61+
| 400 | [OperationOutcome](http://hl7.org/fhir/operationoutcome.html) | Unable to parse the resource or mismatching resource type specified.. |
62+
| 415 | `none` | Unsupported media type |
63+
| 500 | [OperationOutcome](http://hl7.org/fhir/operationoutcome.html) | Server error. |
64+
| 507 | [Problem details](https://datatracker.ietf.org/doc/html/rfc7807)| Insufficient storage. |

‎docs/compliance/third-party-licenses.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -589,14 +589,14 @@ limitations under the License.
589589

590590

591591
<details>
592-
<summary>ConsoleAppFramework 4.2.2</summary>
592+
<summary>ConsoleAppFramework 4.2.3</summary>
593593

594594
## ConsoleAppFramework
595595

596-
- Version: 4.2.2
596+
- Version: 4.2.3
597597
- Authors: Cysharp
598598
- Project URL: https://github.com/Cysharp/ConsoleAppFramework
599-
- Source: [NuGet](https://www.nuget.org/packages/ConsoleAppFramework/4.2.2)
599+
- Source: [NuGet](https://www.nuget.org/packages/ConsoleAppFramework/4.2.3)
600600
- License: [MIT](https://github.com/Cysharp/ConsoleAppFramework/raw/master/LICENSE)
601601

602602

‎docs/setup/schema.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -96,12 +96,14 @@ The `InformaticsGateway` configuration section contains the following sub-sectio
9696
}
9797
},
9898
"storage": {
99-
"bufferRootPath": "./temp",
100-
"tempStorageRootPath": "/incoming",
99+
"localTemporaryStoragePath": "/payloads",
100+
"remoteTemporaryStoragePath": "/incoming",
101101
"bucketName": "monaideploy",
102102
"storageRootPath": "/payloads",
103103
"temporaryBucketName": "monaideploy",
104104
"serviceAssemblyName": "Monai.Deploy.Storage.MinIO.MinIoStorageService, Monai.Deploy.Storage.MinIO",
105+
"watermarkPercent": 75,
106+
"reserveSpaceGB": 5,
105107
"settings": {
106108
"endpoint": "localhost:9000",
107109
"accessKey": "admin",

‎docs/setup/setup.md

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -101,9 +101,9 @@ The Informatics Gateway operates on two storage locations. In the first location
101101

102102
### Temporary Storage of Incoming Dataset
103103

104-
By default, the temporary storage location is set to `/payloads` in the `appsettings.json` file.
104+
By default, the temporary storage location is set to use `Disk` and stores any incoming files inside `/payloads`. This can be modified to user a different location, such as `Memory` or a different path.
105105

106-
To change the temporary storage location, locate the `./InformaticsGateway/storage/temporary` property in the `appsettings.json` file and modify it.
106+
To change the temporary storage path, locate the `InformaticsGateway>storage>localTemporaryStoragePath` property in the `appsettings.json` file and modify it.
107107

108108
> [!Note]
109109
> You will need to calculate the required temporary storage based on the number of studies and the size of each study.
@@ -116,7 +116,7 @@ To change the temporary storage location, locate the `./InformaticsGateway/stora
116116
> the expected number of studies and size of each study. The suggested value for `reserveSpaceGB` is 2x to 3x the
117117
> size of a single study multiplied by the number of configured AE Titles.
118118
119-
### Shared Storage
119+
### Storage Service
120120

121121
Informatics Gateway includes MinIO as the default storage service provider. To integrate with another storage service provider, please refer to the [Data Storage](https://github.com/Project-MONAI/monai-deploy-informatics-gateway/blob/main/guidelines/srs.md#data-storage) section of the SRS.
122122

@@ -138,7 +138,6 @@ Locate the storage section of the configuration in `appsettings.json`:
138138
"accessToken": "password", # Access token or password
139139
"securedConnection": false, # Indicates if connection should be secured using HTTPS
140140
"region": "local", # Region
141-
"executableLocation": "/bin/mc", # Path to minio client
142141
"serviceName": "MinIO" # Name of the service
143142
},
144143
"storageService": "Monai.Deploy.Storage.MinIO.MinIoStorageService, Monai.Deploy.Storage.MinIO", # Fully qualified type name of the storage service

‎src/Configuration/ConfigurationValidator.cs

Lines changed: 5 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ private bool IsStorageValid(StorageConfiguration storage)
8181
var valid = true;
8282
valid &= IsValidBucketName("InformaticsGateway>storage>bucketName", storage.StorageServiceBucketName);
8383
valid &= IsValidBucketName("InformaticsGateway>storage>temporaryBucketName", storage.TemporaryStorageBucket);
84-
valid &= IsNotNullOrWhiteSpace("InformaticsGateway>storage>temporary", storage.TemporaryStorageRootPath);
84+
valid &= IsNotNullOrWhiteSpace("InformaticsGateway>storage>temporary", storage.RemoteTemporaryStoragePath);
8585
valid &= IsValueInRange("InformaticsGateway>storage>watermark", 1, 100, storage.Watermark);
8686
valid &= IsValueInRange("InformaticsGateway>storage>reserveSpaceGB", 1, 999, storage.ReserveSpaceGB);
8787
valid &= IsValueInRange("InformaticsGateway>storage>payloadProcessThreads", 1, 128, storage.PayloadProcessThreads);
@@ -90,8 +90,8 @@ private bool IsStorageValid(StorageConfiguration storage)
9090

9191
if (storage.TemporaryDataStorage == TemporaryDataStorageLocation.Disk)
9292
{
93-
valid &= IsNotNullOrWhiteSpace("InformaticsGateway>storage>bufferRootPath", storage.BufferStorageRootPath);
94-
valid &= IsValidDirectory("InformaticsGateway>storage>bufferRootPath", storage.BufferStorageRootPath);
93+
valid &= IsNotNullOrWhiteSpace("InformaticsGateway>storage>localTemporaryStoragePath", storage.LocalTemporaryStoragePath);
94+
valid &= IsValidDirectory("InformaticsGateway>storage>localTemporaryStoragePath", storage.LocalTemporaryStoragePath);
9595
valid &= IsValueInRange("InformaticsGateway>storage>bufferSize", 1, int.MaxValue, storage.BufferSize);
9696
}
9797

@@ -105,13 +105,9 @@ private bool IsValidDirectory(string source, string directory)
105105
{
106106
if (!_fileSystem.Directory.Exists(directory))
107107
{
108-
valid = false;
109-
_validationErrors.Add($"Directory `{directory}` specified in `{source}` does not exist.");
110-
}
111-
else
112-
{
113-
using var _ = _fileSystem.File.Create(Path.Combine(directory, Path.GetRandomFileName()), 1, FileOptions.DeleteOnClose);
108+
_fileSystem.Directory.CreateDirectory(directory);
114109
}
110+
using var _ = _fileSystem.File.Create(Path.Combine(directory, Path.GetRandomFileName()), 1, FileOptions.DeleteOnClose);
115111
}
116112
catch (Exception ex)
117113
{

‎src/Configuration/StorageConfiguration.cs

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -33,8 +33,8 @@ public class StorageConfiguration : StorageServiceConfiguration
3333
/// Gets or sets the path used for buffering incoming data.
3434
/// Defaults to <c>./temp</c>.
3535
/// </summary>
36-
[ConfigurationKeyName("bufferRootPath")]
37-
public string BufferStorageRootPath { get; set; } = "./temp";
36+
[ConfigurationKeyName("localTemporaryStoragePath")]
37+
public string LocalTemporaryStoragePath { get; set; } = "/payloads";
3838

3939
/// <summary>
4040
/// Gets or sets the number of bytes buffered for reads and writes to the temporary file.
@@ -66,8 +66,8 @@ public class StorageConfiguration : StorageServiceConfiguration
6666
/// Gets or sets root directory path for storing incoming data in the <c>temporaryBucketName</c>.
6767
/// Defaults to <c>/incoming</c>.
6868
/// </summary>
69-
[ConfigurationKeyName("tempStorageRootPath")]
70-
public string TemporaryStorageRootPath { get; set; } = "/incoming";
69+
[ConfigurationKeyName("remoteTemporaryStoragePath")]
70+
public string RemoteTemporaryStoragePath { get; set; } = "/incoming";
7171

7272
/// <summary>
7373
/// Gets or sets the watermark for disk usage with default value of 75%,

‎src/Configuration/Test/ConfigurationValidatorTest.cs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -137,11 +137,11 @@ public void StorageWithInaccessbleDirectory()
137137

138138
var config = MockValidConfiguration();
139139
config.Storage.TemporaryDataStorage = TemporaryDataStorageLocation.Disk;
140-
config.Storage.BufferStorageRootPath = "/blabla";
140+
config.Storage.LocalTemporaryStoragePath = "/blabla";
141141

142142
var valid = new ConfigurationValidator(_logger.Object, _fileSystem.Object).Validate("", config);
143143

144-
var validationMessages = new[] { $"Directory `/blabla` specified in `InformaticsGateway>storage>bufferRootPath` is not accessible: error." };
144+
var validationMessages = new[] { $"Directory `/blabla` specified in `InformaticsGateway>storage>localTemporaryStoragePath` is not accessible: error." };
145145
Assert.Equal(string.Join(Environment.NewLine, validationMessages), valid.FailureMessage);
146146
foreach (var message in validationMessages)
147147
{

‎src/DicomWebClient/API/IDicomWebClient.cs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -74,9 +74,9 @@ public interface IDicomWebClient
7474
/// </summary>
7575
/// <param name="serviceType"><c>ServiceType</c> to be configured</param>
7676
/// <param name="urlPrefix">Url prefix</param>
77-
#pragma warning disable CA1054
77+
#pragma warning disable CA1054
7878
void ConfigureServicePrefix(DicomWebServiceType serviceType, string urlPrefix);
79-
#pragma warning restore CA1054
79+
#pragma warning restore CA1054
8080
/// <summary>
8181
/// Configures the authentication header for the DICOMweb client.
8282
/// </summary>

‎src/DicomWebClient/API/IServiceBase.cs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,8 @@ namespace Monai.Deploy.InformaticsGateway.DicomWeb.Client.API
1919
{
2020
public interface IServiceBase
2121
{
22-
#pragma warning disable CA1054
22+
#pragma warning disable CA1054
2323
bool TryConfigureServiceUriPrefix(string uriPrefix);
24-
#pragma warning restore CA1054
24+
#pragma warning restore CA1054
2525
}
2626
}

‎src/DicomWebClient/CLI/Monai.Deploy.InformaticsGateway.DicomWeb.Client.CLI.csproj

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@
3434
</PropertyGroup>
3535

3636
<ItemGroup>
37-
<PackageReference Include="ConsoleAppFramework" Version="4.2.2" />
37+
<PackageReference Include="ConsoleAppFramework" Version="4.2.3" />
3838
<PackageReference Include="fo-dicom" Version="5.0.3" />
3939
<PackageReference Include="Microsoft.Extensions.Http" Version="6.0.0" />
4040
</ItemGroup>

‎src/DicomWebClient/DicomWebClient.cs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -78,9 +78,9 @@ public void ConfigureServiceUris(Uri uriRoot)
7878
}
7979

8080
/// <inheritdoc/>
81-
#pragma warning disable CA1054
81+
#pragma warning disable CA1054
8282
public void ConfigureServicePrefix(DicomWebServiceType serviceType, string urlPrefix)
83-
#pragma warning restore CA1054
83+
#pragma warning restore CA1054
8484
{
8585
Guard.Against.NullOrWhiteSpace(urlPrefix, nameof(urlPrefix));
8686

‎src/InformaticsGateway/Common/InsufficientStorageAvailableException.cs

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
// Copyright 2021 MONAI Consortium
2+
// Licensed under the Apache License, Version 2.0 (the "License");
3+
// you may not use this file except in compliance with the License.
4+
// You may obtain a copy of the License at
5+
// http://www.apache.org/licenses/LICENSE-2.0
6+
// Unless required by applicable law or agreed to in writing, software
7+
// distributed under the License is distributed on an "AS IS" BASIS,
8+
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
9+
// See the License for the specific language governing permissions and
10+
// limitations under the License.
11+
12+
/*
13+
* Apache License, Version 2.0
14+
* Copyright 2021 NVIDIA Corporation
15+
*
16+
* Licensed under the Apache License, Version 2.0 (the "License");
17+
* you may not use this file except in compliance with the License.
18+
* You may obtain a copy of the License at
19+
*
20+
* http://www.apache.org/licenses/LICENSE-2.0
21+
*
22+
* Unless required by applicable law or agreed to in writing, software
23+
* distributed under the License is distributed on an "AS IS" BASIS,
24+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
25+
* See the License for the specific language governing permissions and
26+
* limitations under the License.
27+
*/
28+
29+
using System;
30+
31+
namespace Monai.Deploy.InformaticsGateway.Common
32+
{
33+
[Serializable]
34+
public class InsufficientStorageAvailableException : Exception
35+
{
36+
public InsufficientStorageAvailableException()
37+
{
38+
}
39+
40+
public InsufficientStorageAvailableException(string message) : base(message)
41+
{
42+
}
43+
44+
public InsufficientStorageAvailableException(string message, Exception innerException) : base(message, innerException)
45+
{
46+
}
47+
48+
protected InsufficientStorageAvailableException(System.Runtime.Serialization.SerializationInfo serializationInfo, System.Runtime.Serialization.StreamingContext streamingContext)
49+
{
50+
throw new NotImplementedException();
51+
}
52+
}
53+
}

‎src/InformaticsGateway/Logging/Log.100.200.ScpService.cs

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,5 +94,8 @@ public static partial class Log
9494

9595
[LoggerMessage(EventId = 211, Level = LogLevel.Warning, Message = "Verification service is disabled: rejecting association.")]
9696
public static partial void VerificationServiceDisabled(this ILogger logger);
97+
98+
[LoggerMessage(EventId = 212, Level = LogLevel.Error, Message = "Failed to process C-STORE request, out of storage space.")]
99+
public static partial void CStoreFailedDueToLowStorageSpace(this ILogger logger, Exception ex);
97100
}
98101
}

‎src/InformaticsGateway/Logging/Log.300.StorageInfoProvider.cs

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
/*
2+
* Copyright 2022 MONAI Consortium
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
using Microsoft.Extensions.Logging;
18+
19+
namespace Monai.Deploy.InformaticsGateway.Logging
20+
{
21+
public static partial class Log
22+
{
23+
// StorageInfoProvider
24+
[LoggerMessage(EventId = 300, Level = LogLevel.Information, Message = "Temporary Storage Path={path}. Storage Size: {totalSize:N0}. Reserved: {reservedSpace:N0}.")]
25+
public static partial void StorageInfoProviderStartup(this ILogger logger, string path, long totalSize, long reservedSpace);
26+
27+
[LoggerMessage(EventId = 301, Level = LogLevel.Information, Message = "Storage Size: {totalSize:N0}. Reserved: {reservedSpace:N0}. Available: {freeSpace:N0}.")]
28+
public static partial void CurrentStorageSize(this ILogger logger, long totalSize, long reservedSpace, long freeSpace);
29+
}
30+
}

‎src/InformaticsGateway/Logging/Log.500.ExportService.cs

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,5 +114,8 @@ public static partial class Log
114114

115115
[LoggerMessage(EventId = 530, Level = LogLevel.Error, Message = "{message}")]
116116
public static partial void ExportException(this ILogger logger, string message, Exception ex);
117+
118+
[LoggerMessage(EventId = 531, Level = LogLevel.Warning, Message = "Export service paused due to insufficient storage space. Available storage space: {availableFreeSpace:D}")]
119+
public static partial void ExportServiceStoppedDueToLowStorageSpace(this ILogger logger, long availableFreeSpace);
117120
}
118121
}

‎src/InformaticsGateway/Logging/Log.600.DataRetrievalService.cs

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,5 +91,8 @@ public static partial class Log
9191

9292
[LoggerMessage(EventId = 622, Level = LogLevel.Warning, Message = "FHIR resource {type}/{id} contains no data.")]
9393
public static partial void FhirResourceContainsNoData(this ILogger logger, string type, string id);
94+
95+
[LoggerMessage(EventId = 623, Level = LogLevel.Warning, Message = "Data retrieval paused due to insufficient storage space. Available storage space: {availableFreeSpace:D}.")]
96+
public static partial void DataRetrievalServiceStoppedDueToLowStorageSpace(this ILogger logger, long availableFreeSpace);
9497
}
9598
}

0 commit comments

Comments
 (0)