Skip to content

Commit 0ec0e96

Browse files
committed
docs(supplemental-docs): md5 checksum fallback for s3
1 parent ebd03ac commit 0ec0e96

File tree

1 file changed

+111
-0
lines changed

1 file changed

+111
-0
lines changed

supplemental-docs/MD5_FALLBACK.md

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# MD5 Checksum Fallback for AWS SDK for JavaScript v3
2+
3+
## Background
4+
5+
Recently the AWS SDKs shipped a feature that [changed default object integrity in S3](https://github.com/aws/aws-sdk-js-v3/issues/6810). The SDKs now default to using more modern checksums (like CRC32) to ensure object integrity, whereas previously MD5 checksums were being used. Some third-party S3-compatible services currently do not support these checksums. To our knowledge, this affects only the S3 `DeleteObjects` operation.
6+
7+
If you wish to fallback to the old behavior of sending MD5 checksums, for operations like `DeleteObjectsCommand` this is how you can do it in AWS SDK for JavaScript v3:
8+
9+
## MD5 fallback
10+
11+
The following code provides a custom S3 client that will use MD5 checksums for DeleteObjects operations while maintaining the default behavior for all other operations.
12+
13+
```javascript
14+
// md5ClientS3.mjs
15+
import { S3Client } from "@aws-sdk/client-s3";
16+
import { createHash } from "crypto";
17+
18+
/**
19+
* Creates an S3 client that uses MD5 checksums for DeleteObjects operations
20+
*/
21+
export function createS3ClientWithMD5() {
22+
const client = new S3Client({});
23+
24+
client.middlewareStack.add(
25+
(next) => async (args) => {
26+
// Check if this is a DeleteObjects command
27+
const isDeleteObjects = args.constructor?.name === "DeleteObjectsCommand" || args.input?.Delete !== undefined;
28+
29+
if (!isDeleteObjects) {
30+
return next(args);
31+
}
32+
33+
// Remove any checksum headers
34+
const headers = args.request.headers;
35+
Object.keys(headers).forEach((header) => {
36+
if (
37+
header.toLowerCase().startsWith("x-amz-checksum-") ||
38+
header.toLowerCase().startsWith("x-amz-sdk-checksum-")
39+
) {
40+
delete headers[header];
41+
}
42+
});
43+
44+
// Calculate and add MD5 for the request body
45+
if (args.request.body) {
46+
const bodyContent = Buffer.from(args.request.body);
47+
const md5Hash = createHash("md5").update(bodyContent).digest("base64");
48+
headers["Content-MD5"] = md5Hash;
49+
}
50+
51+
return next(args);
52+
},
53+
{
54+
step: "build",
55+
name: "addMD5Checksum",
56+
}
57+
);
58+
59+
return client;
60+
}
61+
```
62+
63+
## Usage
64+
65+
Instead of creating a regular S3 client, use the `createS3ClientWithMD5` function:
66+
67+
```javascript
68+
import { DeleteObjectsCommand } from "@aws-sdk/client-s3";
69+
import { createS3ClientWithMD5 } from "./md5ClientS3.mjs";
70+
71+
// Create the client with MD5 support
72+
const client = createS3ClientWithMD5();
73+
74+
// Use it like a normal S3 client
75+
const deleteParams = {
76+
Bucket: "your-bucket",
77+
Delete: {
78+
Objects: [{ Key: "file1.txt" }, { Key: "file2.txt" }],
79+
},
80+
};
81+
82+
try {
83+
const response = await client.send(new DeleteObjectsCommand(deleteParams));
84+
console.log("Successfully deleted objects:", response);
85+
} catch (err) {
86+
console.error("Error:", err);
87+
}
88+
```
89+
90+
## How It Works
91+
92+
The solution adds middleware to the S3 client that:
93+
94+
1. Detects DeleteObjects operations
95+
2. Removes any checksum headers
96+
3. Calculates an MD5 hash of the request body (in the `build` step of the request lifecycle, as per the middleware implementation above)
97+
4. Adds the MD5 hash as a Content-MD5 header
98+
99+
## Usage Notes
100+
101+
- The client can be configured with additional options as needed (region, credentials, etc.)
102+
- If your S3-compatible service supports the SDK's new checksum options or adds support in the future, you should use the standard S3 client instead
103+
104+
## Debugging
105+
106+
To verify that the MD5 checksum is being correctly applied, you can add console logging to the middleware by modifying the code to include logging statements:
107+
108+
```javascript
109+
// Inside the middleware function, add:
110+
console.log("Headers:", JSON.stringify(args.request.headers, null, 2));
111+
```

0 commit comments

Comments
 (0)