Skip to content

Add filterSensitiveLog method to Structure namespaces #170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 50 commits into from
May 27, 2020
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
0011c55
Add basic toString function
trivikr Apr 27, 2020
a45ddcb
Add REDACTED to member if it has SensitiveTrait
trivikr Apr 28, 2020
cfca4b7
Use getMemberTrait to find SensitiveTrait
trivikr Apr 28, 2020
b3f589f
Move code to StructuredMemberWriter
trivikr Apr 28, 2020
e535e9c
Add SENSITIVE_STRING for optional members only if they're present
trivikr Apr 28, 2020
3159785
chore: remove isRequired check as all members are either undefined or…
trivikr Apr 30, 2020
9b2492d
feat: call toString on Structures
trivikr Apr 30, 2020
f4971cd
chore: rename toString to filterSensitiveLog
trivikr Apr 30, 2020
dbc847d
feat: add filterSensitiveLog for Array
trivikr Apr 30, 2020
687c1d0
feat: added support for list of lists
trivikr May 1, 2020
1d686fe
Move map open and end outside if-else block
trivikr May 1, 2020
0253956
Fix issues with writeFilterSensitiveLogForArray
trivikr May 1, 2020
fb40619
Fix java.lang.StackOverflowError
trivikr May 1, 2020
ebd547e
Simplify code by storing member shape in memberShape
trivikr May 1, 2020
5125a70
Use openBlock instead of write
trivikr May 1, 2020
429f753
Pass arrayMember in writeFilterSensitiveLogForArray
trivikr May 1, 2020
667671a
Fix bug with Array<Array<SimpleShape>>
trivikr May 1, 2020
7514dfe
filterSensitiveLog for MapShape inside Structure
trivikr May 1, 2020
4f9abc8
Move reducer function definition outside memberShape comparison
trivikr May 1, 2020
73dbfcf
Remove redundant code by doing iteration only for some shapes
trivikr May 1, 2020
2ab9128
Recursively call isIterationRequired to remove redundant code
trivikr May 1, 2020
afc8591
filterSensitiveLog for Map inside Collection
trivikr May 1, 2020
64a48eb
Explicitly return any from filterSensitiveLog method
trivikr May 1, 2020
d5bf159
filterSensitiveLog for Collection inside Map
trivikr May 1, 2020
513ee78
filterSensitiveLog for Map inside Map
trivikr May 1, 2020
4464b87
Fix bug in filterSensitiveLog for Map of Map
trivikr May 1, 2020
72a0a3a
chore: rename writeFilterSensitiveLogFor<X> to write<X>FilterSensitiv…
trivikr May 13, 2020
3e89a6c
Simplified method isIterationRequired
trivikr May 13, 2020
6a40003
Add period at the end of comments
trivikr May 13, 2020
117de12
Removed braces for variable writes
trivikr May 13, 2020
e7ff2fb
Used positional parameters in write() calls
trivikr May 13, 2020
a75f171
Use destructuring in writeMapFilterSensitiveLog
trivikr May 13, 2020
e00a880
Throw error in filterSensitiveLog where the path should never reach
trivikr May 18, 2020
ba5d2c9
Make writeX methods private
trivikr May 19, 2020
cf46258
Add writeStructureFilterSensitiveLog
trivikr May 19, 2020
7503ee7
Edge case for collection with sensitive trait
trivikr May 19, 2020
28d478b
Edge case for map with Sensitive trait
trivikr May 19, 2020
f8ac270
Refactor to simplify code
trivikr May 19, 2020
81ff15c
Rename config -> structuredMemberWriter
trivikr May 19, 2020
b436936
Move writeFilterSensitiveLog above private functions
trivikr May 19, 2020
f9e6fd7
Remove shape from writeFilterSensitiveLog
trivikr May 19, 2020
c4f2d40
Rename memberShape to memberTarget
trivikr May 19, 2020
3c73a41
Create getSanitizedMemberName method
trivikr May 19, 2020
1bb22be
Fix for ./gradlew test to be successful
trivikr May 20, 2020
9a31c3d
Add tests for filterSensitiveLog
trivikr May 21, 2020
dd7fd26
Updated documentation to clarify getSanitizedMemberName
trivikr May 27, 2020
d4dc6fb
Use structureTarget.hasTrait in writeStructureFilterSensitiveLog
trivikr May 27, 2020
c8fced3
Added tests for callsFilterIn<X>WithSensitiveData
trivikr May 27, 2020
ad357b7
Modify tests for sensitive List/Map/Structure
trivikr May 27, 2020
3556c64
Add tests for member pointing to sensitive structure/list/map
trivikr May 27, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
import software.amazon.smithy.codegen.core.SymbolProvider;
import software.amazon.smithy.codegen.core.SymbolReference;
import software.amazon.smithy.model.Model;
import software.amazon.smithy.model.shapes.MemberShape;
import software.amazon.smithy.model.shapes.StructureShape;
import software.amazon.smithy.model.traits.ErrorTrait;

Expand Down Expand Up @@ -168,8 +169,17 @@ private void renderErrorStructure() {

private void renderStructureNamespace() {
writer.addImport("isa", "__isa", "@aws-sdk/smithy-client");
writer.addImport("SENSITIVE_STRING", "SENSITIVE_STRING", "@aws-sdk/smithy-client");
Symbol symbol = symbolProvider.toSymbol(shape);
writer.openBlock("export namespace $L {", "}", symbol.getName(), () -> {
writer.openBlock("export const filterSensitiveLog = (obj: $L): any => ({", "})",
symbol.getName(),
() -> {
StructuredMemberWriter config = new StructuredMemberWriter(
model, symbolProvider, shape.getAllMembers().values());
config.writeFilterSensitiveLog(writer, shape);
}
);
writer.write("export const isa = (o: any): o is $L => __isa(o, $S);",
symbol.getName(), shape.getId().getName()
);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,16 @@
import java.util.Collection;
import java.util.HashSet;
import java.util.Set;
import software.amazon.smithy.codegen.core.CodegenException;
import software.amazon.smithy.codegen.core.SymbolProvider;
import software.amazon.smithy.model.Model;
import software.amazon.smithy.model.shapes.CollectionShape;
import software.amazon.smithy.model.shapes.MapShape;
import software.amazon.smithy.model.shapes.MemberShape;
import software.amazon.smithy.model.shapes.Shape;
import software.amazon.smithy.model.shapes.StructureShape;
import software.amazon.smithy.model.traits.IdempotencyTokenTrait;
import software.amazon.smithy.model.traits.SensitiveTrait;

/**
* Generates objects, interfaces, enums, etc.
Expand Down Expand Up @@ -65,6 +70,140 @@ void writeMembers(TypeScriptWriter writer, Shape shape) {
}
}

/**
* Recursively writes filterSensitiveLog for CollectionShape.
*/
void writeCollectionFilterSensitiveLog(TypeScriptWriter writer, MemberShape collectionMember) {
Shape memberShape = model.expectShape(collectionMember.getTarget());
if (memberShape instanceof StructureShape) {
// Call filterSensitiveLog on Structure.
writer.write("$T.filterSensitiveLog", symbolProvider.toSymbol(collectionMember));
} else if (memberShape instanceof CollectionShape) {
// Iterate over array items, and call array specific function on each member.
writer.openBlock("item => item.map(", ")",
() -> {
MemberShape nestedCollectionMember = ((CollectionShape) memberShape).getMember();
writeCollectionFilterSensitiveLog(writer, nestedCollectionMember);
}
);
} else if (memberShape instanceof MapShape) {
// Iterate over Object entries, and call reduce to repopulate map.
writer.openBlock("item => Object.entries(item).reduce(", ")",
() -> {
MemberShape mapMember = ((MapShape) memberShape).getValue();
writeMapFilterSensitiveLog(writer, mapMember);
}
);
} else {
// This path should not reach because of recursive isIterationRequired.
throw new CodegenException(String.format(
"CollectionFilterSensitiveLog attempted for %s while it was not required",
memberShape.getType()
));
// For quick-fix in case of high severity issue:
// comment out the exception above and uncomment the line below.
// writer.write("item => item");
}
}

/**
* Recursively writes filterSensitiveLog for MapShape.
*/
void writeMapFilterSensitiveLog(TypeScriptWriter writer, MemberShape mapMember) {
// Reducer is common to all shapes.
writer.openBlock("(acc: any, [key, value]: [string, $T]) => ({", "}), {}",
symbolProvider.toSymbol(mapMember),
() -> {
writer.write("...acc,");
Shape memberShape = model.expectShape(mapMember.getTarget());
if (memberShape instanceof StructureShape) {
// Call filterSensitiveLog on Structure.
writer.write("[key]: $T.filterSensitiveLog(value),",
symbolProvider.toSymbol(mapMember));
} else if (memberShape instanceof CollectionShape) {
writer.openBlock("[key]: value.map(", "),",
() -> {
MemberShape collectionMember = ((CollectionShape) memberShape).getMember();
writeCollectionFilterSensitiveLog(writer, collectionMember);
}
);
} else if (memberShape instanceof MapShape) {
writer.openBlock("[key]: Object.entries(value).reduce(", "),",
() -> {
MemberShape nestedMapMember = ((MapShape) memberShape).getValue();
writeMapFilterSensitiveLog(writer, nestedMapMember);
}
);
} else {
// This path should not reach because of recursive isIterationRequired.
throw new CodegenException(String.format(
"MapFilterSensitiveLog attempted for %s while it was not required",
memberShape.getType()
));
// For quick-fix in case of high severity issue:
// comment out the exception above and uncomment the line below.
// writer.write("[key]: value,");
}
}
);
}

void writeFilterSensitiveLog(TypeScriptWriter writer, Shape shape) {
writer.write("...obj,");
for (MemberShape member : members) {
Shape memberShape = model.expectShape(member.getTarget());
String memberName = TypeScriptUtils.sanitizePropertyName(symbolProvider.toMemberName(member));
if (member.getMemberTrait(model, SensitiveTrait.class).isPresent()) {
// member is Sensitive, hide the value.
writer.write("...(obj.$1L && { $1L: SENSITIVE_STRING }),", memberName);
} else if (memberShape instanceof StructureShape) {
// Call filterSensitiveLog on Structure.
writer.write("...(obj.$1L && { $1L: $2T.filterSensitiveLog(obj.$1L)}),",
memberName, symbolProvider.toSymbol(member));
} else if (memberShape instanceof CollectionShape) {
MemberShape collectionMember = ((CollectionShape) memberShape).getMember();
if (isIterationRequired(collectionMember)) {
// Iterate over array items, and call array specific function on each member.
writer.openBlock("...(obj.$1L && { $1L: obj.$1L.map(", ")}),", memberName,
() -> {
writeCollectionFilterSensitiveLog(writer, collectionMember);
}
);
}
} else if (memberShape instanceof MapShape) {
MemberShape mapMember = ((MapShape) memberShape).getValue();
if (isIterationRequired(mapMember)) {
// Iterate over Object entries, and call reduce to repopulate map.
writer.openBlock("...(obj.$1L && { $1L: Object.entries(obj.$1L).reduce(", ")}),", memberName,
() -> {
writeMapFilterSensitiveLog(writer, mapMember);
}
);
}
}
}
}

/**
* Identifies if iteration is required on MemberShape.
*
* @param memberShape a {@link MemberShape} to check for iteration required.
* @return Returns true if the iteration is required on memberShape.
*/
private boolean isIterationRequired(MemberShape memberShape) {
Shape targetShape = model.expectShape(memberShape.getTarget());
if (targetShape instanceof StructureShape) {
return true;
} if (targetShape instanceof CollectionShape) {
MemberShape collectionMember = ((CollectionShape) targetShape).getMember();
return isIterationRequired(collectionMember);
} else if (targetShape instanceof MapShape) {
MemberShape mapMember = ((MapShape) targetShape).getValue();
return isIterationRequired(mapMember);
}
return false;
}

/**
* Identifies if a member should be required on the generated interface.
*
Expand Down