Skip to content

Add filterSensitiveLog method to Structure namespaces #170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 50 commits into from
May 27, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
0011c55
Add basic toString function
trivikr Apr 27, 2020
a45ddcb
Add REDACTED to member if it has SensitiveTrait
trivikr Apr 28, 2020
cfca4b7
Use getMemberTrait to find SensitiveTrait
trivikr Apr 28, 2020
b3f589f
Move code to StructuredMemberWriter
trivikr Apr 28, 2020
e535e9c
Add SENSITIVE_STRING for optional members only if they're present
trivikr Apr 28, 2020
3159785
chore: remove isRequired check as all members are either undefined or…
trivikr Apr 30, 2020
9b2492d
feat: call toString on Structures
trivikr Apr 30, 2020
f4971cd
chore: rename toString to filterSensitiveLog
trivikr Apr 30, 2020
dbc847d
feat: add filterSensitiveLog for Array
trivikr Apr 30, 2020
687c1d0
feat: added support for list of lists
trivikr May 1, 2020
1d686fe
Move map open and end outside if-else block
trivikr May 1, 2020
0253956
Fix issues with writeFilterSensitiveLogForArray
trivikr May 1, 2020
fb40619
Fix java.lang.StackOverflowError
trivikr May 1, 2020
ebd547e
Simplify code by storing member shape in memberShape
trivikr May 1, 2020
5125a70
Use openBlock instead of write
trivikr May 1, 2020
429f753
Pass arrayMember in writeFilterSensitiveLogForArray
trivikr May 1, 2020
667671a
Fix bug with Array<Array<SimpleShape>>
trivikr May 1, 2020
7514dfe
filterSensitiveLog for MapShape inside Structure
trivikr May 1, 2020
4f9abc8
Move reducer function definition outside memberShape comparison
trivikr May 1, 2020
73dbfcf
Remove redundant code by doing iteration only for some shapes
trivikr May 1, 2020
2ab9128
Recursively call isIterationRequired to remove redundant code
trivikr May 1, 2020
afc8591
filterSensitiveLog for Map inside Collection
trivikr May 1, 2020
64a48eb
Explicitly return any from filterSensitiveLog method
trivikr May 1, 2020
d5bf159
filterSensitiveLog for Collection inside Map
trivikr May 1, 2020
513ee78
filterSensitiveLog for Map inside Map
trivikr May 1, 2020
4464b87
Fix bug in filterSensitiveLog for Map of Map
trivikr May 1, 2020
72a0a3a
chore: rename writeFilterSensitiveLogFor<X> to write<X>FilterSensitiv…
trivikr May 13, 2020
3e89a6c
Simplified method isIterationRequired
trivikr May 13, 2020
6a40003
Add period at the end of comments
trivikr May 13, 2020
117de12
Removed braces for variable writes
trivikr May 13, 2020
e7ff2fb
Used positional parameters in write() calls
trivikr May 13, 2020
a75f171
Use destructuring in writeMapFilterSensitiveLog
trivikr May 13, 2020
e00a880
Throw error in filterSensitiveLog where the path should never reach
trivikr May 18, 2020
ba5d2c9
Make writeX methods private
trivikr May 19, 2020
cf46258
Add writeStructureFilterSensitiveLog
trivikr May 19, 2020
7503ee7
Edge case for collection with sensitive trait
trivikr May 19, 2020
28d478b
Edge case for map with Sensitive trait
trivikr May 19, 2020
f8ac270
Refactor to simplify code
trivikr May 19, 2020
81ff15c
Rename config -> structuredMemberWriter
trivikr May 19, 2020
b436936
Move writeFilterSensitiveLog above private functions
trivikr May 19, 2020
f9e6fd7
Remove shape from writeFilterSensitiveLog
trivikr May 19, 2020
c4f2d40
Rename memberShape to memberTarget
trivikr May 19, 2020
3c73a41
Create getSanitizedMemberName method
trivikr May 19, 2020
1bb22be
Fix for ./gradlew test to be successful
trivikr May 20, 2020
9a31c3d
Add tests for filterSensitiveLog
trivikr May 21, 2020
dd7fd26
Updated documentation to clarify getSanitizedMemberName
trivikr May 27, 2020
d4dc6fb
Use structureTarget.hasTrait in writeStructureFilterSensitiveLog
trivikr May 27, 2020
c8fced3
Added tests for callsFilterIn<X>WithSensitiveData
trivikr May 27, 2020
ad357b7
Modify tests for sensitive List/Map/Structure
trivikr May 27, 2020
3556c64
Add tests for member pointing to sensitive structure/list/map
trivikr May 27, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -158,18 +158,28 @@ private void renderErrorStructure() {
writer.openBlock("export interface $L extends $L {", symbol.getName(), extendsFrom);
writer.write("name: $S;", shape.getId().getName());
writer.write("$$fault: $S;", errorTrait.getValue());
StructuredMemberWriter config = new StructuredMemberWriter(
StructuredMemberWriter structuredMemberWriter = new StructuredMemberWriter(
model, symbolProvider, shape.getAllMembers().values());
config.writeMembers(writer, shape);
structuredMemberWriter.writeMembers(writer, shape);
writer.closeBlock("}"); // interface
writer.write("");
renderStructureNamespace();
}

private void renderStructureNamespace() {
writer.addImport("isa", "__isa", "@aws-sdk/smithy-client");
writer.addImport("SENSITIVE_STRING", "SENSITIVE_STRING", "@aws-sdk/smithy-client");
Symbol symbol = symbolProvider.toSymbol(shape);
writer.openBlock("export namespace $L {", "}", symbol.getName(), () -> {
String objectParam = "obj";
writer.openBlock("export const filterSensitiveLog = ($L: $L): any => ({", "})",
objectParam, symbol.getName(),
() -> {
StructuredMemberWriter structuredMemberWriter = new StructuredMemberWriter(
model, symbolProvider, shape.getAllMembers().values());
structuredMemberWriter.writeFilterSensitiveLog(writer, objectParam);
}
);
writer.write("export const isa = (o: any): o is $L => __isa(o, $S);",
symbol.getName(), shape.getId().getName()
);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,16 @@
import java.util.Collection;
import java.util.HashSet;
import java.util.Set;
import software.amazon.smithy.codegen.core.CodegenException;
import software.amazon.smithy.codegen.core.SymbolProvider;
import software.amazon.smithy.model.Model;
import software.amazon.smithy.model.shapes.CollectionShape;
import software.amazon.smithy.model.shapes.MapShape;
import software.amazon.smithy.model.shapes.MemberShape;
import software.amazon.smithy.model.shapes.Shape;
import software.amazon.smithy.model.shapes.StructureShape;
import software.amazon.smithy.model.traits.IdempotencyTokenTrait;
import software.amazon.smithy.model.traits.SensitiveTrait;

/**
* Generates objects, interfaces, enums, etc.
Expand Down Expand Up @@ -53,7 +58,7 @@ void writeMembers(TypeScriptWriter writer, Shape shape) {

position++;
boolean wroteDocs = !noDocs && writer.writeMemberDocs(model, member);
String memberName = TypeScriptUtils.sanitizePropertyName(symbolProvider.toMemberName(member));
String memberName = getSanitizedMemberName(member);
String optionalSuffix = shape.isUnionShape() || !isRequiredMember(member) ? "?" : "";
String typeSuffix = isRequiredMember(member) ? " | undefined" : "";
writer.write("${L}${L}${L}: ${T}${L};", memberPrefix, memberName, optionalSuffix,
Expand All @@ -65,6 +70,173 @@ void writeMembers(TypeScriptWriter writer, Shape shape) {
}
}

void writeFilterSensitiveLog(TypeScriptWriter writer, String objectParam) {
writer.write("...$L,", objectParam);
for (MemberShape member : members) {
if (isMemberOverwriteRequired(member)) {
Shape memberTarget = model.expectShape(member.getTarget());
String memberName = getSanitizedMemberName(member);
String memberParam = String.format("%s.%s", objectParam, memberName);
writer.openBlock("...($1L.$2L && { $2L: ", "}),", objectParam, memberName, () -> {
if (member.getMemberTrait(model, SensitiveTrait.class).isPresent()) {
// member is Sensitive, hide the value.
writer.write("SENSITIVE_STRING");
} else if (memberTarget instanceof StructureShape) {
writeStructureFilterSensitiveLog(writer, memberTarget, memberParam);
} else if (memberTarget instanceof CollectionShape) {
MemberShape collectionMember = ((CollectionShape) memberTarget).getMember();
writeCollectionFilterSensitiveLog(writer, collectionMember, memberParam);
} else if (memberTarget instanceof MapShape) {
MemberShape mapMember = ((MapShape) memberTarget).getValue();
writeMapFilterSensitiveLog(writer, mapMember, memberParam);
}
});
}
}
}

/**
* Recursively writes filterSensitiveLog for StructureShape.
*/
private void writeStructureFilterSensitiveLog(
TypeScriptWriter writer,
Shape structureTarget,
String structureParam
) {
if (structureTarget.hasTrait(SensitiveTrait.class)) {
// member is Sensitive, hide the value.
writer.write("SENSITIVE_STRING");
return;
}
// Call filterSensitiveLog on Structure.
writer.write("$T.filterSensitiveLog($L)", symbolProvider.toSymbol(structureTarget), structureParam);
}

/**
* Recursively writes filterSensitiveLog for CollectionShape.
*/
private void writeCollectionFilterSensitiveLog(
TypeScriptWriter writer,
MemberShape collectionMember,
String collectionParam
) {
if (collectionMember.getMemberTrait(model, SensitiveTrait.class).isPresent()) {
// member is Sensitive, hide the value.
writer.write("SENSITIVE_STRING");
return;
}

writer.openBlock("$L.map(", ")", collectionParam, () -> {
String itemParam = "item";
Shape collectionMemberTarget = model.expectShape(collectionMember.getTarget());
writer.write("$L => ", itemParam);
if (collectionMemberTarget instanceof StructureShape) {
writeStructureFilterSensitiveLog(writer, collectionMemberTarget, itemParam);
} else if (collectionMemberTarget instanceof CollectionShape) {
MemberShape nestedCollectionMember = ((CollectionShape) collectionMemberTarget).getMember();
writeCollectionFilterSensitiveLog(writer, nestedCollectionMember, itemParam);
} else if (collectionMemberTarget instanceof MapShape) {
MemberShape mapMember = ((MapShape) collectionMemberTarget).getValue();
writeMapFilterSensitiveLog(writer, mapMember, itemParam);
} else {
// This path should not reach because of recursive isIterationRequired.
throw new CodegenException(String.format(
"CollectionFilterSensitiveLog attempted for %s while it was not required",
collectionMemberTarget.getType()
));
// For quick-fix in case of high severity issue:
// comment out the exception above and uncomment the line below.
// writer.write("$1L => $1L", itemParam);
}
});
}

/**
* Recursively writes filterSensitiveLog for MapShape.
*/
private void writeMapFilterSensitiveLog(TypeScriptWriter writer, MemberShape mapMember, String mapParam) {
if (mapMember.getMemberTrait(model, SensitiveTrait.class).isPresent()) {
// member is Sensitive, hide the value.
writer.write("SENSITIVE_STRING");
return;
}

String accParam = "acc"; // accumulator for the reducer
String keyParam = "key"; // key of the Object.entries() key-value pair
String valueParam = "value"; // value of the Object.entries() key-value pair

// Reducer is common to all shapes.
writer.openBlock("Object.entries($L).reduce(($L: any, [$L, $L]: [string, $T]) => ({", "}), {})",
mapParam, accParam, keyParam, valueParam, symbolProvider.toSymbol(mapMember), () -> {
writer.write("...$L,", accParam);
Shape mapMemberTarget = model.expectShape(mapMember.getTarget());
writer.openBlock("[$L]: ", ",", keyParam, () -> {
if (mapMemberTarget instanceof StructureShape) {
writeStructureFilterSensitiveLog(writer, mapMemberTarget, valueParam);
} else if (mapMemberTarget instanceof CollectionShape) {
MemberShape collectionMember = ((CollectionShape) mapMemberTarget).getMember();
writeCollectionFilterSensitiveLog(writer, collectionMember, valueParam);
} else if (mapMemberTarget instanceof MapShape) {
MemberShape nestedMapMember = ((MapShape) mapMemberTarget).getValue();
writeMapFilterSensitiveLog(writer, nestedMapMember, valueParam);
} else {
// This path should not reach because of recursive isIterationRequired.
throw new CodegenException(String.format(
"MapFilterSensitiveLog attempted for %s while it was not required",
mapMemberTarget.getType()
));
// For quick-fix in case of high severity issue:
// comment out the exception above and uncomment the line below.
// writer.write("$L", valueParam);
}

});
}
);
}

/**
* Identifies if iteration is required on member.
*
* @param member a {@link MemberShape} to check for iteration required.
* @return Returns true if the iteration is required on member.
*/
private boolean isIterationRequired(MemberShape member) {
Shape memberTarget = model.expectShape(member.getTarget());
if (memberTarget instanceof StructureShape) {
return true;
} else if (memberTarget instanceof CollectionShape) {
MemberShape collectionMember = ((CollectionShape) memberTarget).getMember();
return isIterationRequired(collectionMember);
} else if (memberTarget instanceof MapShape) {
MemberShape mapMember = ((MapShape) memberTarget).getValue();
return isIterationRequired(mapMember);
}
return false;
}

/**
* Identifies if member needs to be overwritten in filterSensitiveLog.
*
* @param member a {@link MemberShape} to check if overwrite is required.
* @return Returns true if the overwrite is required on member.
*/
private boolean isMemberOverwriteRequired(MemberShape member) {
return (
member.getMemberTrait(model, SensitiveTrait.class).isPresent() || isIterationRequired(member)
);
}

/**
* Returns the member name to be used in generation.
*
* @param member a {@link MemberShape} to be sanitized.
* @return Returns the member name to be used in generation.
*/
private String getSanitizedMemberName(MemberShape member) {
return TypeScriptUtils.sanitizePropertyName(symbolProvider.toMemberName(member));
}

/**
* Identifies if a member should be required on the generated interface.
*
Expand Down
Loading