Skip to content

feat(endpoint): add wire protocol filtering option to copy models script #4068

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 20, 2022
Merged

feat(endpoint): add wire protocol filtering option to copy models script #4068

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5328d59
feat(endpoint): add wire protocol filtering option to copy models script
kuhe Oct 20, 2022
e1ec127
feat(endpoint): copy models throw on not finding skdId or protocol
kuhe Oct 20, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 61 additions & 16 deletions scripts/copy-models/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,53 @@
const yargs = require("yargs");

const { promises: fsPromises } = require("fs");
const { join } = require("path");
const { spawnProcess } = require("../utils/spawn-process");
const { join, resolve } = require("path");

const { models } = yargs(process.argv.slice(2))
const { models, protocols } = yargs(process.argv.slice(2))
.alias("m", "models")
.string("m")
.describe("m", "The path to directory with aws-models.")
.demandOption(["models"])
.alias("p", "protocols")
.string("p")
.describe(
"p",
"List of protocols to copy e.g. awsJson1,restXml, default all, list: \n" +
`awsJson1_1,
restJson1,
awsJson1_0,
awsQuery,
restXml,
ec2Query`
)
.help().argv;

const getSdkId = (model) => {
const { shapes } = model;
const service = Object.values(shapes).find((shape) => shape.type === "service") || {};
const { traits } = service;
for (const [trait, value] of Object.entries(traits)) {
if (trait === "aws.api#service") {
return value.sdkId;
}
}
throw new Error("unable to find SDK ID in model file");
};

const getWireProtocol = (model) => {
const { shapes } = model;
const service = Object.values(shapes).find((shape) => shape.type === "service") || {};
for (const trait of Object.keys(service.traits || {})) {
if (trait.startsWith("aws.protocols#")) {
const parts = trait.split("aws.protocols#");
if (parts.length !== 0) {
return parts.pop();
}
}
}
throw new Error("unable to determine wire protocol in model file");
};

(async () => {
const OUTPUT_DIR = join(__dirname, "..", "..", "codegen", "sdk-codegen", "aws-models");

Expand All @@ -23,34 +60,42 @@ const { models } = yargs(process.argv.slice(2))
.filter((file) => file.isDirectory())
.map((dir) => join(models.toString(), dir.name, `smithy/model.json`));

const observedProtocols = new Set();

for (const smithyModelsFile of smithyModelsFiles) {
try {
// Test if file exists.
await fsPromises.stat(smithyModelsFile);
// File exists, copy it.
const absolutePath = resolve(smithyModelsFile);

try {
const fileContent = (await fsPromises.readFile(smithyModelsFile)).toString();
const model = require(absolutePath);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

switching to require since this is a JSON file

const sdkId = getSdkId(model).toLowerCase().replace(/\s/g, "-");
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

confirmed this is the same set of filenames as before and does not create any new files

test: yarn copy-models -m ...

const protocol = getWireProtocol(model);

const sdkIdRE = /"sdkId": "([^"]*)"/;
const sdkId = fileContent.match(sdkIdRE)[1].toLowerCase().replace(/\s/g, "-");
observedProtocols.add(protocol);

// Copy file.
const outputFile = join(OUTPUT_DIR, `${sdkId}.json`);
await fsPromises.writeFile(outputFile, fileContent);
if (!protocols || protocols.split(",").includes(protocol)) {
// Copy file.
const outputFile = join(OUTPUT_DIR, `${sdkId}.json`);
await fsPromises.writeFile(outputFile, JSON.stringify(model, null, 2));
console.log("Copied", outputFile);
}
} catch (e) {
// Copy failed, log.
console.log(smithyModelsFile);
console.log("Failed to copy", absolutePath);
console.log(e.message);
}
} catch (e) {
// File doesn't exist, ignore.
console.log(e.message);
console.log("File not found", e.message);
}
}

// Prettify copied models
await spawnProcess(join(__dirname, "..", "..", "node_modules", ".bin", "prettier"), [
"--write",
`${OUTPUT_DIR}/*.json`,
]);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removing prettier step to save time. JSON.stringify(, null, 2) is good enough.

console.log("args:", {
models,
protocols,
observedProtocols,
});
})();