Skip to content

Download links from S3; Do not upload release assets to GitHub #2664

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Mar 1, 2020

Conversation

dhalbert
Copy link
Collaborator

@dhalbert dhalbert commented Feb 28, 2020

We now have over 1700 release assets. GitHub releases don't handle this many assets well:

  • The release pages are huge and take a long time to load.
  • Uploading assets can fail sporadically or hit GitHub rate limits.
  • It's difficult to correct the problems above: the release action must be re-run several times and may still continue to fail. Uploading the missing assets is tedious.

We already upload the release assets to S3 without difficulty. This PR changes the https://circuitpython.org/downloads download links to point to a CloudFront distribution (thanks @jwcooper!), aliased from the CNAME https://downloads.circuitpython.org.

We will need to figure out how to get download counts and other statistics, but AWS CloudWatch log files can do that (@jwcooper turned them on), which can then be processed by us with https://s3stat.com, https://qloudstat.com, or us directly parsing the log files.

This PR:

@dhalbert dhalbert requested a review from tannewt February 28, 2020 20:36
Copy link

@sommersoft sommersoft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple fixes/things I can spot. Thanks @dhalbert.

We will need to figure out how to get download counts and other statistics, but AWS CloudWatch log files can do that (@jwcooper turned them on), which can then be processed by us with https://s3stat.com, https://qloudstat.com, or us directly parsing the log files.

After some further investigation/reading, it looks like AWS's Python SDK boto3 should make this a little easier with CloudWatch. I'll keep reading, and see if I can set up a test env if my current S3 won't suffice.

I'll also go ahead and remove adabot's download stats for now, since they're not working anyway. 😄

@dhalbert
Copy link
Collaborator Author

The logs will get sent to the new adafruit-circuit-python-logs bucket.

Thanks for reviewing! I’m out but will add changes later.

@dhalbert dhalbert requested a review from sommersoft February 29, 2020 02:20
@dhalbert dhalbert marked this pull request as ready for review March 1, 2020 16:42
@dhalbert dhalbert merged commit 7528814 into adafruit:5.0.x Mar 1, 2020
@dhalbert dhalbert deleted the downloads-from-s3 branch March 1, 2020 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants