Run E2E Test in docker container with packages already installed #33
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
harrryr
force-pushed
the
build-docker-for-e2e
branch
3 times, most recently
from
574c7bb to
68aeef3
Compare
majanjua-amzn
approved these changes
Apr 8, 2024
This was referenced Apr 8, 2024
zzhlogin
pushed a commit
to zzhlogin/aws-application-signals-test-framework
that referenced
this pull request
Jun 6, 2024
…docker-for-e2e Run E2E Test in docker container with packages already installed
zzhlogin
pushed a commit
to zzhlogin/aws-application-signals-test-framework
that referenced
this pull request
Jun 6, 2024
…docker-for-e2e Run E2E Test in docker container with packages already installed
zzhlogin
pushed a commit
to zzhlogin/aws-application-signals-test-framework
that referenced
this pull request
Jun 6, 2024
…docker-for-e2e Run E2E Test in docker container with packages already installed
zzhlogin
pushed a commit
to zzhlogin/aws-application-signals-test-framework
that referenced
this pull request
Jun 6, 2024
…docker-for-e2e Run E2E Test in docker container with packages already installed
georgeboc
pushed a commit
to georgeboc/aws-application-signals-test-framework
that referenced
this pull request
Jul 8, 2024
…aws-observability#33) *Issue #, if available:* First PR of 3 parts for adding the X-Ray remote sampling support for OTel Python SDK. *Description of changes:* - Python Classes - `AwsXRayRemoteSampler` - extends `opentelemetry.sdk.trace.sampling.Sampler` and implements `should_sample`. - Upon initialization, starts polling for sampling rules by scheduling a threading.Timer to execute a poll after a configurable interval of time. After this interval, it will repeat this process indefinitely by scheduling the same threading.Timer upon completion of the previous timer. - OTel `resource`, Collector `endpoint`, rules `polling_interval` are configurable. - `AwsXRaySamplingClient` - client to call GetSamplingRules - `SamplingRule` - Class for SamplingRules type Testing Script to poll Sampling Rules every 5 seconds: ``` import logging import time from amazon.opentelemetry.distro.sampler.aws_xray_remote_sampler import AwsXRayRemoteSampler from opentelemetry.sdk.resources import Resource logging.basicConfig(level=logging.INFO) sampler = AwsXRayRemoteSampler(Resource.get_empty(), polling_interval=5) time.sleep(15) ``` Output: ``` 88665a53c0dd:sampler jjllee$ python3 mytesting.py INFO:amazon.opentelemetry.distro.sampler.aws_xray_remote_sampler:Got Sampling Rules: {'[{"Attributes": {}, "FixedRate": 0.05, "HTTPMethod": "*", "Host": "*", "Priority": 10000, "ReservoirSize": 100, "ResourceARN": "*", "RuleARN": "arn:aws:xray:us-east-1:999999999999:sampling-rule/Default", "RuleName": "Default", "ServiceName": "*", "ServiceType": "*", "URLPath": "*", "Version": 1}, {"Attributes": {"abc": "1234"}, "FixedRate": 0.11, "HTTPMethod": "*", "Host": "*", "Priority": 20, "ReservoirSize": 1, "ResourceARN": "*", "RuleARN": "arn:aws:xray:us-east-1:999999999999:sampling-rule/test", "RuleName": "test", "ServiceName": "*", "ServiceType": "*", "URLPath": "*", "Version": 1}]'} INFO:amazon.opentelemetry.distro.sampler.aws_xray_remote_sampler:Got Sampling Rules: {'[{"Attributes": {}, "FixedRate": 0.05, "HTTPMethod": "*", "Host": "*", "Priority": 10000, "ReservoirSize": 100, "ResourceARN": "*", "RuleARN": "arn:aws:xray:us-east-1:999999999999:sampling-rule/Default", "RuleName": "Default", "ServiceName": "*", "ServiceType": "*", "URLPath": "*", "Version": 1}, {"Attributes": {"abc": "1234"}, "FixedRate": 0.11, "HTTPMethod": "*", "Host": "*", "Priority": 20, "ReservoirSize": 1, "ResourceARN": "*", "RuleARN": "arn:aws:xray:us-east-1:999999999999:sampling-rule/test", "RuleName": "test", "ServiceName": "*", "ServiceType": "*", "URLPath": "*", "Version": 1}]'} INFO:amazon.opentelemetry.distro.sampler.aws_xray_remote_sampler:Got Sampling Rules: {'[{"Attributes": {}, "FixedRate": 0.05, "HTTPMethod": "*", "Host": "*", "Priority": 10000, "ReservoirSize": 100, "ResourceARN": "*", "RuleARN": "arn:aws:xray:us-east-1:999999999999:sampling-rule/Default", "RuleName": "Default", "ServiceName": "*", "ServiceType": "*", "URLPath": "*", "Version": 1}, {"Attributes": {"abc": "1234"}, "FixedRate": 0.11, "HTTPMethod": "*", "Host": "*", "Priority": 20, "ReservoirSize": 1, "ResourceARN": "*", "RuleARN": "arn:aws:xray:us-east-1:999999999999:sampling-rule/test", "RuleName": "test", "ServiceName": "*", "ServiceType": "*", "URLPath": "*", "Version": 1}]'} 88665a53c0dd:sampler jjllee$ ``` By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice. --------- Co-authored-by: Prashant Srivastava <[email protected]>
harrryr
added a commit
that referenced
this pull request
Jul 31, 2024
*Issue description:* To resolve issues where downloading dependencies caused failures in E2E workflows due to transient issues, we began running E2E workflows in a docker image container with those dependencies already installed in this [PR](#33). As we started scaling our canary to encompass more platforms and regions however, the public ECR storing the image started throttling. Despite efforts to mitigate this throttle by increasing the API limit and also distributing the image to multiple public ECRs, the throttling still occurs and we are currently unable to determine the precise reason why. After discussion, we have decided to revert from using image containers and explore other solutions to mitigate these transient issues such as caching. *Description of changes:* Stop using image containers in the github runners and add back the retry logic to install Terraform and other dependencies. Test run: https://github.com/aws-observability/aws-application-signals-test-framework/actions/runs/10171443800 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
The e2e canary occasionally fails while downloading packages (Terraform, Gradlew, Eksctl) due to transient issues. Currently we have wrapped those packages with retries, but they are still causing issues that is triggering the canary to fail.
Description of changes:
Rather than attempting to download the packages during every canary run, we will build a docker image that already has all the necessary packages installed. This docker image will be built using a dispatch workflow, and stored in the github container registry (Can see under package tab) The canary will run in the docker container, so this should prevent issues such as gradlew failing to build successfully.
Test run: https://github.com/aws-observability/aws-application-signals-test-framework/actions/runs/8525187366
Another test run: https://github.com/aws-observability/aws-application-signals-test-framework/actions/runs/8604239069
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.