Crash observed when updating beyond 1.11.522

[hqho]

Describe the bug

When moving beyond version 1.11.522 we observed the crash in our application.

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

Works on 1.11.522

Current Behavior

Starting test: Create S3 interface with default implementation
s2n_init() failed: 469762137 (FIPS mode is not supported for the libcrypto)
Fatal error condition occurred in /home/jenkins/aws-sdk-cpp/crt/aws-crt-cpp/crt/aws-c-io/source/s2n/s2n_tls_channel_handler.c:233: 0 && "s2n_init() failed"
Exiting Application

Reproduction Steps

# cat /etc/os-release
NAME="SLES"
VERSION="15-SP4"
VERSION_ID="15.4"
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP4"
ID="sles"
ID_LIKE="suse"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:15:sp4"
DOCUMENTATION_URL="https://documentation.suse.com/"

# openssl version
OpenSSL 1.1.1l  24 Aug 2021 SUSE release 150400.7.48.1

# cat /proc/sys/crypto/fips_enabled
1

Possible Solution

No response

Additional Information/Context

No response

AWS CPP SDK version used

1.11.548

Compiler and Version used

9.3.1

Operating System and version

SLES 15SP4

[SergeyRyabinin]

Hello, thank you for this report!
It seems to be an issue in AWS CRT library, CRT team is checking it.

In the meantime,
could you please check what OpenSSL is being used during the cmake config and build stages, and at runtime? Is it the same library?
Do you use any specific cmake config other than defaults?
Also, do you perform cmake install (so that RPATH is removed from the artifacts).

Thanks.

[hqho]

Yes, I confirmed that we are using openssl 1.1.1l for cmake config build stage and runtime.
I am using the defaults for camke config and also using cmake install

[DmitriyMusatkin]

you have fips enabled on the system and openssl 1.1.1 does not support fips as indicated by the "s2n_init() failed: 469762137 (FIPS mode is not supported for the libcrypto)" error.

you either need to disable fips or build against libcrypto that supports fips

[hqho]

Is this a new restriction with the newer version of the SDK? We didn't run into this issue with 1.11.522?

[DmitriyMusatkin]

hard to tell. 1.1.1 never worked with fips, it might not have failed before and instead reverted to no fips mode, which is arguably a worse behavior than error. https://github.com/aws/s2n-tls/releases they own the check and there has been a bunch of fips related changes in the last couple months, so maybe they changed the error.

[SergeyRyabinin]

Hi,
I did not notice that the issue was about OpenSSL 1.1.1 and FIPS enabled environment, so I had to involve CRT team.

It seems that CRT crypto submodules are working as designed and trying to respect the FIPS enabled mode in an unsupported environment.
You might want to disable FIPS mode (if it is okay in your environment / config), or use libcrypto supporting the FIPS.

Best regards,
S

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

[hqho]

I am not sure why is close. However we have a flavor of openssl 1.1.1, which is fips 140-2 complaint. This check is preventing us from updating to the newer version of the SDK. Please refer to this link. https://www.atsec.com/how-can-openssl-survive-fips-140-2-validation-in-2020

These distributions took the OpenSSL 1.1.1 standard code base and forward-ported all FIPS 140-2 patches. The resulting OpenSSL 1.1.1 versions provided by the mentioned distributions attempt to be API (Application Program Interface) and ABI (Application Binary Interface) compatible to the standard OpenSSL 1.1.1 library and yet utlize FIPS 140-2 compliant cipher implementations. When using an OpenSSL version from these distributions, applications may hardly need any changes when they already link to the standard OpenSSL version 1.1.1.

[SergeyRyabinin]

Hello,

I am not sure why is close.

I closed this issue because this is no-op issue for the AWS SDK for CPP.
The reason for the crash observed is the enforced check for FIPS compliance done by the s2n.

The AWS SDK and/or CRT cannot tamper with crypto to "just make it work", there are more things to be considered in FIPS mode to claim compliance, such as but not limited to libcrypto certification.
At this moment, s2n does not recognize your libcrypto as FIPS compliant/certified.
The SDK team would like to suggest you to:

• use AWS-LC libcrypto, it is fips certified, you can switch to aws-lc by specifying "-DUSE_OPENSSL=OFF" cmake argument;
• provide your own FIPS-compliant libcrypto recognized by s2n;
• request s2n to recognize your libcrypto as FIPS-compliant. Note: I have submitted such issue on s2n: S2N does not recognize a version of openssl 1.1.1 as fips 140-2 complaint s2n-tls#5270

Best regards,
S