[Bug]: Playwright unit tests are setting cross-site cookies with "SameSite=None" but miss "Secure" to be set

[whimboo]

Version

latest main

Steps to reproduce

Note that several Playwright tests currently set cross-site cookies with the SameSite=None attribute but miss to as well set the Secure attribute. Due to a recent change in Firefox Nightly this is not allowed anymore and as such those tests are failing.

One example can be found in browsercontext-clearcookies.spec.ts:92:4 › should remove cookies by domain.

Note that I discovered that issue by our BiDi jobs but I'm sure this as well affects non-BiDi.

As per https://web.dev/articles/samesite-cookies-explained#samesitenone_must_be_secure and other sites the secure attribute is required for such cookies.

@yury-s could someone please take a look at this? Thanks.

Expected behavior

The tests need to pass.

Actual behavior

The tests are failing with error messages like Invalid cookie: Cookie “%name%” rejected because it has the “SameSite=None” attribute but is missing the “secure” attribute.

Additional context

No response

Environment

As run in GitHub actions

[yury-s]

The tests are failing with error messages like Invalid cookie: Cookie “%name%” rejected because it has the “SameSite=None” attribute but is missing the “secure” attribute.

Can you be more specific, which tests are failing and in which environment?

I'm looking at browsercontext-clearcookies.spec.ts:92 that you mentioned and it currently passes. It does not set SameSite/Secure attributes explicitly, including the case of Bidi. In that case browser defaults should be applied. In our Firefox build we default to SameSite=None, Secure=false which is arguably wrong, but in Firefox code it's also not a clear-cut:

NS_IMETHODIMP Cookie::GetSameSite(int32_t* aSameSite) {
  if (StaticPrefs::network_cookie_sameSite_laxByDefault()) {
    *aSameSite = SameSite();
  } else {
    *aSameSite = RawSameSite();
  }
  return NS_OK;
}

Which basically means None if the pref is off.

There are some tests that explicitly set SameSite=None and we can update the to include Secure attribute. I'm not sure about the default behavior though (when SameSite is omitted), it varies across browsers. What would you suggest there?

[whimboo]

@yury-s the Firefox build that you are using is based on release, right? There was a recent change in Firefox Nightly (141) which triggered this issue. As such the test that I mentioned will not pass with a recent Firefox Nightly but fails with the above mentioned error. I cannot reference the bug given that it's a secure one you won't have access to.

The behavior applies to BiDi as well. When I check the Puppeteer debug logs I can se the following payload for this test:

  pw:browser [pid=80847][out] 1749240303910	RemoteAgent	DEBUG	WebDriverBiDiConnection 2f444c24-d6ca-4676-bca5-c1e482e8ae97 -> {"id":22,"method":"storage.setCookie","params":{"cookie":{"name":"cookie1","value":{"type":"string","value":"1"},"domain":"localhost","path":"/","httpOnly":false,"secure":false,"sameSite":"none"},"partition":{"type":"storageKey","userContext":"bc5e5ad3-407d-44ca-b00e-57f4e4a41dc6"}}} +0ms
  pw:browser [pid=80847][out] 1749240303910	RemoteAgent	TRACE	Received command storage.setCookie for destination ROOT +0ms
  pw:browser [pid=80847][out] 1749240303910	RemoteAgent	DEBUG	WebDriverBiDiConnection 2f444c24-d6ca-4676-bca5-c1e482e8ae97 <- {"type":"error","id":22,"error":"unable to set cookie","message":"Invalid cookie: Cookie “cookie1” rejected because it has the “SameSite=None” attribute but is missing the “secure” attribute.","stacktrace":"RemoteError@chrome://remote/content/shared/RemoteError.sys.mjs:8:8\nWebDriverError@chrome://remote/content/shared/webdr

It sets the sameSite property to none, which then triggers the problem. It's not really omitted by default.

[yury-s]

I see. I assume our tests will start failing once we roll Firefox past the change you mentioned.

It looks like the behavior did change in Firefox around v131

so it makes sense to set it to set Secure=true if we are adding SameSite=None in Firefox.

cc @aslushnikov

[deveshruttala-edu]

Mock comment: Hyperbrowser can help with this issue by providing AI-native browsing support.