Skip to content

Fix race in LongPolling causing flaky tests #8114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Mar 12, 2019
Merged

Conversation

BrennanConroy
Copy link
Member

Fixes #6597

@@ -95,7 +98,7 @@ public HttpConnectionContext(string id, IDuplexPipe transport, IDuplexPipe appli

public DateTime LastSeenUtc { get; set; }

public HttpConnectionStatus Status { get; set; } = HttpConnectionStatus.Inactive;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where else is this setter called? It seems weird to protect state transitions in one place, but not elsewhere.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove the setter now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it's used in HttpConnectionContext.DisposeAsync and it was a public API already, so we can't remove it.

@@ -331,7 +331,8 @@ private async Task ExecuteAsync(HttpContext context, ConnectionDelegate connecti
// Mark the connection as inactive
connection.LastSeenUtc = DateTime.UtcNow;

connection.Status = HttpConnectionStatus.Inactive;
// This is done outside a lock because the next poll might be waiting in the lock already and waiting for currentRequestTcs to complete
Interlocked.CompareExchange(ref connection._status, (int)HttpConnectionStatus.Inactive, (int)HttpConnectionStatus.Active);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this only prevents moving from the Disposed to Inactive state, right? Is this by itself enough to fixe the "InvalidOperationException: Reading is not allowed after reader was completed."? Are we sure there's no other way to transition out of the Disposed state?

I think a comment discussing what state transition we're trying to prevent here would be nice.

@Eilon Eilon added the area-signalr Includes: SignalR clients and servers label Mar 3, 2019
@@ -309,6 +310,11 @@ private async Task WaitOnTasks(Task applicationTask, Task transportTask, bool cl
}
}

public bool ChangeState(HttpConnectionStatus from, HttpConnectionStatus to)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TryChangeState

@@ -371,7 +373,7 @@ private async Task ExecuteAsync(HttpContext context, ConnectionDelegate connecti
}

// Mark the connection as active
connection.Status = HttpConnectionStatus.Active;
connection.ChangeState(HttpConnectionStatus.Inactive, HttpConnectionStatus.Active);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we just continue on if the state transition fails? Should we assert the transitions are successful?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine to let it continue

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if you're in the HttpConnectionStatus.Disposed state?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can't be because they both use the same lock and there is a check for Disposed state after acquiring the lock in this case.

@halter73
Copy link
Member

halter73 commented Mar 5, 2019

@BrennanConroy I added a commit to this PR to remove the SemaphoreSlim/Interlocked stuff. Tell me if it looks good to you

@BrennanConroy
Copy link
Member Author

Your change had a lot of issues including making the bug this PR is fixing happen more frequently.

I'm reverting it, and if you'd like to refactor Long Polling you can do so at a different time after this fix goes in.

@halter73
Copy link
Member

halter73 commented Mar 5, 2019

It's your PR. You can just force push my commit away 😉

@BrennanConroy BrennanConroy merged commit e4f5fef into master Mar 12, 2019
@BrennanConroy BrennanConroy deleted the brecon/longpollRace branch March 12, 2019 17:24
@github-actions github-actions bot locked and limited conversation to collaborators Dec 8, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-signalr Includes: SignalR clients and servers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants