testing: async test fails if SUT calls an observable with a delay operator (fakeAsync too)

[wardbell]

I'm submitting a ... (check one with "x")

[x] bug report
[ ] feature request
[ ] support request 

Consider these two tests:

afterEach(() => { expect(actuallyDone).toEqual(true); });

// Async
it('should run async test with successful delayed Observable', async(() => {
  let actuallyDone = false;
  let source = Observable.of(true).delay(10);
  source.subscribe(
    val => {
      actuallyDone = true;
    },
    err => fail(err)
  );
}));

// FakeAsync
it('should run async test with successful delayed Observable', fakeAsync(() => {
  let source = Observable.of(true).delay(10);
  source.subscribe(
    val => {
      actuallyDone = true;
    },
    err => fail(err)
  );
  tick();
}));

Current behavior

Test 1 fails with message: Cannot use setInterval from within an async zone test
Test 2 fails with message: Error: 1 periodic timer(s) still in the queue.

In neither test does the actuallyDone value become true;

Expected/desired behavior

The test should not fail.

Reproduction of the problem

See above.

What is the expected behavior?

The test should not fail. We should allow async tests of stuff that calls setInterval. If we can't, we had better offer a message that helps the developer find likely sources of the problem (e.g, Observables).

What is the motivation / use case for changing the behavior?

I have no idea how to test a SUT with an Observable.delay() ... or any observable operator that calls setInterval.

Maybe I'm just using fakeAsync incorrectly. I'd like to know what to do.

Let's say we get that to work. Is that a solution?

I don't think so. It is generally impractical for me, the test author, to anticipate whether fakeAsync is necessary. I don't always know if the SUT (or something it uses ... like a service) makes use of setInterval.

Moreover, a test that was working could suddenly fail simple because someone somewhere modified the observable with an operator that calls setInterval. How would I know?

The message itself requires knowledge of which observables rely upon setInterval. I only guessed that delay did; it's not obvious that it should.

Please tell us about your environment:

• Angular version: 2.0.0-rc.5 (candidate - 16 July 2016)
• Browser: [ Chrome ]
• Language: [TypeScript 1.8.x ]

[juliemr]

I'd like to work on a better long term solution, but as a quick note - you can get the fakeAsync() case working by calling discardPeriodicTasks() at the end.

[choeller]

@juliemr discardPeriodicTasks() prevents the "still in queue" error, but nevertheless the Observable is not executed - so tick doesn't seem to work with Observable.delay at all.

  it('should be able to work with Observable.delay', fakeAsync(() => {
    let actuallyDone=false;
    let source = Observable.of(true).delay(10);
    source.subscribe(
      val => {
        actuallyDone = true;
      },
      err => fail(err)
    );
    tick(100);
    expect(actuallyDone).toBeTruthy(); // Expected false to be truthy.

    discardPeriodicTasks();
  }));

we are currently running into this situation for Asynchroneous tests, that don't complete even when adding multiple tick and detectChanges.

[choeller]

Are there any plans on this? as far as I understand there is currently no way to test components that use setInterval (somewhere under the hood). This seems like a really serious limitation - or am I not getting something?

[choeller]

The "funny" thing is - when I start karma in a debug window and put a breakpoint on the tick-call the test works 😳

  it('should be able to work with Observable.delay', fakeAsync(() => {
    let actuallyDone=false;
    let source = Observable.of(true).delay(10);
    source.subscribe(
      val => {
        actuallyDone = true;
      },
      err => fail(err)
    );
    tick(100); // ---------------> put breakpoint here in dev-console
    expect(actuallyDone).toBeTruthy(); // succeedes
    discardPeriodicTasks();
  }));

so there seem to be some weird things going on there... I'm using zone 0.6.15 and angular commit fc2fe00

[tomwanzek]

I have the same issue with a different use case.

An Angular 2 component that wraps some D3 magic which requires setInterval.

The behavior is exactly as described in the original bug report and I agree with @choeller that this is a serious limitation.

The only thing I noticed is, that as long as I do not run fixture.detectChanges(), I can at least validate the static parts of the component template and e.g. component member initialization done in the constructor. The setInterval is not invoked until the ngOnInit lifecyclehook is hit.

import { TestBed, async } from '@angular/core/testing';
import { TestVoronoiSpirals3Component } from './test-voronoi-spirals-3.component';

import { D3Service, D3 } from 'd3-ng2-service';

describe('Component: TestVoronoiSpirals3', () => {

  beforeEach(() => {
    TestBed.configureTestingModule({
      declarations: [
        TestVoronoiSpirals3Component
      ],
      providers: [
        D3Service
      ]
    });
  });

  it('should create the component', async(() => {
    let fixture = TestBed.createComponent(TestVoronoiSpirals3Component);
    let component = fixture.debugElement.componentInstance;
    expect(component).toBeTruthy();
  }));

  it('should have a d3 member with  relevant D3 members', async(() => {
    let fixture = TestBed.createComponent(TestVoronoiSpirals3Component);
    let component = fixture.debugElement.componentInstance;
    let d3: D3 | undefined = component.d3;
    expect(d3).toBeTruthy('No member d3 defined');
    expect(d3 && d3.voronoi && typeof d3.voronoi === 'function').toBeTruthy('Member function voronoi() of d3 not defined');
  }));

  it(`should have heading h2 with text 'Voronoi Spirals III Example'`, async(() => {
    let fixture = TestBed.createComponent(TestVoronoiSpirals3Component);
    let compiled = fixture.debugElement.nativeElement;
    expect(compiled.querySelector('h2').textContent).toBe('Voronoi Spirals III Example');
  }));

  it('should have exactly one HTMLCanvasElement with dimensions 400x400', async(() => {
    let fixture = TestBed.createComponent(TestVoronoiSpirals3Component);
    let compiled = fixture.debugElement.nativeElement;
    let canvasElements = <NodeListOf<HTMLCanvasElement>>compiled.querySelectorAll('canvas');
    expect(canvasElements.length).toBe(1, 'Incorrect number of canvas elements found');
    expect(canvasElements.length === 1 && canvasElements[0].clientWidth).toBe(400, 'Incorrect canvas width');
    expect(canvasElements.length === 1 && canvasElements[0].clientHeight).toBe(400, 'Incorrect canvas height');
  }));


});

[robertlevy]

as a workaround, I'm wrapping calls to .delay in my code with a check for the presence for jasmine. lame but effective at least for code we have control over

    if(!window.jasmine) {
        myObservable = myObservable.delay(500);
    }

[wardbell]

@choeller One thing to keep in mind ... we can fall back to jasmine.done. I intend to do that on my sample and see if I can get the job done that way.

[choeller]

@wardbell I generally prefer jasmine.done over async and fakeAsync but as far as I understand we can only utilize it, when the async part is happening in the test itself (like with whenStable). I "solved" our use case (testing a component with multiple debounceTime calls) by introducing a complex cascade of setTimeouts in the test and calling done in the deepest cascade, but that is really super hacky...

[wardbell]

Haven't tried yet but with Jasmine clock and spies might plow through it more easily. Will try again when I'm back

[kasiditi]

After my investigation, the problem lies in how RxJS' Scheduler works.
This line on delay.ts, https://github.com/ReactiveX/rxjs/blob/master/src/operator/delay.ts#L82, it does a double check that it's the time to dispatch the notification before actually dispatch the event by comparing the time to the Scheduler's now function.

For AsyncScheduler, which is the default Scheduler for delay() operator, the now function is just a normal Date function. (See https://github.com/ReactiveX/rxjs/blob/master/src/Scheduler.ts#L26-L41) This is why the test pass when you set the breakpoint in @choeller's test (#10127 (comment)). Because Scheduler use the native Date.now function, before you continue running the code again, it have passed the 10 milliseconds already. Changing the delay to 1000 seconds and the test should not pass (unless you have enough patience to wait for 1000 seconds.)

To make the fakeAsync test pass, one has to mock up the Date.now function, but using jasmine.clock().mockDate(...) might not going to work, because the Scheduler already have the native Date.now set in the now property before Jasmine's MockDate is installed, so you have to install MockDate before the Scheduler's now is set, which seems very impractical.

Another workarounds I came up with is to spy on the Scheduler.async.now function to the mocked time, so the test should be like this:

it('should run async test with successful delayed Observable', fakeAsync(() => {
    let actuallyDone = false;
    let currentTime = 0;

    spyOn(Scheduler.async, 'now').and.callFake(() => currentTime);

    let source = Observable.of(true).delay(10);
    source.subscribe(() => {
        actuallyDone = true;
    });

    currentTime = 10;
    tick(10);

    expect(actuallyDone).toEqual(true);
}));

, which is a little hacky. You might also use something like TestScheduler or VirtualTimeScheduler, which I have not used it before, so I don't know if it's going to work or not.

I think the most viable solution is to let the fakeAsync's tick() function mock the Date.now itself, which might be related to the issue #8678.

[mmc41]

This is a pretty serious and surprising limitation! All my mocked http calls using angular-in-memory-web-api apparently uses setInterval behind the scenes, so I can not use Angular's async to test any of them. If I try, the test fails with "Cannot use setInterval from within an async zone test.".

Is feels like somewhat a joke that Angular2 docs really pushes RxJs but If one actually listen and do use RxJs, than the resulting code will not easily test (using any of the angular 2 supported methods).

I had to use plain old jasmine's support for done callbacks to get tests with observables running. Quite a surprise. Much more complicated than I expected.

[JohannesRudolph]

I can confirm this is still a bug in angular 2.1.1. Pretty annoying...

[girishjjain]

Would be good to have some guidance from angular team regarding how to handle this situation with unit tests.