Tests failing under MSBuild and NCover Explorer 3.4

Question

Tests failing under MSBuild and NCover Explorer 3.4

Hi

I'm trying to run my tests using NCover Explorer and all tests where mocking is used are failing because mocks simple don't kick in. After spending a few days on googling I tried:

1. Run NCover explorer using TMockRunner
"C:Program Files (x86)TypemockIsolator.0TMockRunner.exe" -first -link NCover3.0 "C:Program Files (x86)NCoverNCover.Explorer.exe"

2. Run the explorer normally and then run tests using TMockRunner
Path to application to profile:
C:Program Files (x86)TypemockIsolator.0TMockRunner.exe
Arguments:
-first -link NCover3.0 "C:Program Files (x86)Microsoft Visual Studio 10.0Common7IDEMSTest.exe" /noisolation /testcontainer:"ClientTestsinDebugSymantec.Applications.AuthenticationManager.Clients.Tests.dll"

3. Attempted to get any luck using console:
TMockRunner.exe -first -link NCover3.0 "c:Program Files (x86)NCoverNCover.Console.exe" "C:Program Files (x86)M
icrosoft Visual Studio 10.0Common7IDEMSTest.exe" /noisolation /testcontainer:"d:CodeApplicationsAuthenticationManagerClientTestsinDebugSymantec.Appl
ications.AuthenticationManager.Clients.Tests.dll"

4. Did above calling console directly:
"c:Program Files (x86)NCoverNCover.Console.exe" "C:Program Files (x86)Microsoft Visual Studio 10.0Common7ID
MSTest.exe" /noisolation /testcontainer:"d:CodeApplicationsAuthenticationManagerClientTestsinDebugSymantec.Applications.AuthenticationManager.Clients
Tests.dll"

5. Tries to run just MSTest:
"C:Program Files (x86)Microsoft Visual Studio 10.0Common7IDEMSTest.exe" /noisolation /testcontainer:"ClientTe
stsinDebugSymantec.Applications.AuthenticationManager.Clients.Tests.dll"

All no luck. Test with mocks are failing. Full stop. How can I force this to work? Test are failing like mocking framework does not exist. Mocked properties are null etc. No exceptions, no sign that something actually is wrong, just tests failing. The same tests are perfectly fine when executed in VS2010.

Help?

asked Feb 4, 2011 by jimmy_skowronski (680 points)

5 Answers

Answer 1 · 2011-02-04T11:21:49+0000

NCover Explorer is used to view coverage results gathered by running NCover through other means, like NCover.Console.exe.

The very easiest way to get tests running with Typemock and NCover is to use the provided build tasks. Running things manually by using TMockRunner.exe is possible, but 99% of the time you really don't need to.

I have a blog post that uses Typemock, NCover and NUnit in MSBuild to run tests. You can swap running MSTest for NUnit, the principle is the same. Note that the blog post was written in 2008, so the NCover build tasks are just SLIGHTLY different. Not much, but a little, so you'll have to adjust. Again, though, the principle should hold.

Here's that post: http://www.paraesthesia.com/archive/200 ... build.aspx

Answer 2 · 2011-02-07T04:27:52+0000

Thanks for the answer, however tests are still failing when run outside of Visual Studio.

Answer 3 · 2011-02-07T10:56:15+0000

I understand that - the blog post I provided you was specifically for running the tests outside of Visual Studio.

If there are no exceptions, there is possibly something else going on. Generally when you try to run tests and Typemock Isolator isn't enabled when it should be, you'll get an exception that actually says something to the effect of "Typemock Isolator isn't enabled." (It's a long stack trace, but that's the message associated.)

Sometimes it's hard to troubleshoot stuff like this in a larger environment because there are so many moving pieces.

One thing that might be a good idea is to try to set up a very small sample project that doesn't attach to your production code and see if you can get that working. Basically File -> New Project and create a very simple class library with like one class and one method, and then create a unit test that mocks something. See if you can get that working in both environments. If you can't, that might indicate an environmental issue (something on your machine). If you can get it working, see if you can apply the things you learned there to your larger/production project.

Beyond that, you might have to create a small reproduction with like one real/production test that works in VS and fails outside VS and either post that here (along with the code it's testing) so the community can help you or send it to Typemock support so they can. If you do post (or send in) a repro, make sure to include your build script so we/they can see how you're starting up Isolator, NUnit, etc. from outside VS.

Also include information on how you're running your tests inside Visual Studio. TestDriven.NET? CodeRush/Refactor? Resharper? Visual Studio Test Runner?

Answer 4 · 2011-02-08T05:30:19+0000

Hi

I did a very simple test. Class:

  public class Car
  {
    private int ServiceDistance = 15000;
    public int Mileage { get; set; }

    public int DistanceToService()
    {
      return Mileage % ServiceDistance;
    }
  }

Two unit tests:

    [TestMethod]
    public void NotMockedTest()
    {
      Car myCar = new Car();
      myCar.Mileage = 35000;
      int distance = myCar.DistanceToService();
      Assert.AreEqual(5000, distance);
    }

    [TestMethod]
    [Isolated]
    public void MockedTest()
    {
      Car myCarFake = Isolate.Fake.Instance<Car>(Members.CallOriginal);
      Isolate.WhenCalled(() => myCarFake.Mileage).WillReturn(40000);
      int distance = myCarFake.DistanceToService();
      Assert.AreEqual(10000, distance);
    }

Results:
Visual Studio 2010
Tests run by "Run All Tests in Solution" option. Both tests passed.

MSTest.exe

"C:Program Files (x86)Microsoft Visual Studio 10.0Common7IDEMSTest.exe" /noisolation /testcontaine
r:"TypeMockTest.TestsinDebugTypeMockTest.Tests.dll"

Results               Top Level Tests
-------               ---------------
Failed                TypeMockTest.Tests.CarTest.MockedTest
Passed                TypeMockTest.Tests.CarTest.NotMockedTest

MSTest.exe using TMockRunner.exe

"c:Program Files (x86)TypemockIsolator.0TMockRunner.exe" "C:Program Files (x86)Microsoft Visual
 Studio 10.0Common7IDEMSTest.exe" /noisolation /testcontainer:"TypeMockTest.TestsinDebugTypeMockTest.Tests.dll"

Results               Top Level Tests
-------               ---------------
Passed                TypeMockTest.Tests.CarTest.MockedTest
Passed                TypeMockTest.Tests.CarTest.NotMockedTest

NCover.Console using TMockRunner.exe

"c:Program Files (x86)TypemockIsolator.0TMockRunner.exe" -first -link NCover3.0 "c:Program Files
 (x86)NCoverNCover.Console.exe" "C:Program Files (x86)Microsoft Visual Studio 10.0Common7IDEMSTest.exe" /noisolation /testcontainer:"TypeMockTest.Tests
inDebugTypeMockTest.Tests.dll"

Results               Top Level Tests
-------               ---------------
Passed                TypeMockTest.Tests.CarTest.MockedTest
Passed                TypeMockTest.Tests.CarTest.NotMockedTest

No data was collected
For more information on this error, see http://www.ncover.com/lt/no-data

NCover.Console is returning exit code #20000

No coverage was collected.

NCover.Console with //reg using TMockRunner.exe

"c:Program Files (x86)TypemockIsolator.0TMockRunner.exe" -first -link NCover3.0 "c:Program Files
 (x86)NCoverNCover.Console.exe" //reg "C:Program Files (x86)Microsoft Visual Studio 10.0Common7IDEMSTest.exe" /noisolation /testcontainer:"TypeMockTest.T
estsinDebugTypeMockTest.Tests.dll"

Results               Top Level Tests
-------               ---------------
Failed                TypeMockTest.Tests.CarTest.MockedTest
Passed                TypeMockTest.Tests.CarTest.NotMockedTest

Execution Time: 2.9323 s
Symbol Coverage: 61.11%
Branch Coverage: 3.38%

Coverage collected but mocked test failed.

Looks like there is a conflict between those two.

Ncover Explorer
Parameters
Path to application:
c:Program Files (x86)TypemockIsolator.0TMockRunner.exe

Arguments for the application to profile:
-first -link NCover3.0 "C:Program Files (x86)Microsoft Visual Studio 10.0Common7IDEMSTest.exe" /noisolation /testcontainer:"TypeMockTest.TestsinDebugTypeMockTest.Tests.dll"

Register NCover before running checked
Register NCover before globally checked

Mocked test fails.
Error in output:

TMockRunner.exe is returning exit code #1

Build script
Works fine. Tests are passing and coverage is collected.

Summary
I'm unable to get test coverage using NCover, neither console or explorer. Although the build script works, it is not an ideal solution. My devs are using NCover Explorer to run tests and work with coverage data and running the script makes things more painful.

Q1:
So, how I can make NEcover explorer working directly without a need to run the script. At this moment the script is the only option that works.

Q2:
I have to find out why my project tests are failing when run outside of Visual Studio. Any hints? I'm creatign mock objects in TestInitialize method. Can that be the reason?[/code]

Update
I've just noticed that I have the following error when run my project tests with MSTest using TMockRunner:

Test method Symantec.Applications.AuthenticationManager.Clients.Tests.ActiveTest.GetSaml2SecurityToken_Saml2SecurityToken_ReturnSaml2SecurityToken_Test threw exception: 
TypeMock.TypeMockException: 
*** Typemock Isolator is not currently enabled. 
To enable do one of the following:

* To run Typemock Isolator as part of an automated process you can:
   - run tests via TMockRunner.exe command line tool
   - use 'TypeMockStart' tasks for MSBuild or NAnt

* To work with Typemock Isolator inside Visual Studio.NET:
        set Tools->Enable Typemock Isolator from within Visual Studio

For more information consult the documentation (see 'Running' topic)

The command line used here is:

"c:Program Files (x86)TypemockIsolator.0TMockRunner.exe" -first "C:Program Files (x86)Microsoft Visual Studio
 10.0Common7IDEMSTest.exe" /noisolation /testcontainer:"ClientTestsinDebugSymantec.Applications.AuthenticationManager.Clients.Tests.dll"

To make it more bizarre, tests are fine when I removed /noisolation and -first parameters.

Answer 5 · 2011-02-08T14:03:46+0000

OK, I think I see what you're asking for now. Sorry if it seemed like I was a little slow on the uptake, there. I think I just needed some clarification on what you were trying to do.

First,

Next, I just posted a blog article explaining different ways to run unit tests through Isolator outside of Visual Studio:

http://www.paraesthesia.com/archive/201 ... tudio.aspx

What wasn't clear to me was that you're not using build scripts or rebuilding before a test run; you're just running the tests through the NCover Explorer project functionality (which, in turn, runs NCover.Console.exe), similar to the way people use NUnit GUI.

I recommend using build scripts since that's sort of industry standard and allows you to control the whole process in one go - clean, build, test, profile - but if you have a solution that works for you, that's cool. The blog article I posted above shows how to run NCover Explorer using an NCover Explorer "project" and clicking the "Run Coverage" button and have the unit tests execute.

I also see you got error #20000 from NCover - that basically means something got corrupted in your NCover installation. This can happen if you don't start and stop Typemock properly or if something crashes before Typemock can unlink itself. You should probably repair your NCover installation.

http://www.paraesthesia.com/archive/201 ... error.aspx

Hope that helps,
-T

Tests failing under MSBuild and NCover Explorer 3.4

Please log in or register to add a comment.

Please log in or register to answer this question.

5 Answers

Please log in or register to add a comment.

Please log in or register to add a comment.

Please log in or register to add a comment.

Please log in or register to add a comment.

Please log in or register to add a comment.

Products

Help

Company

Community

Contact

Typemock Store (US toll FREE 866–895–4680). Copyright © 2006 Typemock Inc. All rights reserved. Terms of Use.

Choose the product type:

Related questions

Tests failing under MSBuild and NCover Explorer 3.4

Please log in or register to add a comment.

Please log in or register to answer this question.

5 Answers

Please log in or register to add a comment.

Please log in or register to add a comment.

Please log in or register to add a comment.

Please log in or register to add a comment.

Please log in or register to add a comment.

Products

Help

Company

Community

Contact

Typemock Store (US toll FREE 866–895–4680). Copyright © 2006 Typemock Inc. All rights reserved. Terms of Use.

Choose the product type: