lucenenet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shad Storhaug <s...@shadstorhaug.com>
Subject Debugging Help Requested (for Beta Release)
Date Tue, 12 Sep 2017 19:17:34 GMT
Vincent (and/or anyone else willing to help out),

I have managed to get the projects upgraded to the new .csproj format and Visual Studio 2017,
but there have been many challenges getting the tests to run all the way through without crashing,
and I have finally managed to locate the tests that were causing the crashes (in Release mode,
anyway) and manually fail them. This was a temporary fix to get by long enough to setup TeamCity
so it works with the new configuration, which is now in place.

During my search for an answer as to why the test runner crashes, I discovered two causes:


1.       On .NET Standard 2.0, there is a bug that crashes the test runner when Debug.Assert
evaluates to false. See: https://github.com/Microsoft/vstest/issues/1022

2.       Some of the concurrency tests have been converted literally from Java and in those
tests there are either throw statements or NUnit asserts that are causing uncaught exceptions
on background threads.

The second issue wasn't registering as a "problem" because of Microsoft's decision to make
the default behavior in .NET 4.5 ignore uncaught exceptions (https://blogs.msdn.microsoft.com/pfxteam/2011/09/28/task-exception-handling-in-net-4-5/).
Essentially, because in .NET 4.5 the ThrowUnobservedTaskExceptions setting is false on our
test framework, these exceptions are being ignored. In Java, these would have been marshalled
to the main thread causing a failure, meaning we are getting false positives. In .NET Core
2.0, these exceptions are now causing the foreground thread to crash (as one might expect),
so we can now no longer ignore these issues.

However, there is another snag. We are getting more test failures with the new tooling than
we did with the old tooling. And these tests do not fail in Visual Studio with the new tooling.
Although ultimately I would like to get us onto the new tooling across .NET Framework and
.NET Core, for the time being I have reverted the .NET Framework side back to using the NUnit
3 Console (as we had previously) and sure enough those additional test failures went away
again. I suspect the tests (and perhaps even the software) have more concurrency bugs that
are now being revealed by the new .NET Core 2.0 SDK tooling because of a different approach
to concurrency.
See the comparison between 4.8.0-ci0000001123, 4.8.0-ci0000001126, 4.8.0-ci0000001127 that
had a lot of failures on the .NET Core 2.0 and 1.0.4 SDKs (dotnet test), and 4.8.0-ci0000001134
& 4.8.0-ci0000001136 on NUnit 3 Console: https://teamcity.jetbrains.com/viewType.html?buildTypeId=LuceneNet_PortableBuilds_TestOnNet451
(note you can login as a guest to see these)

We are also seeing higher numbers of failures on .NET Core 1.0 and 2.0:

https://teamcity.jetbrains.com/viewType.html?buildTypeId=LuceneNet_PortableBuilds_TestOnDotNetCore10
https://teamcity.jetbrains.com/viewType.html?buildTypeId=LuceneNet_PortableBuilds_TestOnDotNetCore20

Anyway, back to the tests that I "fixed" to make the test runner not crash, which need some
work:

https://github.com/apache/lucenenet/commit/40c42ced8127d61a9e066548ceb54d03868e54a0
https://github.com/apache/lucenenet/commit/62333ec568e3989e73dee65464e89d85c44acb82
https://github.com/apache/lucenenet/commit/b7ab123bed3ce084c1fa0e871f44045c2d7416af
https://github.com/apache/lucenenet/commit/fa94509d3f5b2daea6f2d812112b905e66a60595
https://github.com/apache/lucenenet/commit/60faa9cdcbc9b0e37a2472f5d3cc5af76c96c82a

The first three are the most concerning. I am not so worried about having to remove Debug.Assert
to get them to work for the moment, as in production they won't be there anyway.

Also of concern are the "file locked by another process" errors we are now getting pretty
much anywhere IndexWriter is used. I will need to confirm, but I don't believe we were getting
these errors in the last release and these don't appear to be an issue with the tests. It
would be preferable to have a fix rather than having to revert something in order to eliminate
these errors.

I have added some instructions to the README (https://github.com/apache/lucenenet#visual-studio)
on how to run tests and switch between target frameworks within Visual Studio 2017.

Do note that changing target frameworks doesn't go very smoothly and often requires a restart
of Visual Studio 2017 (either the tests are not discovered, or there is a "build failed" error
that doesn't show any corresponding errors in the list). I am hoping the tooling in Visual
Studio 2017 will improve eventually and allow switching the framework to test on in the UI
as well as perform better, but this is what we have for the time being.

Anyway, I would appreciate some assistance getting the test failures under control so we can
finally release all of these new features, patches, and performance improvements that have
been contributed by the team.

Thanks,
Shad Storhaug (NightOwl888)

P.S. Vincent - you will be pleased to know that the [Debuggable] attribute has now been removed
and your advice of using [MethodImpl(MethodImplOptions.NoInlining)] worked great for the tests
that require method names in the stack trace (https://github.com/apache/lucenenet/commit/4abbd4be4bf65c83562cac5870c802b6e31abf64).

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message