ripple-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brent Lintner <brent.lint...@gmail.com>
Subject Re: Test Coverage Tooling in Ripple
Date Tue, 22 Jan 2013 21:51:53 GMT
Ok, So I DO apologize for the delayed reply (and submission of the code).
This has not been priority for me (and has been on my own time, for the
most part), and I also found a bug after posting that I managed to fix
in-between now and then.

Dan, definitely makes sense to have both node and browser based runners (I
do love my nodejs, lol..). In retrospect, I think I was mostly concerned
about using the tool in at least one place (which is sufficient to gain the
insight I, at least, desired). Although the fact that a small amount of
tests only run in the browser runner is something to address, too, as they
are not covered (hopefully sooner than later). :-(

Brian, appreciate mentioning your experience and insight (personally). I
definitely do not disagree with what you are saying. From my (small)
experience using it on my side project, I've come to see code coverage as
just another tool in one's tool belt to gain insight and understanding.
i.e. Non 100% coverage for me should cause me to ask *why* that code is not
running (and if it is covered into another type of test, such as higher
level system testing), and if it ultimately *should* be running (and under
test), vs blatantly demanding 100% coverage (and ending up with crap like
what you mentioned). Either way, I have yet to use it in a large project,
myself..

PS: I was not aware of the concept of fuzzy testing, although I am now, at
least in (basic) theory (good to know- thanks). :-)

Also, FYI (all): Here is the Pull Request for that feature ->
https://github.com/blackberry/Ripple-UI/pull/703

On Fri, Jan 4, 2013 at 4:11 PM, Brian LeRoux <b@brian.io> wrote:

> A tale of caution. In a past life I once advocated strongly for
> complete coverage on a project. We made it a standard part of the
> process for each feature 'definition of done'. Eventually, bullshit
> tests snuck in because of this, and, worse, the suite was terribly
> slow to run and only getting bigger. The experience left me feeling
> that coverage is a bit of a ceremony if its not being taken super
> seriously. (Later we used a fuzzing tool to help shake out bugs and
> write good/better tests.) It was shitty maintenance that never really
> contributed to code quality, and because of bullshit tests, in some
> ways it created LESS quality and harmed our certainty of the suite
> itself.
>
> These days I tend to think that one should write tests to help you
> write code but only the bare minimum to exercise intent. (And
> obviously always write a test for every bug filed so no regressions
> manifest.) Testing is good and maybe coverage useful to understand the
> weaker areas of code but I would avoid making coverage itself a goal.
>
> $0.02
>
> On Fri, Jan 4, 2013 at 11:18 AM, Gord Tanner <gtanner@gmail.com> wrote:
> > I have been wanting code coverage forever
> >
> > +1
> >
> >
> > On Fri, Jan 4, 2013 at 2:10 PM, Filip Maj <fil@adobe.com> wrote:
> >
> >> Cool stuff Brent!
> >>
> >> +1
> >>
> >> On 1/4/13 10:46 AM, "Dan Silivestru" <dan.silivestru@gmail.com> wrote:
> >>
> >> >I'm 100% behind having some sort of code coverage solution in place.
> >> >Simply
> >> >counting the number of assertions we have doesn't give us as much
> >> >confidence as I would like to have.
> >> >
> >> >I do somewhat disagree that the node tests are more important the the
> >> >browser tests. I would at a minimum put them on par since Ripple does
> run
> >> >in the browser :-) But I think starting with code coverage for node
> only
> >> >is
> >> >a very good first step.
> >> >
> >> >So... long way of me saying... +1  :)
> >> >
> >> >
> >> >On Fri, Jan 4, 2013 at 1:19 PM, Brent Lintner
> >> ><brent.lintner@gmail.com>wrote:
> >> >
> >> >> Hey all,
> >> >>
> >> >> So, I have been using a (recently) new project called CoverJS in one
> of
> >> >>my
> >> >> personal (side projects), and I am finding it really useful and easy
> to
> >> >> use/setup when it comes to test code coverage in JS.
> >> >>
> >> >> https://npmjs.org/package/coverjs
> >> >>
> >> >> My proposal is to add support for test code coverage to Ripple (as
> test
> >> >> coverage is something I've really wanted to see go into the
> development
> >> >> workflow of Ripple). It is still in some early stages, but I think
it
> >> >>would
> >> >> be a great project to adopt (even initially) as the code coverage
> >> >>tooling
> >> >> for this project. If it it needs to be changed, it should not be too
> >> >> difficult to rip out or replace, and this does not affect the normal
> >> >>way of
> >> >> running tests.
> >> >>
> >> >> I.e. Check it out in my fork (first and only commit) -->
> >> >> https://github.com/brentlintner/Ripple-UI/tree/test.cov
> >> >>
> >> >> Since I had already done the setup in my side project, it was quite
> >> >>easy to
> >> >> get it working in Ripple (although I had to wait to submit it until
> an
> >> >> upstream bug was fixed in CoverJS). The only pitfall here is it
> >> >>currently
> >> >> only works when running the tests with the nodejs runner (vs the
> browser
> >> >> based test runner, which, IMO is less primary than the node runner,
> >> >> anyways). However, it is still very useful when testing (even after
> >> >>using
> >> >> it a few times).
> >> >>
> >> >> Thoughts? Yay/Nay?
> >> >>
> >> >> I was hoping to issue a Pull Request soon (if it is a welcomed idea).
> >> >>:-)
> >> >>
> >> >> --
> >> >> Brent
> >> >>
> >> >
> >> >
> >> >
> >> >--
> >> >Dan Silivestru
> >> >+1 (519) 589-3624
> >>
> >>
>



-- 
Brent

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message