From dev-return-142-apmail-ripple-dev-archive=ripple.apache.org@ripple.incubator.apache.org Tue Jan 22 21:52:21 2013 Return-Path: X-Original-To: apmail-ripple-dev-archive@minotaur.apache.org Delivered-To: apmail-ripple-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 945F2E01F for ; Tue, 22 Jan 2013 21:52:21 +0000 (UTC) Received: (qmail 86376 invoked by uid 500); 22 Jan 2013 21:52:21 -0000 Delivered-To: apmail-ripple-dev-archive@ripple.apache.org Received: (qmail 86352 invoked by uid 500); 22 Jan 2013 21:52:21 -0000 Mailing-List: contact dev-help@ripple.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ripple.incubator.apache.org Delivered-To: mailing list dev@ripple.incubator.apache.org Received: (qmail 86343 invoked by uid 99); 22 Jan 2013 21:52:21 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 Jan 2013 21:52:21 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of brent.lintner@gmail.com designates 209.85.214.173 as permitted sender) Received: from [209.85.214.173] (HELO mail-ob0-f173.google.com) (209.85.214.173) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 Jan 2013 21:52:15 +0000 Received: by mail-ob0-f173.google.com with SMTP id dn14so431103obc.32 for ; Tue, 22 Jan 2013 13:51:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=Q76zfjOsVZNvMfTJIFes1rtCQr2w/pZyEC+ux9uMhsQ=; b=k6KVqcVwhTStRpoPjvbfr9KDTwo1x9jfvBDY86mBYtpD6YHAycenHE92QNC/DB7qZC C4SruOgNI109JfGVZ+bjhesxY9cqO7VKYh3agRteQ4Zc350wnaK8JdLM/v0DjzswtbzD o/e/ylq5A/VmhKYTSViALjolxSOjmCrwFaiyLT/JcXX8CMveqS3QoV6GRz1Fcu+nwRi6 bJAdkN88sTsigD3E7VPjMMoB8/bkZUUz7Jv0m4ql95EN6i6x1UV3zsHYjg9nxHHtUv1R UMdg+t81IJwnFCzfrftnbHs54+oxncNP8ADQmhPHsv43/rtqnG5tgRSWN2eDd1Q81LOg Fq0Q== MIME-Version: 1.0 X-Received: by 10.60.29.193 with SMTP id m1mr17609939oeh.36.1358891514062; Tue, 22 Jan 2013 13:51:54 -0800 (PST) Received: by 10.76.34.232 with HTTP; Tue, 22 Jan 2013 13:51:53 -0800 (PST) In-Reply-To: References: Date: Tue, 22 Jan 2013 16:51:53 -0500 Message-ID: Subject: Re: Test Coverage Tooling in Ripple From: Brent Lintner To: dev@ripple.incubator.apache.org Content-Type: multipart/alternative; boundary=e89a8ff242a1fbe85304d3e79583 X-Virus-Checked: Checked by ClamAV on apache.org --e89a8ff242a1fbe85304d3e79583 Content-Type: text/plain; charset=UTF-8 Ok, So I DO apologize for the delayed reply (and submission of the code). This has not been priority for me (and has been on my own time, for the most part), and I also found a bug after posting that I managed to fix in-between now and then. Dan, definitely makes sense to have both node and browser based runners (I do love my nodejs, lol..). In retrospect, I think I was mostly concerned about using the tool in at least one place (which is sufficient to gain the insight I, at least, desired). Although the fact that a small amount of tests only run in the browser runner is something to address, too, as they are not covered (hopefully sooner than later). :-( Brian, appreciate mentioning your experience and insight (personally). I definitely do not disagree with what you are saying. From my (small) experience using it on my side project, I've come to see code coverage as just another tool in one's tool belt to gain insight and understanding. i.e. Non 100% coverage for me should cause me to ask *why* that code is not running (and if it is covered into another type of test, such as higher level system testing), and if it ultimately *should* be running (and under test), vs blatantly demanding 100% coverage (and ending up with crap like what you mentioned). Either way, I have yet to use it in a large project, myself.. PS: I was not aware of the concept of fuzzy testing, although I am now, at least in (basic) theory (good to know- thanks). :-) Also, FYI (all): Here is the Pull Request for that feature -> https://github.com/blackberry/Ripple-UI/pull/703 On Fri, Jan 4, 2013 at 4:11 PM, Brian LeRoux wrote: > A tale of caution. In a past life I once advocated strongly for > complete coverage on a project. We made it a standard part of the > process for each feature 'definition of done'. Eventually, bullshit > tests snuck in because of this, and, worse, the suite was terribly > slow to run and only getting bigger. The experience left me feeling > that coverage is a bit of a ceremony if its not being taken super > seriously. (Later we used a fuzzing tool to help shake out bugs and > write good/better tests.) It was shitty maintenance that never really > contributed to code quality, and because of bullshit tests, in some > ways it created LESS quality and harmed our certainty of the suite > itself. > > These days I tend to think that one should write tests to help you > write code but only the bare minimum to exercise intent. (And > obviously always write a test for every bug filed so no regressions > manifest.) Testing is good and maybe coverage useful to understand the > weaker areas of code but I would avoid making coverage itself a goal. > > $0.02 > > On Fri, Jan 4, 2013 at 11:18 AM, Gord Tanner wrote: > > I have been wanting code coverage forever > > > > +1 > > > > > > On Fri, Jan 4, 2013 at 2:10 PM, Filip Maj wrote: > > > >> Cool stuff Brent! > >> > >> +1 > >> > >> On 1/4/13 10:46 AM, "Dan Silivestru" wrote: > >> > >> >I'm 100% behind having some sort of code coverage solution in place. > >> >Simply > >> >counting the number of assertions we have doesn't give us as much > >> >confidence as I would like to have. > >> > > >> >I do somewhat disagree that the node tests are more important the the > >> >browser tests. I would at a minimum put them on par since Ripple does > run > >> >in the browser :-) But I think starting with code coverage for node > only > >> >is > >> >a very good first step. > >> > > >> >So... long way of me saying... +1 :) > >> > > >> > > >> >On Fri, Jan 4, 2013 at 1:19 PM, Brent Lintner > >> >wrote: > >> > > >> >> Hey all, > >> >> > >> >> So, I have been using a (recently) new project called CoverJS in one > of > >> >>my > >> >> personal (side projects), and I am finding it really useful and easy > to > >> >> use/setup when it comes to test code coverage in JS. > >> >> > >> >> https://npmjs.org/package/coverjs > >> >> > >> >> My proposal is to add support for test code coverage to Ripple (as > test > >> >> coverage is something I've really wanted to see go into the > development > >> >> workflow of Ripple). It is still in some early stages, but I think it > >> >>would > >> >> be a great project to adopt (even initially) as the code coverage > >> >>tooling > >> >> for this project. If it it needs to be changed, it should not be too > >> >> difficult to rip out or replace, and this does not affect the normal > >> >>way of > >> >> running tests. > >> >> > >> >> I.e. Check it out in my fork (first and only commit) --> > >> >> https://github.com/brentlintner/Ripple-UI/tree/test.cov > >> >> > >> >> Since I had already done the setup in my side project, it was quite > >> >>easy to > >> >> get it working in Ripple (although I had to wait to submit it until > an > >> >> upstream bug was fixed in CoverJS). The only pitfall here is it > >> >>currently > >> >> only works when running the tests with the nodejs runner (vs the > browser > >> >> based test runner, which, IMO is less primary than the node runner, > >> >> anyways). However, it is still very useful when testing (even after > >> >>using > >> >> it a few times). > >> >> > >> >> Thoughts? Yay/Nay? > >> >> > >> >> I was hoping to issue a Pull Request soon (if it is a welcomed idea). > >> >>:-) > >> >> > >> >> -- > >> >> Brent > >> >> > >> > > >> > > >> > > >> >-- > >> >Dan Silivestru > >> >+1 (519) 589-3624 > >> > >> > -- Brent --e89a8ff242a1fbe85304d3e79583--