From dev-return-95-apmail-ripple-dev-archive=ripple.apache.org@ripple.incubator.apache.org Fri Jan 4 21:11:34 2013 Return-Path: X-Original-To: apmail-ripple-dev-archive@minotaur.apache.org Delivered-To: apmail-ripple-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8DE49E8E9 for ; Fri, 4 Jan 2013 21:11:34 +0000 (UTC) Received: (qmail 18473 invoked by uid 500); 4 Jan 2013 21:11:34 -0000 Delivered-To: apmail-ripple-dev-archive@ripple.apache.org Received: (qmail 18444 invoked by uid 500); 4 Jan 2013 21:11:34 -0000 Mailing-List: contact dev-help@ripple.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ripple.incubator.apache.org Delivered-To: mailing list dev@ripple.incubator.apache.org Received: (qmail 18432 invoked by uid 99); 4 Jan 2013 21:11:34 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Jan 2013 21:11:34 +0000 X-ASF-Spam-Status: No, hits=0.3 required=5.0 tests=FRT_ADOBE2,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of brian.leroux@gmail.com designates 209.85.220.180 as permitted sender) Received: from [209.85.220.180] (HELO mail-vc0-f180.google.com) (209.85.220.180) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Jan 2013 21:11:28 +0000 Received: by mail-vc0-f180.google.com with SMTP id p16so17112555vcq.11 for ; Fri, 04 Jan 2013 13:11:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=1XIH3+EY0o2pzZoiOUYxwDGjrArK4LG158tuadPmEDE=; b=eQ/DG/v7AU8BSnERX2AFr2z+EBQyqRSe3KUJtTKEXA5qLhMHMTHORM1L9oeuHy6Plj agoGybF7xKfvIcr6V0c0fMyfCAAOw0AOMhvWw6Hvxu1ilm5Sz3XZtgt7pQquPZYexfA6 5Ryq6sQBjwVJ+0WBS6f7qzspyr9ieb1gphEVd15pWTi8Xz7jXdGVKAcbXu1B8WkRSZZ0 PxhYuchIqvleomETbUoKG7iCGKcOF7CABIkV2KVPM57hSWvwt1V6aLZnyOw6ctEZm+0g eQ6wTJrt4YtkFU8U8cuyPlaFsd+YrPCdq+pu3Lcb1wjpC0JbDGIKnkzUd7rB86xB0ryV Lfiw== MIME-Version: 1.0 Received: by 10.52.72.41 with SMTP id a9mr66574312vdv.7.1357333867541; Fri, 04 Jan 2013 13:11:07 -0800 (PST) Sender: brian.leroux@gmail.com Received: by 10.58.221.103 with HTTP; Fri, 4 Jan 2013 13:11:07 -0800 (PST) In-Reply-To: References: Date: Fri, 4 Jan 2013 13:11:07 -0800 X-Google-Sender-Auth: egKBsMZGPBXXN0A505uPVJ2ORuk Message-ID: Subject: Re: Test Coverage Tooling in Ripple From: Brian LeRoux To: dev@ripple.incubator.apache.org Content-Type: text/plain; charset=ISO-8859-1 X-Virus-Checked: Checked by ClamAV on apache.org A tale of caution. In a past life I once advocated strongly for complete coverage on a project. We made it a standard part of the process for each feature 'definition of done'. Eventually, bullshit tests snuck in because of this, and, worse, the suite was terribly slow to run and only getting bigger. The experience left me feeling that coverage is a bit of a ceremony if its not being taken super seriously. (Later we used a fuzzing tool to help shake out bugs and write good/better tests.) It was shitty maintenance that never really contributed to code quality, and because of bullshit tests, in some ways it created LESS quality and harmed our certainty of the suite itself. These days I tend to think that one should write tests to help you write code but only the bare minimum to exercise intent. (And obviously always write a test for every bug filed so no regressions manifest.) Testing is good and maybe coverage useful to understand the weaker areas of code but I would avoid making coverage itself a goal. $0.02 On Fri, Jan 4, 2013 at 11:18 AM, Gord Tanner wrote: > I have been wanting code coverage forever > > +1 > > > On Fri, Jan 4, 2013 at 2:10 PM, Filip Maj wrote: > >> Cool stuff Brent! >> >> +1 >> >> On 1/4/13 10:46 AM, "Dan Silivestru" wrote: >> >> >I'm 100% behind having some sort of code coverage solution in place. >> >Simply >> >counting the number of assertions we have doesn't give us as much >> >confidence as I would like to have. >> > >> >I do somewhat disagree that the node tests are more important the the >> >browser tests. I would at a minimum put them on par since Ripple does run >> >in the browser :-) But I think starting with code coverage for node only >> >is >> >a very good first step. >> > >> >So... long way of me saying... +1 :) >> > >> > >> >On Fri, Jan 4, 2013 at 1:19 PM, Brent Lintner >> >wrote: >> > >> >> Hey all, >> >> >> >> So, I have been using a (recently) new project called CoverJS in one of >> >>my >> >> personal (side projects), and I am finding it really useful and easy to >> >> use/setup when it comes to test code coverage in JS. >> >> >> >> https://npmjs.org/package/coverjs >> >> >> >> My proposal is to add support for test code coverage to Ripple (as test >> >> coverage is something I've really wanted to see go into the development >> >> workflow of Ripple). It is still in some early stages, but I think it >> >>would >> >> be a great project to adopt (even initially) as the code coverage >> >>tooling >> >> for this project. If it it needs to be changed, it should not be too >> >> difficult to rip out or replace, and this does not affect the normal >> >>way of >> >> running tests. >> >> >> >> I.e. Check it out in my fork (first and only commit) --> >> >> https://github.com/brentlintner/Ripple-UI/tree/test.cov >> >> >> >> Since I had already done the setup in my side project, it was quite >> >>easy to >> >> get it working in Ripple (although I had to wait to submit it until an >> >> upstream bug was fixed in CoverJS). The only pitfall here is it >> >>currently >> >> only works when running the tests with the nodejs runner (vs the browser >> >> based test runner, which, IMO is less primary than the node runner, >> >> anyways). However, it is still very useful when testing (even after >> >>using >> >> it a few times). >> >> >> >> Thoughts? Yay/Nay? >> >> >> >> I was hoping to issue a Pull Request soon (if it is a welcomed idea). >> >>:-) >> >> >> >> -- >> >> Brent >> >> >> > >> > >> > >> >-- >> >Dan Silivestru >> >+1 (519) 589-3624 >> >>