Ok, returning back to this, my test case did not want to save the file, only calculate the hash as memory-efficiently as possible without buffereing.  Mission accomplished. 

However now my real implementation does want to save and hash, again as memory-efficiently as possible. 

This is what I've created so far.  

First, getting error at the "while()" loop:
Caught: java.lang.NullPointerException
at java_io_FilterInputStream$read.call(Unknown Source)

Second, I'm really unsatisfied with it. 
1. Declaring all the variables before the loop is... really verbose
2.  It seems like the hash and file write operations should be separated. 

Does anyone have any suggestions?

def baseDir = new File("c:/data")
File localFile
byte[] buffer
DigestInputStream dis
MessageDigest digest
ByteArrayInputStream bais
jarList.each{Map jarMetadata ->
content.filePath = jarMetadata['path']
bais = content.downloadContent(true, [bt_package: testPkg])
localFile = new File(baseDir, jarMetadata['path'])
localFile.withOutputStream {fos->
buffer = new byte[1024]
dis = new DigestInputStream(bais, digest)
digest = MessageDigest.getInstance("SHA1")
while (dis.read(buffer) != -1) {
assert digest.digest().encodeHex() == jarMetadata['sha1']

Gerald R. Wiltse

On Fri, Mar 4, 2016 at 11:30 AM, Winnebeck, Jason <Jason.Winnebeck@windstream.com> wrote:

Only the “short” solution I wrote involved buffering to memory. The “long” solution I wrote involves no buffering. The DigestInputStream is not related to whether or not there is buffering, all it does is it lets you read the stream and get a digest at the same time. If you are just throwing out the data, there’s no need for the DigestInputStream. DigestInputStream just takes the result of every read call and sends the bytes to MessageDigest#update before returning the data. DigestInputStream makes sense if you were, for example, passing that input stream to another method that wrote the content to a file then you wanted to get the digest after the file write was complete.




From: Gerald Wiltse [mailto:jerrywiltse@gmail.com]
Sent: Friday, March 04, 2016 10:08 AM
To: users@groovy.apache.org
Subject: Re: Groovy Hash Calculations


I'm trying to verify the sha1 hash on a file download without saving it to disk, and without buffering the whole thing.  I think this solution might buffer all the data into "content" before going into the eachByte loop. I thought that was the reason for the DigestInputStream.  Do you know if this is correct?

Gerald R. Wiltse



On Fri, Mar 4, 2016 at 9:40 AM, Winnebeck, Jason <Jason.Winnebeck@windstream.com> wrote:

Here is how I would do it. I provide both a “short” solution and a “long” one. I would use the “short” solution if I was making for example a developer only tool and I knew I was only hashing small things for example a configuration file in a build script and want a one-liner. Otherwise I’d use the “long” version.


//If the content is guaranteed to be short:

def content = new ByteArrayInputStream("Here be dragons".bytes)

println MessageDigest.getInstance("SHA1").digest(content.bytes).encodeHex()


//If the content might be arbitrarily long:

content = new ByteArrayInputStream("Here be dragons".bytes)

def digest = MessageDigest.getInstance("SHA1")

content.eachByte(4096) { bytes, len ->

  digest.update(bytes, 0, len)


println digest.digest().encodeHex()




From: Gerald Wiltse [mailto:jerrywiltse@gmail.com]
Sent: Friday, March 04, 2016 9:18 AM
To: users@groovy.apache.org
Subject: Groovy Hash Calculations


Hello All, 


I have this block, it's pretty compressed, just wondering if there is a more groovy way to handle reading the buffer and computing the hash.  


        def messageDigest = MessageDigest.getInstance("SHA1")

        def dis = new DigestInputStream(content, messageDigest)

        byte[] buffer = new byte[1024];

        while (dis.read(buffer) != -1) {}

        def sha1Hex = new BigInteger(1, messageDigest.digest()).toString(16).padLeft(40, '0')


Gerald R. Wiltse


This email message and any attachments are for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message and any attachments.