Groovy: Simple file download from URL – Fixed

The Grails app I\’m working on right now has some cookbook code that takes a list of URLs and downloads the file each URL points to into a staging directory for other code to work on them. There are a couple dozen similar examples on groovy/grails blogs on the net:

def downloadFiles = { sourceUrls->
 def stagingDir = \"/tmp/stagingdir\"
 new File(stagingDir).mkdirs()
 sourceUrls.each { sourceUrl ->
   def filename = sourceUrl.tokenize(\'/\')[-1]
   def file = new FileOutputStream(\"$stagingDir/$filename\")
   def out = new BufferedOutputStream(file)
   out << new URL(sourceUrl).openStream()
   out.close()
 }
}

downloadFiles(
 [\"http://lavezzo.com/saic/mvnBuildLifecycle.png\",
 \"http://lavezzo.com/saic/settings.xml\"
 ])

Looks reasonable, right?

What happens if we call it like this?

downloadFiles(
 [\"http://lavezzo.com/saic/mvnBuildLifecycle.png\",
 \"http://lavezzo.com/saic/I have a space.png\"
 ])

Disaster!  java.net.URL can\’t handle spaces? Now normally, if I were writing the URLs I\’d just add in my own %20s and call it a day. But in this case that array of URL strings is the output of an XmlSlurper pointed at an html file. I have no control over the spaces in that file. java.net.URLEncoder seems like a good place to look, but it turns out that class is intended for use when composing links for html files. It substitutes a + for spaces, which don\’t work in java.net.URL. java.net.URI\’s documentation mentions that it encodes non-US-ASCII characters but not with the URI(String str) constructor. Again, this class seems to assume that you are making this URL yourself and can enter the protocol, port, hostname, etc each in its own constructor argument.

Well it was hard for me to believe but the answer was to separate out JUST the http portion of the URL string I collected from the web page and pass those into the URI(String scheme, String ssp, String fragment) constructor and then call URI\’s toURL() method.  Some Groovy array manipulation convienences made it a little easier:

def downloadFiles = { sourceUrls->
 def stagingDir = \"/tmp/stagingdir\"
 new File(stagingDir).mkdirs()
 sourceUrls.each { sourceUrl ->
   def filename = sourceUrl.tokenize(\'/\')[-1]
   def file = new FileOutputStream(\"$stagingDir/$filename\")
   def protocolUrlTokens = sourceUrl.tokenize(\':\')
   def sourceUrlAsURI = new URI(protocolUrlTokens[0],
       protocolUrlTokens[1..(protocolUrlTokens.size-1)].join(\":\"), \"\")
   def out = new BufferedOutputStream(file)
   out << sourceUrlAsURI.toURL().openStream()
   out.close()
 }
}

downloadFiles(
 [\"http://lavezzo.com/saic/mvnBuildLifecycle.png\",
 \"http://lavezzo.com/saic/I have a space.png\"
 ])

It looks silly to be splitting out the http just to put it back together in the constructor.  Seems like a simple point of improvement in the one argument constructor to URI to parse the String for protocol and then use the three argument constructor internally.

In Charlottesville, Virginia
Jeff

[Ed: Now with SyntaxHighlighter goodness]

Yady’s Alterations

Yady\’s Alterations, a full service clothing alterations and tailoring shop in Downtown Charlottesville has a new website:

http://www.YadysAlterations.com

While they\’ve only been in Charlottesville for a few years, they\’ve been doing this kind of work internationally for a while:

Yady\’s Alterations brings you more than 30 years experience. Our goal is to satisfy our customers with the lowest price in shortest time.

A great family-owned and operated business, they are happy to tackle any job, big or small:

From custom couture to small repairs, bridesmaid\’s dresses to backpacks, costumes to curtains, Yady\’s Alteration has done it all.

Can\’t get the right fit? Arms are too long? Pants are too short? Just a small hole doesn\’t have to ruin it. That backpack would be perfect if it only had that extra feature. We have a solution to these common problems. We alter clothing and add features to equipment, returning your item with a \”factory finished\” look.

They\’re a great \”Green\” asset for our town because they can fix things that we might normally be tempted to trash and replace.

We have the experience to bring your items back to \”good as new\” status. Most zipper repairs are simple and inexpensive. We offer many alternatives for repairing rips and tears with multiple color and fabric choices. We can repair almost anything including tents, packs, parkas, pants, sleeping bags, and dry suits.

We also make dresses, pants, skirts, shirts, coats, jackets, pajamas, nighties, robes, pillows, curtains, and costumes for any event.

I\’m looking forward to many years of success for them in Charlottesville.

In Charlottesville, Virginia, United States
Jeff Lavezzo