Thursday, January 31, 2013

Shake 0.8 is out

Summary: Shake 0.8 is out, ask any questions on StackOverflow

Shake is a build system I have been working on sporadically for the last four years, think of it as a better alternative to writing Makefiles. In the past few weeks I've released versions 0.6, 0.7 and 0.8.

Questions about Shake

Unlike many of my other libraries, Shake invites user questions. It's a complex tool with lots of power to wield, and lots of aspects that emerge from the API, rather than being obvious from it. Therefore, I encourage anyone with any questions about Shake to ask them against the shake-build-system StackOverflow tag (thanks to Lennart Augustsson for creating the tag, as my reputation is too low). I've already asked one question, but I'm sure there are lots of others - "how does Shake/Make compare to monad/arrow?", "why did the Oracle change type?", "how would I define a rule that downloads from the web?". The more questions the easier it will be for future Shake users to find the information they need.

API Change: getDirectoryFiles

There is only one real breaking API change in the above series of versions, getDirectoryFiles now takes a list of FilePatterns. So you can write:

getDirectoryFiles "Configuration" ["*.xml","*.json"]

to find all XML and JSON files in your Configuration directory. The reason for this change is to introduce a new and more powerful matching capability, so you can also write:

getDirectoryFiles "Configuration" ["//*.xml","//*.json"]

And that will find all XML and JSON files anywhere under the Configuration directory. Shake tries hard to issue the minimum number of directory traversals, so searching for a list of patterns results in fewer file system queries than searching for each pattern individually.

2 comments:

Cédric Venet said...

Hi Neil,

Thanks for this great tool. Your paper on ICFP2012 explain it quite well. However I stumbled on issue on my first try to use it. I asked about it at http://stackoverflow.com/questions/14631978/how-to-define-custom-rule-in-shake-development-shake-core-is-hidden

Another small question I had is that from your paper, it seems that two identical directory listings are done each time their is a change. The first to check the validity of the stored value and the second to compute the new value. However, they should return the same things so if the goal is to reduce those directory listing, shouldn't the first listing be reused?
Could we define a generic rule for all the rules who do the same processing for validity checking and value generation?

Neil Mitchell said...

Hi Cédric,

I've replied to your StackOverflow question - the problem has come about since the Shake implementation has changed in a few details since ICFP.

You are indeed right, on each change it will do two directory listings. My argument has always been that the first probably brings it into the disk cache, so the second is cheap. But that's a poor argument! Thinking further I can probably define something like defaultRuleIsStoredValue, to say that the default rule is just to reuse whatever was previously returned by storedValue. That should simplify defining certain rules, and avoid the double traversal - thanks for pointing it out.