tag:blogger.com,1999:blog-7094652.post950936922998981027..comments2024-03-23T14:36:09.980+00:00Comments on Neil Mitchell's Blog (Haskell etc): Build system performance: Shake vs NinjaNeil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.comBlogger6125tag:blogger.com,1999:blog-7094652.post-28730999057170113672014-05-12T20:08:47.611+01:002014-05-12T20:08:47.611+01:00Sean: Thanks, fixed now.Sean: Thanks, fixed now.Neil Mitchellhttps://www.blogger.com/profile/13084722756124486154noreply@blogger.comtag:blogger.com,1999:blog-7094652.post-78573913854655920432014-05-12T09:40:32.882+01:002014-05-12T09:40:32.882+01:00Errata: The binary package link should be http://h...Errata: The binary package link should be http://hackage.haskell.org/package/binary .Anonymoushttps://www.blogger.com/profile/02951502639017426632noreply@blogger.comtag:blogger.com,1999:blog-7094652.post-43921407477806495412014-05-11T05:29:41.612+01:002014-05-11T05:29:41.612+01:00Ian: These numbers are all for Linux, but I do all...Ian: These numbers are all for Linux, but I do all my development on Windows and use Shake with Windows all day long, so I've optimised it for that too. Most optimisations tend to be shared, although I have custom filetime code on each OS, and certain things like parallelising GetModificationTime give a win on Windows but not Linux. For Windows, spawning processes has a terrible overhead compared to Linux, and Shake doesn't spawn process to check a build is up to date, while recursive Make would.<br /><br />Unfortunately my home desktop is now showing its age, so building things like kdelibs takes a while and probably requires updating half my system. I welcome others to try Shake out with cmake - if providing Windows Shake binaries would make that easier, let me know.Neil Mitchellhttps://www.blogger.com/profile/13084722756124486154noreply@blogger.comtag:blogger.com,1999:blog-7094652.post-73838068646053774072014-05-11T05:13:21.505+01:002014-05-11T05:13:21.505+01:00This is all for Linux I assume? Windows would be m...This is all for Linux I assume? Windows would be more fertile ground for optimizing a make replacement. I build on Windows and Linux both, and basically only really notice build times on Windows with my relatively small project. Especially the 'zero build' is a lot slower on Windows. (My theory is that NTFS is horrible, but I'm just guessing.) Though I'm guessing optimizing on Windows doesn't help for Linux though.<br /><br />Anyways if you can parse Ninja, that means Shake can build any cmake project. I suggest kdelibs for a more meaningful benchmark. ;)Ian Monroehttps://www.blogger.com/profile/16953610792430204523noreply@blogger.comtag:blogger.com,1999:blog-7094652.post-54144837361360098292014-05-06T06:22:42.875+01:002014-05-06T06:22:42.875+01:00I think something like sqlite would be a poor fit ...I think something like sqlite would be a poor fit - I describe it as "the database" but it's very much a serial list of record entries, with arbitrary Haskell payloads. If I put it in a database, I'd still have to deserialise the Haskell payloads, which is the bottleneck, and my own log can end up very efficient with the right binary library. It's interesting that both Shake and Ninja converged independently on very similar log structured files.<br /><br />I have considered reading the database in parallel. In particular, if you are reading the db and come across a file entry (which is roughly every single entry) you could check the modification time of that entry and cache it. I've not tried that yet as it requires threads/locks and could potentially go slower given a fast database read. It's one of the things on my list after the binary library rewrite.<br /><br />For learning Shake, I recommend the user manual which is basically a long tutorial: https://github.com/ndmitchell/shake/blob/master/docs/Manual.md#readmeNeil Mitchellhttps://www.blogger.com/profile/13084722756124486154noreply@blogger.comtag:blogger.com,1999:blog-7094652.post-42694080697600597582014-05-06T04:38:50.660+01:002014-05-06T04:38:50.660+01:00Congratulations!
Looks like shake is contender fo...Congratulations!<br /><br />Looks like shake is contender for the best large scale build system now.<br /><br />Did you consider moving database to sqlite or similar backend? Or starting to read database in parallel with other stages?<br /><br />PS I should probably think about moving my Makefiles to shake now... do you recommend any particular tutorial?MichaĆhttps://www.blogger.com/profile/09236312250962578880noreply@blogger.com