The Four Flaws of Haskell

Summary: Last year I made a list of four flaws with Haskell. Most have improved significantly over the last year.

No language/compiler ecosystem is without its flaws. A while ago I made a list of the flaws I thought might harm people using Haskell in an industrial setting. These are not flaws that impact beginners, or flaws that stop people from switching to Haskell, but those that might harm a big project. Naturally, everyone would come up with a different list, but here is mine.

Package Management: Installing a single consistent set of packages used across a large code base used to be difficult. Upgrading packages within that set was even worse. On Windows, anything that required a new network package was likely to end in tears. The MinGHC project solved the network issue. Stackage solved the consistent set of packages, and Stack made it even easier. I now consider Haskell package management a strength for large projects, not a risk.

IDE: The lack of an IDE certainly harms Haskell. There are a number of possibilities, but whenever I've tried them they have come up lacking. The fact that every Haskell programmer has an entrenched editor choice doesn't make it an easier task to solve. Fortunately, with Ghcid there is at least something near the minimum acceptable standard for everyone. At the same time various IDE projects have appeared, notably the Haskell IDE Engine and Intero. With Ghcid the lack of an IDE stops being a risk, and with the progress on other fronts I hope the Haskell IDE story continues to improve.

Space leaks: As Haskell programs get bigger, the chance of hitting a space leak increases, becoming an inevitability. While I am a big fan of laziness, space leaks are the big downside. Realising space leaks were on my flaws list, I started investigating methods for detecting space leaks, coming up with a simple detection method that works well. I've continued applying this method to other libraries and tools. I'll be giving a talk on space leaks at Haskell eXchange 2016. With these techniques space leaks don't go away, but they can be detected with ease and solved relatively simply - no longer a risk to Haskell projects.

Array/String libraries: When working with strings/arrays, the libraries that tend to get used are vector, bytestring, text and utf8-string. While each are individually nice projects, they don't work seamlessly together. The utf8-string provides UTF8 semantics for bytestring, which provides pinned byte arrays. The text package provides UTF16 encoded unpinned Char arrays. The vector package provides mutable and immutable vectors which can be either pinned or unpinned. I think the ideal situation would be a type that was either pinned or unpinned based on size, where the string was just a UTF8 encoded array with a newtype wrapping. Fortunately the foundation library provides exactly that. I'm not brave enough to claim a 0.0.1 package released yesterday has derisked Haskell projects, but things are travelling in the right direction.

It has certainly been possible to use Haskell for large projects for some time, but there were some risks. With the improvements over the last year the remaining risks have decreased markedly. In contrast, the risks of using an untyped or impure language remain significant, making Haskell a great choice for new projects.

Defining your own Shake build system

Summary: The video of my recent Shake talk is now online.

My Haskell eXchange 2015 talk "Defining your own build system with Shake" is now online, with both slides and video. As always, all my talks and papers are available from ndmitchell.com.

This Shake talk took a distinct approach from my previous talks - partly because I now have a better understanding of how large Shake build systems are typically structured. The key theoretical innovation of Shake is monadic dependencies, and the key technical innovation is using Haskell as an embedded language. Putting these pieces together lets Shake users write a build system interpreter, rather than a monolithic build system.

In the talk I suggest that build systems can be split into metadata (changes frequently, can be tracked, custom format for each project relying on conventions) and an interpreter for that metadata (written using Shake). The result seems to work quite nicely, especially in larger projects where everyone is expected to update the metadata, but changes to the interpreter are expected to be rarer and more carefully thought out.

One of the goals of the talk was to convert some of the audience to using Shake. It seems to have worked, as @krisajenkins tweeted:

First fruit of #haskellx - switched my makefiles over to Shake. Parallelised builds & a readable syntax. Much win.

Upcoming talks

I'll be talking at two events this month - hope to see some people there!

9 Oct 2013 - Haskell eXchange 2013

Everyone should use a Generics library - writing HLint with Uniplate

A generics library allows programmers to express only the interesting part of certain tasks, avoiding lots of boilerplate and making the code significantly shorter and more maintainable. Haskell is awash with excellent generics libraries, including Scrap Your Boilerplate, Generics for the Masses, Generic Deriving and Uniplate, yet many developers will never have used any of them. This talk explains how to use the Uniplate generics library, using examples derived from the Haskell Lint tool (HLint), which makes extensive use of Uniplate.

23 Oct 2013 - Runcifest

Colin's Industrial Influence (joint with Malcolm Wallace)

Colin has dabbled in many areas of functional programming, from XML processing to data visualisation, from type searching to generic transformations. While most of Colin's contributions have been made from inside a University, their impact has been felt in industry. In this talk we take a whirlwind tour through some of the projects that have influenced us and our work.

Haskell Exchange 2012 talks

Summary: The talks from the Haskell Exchange 2012 are online.

The Haskell Exchange 2012 took place last week. We had a good range of speakers, who all gave excellent talks. I would like to say thanks to all the speakers and to Skills Matter for organising and running the event. The talks are all available online, and I thought I'd just go through them now:

Simon Peyton Jones argued that purity and types are the key to Haskell. The purity thing is without doubt true - people cheat if they are allowed to, but Haskell discourages impurity both through the IO monad (effects are explicit) and community pressure (don't use unsafePerformIO unsafely). I also think that laziness combined with an aggressively optimising that messes up unsafePerformIO at the slightest opportunity have also helped (impurity always comes back to bite).

Simon Marlow explained the async package, showing how you can turn complex problems into simple ones with a simple mechanism that looks highly reusable. I have previously used a similar idea in my F# coding (although not nearly as thoroughly thought out), and been happy with the results - I suspect Simon's version will be even more useful. I particularly enjoyed the use of STM - it's rarely a concurrency solution I think of (I'm a threads and locks kind of person), but seeing it used so elegantly makes me want to experiment more.

Lennart Augustsson talked about embedded languages - where you write a language that looks a lot like Haskell, inside a Haskell program, only to find out later that it wasn't Haskell at all. Lennart is clearly the expert at these languages, having covered C, Basic and now financial combinators. Once you have these inner languages, you can take them in all kinds of directions, including right down to optimised assembly code.

Blake Rain gave an introduction to Yesod. I had read the Yesod documentation a while back, but the had trouble figuring out where to start, and what it was that mattered. Blake did both the sales pitch, and the beginners guide - I'll certainly be trying a Yesod at some point. In particular I really liked the type-safe routing, that would certainly have some in handy in my ASP developer days.

Duncan Coutts gave updates on the Cloud Haskell work, which in only a year, has gone from a research project to a practical library/tool that can be distributed at scale. The work covers details like how you distribute, why you distribute and how the model was designed (basically, copy Erlang). Another interesting aspect was how the real world development challenges, both the diverse nature of network/cloud computing, and how you can fund turning an academic idea into a user tool.

Rob Harrop showed how Ruby, Erlang and Haskell can all communicate using a message passing framework (AMQP). I certainly prefer Haskell, and go fully Haskell where possible, but a heterogeneous environment provides an easier migration path. Rob showed how to start and stop Haskell processes, switch elements from Ruby to Haskell and back again, all while a website was continuously responding to requests. This development style certainly provides an easy route in for Haskell, but also highlighted that Haskell still lacks some of the nicer sugar over AMQP that Ruby provides.

The final session was an open session where people discussed topics relating to Haskell. This session was fun, but I felt it could be better. I think it meandered at some points, and had a core of people who talked, but a large number of people who just watched. I'm not sure how I'd do things better, but it felt like some of the questions after the talks (Simon Peyton Jones and Duncan Coutts talks in particular) lead to more thorough discussions. I know this can be a problem for Haskell Symposium "Future of Haskell" discussions too, so perhaps there is some scope for tweaking the format?

Haskell Exchange next Wednesday

The Haskell Exchange 2012 is happening all day next Wednesday, October 10th, in London. There is still time to register, and with the discount code HASKELLX-2012-TE1 you save £50, making it £175. This event is very much about learning how people use Haskell, and how you can use Haskell. It isn't academics giving their plans for the future, or perfecting the small details of the language, it is all about what you can do today.

I've been helping Skills Matter organise the program since the start of this year. The final list of speakers is Simon Peyton Jones, Simon Marlow, Lennart Augustsson, Duncan Coutts, Blake Rain and Rob Harrop. You can find details of their talks on the main page. I'm very happy with both the variety and quality of the speakers. It includes people who were there at the beginning of lazy functional languages and also people actively making their living developing things for clients with Haskell.

I've seen four of these speakers talk many times before, and they are always fun and informative. The two speakers who might be less familiar to some in the Haskell community are talking about topics which sound particularly interesting:

• Blake Rain will be talking about Yesod, one of the three Haskell web frameworks. I've read the excellent documentation on Yesod, but I've never seen the overview. I ran an ASP web development company many years ago, and want to see how I could have avoided the problems of untyped development. I want to see how the ideas behind Haskell web frameworks work in practice, with real end-users who care far more about the shade of blue than type safe URL's.
• Rob Harrop will be talking about integrating Haskell components into an existing system via message passing over the network. I hear more and more people structuring complex systems as separate processes that talk to each other with messaging interfaces. Once you have process boundaries, then using a different language for each piece becomes easy. Translating 1 million lines of code to Haskell isn't usually an option, but prototyping one new process might be - this approach seems like a great gateway for Haskell.

I look forward to seeing some of you there! Register now (discount code HASKELLX-2012-TE1).

Haskell eXchange 2012

Skills Matter are having a 1-day Haskell conference in London on October 10th:

Haskell Exchange 2012

The conference is dedicated to practical Haskell, not academic Haskell. Each session will teach you about how Haskell is used in the real world, and include techniques you can apply to your Haskell programming. Speakers include Simon Peyton Jones, Simon Marlow, Lennart Augustsson, Duncan Coutts, Blake Rain and Rob Harrop. Topics include types, parallelism, high-performance concurrency, LLVM, EDSLs, Yesod and Cloud Haskell.

Registration is already open, and the early registration discount applies until 9th September (6 more days). I've been working with Skills Matter since May developing the program, and am very happy with the end result. I'll be there with 69% probability, and am really looking forward to it.