Summary: Stackage LTS is mostly pointless.
Stackage provides a set of precise versions of a subset of Hackage which all place nicely together. At any one time, there are two such version sets:
- Nightly, which aims to be the latest version of all packages.
- LTS (Long Term Support), which with every major release updates every package, and every minor release updates packages that only changed their minor version.
Stack currently defaults to LTS. I don't think LTS fulfils any need, and does cause harm (albeit very mild harm), so should be abandoned. Instead, people should use a specific nightly that suits them, and upgrade to later nightlies on whatever schedule suits them. I share these somewhat half-baked thoughts to hopefully guide how I think Stackage should evolve, and there may be good counterarguments I haven't thought of.
Why is Nightly better?
Nightly always has newer versions of packages. I believe that newer versions of packages are better.
- As a package author, with each new release, I fix bugs and evolve the library in ways which I think are beneficial. I make mistakes, and new versions are how I fix them, so people using old versions of my software are suffering for my past mistakes.
- As a package user, if I find a bug, I will endeavour to report it to the author. If I'm not using the latest version, it's highly likely the author will ask me to upgrade first. If a feature I want gets added, it's going to be to the latest version. The latest version of a package always has better support.
- If the latest version of a package does not suit my needs, that's an important alarm bell, and I need to take action. In some cases, that requires finding a fork or alternative package to do the same thing. In some cases, that involves talking to the author to make them aware of my particular use case. In either case, doing this sooner rather than later makes it easier to find a good solution.
What's the benefit of LTS?
As far as I am aware, LTS major releases are just Nightly at a particular point in time - so if you only ever use x.0 LTS, you might as well just use Nightly. The main benefit of LTS comes from picking a major LTS version, then upgrading to the subsequent minor LTS versions, to access new minor versions of your dependencies.
Upgrading packages is always a risk, as LTS Haskell says. What LTS minor releases do are minimize the risk of having to fix compilation errors, at the cost of not upgrading to the latest packages. However, the reasons I do upgrades are:
- Sometimes, if one package has a known bug that impacts me, I specifically upgrade just that one package. I don't want to upgrade other packages (it introduces unnecessary risk), so I take my package set (be it LTS or Nightly) and replace one constraint.
- Every so often, I want to upgrade all my packages, so that I'm not missing out on improvements and so I'm not falling behind the latest versions - reducing the time to upgrade in future. I do that by picking the latest version of all packages, fixing breakages, and upgrading.
- When upgrading, compile-time errors are a minor issue at worst. Before Stackage, my major headache was finding a compatible set of versions, but now that is trivial. Fixing compile-time errors is usually a little work, but fairly easy and predictable. Checking for regressions is more time consuming, and running the risk of regressions that escape the test suite has a significant cost. Tracking down if there are any resulting regressions is very time-consuming.
I don't see a use case for upgrading "a little bit", since I get a lot less benefit for only marginally less work.
Why is LTS better?
When I asked this question on Twitter, I got two reasons that did seem plausible:
- Matt Parsons suggested that LTS provided a higher likelihood of precompiled binaries shared between projects. That does make sense, but a similar benefit could be achieved by preferring Nightly on the 1st on the month.
- Gabriel Gonzalez suggested that when including a resolver in tutorials, LTS looks much better than a Nightly. I agree, but if there only was Nightly, then it wouldn't look quite as bad.
If Haskell packages had bug fixes that were regularly backported to minor releases, then the case for an LTS version would be stronger. However, my experience is that fixes are rarely backported, even for the rare security issues.
Is this a problem?
Currently everyone developing Haskell has two choices, and thanks to Stack, might not even be aware which one they've ended up making (Stack will pick nightly on its own if it thinks it needs to). Reducing the choices and simplifying the story removes work for the Stackage maintainers, and cognitive burden from the Stackage users.
Stackage was a radical experiment in doing things in an entirely different way. I think it's fair to say it has succeeded. Now, after experience using the features, I think it's right to critically look at tweaks to the model.
Here's another argument against LTS, as a library author. If there is some other LTS accepted package has an upper bound on your library, then your newer releases will not be accepted to LTS, until that other package increases their upper limit.
ReplyDeleteSo, a package that you've never even heard of can stop LTS from ever including the newer versions of your library, making everybody else suffer.
I find this to be the biggest challenge adopting LTS. As Lamport quipped once, "A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable." I'm afraid LTS is playing that role: A package you didn't even know existed prevents you from using the latest version of a library that you need.
This comment has been removed by the author.
ReplyDeleteAn admittedly petty concern, but if every project is based on a different nightly, won't my hard drive be full of random snapshots which I have to spend time downloading and recompiling all the time ? My sturdy Macmini from 2010 despite its many qualities is not the fastest gun around, even my fairly recent macbook does spend time does not shine in those moments.
ReplyDeleteI think I use mostly LTS3.6 on my laptop. works fine, smooth ride (lauding stack at least than 5 times a day for this)
Admittedly, that's bad for feedback to authors. A slower, say monthly, "potentially breaking release" would seem like a good compromise may be ?
Levent: The maintainer agreement is meant to stop that happening: https://github.com/fpco/stackage/blob/master/MAINTAINERS.md (I helped rewrite it earlier this week).
ReplyDeleteNicolas: It sounds like you want to only pick the first nightly of the month, almost as though there should be nighties and monthlies, which are just nightlies that occur on the 1st.
Levent:
ReplyDeleteMy experience is that whenever something fails to support the newest versions of its dependencies, and is holding everybody else back for a long enough window it will get kicked out of stackage until it catches up. I've watched this happen maybe a half dozen times through the stackage issue tracker.
Neil, Edward: That's good to know that there's a "social contract" around this. But it all being "manually enforced" is a problem for the long term sustainability/scalability of this model.
ReplyDeleteI want to make it clear that I like the idea behind LTS and very much applaud the work people are putting into it to create a reliable ecosystem. It's just an inherently hard problem to do so.
As someone on the distant fringe of the Haskell world, I have a perhaps ill-informed question: Parts of this blog post sound like you don't really like the concept of a conservative, more thoroughly vetted/curated set of packages/versions... like a Debian of the Haskell library ecosystem. Is that right? It seems like some potential users would find that quite valuable.
ReplyDeleteLevent: It's still early days, so might move to auto enforcement one day, but I'm not sure - it's working pretty well as is.
ReplyDeleteBenjamin: Stack offers manual curation of packages, with compatibility but not quality curation of versions. LTS and Nightly don't differ there. There is no real vetting of each version.
I am a new Haskeller. I am starting a project. I do not want to be forced into an update merry-go-round during the 18 months of the project. I may need to do minor updates of libraries or GHC because of bugs. What should I do?
ReplyDeleteWhen people ask what level of Haskell I am using, what do I say? Should I say anything other than GHC version. Does "LTS of Dec 3, 2015" carry any meaning next July?
I also want to do continuous integration and something akin to test driven development. I use OS X. I have vim fingers. Where do I go to find out what packages I should install and how to set them up?
Your decision on Stockage should consider the answer to these questions.
Carlton: Minor updates of the libraries are just as able to introduce as remove bugs. If minor updates were strictly less buggy, then your upgrade cycle would be a good idea, but as it is, better the bugs you know - don't upgrade unless you come across a bug that matters, and then upgrade in a targeted way.
ReplyDeleteLTS of Dec 3, 2015 is LTS-3.8. Say that instead and everyone will know exactly what you use forever.
The test and continuous integration stuff, while useful, is outside the scope of Stackage LTS vs Nightly, and has little impact on that.
don't forget docker support. LTS builds map to docker fpco support very cleanly.
ReplyDeleteI like LTS builds for this reason. Behind the curve sure, but stable, which is what they are meant to be.