From spam at scientician.net Fri Jul 3 04:30:24 2015 From: spam at scientician.net (Bardur Arantsson) Date: Fri, 3 Jul 2015 06:30:24 +0200 Subject: UseSandbox Message-ID: Hi all, It seems to me that UseSandbox is slightly strangely defined at the moment. It's currently defined as data UseSandbox = UseSandbox FilePath | NoSandbox but it seems to me that it would be better to define it such that it actually contains relevant information about the sandbox if the UseSandbox constructor case instead of just the filepath. On the one hand this would require a separate "read-the-sandbox-information-we-need" function somewhere before individual commands run, but that would give us the opportunity to actually give relevant warnings in various significant cases; see for example https://github.com/haskell/cabal/issues/2381 Am I missing something which necessitates using only the FilePath? (Perhaps some dependence on sub-command switches, or similar?) Any thoughts, comments...? Regards, From the.dead.shall.rise at gmail.com Fri Jul 3 14:00:10 2015 From: the.dead.shall.rise at gmail.com (Mikhail Glushenkov) Date: Fri, 3 Jul 2015 16:00:10 +0200 Subject: UseSandbox In-Reply-To: References: Message-ID: Hi, On 3 July 2015 at 06:30, Bardur Arantsson wrote: > Hi all, > > Am I missing something which necessitates using only the FilePath? Adding extra info is absolutely fine, it's just that I never needed it. What do you want to add? From spam at scientician.net Fri Jul 3 14:47:25 2015 From: spam at scientician.net (Bardur Arantsson) Date: Fri, 3 Jul 2015 16:47:25 +0200 Subject: UseSandbox In-Reply-To: References: Message-ID: On 07/03/2015 04:00 PM, Mikhail Glushenkov wrote: > Hi, > > On 3 July 2015 at 06:30, Bardur Arantsson wrote: >> Hi all, >> >> Am I missing something which necessitates using only the FilePath? > > Adding extra info is absolutely fine, it's just that I never needed > it. What do you want to add? > Well, I'd like to split up the "get all the information we need" bit from the individual commands. That would perhaps make it easier to validate things like "does GHC_PACKAGE_PATH [or whatever] point at an existing directory" in a consistent manner across all commands and thus hopefully provide better feedback for broken setups/environments. (I also tend to find it good "hygiene" to split things up this way, but that's perhaps more a matter of taste.) See e.g. the issue I pointed to: https://github.com/haskell/cabal/issues/2381 (I don't think a full solution is possible at this time due to the fact that the env variables are only retrieved in the "exec" subcommand). Regards, From the.dead.shall.rise at gmail.com Fri Jul 3 15:53:17 2015 From: the.dead.shall.rise at gmail.com (Mikhail Glushenkov) Date: Fri, 3 Jul 2015 17:53:17 +0200 Subject: UseSandbox In-Reply-To: References: Message-ID: Hi, On 3 July 2015 at 16:47, Bardur Arantsson wrote: > Well, I'd like to split up the "get all the information we need" bit > from the individual commands. UseSandbox is produced by loadConfigOrSandboxConfig. Perhaps you can move your check there? From spam at scientician.net Fri Jul 3 16:09:08 2015 From: spam at scientician.net (Bardur Arantsson) Date: Fri, 3 Jul 2015 18:09:08 +0200 Subject: UseSandbox In-Reply-To: References: Message-ID: On 07/03/2015 05:53 PM, Mikhail Glushenkov wrote: > Hi, > > On 3 July 2015 at 16:47, Bardur Arantsson wrote: >> Well, I'd like to split up the "get all the information we need" bit >> from the individual commands. > > UseSandbox is produced by loadConfigOrSandboxConfig. Perhaps you can > move your check there? > The checks rely on information which the sub-commands will use (or will have to look up a second time)... hence I want to store it in UseSandbox :). Regards, From the.dead.shall.rise at gmail.com Fri Jul 3 21:44:36 2015 From: the.dead.shall.rise at gmail.com (Mikhail Glushenkov) Date: Fri, 3 Jul 2015 23:44:36 +0200 Subject: UseSandbox In-Reply-To: References: Message-ID: Hi, On 3 July 2015 at 18:09, Bardur Arantsson wrote: > The checks rely on information which the sub-commands will use (or will > have to look up a second time)... hence I want to store it in UseSandbox :). Yeah, so you can do the check in loadConfigOrSandboxConfig and then put the result of it in the UseSandbox value that you return. From spam at scientician.net Sat Jul 4 06:53:20 2015 From: spam at scientician.net (Bardur Arantsson) Date: Sat, 4 Jul 2015 08:53:20 +0200 Subject: UseSandbox In-Reply-To: References: Message-ID: On 07/03/2015 11:44 PM, Mikhail Glushenkov wrote: > Hi, > > On 3 July 2015 at 18:09, Bardur Arantsson wrote: >> The checks rely on information which the sub-commands will use (or will >> have to look up a second time)... hence I want to store it in UseSandbox :). > > Yeah, so you can do the check in loadConfigOrSandboxConfig and then > put the result of it in the UseSandbox value that you return. > (Sorry, think I did a personal reply instead of reply-to-list. Here it is again for the rest of you...) Exactamundo. I take it there's no objections on the ground of "but UseSandbox MUST ..."? I took that sentinment from your first post. It'll probably be easier to evaluate these things properly when I produce actual pull req's. Regards, From duncan at well-typed.com Wed Jul 8 13:08:23 2015 From: duncan at well-typed.com (Duncan Coutts) Date: Wed, 08 Jul 2015 14:08:23 +0100 Subject: Hackage security alpha Message-ID: <1436360903.26857.340.camel@well-typed.com> Hi folks, We're doing an alpha release of the hackage security work today and we'd like to invite you all to help test it. In addition to the security improvements it includes automatic use of mirrors (including the server distributing a list of available public mirrors) and includes incremental downloads of the hackage index, so cabal update should be a lot faster. At this alpha stage we would like some but not too many users to try it out, so when things do break we don't have it break for too many people all at once. But subscribers to this list are just the kind of expert users who we'd like to try it out and report issues. In particular we're interested in any problems caused by crazy proxies and annoying things of that ilk. During the beta we'll make the whole thing a bit more user friendly to get more people to try it out. So for the moment you have to grab things from git branches etc. All the details are in this blog post: http://www.well-typed.com/blog/2015/07/hackage-security-alpha/ As it says there, report issues in the github bug tracker. Oh and I don't think we say it in the blog post but the idea is that for any of the new library dependences for the security stuff, if any of them are problematic we can just bundle them with cabal-install (we'll probably just bundle them all). The design deliberately keeps these dependencies to a minimum: SHA256 hashing, ed25519 signing/checking provided by minimal bundled C code. For the alpha the cabal-install integration just uses these as external dependencies. -- Duncan Coutts, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/ From roma at ro-che.info Wed Jul 8 13:37:44 2015 From: roma at ro-che.info (Roman Cheplyaka) Date: Wed, 8 Jul 2015 16:37:44 +0300 Subject: Hackage security alpha In-Reply-To: <1436360903.26857.340.camel@well-typed.com> References: <1436360903.26857.340.camel@well-typed.com> Message-ID: <559D27A8.4000203@ro-che.info> Where exactly should I be looking for /snapshot.json? % curl -D - https://hackage.haskell.org/snapshot.json HTTP/1.1 404 Not Found Server: nginx/1.8.0 Content-Type: text/plain Content-Length: 43 Accept-Ranges: bytes Date: Wed, 08 Jul 2015 13:35:35 GMT Via: 1.1 varnish Age: 0 Connection: keep-alive X-Served-By: cache-fra1245-FRA X-Cache: MISS X-Cache-Hits: 0 X-Timer: S1436362535.049275,VS0,VE132 Page not found: Sorry, it's just not here. On 08/07/15 16:08, Duncan Coutts wrote: > Hi folks, > > We're doing an alpha release of the hackage security work today and we'd > like to invite you all to help test it. > > In addition to the security improvements it includes automatic use of > mirrors (including the server distributing a list of available public > mirrors) and includes incremental downloads of the hackage index, so > cabal update should be a lot faster. > > At this alpha stage we would like some but not too many users to try it > out, so when things do break we don't have it break for too many people > all at once. But subscribers to this list are just the kind of expert > users who we'd like to try it out and report issues. In particular we're > interested in any problems caused by crazy proxies and annoying things > of that ilk. > > During the beta we'll make the whole thing a bit more user friendly to > get more people to try it out. So for the moment you have to grab things > from git branches etc. All the details are in this blog post: > > http://www.well-typed.com/blog/2015/07/hackage-security-alpha/ > > As it says there, report issues in the github bug tracker. > > Oh and I don't think we say it in the blog post but the idea is that for > any of the new library dependences for the security stuff, if any of > them are problematic we can just bundle them with cabal-install (we'll > probably just bundle them all). The design deliberately keeps these > dependencies to a minimum: SHA256 hashing, ed25519 signing/checking > provided by minimal bundled C code. For the alpha the cabal-install > integration just uses these as external dependencies. > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From andres at well-typed.com Wed Jul 8 14:10:26 2015 From: andres at well-typed.com (Andres =?iso-8859-1?B?TPZo?=) Date: Wed, 8 Jul 2015 15:10:26 +0100 Subject: Hackage security alpha In-Reply-To: <559D27A8.4000203@ro-che.info> References: <1436360903.26857.340.camel@well-typed.com> <559D27A8.4000203@ro-che.info> Message-ID: <20150708141026.GK975@nullzig.kosmikus.org> Hi Roman. On Wed, Jul 08, 2015 at 04:37:44PM +0300, Roman Cheplyaka wrote: > Where exactly should I be looking for /snapshot.json? > > % curl -D - https://hackage.haskell.org/snapshot.json They're provided on the two "mirrors", but not on the main Hackage server (yet): https://hackage.haskell.org/security-alpha/mirror1/snapshot.json https://hackage.haskell.org/security-alpha/mirror2/snapshot.json Cheers, Andres -- Andres L?h, Haskell Consultant Well-Typed LLP, http://www.well-typed.com Registered in England & Wales, OC335890 250 Ice Wharf, 17 New Wharf Road, London N1 9RF, England From duncan at well-typed.com Wed Jul 8 18:42:20 2015 From: duncan at well-typed.com (Duncan Coutts) Date: Wed, 08 Jul 2015 19:42:20 +0100 Subject: Hackage security alpha In-Reply-To: <20150708141026.GK975@nullzig.kosmikus.org> References: <1436360903.26857.340.camel@well-typed.com> <559D27A8.4000203@ro-che.info> <20150708141026.GK975@nullzig.kosmikus.org> Message-ID: <1436380940.26857.357.camel@well-typed.com> On Wed, 2015-07-08 at 15:10 +0100, Andres L?h wrote: > Hi Roman. > > On Wed, Jul 08, 2015 at 04:37:44PM +0300, Roman Cheplyaka wrote: > > Where exactly should I be looking for /snapshot.json? > > > > % curl -D - https://hackage.haskell.org/snapshot.json > > They're provided on the two "mirrors", but not on the main Hackage > server (yet): > > https://hackage.haskell.org/security-alpha/mirror1/snapshot.json > https://hackage.haskell.org/security-alpha/mirror2/snapshot.json Right, for the alpha we're just providing these two "mirrors" which are just static file sets (created by the hackage-security tool for managing file based repos). The plan is that for the beta we'll deploy the server side code on hackage.h.o meaning that the main "smart" server will provide all the required security metadata. -- Duncan Coutts, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/ From gershomb at gmail.com Sun Jul 12 16:19:19 2015 From: gershomb at gmail.com (Gershom B) Date: Sun, 12 Jul 2015 12:19:19 -0400 Subject: Improving the "Get Haskell Experience" In-Reply-To: References: Message-ID: I?m ccing cabal-devel, as that list seems pertinent to this discussion too. In general I?m in favor of unifying efforts such as this. For windows, I?d still like the _option_ to get prebuilt library binaries for the full platform db. The modularity means that they can be just downloaded in a separate folder now, rather than installed into e.g. the global db? If so, that?s a nice win. Here is my motivation ? I tried the minghc installer and it mainly worked, but it played terribly with cygwin, which I use pervasively. So I found myself in a situation where one set of paths worked with cygwin, and the other set of paths let me build the network library. I still have some anxieties about those sorts of issues, and being able to ?just download the binaries? would make me feel a bit safer about at least one workaround should things get too muddled up. If we synchronize LTS and ?platform libs,? then I would like A) some way of distinguishing ?blessed platform libs? from ?other libs in LTS? (to solve the ?curation problem? which has always been a goal of the platform) and B) a standardized release schedule for LTS. (And, of course, I assume this will _not_ be for the platform release upcoming, which is due very shortly, but we?ll have a longer lead time to integrate and shake out all the various issues and test, etc.) ?Gershom On July 12, 2015 at 12:04:22 PM, Mark Lentczner (mark.lentczner at gmail.com) wrote: > *tl;dr: We'd like to incorporate stack into Haskell Platform, and stop > shipping pre-built packages, so we banish cabal hell, and have a single > common way to 'get Haskell' that just works.* > > At ICFP '14, there were several long group discussions of the state of > "getting Haskell", including Haskell Platform, Stackage, and other > approaches. Over the last year, there have been a few more public > discussions and evolution on the ideas, and other installer developments. > In particular, Stackage's LTS package sets are a direct development from > this work, and the release of stack has offered new options. > > Today, drawing from all this good work and ideas from so many, we'd would > like to propose a concrete plan: > > - Haskell Platform becomes the standard way to get *GHC* and related > tools: *alex*, *cabal*, *happy*, *hscolour*, and *stack*. It's a > user-friendly, cross-platform installer that gives a standard way to "get > Haskell" for most users. > - Use the *stack* model for package installation: > - The global db has only the GHC packages > - There is a package db for each curated set, Haskell Platform > becomes one such set > - Projects each have their own package db, much like sandboxes. > - Haskell Platform's installer will no longer include pre-built, > pre-installed packages other than GHC's set. Instead, it is configured so > that *stack* can be used to build and install, on as needed, the > corresponding Haskell Platform release packages. > > We think this plan solves many different community needs: > > - We have a clear way to "get Haskell" that works for a wide variety of > use cases. > - HP installer gets much smaller, and about as minimal as a working > installation can get. > - By leaving most packages out of the global database, users of > cabal-install, will now have far fewer problems. Sandbox builds should now > never give users "cabal hell" like warnings. > - By building and installing the Platform packages into it's own package > db, users get the benefit of building and installing these common packages > only once per system, yet can easily bypass them for any given project if > desired. > - Since the Platform packages are now built and installed as needed, > installing on smaller systems or servers without OpenGL will work. > > To do this, we have a bit of work ahead of us: We need to settle on > installation layout for each OS (including getting msys into the Windows > installer); decide on the naming and versioning of the Platform package set > (is it just LTS? does it have a different life cycle? etc...); getting > *stack* ready such a distribution; and configuring (or updating) > *cabal-install* to support the three-layer package db scheme. We think we > can do this in short order. > > We recognize this represents a significant change for the Platform, and > will require buy-in from the various parts, especially *ghc*, *cabal*, and > *stack*. We'd like to get your input. > > - Michael & Mark > _______________________________________________ > Libraries mailing list > Libraries at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries > From michael at snoyman.com Sun Jul 12 16:26:57 2015 From: michael at snoyman.com (Michael Snoyman) Date: Sun, 12 Jul 2015 16:26:57 +0000 Subject: Improving the "Get Haskell Experience" In-Reply-To: References: Message-ID: I'm +1 on nailing down an LTS release cycle, I've been pushing for it, and planning that it would happen after the first few releases. I'm fine with doing it starting with the next release if that's desired. The cygwin/msys conflict is a problematic one. stack has more flexibility addressing this than MinGHC did, since stack can control the PATH more directly. What we do now is, by default, add msys to the beginning of the PATH, and provide a command line option to not use msys at all. I believe this combination has addressed the bug reports we've received in the past. That's not to say I'm opposed to generating binary databases; I'm actually in favor of having that option for all platforms at some point (there's an open stack issue about it[1]). But it's a significant amount of work, and I think if possible we should defer it until we've figured out initial integration points. [1] https://github.com/commercialhaskell/stack/issues/117 On Sun, Jul 12, 2015 at 9:20 AM Gershom B wrote: > I?m ccing cabal-devel, as that list seems pertinent to this discussion > too. In general I?m in favor of unifying efforts such as this. > > For windows, I?d still like the _option_ to get prebuilt library binaries > for the full platform db. The modularity means that they can be just > downloaded in a separate folder now, rather than installed into e.g. the > global db? If so, that?s a nice win. Here is my motivation ? I tried the > minghc installer and it mainly worked, but it played terribly with cygwin, > which I use pervasively. So I found myself in a situation where one set of > paths worked with cygwin, and the other set of paths let me build the > network library. I still have some anxieties about those sorts of issues, > and being able to ?just download the binaries? would make me feel a bit > safer about at least one workaround should things get too muddled up. > > If we synchronize LTS and ?platform libs,? then I would like A) some way > of distinguishing ?blessed platform libs? from ?other libs in LTS? (to > solve the ?curation problem? which has always been a goal of the platform) > and B) a standardized release schedule for LTS. > > (And, of course, I assume this will _not_ be for the platform release > upcoming, which is due very shortly, but we?ll have a longer lead time to > integrate and shake out all the various issues and test, etc.) > > ?Gershom > > > On July 12, 2015 at 12:04:22 PM, Mark Lentczner (mark.lentczner at gmail.com) > wrote: > > *tl;dr: We'd like to incorporate stack into Haskell Platform, and stop > > shipping pre-built packages, so we banish cabal hell, and have a single > > common way to 'get Haskell' that just works.* > > > > At ICFP '14, there were several long group discussions of the state of > > "getting Haskell", including Haskell Platform, Stackage, and other > > approaches. Over the last year, there have been a few more public > > discussions and evolution on the ideas, and other installer developments. > > In particular, Stackage's LTS package sets are a direct development from > > this work, and the release of stack has offered new options. > > > > Today, drawing from all this good work and ideas from so many, we'd would > > like to propose a concrete plan: > > > > - Haskell Platform becomes the standard way to get *GHC* and related > > tools: *alex*, *cabal*, *happy*, *hscolour*, and *stack*. It's a > > user-friendly, cross-platform installer that gives a standard way to "get > > Haskell" for most users. > > - Use the *stack* model for package installation: > > - The global db has only the GHC packages > > - There is a package db for each curated set, Haskell Platform > > becomes one such set > > - Projects each have their own package db, much like sandboxes. > > - Haskell Platform's installer will no longer include pre-built, > > pre-installed packages other than GHC's set. Instead, it is configured so > > that *stack* can be used to build and install, on as needed, the > > corresponding Haskell Platform release packages. > > > > We think this plan solves many different community needs: > > > > - We have a clear way to "get Haskell" that works for a wide variety of > > use cases. > > - HP installer gets much smaller, and about as minimal as a working > > installation can get. > > - By leaving most packages out of the global database, users of > > cabal-install, will now have far fewer problems. Sandbox builds should > now > > never give users "cabal hell" like warnings. > > - By building and installing the Platform packages into it's own package > > db, users get the benefit of building and installing these common > packages > > only once per system, yet can easily bypass them for any given project if > > desired. > > - Since the Platform packages are now built and installed as needed, > > installing on smaller systems or servers without OpenGL will work. > > > > To do this, we have a bit of work ahead of us: We need to settle on > > installation layout for each OS (including getting msys into the Windows > > installer); decide on the naming and versioning of the Platform package > set > > (is it just LTS? does it have a different life cycle? etc...); getting > > *stack* ready such a distribution; and configuring (or updating) > > *cabal-install* to support the three-layer package db scheme. We think we > > can do this in short order. > > > > We recognize this represents a significant change for the Platform, and > > will require buy-in from the various parts, especially *ghc*, *cabal*, > and > > *stack*. We'd like to get your input. > > > > - Michael & Mark > > _______________________________________________ > > Libraries mailing list > > Libraries at haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries > > > > _______________________________________________ > ghc-devs mailing list > ghc-devs at haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael at snoyman.com Sun Jul 12 17:13:37 2015 From: michael at snoyman.com (Michael Snoyman) Date: Sun, 12 Jul 2015 17:13:37 +0000 Subject: Improving the "Get Haskell Experience" In-Reply-To: References: Message-ID: I think it depends somewhat on operating system, since there are permissions issues to be dealt with regarding user vs global installation. In principle though: I think the HP installers would install to a non-stack-specific location, and then stack would simply use that GHC based on its inclusion in the PATH. (I'm also guessing this will tie in nicely with Linux distro packages, which would likely want ghc to live in /usr/bin). On Sun, Jul 12, 2015 at 10:11 AM Dan Burton wrote: > Stack has the capability of downloading and installing GHC on demand, as > well as auto-updating itself. Both of these features, by default, occur in > subdirectories of the user's home directory. (Slightly different on Windows > but same idea.) Would the Platform install GHC to the stack location > directly, or install it system-wide as previously? (Stack can still make > use of GHC anywhere on your path.) > > On Sunday, July 12, 2015, Michael Snoyman wrote: > >> I'm +1 on nailing down an LTS release cycle, I've been pushing for it, >> and planning that it would happen after the first few releases. I'm fine >> with doing it starting with the next release if that's desired. >> >> The cygwin/msys conflict is a problematic one. stack has more flexibility >> addressing this than MinGHC did, since stack can control the PATH more >> directly. What we do now is, by default, add msys to the beginning of the >> PATH, and provide a command line option to not use msys at all. I believe >> this combination has addressed the bug reports we've received in the past. >> >> That's not to say I'm opposed to generating binary databases; I'm >> actually in favor of having that option for all platforms at some point >> (there's an open stack issue about it[1]). But it's a significant amount of >> work, and I think if possible we should defer it until we've figured out >> initial integration points. >> >> [1] https://github.com/commercialhaskell/stack/issues/117 >> >> On Sun, Jul 12, 2015 at 9:20 AM Gershom B wrote: >> >>> I?m ccing cabal-devel, as that list seems pertinent to this discussion >>> too. In general I?m in favor of unifying efforts such as this. >>> >>> For windows, I?d still like the _option_ to get prebuilt library >>> binaries for the full platform db. The modularity means that they can be >>> just downloaded in a separate folder now, rather than installed into e.g. >>> the global db? If so, that?s a nice win. Here is my motivation ? I tried >>> the minghc installer and it mainly worked, but it played terribly with >>> cygwin, which I use pervasively. So I found myself in a situation where one >>> set of paths worked with cygwin, and the other set of paths let me build >>> the network library. I still have some anxieties about those sorts of >>> issues, and being able to ?just download the binaries? would make me feel a >>> bit safer about at least one workaround should things get too muddled up. >>> >>> If we synchronize LTS and ?platform libs,? then I would like A) some way >>> of distinguishing ?blessed platform libs? from ?other libs in LTS? (to >>> solve the ?curation problem? which has always been a goal of the platform) >>> and B) a standardized release schedule for LTS. >>> >>> (And, of course, I assume this will _not_ be for the platform release >>> upcoming, which is due very shortly, but we?ll have a longer lead time to >>> integrate and shake out all the various issues and test, etc.) >>> >>> ?Gershom >>> >>> >>> On July 12, 2015 at 12:04:22 PM, Mark Lentczner ( >>> mark.lentczner at gmail.com) wrote: >>> > *tl;dr: We'd like to incorporate stack into Haskell Platform, and stop >>> > shipping pre-built packages, so we banish cabal hell, and have a single >>> > common way to 'get Haskell' that just works.* >>> > >>> > At ICFP '14, there were several long group discussions of the state of >>> > "getting Haskell", including Haskell Platform, Stackage, and other >>> > approaches. Over the last year, there have been a few more public >>> > discussions and evolution on the ideas, and other installer >>> developments. >>> > In particular, Stackage's LTS package sets are a direct development >>> from >>> > this work, and the release of stack has offered new options. >>> > >>> > Today, drawing from all this good work and ideas from so many, we'd >>> would >>> > like to propose a concrete plan: >>> > >>> > - Haskell Platform becomes the standard way to get *GHC* and related >>> > tools: *alex*, *cabal*, *happy*, *hscolour*, and *stack*. It's a >>> > user-friendly, cross-platform installer that gives a standard way to >>> "get >>> > Haskell" for most users. >>> > - Use the *stack* model for package installation: >>> > - The global db has only the GHC packages >>> > - There is a package db for each curated set, Haskell Platform >>> > becomes one such set >>> > - Projects each have their own package db, much like sandboxes. >>> > - Haskell Platform's installer will no longer include pre-built, >>> > pre-installed packages other than GHC's set. Instead, it is configured >>> so >>> > that *stack* can be used to build and install, on as needed, the >>> > corresponding Haskell Platform release packages. >>> > >>> > We think this plan solves many different community needs: >>> > >>> > - We have a clear way to "get Haskell" that works for a wide variety of >>> > use cases. >>> > - HP installer gets much smaller, and about as minimal as a working >>> > installation can get. >>> > - By leaving most packages out of the global database, users of >>> > cabal-install, will now have far fewer problems. Sandbox builds should >>> now >>> > never give users "cabal hell" like warnings. >>> > - By building and installing the Platform packages into it's own >>> package >>> > db, users get the benefit of building and installing these common >>> packages >>> > only once per system, yet can easily bypass them for any given project >>> if >>> > desired. >>> > - Since the Platform packages are now built and installed as needed, >>> > installing on smaller systems or servers without OpenGL will work. >>> > >>> > To do this, we have a bit of work ahead of us: We need to settle on >>> > installation layout for each OS (including getting msys into the >>> Windows >>> > installer); decide on the naming and versioning of the Platform >>> package set >>> > (is it just LTS? does it have a different life cycle? etc...); getting >>> > *stack* ready such a distribution; and configuring (or updating) >>> > *cabal-install* to support the three-layer package db scheme. We think >>> we >>> > can do this in short order. >>> > >>> > We recognize this represents a significant change for the Platform, and >>> > will require buy-in from the various parts, especially *ghc*, *cabal*, >>> and >>> > *stack*. We'd like to get your input. >>> > >>> > - Michael & Mark >>> > _______________________________________________ >>> > Libraries mailing list >>> > Libraries at haskell.org >>> > http://mail.haskell.org/cgi-bin/mailman/listinfo/libraries >>> > >>> >>> _______________________________________________ >>> ghc-devs mailing list >>> ghc-devs at haskell.org >>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >>> >> -- >> > You received this message because you are subscribed to the Google Groups >> "haskell-stack" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to haskell-stack+unsubscribe at googlegroups.com. >> To post to this group, send email to haskell-stack at googlegroups.com. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/haskell-stack/CAKA2JgKYuBMU5wtbjGV72dDKvSJePWbhoZeRCuJYR9SFoV8EDw%40mail.gmail.com >> >> . > > >> For more options, visit https://groups.google.com/d/optout. >> > > > -- > -- Dan Burton > > -- > You received this message because you are subscribed to the Google Groups > "haskell-stack" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to haskell-stack+unsubscribe at googlegroups.com. > To post to this group, send email to haskell-stack at googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/haskell-stack/CALSygwfNjyxWFL0y4-_5aSyZZVQSeb5705MzZnaM4Q7GA2MLBg%40mail.gmail.com > > . > For more options, visit https://groups.google.com/d/optout. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From duncan.coutts at googlemail.com Tue Jul 14 12:52:46 2015 From: duncan.coutts at googlemail.com (Duncan Coutts) Date: Tue, 14 Jul 2015 13:52:46 +0100 Subject: Request for feedback on spec/proposal for distributing package collections via hackage Message-ID: <1436878366.26857.532.camel@googlemail.com> Hi folks, I'd like to get feedback on a spec/proposal for distributing package collections via hackage. This is currently somewhere beyond vapourware but certainly not a fait accompli and hopefully it is at an appropriate point to get feedback. The basic idea is that package collections are: * useful (IMHO, one of the top two solutions to dependency hell, alongside nix-style package management); and * just as we distribute packages via hackage, we should also be able to easily distribute package collections. One would then use them with tools like cabal and stack. Distributing via hackage (both in the sense of the format/protocol and in the sense of the central community hackage instance) seems natural, and allows taking advantage of much of the infrastructure we have for packages already like: * existing user accounts and management infrastructure on the hackage website * allowing anyone to host collections on their own servers, just as they can host their own package archives currently (either as static file sets or with smart servers) * low barrier for distribution, potentially encouraging more collections to be created potentially covering more use cases * security infrastructure (currently in alpha) * automatic mirroring (currently in alpha) Two obvious examples are stackage-lts and stackage-nightly but if we lower the barrier for distribution then there may well be many more. For example, the existing Linux distros put a lot of effort into selecting and maintain package collections, and some of these collections could be distributed via hackage. In fast several Linux distributions already use Hackage's "distro" feature to advertise which versions of packages are provided by that distro. One can also imagine special-purpose collections, and there's probably cases we've not thought of yet. Package collections are different things from packages, not like "meta packages" that one gets in some package systems. A package collection at it's simplest is just a set of source package identifiers (ie names-version pairs). Like packages, package collections have names and versions and are immutable once distributed. The intention is that users can configure their tool to use collection(s), either by nailing down a specific collection version, or by not specifying a version it would default to the latest version of the named collection. (But the specific behaviour is up to the tool) Use cases: * versioned collections. For some collections the policy by which it's defined naturally uses meaningful versions. * daily collections. These can have a date-form version number imposed on them. * "live" "rolling" collections. These could have a simple monotonic increasing version with no particular meaning attached. For such collections, clients might be configured to use the latest (by not specifying a version), but it's always possible to pick a specific revision. * special-purpose collections. Not necessarily collections aiming to cover a large number of common packages, but aiming to cover some application area, or related stack of packages (e.g. some of the web frameworks). * negative collections. Collections of packages you may specifically want to avoid (e.g. deprecated by their authors, or known-broken). Using such collections would rely on clients that can be configured to treat it negatively. Specifics: A package collection specifies a set of source package ids (id being name-version pair). It also optionally specifies a (partial) flag assignment for any package name. The collection does not specify how tools should treat them. That is, a collection does not specify if it should be treated as a strong or a soft constraint, inclusive or exclusive, positive or negative. Such things are completely up to the client's policy and configuration. Similarly for flag assignments, collections do not specify whether tools should interpret these as strong or soft constraints. Syntax: Package collection names and versions exactly follow those of package names (but they live in a different namespace). For example, "stackage-lts-2.9", or "deprecated-343" (the latter being a "rolling" collection with a meaningless monotonically increasing version). A collection distributed in the archive format is just a text file with one entry per line, such as: foo-1.0 foo-1.1 bar >= 3 && < 4 bar +this -that So each line can be one of: * a simple package id * a package version range, using Cabal version range syntax * a package name with a flag assignment, + for on, - for off The interpretation of the above is that: * both foo-1.0 and foo-1.1 are in the collection (ie union not intersection) * all versions of bar between 3 and 4 are in the collection * the package bar has flag 'this' as True, and flag 'that' as False Of course for some collections the policy is that only one version of any package is included, but this is a policy question and the format itself does not impose this constraint. Hackage archive format: collection files live under a different prefix from package tarballs (but are still considered part of the archive) and are named after the collection id. The collection files are not compressed (but of course http clients and servers can negotiate transport compression). The collection files are not included in nor listed in the existing 00-index.tar.gz, but there's other json format metadata for a client to enumerate the available collections and versions. And like with package tarballs, a client that wants a specific collection version can construct the url and fetch it directly. Security: The hackage security system that's currently in alpha testing can easily be extended to cover collections, similarly to how it covers package tarballs. Misc notes: There is no requirement that a hackage-format repo containing collections be closed. That is, the collections may refer to packages not in that archive. This could be useful for private hackage repos that host a small number of private packages, but also host collections that refer both to the private packages and public ones from the community central hackage. The resolution of package names is done by the clients, and some clients may be configured to union/overlay multiple repos. On the other hand, for the central community hackage it may be sensible to enforce a policy that the collections it distributes be closed (ie refer only to packages distributed via hackage). Questions: Is this sufficiently flexible to fully cover the obvious use cases? Are there any interesting use cases that are excluded? Anything else? Duncan From ezyang at mit.edu Tue Jul 14 19:02:12 2015 From: ezyang at mit.edu (Edward Z. Yang) Date: Tue, 14 Jul 2015 12:02:12 -0700 Subject: Request for feedback on spec/proposal for distributing package collections via hackage In-Reply-To: <1436878366.26857.532.camel@googlemail.com> References: <1436878366.26857.532.camel@googlemail.com> Message-ID: <1436898968-sup-2927@sabre> Hello Duncan, In my eyes, this proposal looks like some sort of generalization of Stackage; and one further use case is "special purpose" collection. My big question: how composable are these collections really? I can't put two collections with conflicting versions together (or can I? Do I union?); and is there any point to having a collection without versions in it? (If Cabal syntax is extended to support depending on collections as well as packages, yes?) The classic use-case for package collections is deployment settings, ala Stack, or even Cargo lockfiles / Bundler Gemfile.lock (versioned collections). In all these use-cases package collections are treated as non-compositional things. http://doc.crates.io/guide.html http://bundler.io/v1.7/rationale.html#checking-your-code-into-version-control Libraries (compositional) do NOT publish lockfiles: only executables (non-compositional) DO. Re the file format, it seems fine; suitable for the lockfile use-case and the Stackage use-case. Less sure about the unioning semantics. Edward Excerpts from Duncan Coutts's message of 2015-07-14 05:52:46 -0700: > Hi folks, > > I'd like to get feedback on a spec/proposal for distributing package > collections via hackage. This is currently somewhere beyond vapourware > but certainly not a fait accompli and hopefully it is at an appropriate > point to get feedback. > > The basic idea is that package collections are: > * useful (IMHO, one of the top two solutions to dependency hell, > alongside nix-style package management); and > * just as we distribute packages via hackage, we should also be > able to easily distribute package collections. > > One would then use them with tools like cabal and stack. Distributing > via hackage (both in the sense of the format/protocol and in the sense > of the central community hackage instance) seems natural, and allows > taking advantage of much of the infrastructure we have for packages > already like: > * existing user accounts and management infrastructure on the > hackage website > * allowing anyone to host collections on their own servers, just > as they can host their own package archives currently (either as > static file sets or with smart servers) > * low barrier for distribution, potentially encouraging more > collections to be created potentially covering more use cases > * security infrastructure (currently in alpha) > * automatic mirroring (currently in alpha) > > Two obvious examples are stackage-lts and stackage-nightly but if we > lower the barrier for distribution then there may well be many more. For > example, the existing Linux distros put a lot of effort into selecting > and maintain package collections, and some of these collections could be > distributed via hackage. In fast several Linux distributions already use > Hackage's "distro" feature to advertise which versions of packages are > provided by that distro. One can also imagine special-purpose > collections, and there's probably cases we've not thought of yet. > > Package collections are different things from packages, not like "meta > packages" that one gets in some package systems. A package collection at > it's simplest is just a set of source package identifiers (ie > names-version pairs). Like packages, package collections have names and > versions and are immutable once distributed. > > The intention is that users can configure their tool to use > collection(s), either by nailing down a specific collection version, or > by not specifying a version it would default to the latest version of > the named collection. (But the specific behaviour is up to the tool) > > Use cases: > > * versioned collections. For some collections the policy by which > it's defined naturally uses meaningful versions. > * daily collections. These can have a date-form version number > imposed on them. > * "live" "rolling" collections. These could have a simple > monotonic increasing version with no particular meaning > attached. For such collections, clients might be configured to > use the latest (by not specifying a version), but it's always > possible to pick a specific revision. > * special-purpose collections. Not necessarily collections aiming > to cover a large number of common packages, but aiming to cover > some application area, or related stack of packages (e.g. some > of the web frameworks). > * negative collections. Collections of packages you may > specifically want to avoid (e.g. deprecated by their authors, or > known-broken). Using such collections would rely on clients that > can be configured to treat it negatively. > > Specifics: > > A package collection specifies a set of source package ids (id being > name-version pair). It also optionally specifies a (partial) flag > assignment for any package name. > > The collection does not specify how tools should treat them. That is, a > collection does not specify if it should be treated as a strong or a > soft constraint, inclusive or exclusive, positive or negative. Such > things are completely up to the client's policy and configuration. > Similarly for flag assignments, collections do not specify whether tools > should interpret these as strong or soft constraints. > > Syntax: > > Package collection names and versions exactly follow those of package > names (but they live in a different namespace). For example, > "stackage-lts-2.9", or "deprecated-343" (the latter being a "rolling" > collection with a meaningless monotonically increasing version). > > A collection distributed in the archive format is just a text file with > one entry per line, such as: > > foo-1.0 > foo-1.1 > bar >= 3 && < 4 > bar +this -that > > So each line can be one of: > * a simple package id > * a package version range, using Cabal version range syntax > * a package name with a flag assignment, + for on, - for off > > The interpretation of the above is that: > * both foo-1.0 and foo-1.1 are in the collection (ie union not > intersection) > * all versions of bar between 3 and 4 are in the collection > * the package bar has flag 'this' as True, and flag 'that' as > False > > Of course for some collections the policy is that only one version of > any package is included, but this is a policy question and the format > itself does not impose this constraint. > > Hackage archive format: > > collection files live under a different prefix from package tarballs > (but are still considered part of the archive) and are named after the > collection id. The collection files are not compressed (but of course > http clients and servers can negotiate transport compression). The > collection files are not included in nor listed in the existing > 00-index.tar.gz, but there's other json format metadata for a client to > enumerate the available collections and versions. And like with package > tarballs, a client that wants a specific collection version can > construct the url and fetch it directly. > > Security: > > The hackage security system that's currently in alpha testing can easily > be extended to cover collections, similarly to how it covers package > tarballs. > > Misc notes: > > There is no requirement that a hackage-format repo containing > collections be closed. That is, the collections may refer to packages > not in that archive. This could be useful for private hackage repos that > host a small number of private packages, but also host collections that > refer both to the private packages and public ones from the community > central hackage. The resolution of package names is done by the clients, > and some clients may be configured to union/overlay multiple repos. > > On the other hand, for the central community hackage it may be sensible > to enforce a policy that the collections it distributes be closed (ie > refer only to packages distributed via hackage). > > Questions: > > Is this sufficiently flexible to fully cover the obvious use cases? Are > there any interesting use cases that are excluded? > > Anything else? > > > Duncan > From duncan.coutts at googlemail.com Tue Jul 14 19:48:18 2015 From: duncan.coutts at googlemail.com (Duncan Coutts) Date: Tue, 14 Jul 2015 20:48:18 +0100 Subject: Request for feedback on spec/proposal for distributing package collections via hackage In-Reply-To: <1436898968-sup-2927@sabre> References: <1436878366.26857.532.camel@googlemail.com> <1436898968-sup-2927@sabre> Message-ID: <1436903298.26857.563.camel@googlemail.com> On Tue, 2015-07-14 at 12:02 -0700, Edward Z. Yang wrote: > Hello Duncan, > > In my eyes, this proposal looks like some sort of generalization of > Stackage; and one further use case is "special purpose" collection. My > big question: how composable are these collections really? I can't put > two collections with conflicting versions together (or can I? Do I > union?); You're right that some use cases of collections only make sense if clients can reasonably flexibly combine collections, e.g. set ops like union, intersection and inversion or difference. I think in principle taking collection unions or intersections makes sense. For example, in stack, as I understand it, you can add a local version of something that is also in the collection. This is of course a union. So extending "narrow" collections by unioning them with extra stuff makes sense. And suppose we had other "wide" collections that didn't nail things down to just one version, then taking an intersection with some other wide or narrow collection makes sense. So in principle, these set-like operations make sense. The code we've got in-progress for cabal-install allows exactly that. But none of that is essential to make use of the general purpose collections like stackage. > and is there any point to having a collection without versions > in it? To be clear, every collection instance has a version. It's just that for some of them -- live or rolling collections -- there is no particular meaning to the version. So this thing about not specifying a version is completely client side and doesn't affect the spec at all. It's just worth pointing out as a use case. As an example, we might have collections that are automagically defined and updated based on some property of the packages. Those are unlikely to have a meaningful version number. > (If Cabal syntax is extended to support depending on collections > as well as packages, yes?) I don't think you're confused here, but just to clarify: packages do not talk about collections. Tools like cabal/stack can be configured to use collections. And yes, where currently to configure cabal-install to use a collection you have to explicitly list every member as a constraint in a cabal.config file (see the cabal.config files distributed from the stackage website), the extension in cabal-install is to support these collections as a first class thing, by name-version or just name (and with an expression language to combine them). > The classic use-case for package collections is deployment settings, ala > Stack, or even Cargo lockfiles / Bundler Gemfile.lock (versioned > collections). In all these use-cases package collections are treated as > non-compositional things. http://doc.crates.io/guide.html > http://bundler.io/v1.7/rationale.html#checking-your-code-into-version-control > Libraries (compositional) do NOT publish lockfiles: only executables > (non-compositional) DO. > > Re the file format, it seems fine; suitable for the lockfile use-case > and the Stackage use-case. Less sure about the unioning semantics. Right, collections and frozen settings are similar, and for the latter composition doesn't make a lot of sense. We have the latter already of course, in the form of "cabal freeze" cabal.config files, and similarly for stack with .yml config files (which can be based off of pre-defined collections). It's likely that cabal freeze will switch to use this collection notation as it's somewhat more intentional (than the big sets of raw constraints). Duncan From duncan.coutts at googlemail.com Tue Jul 14 19:48:22 2015 From: duncan.coutts at googlemail.com (Duncan Coutts) Date: Tue, 14 Jul 2015 20:48:22 +0100 Subject: Request for feedback on spec/proposal for distributing package collections via hackage In-Reply-To: <1436878366.26857.532.camel@googlemail.com> References: <1436878366.26857.532.camel@googlemail.com> Message-ID: <1436903302.26857.564.camel@googlemail.com> On Tue, 2015-07-14 at 13:52 +0100, Duncan Coutts wrote: > Syntax: > > Package collection names and versions exactly follow those of package > names (but they live in a different namespace). For example, > "stackage-lts-2.9", or "deprecated-343" (the latter being a "rolling" > collection with a meaningless monotonically increasing version). > > A collection distributed in the archive format is just a text file with > one entry per line, such as: > > foo-1.0 > foo-1.1 > bar >= 3 && < 4 > bar +this -that > > So each line can be one of: > * a simple package id > * a package version range, using Cabal version range syntax > * a package name with a flag assignment, + for on, - for off Oops, one thing I forgot to mention is another entry syntax: baz That is, a package name with no version or range at all. This is shorthand for the version range style with no version constraint (.cabal files have a slightly odd syntax for that, "baz -any"). This is actually useful if you want to define a negative collection, e.g. all the packages that are deprecated (as a whole, not just single versions). Duncan From gershomb at gmail.com Tue Jul 14 19:50:08 2015 From: gershomb at gmail.com (Gershom B) Date: Tue, 14 Jul 2015 15:50:08 -0400 Subject: Request for feedback on spec/proposal for distributing package collections via hackage In-Reply-To: <1436898968-sup-2927@sabre> References: <1436878366.26857.532.camel@googlemail.com> <1436898968-sup-2927@sabre> Message-ID: > and is there any point to having a collection without versions in it? (If Cabal syntax is extended to support depending on collections as well as packages, yes?) So I think another use-case for collections, besides "version-locked", is sets of "blessed" packages. So we might want a collection for "verified compatible with windows 2008" or "platform packages" or "works with nhc" to be collections. And in those cases, specifying unique versions doesn't seem to make much sense. And one could imagine taking intersections -- i.e. "uhc-compatible _and_ in stackage LTS". In general I like this proposal as very minimal in terms of just providing and collecting data, and allowing that data to then be used by clients in various ways. --gershom On Tue, Jul 14, 2015 at 3:02 PM, Edward Z. Yang wrote: > Hello Duncan, > > In my eyes, this proposal looks like some sort of generalization of > Stackage; and one further use case is "special purpose" collection. My > big question: how composable are these collections really? I can't put > two collections with conflicting versions together (or can I? Do I > union?); and is there any point to having a collection without versions > in it? (If Cabal syntax is extended to support depending on collections > as well as packages, yes?) > > The classic use-case for package collections is deployment settings, ala > Stack, or even Cargo lockfiles / Bundler Gemfile.lock (versioned > collections). In all these use-cases package collections are treated as > non-compositional things. http://doc.crates.io/guide.html > http://bundler.io/v1.7/rationale.html#checking-your-code-into-version-control > Libraries (compositional) do NOT publish lockfiles: only executables > (non-compositional) DO. > > Re the file format, it seems fine; suitable for the lockfile use-case > and the Stackage use-case. Less sure about the unioning semantics. > > Edward > From ben at smart-cactus.org Sat Jul 18 12:00:33 2015 From: ben at smart-cactus.org (Ben Gamari) Date: Sat, 18 Jul 2015 14:00:33 +0200 Subject: New release of GHC 7.8 to support OS X El Capitan In-Reply-To: <550AC623-0CB9-4680-B189-A2CCB63B01A2@iki.fi> References: <550AC623-0CB9-4680-B189-A2CCB63B01A2@iki.fi> Message-ID: <87bnf97k0u.fsf@smart-cactus.org> Oleg Grenrus writes: > Hi, > first some background: there is an issue with having Haskell on the > next OS X ?El Capitan?: [1] caused by security updates in the > operating system. The cause is too old `unix` package, the same issue > as in [2] I would like to understand the root-cause of the issue. It seems that OS X will now raise EPERM instead of EACCES when certain files are accessed. That being said, it's not at all clear to me which system call is failing or why. Could someone familiar with El Capitan describe what exactly is going on here? > Could someone verify that my understanding of the issue is correct, > i.e. how Cabal-the-tool and Cabal-the-library interoperate. > > For me it seems, that one of the easiest ways to fix the issue (for > the users), is to have a new release in 7.8 branch, with updated > `unix` dependency. Or is it there too might hassle to be possible? I will bring this up in next week's GHC meeting. Cheers, - Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 472 bytes Desc: not available URL: From allbery.b at gmail.com Sat Jul 18 13:11:25 2015 From: allbery.b at gmail.com (Brandon Allbery) Date: Sat, 18 Jul 2015 09:11:25 -0400 Subject: New release of GHC 7.8 to support OS X El Capitan In-Reply-To: <87egk57kzq.fsf@smart-cactus.org> References: <550AC623-0CB9-4680-B189-A2CCB63B01A2@iki.fi> <87egk57kzq.fsf@smart-cactus.org> Message-ID: On Sat, Jul 18, 2015 at 7:39 AM, Ben Gamari wrote: > I would like to understand the root-cause of the issue. It seems that > OS X will now raise EPERM instead of EACCES when certain files are > accessed. That being said, it's not at all clear to me which system call > is failing or why. Could someone familiar with El Capitan describe what > exactly is going on here? > The trace showed access("/usr/bin/ar", 2) => -1/EPERM (instead of -1/EACCES). http://apple.stackexchange.com/questions/193368/what-is-the-rootless-feature-in-el-capitan-really appears relevant. Sounds to me like they automatically set a bunch of stuff immutable (chflags(1) schg flag; also see chflags(2), the underlying syscall) and bump the (equivalent of) securelevel so it can't be altered even by root after system boot. (Sadly, Apple did not bother to update the manpages to reflect launchd.) -- brandon s allbery kf8nh sine nomine associates allbery.b at gmail.com ballbery at sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net -------------- next part -------------- An HTML attachment was scrubbed... URL: