From graydon at mozilla.com Thu Feb 2 18:03:28 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Thu, 02 Feb 2012 18:03:28 -0800 Subject: [rust-dev] libuv updated Message-ID: <4F2B4070.9090106@mozilla.com> Hi, I've updated libuv. Normally this might not require mention but I had to make a couple hacks to get it working on mingw (they typically build on msvc) and this meant forking it to my own. Which meant changing the submodule URL. Turns out git caches the submodule URL. So you *might* need to change the submodule URLs in your .git/config to point to graydon/libuv rather than joyent/libuv. Let me know if you see any other breakage. I don't want to fall too far behind libuv. I've also un-cfg'ed all the tests on windows so we get proper library coverage there too. It's supposed to work on windows. That's part of the sales pitch :) -Graydon From rust-dev at tomlee.co Thu Feb 2 23:04:43 2012 From: rust-dev at tomlee.co (Tom Lee) Date: Thu, 2 Feb 2012 23:04:43 -0800 Subject: [rust-dev] libuv updated In-Reply-To: <4F2B4070.9090106@mozilla.com> References: <4F2B4070.9090106@mozilla.com> Message-ID: And those unfamiliar with git submodules (and/or a bit slow on the uptake like myself!) will also want to: cd src/libuv && git checkout master && git pull --rebase *after* you've got graydon/libuv is in your .git/config -- otherwise it continues to reference the last commit to joyent/libuv prior to Graydon's fork. :) (while I'm on the topic, is there a nicer way to ask git to do this for you?) Cheers, Tom On Thu, Feb 2, 2012 at 6:03 PM, Graydon Hoare wrote: > Hi, > > I've updated libuv. Normally this might not require mention but I had to > make a couple hacks to get it working on mingw (they typically build on > msvc) and this meant forking it to my own. Which meant changing the > submodule URL. Turns out git caches the submodule URL. So you *might* need > to change the submodule URLs in your .git/config to point to graydon/libuv > rather than joyent/libuv. > > Let me know if you see any other breakage. I don't want to fall too far > behind libuv. I've also un-cfg'ed all the tests on windows so we get proper > library coverage there too. It's supposed to work on windows. That's part > of the sales pitch :) > > -Graydon > ______________________________**_________________ > Rust-dev mailing list > Rust-dev at mozilla.org > https://mail.mozilla.org/**listinfo/rust-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marijnh at gmail.com Thu Feb 2 23:46:08 2012 From: marijnh at gmail.com (Marijn Haverbeke) Date: Fri, 3 Feb 2012 08:46:08 +0100 Subject: [rust-dev] libuv updated In-Reply-To: References: <4F2B4070.9090106@mozilla.com> Message-ID: >> cd src/libuv && git checkout master && git pull --rebase Thanks! You saved me at least ten minutes of confused stumbling. From graydon at mozilla.com Fri Feb 3 07:34:11 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Fri, 03 Feb 2012 07:34:11 -0800 Subject: [rust-dev] libuv updated In-Reply-To: References: <4F2B4070.9090106@mozilla.com> Message-ID: <4F2BFE73.5010006@mozilla.com> On 02/02/2012 11:01 PM, Tom Lee wrote: > And those unfamiliar with git submodules (and/or a bit slow on the > uptake like myself!) will also want to: > > cd src/libuv && git checkout master && git pull --rebase > > > /after/ you've got graydon/libuv is in your .git/config -- otherwise it > continues to reference the last commit to joyent/libuv prior to > Graydon's fork. :) > > (while I'm on the topic, is there a nicer way to ask git to do this for > you?) I'm confused. As far as I know the submodule code in our configury *should* be doing this for you. Did it not? -Graydon From rust-dev at tomlee.co Fri Feb 3 12:25:18 2012 From: rust-dev at tomlee.co (Tom Lee) Date: Fri, 3 Feb 2012 12:25:18 -0800 Subject: [rust-dev] libuv updated In-Reply-To: <4F2BFE73.5010006@mozilla.com> References: <4F2B4070.9090106@mozilla.com> <4F2BFE73.5010006@mozilla.com> Message-ID: Can't remember how I did the build, sorry. Is it supposed to happen as part of a ./configure or something? It's possible I ran a make without running configure. In any case, the build left the libuv head pointing at the last joyent commit. :( Cheers, Tom On Feb 3, 2012 7:34 AM, "Graydon Hoare" wrote: > On 02/02/2012 11:01 PM, Tom Lee wrote: > >> And those unfamiliar with git submodules (and/or a bit slow on the >> uptake like myself!) will also want to: >> >> cd src/libuv && git checkout master && git pull --rebase >> >> >> /after/ you've got graydon/libuv is in your .git/config -- otherwise it >> continues to reference the last commit to joyent/libuv prior to >> Graydon's fork. :) >> >> (while I'm on the topic, is there a nicer way to ask git to do this for >> you?) >> > > I'm confused. As far as I know the submodule code in our configury > *should* be doing this for you. Did it not? > > -Graydon > > > ______________________________**_________________ > Rust-dev mailing list > Rust-dev at mozilla.org > https://mail.mozilla.org/**listinfo/rust-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From talex5 at gmail.com Sat Feb 4 04:13:23 2012 From: talex5 at gmail.com (Thomas Leonard) Date: Sat, 4 Feb 2012 12:13:23 +0000 Subject: [rust-dev] Cargo requirements In-Reply-To: <4F27345E.1040101@mozilla.com> References: <4F2067CB.6010309@mozilla.com> <4F27345E.1040101@mozilla.com> Message-ID: On 31 January 2012 00:22, Graydon Hoare wrote: > On 1/26/2012 12:36 AM, Thomas Leonard wrote: Thanks. I'm documenting these issues here: http://0install.net/why-not.html I've added your replies to our Google Docs spreadsheet here: https://docs.google.com/spreadsheet/ccc?key=0AlQ_ban3-1YrdFJfdlJwNkpENUdMVXJRbE04d29LN1E >> Yes, simplicity is a big problem. Unfortunately, all package systems start >> simple and grow more complex over time :-( > > All systems of any sort start simple and grow more complex, yes. But one > sometimes "starts new" to see what level of complexity is legacy and can be > safely discarded. We live in a world with pervasive distributed version > control now. We didn't used to. Agreed. Although 0install pre-dates Git, it (like Git) was influenced by Monotone (using hashes of trees of hashes to identify revisions). [...] >> That works if you don't have diamond dependencies (most systems seem >> to start this way). But then what if I have: > > No, we very specifically built the versioning scheme in our crates to permit > loading and using multiple versions of a library in the same binary (with a > deterministic, acyclic dependency graph). IOW the "diamond" doesn't have a > join point unless they're metadata-equivalent versions. This is baked into > rust's library name-mangling scheme. If I understand what you're saying: - if a crate A depends on B, and - B depends on C, then - A does not see (or conflict with) symbols from C. If A does depend on C, it may use a different version without conflicts. That's good. I think all modern systems should do that. Ceylon is the same, I think, and ebox (using 0install) uses the same idea to protect applications from the effects of malicious libraries. But it doesn't remove the need to handle diamond dependencies. Very often (most of the time, I think), if two libraries depend on a common one then it's because they need to communicate using a common API or common set of data-types. For example, if I write an application using GTK to handle windows, and I import a widget library, then it's no good having the widget library using its own version of GTK, because I won't be able to embed its widgets in my windows. As another example, >> use logging (url = "git://github.com/user/logging.git", vers = "1.12.3"); >> use foo (url = "git://github.com/user/foo.git", vers = "1.0"); >> >> and foo has: >> >> use logging (url = "git://github.com/user/logging.git", vers = "1.12.4"); > > You wind up with two copies of logging in your process, and they don't > conflict because all their names are version-qualified and there are no > singletons or global (unless you do unsafe things; don't do that). This is > what we built already. Sure, but now the user needs to manage two separate logging configurations for the same application, or risk them overwriting each other's log files (although this is probably a bad example, as I see that Rust very sensibly has built-in logging support). By the way, does this mean that a Rust program must link against exactly the same version of the library that was used at build time? e.g. if I upgrade libssl then I must rebuild all applications using it? -- Dr Thomas Leonard? ? ? ? http://0install.net/ GPG: 9242 9807 C985 3C07 44A6? 8B9A AE07 8280 59A5 3CC1 GPG: DA98 25AE CAD0 8975 7CDA? BD8E 0713 3F96 CA74 D8BA From niko at alum.mit.edu Sat Feb 4 07:58:41 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Sat, 04 Feb 2012 07:58:41 -0800 Subject: [rust-dev] bind syntax change proposal Message-ID: <4F2D55B1.2090109@alum.mit.edu> Hello, As some of you have no doubt seen, in #1649 [1] I proposed a lighterweight bind syntax which (basically) omits the keyword bind. Here is the proposal: https://github.com/mozilla/rust/wiki/Proposal-for-bindless-bind I would paste it inline below but I think it's much easier to read online, due to the formatting. There are several open-ish questions at the end and I would appreciate your thoughts, as well as an overall "yay or nay" opinion. Thanks. Niko [1] https://github.com/mozilla/rust/issues/1649 From info at bnoordhuis.nl Sat Feb 4 09:35:22 2012 From: info at bnoordhuis.nl (Ben Noordhuis) Date: Sat, 4 Feb 2012 18:35:22 +0100 Subject: [rust-dev] libuv updated In-Reply-To: <4F2B4070.9090106@mozilla.com> References: <4F2B4070.9090106@mozilla.com> Message-ID: On Fri, Feb 3, 2012 at 03:03, Graydon Hoare wrote: > Hi, > > I've updated libuv. Normally this might not require mention but I had to > make a couple hacks to get it working on mingw (they typically build on > msvc) and this meant forking it to my own. Which meant changing the > submodule URL. Turns out git caches the submodule URL. So you *might* need > to change the submodule URLs in your .git/config to point to graydon/libuv > rather than joyent/libuv. > > Let me know if you see any other breakage. I don't want to fall too far > behind libuv. I've also un-cfg'ed all the tests on windows so we get proper > library coverage there too. It's supposed to work on windows. That's part of > the sales pitch :) Graydon, what issues did you run into? We can probably fix it. From graydon at mozilla.com Sat Feb 4 10:26:16 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Sat, 04 Feb 2012 10:26:16 -0800 Subject: [rust-dev] bind syntax change proposal In-Reply-To: <4F2D55B1.2090109@alum.mit.edu> References: <4F2D55B1.2090109@alum.mit.edu> Message-ID: <4F2D7848.7060604@mozilla.com> On 04/02/2012 7:58 AM, Niko Matsakis wrote: > Hello, > > As some of you have no doubt seen, in #1649 [1] I proposed a > lighterweight bind syntax which (basically) omits the keyword bind. Here > is the proposal: > > https://github.com/mozilla/rust/wiki/Proposal-for-bindless-bind > > I would paste it inline below but I think it's much easier to read > online, due to the formatting. There are several open-ish questions at > the end and I would appreciate your thoughts, as well as an overall "yay > or nay" opinion. I'm mostly ok with this; answers to your issues follow: 1. I'm ok with binding on operators, yeah. We've lacked a good way to easily denote op+ and such for a while, and _+_ fits nicely in our syntax. Might get difficult to infer the overload though. Tread lightly. 2. This one is, I think, not OK. It's effectively changing a bunch of expressions from call-by-value to non-memoizing call-by-need due to a _ in the neighbourhood. That's too hazardous. IOW I think it needs to remain: f(a.b, _) => { let x = a.b; {|y| f(x,y) } } The counterargument you put forth is "consolidated code path", but ... I don't think that's sufficient justification. For constants it makes no difference, but for mutable values and expressions with side effects it makes a big, scary difference. Bind is a notation borrowed from Sather, incidentally, and it binds by value. Even if you want to look to more-mainstream languages for precedent, you would look to the capture rules for lambda and currying. Both of those, in most reasonable languages, capture values rather than expressions (or variables). Indeed, the "closures from for-loops close over the variable not the value" problem was being discussed as a major js and C# anti-feature on #rust the other day. I don't think it's right to be capturing un-evaluated expressions like this. 3. I don't entirely understand the concern, and I'm not sure it still stands if you do what I'm requesting in #2 anyways, so going to defer. Maybe we can talk more in person or on IRC :) 4. It's not too magical. The only major thing this proposal involves is kicking out the `bind` keyword and generalizing the notation to hit expression-positions we didn't previously know how to bind. So, 'bind' already had a bug open for binding self in method calls, and I think people had at least mentioned binding operators before (or auto-currying them when quoted a certain way). Sather supported binding more things than our 'bind' does presently anyways: http://www.icsi.berkeley.edu/~sather/faq.html#17 So ... I'm ok with expanding its reach. I'm also ok with just kicking it out altogether, since we have Real Lambdas now. It's redundant sugar, one way or another, given environment capture. I have little opinion on keeping or removing the keyword, if bind of any sort survives. Rust started very keyword-y (due to my aesthetic preferences, modeled on Napier88, Alef and Sather) but there has been virtually unanimous, steady pressure to eliminate keywords from everyone else I show it to, and it's hard to keep up such an argument indefinitely. I am clearly in the same aesthetic minority here as with my taste for for Boston City Hall. -Graydon From graydon at mozilla.com Sat Feb 4 11:56:53 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Sat, 04 Feb 2012 11:56:53 -0800 Subject: [rust-dev] libuv updated In-Reply-To: References: <4F2B4070.9090106@mozilla.com> Message-ID: <4F2D8D85.5060501@mozilla.com> On 04/02/2012 9:35 AM, Ben Noordhuis wrote: > Graydon, what issues did you run into? We can probably fix it. Oh, I just ifdef'ed out a couple API calls that mingw headers don't have. It's probably mingw's bug to fix; I don't think it's anything "serious", just doesn't compile. The markers are the #ifndefs I injected in the single additional commit in my branch. https://github.com/graydon/libuv/commit/1170ffba3ac5191930b40c897d4569a9d8a296a3 -Graydon From krux02 at googlemail.com Sat Feb 4 06:21:58 2012 From: krux02 at googlemail.com (=?ISO-8859-1?Q?Arne_D=F6ring?=) Date: Sat, 4 Feb 2012 15:21:58 +0100 Subject: [rust-dev] Suggestions Message-ID: Hi Maybe I should introduce myself first. I am Arne a Computer Science student and my bachelor degree is not far away anymore. I am very interested in new programming languages and lately I found out about Rust. Because of the fact, that in the last time my most used language is Scala I was kind of interested weather there is something that behaves to C++ similar as Scala behaves to Java. And rust seems in my opinion to be tho closest to Scala. Now I would like to share my experience with Scala, so that Rust can make things right, that were also right in Scala. Normally I really prefer Forums and other less direct forms of communication to introduce my Suggestions, because then more people are able to read them, but Rust doesn't have an official discussion forum (I would really like that). First of all, generic Types with angle brackets. We all know generic types from C++, Java, C# etc, and they all use angle brackets for generic types. at least they call it angle brackets, but in fact they are the less and greater sign <> angle brackets are these ??. This leads to many probles if you also want to use the < sign as an actual less sign. We all know that in C++ it was long time not possible to have >> as two closing angle brackets of templates, because it was parsed as a right shift operator. And there are a lot of ambiguous situations, for example if you want to use < within generic types (Scala does if for definition of inheritance ). html forbids to use <,> at all. The D programming language had an article about it, but I cant find it anymore. Scala uses those [] (array index is with normal braces ()). Afer a little while were it was different to me, I really liked it, because they are real brackets. I would really like to see those brackets it Rust. The second suggestion is concerning tho #fmt macro. #fmt works like printf, but its string is parsed at compile time, so that errors might be thrown when the string is incorrect. So when you unwind this format string at compile time, you know also all variable names and their types from scope. So it might be more natural to directly import variable names from the scope inside of the String, like it is done in many scripting languages. This is also done in tho Programming language Nemerle. Arne From marijnh at gmail.com Sat Feb 4 13:05:53 2012 From: marijnh at gmail.com (Marijn Haverbeke) Date: Sat, 4 Feb 2012 22:05:53 +0100 Subject: [rust-dev] bind syntax change proposal In-Reply-To: <4F2D7848.7060604@mozilla.com> References: <4F2D55B1.2090109@alum.mit.edu> <4F2D7848.7060604@mozilla.com> Message-ID: First, I like this a lot. I think applying it to operators as well is definitely a good thing. I don't feel strongly about these closures needing to copy the bound values. If you consider them a syntax for currying, one would expect them to copy, but you could also look at them as a shorthand for block/lambda syntax, in which case not copying makes perfect sense. The difference might be moot, actually, since fn@ will copy anyway, and blocks don't escape the call that uses them mutation of the closed over variables will rarely be an issue. Best, Marijn From banderson at mozilla.com Sat Feb 4 13:14:29 2012 From: banderson at mozilla.com (Brian Anderson) Date: Sat, 4 Feb 2012 13:14:29 -0800 (PST) Subject: [rust-dev] Suggestions In-Reply-To: Message-ID: <272652566.2411218.1328390069431.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> ----- Original Message ----- > From: "Arne D?ring" > To: rust-dev at mozilla.org > Sent: Saturday, February 4, 2012 6:21:58 AM > Subject: [rust-dev] Suggestions > > Hi Hi, Arne. > Maybe I should introduce myself first. I am Arne a Computer Science > student and my bachelor degree is not far away anymore. I am very > interested in new programming languages and lately I found out about > Rust. Because of the fact, that in the last time my most used > language > is Scala I was kind of interested weather there is something that > behaves to C++ similar as Scala behaves to Java. And rust seems in my > opinion to be tho closest to Scala. Now I would like to share my > experience with Scala, so that Rust can make things right, that were > also right in Scala. Normally I really prefer Forums and other less > direct forms of communication to introduce my Suggestions, because > then more people are able to read them, but Rust doesn't have an > official discussion forum (I would really like that). Noted. > First of all, generic Types with angle brackets. We all know generic > types from C++, Java, C# etc, and they all use angle brackets for > generic types. at least they call it angle brackets, but in fact they > are the less and greater sign <> angle brackets are these ??. This > leads to many probles if you also want to use the < sign as an actual > less sign. We all know that in C++ it was long time not possible to > have >> as two closing angle brackets of templates, because it was > parsed as a right shift operator. And there are a lot of ambiguous > situations, for example if you want to use < within generic types > (Scala does if for definition of inheritance ). html forbids to use > <,> at all. The D programming language had an article about it, but I > cant find it anymore. Scala uses those [] (array index is with normal > braces ()). Afer a little while were it was different to me, I really > liked it, because they are real brackets. I would really like to see > those brackets it Rust. We don't have the same problems as C++ here because we made a compromise that wherever there would be a potential ambiguity the angle brackets have to be proceeded with a double colon. So for function calls, for example you have to write `do_something::(0)`. In most cases this ugliness is ameliorated because type inference will figure out the type parameters and they don't have to be written at all. That said, this is one of my nits with the language as well. In an earlier version we did use square brackets for type parameters, and i was ok with it (but then there was an ambiguity between array indexing with () and function calls). > The second suggestion is concerning tho #fmt macro. #fmt works like > printf, but its string is parsed at compile time, so that errors > might > be thrown when the string is incorrect. So when you unwind this > format > string at compile time, you know also all variable names and their > types from scope. So it might be more natural to directly import > variable names from the scope inside of the String, like it is done > in > many scripting languages. This is also done in tho Programming > language Nemerle. I would be interested in seeing an extension that explores other ways to do string formatting. I think #fmt uses a printf style because it is well-known and was an obvious early approach to take. It is a bit unwieldy though and other languages have done this in a nicer way. From pwalton at mozilla.com Sat Feb 4 13:17:54 2012 From: pwalton at mozilla.com (Patrick Walton) Date: Sat, 04 Feb 2012 13:17:54 -0800 Subject: [rust-dev] Suggestions In-Reply-To: References: Message-ID: <4F2DA082.3020309@mozilla.com> On 02/04/2012 06:21 AM, Arne D?ring wrote: > The second suggestion is concerning tho #fmt macro. #fmt works like > printf, but its string is parsed at compile time, so that errors might > be thrown when the string is incorrect. So when you unwind this format > string at compile time, you know also all variable names and their > types from scope. So it might be more natural to directly import > variable names from the scope inside of the String, like it is done in > many scripting languages. This is also done in tho Programming > language Nemerle. Well, #fmt is a macro, and I would generally be opposed to macros implicitly capturing the local variables in scope -- that's unhygienic. Patrick From pwalton at mozilla.com Sat Feb 4 13:50:31 2012 From: pwalton at mozilla.com (Patrick Walton) Date: Sat, 04 Feb 2012 13:50:31 -0800 Subject: [rust-dev] bind syntax change proposal In-Reply-To: <4F2D7848.7060604@mozilla.com> References: <4F2D55B1.2090109@alum.mit.edu> <4F2D7848.7060604@mozilla.com> Message-ID: <4F2DA827.30908@mozilla.com> I'm ok with this, but I agree with Graydon re. the second issue, and regarding nesting I think the rules should be as simple as possible. The danger of eliminating the keyword "bind" is that now the lambda is implicit and it needs to be very clear what's going on -- the eye needs to be able to infer where the lambda is. I'm a little worried about how mind-twisting the scoping examples you gave were. I think solution #2 -- that we need to accept _.b(_) but only if b() turns out to be a method, as you said -- is the right one. (Also, I hate to bring it up again, but '->' for methods solves this...) Supporting bind on operators is a bit more perilous from a readability standpoint, I'd tend to think, because now it's not just PrimaryExpressions (in the Java sense) that a reader has to worry about to find where the lambdas are but many grammar productions. To make this concrete, the grammar for PrimaryExpression could be something like (simplified): PrimaryExpression ::== MethodCall | FunctionCall | '(' Expression ')' | Identifier | Literal MethodCall ::== PrimaryOrUnderscore '.' Identifier '(' ExpressionOrUnderscore (',' ExpressionOrUnderscore)* ')' FunctionCall ::== PrimaryOrUnderscore '(' ExpressionOrUnderscore (',' ExpressionOrUnderscore)* ')' PrimaryOrUnderscore ::== '_' | PrimaryExpression ExpressionOrUnderscore ::== '_' | Expression Here, assuming I understand your proposal correctly, the rule is relatively simple: where there's an underscore, the lambda is always positioned before the nearest parent PrimaryExpression. But with operators, the rules become more complex. I don't know if most programmers would have an intuition about what "_ + _ * _" means, for example. But generally I like the proposal as a whole -- it's a nice bit of syntactic sugar. Patrick From niko at alum.mit.edu Sat Feb 4 17:37:48 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Sat, 04 Feb 2012 17:37:48 -0800 Subject: [rust-dev] bind syntax change proposal In-Reply-To: <4F2D7848.7060604@mozilla.com> References: <4F2D55B1.2090109@alum.mit.edu> <4F2D7848.7060604@mozilla.com> Message-ID: <4F2DDD6C.8010704@alum.mit.edu> On 2/4/12 10:26 AM, Graydon Hoare wrote: > 1. I'm ok with binding on operators, yeah. We've lacked a good way to > easily denote op+ and such for a while, and _+_ fits nicely in our > syntax. Might get difficult to infer the overload though. Tread lightly. Yes, when you go beyond curried arguments to calls?for example with receivers?the inference is somewhat limited. It must come from the expected type, basically. If that's insufficient then you must use fn@ or some other form with explicit type annotations. Anyway, I think I would prefer one of two paths: - function calls and methods only (methods being enforced in `typeck`); - an expanded set of operations containing: - fields - function calls - unary operators like `*_` (hmm, those look odd) - a series of binary operators (`_ + _ - _`) - constructors (`(_, _)`, `[_, _]`) In other words, either allow very little, or be quite general. Seems like anything in between is confusing. > 2. This one is, I think, not OK. It's effectively changing a bunch of > expressions from call-by-value to non-memoizing call-by-need due to a > _ in the neighbourhood. That's too hazardous. IOW I think it needs to > remain: > > f(a.b, _) => { let x = a.b; {|y| f(x,y) } } > > The counterargument you put forth is "consolidated code path", but ... > I don't think that's sufficient justification. For constants it makes > no difference, but for mutable values and expressions with side > effects it makes a big, scary difference. I agree it's a change. FWIW, I didn't intend to argue that we should make this change to achieve a consolidated code path. I think we ought to decide what the language should do first and let the code work itself out, of course. I didn't express this well. Basically the question is whether `_` should be a notation for currying (as `bind` is today) or a shorthand for closures (as I was proposing). If we decide it should be currying notation, then I agree the semantics should stay as they are and the code will just have to have two paths. I don't really think either way is more hazardous?both are surprising depending on what you expect `_` to be (Scala, for example, supports `_` as a shorthand for closures, so `c() + _` does not evaluate `c()`). I do think there is a case for `_` as closure notation: it's one less concept. However, I am ok with either way, depending on what people want. I think if `_` is a currying notation, though, I personally would prefer to restrict its use to calls rather than allow a wide range of expressions. Anyway, let's discuss more, gotta' run... Niko From niko at alum.mit.edu Sat Feb 4 17:55:51 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Sat, 04 Feb 2012 17:55:51 -0800 Subject: [rust-dev] bind syntax change proposal In-Reply-To: <4F2DA827.30908@mozilla.com> References: <4F2D55B1.2090109@alum.mit.edu> <4F2D7848.7060604@mozilla.com> <4F2DA827.30908@mozilla.com> Message-ID: <4F2DE1A7.1080808@alum.mit.edu> On 2/4/12 1:50 PM, Patrick Walton wrote: > Here, assuming I understand your proposal correctly, the rule is > relatively simple: where there's an underscore, the lambda is always > positioned before the nearest parent PrimaryExpression. But with > operators, the rules become more complex. I don't know if most > programmers would have an intuition about what "_ + _ * _" means, for > example. The rules you specify are precisely what I had in mind as the more limited proposal. I was also worried about programmers being able to deduce the scope of the `_`. And no, I don't quite know what `_ + _ * _` should mean, really. =) Anyway, I am happy to update my patch to make `_` be currying (as both you and graydon preferred). I think for now what I'll do is try to slice up my patch into pieces: for example, first the improved inference (which is good for closures in general), then maybe the syntax compiling to the same bind AST and trans as we have now, and finally get method call binds working. Niko From andre.pang at gmail.com Sun Feb 5 15:39:36 2012 From: andre.pang at gmail.com (=?iso-8859-1?Q?Andr=E9_Pang?=) Date: Sun, 5 Feb 2012 15:39:36 -0800 Subject: [rust-dev] error: can't find crate for 'core' Message-ID: Hi! I'm trying out rust for the first time, and successfully compiled and installed rustc. I hit this error attempting to compile "hello world": > ~ % cat hello.rs > use std; > fn main(args: [str]) { > std::io::println("hello world from '" + args[0] + "'!"); > } > ~ % rustc hello.rs > hello.rs:1:0: 1:0 error: can't find crate for 'core' > hello.rs:1 use std; > ^ Googling turned up a possible problem of not running "make all" before "make install" , but make all does nothing for me, so everything looks built. "make check" passes fine. This happens to me with both the 0.1 tarball and a github clone from Feb 5 2012 (91b6dc5c8ed8d839006de4ea6a7e8cd6727db93d). Platform is Mac OS X 10.6.7. Any clues? Perhaps relatedly, cargo looks like it's complaining alot. "cargo init" warns about "warning: signature verification failed for sources.json", but finally appears to do something if I run it from src/cargo/ (where sources.json is). "cargo sync" then passes, but any "cargo install" commands immediately fail with the same error; e.g. > ~ % cargo install sqlite > warning: Malformed source json: erickt (missing description) > warning: Malformed source json: erickt (missing description) > info: Installing with git from https://github.com/linuxfood/rustsqlite... > Cloning into /Users/andre/.cargo/work/QubPvewYAX3iGg2h... > remote: Counting objects: 64, done. > remote: Compressing objects: 100% (28/28), done. > remote: Total 64 (delta 36), reused 62 (delta 34) > Unpacking objects: 100% (64/64), done. > error: rustc failed: 101 > ./sqlite.rc:1:0: 1:0 error: can't find crate for 'core' > ./sqlite.rc:1 #[link( From ozone at algorithm.com.au Sun Feb 5 16:00:07 2012 From: ozone at algorithm.com.au (=?iso-8859-1?Q?Andr=E9_Pang?=) Date: Sun, 5 Feb 2012 16:00:07 -0800 Subject: [rust-dev] error: can't find crate for 'core' Message-ID: <86C09E07-9370-49DC-A5C0-BB45914A84B8@algorithm.com.au> Hi! I'm trying out rust for the first time, and successfully compiled and installed rustc. I hit this error attempting to compile "hello world": > ~ % cat hello.rs > use std; > fn main(args: [str]) { > std::io::println("hello world from '" + args[0] + "'!"); > } > ~ % rustc hello.rs > hello.rs:1:0: 1:0 error: can't find crate for 'core' > hello.rs:1 use std; > ^ Googling turned up a possible problem of not running "make all" before "make install" , but make all does nothing for me, so everything looks built. "make check" passes fine. This happens to me with both the 0.1 tarball and a github clone from Feb 5 2012 (91b6dc5c8ed8d839006de4ea6a7e8cd6727db93d). Platform is Mac OS X 10.6.7. Any clues? Perhaps relatedly, cargo looks like it's complaining alot. "cargo init" warns about "warning: signature verification failed for sources.json", but finally appears to do something if I run it from src/cargo/ (where sources.json is). "cargo sync" then passes, but any "cargo install" commands immediately fail with the same error; e.g. > ~ % cargo install sqlite > warning: Malformed source json: erickt (missing description) > warning: Malformed source json: erickt (missing description) > info: Installing with git from https://github.com/linuxfood/rustsqlite... > Cloning into /Users/andre/.cargo/work/QubPvewYAX3iGg2h... > remote: Counting objects: 64, done. > remote: Compressing objects: 100% (28/28), done. > remote: Total 64 (delta 36), reused 62 (delta 34) > Unpacking objects: 100% (64/64), done. > error: rustc failed: 101 > ./sqlite.rc:1:0: 1:0 error: can't find crate for 'core' > ./sqlite.rc:1 #[link( From banderson at mozilla.com Sun Feb 5 16:04:41 2012 From: banderson at mozilla.com (Brian Anderson) Date: Sun, 5 Feb 2012 16:04:41 -0800 (PST) Subject: [rust-dev] error: can't find crate for 'core' In-Reply-To: Message-ID: <176886427.2619043.1328486681788.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> ----- Original Message ----- > From: "Andr? Pang" > To: rust-dev at mozilla.org > Sent: Sunday, February 5, 2012 3:39:36 PM > Subject: [rust-dev] error: can't find crate for 'core' > > Hi! I'm trying out rust for the first time, and successfully > compiled and installed rustc. I hit this error attempting to > compile "hello world": > > > ~ % cat hello.rs > > use std; > > fn main(args: [str]) { > > std::io::println("hello world from '" + args[0] + "'!"); > > } > > ~ % rustc hello.rs > > hello.rs:1:0: 1:0 error: can't find crate for 'core' > > hello.rs:1 use std; > > ^ If you run again with "RUST_LOG=rustc::metadata::creader" you should get a bunch of information about where rustc is looking for the core crate. For example, when I compile something I see the following: ... rust: "inspecting file /home/banderson/Dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustc/x86_64-unknown-linux-gnu/lib/librustc.so" rust: "skipping /home/banderson/Dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustc/x86_64-unknown-linux-gnu/lib/librustc.so, doesn't look like libcore-*. so" rust: "inspecting file /home/banderson/Dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustc/x86_64-unknown-linux-gnu/lib/libcore-14bd852465126fe7-0.1.so" rust: "/home/banderson/Dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustc/x86_64-unknown-linux-gnu/lib/libcore-14bd852465126fe7-0.1.so is a candidate" rust: "matching 0 metadata requirements against 4 items" rust: "crate metadata:" rust: " name = "core"" rust: " vers = "0.1"" rust: " uuid = "c70c24a7-5551-4f73-8e37-380b11d80be8"" rust: " url = "http://rust-lang.org/src/core"" rust: "found /home/banderson/Dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustc/x86_64-unknown-linux-gnu/lib/libcore-14bd852465126fe7-0.1.so with matching metadata" On a 64-bit OS X system I would expect your rustc to find libcore at "../lib/rustc/x86_64-apple-darwin/lib/", with the path starting from the directory your rustc binary lives in. First I would check that you have a library called libcore-*-*.so living at that location and that rustc appears to be searching that location for crates. From kevin at atkinson.dhs.org Sun Feb 5 19:09:49 2012 From: kevin at atkinson.dhs.org (Kevin Atkinson) Date: Sun, 5 Feb 2012 20:09:49 -0700 (MST) Subject: [rust-dev] Suggestions In-Reply-To: <4F2DA082.3020309@mozilla.com> References: <4F2DA082.3020309@mozilla.com> Message-ID: On Sat, 4 Feb 2012, Patrick Walton wrote: > On 02/04/2012 06:21 AM, Arne D?ring wrote: >> The second suggestion is concerning tho #fmt macro. #fmt works like >> printf, but its string is parsed at compile time, so that errors might >> be thrown when the string is incorrect. So when you unwind this format >> string at compile time, you know also all variable names and their >> types from scope. So it might be more natural to directly import >> variable names from the scope inside of the String, like it is done in >> many scripting languages. This is also done in tho Programming >> language Nemerle. > > Well, #fmt is a macro, and I would generally be opposed to macros implicitly > capturing the local variables in scope -- that's unhygienic. Personally I do not see the difference between #fmt("%s", some_str) and #fmt("$some_str"); as far as hygiene is concerned. In both cases the variable is provided by the user of the macro, it is just in the later case the variable happens to be embedded within a string. From ozone at algorithm.com.au Sun Feb 5 19:24:23 2012 From: ozone at algorithm.com.au (=?iso-8859-1?Q?Andr=E9_Pang?=) Date: Sun, 5 Feb 2012 19:24:23 -0800 Subject: [rust-dev] error: can't find crate for 'core' In-Reply-To: <176886427.2619043.1328486681788.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> References: <176886427.2619043.1328486681788.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Message-ID: On Feb 5, 2012, at 4:04 PM, Brian Anderson wrote: >> Hi! I'm trying out rust for the first time, and successfully >> compiled and installed rustc. I hit this error attempting to >> compile "hello world": >> >>> ~ % cat hello.rs >>> use std; >>> fn main(args: [str]) { >>> std::io::println("hello world from '" + args[0] + "'!"); >>> } >>> ~ % rustc hello.rs >>> hello.rs:1:0: 1:0 error: can't find crate for 'core' >>> hello.rs:1 use std; >>> ^ > > If you run again with "RUST_LOG=rustc::metadata::creader" you should get a bunch of information about where rustc is looking for the core crate. For example, when I compile something I see the following: > > ... > rust: "inspecting file /home/banderson/Dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustc/x86_64-unknown-linux-gnu/lib/librustc.so" > rust: "skipping /home/banderson/Dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustc/x86_64-unknown-linux-gnu/lib/librustc.so, doesn't look like libcore-*. > so" > rust: "inspecting file /home/banderson/Dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustc/x86_64-unknown-linux-gnu/lib/libcore-14bd852465126fe7-0.1.so" > rust: "/home/banderson/Dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustc/x86_64-unknown-linux-gnu/lib/libcore-14bd852465126fe7-0.1.so is a candidate" > rust: "matching 0 metadata requirements against 4 items" > rust: "crate metadata:" > rust: " name = "core"" > rust: " vers = "0.1"" > rust: " uuid = "c70c24a7-5551-4f73-8e37-380b11d80be8"" > rust: " url = "http://rust-lang.org/src/core"" > rust: "found /home/banderson/Dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustc/x86_64-unknown-linux-gnu/lib/libcore-14bd852465126fe7-0.1.so with matching metadata" Hmm, I get something rather odd if I try this: ~ % RUST_LOG=rustc::metadata::creader rustc ~/hello.rs /Users/andre/hello.rs:1:0: 1:0 error: can't find crate for 'core' /Users/andre/hello.rs:1 use std; ^ rust: upcall fail 'explicit failure', src/comp/driver/diagnostic.rs:52 rust: upcall fail 'explicit failure', src/comp/driver/rustc.rs:184 rust: domain main @0x10182be00 root task failed I tried it with the compiler built at x86_64-apple-darwin/stage3/bin/ too; same result. From masklinn at masklinn.net Mon Feb 6 00:07:04 2012 From: masklinn at masklinn.net (Masklinn) Date: Mon, 6 Feb 2012 09:07:04 +0100 Subject: [rust-dev] Suggestions In-Reply-To: <272652566.2411218.1328390069431.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> References: <272652566.2411218.1328390069431.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Message-ID: <491353D2-2BAF-48DC-9874-D04219EDB44C@masklinn.net> On 2012-02-04, at 22:14 , Brian Anderson wrote: > > I would be interested in seeing an extension that explores other ways to do string formatting. I think #fmt uses a printf style because it is well-known and was an obvious early approach to take. It is a bit unwieldy though and other languages have done this in a nicer way. These ways may or may not work in Rust though, not sure. String formatting approaches I know of: = printf-style (most C-derived languages, numerous others as well) "this is a string %s", interpolated + well known + pretty simple + can be statically verified - positional interpolation only (annoying for e.g. translations where the order of interpolated terms could be changed) - formatting string is unclear and unwieldy as it becomes big - formatting mini-language is pretty horrible == printf-style extensions: keyword (Python) "this is a string %(interpolated)s", {'interpolated': somevar} + in-band name can be different (and more descriptive for the new context) than the variable name + when interpolating multiple values, values can easily be moved around, also allows for easily interpolating the same value in multiple places + can ignore some of the values provided for interpolation if desired/desirable + can be statically verified when using structures/records - redundancy between the format string and the map/structure provided, Python can also pick the local environment as a dict (`locals()`) but that brings other risks. = Direct variable interpolation (Perl, PHP, shells) "this is a string $interpolated" + simple and (usually) obvious - harder to statically check - needs a standard conversion protocol or interface (a -> String) == extensions: expressions (PHP, Ruby) "this is a string {$interpolated}" "this is a string #{interpolated}" In this case, the format string contains an expression shim, which fully evaluates its expressions in the context of the surrounding code. It's possible to call any expression construct in the format string. + interpolated values can be formatted using standard language functions and methods - untrusted formatting input are code injection vectors (hard to use for e.g. translations) = C#-string formatting (C#, Python >= 2.6, preferred to printf-style in Python 3) "this is a string {}", interpolated "this is a string {0}", interpolated "this is a string {interpolated}", {'interpolated': somevar} + terseness even superior to printf-style + more flexibility (e.g. moving positional arguments around by providing an integer index to `{}`) + can easily be extended with things like attributes or index access (as Python does) - formatting mini-language just as bad as printf-style - also requires a standard protocol or interface (? la Show) to convert values to strings = Common-lisp's format you probably don't want to know From olson.jeffery at gmail.com Mon Feb 6 09:18:07 2012 From: olson.jeffery at gmail.com (Jeffery Olson) Date: Mon, 6 Feb 2012 09:18:07 -0800 Subject: [rust-dev] net, libuv, etc Message-ID: First of all, hello list. I'm pretty excited about rust and hope to have an opportunity to make contributions and learn. I'm curious about the current state of the 'net' module and libuv. Looking at the docs, there's just some stubs around IPv4 stuff. I've heard around that there's interest in building the module atop libuv. Is libuv going to be a core dependency? Is there an existing wrapper? I hope you will ship a low-level wrapper around libuv in the stdlib, as the event loop abstraction is pretty useful (beyond libuv's AIO). Anyways, I'm inquiring to find out the status/priority of this work and find out if there's a contribution I can make, here. I'd be interested in doing the work around adding a low-level libuv binding to the stdlib, if this is something the core stakeholders would be interested in. I assume this would also include adding libuv to the build. I'd be interested to here your thoughts and any existing plans about this. Thanks, Jeff From niko at alum.mit.edu Mon Feb 6 10:07:18 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Mon, 06 Feb 2012 10:07:18 -0800 Subject: [rust-dev] bind syntax change proposal In-Reply-To: <4F2DE1A7.1080808@alum.mit.edu> References: <4F2D55B1.2090109@alum.mit.edu> <4F2D7848.7060604@mozilla.com> <4F2DA827.30908@mozilla.com> <4F2DE1A7.1080808@alum.mit.edu> Message-ID: <4F3016D6.9000702@alum.mit.edu> On 2/4/12 5:55 PM, Niko Matsakis wrote: > Anyway, I am happy to update my patch to make `_` be currying (as both > you and graydon preferred). I think I spoke too soon about being happy. =) I've been thinking about the patch some more and I am not sure how I feel about `_` being currying. This is not because I'm opposed to a currying semantics as opposed to a closure semantics?though I'm not sure that it's really better?but because I don't yet see how to implement it well. I may go ahead and make the current `bind` parse without the keyword `bind`, since that seems universally popular, but that doesn't actually address the implementation problems I was trying to address (you can't bind methods, nor receivers, nor other expressions). The problem is that addressing those with currying semantics will be more invasive and I just don't want to spend the time on it right now. I was thinking I might prefer to just have the iter library essentially take a "manual currying" approach, where `map()` is not defined like so: fn map,B>(self: IA, map_fn: fn(A) -> B, consume_fn: fn(B)) { self.iter {|a| consume_fn(map_fn(a)) } } but rather like so: fn map,B>(self: IA, map_fn: fn(A) -> B) -> fn@(fn(B)) { fn@(map_fn: fn(B)) { self.iter {|a| consume_fn(map_fn(a)) } } } I didn't want to do this before because it's less efficient in some cases, and a nice bind syntax would obviate the need. But it'd be easier to implement and, in the common case, prettier to use (no underscores at all): something.do_filter {|t| ... }.do_map {|t| ... }.do {|t| ... } In any case, I am wondering a bit about the design of the library. I'd like to keep the "deforested by default" approach, but perhaps change the API a bit to allow hints to be passed down about the eventual size of the data structure. It'd be nice if the vector could be pre-allocated to the right size, basically. Anyway, I hope to toy around with this a bit today in parallel with CCI work. Niko From graydon at mozilla.com Mon Feb 6 10:53:25 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Mon, 06 Feb 2012 10:53:25 -0800 Subject: [rust-dev] net, libuv, etc In-Reply-To: References: Message-ID: <4F3021A5.4000606@mozilla.com> On 06/02/2012 9:18 AM, Jeffery Olson wrote: > I'd be interested to here your thoughts and any existing plans about this. Sure! - Rust has a 3-layer standard library: - librustrt, C++ code implementing language primitives and OS wrappers we can't figure out how to do in rust yet. - libcore, rust code imported-and-linked by default with all crates (unless you explicitly ask to omit it). - libstd, larger blobs of library code that are less certain to be used by "all reasonable programs". - The core/std rationale is explained elsewhere: https://mail.mozilla.org/pipermail/rust-dev/2011-December/001037.html - We provide, and will continue to provide, direct bindings to synchronous libc IO functions (fopen, fread, etc.) in libcore, but these are mostly for "simple cases". We're assuming more complex IO scenarios will want to switch to uv-based multiplexed IO-and-events. - Libuv-the-C-code is in the build already. We submodule it in from joyent from time to time. I updated it most recently on Friday. - We intend it to be a core dependency, actually just statically linked into our runtime library. Already is. - Libuv bindings exist in two forms in libstd currently, both incomplete: uv and uvtmp. They should be merged, cleaned up, made reliable and extended; uvtmp is the new one and should probably be renamed uv, with the old one removed. - Note that uv is currently in libstd. All of io is, and much of os. This is unintentional. We didn't manage to move them to core by the time we did 0.1. They should be in core as they are interfaces to services provided by librustrt (criterion #2). I'm in the process of moving os to libcore, hopefully this week. - Mostly we just lack sustained focus on uv. People keep getting into the work and then getting distracted with other things. This is not meant as criticism of the people who have worked on it or are presently working on it. It's just a fact: we have too much to do right now so everyone gets time-sliced a fair bit. - Currently brson is working closest to uv. In particular there's a small quantity of reorganizing the runtime scheduler and thread abstractions that has to be done before uv can be used safely and sanely from the rust libraries we have in mind. He's working on that presently, then may or may not return to hacking directly on the uv bindings. Others poke in from time to time from the servo team. - Read the changelogs on src/libstd/uv.rs and uvtmp.rs, as well as the C++ file src/rt/rust_uv.cpp to see who's been working on what. I'm certain you'd be welcome to lend a hand completing and extending the uv interface. Thanks for your interest, -Graydon From dherman at mozilla.com Mon Feb 6 11:36:37 2012 From: dherman at mozilla.com (David Herman) Date: Mon, 6 Feb 2012 11:36:37 -0800 Subject: [rust-dev] Suggestions In-Reply-To: <4F2DA082.3020309@mozilla.com> References: <4F2DA082.3020309@mozilla.com> Message-ID: <2F5A3EA1-4E95-4F17-A4AB-49655F3F1D98@mozilla.com> On Feb 4, 2012, at 1:17 PM, Patrick Walton wrote: > On 02/04/2012 06:21 AM, Arne D?ring wrote: >> The second suggestion is concerning tho #fmt macro. #fmt works like >> printf, but its string is parsed at compile time, so that errors might >> be thrown when the string is incorrect. So when you unwind this format >> string at compile time, you know also all variable names and their >> types from scope. So it might be more natural to directly import >> variable names from the scope inside of the String, like it is done in >> many scripting languages. This is also done in tho Programming >> language Nemerle. > > Well, #fmt is a macro, and I would generally be opposed to macros implicitly capturing the local variables in scope -- that's unhygienic. I don't quite understand Arne's description, but I don't see anything unhygienic about it either. Dave From graydon at mozilla.com Mon Feb 6 12:31:35 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Mon, 06 Feb 2012 12:31:35 -0800 Subject: [rust-dev] bind syntax change proposal In-Reply-To: <4F3016D6.9000702@alum.mit.edu> References: <4F2D55B1.2090109@alum.mit.edu> <4F2D7848.7060604@mozilla.com> <4F2DA827.30908@mozilla.com> <4F2DE1A7.1080808@alum.mit.edu> <4F3016D6.9000702@alum.mit.edu> Message-ID: <4F3038A7.9070803@mozilla.com> On 2/6/2012 10:07 AM, Niko Matsakis wrote: > I've been thinking about the patch some more and I am not sure how I > feel about `_` being currying. This is not because I'm opposed to a > currying semantics as opposed to a closure semantics?though I'm not sure > that it's really better?but because I don't yet see how to implement it > well. Ok. I hate to be a stick in the mud about this but part of what made 'bind' appeal to me initially was exactly that it encouraged the formation of closures that were less-coupled to their environment than those formed from function literals. The values got bound, never the variables. I guess I was over-eager about this decoupling, but I still really prefer to err on the side of decoupled behaviors by default. Capturing a mutable variable someone else might mutate at a distance is exactly the sort of footgun I prefer be non-default behavior in rust. The sort of thing you have to mean-to-say to have it happen. (I often express my preferences here in terms of balancing expressiveness of some desirable results against anti-expressiveness of undesirable results, like notational-ambiguity or module-coupling. IOW trying to balance the ease of expressing meaning-X in the language with defense against accidentally expressing anything _beyond_ meaning-X.) > keyword `bind`, since that seems universally popular, but that doesn't > actually address the implementation problems I was trying to address > (you can't bind methods, nor receivers, nor other expressions). The > problem is that addressing those with currying semantics will be more > invasive and I just don't want to spend the time on it right now. Ok. > but rather like so: > > fn map,B>(self: IA, map_fn: fn(A) -> B) -> fn@(fn(B)) { > fn@(map_fn: fn(B)) { > self.iter {|a| consume_fn(map_fn(a)) } > } > } Yeah, that might well work better. Iter is a real pressure-cooker for these features. It's exciting to see you bouncing around in this design space, and I'd expect several passes of (re-)design. Keep notes on what goes wrong each time, if you can! At very least keep posting updates on what-works / what-doesn't to the list so we can follow along and refer-back in the future :) (We do not universally do this, much to our own loss.) -Graydon From banderson at mozilla.com Mon Feb 6 12:36:27 2012 From: banderson at mozilla.com (Brian Anderson) Date: Mon, 6 Feb 2012 12:36:27 -0800 (PST) Subject: [rust-dev] net, libuv, etc In-Reply-To: Message-ID: <80628441.3452319.1328560587436.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> ----- Original Message ----- > From: "Jeffery Olson" > To: rust-dev at mozilla.org > Sent: Monday, February 6, 2012 9:18:07 AM > Subject: [rust-dev] net, libuv, etc > > First of all, hello list. I'm pretty excited about rust and hope to > have an opportunity to make contributions and learn. There is definitely an opportunity to make a big impact implementing the uv bindings. > I'm curious about the current state of the 'net' module and libuv. > Looking at the docs, there's just some stubs around IPv4 stuff. I've > heard around that there's interest in building the module atop libuv. > Is libuv going to be a core dependency? Is there an existing wrapper? > > I hope you will ship a low-level wrapper around libuv in the stdlib, > as the event loop abstraction is pretty useful (beyond libuv's AIO). That is what I would like as well: a low-level uv API that mirrors as close as possible the C API, then a high-level API for async I/O. uv does two things that Rust can't cope with yet: blocking in native code, and calling C callbacks. Accomplishing either of these things from Rust currently requires writing a lot of C glue code. To that end we have a module in std called uvtmp that calls a bunch of wrapper functions in the runtime and presents an interface that is quite dissimilar to uv's API. The corresponding runtime code lives in src/rt/rust_uvtmp.cpp. This was hacked together quickly for the use of another project (called servo). To allow blocking code we are going to add first class schedulers to the language, so if you want to start a uv event loop you can start up your own scheduler and take an entire thread to yourself. This is what I'm working on now and it should be usable this week. The bug ticket is here: https://github.com/mozilla/rust/issues/1721 The ticket for allowing callbacks from C into Rust is here: https://github.com/mozilla/rust/issues/1732 The bug ticket for the uv bindings is here: https://github.com/mozilla/rust/issues/10 As to how you can help us get to the end goal, probably there are three good options. First, you could implement callbacks from C to Rust. This is a pretty hairy change to both the compiler and the runtime, and we don't have an agreed-upon design, but I can help you with this if you want to pursue it. The second thing you could do is review what's going in std::uvtmp and coax it into an API that properly mirrors the uv API, and redesign the runtime code and interface with a mind toward eliminating them when the aforementioned Rust features materialize. I outlined in issue #10 a strategy for implementing the uv bindings that I think does not require C-to-Rust callbacks but still mirrors the uv API. Come talk about it on IRC. Regards, Brian From banderson at mozilla.com Mon Feb 6 12:36:30 2012 From: banderson at mozilla.com (Brian Anderson) Date: Mon, 6 Feb 2012 12:36:30 -0800 (PST) Subject: [rust-dev] net, libuv, etc In-Reply-To: <4F3021A5.4000606@mozilla.com> Message-ID: <1413273738.3452488.1328560590705.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> ----- Original Message ----- > From: "Graydon Hoare" > To: "Jeffery Olson" > Cc: rust-dev at mozilla.org > Sent: Monday, February 6, 2012 10:53:25 AM > Subject: Re: [rust-dev] net, libuv, etc > > On 06/02/2012 9:18 AM, Jeffery Olson wrote: > > > I'd be interested to here your thoughts and any existing plans > > about this. > > > - Libuv bindings exist in two forms in libstd currently, both > incomplete: uv and uvtmp. They should be merged, cleaned up, > made reliable and extended; uvtmp is the new one and should > probably be renamed uv, with the old one removed. While uvtmp is the module capable of doing 'real work', my intention was the opposite. uvtmp does almost all of its work in C++ (and is quite hastily designed), and I really want to avoid that if we can. Of course, the features necessary to write uv bindings the way I want to don't exist yet, so I understand if we end up going that route. From banderson at mozilla.com Mon Feb 6 12:46:38 2012 From: banderson at mozilla.com (Brian Anderson) Date: Mon, 6 Feb 2012 12:46:38 -0800 (PST) Subject: [rust-dev] error: can't find crate for 'core' In-Reply-To: Message-ID: <476469815.3463680.1328561198691.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> ----- Original Message ----- > From: "Andr? Pang" > To: "Brian Anderson" > Cc: rust-dev at mozilla.org > Sent: Sunday, February 5, 2012 7:24:23 PM > Subject: Re: [rust-dev] error: can't find crate for 'core' > > On Feb 5, 2012, at 4:04 PM, Brian Anderson wrote: > > >> Hi! I'm trying out rust for the first time, and successfully > >> compiled and installed rustc. I hit this error attempting to > >> compile "hello world": > >> > >>> ~ % cat hello.rs > >>> use std; > >>> fn main(args: [str]) { > >>> std::io::println("hello world from '" + args[0] + "'!"); > >>> } > >>> ~ % rustc hello.rs > >>> hello.rs:1:0: 1:0 error: can't find crate for 'core' > >>> hello.rs:1 use std; > >>> ^ > > > > If you run again with "RUST_LOG=rustc::metadata::creader" you > > should get a bunch of information about where rustc is looking for > > the core crate. For example, when I compile something I see the > > following: > > > > ... > > rust: "inspecting file > > /home/banderson/Dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustc/x86_64-unknown-linux-gnu/lib/librustc.so" > > rust: "skipping > > /home/banderson/Dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustc/x86_64-unknown-linux-gnu/lib/librustc.so, > > doesn't look like libcore-*. > > so" > > rust: "inspecting file > > /home/banderson/Dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustc/x86_64-unknown-linux-gnu/lib/libcore-14bd852465126fe7-0.1.so" > > rust: > > "/home/banderson/Dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustc/x86_64-unknown-linux-gnu/lib/libcore-14bd852465126fe7-0.1.so > > is a candidate" > > rust: "matching 0 metadata requirements against 4 items" > > rust: "crate metadata:" > > rust: " name = "core"" > > rust: " vers = "0.1"" > > rust: " uuid = "c70c24a7-5551-4f73-8e37-380b11d80be8"" > > rust: " url = "http://rust-lang.org/src/core"" > > rust: "found > > /home/banderson/Dev/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustc/x86_64-unknown-linux-gnu/lib/libcore-14bd852465126fe7-0.1.so > > with matching metadata" > > Hmm, I get something rather odd if I try this: > > ~ % RUST_LOG=rustc::metadata::creader rustc ~/hello.rs > /Users/andre/hello.rs:1:0: 1:0 error: can't find crate for 'core' > /Users/andre/hello.rs:1 use std; > ^ > rust: upcall fail 'explicit failure', > src/comp/driver/diagnostic.rs:52 > rust: upcall fail 'explicit failure', src/comp/driver/rustc.rs:184 > rust: domain main @0x10182be00 root task failed > > I tried it with the compiler built at x86_64-apple-darwin/stage3/bin/ > too; same result. OK. That's unexpected. I'm not sure what the next thing you should do is. If you write '#[no_core];' at the top of the file it will not try to link to libcore. At least then you could see if it manages to find libstd successfully. From graydon at mozilla.com Mon Feb 6 12:48:46 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Mon, 06 Feb 2012 12:48:46 -0800 Subject: [rust-dev] net, libuv, etc In-Reply-To: <1413273738.3452488.1328560590705.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> References: <1413273738.3452488.1328560590705.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Message-ID: <4F303CAE.1040505@mozilla.com> On 2/6/2012 12:36 PM, Brian Anderson wrote: > While uvtmp is the module capable of doing 'real work', my intention was the opposite. uvtmp does almost all of its work in C++ (and is quite hastily designed), and I really want to avoid that if we can. Of course, the features necessary to write uv bindings the way I want to don't exist yet, so I understand if we end up going that route. Whoops! My mistake. -Graydon From graydon at mozilla.com Tue Feb 7 14:52:38 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Tue, 07 Feb 2012 14:52:38 -0800 Subject: [rust-dev] Cargo requirements In-Reply-To: References: <4F2067CB.6010309@mozilla.com> <4F27345E.1040101@mozilla.com> Message-ID: <4F31AB36.9020705@mozilla.com> On 04/02/2012 4:13 AM, Thomas Leonard wrote: > But it doesn't remove the need to handle diamond dependencies. Very > often (most of the time, I think), if two libraries depend on a common > one then it's because they need to communicate using a common API or > common set of data-types. This is true to an extent. Within Rust (when the versioning mechanism is finished, tested, integrated into cargo) we'll handle disagreeing-diamonds ok. For libraries outside Rust, there are mechanisms people have worked out for each OS to try to manage such disagreeing-diamonds too (sonames, which don't work because they don't sort and don't get bumped often enough; symbol versioning, which might work except nobody uses it; SxS, which works more like rust, but gets enormous). What I'm saying, repeating here since it didn't seem to stick with the last message, is is that there is a threshold at which any such version-everything solution *must give up*. You cannot version the entire universe. Not even the software-and-hardware universe. Both because it's too big and because (as I get to down below) each level is itself non-computable. > For example, if I write an application using GTK to handle windows, > and I import a widget library, then it's no good having the widget > library using its own version of GTK, because I won't be able to embed > its widgets in my windows. I know, it's awful, and you can't always solve this. Even if you put the GTKs *right next to each other* in the target environment you can't even solve this, because they have global PLT-resolved symbols they fail to version and the loader may pick some from one library and some from another as it lazily resolves stuff. > Sure, but now the user needs to manage two separate logging > configurations for the same application, or risk them overwriting each > other's log files (although this is probably a bad example, as I see > that Rust very sensibly has built-in logging support). > > By the way, does this mean that a Rust program must link against > exactly the same version of the library that was used at build time? > e.g. if I upgrade libssl then I must rebuild all applications using > it? No, we'll probably support some level of version-range or prefix-matching. Despite the fact that this *weakens* the likelihood of proper functioning. I accept, however, that precise-versioning and loose-coupled-upgrades are in inherent engineering-tradeoff with one another. So I'm ok picking a point on the tradeoff spectrum based on my sense of taste and annoyance (in consultation with others). I don't believe it's solvable. ... Really, as I said above, I feel like this whole conversation is stuck on your assumption that the problem is actually solvable (or even precisely denotable). And in a very thorough and concrete way, I don't think it is. For perhaps 4 broad reasons (along with a zillion technical details): 1. There's tension between precise-coupling (for correctness) and loose-coupling (for ease of use). This is why versioning systems often support imprecise wildcard matching, symbolic names, etc. And as soon as you have symbolic names you have a problem of naming authority, which -- if you take seriously -- you wind up having to invent something like PKI in order to solve. Punt to DNS or GPG whenever possible, maybe, but Naming Is Hard and it's usually a source of endless assumption-mismatch, precisely because names occupy a weird, ill-defined neither-zone between structured data and opaque nonces. 2. A -> B dependency when A and B are turing-complete artifacts is generally non-computable. I.e. you can't even *define* when you have "captured" a dependency accurately. You can capture a declared dependency but it doesn't actually guarantee there's no undeclared dependency. This is why many people are taking to using bit-identity of the delivered artifacts (git content addressing, "ship the DLLs side-by-side with the .exe", or just shipping whole system images / VM images / etc. rather than symbolically-described "packages") 3. Enumerating the "dependencies of A" depends on what you want to *do* with A. 0install talks about executing, but we need to compile, bootstrap-compile, recompile-LLVM-and-self-host-with-it, build docs, build debugging output, run tests ... you wind up reinventing build systems in general in order to handle all the things a developer wants to capture the "dependencies for". 4. The *entire* environment is a dependency. It's often not just non-computable but physically off limits to versioning. I.e. you can version a kernel but not the firmware of the machine it's running, on because the firmware isn't under software control, can't be analyzed/sandboxed/controlled. This goes double for the network environment in which something runs or, heaven forbid, the physical environment in which the system winds up instantiated. The problem is just ... bottomless. I'm sorry. I know you've spent a lot of energy on versioning and built an ambitious system. I'm sympathetic. I did so too, a long time back. But I'm relatively convinced at this point that there's no Correct Solution, just a bunch of wrong ones that have different forms of associated pain and cognitive load, and at some point you have to draw a line and get on with whatever else you were Actually Doing, rather than further work on versioning. We're doing a programming language. I'm ok developing cargo and rustc based on perceived and encountered needs, rather than striving for perfection. -Graydon From niko at alum.mit.edu Tue Feb 7 15:24:34 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Tue, 07 Feb 2012 15:24:34 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory Message-ID: <4F31B2B2.9070009@alum.mit.edu> As part of the work I'm doing on preparing a region proposal, I wanted to spin off a part of it which proposed to address Dan's bug [1] without the need for alias analysis. The idea is fairly simple. I've written it up on the Wiki: https://github.com/mozilla/rust/wiki/Proposal-to-address-Dan%27s-bug I would appreciate feedback. I am pretty sure that the idea is sound, but the problem is subtle and I may be missing something, so that kind of feedback would be particularly helpful! Also, I would like to know what else alias analysis is used for besides Dan's bug (hopefully those remaining uses can be addressed via regions). thanks, Niko [1] if you don't know what Dan's bug is, please read the wiki page. From graydon at mozilla.com Tue Feb 7 16:23:29 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Tue, 07 Feb 2012 16:23:29 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <4F31B2B2.9070009@alum.mit.edu> References: <4F31B2B2.9070009@alum.mit.edu> Message-ID: <4F31C081.8000009@mozilla.com> On 2/7/2012 3:24 PM, Niko Matsakis wrote: > I would appreciate feedback. I am pretty sure that the idea is sound, > but the problem is subtle and I may be missing something, so that kind > of feedback would be particularly helpful! Also, I would like to know > what else alias analysis is used for besides Dan's bug (hopefully those > remaining uses can be addressed via regions). Hm. I am confused at the description of the "hole" in the type system. I was under the impression that this was the distinction between immutable values and immutably-rooted values (those contained within a path-of-immutable-references). Am I misunderstanding? -Graydon From pwalton at mozilla.com Tue Feb 7 16:27:39 2012 From: pwalton at mozilla.com (Patrick Walton) Date: Tue, 07 Feb 2012 16:27:39 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <4F31C081.8000009@mozilla.com> References: <4F31B2B2.9070009@alum.mit.edu> <4F31C081.8000009@mozilla.com> Message-ID: <4F31C17B.1090500@mozilla.com> On 2/7/12 4:23 PM, Graydon Hoare wrote: > Hm. I am confused at the description of the "hole" in the type system. I > was under the impression that this was the distinction between immutable > values and immutably-rooted values (those contained within a > path-of-immutable-references). Am I misunderstanding? It's not a hole per se, but our concept of immutability is pretty unique among languages I know of. (C++ doesn't work this way, in particular.) Moreover, our notion of immutability makes it more difficult than it could be to prevent Dan's bug. Patrick From graydon at mozilla.com Tue Feb 7 16:31:54 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Tue, 07 Feb 2012 16:31:54 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <4F31C17B.1090500@mozilla.com> References: <4F31B2B2.9070009@alum.mit.edu> <4F31C081.8000009@mozilla.com> <4F31C17B.1090500@mozilla.com> Message-ID: <4F31C27A.1030005@mozilla.com> On 2/7/2012 4:27 PM, Patrick Walton wrote: > On 2/7/12 4:23 PM, Graydon Hoare wrote: >> Hm. I am confused at the description of the "hole" in the type system. I >> was under the impression that this was the distinction between immutable >> values and immutably-rooted values (those contained within a >> path-of-immutable-references). Am I misunderstanding? > > It's not a hole per se, but our concept of immutability is pretty unique > among languages I know of. (C++ doesn't work this way, in particular.) > Moreover, our notion of immutability makes it more difficult than it > could be to prevent Dan's bug. Really? I think our concept quite closely mirrors most languages that mix mutable and immutable types. Either can nest in either, and saying something is mutable or immutable is only a statement about one level of nesting. It *used* to do deep immutability analysis when assigning kinds. But it doesn't do that anymore. Now it only does deep immutability analysis when looking at reference safety. Which, I think, it kinda has to. Or are you saying that it's rare to do *that* deep analysis? I agree with that. I also think it's one of our more pleasant features. Other systems must either defer to "the GC keeps everything alive" or "any reference can become invalid at any moment" (C++). -Graydon From niko at alum.mit.edu Tue Feb 7 18:51:48 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Tue, 07 Feb 2012 18:51:48 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <4F31C27A.1030005@mozilla.com> References: <4F31B2B2.9070009@alum.mit.edu> <4F31C081.8000009@mozilla.com> <4F31C17B.1090500@mozilla.com> <4F31C27A.1030005@mozilla.com> Message-ID: <4F31E344.9090300@alum.mit.edu> On 2/7/12 4:31 PM, Graydon Hoare wrote: > Really? I think our concept quite closely mirrors most languages that > mix mutable and immutable types. Either can nest in either, and saying > something is mutable or immutable is only a statement about one level > of nesting. I agree that we mirror existing languages, but I disagree that this makes it safe. Most languages with which I'm familiar, like O'Caml, don't have this issue because they do not have interior record types. Languages which do, namely C++, have special rules. For example, in C++ you cannot have a const field of value type unless all fields within that type are const. Compiling this program in gcc, for example: > struct foo { const int x; }; > struct bar { foo f; }; > > void test(bar *b, foo f) { > b->f = f; > } yields the following error: "non-static const member ?const int foo::x?, can't use default assignment operator". Niko From niko at alum.mit.edu Tue Feb 7 19:00:08 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Tue, 07 Feb 2012 19:00:08 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <4F31C081.8000009@mozilla.com> References: <4F31B2B2.9070009@alum.mit.edu> <4F31C081.8000009@mozilla.com> Message-ID: <4F31E538.2030908@alum.mit.edu> On 2/7/12 4:23 PM, Graydon Hoare wrote: > On 2/7/2012 3:24 PM, Niko Matsakis wrote: > > Hm. I am confused at the description of the "hole" in the type system. > I was under the impression that this was the distinction between > immutable values and immutably-rooted values (those contained within a > path-of-immutable-references). Am I misunderstanding? I am talking about our type system as implemented. We treat immutable fields as covariantly typed, meaning that {x:T1} <: {x:T2} if T1 <: T2. This is sound so long as the field x is really immutable; but it is not. Here is an example of where this leads to bad results: > type T = { f: @const int }; > > fn foo(&t: T, v: @const int) { > t = {f:v}; > } > > fn main() { > let h = @3; // note: h is immutable > let g = @mutable {f: @mutable 4}; > #error["h=%? g=%?", h, g]; // prints "h=@3 g=@(@4)" > foo(*g, h); // after this point, g.f == h but with a new type > #error["h=%? g=%?", h, g]; // prints "h=@3 g=@(@3)" > *g.f = 5; > #error["h=%? g=%?", h, g]; // prints "h=@5 g=@(@5)" > } You can construct similar examples using vectors or fn types. If we were to make more use of subtyping, for example via refinement types such as those proposed in #1679, there would be many more such examples as well. Niko From niko at alum.mit.edu Tue Feb 7 19:02:54 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Tue, 07 Feb 2012 19:02:54 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <4F31E344.9090300@alum.mit.edu> References: <4F31B2B2.9070009@alum.mit.edu> <4F31C081.8000009@mozilla.com> <4F31C17B.1090500@mozilla.com> <4F31C27A.1030005@mozilla.com> <4F31E344.9090300@alum.mit.edu> Message-ID: <4F31E5DE.8090905@alum.mit.edu> On 2/7/12 6:51 PM, Niko Matsakis wrote: > For example, in C++ you cannot have a const field of value type unless > all fields within that type are const. Sorry, this is not what I meant to write. The rule, in fact, is that the values with const value fields are not assignable (which is basically what I proposed, except that in C++ things are mutable by default and they are not in Rust). Hat tip to pcwalton for pointing out this rule in C++ a while back.. Niko From niko at alum.mit.edu Tue Feb 7 19:06:07 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Tue, 07 Feb 2012 19:06:07 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <4F31C17B.1090500@mozilla.com> References: <4F31B2B2.9070009@alum.mit.edu> <4F31C081.8000009@mozilla.com> <4F31C17B.1090500@mozilla.com> Message-ID: <4F31E69F.6050308@alum.mit.edu> On 2/7/12 4:27 PM, Patrick Walton wrote: > It's not a hole per se As I pointed out elsewhere, I think this is a genuine hole. It allows you to create an immutable box that can later be mutated, as I showed. Niko From pwalton at mozilla.com Tue Feb 7 19:16:47 2012 From: pwalton at mozilla.com (Patrick Walton) Date: Tue, 07 Feb 2012 19:16:47 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <4F31E69F.6050308@alum.mit.edu> References: <4F31B2B2.9070009@alum.mit.edu> <4F31C081.8000009@mozilla.com> <4F31C17B.1090500@mozilla.com> <4F31E69F.6050308@alum.mit.edu> Message-ID: <488b6407-31ed-4fdc-aacf-66293953b457@email.android.com> Ah yes, I remember now. Can this also be used to break refinement types (i.e. break typestate)? We have subtyping via refinements. Patrick -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. Niko Matsakis wrote: On 2/7/12 4:27 PM, Patrick Walton wrote: > It's not a hole per se As I pointed out elsewhere, I think this is a genuine hole. It allows you to create an immutable box that can later be mutated, as I showed. Niko -------------- next part -------------- An HTML attachment was scrubbed... URL: From graydon at mozilla.com Tue Feb 7 19:26:03 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Tue, 07 Feb 2012 19:26:03 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <4F31E538.2030908@alum.mit.edu> References: <4F31B2B2.9070009@alum.mit.edu> <4F31C081.8000009@mozilla.com> <4F31E538.2030908@alum.mit.edu> Message-ID: <4F31EB4B.10108@mozilla.com> On 2/7/2012 7:00 PM, Niko Matsakis wrote: > I am talking about our type system as implemented. We treat immutable > fields as covariantly typed, meaning that {x:T1} <: {x:T2} if T1 <: T2. > This is sound so long as the field x is really immutable; but it is not. > Here is an example of where this leads to bad results: Goodness. You're quite right, this should have never occurred. Ok. *shakes fist at subtyping*[1] -Graydon [1] which I never wanted[3], exactly because most structural variance rules that look attractive on paper produce these kinds of nightmares when you actually try to implement them in terms of structures in memory. Likewise record extension, sum-restriction, field permutation, etc. etc. [2] yes, I know we needed[3] something to deal with const / mutable? [3] life's unfair[4] sometimes. [4] No pony. From marijnh at gmail.com Wed Feb 8 00:05:12 2012 From: marijnh at gmail.com (Marijn Haverbeke) Date: Wed, 8 Feb 2012 09:05:12 +0100 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <4F31B2B2.9070009@alum.mit.edu> References: <4F31B2B2.9070009@alum.mit.edu> Message-ID: Help me understand -- does this mean 'let x = {a: 10}; x = {a: 20};' would be illegal (assigning to non-assignable type)? Also, you only mention pattern matching, but function calls also create references to the arguments. Would it be forbidden to do f(foo.x) where x is a mutable field? From niko at alum.mit.edu Wed Feb 8 06:36:28 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Wed, 08 Feb 2012 06:36:28 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: References: <4F31B2B2.9070009@alum.mit.edu> Message-ID: <4F32886C.9070600@alum.mit.edu> On 2/8/12 12:05 AM, Marijn Haverbeke wrote: > Help me understand -- does this mean 'let x = {a: 10}; x = {a: 20};' > would be illegal (assigning to non-assignable type)? Yes. What would be legal would be: let x = @{a: 10}; x = @{a: 20}; or something similar. But `x = {a: 20}` effectively overwrites x.a, which was declared as immutable. > Also, you only mention pattern matching, but function calls also > create references to the arguments. Would it be forbidden to do > f(foo.x) where x is a mutable field? No, it is legal to do `f(foo.x)` but, depending on the type of `foo.x`, `f()` might be restricted in what it can do with the reference. For example: fn f(&x: {a: int}) { x = {a: 20}; // illegal, x is not an assignable type } As far as I can see, there isn't much point to have mutable slots (e.g., by-mut-ref, vectors, @, fields, local variables) of non-assignable type, so perhaps that should simply be illegal. In which case, the declaration of `f()` above would not be illegal, you would have to use an immutable reference `&&` because `{a: int}` is not an assignable type. This would probably be my preference. Niko From marijnh at gmail.com Wed Feb 8 06:54:12 2012 From: marijnh at gmail.com (Marijn Haverbeke) Date: Wed, 8 Feb 2012 15:54:12 +0100 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <4F32886C.9070600@alum.mit.edu> References: <4F31B2B2.9070009@alum.mit.edu> <4F32886C.9070600@alum.mit.edu> Message-ID: > Yes. What would be legal would be: > > ? ?let x = @{a: 10}; > ? ?x = @{a: 20}; That seems a rather heavy-handed restriction. > No, it is legal to do `f(foo.x)` but, depending on the type of `foo.x`, > `f()` might be restricted in what it can do with the reference. ?For > example: How about this: let x = @{mutable y: 10}; let f = fn@(u: int) { x.y = 20; log(error, u); } f(x.y); // Where f's arg is passed by reference, whichever notation you'd use for that From niko at alum.mit.edu Wed Feb 8 07:30:51 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Wed, 08 Feb 2012 07:30:51 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: References: <4F31B2B2.9070009@alum.mit.edu> <4F32886C.9070600@alum.mit.edu> Message-ID: <4F32952B.3070300@alum.mit.edu> On 2/8/12 6:54 AM, Marijn Haverbeke wrote: > That seems a rather heavy-handed restriction. Perhaps? All I am saying is that assigning to value types that contain immutable fields is illegal, because it overwrites those immutable fields. You can declare the record like: let x = {mutable a: 10}; x = {mutable a: 20}; if you wish to allow mutability. The fact that this notation is painful is more a by-product of using structural record types than anything else. An alternative is that declaring a field as immutable does not in fact mean that it will not be overwritten. We could make that sound (no covariance, for example), but it means we (and the language's users) have to ultimately treat all fields as potentially mutable. So I'm not sure what the point is. Maybe there are other alternatives? > How about this: > let x = @{mutable y: 10}; > let f = fn@(u: int) { x.y = 20; log(error, u); } > f(x.y); // Where f's arg is passed by reference, whichever > notation you'd use for that Yes, this is precisely the kind of problem I was looking for! Well, this example itself I think is ok: it prints "20" (the wonders of aliasing). But an example like this: let x = @{mutable f: some(10)}; let f = fn@(o: option) { alt o { some(f) { x.f = none; log(f); } } f(x.f); is certainly bad. We basically have to treat && mode as potentially mutable for the purposes of matching in alt statements. So the mutability algorithm from mut.rs needs to be somewhat adjusted to yield a 3-way answer: immutable, const, or mutable (actually I can't recall if it does this already or not). One annoyance is that we don't have a way to express a reference to some arbitrary immutable field. Moving to something like regions would allow a type modifier `&` that supported `const` and `mut` just like `@`, which would make it possible to express that (above, `o` would have type `&const option`: `&option` would be a type error, as that would mean a ptr to immutable memory, and `x.f` is not immutable). Niko From niko at alum.mit.edu Wed Feb 8 08:03:35 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Wed, 08 Feb 2012 08:03:35 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory Message-ID: <7nnlkuj13k0qcn6bq89a7fsx.1328717015431@email.android.com> I'm on the train so I will be brief. I wanted to clarify a few things 1. the type hole appears in the language as it is today, not some hypothetical future without alias analysis or with regions. It occurred to me that this might not be obvious. 2. Making fields invariant but still allowing assignments that overwrite immutable fields would indeed give weaker guarantees but not quite as weak as I implied. ?For example a type @T would be immutable as declared. It is only references where the declared mutability of fields would have a murky meaning (I think, anyway) 3. Regarding structural types, I shouldn't have dragged them into this. :) There are many ways to address the verbosity of mutable fields, of which nominal record types are but one. Hope that makes sense. I can clarify more when I am not typing on my phone. :) Niko -------- Original message -------- Subject: Re: [rust-dev] RFC: Addressing Dan's bug through immutable memory From: Niko Matsakis To: Marijn Haverbeke CC: "rust-dev at mozilla.org" On 2/8/12 6:54 AM, Marijn Haverbeke wrote: > That seems a rather heavy-handed restriction. Perhaps?? All I am saying is that assigning to value types that contain immutable fields is illegal, because it overwrites those immutable fields.? You can declare the record like: ???? let x = {mutable a: 10}; ???? x = {mutable a: 20}; if you wish to allow mutability.? The fact that this notation is painful is more a by-product of using structural record types than anything else. An alternative is that declaring a field as immutable does not in fact mean that it will not be overwritten.? We could make that sound (no covariance, for example), but it means we (and the language's users) have to ultimately treat all fields as potentially mutable.? So I'm not sure what the point is. Maybe there are other alternatives? > How about this: >????? let x = @{mutable y: 10}; >????? let f = fn@(u: int) { x.y = 20; log(error, u); } >????? f(x.y); // Where f's arg is passed by reference, whichever > notation you'd use for that Yes, this is precisely the kind of problem I was looking for!? Well, this example itself I think is ok: it prints "20" (the wonders of aliasing).? But an example like this: ???? let x = @{mutable f: some(10)}; ???? let f = fn@(o: option) { ???????? alt o { some(f) { x.f = none; log(f); } ???? } ???? f(x.f); is certainly bad.? We basically have to treat && mode as potentially mutable for the purposes of matching in alt statements.? So the mutability algorithm from mut.rs needs to be somewhat adjusted to yield a 3-way answer: immutable, const, or mutable (actually I can't recall if it does this already or not). One annoyance is that we don't have a way to express a reference to some arbitrary immutable field.? Moving to something like regions would allow a type modifier `&` that supported `const` and `mut` just like `@`, which would make it possible to express that (above, `o` would have type `&const option`: `&option` would be a type error, as that would mean a ptr to immutable memory, and `x.f` is not immutable). Niko -------------- next part -------------- An HTML attachment was scrubbed... URL: From pwalton at mozilla.com Wed Feb 8 08:39:55 2012 From: pwalton at mozilla.com (Patrick Walton) Date: Wed, 08 Feb 2012 08:39:55 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: References: <4F31B2B2.9070009@alum.mit.edu> <4F32886C.9070600@alum.mit.edu> Message-ID: <4F32A55B.6030901@mozilla.com> On 02/08/2012 06:54 AM, Marijn Haverbeke wrote: >> Yes. What would be legal would be: >> >> let x = @{a: 10}; >> x = @{a: 20}; > > That seems a rather heavy-handed restriction. C++ has the same restriction. In practice I've found that it rarely comes up. Patrick From marijnh at gmail.com Wed Feb 8 08:48:56 2012 From: marijnh at gmail.com (Marijn Haverbeke) Date: Wed, 8 Feb 2012 17:48:56 +0100 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <4F32A55B.6030901@mozilla.com> References: <4F31B2B2.9070009@alum.mit.edu> <4F32886C.9070600@alum.mit.edu> <4F32A55B.6030901@mozilla.com> Message-ID: > C++ has the same restriction. In practice I've found that it rarely comes > up. Obviously, since const field are rare in C. In Rust, one rarely uses a record type that doesn't have immutable fields. From pwalton at mozilla.com Wed Feb 8 08:58:30 2012 From: pwalton at mozilla.com (Patrick Walton) Date: Wed, 08 Feb 2012 08:58:30 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: References: <4F31B2B2.9070009@alum.mit.edu> <4F32886C.9070600@alum.mit.edu> <4F32A55B.6030901@mozilla.com> Message-ID: <4F32A9B6.4040303@mozilla.com> On 02/08/2012 08:48 AM, Marijn Haverbeke wrote: >> C++ has the same restriction. In practice I've found that it rarely comes >> up. > > Obviously, since const field are rare in C. In Rust, one rarely uses a > record type that doesn't have immutable fields. Fair enough. Still: (1) This is a type unsoundness. I think we have to fix it, because as it stands typestate means nothing -- it can be broken in the safe language. (2) The type unsoundness notwithstanding, I think it's more logical for immutable to mean truly immutable. Right now it's possible to mutate all immutable fields. Patrick From niko at alum.mit.edu Wed Feb 8 09:18:54 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Wed, 08 Feb 2012 09:18:54 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: References: <4F31B2B2.9070009@alum.mit.edu> <4F32886C.9070600@alum.mit.edu> <4F32A55B.6030901@mozilla.com> Message-ID: <4F32AE7E.6090306@alum.mit.edu> On 2/8/12 8:48 AM, Marijn Haverbeke wrote: >> C++ has the same restriction. In practice I've found that it rarely comes >> up. > Obviously, since const field are rare in C. In Rust, one rarely uses a > record type that doesn't have immutable fields. The question, I guess, is how often we reassign an entire record at a time. But, as pcwalton said, this is a type hole, so I don't think it's a question of whether we fix it, it's a question of how. To my mind, there are two viable options: 1. Introduce a distinction between assignable and non-assignable types, as I proposed. This may mean that sometimes you have to use a boxed type where you could otherwise use an interior type, because the type is defined with immutable fields and you wish to mutate them (this is, after all, what you would be doing in Java, O'Caml, or most other languages). I don't view box allocation as a penalty: it's probably going to be the default much of the time. To my mind, interior types are the "bonus", great when they work but not always an option. That's certainly how I've found it to be in C and C++ and I would expect no different in Rust. 2. Remove the concept of mutable and immutable fields and replace it with something else. I can think of various options but I chose option #1 because it's going to be a smaller delta on what we have. Also, the same concept (assignable types) is required if we move to types of dynamic size (e.g., fixed-length arrays or records containing inline arrays of unknown size). Still, maybe we want to discuss other ways of controlling mutability besides the current approach... I certainly don't feel wedded to what I proposed. I just want to be able to statically tell if a given memory location is mutable or not. In principle, as I wrote before, we could patch up the big hole by being more restrictive about variance. There may be other holes. But frankly I feel pretty strongly that if a field is declared immutable, it ought to mean that no matter how many times you read that memory location, you get the same result. This property would not be preserved in such a system and I think that's undesirable. It's also true that getting rid of field variance is itself rather ham-fisted and will lead to numerous unnecessary type errors, though perhaps improved inference might help. Niko From graydon at mozilla.com Wed Feb 8 09:40:54 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Wed, 08 Feb 2012 09:40:54 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <4F32AE7E.6090306@alum.mit.edu> References: <4F31B2B2.9070009@alum.mit.edu> <4F32886C.9070600@alum.mit.edu> <4F32A55B.6030901@mozilla.com> <4F32AE7E.6090306@alum.mit.edu> Message-ID: <4F32B3A6.20308@mozilla.com> On 08/02/2012 9:18 AM, Niko Matsakis wrote: > In principle, as I wrote before, we could patch up the big hole by being > more restrictive about variance. There may be other holes. But frankly I > feel pretty strongly that if a field is declared immutable, it ought to > mean that no matter how many times you read that memory location, you > get the same result. This property would not be preserved in such a > system and I think that's undesirable. It's also true that getting rid > of field variance is itself rather ham-fisted and will lead to numerous > unnecessary type errors, though perhaps improved inference might help. A communication-note: I understand you've developed strong thoughts about this, as has pcwalton, and suspect it results from a lot of high-bandwidth, in-person conversations in mountain view. I respectfully ask you to move *slowly* and at a somewhat laborious, tedious-to-you pace on the topic, for the "remoties" (marijn and myself). We've not been party to *any* of the in-person whiteboarding or conversing that leads to this conclusion, so the "to my mind ... only two options" statements come off as very immediate, certain and restrictive. IOW, there's a lot to think and talk about in this topic, and what seem like cut-and-dried conclusions to you are "the first time we've heard mention of it" to us. (To me, for example, the notion that "var x = ..." would somehow lead to x being "unassignable" seems curious and equally wrong. Because the whole notion of a mutable slot (such as 'x') implies the ablility to change its entire contents; that implicitly means "all of its substructure"; otherwise you'd have to 'mutable'-annotate every single bit in a substructure. I'd be curious to see what actually happens if we lose mutability covariance on interior-structural types, or even explore why you think those are the only two options. Fwiw the only place I think I *use* mutability-subtyping is on vector types, which are exterior. It feels premature to try to couple this decision to the interior-vector problem, which is even more involved since the variance of fixed vs. dynamic size types also comes into play ... all this says that there's a lot of conversation we've *not had* yet.) -Graydon From graydon at mozilla.com Wed Feb 8 10:35:52 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Wed, 08 Feb 2012 10:35:52 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <7nnlkuj13k0qcn6bq89a7fsx.1328717015431@email.android.com> References: <7nnlkuj13k0qcn6bq89a7fsx.1328717015431@email.android.com> Message-ID: <4F32C088.4070607@mozilla.com> On 2/8/2012 8:03 AM, Niko Matsakis wrote: > 1. the type hole appears in the language as it is today, not some > hypothetical future without alias analysis or with regions. It occurred > to me that this might not be obvious. Understood. Thankfully we do not have enterprise customers breathing down our necks for a hot patch just now, so we don't have to have a fix for it in by the end of the day. We can take our time to talk through the problem thoroughly. > 2. Making fields invariant but still allowing assignments that overwrite > immutable fields would indeed give weaker guarantees but not quite as > weak as I implied. For example a type @T would be immutable as declared. > It is only references where the declared mutability of fields would have > a murky meaning (I think, anyway) I agree these issues are murky! I don't mean to imply I have some equally-well-thought but opposite belief. Merely that taking our cues from C++ here might not work very well either: their subtyping-on-const rules are often bizarre and related to many matters we don't face, and they start with the assumption that everything-is-mutable, and they don't have safe-alias analysis ... I think it's useful to *look* at what they come up with, but we might well wind up someplace other than they did. It's also useful to look at cyclone, D, the MLs, and various other players in this space. > 3. Regarding structural types, I shouldn't have dragged them into this. > :) There are many ways to address the verbosity of mutable fields, of > which nominal record types are but one. Fair. I've only been carrying on with the term "structural" in response as a short hand for denoting "interior types composed of records, resources, tags, tuples, and scalars". But that seemed like a long noun-phrase. Perhaps "structured" is a better word :) > Hope that makes sense. I can clarify more when I am not typing on my > phone. :) Yeah. We'll talk more. -Graydon From talex5 at gmail.com Wed Feb 8 10:47:21 2012 From: talex5 at gmail.com (Thomas Leonard) Date: Wed, 8 Feb 2012 18:47:21 +0000 Subject: [rust-dev] Cargo requirements In-Reply-To: <4F31AB36.9020705@mozilla.com> References: <4F2067CB.6010309@mozilla.com> <4F27345E.1040101@mozilla.com> <4F31AB36.9020705@mozilla.com> Message-ID: On 7 February 2012 22:52, Graydon Hoare wrote: > On 04/02/2012 4:13 AM, Thomas Leonard wrote: > >> But it doesn't remove the need to handle diamond dependencies. Very >> often (most of the time, I think), if two libraries depend on a common >> one then it's because they need to communicate using a common API or >> common set of data-types. > > This is true to an extent. Within Rust (when the versioning mechanism is > finished, tested, integrated into cargo) we'll handle disagreeing-diamonds > ok. Sorry, I'm still not clear about this. For example, I wrote a simple GTK text editor that uses the GtkSourceView library. By "handle disagreeing-diamonds", do you meaning that a) my program and GtkSourceView see a single version of GTK, compatible with both of them, or b) they each see a version of GTK they are compatible with, but not necessarily the same one (so I cannot embed the GtkSourceView widget into my GtkWindow)? (assuming here that GTK and GtkSourceView are Rust libraries, or Rust wrappers for the C libraries) > For libraries outside Rust, there are mechanisms people have worked out for > each OS to try to manage such disagreeing-diamonds too (sonames, which don't > work because they don't sort and don't get bumped often enough; symbol > versioning, which might work except nobody uses it; SxS, which works more > like rust, but gets enormous). Sure. > What I'm saying, repeating here since it didn't seem to stick with the last > message, is is that there is a threshold at which any such > version-everything solution *must give up*. I don't recall saying anything else. I'm just trying to find out how Rust plans to cope with some common packaging problems. My experience from other languages is that situations such as the GTK one above are very common. > You cannot version the entire universe. Not even the software-and-hardware > universe. Both because it's too big and because (as I get to down below) > each level is itself non-computable. You mean the problem of selecting a set of versions such that the program is guaranteed to compile and run correctly? Of course we can't solve that. But any piece of software will come with some requirements from the author, like "You need GTK >= 2.12 to compile this program". These requirements have to be satisfied somehow. Either the computer does it, or the human does it. >> For example, if I write an application using GTK to handle windows, >> and I import a widget library, then it's no good having the widget >> library using its own version of GTK, because I won't be able to embed >> its widgets in my windows. > > I know, it's awful, and you can't always solve this. Even if you put the > GTKs *right next to each other* in the target environment you can't even > solve this, because they have global PLT-resolved symbols they fail to > version and the loader may pick some from one library and some from another > as it lazily resolves stuff. If the application and the library really do require different versions of GTK (e.g. <2 and >2) then you can't solve it. But if they require e.g. <2.12 and >2.10 then selecting 2.10.1 would be fine. >> Sure, but now the user needs to manage two separate logging >> configurations for the same application, or risk them overwriting each >> other's log files (although this is probably a bad example, as I see >> that Rust very sensibly has built-in logging support). >> >> By the way, does this mean that a Rust program must link against >> exactly the same version of the library that was used at build time? >> e.g. if I upgrade libssl then I must rebuild all applications using >> it? > > > No, we'll probably support some level of version-range or prefix-matching. > Despite the fact that this *weakens* the likelihood of proper functioning. > > I accept, however, that precise-versioning and loose-coupled-upgrades are in > inherent engineering-tradeoff with one another. So I'm ok picking a point on > the tradeoff spectrum based on my sense of taste and annoyance (in > consultation with others). I don't believe it's solvable. Naturally. It's up to the programmer how tight to make the bounds. Incidentally, keeping the metadata outside of the source code can help with this (when a new GTK comes out and breaks your program, you can just update the metadata file on the web-server with a tighter requirement rather than making a whole new release). > Really, as I said above, I feel like this whole conversation is stuck on > your assumption that the problem is actually solvable (or even precisely > denotable). And in a very thorough and concrete way, I don't think it is. > For perhaps 4 broad reasons (along with a zillion technical details): I don't think I ever claimed any such thing (I only asked how Rust plans to handle dependencies). I do believe that, given a set of version constraints specified by the programmers of the various components, there exist algorithms to select a set of versions that meet all of them (if any such selection exists). Indeed, 0install uses such an algorithm. If the programmer doesn't state a requirement then 0install doesn't consider it. > ?1. There's tension between precise-coupling (for correctness) and > ? ?loose-coupling (for ease of use). This is why versioning systems > ? ?often support imprecise wildcard matching, symbolic names, etc. > ? ?And as soon as you have symbolic names you have a problem of > ? ?naming authority, which -- if you take seriously -- you wind up > ? ?having to invent something like PKI in order to solve. Punt to > ? ?DNS or GPG whenever possible, maybe, but Naming Is Hard and it's > ? ?usually a source of endless assumption-mismatch, precisely because > ? ?names occupy a weird, ill-defined neither-zone between structured > ? ?data and opaque nonces. > > ?2. A -> B dependency when A and B are turing-complete artifacts is > ? ?generally non-computable. I.e. you can't even *define* when you > ? ?have "captured" a dependency accurately. You can capture a declared > ? ?dependency but it doesn't actually guarantee there's no undeclared > ? ?dependency. This is why many people are taking to using bit-identity > ? ?of the delivered artifacts (git content addressing, "ship the DLLs > ? ?side-by-side with the .exe", or just shipping whole system images > ? ?/ VM images / etc. rather than symbolically-described "packages") > > ?3. Enumerating the "dependencies of A" depends on what you want to > ? ?*do* with A. 0install talks about executing, but we need to compile, > ? ?bootstrap-compile, recompile-LLVM-and-self-host-with-it, build docs, > ? ?build debugging output, run tests ... you wind up reinventing build > ? ?systems in general in order to handle all the things a developer > ? ?wants to capture the "dependencies for". > > ?4. The *entire* environment is a dependency. It's often not just > ? ?non-computable but physically off limits to versioning. I.e. you > ? ?can version a kernel but not the firmware of the machine it's > ? ?running, on because the firmware isn't under software control, > ? ?can't be analyzed/sandboxed/controlled. This goes double for > ? ?the network environment in which something runs or, heaven forbid, > ? ?the physical environment in which the system winds up instantiated. > > The problem is just ... bottomless. I'm sorry. I know you've spent a lot of > energy on versioning and built an ambitious system. I'm sympathetic. I did > so too, a long time back. But I'm relatively convinced at this point that > there's no Correct Solution, just a bunch of wrong ones that have different > forms of associated pain and cognitive load, and at some point you have to > draw a line and get on with whatever else you were Actually Doing, rather > than further work on versioning. We're doing a programming language. I'm ok > developing cargo and rustc based on perceived and encountered needs, rather > than striving for perfection. Sure. 0install is the same. In 2003, it worked just as Cargo does now (you could depend on a specific version, or on the "latest" version). Later, that turned out to be too limited. In 2005, it gained version ranges and a naive solver (that just walked the tree depth first selecting the best available version for each component as it went). That was OK for smaller programs. As the number of (declared) dependencies got larger, that broke down and we replaced it with a proper SAT solver that always finds a correct solution (if one exists, or aborts otherwise). -- Dr Thomas Leonard? ? ? ? http://0install.net/ GPG: 9242 9807 C985 3C07 44A6? 8B9A AE07 8280 59A5 3CC1 GPG: DA98 25AE CAD0 8975 7CDA? BD8E 0713 3F96 CA74 D8BA From niko at alum.mit.edu Wed Feb 8 11:58:35 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Wed, 08 Feb 2012 11:58:35 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <4F32B3A6.20308@mozilla.com> References: <4F31B2B2.9070009@alum.mit.edu> <4F32886C.9070600@alum.mit.edu> <4F32A55B.6030901@mozilla.com> <4F32AE7E.6090306@alum.mit.edu> <4F32B3A6.20308@mozilla.com> Message-ID: <4F32D3EB.8070708@alum.mit.edu> On 2/8/12 9:40 AM, Graydon Hoare wrote: > I understand you've developed strong thoughts about this, as has > pcwalton, and suspect it results from a lot of high-bandwidth, > in-person conversations in mountain view. I respectfully ask you to > move *slowly* and at a somewhat laborious, tedious-to-you pace on the > topic, for the "remoties" (marijn and myself). We've not been party to > *any* of the in-person whiteboarding or conversing that leads to this > conclusion, so the "to my mind ... only two options" statements come > off as very immediate, certain and restrictive. I'm sorry if I came across as wanting to shut off debate. That wasn't my intention. I woke up quite early today to take care of my daughter's teething episode so I am feeling a bit grouchy. Perhaps that's coming through. If you look at what I wrote, the "second option" was basically "do something else." That is, I think there is a fairly wide set of solutions to address this problem. I happen to think the one I proposed is the closest to what we have today and something which may in fact be necessary for other use cases. But basically I think whatever we choose ought to enable the compiler and user to identify immutable memory with as little context as possible, as it is very useful for optimization and correctness (as illustrated by the Dan's bug problem). Just limiting variance still presents a murky story, where given a reference to an instance of type T you don't really know whether its fields are immutable or not. I suppose you would have to think of the lack of a "mutable" declaration as "potentially immutable"; in the context of an @T or ~T pointer, those fields would truly be immutable. I'm not sure whether having field-level mutability is the best fit in that case, maybe mutability modifiers in types would be better (`mut T` etc). Niko From graydon at mozilla.com Wed Feb 8 13:35:39 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Wed, 08 Feb 2012 13:35:39 -0800 Subject: [rust-dev] reorg of os / libc bindings Message-ID: <4F32EAAB.9020505@mozilla.com> Hi, I'm reorganizing the OS and C library bindings in libstd (and moving them to libcore). These are presently a highly-duplicated hodge-podge of declarations scattered between modules that were placed in separate per-platform files largely due to an old restriction in how we did conditional compilation. So it needs cleanup, and I've got quite a bit sketched on what to put in that cleanup already. I'm less sure on the external interface and module structure to adopt. I'm starting with the "lower level", which is the native libc part itself, before moving up to the Rust-level "os" module on top. The contents of a libc varies by platform. There are two conflicting tensions in the matter: - The definitions in a libc are typically present in "Compatibility blocks" based on particular standards[1] (c89, c99, posix88, posix08..) Sometimes when writing client code the presence of such a compatibility block is a nice requirement to rest your code on. - More-casual users often don't really care which compatibility block some functionality comes from. You don't usually care that you get SEEK_SET from c89 but O_RDWR from posix88. People tend to just cargo-cult the #include lines until their code compiles. So what I'm tentatively doing is putting *everything* you might want to access from libc in core::libc, with a bunch of submodules for the individual specs. Then importing and re-exporting all the symbols found in the submodules at the core::libc::* level. So if you want to refer to just "libc::creat" you can do so, but if you are being fussy you can do "import libc::posix88;" and then use "posix88::creat" when you want it. So, questions: - Is this structure roughly ok? Useful? Silly? - Should I put windows-specific parts of msvcrt in there as well? - Is it wrong to try to "gather up" the bits found in common between the ANSI C standard library[2] and the POSIX C standard library[3]? - Is it wrong to call this library 'core::libc', is something else better? -Graydon [1] http://upload.wikimedia.org/wikipedia/commons/4/46/SUS_History.svg [2] http://en.wikipedia.org/wiki/C_Run-Time_Library#Header_files [3] http://en.wikipedia.org/wiki/C_POSIX_library From graydon at mozilla.com Wed Feb 8 17:04:52 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Wed, 08 Feb 2012 17:04:52 -0800 Subject: [rust-dev] Cargo requirements In-Reply-To: References: <4F2067CB.6010309@mozilla.com> <4F27345E.1040101@mozilla.com> <4F31AB36.9020705@mozilla.com> Message-ID: <4F331BB4.1050601@mozilla.com> On 2/8/2012 10:47 AM, Thomas Leonard wrote: > Sorry, I'm still not clear about this. For example, I wrote a simple > GTK text editor that uses the GtkSourceView library. By "handle > disagreeing-diamonds", do you meaning that > > a) my program and GtkSourceView see a single version of GTK, > compatible with both of them, or > b) they each see a version of GTK they are compatible with, but not > necessarily the same one (so I cannot embed the GtkSourceView widget > into my GtkWindow)? A "disagreeing" diamond would be one where there's no single version that satisfies both. If they can be made to agree -- if there's a solvable constraint-set -- then it's just a matter of choosing the right one. Which is tricky, as you point out (might need a constraint-solver) but at least doesn't require additional machinery once you've chosen a version. > If the application and the library really do require different > versions of GTK (e.g.<2 and>2) then you can't solve it. IME this sort of situation is quite common too and it's what our versioning mechanisms are built to accommodate. You can't usually solve it in GTK-land because of the way PLTs on unversioned C symbols work, and the general lack of symbol-versioning within C libraries. Our symbols are all versioned. > I don't think I ever claimed any such thing (I only asked how Rust > plans to handle dependencies). Sorry, my tone was too negative and full of hyperbole. I apologize for putting words in your mouth. I just can't help the feeling that this conversation is geared towards trying to convince me that there's some aspect of baseline correctness that is so hard to get right that we can't grow it ourselves (or copy from 0install, or apt, or any other dependency-resolver) as we encounter it, but should instead give up now and just use it. I don't really feel that's a good option, for a bunch of cognitive complexity and UI / role reasons I've pointed out already. I don't debate that 0install does more (and more correctly) than cargo currently does. I hope it's ok if we crib logic from it when and if we need to. But I don't feel that's a conclusive argument that we ought to adopt it wholesale as a system dependency. -Graydon From banderson at mozilla.com Wed Feb 8 17:21:59 2012 From: banderson at mozilla.com (Brian Anderson) Date: Wed, 8 Feb 2012 17:21:59 -0800 (PST) Subject: [rust-dev] How to write Rust code that blocks natively In-Reply-To: <847248023.5183339.1328749638787.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Message-ID: <236149415.5190209.1328750519078.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Hey. One of the several painful things about creating bindings for native libraries is that calling blocking functions prevents other tasks from making progress. This makes using things like sockets or libuv problematic. No more - a solution is here! The runtime now allows for schedulers to be created and added dynamically, and by creating your own scheduler you can block for as long as you like without interfering with tasks on other schedulers. We currently have a single function that can be used to take advantage of this: core::task::spawn_sched. It spawns a new task into a new scheduler and takes an argument for the number of OS threads the scheduler should use. spawn_sched(1) {|| // do some blocky sockets business } // go on my merry way The scheduler will die when all tasks running on it die. By default (when not using spawn_sched), tasks inherit their schedulers, so if your blocky task wants to spawn other tasks that shouldn't be interfered with it will need to do so through an intermediary on a different scheduler (which is easy enough to do with unique closures). This interface will continue to change. Regards, Brian From banderson at mozilla.com Wed Feb 8 17:32:13 2012 From: banderson at mozilla.com (Brian Anderson) Date: Wed, 8 Feb 2012 17:32:13 -0800 (PST) Subject: [rust-dev] reorg of os / libc bindings In-Reply-To: <4F32EAAB.9020505@mozilla.com> Message-ID: <1540243454.5196266.1328751132989.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> ----- Original Message ----- > From: "Graydon Hoare" > To: rust-dev at mozilla.org > Sent: Wednesday, February 8, 2012 1:35:39 PM > Subject: [rust-dev] reorg of os / libc bindings > > Hi, > > - Is this structure roughly ok? Useful? Silly? I think from an end-user perspective such fine-grained organization is not that useful (you'll always just use the reexported version) but +1 for keeping things neat and clean. > - Should I put windows-specific parts of msvcrt in there as well? I think so. > - Is it wrong to try to "gather up" the bits found in common > between > the ANSI C standard library[2] and the POSIX C standard > library[3]? No opinion. > - Is it wrong to call this library 'core::libc', is something else > better? core::libc is good. From graydon at mozilla.com Wed Feb 8 17:59:58 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Wed, 08 Feb 2012 17:59:58 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <4F32D3EB.8070708@alum.mit.edu> References: <4F31B2B2.9070009@alum.mit.edu> <4F32886C.9070600@alum.mit.edu> <4F32A55B.6030901@mozilla.com> <4F32AE7E.6090306@alum.mit.edu> <4F32B3A6.20308@mozilla.com> <4F32D3EB.8070708@alum.mit.edu> Message-ID: <4F33289E.5070106@mozilla.com> On 2/8/2012 11:58 AM, Niko Matsakis wrote: > On 2/8/12 9:40 AM, Graydon Hoare wrote: >> I understand you've developed strong thoughts about this, as has >> pcwalton, and suspect it results from a lot of high-bandwidth, >> in-person conversations in mountain view. I respectfully ask you to >> move *slowly* and at a somewhat laborious, tedious-to-you pace on the >> topic, for the "remoties" (marijn and myself). We've not been party to >> *any* of the in-person whiteboarding or conversing that leads to this >> conclusion, so the "to my mind ... only two options" statements come >> off as very immediate, certain and restrictive. > > I'm sorry if I came across as wanting to shut off debate. That wasn't my > intention. I woke up quite early today to take care of my daughter's > teething episode so I am feeling a bit grouchy. Perhaps that's coming > through. Oh no, nothing of the sort. I miscommunicated. Probably this stems from my feeling overwhelmed and outpaced lately ... er ... since around this time last year? Every few months I manage a brief period where I think I understand the whole language, and then it races ahead away from my understanding again. For a project I started, it's an unnerving feeling. IOW if possible, please try to hear any time I ask for a slower pace as, er, flattery? You're _really good_ at this stuff -- much better than I am -- and I'm just trying to keep up. The whole team is well beyond my skill-level really. I'm not exaggerating; read the commit logs if you have any doubt. And while I totally trust that you have these things thought-through very well, part of me (perhaps a part that needs to learn to shut up) still wants to know what's going on, and why it's right. Particularly with language-semantic changes. But I gather I'm communicating that desire in an over-critical, unhelpful way, that comes across as "stop energy"[1] that shoots people down. I apologize for that. I'll try to keep such comments more focused on the meat of the technical content, and less knee-jerk conservative moaning. You're totally doing the right thing by bringing this up (and many other issues queued up on the back-burner; I'm aware of how very many changes we have pending), and it's much more important to maintain a positive and productive sense of "forward motion" than it is for me to feel entirely clear and convinced of everything a priori. -Graydon [1] http://www.userland.com/whatIsStopEnergy From niko at alum.mit.edu Wed Feb 8 19:06:49 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Wed, 08 Feb 2012 19:06:49 -0800 Subject: [rust-dev] How to write Rust code that blocks natively In-Reply-To: <236149415.5190209.1328750519078.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> References: <236149415.5190209.1328750519078.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Message-ID: <4F333849.1060408@alum.mit.edu> This is very cool. On 2/8/12 5:21 PM, Brian Anderson wrote: > Hey. > > One of the several painful things about creating bindings for native libraries is that calling blocking functions prevents other tasks from making progress. This makes using things like sockets or libuv problematic. No more - a solution is here! > > The runtime now allows for schedulers to be created and added dynamically, and by creating your own scheduler you can block for as long as you like without interfering with tasks on other schedulers. > > We currently have a single function that can be used to take advantage of this: core::task::spawn_sched. It spawns a new task into a new scheduler and takes an argument for the number of OS threads the scheduler should use. > > spawn_sched(1) {|| > // do some blocky sockets business > } > > // go on my merry way > > The scheduler will die when all tasks running on it die. By default (when not using spawn_sched), tasks inherit their schedulers, so if your blocky task wants to spawn other tasks that shouldn't be interfered with it will need to do so through an intermediary on a different scheduler (which is easy enough to do with unique closures). > > This interface will continue to change. > > Regards, > Brian > _______________________________________________ > Rust-dev mailing list > Rust-dev at mozilla.org > https://mail.mozilla.org/listinfo/rust-dev From graydon at mozilla.com Wed Feb 8 19:07:46 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Wed, 08 Feb 2012 19:07:46 -0800 Subject: [rust-dev] How to write Rust code that blocks natively In-Reply-To: <236149415.5190209.1328750519078.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> References: <236149415.5190209.1328750519078.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Message-ID: <4F333882.4050207@mozilla.com> On 2/8/2012 5:21 PM, Brian Anderson wrote: > We currently have a single function that can be used to take advantage of this: core::task::spawn_sched. It spawns a new task into a new scheduler and takes an argument for the number of OS threads the scheduler should use. > > spawn_sched(1) {|| > // do some blocky sockets business > } > > // go on my merry way \o/ Great, thanks. -Graydon From sebastian.sylvan at gmail.com Wed Feb 8 19:22:30 2012 From: sebastian.sylvan at gmail.com (Sebastian Sylvan) Date: Wed, 8 Feb 2012 19:22:30 -0800 Subject: [rust-dev] How to write Rust code that blocks natively In-Reply-To: <236149415.5190209.1328750519078.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> References: <847248023.5183339.1328749638787.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> <236149415.5190209.1328750519078.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Message-ID: On Wed, Feb 8, 2012 at 5:21 PM, Brian Anderson wrote: > Hey. > > One of the several painful things about creating bindings for native libraries is that calling blocking functions prevents other tasks from making progress. This makes using things like sockets or libuv problematic. No more - a solution is here! > You might want to look at User Mode Scheduling http://msdn.microsoft.com/en-us/library/windows/desktop/dd627187(v=vs.85).aspx It's windows only, and only 64-bit at that, but it allows you to schedule threads manually. Basically if a thread calls in to the system (or manually calls yield), the system switches to your scheduler instead of blocking the OS thread. -- Sebastian Sylvan From graydon at mozilla.com Thu Feb 9 09:30:01 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Thu, 09 Feb 2012 09:30:01 -0800 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <1328777580.69806.YahooMailClassic@web35606.mail.mud.yahoo.com> References: <1328777580.69806.YahooMailClassic@web35606.mail.mud.yahoo.com> Message-ID: <4F340299.4090008@mozilla.com> (forwarded to list on request) On 09/02/2012 12:53 AM, Alex Burr wrote: > FWIW,the issue of mutability was discussed a lot on the bitc-dev mailing > list. > A particularly troubling case was if you have a struct foo which is > marked immutable, which > contains a field bar which is not, what happens if you take a reference > to bar and pass it > to a function F? > > The type of F doesn't know anything about foo, so F could potentially > mutate it. > > I don't know enough about rust to see if you have this problem. Yeah. That's more or less what we're talking about here. We have an existing sort of approach to this that, sadly, has a soundness hole in it due to its subtyping rules. My initial hope is that maybe we can fix the subtyping rules. But others are, I think, proposing we keep the subtyping rules as they are and adopt a secondary definition similar to what C++ does, differentiating "assignability" from non. Possibly buying us some more wiggle room in (or even freedom-from-needing) our safe-reference checker. And possibly (?) working better with some other future work (regions, interior vectors). I don't actually know how C++ goes about deriving and propagating that assignability definition -- or whether it simply has to do with its auto-generation of op= methods -- so have to look more into what it means there before I can suitably comment. It might be similar to our reference-checker's current synthetic concept of "immutably rooted" values. It might wind up working well, not sure. There might be several other approaches too; it's an old and quite general sort of problem. Really I think the question hinges on intuitions about transitivity, figuring out which is more counterintuitive: - Something you mark as mutable that you find yourself restricted from mutating due to its sub- or super-structure being immutable. - Something marked as immutable that still manages to change bits-in-memory due to part of its sub- or super-structure being mutated. Neither is really great. Depending on how you model things, the latter may be a soundness hole (I had thought our safe-reference checker prevented it, but apparently not). As with so many things in languages, we're dancing around a tension between unpleasant results, looking for a nice compromise, or at most a compelling reason why one or the other is more-necessary. -Graydon From graydon at mozilla.com Thu Feb 9 11:48:11 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Thu, 09 Feb 2012 11:48:11 -0800 Subject: [rust-dev] RFC: glob exports (bug 1238) Message-ID: <4F3422FB.1030602@mozilla.com> I don't think this is terribly contentious, but worth checking: A bug dylukes opened back in december is definitely biting my progress (on the same area it was biting him, of course): exporting by glob. In particular, some combination of importing stuff from a submodule and re-exporting https://github.com/mozilla/rust/issues/1238 Anyone mind if I implement it? It's a pretty direct generalization of what we permit for imports. Should just make organizing library code a little less arduous. -Graydon From banderson at mozilla.com Thu Feb 9 12:02:21 2012 From: banderson at mozilla.com (Brian Anderson) Date: Thu, 9 Feb 2012 12:02:21 -0800 (PST) Subject: [rust-dev] RFC: glob exports (bug 1238) In-Reply-To: <4F3422FB.1030602@mozilla.com> Message-ID: <846505961.5622392.1328817741063.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> ----- Original Message ----- > From: "Graydon Hoare" > To: rust-dev at mozilla.org > Sent: Thursday, February 9, 2012 11:48:11 AM > Subject: [rust-dev] RFC: glob exports (bug 1238) > > I don't think this is terribly contentious, but worth checking: A bug > dylukes opened back in december is definitely biting my progress (on > the > same area it was biting him, of course): exporting by glob. In > particular, some combination of importing stuff from a submodule and > re-exporting > > https://github.com/mozilla/rust/issues/1238 > > Anyone mind if I implement it? It's a pretty direct generalization of > what we permit for imports. Should just make organizing library code > a > little less arduous. I agree. Something like this is the easiest way to do what you are trying to do with libc, and will make the float/f32, etc. situation better. From niko at alum.mit.edu Thu Feb 9 13:01:04 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Thu, 09 Feb 2012 13:01:04 -0800 Subject: [rust-dev] RFC: glob exports (bug 1238) In-Reply-To: <4F3422FB.1030602@mozilla.com> References: <4F3422FB.1030602@mozilla.com> Message-ID: <4F343410.8030505@alum.mit.edu> On 2/9/12 11:48 AM, Graydon Hoare wrote: > Anyone mind if I implement it? It's a pretty direct generalization of > what we permit for imports. Should just make organizing library code a > little less arduous. +1 From theoneandonlykosta at googlemail.com Thu Feb 9 13:41:04 2012 From: theoneandonlykosta at googlemail.com (Kosta Welke) Date: Thu, 9 Feb 2012 22:41:04 +0100 Subject: [rust-dev] RFC: Addressing Dan's bug through immutable memory In-Reply-To: <4F340299.4090008@mozilla.com> References: <1328777580.69806.YahooMailClassic@web35606.mail.mud.yahoo.com> <4F340299.4090008@mozilla.com> Message-ID: Graydon Hoare : > Really I think the question hinges on intuitions about transitivity, figuring out which is more counterintuitive: > > - Something you mark as mutable that you find yourself restricted > from mutating due to its sub- or super-structure being immutable. > > - Something marked as immutable that still manages to change > bits-in-memory due to part of its sub- or super-structure being > mutated. > > Neither is really great. Exactly. However, as far as I understood this language, immutability is the default. So if you have an immutable struct that imports a mutable substructure, the substructure should become immutable, too. Immutable wins. This of course means that if a function expects a mutable substructure, that function does not apply. However, you can always call the function on a copy of the structure, if you must. Just my two cents, Kosta PS: I'm new here btw. And I have written about 5 lines of Rust in total so far. From niko at alum.mit.edu Thu Feb 9 20:07:51 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Thu, 09 Feb 2012 20:07:51 -0800 Subject: [rust-dev] Python-like slicing for vectors, strings Message-ID: <4F349817.7070709@alum.mit.edu> Any objections to this proposal? https://github.com/mozilla/rust/issues/1799 Niko From niko at alum.mit.edu Fri Feb 10 06:32:04 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Fri, 10 Feb 2012 06:32:04 -0800 Subject: [rust-dev] type holes and identifying immutable memory Message-ID: <4F352A64.6040801@alum.mit.edu> So, I wanted to take a step back and look at this type hole concerning mutability. I think there are a number of intertwined concerns. Anyway, I've been thinking over the "menu" of possible solutions and I wanted to lay out the ones that I see at the moment. This is not to say that there are not others. I apologize that this is kind of a long mail. 1. *Create a class of assignable types.* This is what I initially proposed. I have a patch in a sandbox that implements this, hopefully today I can come back with some data about how disruptive this is to the existing codebase. 2. *Give the type checker a clearer view of when memory is mutable. *The worst part of the problem right now is that if you have a parameter declared like so: `&x: {f: T}`, the field `f` is considered immutable by the type checker, although in fact the mode `&` allows `x` to be overwritten. This combines with subtyping to create an inconsistent view over mutable memory: the callee can think that the memory has type T but the caller thinks it has type T' where T' <: T. Bad. If the type checker were aware of modes, we could have the type checker treat the field `f` as mutable even though it is declared immutable. Integrating modes in the type checker is somewhat unappealing to me, because they are supposed to be orthogonal, so an alternative would be to integrate first-class reference types as is proposed in the regions proposal. This would mean that the parameter `x` would be declared as `x: &mut {f: T}`. Now the type checker can more easily see that the field `f` may be mutated. (Note that the use of `&` types does not require the full region proposal, we could impose arbitrary limitations that make them basically equivalent to modal arguments). The main downside to option #2 from my point of view is that it presents a kind of complex story. I was initially opposed to it because it complicates the type checker and it seems hard to explain: we say fields are immutable by default but that's not really true (although it is true in a lot of important cases). Therefore, I was thinking about a third option that builds on the `&` types to provide a simpler story about mutability. I think this more closely meets peoples intutions: 3. *Change mutability to be "all-or-nothing".* This actually builds on solution #2. The goal is to address the complicated story about when fields are mutable or immutable that results. The basic idea is that we do not mark individual fields of a record as mutable or immutable, but rather the entire record itself. Interior records then naturally inherit the mutability of their container, and records stored in mutable local variables can be overridden without problems. I spelled out this idea a bit in , though this is more of a sketch than a detailed proposal. The main downside of this option is that we lose the ability to specify "field x is mutable but field y is not" (but then, that was the idea: the mutability of fields is supposed to come from the outside now). *Another cross-cutting concern so to speak has to do with the alias checker.* One of the motivations for exploring this issue was to try and find alternatives that do not require an alias checker to be sound: we could still keep the alias checker in the code base, as it can provide very useful guarantees for reasoning about your program, but I would prefer if the user has to "opt-in" to use it. [1] This way there would be no problems with functions that invoke closures having to make extra copies simply to please the alias checker and so forth. Both options #2 and #3 will basically result in the type checker being more conservative about when fields are considered potentially mutable. Right now we consider any field that is not declared mutable as immutable. Only option #1 allows us to keep doing that. This in turn means that things like the Dan's bug proposal I put forward can get away with making fewer copies at runtime. However, it may be that having (1) immutable local variable declarations by default as proposed in #1273 and (2) immutable references of the form (rather than read-only references, as we have today) could address that last concern satisfactorily. At some point I thought that options #2 and #3 would still require an alias checker to be sound, but I am not sure why I thought that at the moment. It seems to me that any potentially mutable piece of memory storing an enum would either have to be in a mutable local variable or reachable via a pointer with a `mut` or `const` qualifier, regardless of how many times it is aliased: in either of those cases, the type system can clearly see that it is mutable and enforce that it cannot be matched with a copy to immutable memory. Niko From niko at alum.mit.edu Fri Feb 10 06:48:39 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Fri, 10 Feb 2012 06:48:39 -0800 Subject: [rust-dev] type holes and identifying immutable memory In-Reply-To: <4F352A64.6040801@alum.mit.edu> References: <4F352A64.6040801@alum.mit.edu> Message-ID: <4F352E47.90709@alum.mit.edu> On 2/10/12 6:32 AM, Niko Matsakis wrote: > we could still keep the alias checker in the code base, as it can > provide very useful guarantees for reasoning about your program, but I > would prefer if the user has to "opt-in" to use it. [1] I forgot to spell out what I had in mind. Basically it'd be a safe version of ANSI C's restrict keyword. A function could declare that its parameters MUST not be aliased. I'm not sure how well this would work in practice but it could be pretty nice to have. In particular, I could imagine it being combined with a check predicate of sorts, so that you could write something like: fn foo(x: @T, y: @T) { check distinct(x, y); memmove(x, y); // or whatever } fn memmove(x: @T, y: @T) : distinct(x, y) { ... } Anyway, this is just a random thought. I have to say though that my opposition to alias analysis has somewhat softened now that I better understand the guarantees that the alias checker is trying to enforce. Before I had a vague sense of what they were and it felt kind of frustrating to have to insert copies for reasons I did not fully understand. However, if we try to move to a more flexible system like regions, I do not think it will be possible to simply forbid aliases as we do now, because part of the goal of that system is to allow for pointers to be used more naturally, which often includes aliasing. Niko From talex5 at gmail.com Fri Feb 10 11:08:02 2012 From: talex5 at gmail.com (Thomas Leonard) Date: Fri, 10 Feb 2012 19:08:02 +0000 Subject: [rust-dev] Cargo requirements In-Reply-To: <4F331BB4.1050601@mozilla.com> References: <4F2067CB.6010309@mozilla.com> <4F27345E.1040101@mozilla.com> <4F31AB36.9020705@mozilla.com> <4F331BB4.1050601@mozilla.com> Message-ID: On 9 February 2012 01:04, Graydon Hoare wrote: [...] > I just can't help the feeling that this conversation is geared towards > trying to convince me that there's some aspect of baseline correctness that > is so hard to get right that we can't grow it ourselves (or copy from > 0install, or apt, or any other dependency-resolver) as we encounter it, but > should instead give up now and just use it. Not at all. Writing a package system is pretty easy (which is why there are so many of them). The most interesting and non-obvious bit of 0install is the constraint solver, which we took, with minor changes, from the algorithms in the literature. See: http://stackoverflow.com/questions/2522362/what-are-the-cs-fundamentals-behind-package-dependency-management Getting security and decentralisation right at the start is quite important (if you start off by assuming that all packages should be trusted with root access, and centrally named, that's hard to fix later). But I don't see cargo making those mistakes. I just think it's a bit sad that every time I want to use a different language I have to learn to use a different installation tool, with its own documentation, syntax, solver, security features, and file-system locations, each incompatible with all the others. But maybe there's no point during the development of a new packaging system where the quickest way to add the next feature is to adopt some existing system. > I don't really feel that's a good option, for a bunch of cognitive > complexity and UI / role reasons I've pointed out already. I don't debate > that 0install does more (and more correctly) than cargo currently does. I > hope it's ok if we crib logic from it when and if we need to. But I don't > feel that's a conclusive argument that we ought to adopt it wholesale as a > system dependency. Crib as much as you like :-) -- Dr Thomas Leonard? ? ? ? http://0install.net/ GPG: 9242 9807 C985 3C07 44A6? 8B9A AE07 8280 59A5 3CC1 GPG: DA98 25AE CAD0 8975 7CDA? BD8E 0713 3F96 CA74 D8BA From niko at alum.mit.edu Fri Feb 10 15:25:29 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Fri, 10 Feb 2012 15:25:29 -0800 Subject: [rust-dev] type holes and identifying immutable memory In-Reply-To: <4F352A64.6040801@alum.mit.edu> References: <4F352A64.6040801@alum.mit.edu> Message-ID: <4F35A769.7030802@alum.mit.edu> On 2/10/12 6:32 AM, Niko Matsakis wrote: > 3. *Change mutability to be "all-or-nothing".* This actually builds > on solution #2. The goal is to address the complicated story about > when fields are mutable or immutable that results. The basic idea is > that we do not mark individual fields of a record as mutable or > immutable, but rather the entire record itself. Interior records then > naturally inherit the mutability of their container, and records > stored in mutable local variables can be overridden without problems. > I spelled out this idea a bit in > , though > this is more of a sketch than a detailed proposal. The main downside > of this option is that we lose the ability to specify "field x is > mutable but field y is not" (but then, that was the idea: the > mutability of fields is supposed to come from the outside now). Thinking more about this, I am not sure that option #3 has any point. It is basically the same as option #2 but less expressive. It's a case of "when I started playing with this idea, it was different, but I found inconsistencies and over time it evolved to be the same thing as I started with." Niko From marijnh at gmail.com Sat Feb 11 00:17:52 2012 From: marijnh at gmail.com (Marijn Haverbeke) Date: Sat, 11 Feb 2012 09:17:52 +0100 Subject: [rust-dev] type holes and identifying immutable memory In-Reply-To: <4F352A64.6040801@alum.mit.edu> References: <4F352A64.6040801@alum.mit.edu> Message-ID: I feel that I simply don't have the type-theoretic background to say anything definite about this. I'm going to change my stance from 'grumbling darkly in the background' to 'I guess you guys know what you're doing, I'm not objecting'... > hopefully today I can come back with some data about how disruptive this is to the existing codebase. Right. This is important. If a change end up introducing a lot of awfulness in existing code, that should be a reason to re-think it. From banderson at mozilla.com Mon Feb 13 13:31:22 2012 From: banderson at mozilla.com (Brian Anderson) Date: Mon, 13 Feb 2012 13:31:22 -0800 (PST) Subject: [rust-dev] The state of Rust on FreeBSD In-Reply-To: <568609600.7265560.1329168052400.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Message-ID: <376065250.7276815.1329168682738.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Hi. Folks periodically express a desire to run Rust on FreeBSD, and until now we've had to say that it wasn't possible. Thanks to a big effort over the last month by Jyun-Yan You we can now build Rust and run the entire test suite on FreeBSD. While I can't claim that it is 'officially supported', I do want to encourage folks interested in using this combo to give it a shot. I'll do my best to ensure that snapshots continue to be available. There are FreeBSD-specific notes on the getting started wiki page[1]. You should also probably be aware that the FreeBSD build has an issue that prompted me to temporarily turn off multithreading[2]. Regards, Brian [1] - https://github.com/mozilla/rust/wiki/Doc-getting-started [2] - https://github.com/mozilla/rust/issues/1825 From graydon at mozilla.com Mon Feb 13 18:19:01 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Mon, 13 Feb 2012 18:19:01 -0800 Subject: [rust-dev] The state of Rust on FreeBSD In-Reply-To: <376065250.7276815.1329168682738.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> References: <376065250.7276815.1329168682738.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Message-ID: <4F39C495.2010803@mozilla.com> On 2/13/2012 1:31 PM, Brian Anderson wrote: > Folks periodically express a desire to run Rust on FreeBSD, and until now we've had to say that it wasn't possible. Thanks to a big effort over the last month by Jyun-Yan You we can now build Rust and run the entire test suite on FreeBSD. Great, thanks! Hopefully I can put together a more-generic form of "new platform process" for the next- and next-next platforms; we'll need to start work on an ARM port eventually too :) -Graydon From graydon at mozilla.com Mon Feb 13 18:40:02 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Mon, 13 Feb 2012 18:40:02 -0800 Subject: [rust-dev] type holes and identifying immutable memory In-Reply-To: <4F35A769.7030802@alum.mit.edu> References: <4F352A64.6040801@alum.mit.edu> <4F35A769.7030802@alum.mit.edu> Message-ID: <4F39C982.2000207@mozilla.com> On 2/10/2012 3:25 PM, Niko Matsakis wrote: > Thinking more about this, I am not sure that option #3 has any point. It > is basically the same as option #2 but less expressive. It's a case of > "when I started playing with this idea, it was different, but I found > inconsistencies and over time it evolved to be the same thing as I > started with." Yeah, it seemed related.. Earlier last week we looked at the regions proposal[1], and I think most of us liked the sound of it, despite some reservations about where we draw the complexity line. Option #2 here sounds to me a closest to a subset of the regions proposal, maybe starting with a form where the first class references are all in the same region, and can't be assigned. Or something similarly restrictive. I'm most comfortable seeing you (or whoever wants to attack this) wander down that road for a while and see what comes out. Failing that, #1 won't be the end of the world. It just feels a bit like the wrong direction to me, longer-term. I guess if you have it working now ... hmm, how intrusive is it? -Graydon [1] https://github.com/mozilla/rust/wiki/Proposal-for-regions From andrew.pennebaker at gmail.com Tue Feb 14 08:30:55 2012 From: andrew.pennebaker at gmail.com (Andrew Pennebaker) Date: Tue, 14 Feb 2012 11:30:55 -0500 Subject: [rust-dev] Lisp (apply) / (curry) in Rust? Message-ID: I'm porting the QuickCheck unit test library to Rust, and I need to find a way to do Lisp's (apply) or (curry). Rust has a fantastic type system, but I'm not sure how to use templates to write quickcheck::for_all. In a dynamic language, I wouldn't have to worry about the types. Example: use rustcheck; import rustcheck::*; use std; fn prop_even(x : int) -> bool { ret x % 2 == 0; } fn gen_even() -> int { let i : int = rustcheck::gen_int(); if i % 2 == 0 { ret i; } else { ret i + 1; } } fn main() { for_all(prop_even, [gen_int]); for_all(prop_even, [gen_even]); } Desired Output: $ make $ ./example *** Failed! 37 +++ OK, passed 100 tests. In other words, for_all uses the passed in generators to create 100 random test cases for the prop_even function. Because gen_int returns an odd number 50% of the time, prop_even will quickly fail, and for_all will print the offending value (e.g. 37). Assuming gen_even really returns a random even integer, when for_all calls is 100 times, every value passed to prop_even will result in true, and so for_all reports that it passes 100 test cases. Again, the type system is the problem: it's clearly wonderful and declarative, like Haskell and Erlang, where QuickCheck originates, but I'm not sure how to properly wield Rust's type system to port over QuickCheck's for_all function. Cheers, Andrew Pennebaker www.yellosoft.us -------------- next part -------------- An HTML attachment was scrubbed... URL: From niko at alum.mit.edu Tue Feb 14 10:23:34 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Tue, 14 Feb 2012 10:23:34 -0800 Subject: [rust-dev] Lisp (apply) / (curry) in Rust? In-Reply-To: References: Message-ID: <4F3AA6A6.9090409@alum.mit.edu> On 2/14/12 8:30 AM, Andrew Pennebaker wrote: > Again, the type system is the problem: it's clearly wonderful and > declarative, like Haskell and Erlang, where QuickCheck originates, but > I'm not sure how to properly wield Rust's type system to port over > QuickCheck's for_all function. I just posted this on stackoverflow in response to your query there, but would this work for you: | fn for_all(test: fn(A) -> bool, generators: [fn() -> A]) -> bool { vec::all(generators) {|gen| test(gen()) } }| Niko From banderson at mozilla.com Tue Feb 14 14:18:55 2012 From: banderson at mozilla.com (Brian Anderson) Date: Tue, 14 Feb 2012 14:18:55 -0800 (PST) Subject: [rust-dev] Writing callbacks from C into Rust In-Reply-To: <1251289447.8214865.1329252069557.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Message-ID: <218702780.8298147.1329257935162.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Hey! We now have a long-desired mechanism for calling from C back into Rust functions. This should make writing bindings for certain libraries much easier. The short version is that we have a new type of function declaration, a 'crust' function ("C-to-Rust" - cute, or obnoxious?) that follows the C ABI and can be used as a callback from C functions. It looks like this crust fn cb(data: ctypes::uintptr_t) -> ctypes::uintptr_t { ... do whatever rusty stuff you like ... } These are special in several ways * They cannot be called from Rust code * Their value can be taken, but they always have type *u8 * A task in a crust callback that fails results in the runtime aborting abrubtly * Tasks cannot be killed while in a crust callback Their use should generally be encapsulated into libraries that can deal with their special nature. My general suggestion for using them successfully is: * Use them primarily to dispatch messages to other tasks to avoid catastrophic failure * If you are using them from an event loop then first put yourself into a new, single-threaded scheduler using spawn_sched to avoid blocking other tasks Additionally, the inability to kill tasks running in callbacks places an additional large burden on those using callbacks to handle long-running event loops (like libuv) - if the runtime is failing you have to ensure that your callbacks can detect this situation and exit cleanly. Otherwise your task will keep the runtime alive after all other tasks have died. My current suggestion is to have a monitor task (on a different scheduler) that detects failure and notifies the event loop callbacks via a message that it needs to terminate (using the non-blocking comm::peek function to look into the message queue). With libuv we could also inject a cleanup function into the event loop with the async_t handle. Anyway, here's an example of running the uv event loop using no C code: use std; // A selection from the uv API, reexported from the runtime with a rust_ prefix. #[link_name = "rustrt"] native mod uv { fn rust_uv_loop_new() -> *loop_t; fn rust_uv_loop_delete(loop: *loop_t); fn rust_uv_default_loop() -> *loop_t; fn rust_uv_run(loop: *loop_t) -> ctypes::c_int; fn rust_uv_unref(loop: *loop_t); fn rust_uv_idle_init(loop: *loop_t, idle: *idle_t) -> ctypes::c_int; fn rust_uv_idle_start(idle: *idle_t, cb: idle_cb) -> ctypes::c_int; } type opaque_cb = *u8; type handle_type = ctypes::enum; type close_cb = opaque_cb; type idle_cb = opaque_cb; // Redefine various uv structs. Most fields we don't care about type handle_private_fields = { a00: ctypes::c_int, a01: ctypes::c_int, a02: ctypes::c_int, a03: ctypes::c_int, a04: ctypes::c_int, a05: ctypes::c_int, a06: int, a07: int, a08: int, a09: int, a10: int, a11: int, a12: int }; type handle_fields = { loop: *loop_t, type_: handle_type, close_cb: close_cb, mutable data: *ctypes::void, private: handle_private_fields }; type handle_t = { fields: handle_fields }; type loop_t = int; type idle_t = { fields: handle_fields /* private: idle_private_fields */ }; fn handle_fields_new() -> handle_fields { { loop: ptr::null(), type_: 0u32, close_cb: ptr::null(), mutable data: ptr::null(), private: { a00: 0i32, a01: 0i32, a02: 0i32, a03: 0i32, a04: 0i32, a05: 0i32, a06: 0, a07: 0, a08: 0, a09: 0, a10: 0, a11: 0, a12: 0 } } } fn idle_new() -> idle_t { { fields: handle_fields_new() } } // The C ABI callback function. When this is called it will // automatically switch to the Rust stack, and we can run arbitrary // Rust code (but we must not fail!). crust fn idle_cb(handle: *ctypes::void, _status: ctypes::c_int) unsafe { std::io::println("I'm a-callin' back"); let idle: *idle_t = unsafe::reinterpret_cast(handle); // Drop the reference count of the event loop so // that it exits uv::rust_uv_unref((*idle).fields.loop); } fn main() unsafe { // Put ourselves into a new scheduler so that we // run the native event loop without blocking other tasks task::spawn_sched(1u) {|| let loop = uv::rust_uv_loop_new(); let h = idle_new(); uv::rust_uv_idle_init(loop, ptr::addr_of(h)); // Take the value of our callback function let callback = idle_cb; uv::rust_uv_idle_start(ptr::addr_of(h), callback); // Begin the event loop uv::rust_uv_run(loop); uv::rust_uv_loop_delete(loop); }; } From andrew.pennebaker at gmail.com Fri Feb 17 07:18:18 2012 From: andrew.pennebaker at gmail.com (Andrew Pennebaker) Date: Fri, 17 Feb 2012 10:18:18 -0500 Subject: [rust-dev] [rust] block syntax redundant (#1854) In-Reply-To: References: Message-ID: ssylvan, as a Lisper I would be in favor of having fn name(args) ... desugar to let name = fn(args) ... where fn is essentially lambda. Cheers, Andrew Pennebaker www.yellosoft.us On Thu, Feb 16, 2012 at 10:11 PM, ssylvan < reply+i-3256181-1278e5d081fb7d85e61536ab04aa4a1ed31f5d52-5316 at reply.github.com > wrote: > How about go the other way and make the normal function definition just be > a let statenent that binds a lambda of some sort? > > I really like the idea of unifying syntaxes though, but it's worth > considering changing the declaration to be more like the expression, > instead of going the other way. Also, it would be cool if there wasn't two > ways to declare a function (either by using special-case function > declaration syntax, or just creating a lambda, of some sort, and assining > it to a variable). > > Maybe get rid of special function declarations, and do something like: > > ``` > // just declare a variable that happens to be a function... > let foo = ( x : int ) -> int { > x+5 > }; > > // alternative form, do inference the other way, either would be legal > let foo2 : (int) -> int = x { > x+5 > }; > > // full syntax, all types explicit, no inference > let foo3 : (int)->int = (x : int) -> int { > x+5 > }; > ``` > > These all sort of look enough like function definitions, but are really > just declaring a variable like any other. > > --- > Reply to this email directly or view it on GitHub: > https://github.com/mozilla/rust/issues/1854#issuecomment-4014461 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From niko at alum.mit.edu Fri Feb 17 07:37:29 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Fri, 17 Feb 2012 07:37:29 -0800 Subject: [rust-dev] [rust] block syntax redundant (#1854) In-Reply-To: References: Message-ID: <4F3E7439.5000109@alum.mit.edu> On 2/17/12 7:18 AM, Andrew Pennebaker wrote: > ssylvan, as a Lisper I would be in favor of having fn name(args) ... > desugar to let name = fn(args) ... where fn is essentially lambda. I am not in favor of removing named fn items for several reasons. 1. I like that named functions do not inherit their environment. I find that for long functions I typically only want one or two variables from the environment, and inheriting more than that is inviting bugs, as those other variables are not meant to be used. This could admittedly be addressed by requiring explicit capture clauses. 2. In addition, the order of closure construction is very significant in Rust, particularly when you do things like move data into a closure. But declarations like "fn foo()" do not look like side-effecting statements, although they would be. Also, item decl order is generally insignificant in Rust, but this would no longer be the case. 3. Finally, the issue of mutually recursive functions would become quite annoying. As I understand it, this proposal would mean that functions inherit the names that appear before their declaration, making it hard to write a mutually recursive function declaration. If you decide to inherit all names from the parent, then it becomes quite challenging to know whether all variables that might be used have been initialized. You end up needing a `let rec` form or something like that. No doubt these issues can be addressed, but just making "fn foo(x,y)" sugar for "let foo = {|x, y| ...}" will not be very usable by itself (imo). Niko From zackcorr95 at gmail.com Sun Feb 19 01:42:52 2012 From: zackcorr95 at gmail.com (Zack Corr) Date: Sun, 19 Feb 2012 19:42:52 +1000 Subject: [rust-dev] Why exactly do we need low-level libuv bindings? Message-ID: The current target for std::uv are a set of low-level libuv bindings, closely mimicking the C API. Why exactly do they need to be low-level? A large majority of the std modules are already high level, it feels kind of odd to have a low level module in there. Wouldn't it be nicer (and just as powerful) to have it set out like below with interfaces: fn callback(handle: timer, status: int) { io::println("Done!"); } fn main() { let loop = uv::mk_loop(); let timer = uv::mk_timer(loop); timer.start(callback, 5000, 0); loop.run(); } vs. (based off the current state of uv.rs) crust fn callback(handle: ctypes::intptr_t, status: ctypes::c_int) { io::println("Done!"); } fn main() { let loop = uv::loop_new(); let timer = uv::timer_new(); uv::timer_init(ptr::addr_of(loop), ptr::addr_of(timer)); uv::timer_start(ptr::addr_of(timer), ptr::addr_of(callback), 5000, 0); uv::run(loop); uv::loop_destroy(loop); } So what exactly do we need low level bindings of libuv for? -------------- next part -------------- An HTML attachment was scrubbed... URL: From banderson at mozilla.com Sun Feb 19 14:21:06 2012 From: banderson at mozilla.com (Brian Anderson) Date: Sun, 19 Feb 2012 14:21:06 -0800 (PST) Subject: [rust-dev] Why exactly do we need low-level libuv bindings? In-Reply-To: Message-ID: <1220151951.11581599.1329690066545.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> ----- Original Message ----- > From: "Zack Corr" > To: Rust-dev at mozilla.org > Sent: Sunday, February 19, 2012 1:42:52 AM > Subject: [rust-dev] Why exactly do we need low-level libuv bindings? > > > The current target for std::uv are a set of low-level libuv bindings, > closely mimicking the C API. Why exactly do they need to be > low-level? We don't need them from an end user perspective, but the task of creating bindings to this complex API is difficult enough that, in my opinion, it would be cleaner and more maintainable to design it in two layers. Others have disagreed with me on this before though. From graydon at mozilla.com Sun Feb 19 16:17:30 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Sun, 19 Feb 2012 16:17:30 -0800 Subject: [rust-dev] Why exactly do we need low-level libuv bindings? In-Reply-To: <1220151951.11581599.1329690066545.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> References: <1220151951.11581599.1329690066545.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Message-ID: <4F41911A.3030905@mozilla.com> On 19/02/2012 2:21 PM, Brian Anderson wrote: > ----- Original Message ----- >> From: "Zack Corr" >> To: Rust-dev at mozilla.org >> Sent: Sunday, February 19, 2012 1:42:52 AM >> Subject: [rust-dev] Why exactly do we need low-level libuv bindings? >> >> >> The current target for std::uv are a set of low-level libuv bindings, >> closely mimicking the C API. Why exactly do they need to be >> low-level? > > We don't need them from an end user perspective, but the task of creating bindings to this complex API is difficult enough that, in my opinion, it would be cleaner and more maintainable to design it in two layers. Others have disagreed with me on this before though. I concur with Brian here. And I think there's a pretty explicit reason: C interfaces are semantically "thinner" than rust interfaces. A "good" (easy, friendly) user-facng rust interface typically takes and returns some combination of closures, interfaces, enums, vectors, polymorphic functions, etc. Whereas a C interface is typically just calls to static functions passing either scalar C types or raw pointers to buffers / opaque C structures. It's much, much easier to work with the C values when in rust than it is to work with the rust values when in C. That is, to write the code that "translates down" from the richer rust semantics to the sparser C semantics _in rust_ rather than on the C side of the FFI. You have to "make no mistakes" no matter which side you write it on (you can shoot yourself in the foot in C or in "unsafe rust" just as easily) but it happens to be a lot _easier_ to write the code that looks at the rust values when writing in rust. We have pattern matching and polymorphic dispatch and whatnot. So the easiest way to write a friendly library binding, it seems, is to write a minimal (as minimal as possible) binding to the C part, and then do a second layer, in rust, that does that translation-down from rust-friendly form. This is what we're going to be doing as we redo the libc / os interface. The one place this wasn't true, until recently, was in task activation and single-threaded callbacks. When C needed to do that, we couldn't model its need in rust, so had to push logic -- often far too much -- into C code. But Brian recently cracked that nut via the new explicit schedulers, so I *think* we're ok again (fingers crossed). -Graydon From zackcorr95 at gmail.com Sun Feb 19 17:42:49 2012 From: zackcorr95 at gmail.com (Zack Corr) Date: Mon, 20 Feb 2012 11:42:49 +1000 Subject: [rust-dev] Why exactly do we need low-level libuv bindings? In-Reply-To: <4F41911A.3030905@mozilla.com> References: <1220151951.11581599.1329690066545.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> <4F41911A.3030905@mozilla.com> Message-ID: On Mon, Feb 20, 2012 at 10:17 AM, Graydon Hoare wrote: > So the easiest way to write a friendly library binding, it seems, is to > write a minimal (as minimal as possible) binding to the C part, and then do > a second layer, in rust, that does that translation-down from rust-friendly > form. This is what we're going to be doing as we redo the libc / os > interface. > This makes sense. I would be willing to help on the libuv bindings front, but let me get this straight before I go any further. An ideal setup of the low-level/high-level system would look like this: std::uv - low-level bindings to the libuv C API. std::event/std::ev - high-level bindings to the event loop part (timers and such) of std::uv. std::fs - high-level bindings to fs functions of std::uv as well as other filesystem functions. std::net - high-level bindings to the networking parts of std::uv as well as other networking functions. Or am I misinterpreting how you would ideally like it set out? P.S. Incase anyone misunderstood, I wasn't challenging this decision, just trying to understand why exactly that way was best. P.S.S. I would suggest that std::net IPv6 parsing and formatting uses the libuv wrapper for this, because IPv6 addresses are very complicated and using a low-level function for it that handles the many shortcuts in the addresses would work better. -------------- next part -------------- An HTML attachment was scrubbed... URL: From banderson at mozilla.com Sun Feb 19 19:26:33 2012 From: banderson at mozilla.com (Brian Anderson) Date: Sun, 19 Feb 2012 19:26:33 -0800 (PST) Subject: [rust-dev] Why exactly do we need low-level libuv bindings? In-Reply-To: Message-ID: <1516229127.11607661.1329708393648.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> ----- Original Message ----- > From: "Zack Corr" > To: "Graydon Hoare" > Cc: Rust-dev at mozilla.org > Sent: Sunday, February 19, 2012 5:42:49 PM > Subject: Re: [rust-dev] Why exactly do we need low-level libuv bindings? > > This makes sense. I would be willing to help on the libuv bindings > front, but let me get this straight before I go any further. An > ideal setup of the low-level/high-level system would look like this: > > std::uv - low-level bindings to the libuv C API. > std::event/std::ev - high-level bindings to the event loop part > (timers and such) of std::uv. > std::fs - high-level bindings to fs functions of std::uv as well as > other filesystem functions. > std::net - high-level bindings to the networking parts of std::uv as > well as other networking functions. Possibly. I honestly haven't given much thought to what happens after we have the libuv bindings. As you suggest though, the libuv API itself is not very Rustlike, so maybe it does make sense to have another intermediate module, so std::uv::raw - Literally the C API exposed to Rust with all the struct redefinitions, etc. that entails std::uv - A friendlier API along the lines you suggested Beyond that possibly the module layout you suggest makes sense. It's still a ways off and there's a lot of experimenting to do yet. Things that I imagine we will want to build off of uv include: * async network I/O * syncronous network I/O - With our task model we will have the luxury of presenting a synchronous API that is built around the asynchronous API. It will block individual tasks, but not any threads. * async file I/O & sync file I/O * timers - This is honestly the thing I want the most right now. There is literally no way to sleep for a specified duration in Rust. * we will also want to structure things, I think, so that by default all these API's use some default event loop (which we have no mechanism to create yet), but can also be used with a supplied event loop. I've heard opposition to this idea though since it requires global state. Probably all of this stuff will be promoted to core eventually. fzzzy and I have been dabbling with libuv recently, and pfox has been trying to clean up the existing uv code this week. More hands would be welcome and I recommend stopping by IRC during work hours Pacific time to discuss plans. > P.S.S. I would suggest that std::net IPv6 parsing and formatting uses > the libuv wrapper for this, because IPv6 addresses are very > complicated and using a low-level function for it that handles the > many shortcuts in the addresses would work better. Agreed that anything that uv supplies we should attempt to reuse. From olson.jeffery at gmail.com Mon Feb 20 05:38:21 2012 From: olson.jeffery at gmail.com (Jeffery Olson) Date: Mon, 20 Feb 2012 05:38:21 -0800 Subject: [rust-dev] Why exactly do we need low-level libuv bindings? In-Reply-To: References: <1516229127.11607661.1329708393648.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Message-ID: (whoops. didn't go to list) > it would be cleaner and more maintainable to design it in two layers. Others have disagreed with me on this before though. My one gripe with this, and it was exposed in your email on c->rust mapping, is that you end up defining various libuv data structures in rust (including stuff defined in UV_HANDLE_PRIVATE_FIELDS), which strikes me as brittle, from a long-term maint. perspective. An elegant solution to address this doesn't really occur to me (header file/#define parsing in rustc?). Also, there are a number of unsafe operations we have to do in terms of managing data in the handles for their lifetime (kernel mallocs and frees). They'll be unsafe:: operations in rust, or the usual malloc/frees in C. There's no way around the pain, atm. So it isn't like hewing to rust gets us anything, here. In related news, my work on a proposal for libuv bindings is nearing a point where I can hopefully put forward a pull-request later today or tomorrow. It is by no means a complete mapping, but the patterns are in place ?and I'm reasonably happy with how it turned out. It's completely async, threadsafe, and retains the "character" of the C libuv api while leveraging the useful language opportunities that rust puts forward. Cheers, Jeff On Sun, Feb 19, 2012 at 7:26 PM, Brian ?Anderson wrote: > ----- Original Message ----- >> From: "Zack Corr" >> To: "Graydon Hoare" >> Cc: Rust-dev at mozilla.org >> Sent: Sunday, February 19, 2012 5:42:49 PM >> Subject: Re: [rust-dev] Why exactly do we need low-level libuv bindings? >> >> This makes sense. I would be willing to help on the libuv bindings >> front, but let me get this straight before I go any further. An >> ideal setup of the low-level/high-level system would look like this: >> >> std::uv - low-level bindings to the libuv C API. >> std::event/std::ev - high-level bindings to the event loop part >> (timers and such) of std::uv. >> std::fs - high-level bindings to fs functions of std::uv as well as >> other filesystem functions. >> std::net - high-level bindings to the networking parts of std::uv as >> well as other networking functions. > > Possibly. I honestly haven't given much thought to what happens after we have the libuv bindings. As you suggest though, the libuv API itself is not very Rustlike, so maybe it does make sense to have another intermediate module, so > > std::uv::raw - Literally the C API exposed to Rust with all the struct redefinitions, etc. that entails > std::uv - A friendlier API along the lines you suggested > > Beyond that possibly the module layout you suggest makes sense. It's still a ways off and there's a lot of experimenting to do yet. Things that I imagine we will want to build off of uv include: > > * async network I/O > * syncronous network I/O - With our task model we will have the luxury of presenting a synchronous API that is built around the asynchronous API. It will block individual tasks, but not any threads. > * async file I/O & sync file I/O > * timers - This is honestly the thing I want the most right now. There is literally no way to sleep for a specified duration in Rust. > * we will also want to structure things, I think, so that by default all these API's use some default event loop (which we have no mechanism to create yet), but can also be used with a supplied event loop. I've heard opposition to this idea though since it requires global state. > > Probably all of this stuff will be promoted to core eventually. > > fzzzy and I have been dabbling with libuv recently, and pfox has been trying to clean up the existing uv code this week. More hands would be welcome and I recommend stopping by IRC during work hours Pacific time to discuss plans. > >> P.S.S. I would suggest that std::net IPv6 parsing and formatting uses >> the libuv wrapper for this, because IPv6 addresses are very >> complicated and using a low-level function for it that handles the >> many shortcuts in the addresses would work better. > > Agreed that anything that uv supplies we should attempt to reuse. > _______________________________________________ > Rust-dev mailing list > Rust-dev at mozilla.org > https://mail.mozilla.org/listinfo/rust-dev From graydon at mozilla.com Mon Feb 20 08:25:21 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Mon, 20 Feb 2012 08:25:21 -0800 Subject: [rust-dev] Why exactly do we need low-level libuv bindings? In-Reply-To: References: <1516229127.11607661.1329708393648.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Message-ID: <4F4273F1.9000407@mozilla.com> On 20/02/2012 5:38 AM, Jeffery Olson wrote: > (whoops. didn't go to list) > >> it would be cleaner and more maintainable to design it in two layers. Others have disagreed with me on this before though. > > My one gripe with this, and it was exposed in your email on c->rust > mapping, is that you end up defining various libuv data structures in > rust (including stuff defined in UV_HANDLE_PRIVATE_FIELDS), which > strikes me as brittle, from a long-term maint. perspective. An elegant > solution to address this doesn't really occur to me (header > file/#define parsing in rustc?). Yeah. We're going to have to do something about that kind of muddle eventually. I suspect it'll involve the clang bindings erickt has been poking around with, possibly driven by a syntax extension or attribute plugin, when we support those. (Alternatively a bindings-generation tool. As usual, some tension between "compiler plugin" and "secondary tool". A design rule of thumb to differentiate the two has not yet become clear to me.) -Graydon From ted.horst at earthlink.net Mon Feb 20 11:44:26 2012 From: ted.horst at earthlink.net (Ted Horst) Date: Mon, 20 Feb 2012 13:44:26 -0600 Subject: [rust-dev] datetime module and some questions Message-ID: Hi, I started working on a datetime module that you can check out at: https://github.com/tedhorst/rust_datetime While working on this I had some questions about rust style or idioms. The first one was about how to implement what would be static arrays in C, e.g.: static int MONTHLENGTH[][13] = {{0, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31}, {0, 31, 29, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31}}; Rust doesn't support const vectors, so I ended up having to do this in a function using alts: fn month_length(m: u8, ly: bool) -> u8 { alt ly { true { alt check m { 1_u8 { 31_u8 } 2_u8 { 29_u8 } 3_u8 { 31_u8 } 4_u8 { 30_u8 } 5_u8 { 31_u8 } 6_u8 { 30_u8 } 7_u8 { 31_u8 } 8_u8 { 31_u8 } 9_u8 { 30_u8 } 10_u8 { 31_u8 } 11_u8 { 30_u8 } 12_u8 { 31_u8 } } } false { alt check m { 1_u8 { 31_u8 } 2_u8 { 28_u8 } 3_u8 { 31_u8 } 4_u8 { 30_u8 } 5_u8 { 31_u8 } 6_u8 { 30_u8 } 7_u8 { 31_u8 } 8_u8 { 31_u8 } 9_u8 { 30_u8 } 10_u8 { 31_u8 } 11_u8 { 30_u8 } 12_u8 { 31_u8 } } } } } Is there a better way to do this in rust? Ted From pwalton at mozilla.com Mon Feb 20 12:03:59 2012 From: pwalton at mozilla.com (Patrick Walton) Date: Mon, 20 Feb 2012 12:03:59 -0800 Subject: [rust-dev] datetime module and some questions In-Reply-To: References: Message-ID: <4F42A72F.4040104@mozilla.com> On 02/20/2012 11:44 AM, Ted Horst wrote: > Is there a better way to do this in rust? I think this is the sincerest form of a feature request for const vectors. :) There's a bug on it here: https://github.com/mozilla/rust/issues/571 Patrick From graydon at mozilla.com Mon Feb 20 18:21:38 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Mon, 20 Feb 2012 18:21:38 -0800 Subject: [rust-dev] Process and workflow updates Message-ID: <4F42FFB2.5060900@mozilla.com> Hi, We've been losing bugs and stepping on one another's toes a bit recently, so I spent last week doing bug triage and redid much of the bug tagging. The results are not exactly wonderful, but if we ever manage to squeeze github into supporting AND or NOT queries on tags (as one might expect) it should be a vaguely-useful scheme. Meanwhile it at least looks less like angry fruit salad, in terms of colour schemes. I've documented the new tagging scheme over in the "issue tracking" part of our development policy: https://github.com/mozilla/rust/wiki/Note-development-policy I've also written up the last round of "RFC policy" that we have been pushing around inside the core developer group at mozilla lately. The idea here is to minimize major changes that surprise people. So I've written up what we agreed to at the end of the last meeting: https://github.com/mozilla/rust/wiki/Note-RFC-process Several people have been slinging around emails with "RFC" in the subject line recently and/or tagging minor bugs as such. This policy should clarify when to use that term a bit more. Hopefully none of this affects anyone's workflow much beyond what one would expect by shipping programs and starting to get users. Changes get a bit more bureaucratic so there are fewer surprises. Film at 11. Thanks for your attention, back to work! -Graydon From zackcorr95 at gmail.com Mon Feb 20 22:56:01 2012 From: zackcorr95 at gmail.com (Zack Corr) Date: Tue, 21 Feb 2012 16:56:01 +1000 Subject: [rust-dev] Why exactly do we need low-level libuv bindings? In-Reply-To: References: <1516229127.11607661.1329708393648.JavaMail.root@zimbra1.shared.sjc1.mozilla.com> Message-ID: On Mon, Feb 20, 2012 at 11:38 PM, Jeffery Olson wrote: > In related news, my work on a proposal for libuv bindings is nearing a > point where I can hopefully put forward a pull-request later today or > tomorrow. It is by no means a complete mapping, but the patterns are > in place and I'm reasonably happy with how it turned out. It's > completely async, threadsafe, and retains the "character" of the C > libuv api while leveraging the useful language opportunities that rust > puts forward. This sounds promising. I think I'll wait for you to submit a pull request for this before I do anything with std::uv. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ted.horst at earthlink.net Tue Feb 21 06:47:04 2012 From: ted.horst at earthlink.net (Ted Horst) Date: Tue, 21 Feb 2012 08:47:04 -0600 Subject: [rust-dev] datetime module and some questions In-Reply-To: <4F42A72F.4040104@mozilla.com> References: <4F42A72F.4040104@mozilla.com> Message-ID: <8FEE5E1B-853D-4448-8F7A-B68DE81236B4@earthlink.net> Ok, so put me down as +1 on const vectors in particular and const expressions in general. My next question was about constructors. I want to have multiple impls of an face (say date as u32 number of days and date as a record of year, month, day). I want to have constructors so that I can check ranges or normalize values for these impls, but I don't see how to do that now. I think I need to wait for classes to be implemented to get constructors. Is that right? In the mean time, I am constructing dates like: let d = (0_u32 as date).from_str("2001-04-01"); That feels a little funny to me. Is there a more elegant way to do this? Ted On 2012-02-20, at 14:03, Patrick Walton wrote: > On 02/20/2012 11:44 AM, Ted Horst wrote: >> Is there a better way to do this in rust? > > I think this is the sincerest form of a feature request for const vectors. :) > > There's a bug on it here: https://github.com/mozilla/rust/issues/571 > > Patrick > _______________________________________________ > Rust-dev mailing list > Rust-dev at mozilla.org > https://mail.mozilla.org/listinfo/rust-dev From marijnh at gmail.com Tue Feb 21 07:08:38 2012 From: marijnh at gmail.com (Marijn Haverbeke) Date: Tue, 21 Feb 2012 16:08:38 +0100 Subject: [rust-dev] datetime module and some questions In-Reply-To: <8FEE5E1B-853D-4448-8F7A-B68DE81236B4@earthlink.net> References: <4F42A72F.4040104@mozilla.com> <8FEE5E1B-853D-4448-8F7A-B68DE81236B4@earthlink.net> Message-ID: > let d = (0_u32 as date).from_str("2001-04-01"); There's no reason to cast to an iface just to call a method on a value. 0_u32.from_str("...") should be equivalent. If you want to have different from_str constructor functions, though, just give them different names instead of using a kludge like this. From ted.horst at earthlink.net Tue Feb 21 07:55:24 2012 From: ted.horst at earthlink.net (Ted Horst) Date: Tue, 21 Feb 2012 09:55:24 -0600 Subject: [rust-dev] datetime module and some questions In-Reply-To: References: <4F42A72F.4040104@mozilla.com> <8FEE5E1B-853D-4448-8F7A-B68DE81236B4@earthlink.net> Message-ID: <9A12468F-4943-46A1-8C58-DE01FFB68452@earthlink.net> It wasn't the cast that felt funny to me, it was the role of 0_u32 in the expression. Its only purpose is to give access to the date impl over u32 which is ok, but I'm used to having something like class methods for that. Ted On 2012-02-21, at 09:08, Marijn Haverbeke wrote: >> let d = (0_u32 as date).from_str("2001-04-01"); > > There's no reason to cast to an iface just to call a method on a > value. 0_u32.from_str("...") should be equivalent. > > If you want to have different from_str constructor functions, though, > just give them different names instead of using a kludge like this. From marijnh at gmail.com Tue Feb 21 08:03:20 2012 From: marijnh at gmail.com (Marijn Haverbeke) Date: Tue, 21 Feb 2012 17:03:20 +0100 Subject: [rust-dev] datetime module and some questions In-Reply-To: <9A12468F-4943-46A1-8C58-DE01FFB68452@earthlink.net> References: <4F42A72F.4040104@mozilla.com> <8FEE5E1B-853D-4448-8F7A-B68DE81236B4@earthlink.net> <9A12468F-4943-46A1-8C58-DE01FFB68452@earthlink.net> Message-ID: > It wasn't the cast that felt funny to me, it was the role of 0_u32 in the expression. ?Its only purpose is to give access to the date impl over u32 which is ok, but I'm used to having something like class methods for that. Right. As I said, just use constructor functions with different names. We may end up supporting this directly in our class system at some point, but that seems an easy enough workaround. From ted.horst at earthlink.net Tue Feb 21 08:11:15 2012 From: ted.horst at earthlink.net (Ted Horst) Date: Tue, 21 Feb 2012 10:11:15 -0600 Subject: [rust-dev] datetime module and some questions In-Reply-To: References: <4F42A72F.4040104@mozilla.com> <8FEE5E1B-853D-4448-8F7A-B68DE81236B4@earthlink.net> <9A12468F-4943-46A1-8C58-DE01FFB68452@earthlink.net> Message-ID: <8AC3A617-AD52-4430-BE90-F0D0DD6C4EDB@earthlink.net> Just to be clear, do you mean a function outside the iface and impl? fn create_u32_date_from_str(ds: str) { ... } Ted On 2012-02-21, at 10:03, Marijn Haverbeke wrote: >> It wasn't the cast that felt funny to me, it was the role of 0_u32 in the expression. Its only purpose is to give access to the date impl over u32 which is ok, but I'm used to having something like class methods for that. > > Right. As I said, just use constructor functions with different names. > We may end up supporting this directly in our class system at some > point, but that seems an easy enough workaround. From marijnh at gmail.com Tue Feb 21 08:15:42 2012 From: marijnh at gmail.com (Marijn Haverbeke) Date: Tue, 21 Feb 2012 17:15:42 +0100 Subject: [rust-dev] datetime module and some questions In-Reply-To: <8AC3A617-AD52-4430-BE90-F0D0DD6C4EDB@earthlink.net> References: <4F42A72F.4040104@mozilla.com> <8FEE5E1B-853D-4448-8F7A-B68DE81236B4@earthlink.net> <9A12468F-4943-46A1-8C58-DE01FFB68452@earthlink.net> <8AC3A617-AD52-4430-BE90-F0D0DD6C4EDB@earthlink.net> Message-ID: > Just to be clear, do you mean a function outside the iface and impl? Yes, that's what I meant. From niko at alum.mit.edu Fri Feb 24 22:08:13 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Fri, 24 Feb 2012 22:08:13 -0800 Subject: [rust-dev] warning: ast serialization Message-ID: <4F487ACD.10208@alum.mit.edu> I am planning to push the first version of the Cross-Crate Inlining (CCI) work. The basic idea here is that functions marked `#[inline]` will be serialized into the crate metadata. When such a function is called, a local copy will be made in the current crate. That means that if the crate is later updated, dynamic linking will not pick up the new definition, so changes to functions marked as `#[inline]` may require recompilation for all users of a library. The feature works for simple examples but does not yet work on its intended target, things like `vec::iter()` (compilation crashes). I am sending this e-mail as warning because the current scheme uses an auto-generated module called `astencode_gen`. This is generated by a program called `serializer` which examines the rustc source. This scheme is, however, somewhat fragile: changes to the AST source require that `astencode_gen` be regenerated. This can be done by running the `src/etc/gen-astencode` script, as described below. It's annoying and I have a plan to convert the serializer into a syntax extension in the near future which should address these problems. For the time being, however, if you make changes to the AST, you generally have to follow a procedure like this: 1. Change the AST. 2. Run gen-astencode. Likely this script will fail to execute because rustc is not yet type-checking. It will, however, leave dummy versions of the various encoding functions in astencode_gen which simply fail when executed. 3. You can now fix the various typing errors and get the compiler to build. 4. Now, re-run gen-astencode. This will generate the real versions of the serialization functions. 5. Now, re-compile, and you will have a functioning compiler. Niko From niko at alum.mit.edu Wed Feb 29 07:40:30 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Wed, 29 Feb 2012 07:40:30 -0800 Subject: [rust-dev] a proposal for break, ret, and cont Message-ID: <4F4E46EE.4020609@alum.mit.edu> So, I've been thinking more about supporting "break", "ret", and "cont" in all loops (including those implemented in library code). I originally proposed in #1619 a system that supported "break" and "cont" based on enums. But I was thinking that we could get a bit more ambitious and support ret and friends without undue difficulty. In addition, we could get better syntactic sugar to boot. There is nothing amazing in this proposal, the main new part is to repurpose the `for` loop to support the various bits of syntactic sugar needed to make iteration easy to use for the end user. The basic return values would be: enum loop_ctl { lc_break, lc_cont, lc_ret(T) } enum may_return { mr_cont, mr_ret(T) } So you would write something like vec::iter as: fn iter(v: [T], f: fn(T) -> loop_ctl) -> may_return { let i = 0u, n = vec::len(v); while i < n { alt f(v[i]) { lc_break { break; } lc_ret(v) { ret mr_ret(v); } lc_cont { /* fallthrough */ } } } ret mr_cont; } The main problem now is that if users write: vec::iter(v) {|i| ret foo; } then they will ignore the return value, and so the ret will be lost. To address this, we can repurpose the `for` loop form (or use a macro), so that you can write: for i in v { ... } Here `v` can be any iterable thing, not just vectors. It's basically syntactic sugar for alt (v.iter { ...; cont; }) { mr_cont { /* fallthrough */ } mr_ret(v) { ret v; } } Note that the compiler adds an implicit "continue" command at the end of the loop body. In addition, any `ret`, `break`, or `cont` statements which appear inside the loop body are translated into returns out of the closure that represents the body. This should then work fine for nested loops as well. The main concern I can see would be Graydon's concern that writing all loops using an `alt` would impair performance or be a pain to use. I am not *so* concerned about it being a pain, a simple macro like: #macro[loop]{alt ... { lc_break { break; } lc_ret(v) { ret mr_ret(v); } lc_cont { /* fallthrough */ } }} would cover most cases, I should think. (I don't actually know our macro syntax.) The performance would be worth experimenting with: I imagine that inlining will help and I can imagine LLVM-level optimizations to convert break/cont into their ideal equivalents, but if they don't exist it'd take some effort to write them. Niko From niko at alum.mit.edu Wed Feb 29 12:58:49 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Wed, 29 Feb 2012 12:58:49 -0800 Subject: [rust-dev] a proposal for break, ret, and cont In-Reply-To: <4F4E46EE.4020609@alum.mit.edu> References: <4F4E46EE.4020609@alum.mit.edu> Message-ID: <4F4E9189.5000905@alum.mit.edu> Marijn suggested a simplification which I like. The types can be: enum loop_ctl = { lc_break, lc_cont, lc_ret }; enum may_return = { mr_cont, mr_ret }; In other words, the return value is not carried in the types. Instead, the `for` loop transform stores the return value in a local variable. In other words, something like: for i in v { ret ...; } becomes: { let _r_tmp = none; if v.iter({|i| _r_tmp = some(...); ret lc_ret; }) == mr_ret { ret option::unwrap(_r_tmp); } } Niko On 2/29/12 7:40 AM, Niko Matsakis wrote: > So, I've been thinking more about supporting "break", "ret", and > "cont" in all loops (including those implemented in library code). I > originally proposed in #1619 a system that supported "break" and > "cont" based on enums. But I was thinking that we could get a bit > more ambitious and support ret and friends without undue difficulty. > In addition, we could get better syntactic sugar to boot. There is > nothing amazing in this proposal, the main new part is to repurpose > the `for` loop to support the various bits of syntactic sugar needed > to make iteration easy to use for the end user. > > The basic return values would be: > > enum loop_ctl { lc_break, lc_cont, lc_ret(T) } > enum may_return { mr_cont, mr_ret(T) } > > So you would write something like vec::iter as: > > fn iter(v: [T], f: fn(T) -> loop_ctl) -> may_return { > let i = 0u, n = vec::len(v); > while i < n { > alt f(v[i]) { > lc_break { break; } > lc_ret(v) { ret mr_ret(v); } > lc_cont { /* fallthrough */ } > } > } > ret mr_cont; > } > > The main problem now is that if users write: > > vec::iter(v) {|i| ret foo; } > > then they will ignore the return value, and so the ret will be lost. > > To address this, we can repurpose the `for` loop form (or use a > macro), so that you can write: > > for i in v { ... } > > Here `v` can be any iterable thing, not just vectors. It's basically > syntactic sugar for > > alt (v.iter { ...; cont; }) { > mr_cont { /* fallthrough */ } > mr_ret(v) { ret v; } > } > > Note that the compiler adds an implicit "continue" command at the end > of the loop body. In addition, any `ret`, `break`, or `cont` > statements which appear inside the loop body are translated into > returns out of the closure that represents the body. This should then > work fine for nested loops as well. > > The main concern I can see would be Graydon's concern that writing all > loops using an `alt` would impair performance or be a pain to use. I > am not *so* concerned about it being a pain, a simple macro like: > > #macro[loop]{alt ... { > lc_break { break; } > lc_ret(v) { ret mr_ret(v); } > lc_cont { /* fallthrough */ } > }} > > would cover most cases, I should think. (I don't actually know our > macro syntax.) The performance would be worth experimenting with: I > imagine that inlining will help and I can imagine LLVM-level > optimizations to convert break/cont into their ideal equivalents, but > if they don't exist it'd take some effort to write them. > > Niko From marijnh at gmail.com Wed Feb 29 13:12:40 2012 From: marijnh at gmail.com (Marijn Haverbeke) Date: Wed, 29 Feb 2012 22:12:40 +0100 Subject: [rust-dev] a proposal for break, ret, and cont In-Reply-To: <4F4E9189.5000905@alum.mit.edu> References: <4F4E46EE.4020609@alum.mit.edu> <4F4E9189.5000905@alum.mit.edu> Message-ID: I was actually thinking there wouldn't have to be a special may_return type at all. The returned value can be an option, and return can store to that and then break. Code after the call to the iterator then checks for a `some` value in the return slot and, if found, returns it. From graydon at mozilla.com Wed Feb 29 14:01:12 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Wed, 29 Feb 2012 14:01:12 -0800 Subject: [rust-dev] a proposal for break, ret, and cont In-Reply-To: References: <4F4E46EE.4020609@alum.mit.edu> <4F4E9189.5000905@alum.mit.edu> Message-ID: <4F4EA028.9030900@mozilla.com> On 12-02-29 01:12 PM, Marijn Haverbeke wrote: > I was actually thinking there wouldn't have to be a special may_return > type at all. The returned value can be an option, and return can store > to that and then break. Code after the call to the iterator then > checks for a `some` value in the return slot and, if found, returns > it. This is coming into focus a little better. I like where it's going. If we make these two assumptions though: - Repurpose the 'for' loop to avoid boilerplate - Store-into-option to avoid a polymorphic tag Can we not, again, just get away with 'bool'? That is: for i in v(z) { ret q; } becomes: { let tmp = none; v.iter(z) {|i| tmp = some(q); ret false; } alt tmp { some(q) { ret q; } _ { } } } and the iter can be written pleasantly as: fn iter(self: [int], x: foo, f: fn(int) -> bool) { let i = 0; while f(i) { i += 1; } } Two questions: is this tolerable, and does it nest properly? I think it's tolerable, personally. I'd appreciate hearing other opinions on that. Meanwhile let's look at nesting... for i in a { for j in b { ret q; } } does not work if the nesting translates, naively, that is, if it becomes: let tmp1 = none; a.iter() {|i| let tmp2 = none; b.iter() {|j| tmp2 = some(q); ret false; } alt tmp2 { some(q) { ret q; } _ {} } } alt tmp { some(q) { ret q; } _ {} } This is a bad translation -- won't even compile -- since the inner block winds up with the wrong type. And anyways control flow doesn't propagate right. I think there's a right translation though. If it instead becomes: let tmp = none; a.iter() {|i| b.iter() {|J| tmp = some(q); ret false; } if tmp != none { ret false; } } alt tmp { some(q) { ret q; } _ {} } Then I think it's ok. IOW there's subtlety to the translation, it has to be nesting aware. In particular: - There's only _one_, outermost option to write to. All early rets write to it. - Any nested for loop has to inspect the outer option at its end, and ret false to break its enclosing loop, if the option changed to 'some'. I _think_ that's enough to make the translation work. Anyone see holes in it? I'm definitely interested in "getting this right", it's just somewhat ... subtle. -Graydon From graydon at mozilla.com Wed Feb 29 14:07:44 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Wed, 29 Feb 2012 14:07:44 -0800 Subject: [rust-dev] [rust] block syntax redundant (#1854) In-Reply-To: <4F3E7439.5000109@alum.mit.edu> References: <4F3E7439.5000109@alum.mit.edu> Message-ID: <4F4EA1B0.8020700@mozilla.com> On 12-02-17 07:37 AM, Niko Matsakis wrote: > On 2/17/12 7:18 AM, Andrew Pennebaker wrote: >> ssylvan, as a Lisper I would be in favor of having fn name(args) ... >> desugar to let name = fn(args) ... where fn is essentially lambda. > > I am not in favor of removing named fn items for several reasons. Agreed, for all reasons given. -Graydon From niko at alum.mit.edu Wed Feb 29 16:41:10 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Wed, 29 Feb 2012 16:41:10 -0800 Subject: [rust-dev] a proposal for break, ret, and cont In-Reply-To: <4F4EA028.9030900@mozilla.com> References: <4F4E46EE.4020609@alum.mit.edu> <4F4E9189.5000905@alum.mit.edu> <4F4EA028.9030900@mozilla.com> Message-ID: <4F4EC5A6.3040905@alum.mit.edu> This sounds fine. The only downside I can see to Marijn's technique is that there is no way for an iteration function to distinguish "broke out of the loop" from "returned out of the loop". This may or may not be important? I could imagine that the iterator may want to perform actions on a break which conceptually follow the loop but not perform those actions on a return, but I can't actually come up with any convincing examples. As far as nesting, I think you can do it in various ways, but as long as we're going to build for loops into the language it makes sense to do nesting the more efficient way that you describe: that is, the outer for loop creates a location to store to and inner for loops all refer to that same location. Just for reference, though, the naive translation of a nested loop pair works fine as long as you translate ret consistently---outside of a for loop, it returns from the method, inside it stores to the innermost temp and breaks. In that case, the inner for loop expands in the first round to: for i in a { let tmp2 = none; b.iter { |j| tmp2 = some(...); ret false; } // <-- this is a "native" return if tmp2 != none { ret option::unwrap(tmp2); // <-- this return must still be desugared } } now when we de-sugar the outer loop we get: let tmp = none; a.iter { |i| let tmp2 = none; b.iter { |j| tmp2 = some(...); ret false; } // <-- this is a "native" return if tmp2 != none { tmp = some(option::unwrap(tmp2)); ret false; // <-- this is a "native" return } } if tmp != none { ret option::unwrap(tmp); // <-- this is a native return // as there are no more for loops } Of course, here you end up with two temporaries where only one is needed so it makes more sense to be smart about it. But it's good to know that the translation does compose: I'd be worried otherwise. Niko On 2/29/12 2:01 PM, Graydon Hoare wrote: > On 12-02-29 01:12 PM, Marijn Haverbeke wrote: >> I was actually thinking there wouldn't have to be a special may_return >> type at all. The returned value can be an option, and return can store >> to that and then break. Code after the call to the iterator then >> checks for a `some` value in the return slot and, if found, returns >> it. > > This is coming into focus a little better. I like where it's going. If > we make these two assumptions though: > > - Repurpose the 'for' loop to avoid boilerplate > - Store-into-option to avoid a polymorphic tag > > Can we not, again, just get away with 'bool'? That is: > > for i in v(z) { ret q; } > > becomes: > > { > let tmp = none; > v.iter(z) {|i| tmp = some(q); ret false; } > alt tmp { > some(q) { ret q; } > _ { } > } > } > > and the iter can be written pleasantly as: > > fn iter(self: [int], x: foo, f: fn(int) -> bool) { > let i = 0; > while f(i) { i += 1; } > } > > Two questions: is this tolerable, and does it nest properly? > > I think it's tolerable, personally. I'd appreciate hearing other > opinions on that. > > Meanwhile let's look at nesting... > > for i in a { > for j in b { > ret q; > } > } > > does not work if the nesting translates, naively, that is, if it becomes: > > let tmp1 = none; > a.iter() {|i| > let tmp2 = none; > b.iter() {|j| > tmp2 = some(q); ret false; > } > alt tmp2 { > some(q) { ret q; } > _ {} > } > } > alt tmp { > some(q) { ret q; } > _ {} > } > > This is a bad translation -- won't even compile -- since the inner > block winds up with the wrong type. And anyways control flow doesn't > propagate right. I think there's a right translation though. If it > instead becomes: > > let tmp = none; > a.iter() {|i| > b.iter() {|J| > tmp = some(q); ret false; > } > if tmp != none { ret false; } > } > alt tmp { > some(q) { ret q; } > _ {} > } > > Then I think it's ok. IOW there's subtlety to the translation, it has > to be nesting aware. In particular: > > - There's only _one_, outermost option to write to. All early rets > write to it. > - Any nested for loop has to inspect the outer option at its end, > and ret false to break its enclosing loop, if the option changed to > 'some'. > > I _think_ that's enough to make the translation work. Anyone see > holes in it? I'm definitely interested in "getting this right", it's > just somewhat ... subtle. > > -Graydon From graydon at mozilla.com Wed Feb 29 16:47:10 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Wed, 29 Feb 2012 16:47:10 -0800 Subject: [rust-dev] a proposal for break, ret, and cont In-Reply-To: <4F4EC5A6.3040905@alum.mit.edu> References: <4F4E46EE.4020609@alum.mit.edu> <4F4E9189.5000905@alum.mit.edu> <4F4EA028.9030900@mozilla.com> <4F4EC5A6.3040905@alum.mit.edu> Message-ID: <4F4EC70E.5090808@mozilla.com> On 12-02-29 04:41 PM, Niko Matsakis wrote: > This sounds fine. The only downside I can see to Marijn's technique is > that there is no way for an iteration function to distinguish "broke out > of the loop" from "returned out of the loop". This may or may not be > important? I could imagine that the iterator may want to perform actions > on a break which conceptually follow the loop but not perform those > actions on a return, but I can't actually come up with any convincing > examples. Yeah, I can't either. And since we don't presently know, in other loops (while, vector-for), whether we exited them by loop-condition-completion or explicit-break, I can't see how this would deserve more-special consideration. If you have a Really Special Loop you can arrange a more subtle protocol between it and its iteratee. > Of course, here you end up with two temporaries where only one is needed > so it makes more sense to be smart about it. But it's good to know that > the translation does compose: I'd be worried otherwise. Oh, weird! I had somehow convinced myself that it doesn't. Um .. well, if you can convince yourself it does, I don't mind the "slightly less efficient" version. Compositionality in the translation is probably more important than saving an extra copy. So ... do we have a translation that actually gets us TCP-preservation on ret? If so that's _great_. We've been banging our heads against this and coming away disappointed since, er, possibly a year ago? -Graydon From niko at alum.mit.edu Wed Feb 29 17:11:23 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Wed, 29 Feb 2012 17:11:23 -0800 Subject: [rust-dev] a proposal for break, ret, and cont In-Reply-To: <4F4EC70E.5090808@mozilla.com> References: <4F4E46EE.4020609@alum.mit.edu> <4F4E9189.5000905@alum.mit.edu> <4F4EA028.9030900@mozilla.com> <4F4EC5A6.3040905@alum.mit.edu> <4F4EC70E.5090808@mozilla.com> Message-ID: <4F4ECCBB.80908@alum.mit.edu> On 2/29/12 4:47 PM, Graydon Hoare wrote: > Yeah, I can't either. And since we don't presently know, in other > loops (while, vector-for), whether we exited them by > loop-condition-completion or explicit-break, I can't see how this > would deserve more-special consideration. If you have a Really Special > Loop you can arrange a more subtle protocol between it and its iteratee. I agree, the simplicity of having the main iteration protocol be: fn(fn(T) -> bool) is very, very appealing. I particularly like that the translation will end up being pretty efficient. Even better, if there is no `ret` statement within a for loop, the compiler need not emit the temporary not any checks of the temporary, so there will be no runtime price paid. > So ... do we have a translation that actually gets us TCP-preservation > on ret? If so that's _great_. We've been banging our heads against > this and coming away disappointed since, er, possibly a year ago? Presuming (of course) that the iterator respects "break", and that the iterator didn't intend to perform any action after the iteration terminated, I think it's TCP-preserving. These seem like reasonable assumptions to me. Also, now more than ever, I think we should still prohibit explicit `break`, `cont`, and `ret` statements in sugared lambdas. That is, these examples should result in an error: v.iter {|i| ret; } spawn {|| ret; } but these would work: v.iter(fn&(i) { ret; }) spawn(fn~() { ret; }) `break` and `cont` already fail of course. This seems particularly important as it is likely to be confusing that `ret` works one way in a for loop and another in in a `v.iter {||...}` loop. I think though that `fn` lambdas don't look they are trying to be "normal syntax" but rather nested functions, so having `ret` return out of that nested function makes sense. While we're on the topic, what about changing the keyword `cont` to `next` (or `continue`, although it is long)? To be perfectly frank, I find the current name rather suggestive of another word that ought not to be uttered in polite company, particularly when spoken out loud. Kind of silly I know, but there it is. Niko From graydon at mozilla.com Wed Feb 29 17:42:26 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Wed, 29 Feb 2012 17:42:26 -0800 Subject: [rust-dev] a proposal for break, ret, and cont In-Reply-To: <4F4ECCBB.80908@alum.mit.edu> References: <4F4E46EE.4020609@alum.mit.edu> <4F4E9189.5000905@alum.mit.edu> <4F4EA028.9030900@mozilla.com> <4F4EC5A6.3040905@alum.mit.edu> <4F4EC70E.5090808@mozilla.com> <4F4ECCBB.80908@alum.mit.edu> Message-ID: <4F4ED402.2010008@mozilla.com> On 12-02-29 05:11 PM, Niko Matsakis wrote: > Also, now more than ever, I think we should still prohibit explicit > `break`, `cont`, and `ret` statements in sugared lambdas. Tricky. Plausible. I've less of an opinion on this, assuming we've moved a large number of blocks back to for-loop form. Might as prohibit, see how bad it hurts. > While we're on the topic, what about changing the keyword `cont` to > `next` (or `continue`, although it is long)? 'next' is ok with me. Need to collect up a summary of all these changes to a (.. the, existing?) RFC and let it stew a bit in the tracker, but I don't expect that to be too objectionable. -Graydon From niko at alum.mit.edu Wed Feb 29 17:45:10 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Wed, 29 Feb 2012 17:45:10 -0800 Subject: [rust-dev] a proposal for break, ret, and cont In-Reply-To: <4F4ECCBB.80908@alum.mit.edu> References: <4F4E46EE.4020609@alum.mit.edu> <4F4E9189.5000905@alum.mit.edu> <4F4EA028.9030900@mozilla.com> <4F4EC5A6.3040905@alum.mit.edu> <4F4EC70E.5090808@mozilla.com> <4F4ECCBB.80908@alum.mit.edu> Message-ID: <4F4ED4A6.7080606@alum.mit.edu> On 2/29/12 5:11 PM, Niko Matsakis wrote: > Presuming (of course) that the iterator respects "break", and that the > iterator didn't intend to perform any action after the iteration > terminated, I think it's TCP-preserving. These seem like reasonable > assumptions to me. I was thinking about this more. It's not entirely clear to me what TCP-preserving means in this context. The best I could come up with is this. If you had a while loop written like: while { } and you refactored this into an iterator: fn iterator(f: fn(T) -> bool) { while { if (!f()) ret; } } then doing: for i in iterator() { } would be the same. I believe this holds. Niko From graydon at mozilla.com Wed Feb 29 17:52:34 2012 From: graydon at mozilla.com (Graydon Hoare) Date: Wed, 29 Feb 2012 17:52:34 -0800 Subject: [rust-dev] a proposal for break, ret, and cont In-Reply-To: <4F4ED4A6.7080606@alum.mit.edu> References: <4F4E46EE.4020609@alum.mit.edu> <4F4E9189.5000905@alum.mit.edu> <4F4EA028.9030900@mozilla.com> <4F4EC5A6.3040905@alum.mit.edu> <4F4EC70E.5090808@mozilla.com> <4F4ECCBB.80908@alum.mit.edu> <4F4ED4A6.7080606@alum.mit.edu> Message-ID: <4F4ED662.2020809@mozilla.com> On 12-02-29 05:45 PM, Niko Matsakis wrote: > then doing: > > for i in iterator() { } > > would be the same. > > I believe this holds. That's what it means, yeah. "TCP-preserving-return" is a somewhat common gotcha in languages that have the temerity to mix closures and explicit flow control. http://gafter.blogspot.com/2006/08/tennents-correspondence-principle-and.html http://yehudakatz.com/ http://techscursion.com/2012/02/tennent-correspondence-principle.html etc. etc. -Graydon From niko at alum.mit.edu Wed Feb 29 19:06:49 2012 From: niko at alum.mit.edu (Niko Matsakis) Date: Wed, 29 Feb 2012 19:06:49 -0800 Subject: [rust-dev] a proposal for break, ret, and cont In-Reply-To: <4F4ED662.2020809@mozilla.com> References: <4F4E46EE.4020609@alum.mit.edu> <4F4E9189.5000905@alum.mit.edu> <4F4EA028.9030900@mozilla.com> <4F4EC5A6.3040905@alum.mit.edu> <4F4EC70E.5090808@mozilla.com> <4F4ECCBB.80908@alum.mit.edu> <4F4ED4A6.7080606@alum.mit.edu> <4F4ED662.2020809@mozilla.com> Message-ID: <4F4EE7C9.8020208@alum.mit.edu> On 2/29/12 5:52 PM, Graydon Hoare wrote: > That's what it means, yeah. "TCP-preserving-return" is a somewhat common > gotcha in languages that have the temerity to mix closures and explicit > flow control. Yes, I just mean that the way TCP is usually phrased is: "I can convert some expr X into (fn() { X })() and it has precisely the same semantics." Clearly this is not the case here. We (would) support a more limited variant where X is a kind of loop. Niko