From justin at presidentbeef.com Wed Dec 2 07:01:23 2009 From: justin at presidentbeef.com (Justin Collins) Date: Wed Dec 2 07:01:34 2009 Subject: [Neko] Neko Tutorial Message-ID: <4B1602B3.1030307@presidentbeef.com> Hello all, A while ago I put together a little site with some information about getting started with Neko. I belatedly realized I probably should have just contributed to the Neko wiki, but by then I was about done with it. Anyhow, hopefully it can be helpful to a few people. Any suggestions, corrections, or contributions are welcome. http://www.nekotutorial.org/ -Justin From sdk at disktree.net Wed Dec 2 08:43:41 2009 From: sdk at disktree.net (tong) Date: Wed Dec 2 08:44:03 2009 Subject: [Neko] Neko Tutorial In-Reply-To: <4B1602B3.1030307@presidentbeef.com> References: <4B1602B3.1030307@presidentbeef.com> Message-ID: <1259739821.2755.7.camel@desktop> On Tue, 2009-12-01 at 22:01 -0800, Justin Collins wrote: > Hello all, > > A while ago I put together a little site with some information about > getting started with Neko. I belatedly realized I probably should have > just contributed to the Neko wiki, but by then I was about done with it. > > Anyhow, hopefully it can be helpful to a few people. Any suggestions, > corrections, or contributions are welcome. > > http://www.nekotutorial.org/ > > -Justin > hey, cool! +1 for merging withthe neko wiki ? /t. ? -- [) | 5 |< ? |2 3 3 From alex55miller at gmail.com Mon Dec 14 16:34:28 2009 From: alex55miller at gmail.com (Alex Miller) Date: Mon Dec 14 16:34:29 2009 Subject: [Neko] patches for upstream In-Reply-To: <585fd5b80912140730x3483f2ecqb23666a4d7f159ed@mail.gmail.com> References: <585fd5b80912140730x3483f2ecqb23666a4d7f159ed@mail.gmail.com> Message-ID: <585fd5b80912140734r28a273b3y78d94fc8bdaad8f3@mail.gmail.com> Skipped content of type multipart/alternative-------------- next part -------------- A non-text attachment was scrubbed... Name: neko-1.8.1-64bit.patch Type: application/x-download Size: 370 bytes Desc: not available Url : http://lists.motion-twin.com/pipermail/neko/attachments/20091214/a1575e12/neko-1.8.1-64bit.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: neko-1.8.1-mysql_fpic.patch Type: application/x-download Size: 444 bytes Desc: not available Url : http://lists.motion-twin.com/pipermail/neko/attachments/20091214/a1575e12/neko-1.8.1-mysql_fpic.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: neko-1.8.1-parallel.patch Type: application/x-download Size: 1176 bytes Desc: not available Url : http://lists.motion-twin.com/pipermail/neko/attachments/20091214/a1575e12/neko-1.8.1-parallel.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: neko-1.8.1-gentoo.patch Type: application/x-download Size: 3155 bytes Desc: not available Url : http://lists.motion-twin.com/pipermail/neko/attachments/20091214/a1575e12/neko-1.8.1-gentoo.bin From alex55miller at gmail.com Sun Dec 20 18:33:46 2009 From: alex55miller at gmail.com (Alex Miller) Date: Sun Dec 20 18:33:48 2009 Subject: [Neko] Re: patches for upstream In-Reply-To: <585fd5b80912140734r28a273b3y78d94fc8bdaad8f3@mail.gmail.com> References: <585fd5b80912140730x3483f2ecqb23666a4d7f159ed@mail.gmail.com> <585fd5b80912140734r28a273b3y78d94fc8bdaad8f3@mail.gmail.com> Message-ID: <585fd5b80912200933o2fbde41ex583dfdbb7779fbc@mail.gmail.com> This has a list of macros that are defined for amd64 platforms: http://predef.sourceforge.net/prearch.html#sec3 This would be the far better way to handle the enabling or disabling of the _64BITS in a platform independent way. On Mon, Dec 14, 2009 at 9:34 AM, Alex Miller wrote: > I've just finished making an ebuild for neko, and in the process, had to > patch a few things to get the install to work cleanly. Therefore, they're > getting sent upstream, and you're welcome to do whatever you'd like to with > them. > > neko-1.8.1-64bit.patch > From a package manager's perspective, it's not very easy to comment or > uncomment the 'CFLAGS += -D_64BITS' line. This patch uses the host's $CHOST > setting, (another option is to replace ${CHOST} with $(shell $(CC) > -dumpmachine) if you'd prefer not to rely on the env. variable) to check if > the host system is amd64 or not. > > neko-1.8.1-mysql_fpic.patch > In compiling the mysql module, you link to a static library with the > assumption that it's been compiled to be position independent. This is not > always the case... see mysql's bug about it. This changes the line so that > it will link against the .so version of the file, which is position > independent. > > neko-1.8.1-parallel.patch > You're missing a few dependencies in the makefile, so as it stands, you > must build neko with only one process running at a time, ie. make -j1. This > patch adds the missing dependencies (mainly just bin/neko and bin/std.ndll > when they're being used as part of a command) so that builds can be done in > parallel. > > neko-1.8.1-gentoo.patch > For the sake of completeness, I'm attaching this one also. It doesn't work > out well from a package manager's perspective if the program you're trying > to install waits for user's input. This removes the lines that ask where a > missing file can be found. > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.motion-twin.com/pipermail/neko/attachments/20091220/bce35781/attachment.htm From ryand-ruby at zenspider.com Wed Dec 23 11:42:36 2009 From: ryand-ruby at zenspider.com (Ryan Davis) Date: Wed Dec 23 11:42:39 2009 Subject: [Neko] Sanity check / suggestions Message-ID: I'm doing a proof of concept to see if nekovm would be a good target for a ruby backend (I didn't see anything over the last year in the ML archives to that effect). I'm getting wildly different times based on how I go about hand-translating. I'd love some hints/tips to try to see if this POC will pan out or not. The goal is for a pure-OO language like ruby to translate down to neko and be at least as performant as MRI ruby (which is admittedly rather slow, esp in method dispatch). Ruby, like neko, has immediates for small integers, but they're pure OO, so I was boxing them in neko so I could simulate actual method calls (tho I wasn't using call/apply yet). The performance was surprising to me (from 5x faster to ~18x slower than ruby), and I'm guessing it is mainly due to the amount of allocations I was doing. No real profiling data tho (how?). I'm sure in some places I'm just missing the point and could use a nudge. // fib.neko: puts = function(o) { $print(o); $print("\n") } // minimal boxing function O = function(v) { switch $typeof(v) { $tint => { var o = $new(Integer) o.value = v o } default => v }; } // Light Object "class", not fully wired up for a full OO system Object = $new(null) Object.superclass = null Object.to_s = function() "" // Simple Integer "class", not fully wired up for full OO system. can inherit Integer = $new(null) Integer.superclass = Object Integer.add = function(x, y) x + y Integer.__add = function(o) O($iadd(this.value, o.value)) Integer.__sub = function(o) O($isub(this.value, o.value)) Integer.__compare = function(o) $compare(this.value, o.value) Integer.to_s = function() $string(this.value) $objsetproto(Integer,Object) //////////////////////////////////////////////////////////// // In sorted order: // unboxed "OO" version, Directly calls Integer.add. t= 1.27s // totally incorrect translation, but speedy! fib = function(n) if n <= 2 1 else Integer.add(fib(n-1), fib(n-2)) // pure ruby version: t= 7.07s // un/box locally. will break if user overrides Integer#+/-. t= 16.44s fib2_4 = function(n) if n.value <= 2 O(1) else O(fib2(O(n.value - 1)).value + fib2(O(n.value - 2)).value) // strip down the comparison to primatives. cached literals t= 70.98s one = O(1) two = O(2) fib2_2 = function(n) if n.value <= 2 O(1) else fib2(n - one) + fib2(n - two) // Fully OO version, with cached literals t=100.52s // calls "+" and lets the system figure it out. fib2_1 = function(n) if n <= two one else fib2(n - one) + fib2(n - two) // don't use one/two globals, box everything. t=127.14s fib2_3 = function(n) if n <= O(2) O(1) else fib2(n - O(1)) + fib2(n - O(2)) fib2 = fib2_4 // fastest OO version puts(fib(10)) // => 55, yay! puts(fib2(O(10)).to_s()) // => 55, yay! iters = 100000 if true { // OO version var i = 0 var j = O(0) while i < iters { j += fib2(O(10)) i += 1 } puts(j.to_s()) } else { // unboxed "OO" version var i = 0 var j = 0 while i < iters { j += fib(10) i += 1 } puts(j) } From justin at presidentbeef.com Wed Dec 23 23:18:09 2009 From: justin at presidentbeef.com (Justin Collins) Date: Wed Dec 23 23:18:36 2009 Subject: [Neko] Sanity check / suggestions In-Reply-To: References: Message-ID: <4B329721.1050508@presidentbeef.com> Ryan Davis wrote: > I'm doing a proof of concept to see if nekovm would be a good target for a ruby backend (I didn't see anything over the last year in the ML archives to that effect). I'm getting wildly different times based on how I go about hand-translating. I'd love some hints/tips to try to see if this POC will pan out or not. > > The goal is for a pure-OO language like ruby to translate down to neko and be at least as performant as MRI ruby (which is admittedly rather slow, esp in method dispatch). Ruby, like neko, has immediates for small integers, but they're pure OO, so I was boxing them in neko so I could simulate actual method calls (tho I wasn't using call/apply yet). The performance was surprising to me (from 5x faster to ~18x slower than ruby), and I'm guessing it is mainly due to the amount of allocations I was doing. No real profiling data tho (how?). > A profiler is on my wish-list. You can use neko -stats, but it doesn't tell you about Neko function calls. Without commenting on the code, I don't personally see this as a viable thing to do. Ruby's object model and scoping rules do not fit the NekoVM very well, so you will have to build a lot of that yourself instead of being able to take advantage of built-in functionality. In my experience, the NekoVM seem to only be fast when you do things the way it expects. Of course, this may be my own fault, I don't know :) -Justin From geetha at angleritech.com Mon Dec 28 04:48:35 2009 From: geetha at angleritech.com (Geetha, ANGLER - EIT) Date: Mon Dec 28 05:00:41 2009 Subject: [Neko] installing neko on mac Message-ID: <16F1D498F4CD4545BB3C49972B5082714038FF@itech.angleritech.com> Hi all, I am new to neko. I want to build neko from source on Mac. Please help me how to do that. Thanks Sagi.. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.motion-twin.com/pipermail/neko/attachments/20091228/9cef7f40/attachment.htm From niel.drummond at grumpytoad.org Tue Dec 29 09:23:54 2009 From: niel.drummond at grumpytoad.org (Niel Drummond) Date: Tue Dec 29 09:24:13 2009 Subject: [Neko] installing neko on mac In-Reply-To: <16F1D498F4CD4545BB3C49972B5082714038FF@itech.angleritech.com> References: <16F1D498F4CD4545BB3C49972B5082714038FF@itech.angleritech.com> Message-ID: <4B39BC9A.7030504@grumpytoad.org> Geetha, ANGLER - EIT wrote: > Hi all, > > I am new to neko. I want to build neko from source on Mac. > > Please help me how to do that. > > Thanks > Sagi.. If you use darwinports (macports) to install the boehm gc garbage collector (boehmgc-devel), then you can download the neko source tarball and change the top of the makefile to read the following: @@@ ## CONFIG INSTALL_PREFIX = /usr/local CFLAGS = -Wall -O3 -fPIC -fomit-frame-pointer -I vm -I /opt/local/include -D_GNU_SOURCE EXTFLAGS = -pthread MAKESO = $(CC) -shared -dynamiclib -WBsymbolic LIBNEKO_NAME = libneko.so LIBNEKO_LIBS = -L/opt/local/lib -ldl -lgc -lm NEKOVM_FLAGS = -Lbin -lneko STD_NDLL_FLAGS = ${NEKOVM_FLAGS} INSTALL_FLAGS = NEKO_EXEC = DYLD_LIBRARY_PATH=../bin:${LD_LIBRARY_PATH} LD_LIBRARY_PATH=../bin:${LD_LIBRARY_PATH} NEKOPATH=../boot:../bin ../bin/neko @@@ this seems to compile on my OSX 10.5.8 - note though I still had to run nekoc using "LD_LIBRARY_PATH=/opt/local/lib nekoc" due to the non-standard location of the gc. - Niel From ryand-ruby at zenspider.com Tue Dec 29 11:48:32 2009 From: ryand-ruby at zenspider.com (Ryan Davis) Date: Tue Dec 29 11:48:36 2009 Subject: [Neko] Sanity check / suggestions In-Reply-To: <4B329721.1050508@presidentbeef.com> References: <4B329721.1050508@presidentbeef.com> Message-ID: On Dec 23, 2009, at 14:18 , Justin Collins wrote: > Ryan Davis wrote: >> I'm doing a proof of concept to see if nekovm would be a good target for a ruby backend (I didn't see anything over the last year in the ML archives to that effect). I'm getting wildly different times based on how I go about hand-translating. I'd love some hints/tips to try to see if this POC will pan out or not. >> >> The goal is for a pure-OO language like ruby to translate down to neko and be at least as performant as MRI ruby (which is admittedly rather slow, esp in method dispatch). Ruby, like neko, has immediates for small integers, but they're pure OO, so I was boxing them in neko so I could simulate actual method calls (tho I wasn't using call/apply yet). The performance was surprising to me (from 5x faster to ~18x slower than ruby), and I'm guessing it is mainly due to the amount of allocations I was doing. No real profiling data tho (how?). >> > > > A profiler is on my wish-list. You can use neko -stats, but it doesn't tell you about Neko function calls. > > Without commenting on the code, I don't personally see this as a viable thing to do. Ruby's object model and scoping rules do not fit the NekoVM very well, so you will have to build a lot of that yourself instead of being able to take advantage of built-in functionality. In my experience, the NekoVM seem to only be fast when you do things the way it expects. Of course, this may be my own fault, I don't know :) OK. Since it seems obvious that nobody is going to actually comment on the code itself... building it myself isn't that big of a deal. That's what compilers are for, right? That said, the rough (but not sanity checked) numbers I got seem to indicate that what you say has some merit; that leading too far off the beaten path of neko has some major penalty points in performance. I don't think it needs to be this way necessarily. I could have gone a more OO/VM approach and written a general purpose method dispatch that allowed for immediate fixnums (like ruby _and_ neko) so I could get the benefits of neko's built in immediates, but emulate a pure OO approach for them. I haven't gone that direction in neko yet so I don't know what the pros/cons are yet. Still not sure if I should bother at this stage. *shrug* From ncannasse at motion-twin.com Tue Dec 29 12:19:41 2009 From: ncannasse at motion-twin.com (Nicolas Cannasse) Date: Tue Dec 29 12:19:43 2009 Subject: [Neko] Sanity check / suggestions In-Reply-To: References: Message-ID: <4B39E5CD.8070506@motion-twin.com> Ryan Davis a ?crit : > I'm doing a proof of concept to see if nekovm would be a good target for a ruby backend (I didn't see anything over the last year in the ML archives to that effect). I'm getting wildly different times based on how I go about hand-translating. I'd love some hints/tips to try to see if this POC will pan out or not. > > The goal is for a pure-OO language like ruby to translate down to neko and be at least as performant as MRI ruby (which is admittedly rather slow, esp in method dispatch). Ruby, like neko, has immediates for small integers, but they're pure OO, so I was boxing them in neko so I could simulate actual method calls (tho I wasn't using call/apply yet). The performance was surprising to me (from 5x faster to ~18x slower than ruby), and I'm guessing it is mainly due to the amount of allocations I was doing. No real profiling data tho (how?). > > I'm sure in some places I'm just missing the point and could use a nudge. > > // fib.neko: > > puts = function(o) { $print(o); $print("\n") } > > // minimal boxing function > O = function(v) { > switch $typeof(v) { > $tint => { > var o = $new(Integer) > o.value = v > o Try instead a $new(IntegerProto) where IntegerProto only have "value = null" and its prototype set to "Integer", this should greatly reduce the amount of allocations since here you're copying the whole method table everytime. Nicolas From justin at presidentbeef.com Tue Dec 29 21:08:32 2009 From: justin at presidentbeef.com (Justin Collins) Date: Tue Dec 29 21:08:58 2009 Subject: [Neko] Sanity check / suggestions In-Reply-To: References: Message-ID: <4B3A61C0.7060007@presidentbeef.com> Ryan Davis wrote: > I'm doing a proof of concept to see if nekovm would be a good target for a ruby backend (I didn't see anything over the last year in the ML archives to that effect). I'm getting wildly different times based on how I go about hand-translating. I'd love some hints/tips to try to see if this POC will pan out or not. > > The goal is for a pure-OO language like ruby to translate down to neko and be at least as performant as MRI ruby (which is admittedly rather slow, esp in method dispatch). Ruby, like neko, has immediates for small integers, but they're pure OO, so I was boxing them in neko so I could simulate actual method calls (tho I wasn't using call/apply yet). The performance was surprising to me (from 5x faster to ~18x slower than ruby), and I'm guessing it is mainly due to the amount of allocations I was doing. No real profiling data tho (how?). > > I'm sure in some places I'm just missing the point and could use a nudge. > > // fib.neko: > > puts = function(o) { $print(o); $print("\n") } > > // minimal boxing function > O = function(v) { > switch $typeof(v) { > $tint => { > var o = $new(Integer) > o.value = v > o > } > default => > v > }; > } > > // Light Object "class", not fully wired up for a full OO system > Object = $new(null) > Object.superclass = null > Object.to_s = function() "" > > // Simple Integer "class", not fully wired up for full OO system. can inherit > Integer = $new(null) > Integer.superclass = Object > Integer.add = function(x, y) x + y > Integer.__add = function(o) O($iadd(this.value, o.value)) > Integer.__sub = function(o) O($isub(this.value, o.value)) > Integer.__compare = function(o) $compare(this.value, o.value) > Integer.to_s = function() $string(this.value) > $objsetproto(Integer,Object) > > //////////////////////////////////////////////////////////// > // In sorted order: > > // unboxed "OO" version, Directly calls Integer.add. t= 1.27s > // totally incorrect translation, but speedy! > fib = function(n) > if n <= 2 > 1 > else > Integer.add(fib(n-1), fib(n-2)) > > // pure ruby version: t= 7.07s > > // un/box locally. will break if user overrides Integer#+/-. t= 16.44s > fib2_4 = function(n) > if n.value <= 2 > O(1) > else > O(fib2(O(n.value - 1)).value + fib2(O(n.value - 2)).value) > > // strip down the comparison to primatives. cached literals t= 70.98s > one = O(1) > two = O(2) > fib2_2 = function(n) > if n.value <= 2 > O(1) > else > fib2(n - one) + fib2(n - two) > > // Fully OO version, with cached literals t=100.52s > // calls "+" and lets the system figure it out. > fib2_1 = function(n) > if n <= two > one > else > fib2(n - one) + fib2(n - two) > > // don't use one/two globals, box everything. t=127.14s > fib2_3 = function(n) > if n <= O(2) > O(1) > else > fib2(n - O(1)) + fib2(n - O(2)) > > fib2 = fib2_4 // fastest OO version > > puts(fib(10)) // => 55, yay! > puts(fib2(O(10)).to_s()) // => 55, yay! > > iters = 100000 > > if true { // OO version > var i = 0 > var j = O(0) > while i < iters { > j += fib2(O(10)) > i += 1 > } > puts(j.to_s()) > } else { // unboxed "OO" version > var i = 0 > var j = 0 > while i < iters { > j += fib(10) > i += 1 > } > puts(j) > } > > Just as a side note, you can do puts = function(o) { $print(o, "\n") } and $print will call $string() on anything passed in, which in turn calls .__string() on any arguments which implement it. Following Nicolas' suggestion, this change made fib2_4 twice as fast for me: O = function(v) { switch $typeof(v) { $tint => { var o = $new(null) $objsetproto(o, Integer) o.value = v o } default => v }; } Here is another implementation which makes fib2_3 twice as fast for me (~32s vs. ~14s). Unless I missed something, that actually puts it as fast as (or maybe slightly faster than) Ruby 1.8.7. //No changes to Object Integer = $new(null) Integer.cache = $hnew(10) //keep a cache of integers Integer.new = function(num) { var n; if((n = $hget(this.cache, num, null)) == null) { n = $new(null) $objsetproto(n, this) n.value = num $hset(this.cache, num, n, null) } n } Integer.superclass = Object Integer.add = function(x, y) x + y Integer.__add = function(o) Integer.new($iadd(this.value, o.value)) Integer.__sub = function(o) Integer.new($isub(this.value, o.value)) Integer.__compare = function(o) $compare(this.value, o.value) Integer.to_s = function() $string(this.value) $objsetproto(Integer,Object) iters = 100000 var i = 0 var j = Integer.new(0) while i < iters { j += fib2(Integer.new(10)) i += 1 } puts(j.to_s()) -Justin From ncannasse at motion-twin.com Wed Dec 30 10:13:21 2009 From: ncannasse at motion-twin.com (Nicolas Cannasse) Date: Wed Dec 30 10:13:24 2009 Subject: [Neko] Sanity check / suggestions In-Reply-To: <4B3A61C0.7060007@presidentbeef.com> References: <4B3A61C0.7060007@presidentbeef.com> Message-ID: <4B3B19B1.1040209@motion-twin.com> > Here is another implementation which makes fib2_3 twice as fast for me > (~32s vs. ~14s). Unless I missed something, that actually puts it as > fast as (or maybe slightly faster than) Ruby 1.8.7. > > //No changes to Object > > Integer = $new(null) > Integer.cache = $hnew(10) //keep a cache of integer [...] The issue here is that you will waste a lot of memory. What about my proposal, which was : IntegerProto = $new(null); IntegerProto.value = null; // allocate object slot $objsetproto(IntegerProto,Integer); Integer.new = function(num) { var o = $new(IntegerProto); o.value = num; o; } You could also maybe allocate a few static integers for small ones : iconsts = $amake(256); var i = 0; while( i < 256 ) { var o = $new(IntegerProto); o.value = i; iconsts[i] = i; } Integer.new = function(num) { var o = iconsts[num]; if( o == null ) { o = $new(IntegerProto); o.value = num; } o; } From justin at presidentbeef.com Wed Dec 30 10:43:36 2009 From: justin at presidentbeef.com (Justin Collins) Date: Wed Dec 30 10:43:44 2009 Subject: [Neko] Sanity check / suggestions In-Reply-To: <4B3B19B1.1040209@motion-twin.com> References: <4B3A61C0.7060007@presidentbeef.com> <4B3B19B1.1040209@motion-twin.com> Message-ID: <4B3B20C8.4000503@presidentbeef.com> Nicolas Cannasse wrote: >> Here is another implementation which makes fib2_3 twice as fast for >> me (~32s vs. ~14s). Unless I missed something, that actually puts it >> as fast as (or maybe slightly faster than) Ruby 1.8.7. >> >> //No changes to Object >> >> Integer = $new(null) >> Integer.cache = $hnew(10) //keep a cache of integer > [...] > > The issue here is that you will waste a lot of memory. You'd certainly need some limit or LRU policy for the cache, but it does speed things up in the tests I've used. There's definitely some trade-off there, though. It could save memory by having a single copy of each Integer used, and checking the cache seems to be faster than allocating a new object. On the other hand, there is some penalty for checking the cache, and those Integers will hang around forever and not be garbage collected, thus wasting memory. A more sophisticated cache management policy might not be worth the overhead, I don't know. > > What about my proposal, which was : > > IntegerProto = $new(null); > IntegerProto.value = null; // allocate object slot > $objsetproto(IntegerProto,Integer); > > Integer.new = function(num) { > var o = $new(IntegerProto); > o.value = num; > o; > } What is the advantage of having the intermediate IntegerProto object? Does allocating the value slot on it instead of the new object make a difference somehow? I'm curious. -Justin