--- Log opened Sat Dec 15 00:00:19 2012 01:34 < rvagg> Raynos: are you on a mac? 01:35 < Raynos> no 01:36 < rvagg> someone on a mac want to help me sort out LevelUP's compile? it'll be relatively painless 01:36 < rvagg> I'm just trying to remove ugly compile warnings 01:40 < CoverSlide> rvagg: j0 01:50 < jjjjohnnny> substack: whats that game tiling module u spoke of 01:54 < substack> jjjjohnnny: found it https://github.com/indutny/isodrive 01:55 < substack> http://indutny.github.com/isodrive/demo.html 01:55 < CoverSlide> hm crazy 01:55 < CoverSlide> i remember doing this in flash 01:55 < substack> it even fades out obstructing sprites 01:56 < CoverSlide> that's pretty awesome 01:58 < CoverSlide> I've been tossing around an idea in my head about a game drawing engine using multiple layers, and the canvases would be larger than the viewport, and moving around would just move the canvases via css so there wouldn't be a redraw for everything, only when the edge of that canvas is reached, and it would snap back to the opposite corner, and only redraw then 01:58 < CoverSlide> but haven't had the time to implement it 01:59 < CoverSlide> not sure if that would be better than the single canvas approach. 01:59 < CoverSlide> I think too often people think only in one canvas 02:12 < rvagg> CoverSlide: sorry, netsplit, did you get what I said about levelup and did you respond? 02:50 < substack> some of the bouncy tests now pass with http-raw! 02:54 < defunctzombie> \o/ 02:54 < defunctzombie> is http ready for some localtunnel action? 02:54 < defunctzombie> *http-raw 02:54 < substack> you can try it 02:55 < substack> http-raw is so simple that there's not much that could go wrong 02:55 < fotoverite> \o/ 02:55 < substack> but I haven't tested it very heavily just yet 02:55 < defunctzombie> kk 02:55 < defunctzombie> I will give it a go 02:55 < defunctzombie> it should simplify quite a bit of my code 02:55 < substack> the new bouncy will have (req, res, bounce) signature now too 02:56 < substack> since http-raw uses the http server already to generate the response object 02:56 < defunctzombie> what is the bounce param? 03:00 < substack> it does a net.connect() and does some convenient url-rewriting 03:01 < defunctzombie> does it net connect for every request? 03:01 < defunctzombie> or does it keep a pool of backend connections? 03:02 < substack> it doesn't keep a pool 03:02 < substack> if you want a pool you can build that yourself 03:02 < defunctzombie> k 03:02 < substack> if you want a pool it's much simpler to use http-raw directly 03:03 < defunctzombie> seems that way 04:21 < defunctzombie> substack: what happens when I write to req.createRawStream() ? 04:21 < defunctzombie> does that go back as a response? 04:21 < defunctzombie> basically the same as writing to the 'res' object? 04:24 < substack> it's not the same as writing to res 04:24 < substack> it writes to the raw socket 04:25 < substack> when you write to res, it gets framed 04:27 < defunctzombie> k 04:27 < defunctzombie> that is waht I want :) 04:34 < substack> ANTS 04:34 < mbalho> im hooking up a kinect to the ar drone 04:34 < mbalho> WOOT 04:35 < fotoverite> woot 04:35 < ik> woot 04:42 < defunctzombie> substack: server.js | 295 +++++++++++++++++++++----------------------------------------------------------- 04:42 < substack> \o/ 04:42 < defunctzombie> use of http-raw over the shit I had 04:43 < substack> what project is this for? 04:43 < substack> and does it work? 04:43 < defunctzombie> https://github.com/shtylman/localtunnel 04:43 < defunctzombie> yes, it works I just need to put back some queuing 04:43 < defunctzombie> and check that it works with websocket and some other edge cases 04:43 < defunctzombie> the basic tests I have seem to pass 04:43 < defunctzombie> but they are not super great and only check super simple functionality 04:44 < substack> neat stuff 04:46 < defunctzombie> I am using the parser to detect finished responses and that seems to be working pretty well 04:46 < defunctzombie> that is the equivalent to keepalive basically 05:03 < Raynos> defunctzombie: is that you? :D 05:03 < Raynos> shit it is 05:03 < Raynos> localtunnel is awesome! 05:03 < defunctzombie> Raynos: thank you :) 05:37 < Raynos> defunctzombie: https://github.com/shtylman/node-spaceport vs seaport 05:38 < defunctzombie> Raynos: no central server 05:38 < defunctzombie> thing avahi or zeroconf 05:38 < Raynos> how do you do no central server? 05:38 < defunctzombie> multicast 05:39 < defunctzombie> look into how zeroconf and avahi work 05:39 < defunctzombie> mDNS 05:39 < defunctzombie> is the underlying idea, I just didn't implement that particular spec 05:39 < Raynos> what is multicast? 05:39 < defunctzombie> but the idea is based off some of those same concepts 05:40 < defunctzombie> think of it like pub/sub except at the network level 05:40 < defunctzombie> http://en.wikipedia.org/wiki/Multicast 05:40 < Raynos> does it work across the internets? 05:40 < defunctzombie> no 05:40 < defunctzombie> it is for local networks 05:41 < substack> I don't like local networks 05:41 < Raynos> Oh 05:41 < Raynos> I see 05:41 < substack> the very idea of them 05:41 < defunctzombie> producers publish messages by sending them on specific ip/ports 05:41 < Raynos> that makes sense 05:41 < Raynos> thats useless :P 05:41 < defunctzombie> it is very useful 05:41 < Raynos> defunctzombie: make it work OVER THE INTERNETS 05:41 < defunctzombie> for certain use cases ;) 05:41 < defunctzombie> you can 05:41 < Raynos> how do I communicate across data centers? 05:41 < substack> yes the internet is kind of the only network we need 05:41 < defunctzombie> if you setup two endpoints 05:41 < Raynos> defunctzombie: so you set up two centralized end points? :D 05:42 < substack> if not for IPv4 scarcity we wouldn't even have this problem 05:42 < defunctzombie> Raynos: that or more 05:42 < Raynos> substack: what is "this problem" ? 05:42 < defunctzombie> not saying multicast itself is the golden solution 05:42 < defunctzombie> it has specific use cases 05:42 < substack> Raynos: having local networks 05:42 < Raynos> Oh 05:42 < defunctzombie> and the pub/sub is supported at the switch level 05:42 < Raynos> Because of IP4 05:42 < substack> the point of networks is that you can get from every node to every other node 05:42 < Raynos> we have local networks behind a single IP4 IP address? 05:42 < defunctzombie> and baked into the network stack 05:43 < Raynos> https://github.com/shtylman/node-spaceport/blob/master/lib/constants.js#L6 05:43 < Raynos> Whats that ip adfdress 05:43 < defunctzombie> I just picked it 05:43 < defunctzombie> it is a multicast address 05:43 < defunctzombie> a subset of ip addresses are special 05:43 < defunctzombie> reserved for multicast use 05:44 < Raynos> I dont understand this at all :D 05:44 < substack> who even has local networks anymore 05:44 < substack> aside from wifi routers 05:45 < defunctzombie> Raynos: you should :) networking is cool 05:45 < Raynos> i am kind of trying to build p2p networking infrastructure 05:45 < defunctzombie> substack: local networks are where you hide all your secrets :) 05:45 < defunctzombie> Raynos: if you are trying to build p2p then you should learn about some of this stuff ;) 05:45 < defunctzombie> maybe I should send you my networking textbook :p 05:45 < Raynos> no! 05:45 < defunctzombie> quite possibly the most useful class I took in college 05:46 < substack> defunctzombie: hide your secrets out in the open 05:46 < substack> behind strong crypto 05:46 < defunctzombie> stenography! 05:46 < substack> Raynos: ignore all this mdns noise 05:46 < substack> it's useless 05:47 < defunctzombie> it probably won't help whatever you are trying to build 05:47 < defunctzombie> it isn't useless 05:47 < defunctzombie> it has specific use cases 05:47 < defunctzombie> use cases where other things are not appropriate 05:47 < defunctzombie> or are not as important as you think 05:47 < defunctzombie> multicast is great if you have a data pipe just publishing info 05:48 < defunctzombie> and you don't care who is getting it 05:48 < defunctzombie> nor do you want to manage the connections 05:48 < defunctzombie> you spew info 05:48 < defunctzombie> and whomever wants it, gets it 05:48 < defunctzombie> anyhow... you should at least learn about udp, multicast, broadcast etc 05:48 < substack> but only if you've configured your local network to even support that 05:48 < substack> and only if all the nodes are on a local network in the first place 05:48 < substack> useless 05:49 < Raynos> substack: make secure-peer work in browsers! 05:49 < defunctzombie> substack: that is easy in many cases 05:49 < substack> I hope idiosyncratic infrastructure like that goes away. 05:49 < Raynos> https://github.com/substack/secure-peer#secure-peer ! 05:49 < substack> everything should just be universally routable 05:49 < defunctzombie> there are things I don't want exposed 05:49 < substack> and you should communicate explicitly with exactly who you want to communicate with 05:49 < substack> none of this implicit topology-based routing 05:49 < defunctzombie> na 05:50 < substack> use p2p algorithms instead 05:50 < defunctzombie> this is not about that stuff 05:50 < substack> for efficient broadcasting 05:50 < defunctzombie> there is a data publisher 05:50 < defunctzombie> no 05:50 < defunctzombie> doesn't work 05:50 < defunctzombie> not when you have high amounts of data 05:50 < defunctzombie> and consumers should receive it at the same time 05:50 < substack> Raynos: crypto-browserify just needs wrappers for rc4 and diffie-hellman 05:50 < substack> then it might start just working 05:50 < defunctzombie> all of this is way easy to do anyway 05:51 < Raynos> substack: how do you get private keys in browsers :D 05:51 < defunctzombie> anyhow.. not here to convince anyone of it :) 05:51 < defunctzombie> I use it, it works 05:51 < defunctzombie> it isn't for everyone or everything 05:51 < substack> defunctzombie: it only works when all the nodes happen to be on the same local network, which is a sufficiently rare condition to be pretty useless 05:51 < defunctzombie> and if you don't know what multicast or broadcast is then you are missing out 05:51 < defunctzombie> substack: it is a sufficiently normal condition for many things I do 05:52 < defunctzombie> and if they are not I can easily udp tunnel 05:52 < defunctzombie> trivially 05:52 < defunctzombie> my nodes don't happen to be on the same local network... I put them on the same local network because that is easy to do 05:53 < Raynos> defunctzombie: does it work over webrtc :D 05:53 < defunctzombie> I know nothing about webrtc 05:53 < defunctzombie> multicast has been around for forever so that I do know something about :) 05:53 < defunctzombie> webrtc sounds tcp to me tho 05:53 < defunctzombie> maybe not 05:53 < defunctzombie> none of this is tcp 05:54 < Raynos> webrtc allows for udp connection 05:54 < defunctzombie> tcp is a bad solution to certain packet scenarios 05:54 < defunctzombie> cool 05:54 < Raynos> but you cant just send things to ports or ip addresses 05:54 < defunctzombie> not too surprising 05:55 < defunctzombie> again, multicast is not designed for WAN 05:55 < defunctzombie> so that doesn't shock me 05:59 < substack> multicast tightly couples the layer 3 addressing with the layer 1 physical topologies 05:59 < substack> for something that should be a layer 7 concern 06:01 < defunctzombie> what is your solution for what multicast provides then? 06:02 < defunctzombie> it doesn't so much as couple them as provide a way to do something with the support of the network that is already in place 06:03 < substack> at the webrtc meetup http://peer5.com/ presented an efficient peer-to-peer approach for distributed latency-critical transmission 06:03 < defunctzombie> LAN is a thing and I don't believe everything should be WAN 06:04 < defunctzombie> substack: if this is routing traffic through peers before reaching destination that is a no go 06:04 < defunctzombie> presents different delivery times to different peers 06:04 < defunctzombie> I think peer5 is a cool solution 06:04 < defunctzombie> to a different problem 06:05 < Raynos> defunctzombie: you cant do that 06:05 < Raynos> you cant garantuee real time networking 06:06 < defunctzombie> check out the HFT space and then tell me again what you think they can or can't guarantee ;) 06:06 < defunctzombie> and yes, I udnerstand what you mean by not guaranteeing real time networking 06:07 < defunctzombie> however, routing through peers also introduces different latencies 06:07 < defunctzombie> and load on peers which you may not wish to have 06:07 < defunctzombie> or other peers may not wish to expose themselves to 06:08 < defunctzombie> in the case of spaceport, how would I do p2p? 06:08 < defunctzombie> lets say I don't use multicast, what do I do then to bootstrap? 06:12 < dominictarr> the internet is build with the _assumption_ that you can't guarantee network performance. 06:13 < dominictarr> if you can pay to dig a really straight ditch between Chicago and New York, or pay for co-localtion (i.e. control everthing) then you can. 06:16 < defunctzombie> dominictarr: yep 06:16 < defunctzombie> that is exactly what people do 06:16 < defunctzombie> seriously :) 06:25 < dominictarr> yeah, I read about it! 06:26 < dominictarr> Raynos, substack : defunctzombie is right about multicast. we need to be aware of multicast 06:27 < dominictarr> if we are gonna imagine a hyper networked future where every-fucking-thing has a computer and wifi in it, then we'll probably need wifi 06:27 < dominictarr> the problem is that TCP creates a telephone metaphone (with a CONNECTION) 06:28 < dominictarr> but that only really relates to the physical reality if you have a series of cables between those two places 06:28 < substack> it just seems really narrow in its applicability 06:28 < substack> I can't see myself ever needing something like mdns 06:28 < dominictarr> yes maybe 06:28 < substack> and I see plenty of people reaching for mdns when it doesn't make any sense 06:29 < dominictarr> I'm not saying use mdns 06:29 < dominictarr> the problem with TCP is when you put it over radio 06:30 < dominictarr> this is why wifi doesn't scale 06:31 < dominictarr> you only have a small number of channels (4 on .bgn 2.4 GHz wifi - check your iwconfig to see what you are on) 06:32 < dominictarr> the thing is: radio is inherently multicast 06:33 < dominictarr> you can mung telephones through radio, but it doesn't scale. that stuff always breaks when you get enough people using it all the same time 06:35 < dominictarr> if you can use a radio metaphor instead, then perhaps you can make that work. I don't know though. 06:36 < dominictarr> I don't know enough data structures/patterns that are designed for multicast. 06:36 < dominictarr> but they are out there, I poked around a little the other day. 06:43 < defunctzombie> dominictarr: multicast is not a super widely known thing so not as much out there for it 06:44 < defunctzombie> it also has limited applicability given that the network has to support it 06:44 < dominictarr> that makes it the perfect academic problem! 06:44 < defunctzombie> haha 06:44 < dominictarr> you can work on interesting problems with no one concerned about the real world bothering you! 06:46 < dominictarr> http://www.youtube.com/watch?v=GwMr8Xl7JMQ 06:46 < dominictarr> ^ how governments attempt to sensor tor 06:46 < defunctzombie> nice.. I gotta watch this 06:49 < Raynos> watching that too 06:49 < Raynos> dominictarr: randpeer videos dissappeared :( 06:49 < dominictarr> IT'S A CONSIPRACY! 06:49 < LOUDBOT> SWEATING IS THE KEY TO SUCCESS 06:50 < defunctzombie> wow 06:59 < dominictarr> Raynos, here it is! http://www.youtube.com/watch?v=MNzMNjW8oGs 07:00 < Raynos> nice! 07:00 < Raynos> how did you find it 07:04 < dominictarr> I searched for the name, and 22C3 07:04 < dominictarr> (the name of the talk) 07:24 < substack> haha https://github.com/thlorenz/browserify-ftw 07:32 < defunctzombie> substack: wow 07:32 < defunctzombie> that is awesome 07:35 < substack> one of domenic's co-workers 07:35 < substack> he's at lab49 now 07:35 < substack> infecting them with browserify no doubt! 07:37 < defunctzombie> excellent... mwahhaha 07:44 < Raynos> thlorenz is pretty cool 07:55 < substack> oh goodness 07:56 < substack> I ran into the http.globalAgent.maxSockets issue again 07:56 < substack> this time in a test 07:56 < substack> why is that still a thing 07:56 < defunctzombie> hahaha 07:56 < defunctzombie> the sad thing is that noone changes that 07:56 < substack> pooling is pretty much never what you actually want 07:56 < defunctzombie> when tehy can easily handle more 07:56 < substack> I just set it to Number.MAX_VALUE 07:57 < defunctzombie> haha 07:57 < substack> because fuck connection pooling 07:57 < substack> it just causes bugs 07:57 < substack> correctness > performance 07:57 < Raynos> That TOR stuff 07:57 < Raynos> is actually moving. 07:58 < Raynos> wait how does maxSockets work? 07:58 < Raynos> and why is it pooling? 07:58 < defunctzombie> the tor stuff is awesome 07:58 < defunctzombie> Raynos: it intercepts http requests 07:59 < defunctzombie> outgoing 07:59 < defunctzombie> based on host iirc 07:59 < substack> Raynos: when you make outgoing http connections with the http module the connections get transparently pooled to work with keep-alive 07:59 < substack> but that is the worst possible thing for them to do 07:59 < substack> because if you try to make too many outgoing http connections at once they will just hang inexplicably 07:59 < substack> until enough of them drop off from timeouts 07:59 < substack> it's really really shit if you have long-lived http connections 08:00 < substack> because you'll fill up the pool and then be completely fucked until you figure out why none of your new connections are going through 08:00 < Raynos> oh 08:00 < Raynos> fuck that 08:00 < Raynos> its a restriction 08:00 < Raynos> wtf 08:00 < defunctzombie> everyone needs to see this tor video 08:38 < Raynos> :/ 08:39 < substack> defunctzombie: http-raw still needs upgrade support 08:39 < substack> adding that now 08:39 < substack> so in 'upgrade' handlers you can do req.createRawStream() 08:39 < defunctzombie> coo 08:39 < defunctzombie> and I can detect the upgrade request? 08:42 < defunctzombie> I will aim to finish the localtunnel refactoring this weekend 08:43 < dominictarr> substack, juliangruber Raynos do you guys ever use the 'sync' event on scuttlbutt? 08:43 < Raynos> not enough that you cant break back compat 08:45 < dominictarr> yeah, and I want to change that so it works properly 08:49 < Raynos> anyone know how to gaurd against XSS. I realise I use innerHTML :( 08:50 < dominictarr> convert < to > or whatever 08:51 < dominictarr> substack, we need robots that just want to make music like this http://www.youtube.com/watch?feature=endscreen&v=bihaL1risM0&NR=1 08:53 < substack> I don't use 'sync' no 08:56 < dominictarr> okay good. 08:57 < dominictarr> so, the API will be that the stream will emit 'sync' when it starts tailing. 08:57 < dominictarr> i.e. now the changes are realtime, not historical. 08:57 < dominictarr> it's emits on the stream, not the scuttlebutt instance, because it's only with respect to that connection 08:58 < Raynos> EVERY PROTOCOL SHOULD BE VERSIONED 08:58 < LOUDBOT> POINT THAT FUCKING CANNON AT THE POLITICIANS 09:21 < substack> defunctzombie_zz: ok http-raw@0.1.0 now inserts req.createRaw{,Body}Stream() for upgrade events too 10:26 < substack> ok http-raw@0.2.0 now has tls support 10:37 < substack> all tests now passing on the latest bouncy using http-raw 10:38 < substack> now updating the api 10:39 < substack> actually meh 10:40 < substack> just added a cb.length === 3 check instead to send the response 11:26 < substack> bouncy@2.0.0 now out 11:28 < substack> uses http-raw 11:29 < substack> api is mostly the same except for res.respond() went away 11:29 < substack> now you can just do bouncy(function (req, res, bounce) {}) 19:57 < defunctzombie> man...why the hell don't I get subscription objects from EventEmitter.on 19:57 < defunctzombie> makes it super annoying to disconnect events 20:16 < chrisdickinson> Raynos: protect against xss? in what context/module? 20:23 < defunctzombie> substack: req.upgrade is set to true immediately after calling req.createRawStream 20:23 < defunctzombie> so before calling that it is false 20:24 < defunctzombie> and after calling that req.upgrade always returns true 20:28 < gildean> substack: btw. are the raw http headers accessible when using the new bouncy? 20:29 < gildean> as in, if i wanted to create a simple proxy but also print out the raw data out somewhere that's flowing through the proxy? 20:33 < defunctzombie> substack: also, what happens if I call createRawStream twice? 20:42 < defunctzombie> you need to set your editor to remove trailing spaces :p 20:49 < defunctzombie> substack: I have an http-raw fix 20:49 < defunctzombie> if you call .pause() 20:50 < defunctzombie> it won't emit the saved buffers right away 20:50 < defunctzombie> do you want this in http-raw? 20:50 < defunctzombie> seems like a useful thing 20:58 < Raynos> chrisdickinson: say I cache templates in local db 20:58 < Raynos> and use innerHTML to turn them into DOM elements 20:58 < Raynos> how do I protect againsrt compromised templates in local db 21:07 < chrisdickinson> Raynos: so, ideally you escape text contents that are inserted into the html using document.body.createElement('div').appendChild(createTextNode("text")).innerHTML (or a regex) 21:07 < chrisdickinson> innerText unfortunately does not work until fairly recent firefox versions 21:09 < Raynos> Well no 21:09 < Raynos> I have a template thats html 21:09 < Raynos> I want to verify that my html template 21:09 < Raynos> is correct 21:09 < Raynos> I guess I can check for script nodes in the create element 21:10 < chrisdickinson> so, in that case, i'd use querySelector with a list of inputs that may not have any non-textnode children 21:11 < chrisdickinson> you could generate that list by querying for "[data-safe]" for nodes that should escape their children 21:11 < chrisdickinson> (eep, i mean "data-escaped") 21:11 < chrisdickinson> and it might be easier to just query for "[data-escaped]" and assert that all of those nodes matched only contain textnode children. 21:12 < chrisdickinson> since