Sven Slootweg (soft-deprecated) @joepie91@pixie.town

@technomancy For what it's worth, for some reason this has never worked reliably cross-browser (many web frameworks had internal workarounds that emulated them for this reason) and I guess they just gave up trying to make it happen...

@amy Belated addition: the example of Validatem use that I linked could indeed be expressed in most static type systems, but Validatem is what I use everywhere by default, including many cases where it couldn't be; the list of validators can give an idea of that: https://validatem.cryto.net/modules/

There just aren't very many of these cases in Promistreams specifically, so it's hard to show that in context 😅

@Kaliah Same!

@jalcine I remember they were using it back in the dotcom boom. Don't recall exactly how they got it, wouldn't surprise me if it were an early acquisition.

I believe such domain names back then went for thousands rather than millions of dollars :)

@sofia@chaos.social Ah, that was a *different* batch of GPUs than the one I was thinking of. The one I was thinking of was when Bitcoin started being ASIC-mined and everyone was trying to get rid of their cards, it was all regular cards back then.

The second batch (when other cryptocurrencies started tanking) were indeed different, though they weren't exactly locked down, rather they didn't have display outputs and not all of them could be rerouted through another port in the system. Could still be used for general-purpose OpenCL compute though, not just cryptocurrency, just not for games.

I do know that even of these, a fair subset could be rerouted so that eg. the display ports of the on-board GPU on the mobo could be used with the silicon in the no-display-output cards, but this depended on both the card and the motherboard IIRC.

The others probably ended up getting reused for LLM stuff...

@sofia@chaos.social If it's anything like what happened with cryptocurrency, eBay will be flooded with cheap "barely used" (uhuh) GPUs all of a sudden, mostly specific models.

@amy (Response to your usecase at the bottom of this post, beyond the line separator, feel free to skip all the stuff above it if you're not in the mood for discussion about types/validation!)

To clarify on the inference point: I meant when not using TS yourself :) tsc has some very limited inference abilities on "just" JS, when it doesn't have explicit type annotations to go off anywhere, but it gets very easily confused, and doesn't infer anywhere near as well in that case as eg. Tern does.

I've found that it usually only takes 3 steps of indirection or so before it loses track of what types are produced by what, and it doesn't understand conditional cases at all (whereas eg. Tern would consider the union of possible types in its inference), and you end up with it just not knowing the types of anything. I've heard complaints that some JS libraries without explicit type definitions can cause this kind of 'type loss' even in TS codebases.

None of this happens if you're writing code in TS yourself AFAIK; as long as it regularly gets explicit type specifications, and everything inbetween also has type definitions, it seems to do fine.

On the validation: yep, it's runtime validation, the motivations are explained a bit more here, albeit in slightly more fiery language due to a number of unpleasant past discussions with other folks: https://www.npmjs.com/package/@validatem/core (though that's not required reading for using Promistreams). I just personally haven't found much value in static typing beyond that, and strong typing (ie. no type coercion) unfortunately isn't a thing in JS either way.

The specific example I linked makes use of `@validatem/wrap-value-as-option` to accept either a single value or an options object, and if it gets a single value, it turns it into an options object with that value as the `onRequest` property. That's because often you don't need to specify any custom teardown code for a source stream, so then you can just pass the "value-producing" callback directly to it and you're done.

--

Onto your actual usecase, if you just need to wrap a single socket at a time, then yeah, the experimental stuff is not needed; just a fromNodeStream.fromReadable(socket) and fromNodeStream.fromWritable(socket) should provide what you need, I think? And then the resulting streams can be used in a normal Promistreams pipeline, and it *should* now correctly handle the socket being closed on either end (by happy-aborting the other side). Please let me know if it doesn't.

The resulting sink stream should take Buffers, and the resulting source stream should produce Buffers, so then you could use a stream/pipeline-generating function like the third example in the introductory post to generate the same 'transform' (logger?) and use that on both ends. And then do whatever else needs to happen in the pipeline, basically. And perhaps pass them around bundled in an object if needed.

I'm not sure what the rest of your usecase looks like - if you're okay with sharing more detail (basically an end-to-end description of where the data comes from and where it ultimately goes to in what form), then I can probably provide a more concrete example of how that might be done in a Promistreams pipeline.

Every pipeline needs to be actively 'driven', so unless you're feeding all the received data immediately back into the sending side (after logging), some kind of other destination will need to be specified as a Promistream, using a Node stream wrapper or otherwise :)

@bananas Same for me with normal bread, but with this flatbread it ended up working great :)

Also just had pizzas, and that worked great as well! Not soggy at all.

@amy So the duplex streams, and specifically sockets, are a bit of a weird case - it *does* support these, but you need to explicitly specify in from-node-stream whether you want to wrap the readable or writable side (and so just have wrap the same duplex stream twice, once on either end, and it'll work fine).

I did look into duplex streams for Promistreams originally but concluded that they break almost every streams mechanic (or make it unreliable), and ultimately don't actually offer anything extra beyond what "an object containing a sink and a source stream as its properties" would, so that's the intended way to do duplex-like streams in Promistreams.

Also, I've actually been experimenting a lot lately with more ergonomic representation of network sockets in Promistreams, the idea being that a 'server' is just represented as a stream of clients, and a 'client' has a .receive and a .send property which are Promistreams representing the respective sides of the underlying stream.

Many of my recent changes to the low-level abstractions and the spec have actually been exactly to make this sort of thing work reliably! I'm not entirely happy with where it is yet (needs more testing), but a rough example of what that looks like is here: https://gist.github.com/joepie91/b583ce10f9dd63e3ad746c9a2a1f37e0?ts=4

(But you can just use from-node-stream today on a regular duplex socket and it'll just work fine, the example above is just experimentation beyond that!)

On the types point; I don't use TS, but I do generally use very strict argument validation on library surfaces (stricter than what TS can check in places), and with understandable error messages, so it'll tell you if anything invalid is passed in. Quite a few of the stream packages probably don't have this set up yet, but will in the future.

Here's an example of a stream package with relatively complex validation rules: https://git.cryto.net/promistream/simple-source/src/branch/master/index.js#L17-L23 (though most packages are single-argument).

This won't help type propagation in TS of course, at least until tsc improves its inference. I'd be open to someone contributing (maintenance for) TS definitions, but that's not really something I'd want to commit to myself, as I don't use it. I can understand if that's a reason not to use it!

Does that answer your points, or did I miss or misunderstand anything?

(As an aside, I wasn't sure if your problem description implied needing to insert streams inside of an existing unmodifiable Node.js streams pipeline - but if that is the case, that *will* be possible, I just have not gotten around to writing the to-node-stream module yet. Once I do, it will also support creating Duplex Node streams from two Promistreams.)