<div dir="ltr"><div>I'm not sure it's possible to do that efficiently.<br></div><div><br></div><div>Here's a hypothetical situation: you have a stream with elements tagged as 1 and 2, but none tagged as 0. If somebody applies an operation to the stream of 0 elements (say stdoutLn), we have to process every single element—and perform every single effect—in the input stream before we know that the stream at index 0 is empty. In general, if we apply an operation to an element of one of the output streams, we'd have to process at minimum all the input elements up to and including that particular element. The important thing is that, semantically, the output of your demux operation is not n independent streams, but n views into a single stream.</div><div><br></div><div>It's probably possible to implement a version of this function that doesn't process the *entire* input stream up-front—it would just process as much of the input as it needed when you look at any given element in the output—but it probably needs a different type than just Vector to make it work correctly, and I'm not sure how to do that. More importantly, the behavior of this function would still be confusing; it might *look* like you have several distinct streams, but you'd have to do the effects and store the results of every single step in the input stream even if you only used one of your demuxed streams.</div><div><br></div><div>Does that explanation make sense? I haven't done much streaming stuff in a while, so I'm struggling a bit with how to express my intuitions about it.<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Apr 15, 2020 at 9:42 AM ☂Josh Chia (謝任中) <<a href="mailto:joshchia@gmail.com">joshchia@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I have a streaming package question.<div><br></div><div>Suppose I have a:</div><div><font face="monospace">Stream (Of (Int, a)) m ()</font></div><div><br></div><div>I would like to demultiplex it into <font face="monospace">Vector (Stream (Of a) m ())</font> using the <font face="monospace">Int</font> as the index of an item into the <font face="monospace">Vector</font> of output streams.</div><div><br></div><div>How can I do this efficiently (constant memory and linear time)?</div><div><br></div><div>Does the following work?</div><div><div style="color:rgb(147,161,161);background-color:rgb(0,24,30);font-family:"Droid Sans Mono",monospace,monospace,"Droid Sans Fallback";font-size:14px;line-height:19px;white-space:pre-wrap"><div><div style="line-height:19px"><div><div style="line-height:19px"><div><span style="color:rgb(133,153,0)">import</span> <span style="color:rgb(133,153,0)">qualified</span> Streaming.Prelude <span style="color:rgb(133,153,0)">as</span> SP</div><div><span style="color:rgb(133,153,0)"></span></div></div></div><div><span style="color:rgb(133,153,0)">import</span> <span style="color:rgb(133,153,0)">qualified</span> Data.Vector <span style="color:rgb(133,153,0)">as</span> V</div><div><span style="color:rgb(133,153,0)"></span></div></div></div><div><span style="color:rgb(133,153,0)"><br></span></div><div><span style="color:rgb(133,153,0)">type</span> <span style="font-weight:bold">StreamOf</span> <span style="color:rgb(38,139,210)">a</span> <span style="color:rgb(38,139,210)">m</span> <span style="color:rgb(38,139,210)">r</span> <span style="color:rgb(133,153,0)">=</span> <span style="font-weight:bold">Stream</span> (<span style="font-weight:bold">Of</span> <span style="color:rgb(38,139,210)">a</span>) <span style="color:rgb(38,139,210)">m</span> <span style="color:rgb(38,139,210)">r</span></div><br><div><span style="color:rgb(38,139,210)">demuxStream</span> <span style="color:rgb(133,153,0)">::</span> <span style="color:rgb(133,153,0)">forall</span> <span style="color:rgb(38,139,210)">a</span> <span style="color:rgb(38,139,210)">m</span>. <span style="font-weight:bold">MonadIO</span> <span style="color:rgb(38,139,210)">m</span></div><div> <span style="color:rgb(133,153,0)">=></span> <span style="font-weight:bold">Int</span> <span style="color:rgb(133,153,0)">-></span> <span style="font-weight:bold">StreamOf</span> (<span style="font-weight:bold">Int</span>, <span style="color:rgb(38,139,210)">a</span>) <span style="color:rgb(38,139,210)">m</span> () <span style="color:rgb(133,153,0)">-></span> <span style="color:rgb(38,139,210)">m</span> (<span style="font-weight:bold">Vector</span> (<span style="font-weight:bold">StreamOf</span> <span style="color:rgb(38,139,210)">a</span> <span style="color:rgb(38,139,210)">m</span> ()))</div><div>demuxStream numSyms stream <span style="color:rgb(133,153,0)">=</span></div><div> <span style="color:rgb(133,153,0)">let</span> emptyStreams <span style="color:rgb(133,153,0)">=</span> V<span style="color:rgb(133,153,0)">.</span>replicate numSyms (pure <span style="color:rgb(181,137,0)">()</span>)</div><div> processItem v (iD, x) <span style="color:rgb(133,153,0)">=</span> V<span style="color:rgb(133,153,0)">.</span>modify (<span style="color:rgb(133,153,0)">\</span>vm <span style="color:rgb(133,153,0)">-></span> VM<span style="color:rgb(133,153,0)">.</span>modify vm (<span style="color:rgb(133,153,0)">>></span> SP<span style="color:rgb(133,153,0)">.</span>yield x) iD) v</div><div> <span style="color:rgb(133,153,0)">in</span> SP<span style="color:rgb(133,153,0)">.</span>fold_ processItem emptyStreams id stream</div></div></div><div><br></div><div>My guess is that it takes more than constant memory as it goes through the entire input stream before returning.</div><div><br></div><div>Josh</div></div>
_______________________________________________<br>
Haskell-Cafe mailing list<br>
To (un)subscribe, modify options or view archives go to:<br>
<a href="http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe" rel="noreferrer" target="_blank">http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe</a><br>
Only members subscribed via the mailman list are allowed to post.</blockquote></div>