[Haskell-beginners] R/W from/to partitions (on linux)

[Silent Leaf]

btw i just tried my comparison of lists, and it seems there's an
intermediate type holding the concept of boundaries and so on, aka it does
not need to compute them beyond comparing their boundaries. clever haskell!
a bit too much, you never know how smart it exactly is...

2017-06-26 13:33 GMT+02:00 Silent Leaf <silent.leaf0 at gmail.com>:

> i'm trying to get for which purpose BS.strict and BS.lazy are adapted. am
> i right to think that:
>
> lazy bytestrings are not optimal when needing to read huge files of
> several gigabytes, bc they are only lazy to the extent that the very long
> bytestring is not read to the end. aka it's like haskell's natural
> laziness, if i compute [0..2^20] == [0..2^20] it won't just take a lot of
> time but will also (try to) keep in memory both huge lists, aka the
> operation is not intrinsically on minimal resources, comparing each item of
> each list and discarding the previous ones at the same time; more generally
> the lists or bytestrings, be they lazy, are still only deallocated when the
> whole variable (the whole list/array) is dereferenced.
>
> thus if i need say to read very huge files and compare them together block
> by block, it's better for me to use strict bytestrings for each pair of
> chunks and manually move the handle from one block to the other, looping
> over the whole file. correct?
>
> 2017-06-26 12:53 GMT+02:00 Silent Leaf <silent.leaf0 at gmail.com>:
>
>> yes, conduits really seem optimal! i'll probably use that in my final
>> version, thanks!
>> however i'll first try without using pre-made tools, to get the handle
>> (ha) of manually using (binary) files with haskell, as after all my program
>> isn that complicated.
>>
>> 2017-06-26 10:42 GMT+02:00 Stefan Risberg <steffenomak at gmail.com>:
>>
>>> I would use some streaming library instead of lazy bytestring to keep
>>> memory at reasonably low levels. It will also help with reading chunks, and
>>> then composing actions on it.
>>>
>>> For library need you got: conduits, iostreams and pipes
>>>
>>>
>>>
>>>
>>> On 26 Jun. 2017 10:37, "Silent Leaf" <silent.leaf0 at gmail.com> wrote:
>>>
>>> i'm reading on the doc of BS.Lazy.hGetContents:
>>> "Once EOF is encountered, the Handle is closed."
>>>
>>> what does that imply if i'm using it inside of withFile? no risk of
>>> getting prematurely out of the function right? that doesn seem possible in
>>> a pure function but i'm asking either way.
>>> if say i do something that reads the whole file, say calculating its
>>> length, does it mean since EOF will be reached i'll have to open the file
>>> again? i think i'm a bit lost...
>>>
>>> i'm trying to find how to read big chunks of two files, do stuff with
>>> each pair of chunk, and so on till the EOF, which may or may not happen at
>>> the same time for both... i don't really know how lazy bytestrings handle,
>>> for example, taking too much from a file. one way would be to calculate the
>>> length, ofc, but for files (partition) of several dozens of gigabytes, it's
>>> a bit delicate... the ideal would be to get the length from the system
>>> itself rather than calculate the whole string ...
>>>
>>> 2017-06-26 7:57 GMT+02:00 Silent Leaf <silent.leaf0 at gmail.com>:
>>>
>>>> Darn quick answer! Thanks Sylvain, that may be all i need to start!
>>>>
>>>> 2017-06-26 7:51 GMT+02:00 Sylvain Henry <sylvain at haskus.fr>:
>>>>
>>>>> Hi,
>>>>>
>>>>> It is not Haskell specific. You just have to read from the partition
>>>>> device special file (e.g., something like /dev/sdb2) as you would do with a
>>>>> normal file. You must have the permission to do so (e.g., be root). Be
>>>>> careful as you can destroy your system if you write something incorrect in
>>>>> your partitions.
>>>>>
>>>>> Repositioning handles: https://www.stackage.org/haddo
>>>>> ck/lts-8.20/base-4.9.1.0/System-IO.html#g:13
>>>>>
>>>>> Read/write: https://www.stackage.org/haddo
>>>>> ck/lts-8.20/base-4.9.1.0/System-IO.html#v:hPutBuf
>>>>>
>>>>> Sylvain
>>>>>
>>>>> On 26/06/2017 07:35, Silent Leaf wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'd like to be able to read and write from/to partitions directly.
>>>>> I've had trouble with the documentation (honestly i can't find anything,
>>>>> and any mention of partitions leads to mathematical partitioning of lists
>>>>> or whatever).
>>>>>
>>>>> I obviously would need to be able to write or read from a specific
>>>>> position in the partition. Mind you that would be good too for files (that
>>>>> is, being able to read/write from a specific position in it) since i plan
>>>>> on making disk images.
>>>>>
>>>>> Thanks in advance!
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Beginners mailing listBeginners at haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Beginners mailing list
>>>>> Beginners at haskell.org
>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
>>>>>
>>>>>
>>>>
>>>
>>> _______________________________________________
>>> Beginners mailing list
>>> Beginners at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
>>>
>>>
>>>
>>> _______________________________________________
>>> Beginners mailing list
>>> Beginners at haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/beginners
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/beginners/attachments/20170626/11a7fd51/attachment.html>