<HTML><BODY><div id="composeWebView_editable_content" data-mailruapp-compose-id="composeWebView_editable_content" style="text-align: left;"><div>I have a (let's call it) database of heterogeneous records.</div><div><br></div><div>They're not Haskell records, but anonymous/extensible type-labelled rows. (Could be tuples, could be HLists, could be Lens-like, could be something fancier.)</div><div><br></div><div>There's a small number (dozens) of distinct row types, each with a large number (thousands) of rows. <span style="background-color: rgba(255, 255, 255, 0);">The variety of row-types is not predictable in advance. And indeed a row might 'morph' over time with fields added/removed.</span></div><div><br></div><div>So the obvious answer of putting the lot into a giant HList (each element of the list being a row) isn't going to scale. I could have a type-indexed HList in which each element is a Set of homogeneous rows. But performance still suffers from scanning along the list to find the right type index.</div><div><br></div><div>Is there something better? On hackage there's two packages called HSet, neither giving very much help about their suitability:</div><div><br></div><div>* `hset` (lower case) [AlekseyUymanov] seems isomorphic to a type-indexed HList.</div><div> ie Must be unique type in each element (could be a Set type, I guess)</div><div><br></div><div>* `HSet` (upper case) [athanclark] "Faux heterogeneous sets" seems a lot meatier</div><div> why the "Faux"?</div><div> built over hashtables in the ST monad.</div><div><br></div><div>Has anybody used these? Can give guidance on what they can and can't?</div><div><br></div><div>Bonus questions:</div><div><br></div><div>Given a filter specifying a restriction on (some) fields of rows, I want to get a heterog subset:</div><div>* all rows with at least those fields, matching those restrictions.</div><div>* the restriction might be merely "has field labelled L".</div><div><br></div><div>GIven a candidate row for insertion, I want first to scan for quasi-duplicates:</div><div>* any existing row with a subset of the given fields, and the same value at those fields.</div><div>* any existing row with a superset of the given fields, and the same value at those fields in common.</div><div>* ignore records with only a partial overlap of fields.</div><div><br></div><div>One possible data structure: a "vertical store".</div><div>Give each row a Globally Unique Id.</div><div>Have a separate set for each possible field,</div><div>where the set elements are field value (key) to set of GUId -- records with that value.</div><div><br></div><div>Then I have a different bonus question:</div><div>* how to retrieve all field values for a given GUId?</div><div><br></div><div><br></div><div>Thanks</div><div>AntC</div><div id="mail-app-auto-signature"></div> <div id="composeWebView_previouse_content" data-mailruapp-compose-id="composeWebView_previouse_content"></div></div></BODY></HTML>