[Haskell-cafe] A little bit OT: Google as a global directory/resolver for cabalized packages?

[Dimitry Golubovsky]

Isaac,

Isaac Jones wrote:


>>it returns URLs of all Google-indexed .cabal files (currently 7- just this few?)
> 
> 
> That's certainly not all of them.  Just in the fptools tree we have:
> 
> Cabal.cabal
> GLUT.cabal

[skip]

Well, I think package maintainers might think why Google does not see 
them (unless hidden intentionally). I picked keywords (i. e. Cabal tags) 
which I believe every package should have. Unfortunately, just 
"filetype:cabal" does not work in Google.

>>Using Google API might be an alternative, but it returns only up to 10
>>results (per their docs) which might be enough to find all mirrors of
>>a single package, but not the whole list of available packages.

[Commenting on myself] BTW, one of search parameters of Google API is 
"start: Zero-based index of the first desired result". So if we got 
first 10 URLs, and then set start=10, 20, 30 etc., then perhaps it will 
be like clicking 1 2 3 etc. on the bottom of Google  results page.

>>One weakness: too easy to spam.
> 
> 
> We could possibly overcome that by using the cryptographic signing
> process similar to what I implemented for apt-secure in Debian.  I've

Spamming, I mean, putting fake .cabal files with garbage inside (but 
containing correct tags), which will clutter up Google search results 
and will make more requests necessary.

PS Yahoo recently announced its own search API which seems to be more 
liberal on number of search requests per day, and is done via plain HTTP 
GET request, not WSDL. Unfortunately, Yahoo does not provide search by 
file suffix.

Dimitry Golubovsky
Middletown, CT