d2jsp
Log InRegister
d2jsp Forums > Off-Topic > Computers & IT > Programming & Development > How Do Google Links Work? > I Need To Extract Random Url From Google
Add Reply New Topic New Poll
Retired Moderator
Posts: 31,238
Joined: Nov 26 2006
Gold: 11,222.12
Trader: Trusted
Aug 6 2012 09:01am
Hello

I'm making a script in bash, that counts the occurrences of all the words it find on the web

in short what I do is make a dictionary with random words and then perform google searches, extract a random url from the search and use it to count the words.

The only problem is that this way it finds A LOT of wikipedia or dictionary pages and I wanted to know if there was a way to know how to specify the page of the google search in the google link

at the moment what i do is make a variable like this

search="www.google.com/search?q=$wordfromdictionary"

but this way it sends me only to the first page of the search and if I try to understand what's their algorithm to calculate the other pages, I can't figure out

I hope I was enough clear and you can help me

thanks
Member
Posts: 13,630
Joined: Dec 4 2009
Gold: 0.00
Aug 6 2012 09:10am
Just looking at how the link changes based on actually going to Google,

you'll want to add &start=$number, and $number has to be an increment of 10.
Retired Moderator
Posts: 31,238
Joined: Nov 26 2006
Gold: 11,222.12
Trader: Trusted
Aug 6 2012 10:00am
thanks a lot :D
Member
Posts: 2,579
Joined: Jun 1 2012
Gold: 1,524.00
Aug 6 2012 11:57am
You can also exclude pages by including something like this in the search term.

stuff im searching for -site:wikipedia.org

This post was edited by Schwag on Aug 6 2012 11:57am
Go Back To Programming & Development Topic List
Add Reply New Topic New Poll