d2jsp
Log InRegister
d2jsp Forums > Off-Topic > Computers & IT > Programming & Development > Link To Masses Of External Data In Openoffice?
12Next
Add Reply New Topic New Poll
Member
Posts: 39,789
Joined: Jul 11 2008
Gold: 15.30
Jul 23 2013 01:05am
I'm looking for a way to link massive amounts of data in OpenOffice for a sim league I'm in. Here's an example of what I'm trying to link to my spreadsheet.
http://rslsim.atspace.com/html/players/player1stats.htm
You see player1stat. There are roughly 600 players though, and linking everyone separately would be way to tedious. The only thing that changes in the link is the player number for example:
http://rslsim.atspace.com/html/players/player1stats.htm
http://rslsim.atspace.com/html/players/player2stats.htm
....
http://rslsim.atspace.com/html/players/player600stats.htm
etc etc

I want to import all of them into a spreadsheet. What's the most efficient way to do this? Another program? Another way? Please help.
Member
Posts: 10,812
Joined: Oct 15 2009
Gold: Locked
Warn: 20%
Jul 23 2013 04:40am
Probably a better way, but you can write a little application to download these pages for you, and save the information in a comma delimited format that you want, then import them into your document.

Each player seems to have an uniquely assigned ID number, and a webpage with their stats located at http://rslsim.atspace.com/html/players/playerID.htm
Just write your app to snatch all of these pages you are interested in.

Only problem I would see is that when the information changes on the website, you'll have to re-download the data, and not sure what else you will have to do (depending on what your doing).

This post was edited by Azrad on Jul 23 2013 04:41am
Member
Posts: 3,386
Joined: May 4 2013
Gold: 1,780.00
Jul 23 2013 05:06am
Code
$ ruby -rmechanize -e'n=(1..600); a=Mechanize.new; n.each{|j| o=a.get("http://rslsim.atspace.com/html/players/player#{j}stats.htm"); b=o.search("tr"); [1,2,4].each{|q| puts b[q].search("td").map{|i| i.text}.join(",") } } ' > data.csv

open data.csv

you can alter (1..600) to specify user range you want to get

This post was edited by nuvo on Jul 23 2013 05:07am
Member
Posts: 39,789
Joined: Jul 11 2008
Gold: 15.30
Jul 23 2013 10:11am
Quote (nuvo @ Jul 23 2013 07:06am)
Code
$ ruby -rmechanize -e'n=(1..600); a=Mechanize.new; n.each{|j| o=a.get("http://rslsim.atspace.com/html/players/player#{j}stats.htm"); b=o.search("tr"); [1,2,4].each{|q| puts b[q].search("td").map{|i| i.text}.join(",") } } ' > data.csv

open data.csv

you can alter (1..600) to specify user range you want to get


Is this for excel? I only have open office.
Member
Posts: 3,386
Joined: May 4 2013
Gold: 1,780.00
Jul 23 2013 12:09pm
you can open .csv in openoffice just fine

just need to generate the file with ruby, code is above :^

(needs gem mechanize: gem install mechanize)
Member
Posts: 39,789
Joined: Jul 11 2008
Gold: 15.30
Aug 3 2013 10:24pm
Need a better solution that ruby shit never worked.
Member
Posts: 10,812
Joined: Oct 15 2009
Gold: Locked
Warn: 20%
Aug 3 2013 10:59pm
Quote (YoYoYoYoMan @ Aug 3 2013 09:24pm)
Need a better solution that ruby shit never worked.
Yeah I don't know about the details of that command, but the basic idea is right at the very least. Write a script to download the data and store it, then get it into excel anyway you want.

This post was edited by Azrad on Aug 3 2013 11:00pm
Member
Posts: 39,789
Joined: Jul 11 2008
Gold: 15.30
Aug 4 2013 12:10am
Quote (Azrad @ Aug 4 2013 12:59am)
Yeah I don't know about the details of that command, but the basic idea is right at the very least. Write a script to download the data and store it, then get it into excel anyway you want.


No idea how to do that lol
Member
Posts: 16,144
Joined: Mar 27 2008
Gold: 14,618.00
Aug 4 2013 05:02am
i recommend octave
Member
Posts: 39,789
Joined: Jul 11 2008
Gold: 15.30
Aug 4 2013 12:09pm
Quote (Richter @ Aug 4 2013 07:02am)
i recommend octave


wouldnt know how to use it lol
Go Back To Programming & Development Topic List
12Next
Add Reply New Topic New Poll