you can skip the bloated classes altogether if you use groovy

Here is some example code from my VBA project out of Excel. I am a novice coder in general (trying to get better) so the code isn't as efficient as possible, but it does/did work.

Lets say I had a list of all pitchers from 2004 and was trying to get the "Batted Ball Table" (5th table down on the page) from this website: http://www.fangraphs.com/statss.aspx?playerid=3&position=P#battedball

From that table I would collect all the info from 2004

I have seen now, at least when looking at the inspect element / I see that the table is probably populated with javascript / jquery type of stuff.

I am trying to re-create my program below but take it out of Excel



This post was edited by yamamotocannon on Dec 8 2014 06:53am

That is an odata query in your URL , you can use that with an http request, when I get to my work laptop ill look at the website and see if .net is the choice for you

Also does your excel program work and get what you need?

This post was edited by t9x on Dec 8 2014 06:57am

The excel program does work, just really inefficient. I have experimented with things like screen updating off, but since data is physically copied and pasted at some point into the cells I think it runs slow / can be unwieldly.

Thanks for taking a peek at it

This post was edited by yamamotocannon on Dec 8 2014 07:20am

They aren't hiding anything in java, I don't know much about baseball so I don't know what order the columns come in but it seems like they are doing this:

the "dashboard" is 3 different tables.

Season Team W L SV G GS IP // This is one table
K/9 BB/9 HR/9 BABIP LOB% GB% HR/FB // This is another table
ERA FIP xFIP WAR // This is the 3rd table

Those 3 tables make up "dashboard", they can be viewed in the page source and basically retrieved any way you want, except by element ID, because they dont have an ID, its just printed as text.


Its kind of hard to see how they are feeding it, because when you load the page, there are 500+ things being loaded into your browser, they use alot of different ad services and information trackers.

But if you have google chrome or some form of developer console in your browser, you need to open it, look at the Network tab, and try to find the GET request that returns the information that fills the tables, it will only have the information in the tables and they should have an ID for each row value

I would find it for you, but I am now at work and there is literally 500+ things being loaded into the browser from that page, this is probably why your app is so slow, the generic request for the page has too much with it, things that see what you click to generate ads.

One of the requests being sent will be the one you are looking for, this will speed up your app a lot.

edit: when looking for the correct request, take the URL given and put it in your address bar, it should give you a source that only contains the info in the tables, there could be a different request for each table as this website is a bit sloppy

This post was edited by t9x on Dec 8 2014 08:26am

So the data I am looking for will always appear in some format in one of those network pages? I actually looked through several hundred of them last night and was worried it would be in JavaScript and unintelligible to me.

So for instance I could skip looking at the css / .js files because they will never have the table data reside within them?

Thanks for your time

Edit:

Also I would expect to see the table info in the response body on the network tab?

This post was edited by yamamotocannon on Dec 8 2014 08:53am

yes you will see the table it will be like

table1
{
id1: value
id2: value
id3: value
}