d2jsp
Log InRegister
d2jsp Forums > Off-Topic > Computers & IT > Programming & Development > Scrub Data From Website To Google Sheets
Add Reply New Topic New Poll
Member
Posts: 49,472
Joined: Aug 6 2006
Gold: 0.00
Mar 2 2024 10:23pm
trying to scrub some data from a website to a google sheet and getting stuck. paying for help.

Code
=IMPORTXML("https://maxroll.gg/d2/d2planner/9uox01ty", "/html/body/div[5]/section/div/div/div[3]/div[2]/div[1]/div[2]/div[2]/div[2]/div/div[1]/div/div/div[2]/div/div[3]/ul/li[1]/span/span")
Member
Posts: 6,991
Joined: Apr 16 2019
Gold: 50.00
Mar 2 2024 11:08pm
I'll be honest I've never used TamperMonkey before so there may be better solutions BUT.

Code
// ==UserScript==
// @name _Grab span values on keypress
// @description Help
// @version 1.0
// @match https://maxroll.gg/d2/d2planner/b7q00yjw
// @require https://ajax.googleapis.com/ajax/libs/jquery/2.1.0/jquery.min.js
// @grant GM_addStyle
// @grant GM.getValue
// ==/UserScript==
//- The @grant directives are needed to restore the proper sandbox.
/* global $ */

$(window).keydown (keyboardShortcutHandler);

function keyboardShortcutHandler (zEvent) {
if (zEvent.key == "Enter") {
grabStatsFromSpans ();

zEvent.preventDefault ();
zEvent.stopPropagation ();
return false;
}
}
function grabStatsFromSpans () {
var zStats = $(".d2p-ItemLink").map (
(K, node) => node.textContent.trim ()
).get ();
console.log ("zStats: ", zStats);
}


So if you just put this into a new script, reload the site and press "Enter" you can then go into the Web Browser console and grab an array of all of the Equipment Items listed.

Comes out looking like this - Which you can copy straight from the console.

Code
[
"Call to Arms Crystal Sword",
"Call to Arms War Scepter",
"Fortitude Sacred Armor",
"Hustle Sacred Armor",
"Phoenix Vortex Shield",
"Spirit Gilded Shield",
"Arreat's Face",
"Gore Rider",
"Highlord's Wrath",
"Nature's Peace",
"Raven Frost",
"Shaftstop",
"Steel Carapace",
"Steelrend",
"Stormshield",
"Verdungo's Hearty Cord",
"Wisp Projector",
"Angelic Halo",
"Dread Horn Armet",
"Dread Horn Armet",
"Plague Grip Vampirebone Gloves",
"Havoc Coil Ring",
"Loath Hide Sacred Armor",
"Rune Casque Diadem",
"Bitter Song Berserker Axe",
"Bramble Visage Corona",
"Wraith Casque Guardian Crown",
"Bitter Song Berserker Axe",
"Rune Casque Diadem",
"Carrion Visage Guardian Crown",
"Blood Flange Legendary Mallet",
"Blood Flange Legendary Mallet",
"Glyph Weaver Seraph Rod",
"Bitter Finger Crusader Gauntlets",
"Dread Whorl Ring",
"Doom Torc Amulet",
"Echoing Ancient Sword",
"Hellfire Torch",
"Annihilus",
"Fine Small Charm of Vita",
"Lion Branded Grand Charm of Vita",
"Steel Large Charm",
"Steel Grand Charm of Vita",
"Dread Eye Jewel",
"Dread Eye Jewel",
"Ruby Jewel of Fervor",
"Ivory Jewel of Fervor",
"Ruby Jewel of Carnage"
]


You can easily modify this to remove white spaces/commas ect.

And then you should be able to go through and create new scrapes for the Stats if that's what you were after. I didn't know which part of this you actually wanted to export.

Not the end all but a good start. I can help you further probably if you need, but if that gets you to where you need to be then consider this a learning experience for myself and free work :)

As a site note, you probably want the item descriptions as well (If you're after items) so you could just grab a second array (or probably a multidimensional array would be best) and grab the others.

But I'm too tired to think tonight so let me know tomorrow if you still need help!

Code

var zStats = $(".d2p-item-description").map (
(K, node) => node.textContent.trim ()
).get ();
console.log ("Items: ", zStats);


This post was edited by 3oDAtlas on Mar 2 2024 11:14pm
Member
Posts: 49,472
Joined: Aug 6 2006
Gold: 0.00
Mar 2 2024 11:50pm
im trying to get attack rating, defense, min damage, max damage, and a few other things, I can find them in the dev mode and get the XPath, but im not getting the correct output in the cells
Go Back To Programming & Development Topic List
Add Reply New Topic New Poll