d2jsp
d2jsp Forums > Programmer's Haven > Javascript Task: Scrape Web Text. > 500fg
Add Reply New Topic New Poll
kdog3682
#1 Sep 24 2019 06:23am
Group: Member
Posts: 14,899
Joined: Jan 3 2008
Gold: 143,279.75
Requirements:

* You can use any framework/library you like, as long as it is javascript.
* You don't have to use a framework.

Task: Given an array of codepen.io links ... compile them into a .txt document.

Example:

const inputArray = [
https://codepen.io/WebDevSimplified/pen/EdEjyx
]

Output would yield:

A. https://codepen.io/WebDevSimplified/pen/EdEjyx.html
B. https://codepen.io/WebDevSimplified/pen/EdEjyx.css
C. https://codepen.io/WebDevSimplified/pen/EdEjyx.js

All of the text on A, B, and C would be pasted together on a single .txt document.
waraholic
#2 Sep 24 2019 07:29pm
Group: Member
Posts: 1,024
Joined: Jul 8 2008
Gold: 2,609.50
Using Node v 8.11.4
Code
const http = require('https');
const fs = require('fs');

const inputArray = [
'https://codepen.io/WebDevSimplified/pen/EdEjyx',
'https://codepen.io/vcurd/pen/rNBPQWr',
];

inputArray.forEach(url => {
['html', 'css', 'js'].forEach(extension => {
http.get(`${url}.${extension}`, (res) => {
let data = [];
res.on('data', chunk => {
data.push(chunk);
});
;
res.on('end', () => {
fs.writeFile(`${url.substring(19).replace(/\//g, '_')}.txt`,
data,
{ flag: 'a' },
(err) => {
if (err) throw err;
});
});
});
});
})
Go Back To Programmer's Haven Topic List
Add Reply New Topic New Poll