d2jsp
Log InRegister
d2jsp Forums > Off-Topic > Computers & IT > Programming & Development > Youtube Metadata To Database/xls? > Or A Better Youtube Search?
Add Reply New Topic New Poll
Member
Posts: 32,925
Joined: Jul 23 2006
Gold: 3,804.50
May 11 2013 03:31am
personally i find youtube sucks when it comes to browsing/searching videos in a channel, searching for comments, etc

Anyone know if there's already a good app for this?

i wrote an app a while back to save to csv for a chess guy so it would be easier to search for openings and such. but the api only let me fetch 50 videos at a time (he had 4000+), and i had to add a slight delay to avoid getting a temp block for spamming. took over an hour just for that one channel, and i wanna follow multiple channels. and i'd have to run it regularly to get up to date data.

/edit: metadata = title, description, length, number of likes, dislikes, upload date, number of comments, number of views, etc etc

This post was edited by carteblanche on May 11 2013 03:52am
Member
Posts: 10,812
Joined: Oct 15 2009
Gold: Locked
Warn: 20%
May 11 2013 02:49pm
according to the documentation:
Code
https://developers.google.com/youtube/2.0/developers_guide_protocol#User_Uploaded_Videos

Code
https://gdata.youtube.com/feeds/api/users/userId/uploads

with UserId replaced with their youtube name should work... but when I tried it, it certainly didn't list all the videos by the user, maybe only 20 or 30 out of like 600


This post was edited by Azrad on May 11 2013 02:53pm
Member
Posts: 10,812
Joined: Oct 15 2009
Gold: Locked
Warn: 20%
May 11 2013 03:47pm
well here is one way:

Code
https://gdata.youtube.com/feeds/api/videos?author=USERNAME&start-index=1&max-results=50

as you said that will give you 50, then hit it again with start-index=51
Code
https://gdata.youtube.com/feeds/api/videos?author=USERNAME&start-index=51&max-results=50

which will give you the next 50
just keep increasing the start index by 50 and hitting the site till you get em all

I don't know if this is what you were doing originally, but I don't think there is a better way.

This post was edited by Azrad on May 11 2013 03:54pm
Member
Posts: 32,925
Joined: Jul 23 2006
Gold: 3,804.50
May 11 2013 06:30pm
Quote (Azrad @ May 11 2013 05:47pm)
well here is one way:

Code
https://gdata.youtube.com/feeds/api/videos?author=USERNAME&start-index=1&max-results=50

as you said that will give you 50, then hit it again with start-index=51
Code
https://gdata.youtube.com/feeds/api/videos?author=USERNAME&start-index=51&max-results=50

which will give you the next 50
just keep increasing the start index by 50 and hitting the site till you get em all

I don't know if this is what you were doing originally, but I don't think there is a better way.


yeah, that's what i'm doing. but 50 at a time takes several hours because i need to pause 1-2s after each time to avoid temp-bans. was wondering if there was an app out there that did a better search.
Member
Posts: 10,812
Joined: Oct 15 2009
Gold: Locked
Warn: 20%
May 11 2013 07:04pm
Quote (carteblanche @ May 11 2013 05:30pm)
yeah, that's what i'm doing. but 50 at a time takes several hours because i need to pause 1-2s after each time to avoid temp-bans. was wondering if there was an app out there that did a better search.


god damn how many videos you trying to get? :evil:
Lets say it takes 10 seconds to get 50. Should be able to get like 300 per minute, or 18,000 an hour!
Oh I guess you are doing more than just collecting each video's title/url?

/e
Oh we should play some chess sometime!

This post was edited by Azrad on May 11 2013 07:08pm
Member
Posts: 32,925
Joined: Jul 23 2006
Gold: 3,804.50
May 11 2013 07:50pm
i dont play much. i have a game going with a coworker and we usually only make 2-3 moves per day, whenever he walks by. but i enjoy watching the 1-2 minute warzone tournys and 5 min pawn storms (Kings indian defence is the most exciting).
Go Back To Programming & Development Topic List
Add Reply New Topic New Poll