1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 | def work(target,words)
begin
resp_1 = Net::HTTP.get_response(URI.parse(target))
rescue => e
puts e
sleep 1
return nil
end
doc = Nokogiri::HTML.parse(resp_1.body, nil,'utf-8')
script_tag = doc.css('script')
json_str = ""
script_tag.each_with_index {|element,i|
if i == 40 then
json_str = element.to_s[58..-11]
end
}
script_tag = nil
title_tag = doc.css('title')
doc = nil
mmmm = title_tag[0].to_s.match(/#{words}/i)
if mmmm == nil
return nil
end
title_tag = nil
begin
script40 = JSON.parse(json_str)
rescue => e
puts e
return nil
end
videoid_list2 = []
videotitle_list2 = []
script40.each {|y,x|
if y.to_s == "contents"
match1 = x.to_s.match(/\{\"title\"=\>\{\"runs\"=\>\[\{\"text\"=\>\"(.*?)\"/)
if match1 != nil
# puts""
# puts"-"*30
# puts $1
# puts $~
# puts"-"*30
tempstrings = $'
while tempstrings.match(/\"title\"=\>\{\"accessibility\"=\>\{\"accessibilityData\"=\>\{\"label\"=\>"(.*?)\"\}\},/) do
if $0 == nil
break
end
# puts" _"*20
# puts""
# puts $1
tempstrings = $'
videotitle_list2.push($1)
match_videoid = /\"commandMetadata\"=\>\{\"webCommandMetadata\"=\>\{\"url\"=\>\"\/watch\?v=(.{11}\"),/ =~ $'
if match_videoid != nil
# puts ("\"" + $1)
videoid_list2.push("\"" + $1)
end
end
else
# puts "-"*30
# puts y x
# puts "can't find the title"
next
end
end
}
videoid_list2.uniq!
videotitle_list2.uniq!
videoid_list3 = []
videotitle_list3 = []
videoid_list2.each_with_index {|content,ind|
if videotitle_list2[ind] == nil
# puts "-"*30
# puts "error"
# puts ind,content
# puts "https://www.youtube.com./watch?v=#{content[1..-2]}"
# puts "-"*30
next
end
mmmm = videotitle_list2[ind].match(/#{words}/i)
if mmmm == nil
# puts "-"*30
# puts ind,content
# puts videotitle_list2[ind]
# puts "skip"
next
end
#puts "-"*30
#puts ind,content
videoid_list3.push(content[0..-1])
#puts "https://www.youtube.com./watch?v=#{content[1..-2]}"
#puts videotitle_list2[ind]
videotitle_list3.push(videotitle_list2[ind])
}
videoid_list2.clear
videotitle_list2.clear
ziped_list = videoid_list3.zip(videotitle_list3)
# ziped_list.each_with_index do |list,ind|
# puts "#{ind}: #{list[0]}"
# puts "#{ind}: #{list[1]}"
# end
return ziped_list
end
|