web.csv is csv file containing urls and their titles + image urls as presented in html files

search

webcsv | grep -i "<search_query>"

search example, multiple words "download debian"

cat web.csv | grep -i "download" | grep -i "debian"

image search
(searchimg searches from web.csv, *tmp commands search from web-tmp.csv)

webcsv | webcsv_simg "<search_query>" | head -n 100 | webcsv_imgui

crawl

1
2
3
webcsv_crawl <start_url>
# additional arguments are passed to wget2, eg:
webcsv_crawl --header 'Cookie: your_session_cookie' "start_url" --max-threads 3

you can stop crawling at any time you wish with ctlr+c
after crawling, update web.csv

webcsv_parse
webcsv_adfilter

upload & generate new link

webcsv_ipfsup

you can generate data uri as well for better decentralization

webcsv_datauri

install

  1. download webcsv

https://h.hostr.cc/p/12daa8df16d9a5d232c14601571a5bf155329a0725aa31af58e7826f8ebd6f1c/d/fileloader.html#1784ae4b7c84bef48cbb56ed10c73366f8ac1253cf8fb6746e91a0edd0a8aeca/quiet/inlineall

  1. source webcsv
    
  2. download web.csv.gz into ~/webcsv/

there is no single place which guarantees best file, here is one link to web.csv.gz, but you might have better one, use which ever version you want

https://h.hostr.cc/p/12daa8df16d9a5d232c14601571a5bf155329a0725aa31af58e7826f8ebd6f1c/d/fileloader.html#fee0e151ab472cd183128a69bd0c013416153829c92161af794ea7669c50d97d/inline/quiet

  1. gzip -dc web.csv.gz > web.csv
Edit

Pub: 14 Nov 2023 10:51 UTC

Edit: 16 Nov 2023 15:20 UTC

Views: 955