Skip to content

Web crawler.

License

Notifications You must be signed in to change notification settings

RecursionHs/WebVideoBot

Repository files navigation

Web Vedeo Bot

java-1.8license: MIT

  • Easy install, all dependencies are already included, just need to have a java environment.

porn_flow

Environment, Architecture

  • Java1.8

  • Crawler4j

  • Spring Boot x H2 Db

How to use

git clone https://github.com/tim232385/WebVideoBot.git
cd WebVideoBot
java -jar PornBot.jar
**DOWNLOAD_VIDEO default is N**See Default Configuration

Demoh2_console

Database Description

http://localhost:8000/h2-console/

JDBC URL: jdbc:h2:~/porn/porn-db

User Name: sa

Password: empty

Record Table:

 Table_Name :PORN_RECORD viewKey :The website's video unique key. imageUrl :Image url of video. linkUrl :Video jump to Website`s link videoUrl :Video adrress. videoTitle :Title of video. videoDuration :Video click count. videoQuality :Defualt quality - 240, 480, 960, 1280p. download :Has been downloaded. True or false. createdTime :The record created time. filePath :The video downloaded path. 

Configuration

 FILE_PATH :Video download path. VIDEO_DOWNLOAD_SIZE :Maximum download size of the video.(Byte) MAX_PAGE_SIZE :Crawling page size. CONCURRENT_THREAD_SIZE :Muilti-threads request amount. START_URL :Crawling url. DOWNLOAD_VIDEO :Download video. Y or N. 

See Default Configuration

About

Web crawler.

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Java100.0%