java.md

Java Web Scraping

This list contains Java libraries related to web scraping and data processing

• FooLanguage Web Scraping
  • Network
  • Web-scraping Frameworks
  • HTML/XML Parsing
  • Text processing
  • Specific Formats Processing
  • Natural Language Processing
  • Browser automation and emulation
  • Multiprocessing
  • Queue
  • Email
  • URL and Network Address Manipulation
  • Web Content Extracting
  • Asynchronous
  • WebSocket
  • DNS Resolving
  • Computer Vision
  • Proxy Server
  • Other FooLanguage Lists

Network

• General
  • Apache HttpClient
  • okhttp3
• Asynchronous
  • Apache Async HttpClient
  • AsyncHttpClient

Web-Scraping Frameworks

• Full Featured Crawlers

  • ACHE Crawler
  • Apache Nutch

• Other

  • Crawler4j
  • StormCrawler

HTML/XML Parsing

• Apache Tika

Text Processing

Libraries for parsing and manipulating plain texts.

• General
  • Apache Tika

Specific Formats Processing

Libraries for parsing and manipulating specific text formats.

• General

  • Apache Tika

• Something

  • TODO

Natural Language Processing

Libraries for working with human languages.

• Apache OpenNLP
• Stanford CoreNLP
• Apache Tika

Browser automation and emulation

• htmlunit

Multiprocessing

• TODO

Asynchronous

Libraries for asynchronous networking programming.

• TODO

Queue

• TODO

Email

Libraries for parsing email.

• TODO

URL and Network Address Manipulation

Libraries for parsing/modifying URLs and network addresses.

• URL
  • TODO
• Network Address
  • TODO

Web Content Extracting

Libraries for extracting web contents.

• Text and Meta Data from HTML pages
  • Boilerpipe
  • Apache Tika

WebSocket

Libraries for working with WebSocket.

• TODO

DNS Resolving

• dnsjava
• spotify-dns-java

Computer Vision

• TODO

Proxy Server

• TODO

Other FooLanguage lists

• TODO