File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
The moose likes Other Open Source Projects and the fly likes crawl dynamic web page using htmlunit Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login

Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Products » Other Open Source Projects
Bookmark "crawl dynamic web page using htmlunit" Watch "crawl dynamic web page using htmlunit" New topic

crawl dynamic web page using htmlunit

Shanmuga Narayanan.R

Joined: Sep 29, 2013
Posts: 3

I am using Windows XP OS and platform is on Java.

How to crawl data from a dynamic webpage, which uses infinite scrolling to retrieve data dynamically, just like facebook's news feed or like flipkart's loading products dynamically, link :,y63&otracker=nmenu_sub_books-media_Science%20%26%20Technology.

I searched the web got this link :

Is there any other tool which can crawl dynamic contents as mentioned above .

Shanmuga Narayanan.R
It is sorta covered in the JavaRanch Style Guide.
subject: crawl dynamic web page using htmlunit
Similar Threads
Java code for extracting data from a HTML Table from a web page
How do I crawl web pages using JSoup or HtmlUnit?
Http Form Post Through Java Code
How to scroll down a web page using htmlunit 2.12
Vulnerability tracker