File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
The moose likes Java in General and the fly likes Data mining a webpage Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login

Win a copy of REST with Spring (video course) this week in the Spring forum!
JavaRanch » Java Forums » Java » Java in General
Bookmark "Data mining a webpage" Watch "Data mining a webpage" New topic

Data mining a webpage

Gorby Green

Joined: Nov 23, 2002
Posts: 12
I want to create at data mining applikation that collects info from a dropdown list on a webpage. My purpose with this is to store that information into a single xml-file.
What is the best way to do this?

Does anyone know some good site or some good api?

My example:
I would like to store data from date and country- droplists from this html- site.

<form name="date" id="date" method="post" action="">
<select name="select">
<option value="20060101">01 Jan 06</option>
<option value="20060102">02 Jan 06</option>
<option value="20060103">03 Jan 06</option>
<option value="20060104">04 Jan 06</option>
<form name="country" id="country" method="country" action="">
<select name="select">
<option value="au">Australia</option>
<option value="dk">Denmark</option>
<option value="fi">Finland</option>
<option value="/fr/">France</option>
<option value="/de/">Germany</option>

Ulf Dittmer

Joined: Mar 22, 2005
Posts: 42959
For all these purposes I recommened HttpUnit (on SourceForge). It's meant to be a web testing extension to JUnit, but can very nicely be used for accessing web pages programmatically.
Tom Blough
Ranch Hand

Joined: Jul 31, 2003
Posts: 263
Search the forums. This topic has been covered recently.


Tom Blough<br /> <blockquote><font size="1" face="Verdana, Arial">quote:</font><hr>Cum catapultae proscriptae erunt tum soli proscripti catapultas habebunt.<hr></blockquote>
I agree. Here's the link:
subject: Data mining a webpage
It's not a secret anymore!