Cameron ax

Greenhorn
+ Follow
since Oct 04, 2012
Merit badge: grant badges
For More
Cows and Likes
Cows
Total received
0
In last 30 days
0
Total given
0
Likes
Total received
0
Received in last 30 days
0
Total given
5
Given in last 30 days
0
Forums and Threads
Scavenger Hunt
expand Ranch Hand Scavenger Hunt
expand Greenhorn Scavenger Hunt

Recent posts by Cameron ax

Paul Clapham wrote:Also, from your eBay User Agreement:

You agree that you will not use any robot, spider, scraper, or other automated means to access our sites for any purpose without our express handwritten permission.



Surely, I won't do any thing against eBay policy. It just hapens it is a good study case.
11 years ago

Bear Bibeault wrote:ebay has a very rich API for getting all sorts of information. The way that you are going about this is like trying to drive a car standing on your head in the seat while wearing a straightjacket.



LOL.. Life would be much less fun, if we do not do things we think itcould not be done.
11 years ago
It is when I click "Customise view", there is a menu jump out, then I need to click "seller information" checkbox to display all the seller information.

I will try to find which one work for that, I may take question back to here if I am failed to find.

Thank you so much.
11 years ago
I never use HtmlUnit before. It seems If I want to execute javascript, I need to work out what is javascript function name.
Is that right?

like eBay page, How could I find out which javascript I should execute?
11 years ago
Here is my example

This is a link for searching iMac in eBay.
eBay.com

When I look at its HTML Scource, I find <div class="s2 distLoc"> content all the information about seller.
here is my code.



But with this code, I cannot reach the data I expected.
the <div class="s2 distLoc"> is totally blank until execute one javascript function in "Customise view" -> "Seller information".

Can you tell how to do that?
11 years ago
Yes, I use HtmlCleaner for parsering HTML file.

now, I do not have a example. I will find a example. Please check my poster later.

Thank you for your help. Appreciated.
I have been searching this for a looong time.
Have not clue at the all.
11 years ago
HtmlUnit.

I will give it a try. thank you.

I have another question about online html parser.

some html file, when I open html source from a web browser, I can see the data in there.
But when I read this html page from java. It can not reach the data.
I saved this html file, and read it as local file, then I can read the data from there.

Why is that?
11 years ago
Hello everyone.

I am not sure there is a way can get data from a html file after it runs completed.

Most html file come with <script>,

when I try to parser it in java, I can only get the data before it runs its script.

but html file could be very different, after it runs its script completed.

my question is how to get the data when it finished.

I have been serched it in the past two weeks, and could not find any.

I take javascript as an example.

Here is my test.html for testing



And here is my Java file




Surly, it cannot get both of text from my test html file.
anyone has a better idea?

Does anyone have any better idea?
Appreciated any help.
11 years ago
I tried to use extractAllNodesThatMatch method. first, I use this method to get all <table> tag from webpage, it return nodelist, I call it tablelist.
after that, I tried to use this method to get <span> from tablelist.
But somehow, it does not work.

For now, I use regular expression method to get the content I want. but I am still looking for some simple method to parser html file.

The target website I listed is for a example. To explain what is my target HTML page will look like.

I just want to know is there method in HTMLparser library could recognize DIV tag which might under a TABLE tag. Does any one has any experience about grab data from a HTML page like this. That is all.

Thank you for quick respond and your help.
Sorry,
I thought you are asking my target html page I want to parser.

so you are askig about htmlparser software?

yes, I do have a link for this,

http://htmlparser.sourceforge.net/

That is the one I am using.
Thank you,
This is for a general question, not for one particular case.
I do not have a link for this.


I want to make a small program to grab data from a website like amazon, ebay, or any online shopping website.
I found there is always a <div> or <span> of product info is placed under <table> tag.
I knnow how to get all the <div> tag form one single page.
but I want to know if there is a way to get all <div>s from one particular <table> tag? and what is method for that in htmlparser library?
Thank you
Hello everyone.

I am tryig use htmlparser library to grab data (price, and item name) from a webpage which is very like the following one.

<table class="item">
<tr>
<td>
<div class="title">Desktop</div>
</td>
<td>
<div class="price">$1,200</div>
</td>
</tr>
</table>

My code is very complex.

I used two parsers, one for searching <div> tag which class equal to "title", the other one is for searching <div> tag which class equal to "price".
I do not know htmlparser library well. I just start to use it two weeks ago, and I find it is very hard to find any sample about it in google.
Does anyone have any better idea?
Appreciated any help.

Jesper de Jong wrote:Welcome to the Ranch.

TextPad is just a text editor. As far as I know, it doesn't know anything about Java by itself. So, there's no way to import Java code in any way.

When you compile your code that uses the HTML parser library, you must specify that you're using the library by using the -classpath (or -cp) option on the command line. So, the command to compile your source code would become something like:

C:\MyProject> javac -cp C:\SomeDir\HtmlParser.jar com\mypackage\MyProgram.java

And to run it, you'd specify:

C:\MyProject> java -cp C:\SomeDir\HtmlParser.jar;. com.mypackage.MyProgram

Note: The ";." in the second command is to include the current directory "." into the classpath, because that's where Java is going to look for your own class com.mypackage.MyProgram.



Sorry to bother you once again.

Say I have unziped my htmlparser files on C:\lib, and I placed my program file on C:\myProgram,

So in DOS window, I run this command

javac -cp C:\lib\htmlparser.jar C:\myProgram\myProgram.java

it completed successfully

then I run java myProgram

It gave me NoClassDefFounndError.

But when I use set classpath method

It does work very well.

Why is that?
11 years ago

Campbell Ritchie wrote:To set the classpath for a particular project, do exactly what you did earlier. As I said, far better than using an environment variable. As far as I know, that sets the classpath for that particular command window, not the environment, but I might be mistaken.
If you want it for an application wrapped in a jar file, that is different. Look in the Java Tutorials under deployment; you will probably find this section most useful for this particular question.


Thank you so much.
That is what I have been looking for.

I think what I did earlier did change environment variable. I run that command once, after that, I find I can load htmparser.jar from everywhere.
11 years ago