| Author |
Breadth first search
|
Johny Kk
Greenhorn
Joined: Oct 23, 2006
Posts: 8
|
|
Hello Friends,
I have to implement a breadth-first web crawler. I have to stop the search based on the level which will be given by user. Not sure how to implement this logic? Please advice if you have any thoughts on this.
Thanks in advance.
Regards
~KK
|
 |
Campbell Ritchie
Sheriff
Joined: Oct 13, 2005
Posts: 32708
|
|
|
You would do well to write down the algorithm in simple words on paper. The pseudo-code method. Also draw a picture of a tree, and go over it with a pencil showing the order you are visiting the nodes.
|
 |
Johny Kk
Greenhorn
Joined: Oct 23, 2006
Posts: 8
|
|
I am doing it now. But somehow I am not able to catch the logic.
Thanks
|
 |
Campbell Ritchie
Sheriff
Joined: Oct 13, 2005
Posts: 32708
|
|
|
This is what I found on Google: 1 2 3. And lots more.
|
 |
Rob Spoor
Sheriff
Joined: Oct 27, 2005
Posts: 19216
|
|
If you need to find out at which level a URL is, just store it with the URL. Create a custom class that holds both the level of the URL and the URL itself.
One thing to think about: what will you do when a URL at level 2 links back to a URL on level 1? Do you skip it (as you've already searched it), or do you check it again as level 3?
|
SCJP 1.4 - SCJP 6 - SCWCD 5
How To Ask Questions How To Answer Questions
|
 |
Johny Kk
Greenhorn
Joined: Oct 23, 2006
Posts: 8
|
|
Thanks Campbell and Rob.
@ Rob... I will skip it. I have to store only unique URL's. I have a custom class which does this for me.
Thanks
~KK
|
 |
 |
|
|
subject: Breadth first search
|
|
|