• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

How does Javaranch allow dynamic pages to be searchable ?

 
Ken Loh
Ranch Hand
Posts: 190
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Please pardon my ignorance, but my understanding has it that only static pages can be searchable by search engines as their spiders and web-crawlers can scout the internet.

However, I have noticed that many dynamic pages are being listed in search engines' search results. One good example is Javaranch's postings.

Could anyone enlighten me how this could happen ?

What should a web content provider do in order to allow contents from their dynamic pages to be searchable ?

Any info would be very much appreciated.
[ March 04, 2005: Message edited by: Ken Loh ]
 
Andrew Monkhouse
author and jackaroo
Marshal Commander
Pie
Posts: 11887
203
C++ Firefox Browser IntelliJ IDE Java Mac Oracle
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Ken,
Please pardon my ignorance, but my understanding has it that only static pages can be searchable by search engines as their spiders and web-crawlers can scout the internet.

Not really. Any page that can be reached via an external link can be searched by a search engine.

Search engines generally cannot handle choice based links (choose your language and product and we will take you to that page ...) or non obvious links (click the button within the shockwave screen to progress ...).

Search engine companies (such as Google and Yahoo) used to have business rules against following dynamically generated pages. From memory they did not index a page they could not guarantee that they could return to. To get around this, some sites (such as JavaRanch) used to have static snapshots of the dynamic pages generated regularly, which the search engines trawled. If you go to the static page, you get redirected (almost instantly) to the dynamic page. This Google search shows that there are still pages indexed that way. (In case it changes, the static page it references is this one).


From memory some of these search engine company's business rules were relaxed as the popularity of Wikis and Blogs took off.

As to how Google are indexing JavaRanch now - I am no longer sure. There are some searches (such as the one above) that still refer to the static pages. There are others that go directly to the dynamic page. Possibly they have more advanced search engines now which realise that anytime they go to "this link" they will always get this page. Therefore it is safe for them to index directly.

So, not sure if I have helped you or not
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic