aspose file tools*
The moose likes JForum and the fly likes Reindex Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Products » JForum
Bookmark "Reindex" Watch "Reindex" New topic
Author

Reindex

Migrated From Jforum.net
Ranch Hand

Joined: Apr 22, 2012
Posts: 17424
I reindexed all my posts (postId=0 to postId=99999) after which I couldn't find some posts (via search).

I looked into net.jforum.search.LuceneReindexer
I saw lastPostId was limited by
int fetchCount = SystemGlobals.getIntValue(ConfigKeys.LUCENE_INDEXER_DB_FETCH_COUNT);

Which means only postId=0 to postId=50 (the default FETCH_COUNT value) were reindexed.

In SystemsGlobals.properties we can read:



This value is only read in the LuceneReindexer.reindex(). I changed it to ConfigKeys.LUCENE_INDEXER_RAM_NUMDOCS (10000 by default) and all my posts where finaly indexed.

I'm not familiar at all with Lucene, my question is: what do you mean by "Number of posts to retrieve on each read from the database"?
Why is it used in the reindex process? I sometimes reindex the database and want to be sure, every post was reindexed. What should I do?

Thanks
[originally posted on jforum.net by Exo7]
Migrated From Jforum.net
Ranch Hand

Joined: Apr 22, 2012
Posts: 17424
"on each read" == "while there are data in the database, retrieve them as 50 by 50, to not overload the system memory".

When you downloaded JForum? I upgraded the package yesterday (October 13) because some bug in a lucene query, which may be related to your problem (or not).

Are you able to always reproduce this problem? Which options in the reindex page you selected?

Rafael
[originally posted on jforum.net by Rafael Steil]
Migrated From Jforum.net
Ranch Hand

Joined: Apr 22, 2012
Posts: 17424
I downloaded the new version 3 days ago, I corrected the lucene query myself after having warned you about it.

Rafael Steil wrote:Are you able to always reproduce this problem? Which options in the reindex page you selected?

Yes:
Lucene Statistics > By Message ID From 0 to 99999 > Check if message exists before adding to index AND Recreate index from scratch

It only reindex from 0 to 50.

<br /> <br />
Rafael Steil wrote:"on each read" == "while there are data in the database, retrieve them as 50 by 50, to not overload the system memory".
<br /> <br /> Ok, I may find the problem: <br /> <br /> Let's say I only have 3 posts in my DB, with postId 300, 301 and 302 <br /> The reindex method will first check from 0 to 50 <br /> The List l = dao.getPostsToIndex(firstPostId, toPostId); <br /> is empty. <br /> <br /> And hasMorePosts = hasMorePosts && l.size() > 0; will be set to false.
The loop is broken.

None of my three posts has been reindexed.
[originally posted on jforum.net by Exo7]
Migrated From Jforum.net
Ranch Hand

Joined: Apr 22, 2012
Posts: 17424
Yep, I understand the problem now. A fix is in the CVS already.

A workaround is to reindex by date instead of post id or, of course, start with the first post id that's in the database. But anyway, a fix for it exists. I'll update the package in the next days.

Rafael
[originally posted on jforum.net by Rafael Steil]
Migrated From Jforum.net
Ranch Hand

Joined: Apr 22, 2012
Posts: 17424
Hi,
I have this similar issue.
I tried using the reindex by date but the system indexes only some messages, I have 16 messages at the wnd of reinex process the number of documents that are indexed is always 10.

Because I�m using Oracle I also fixed a query, in Oracle.sql adding under
SearchModel



Where can I download the fix from CVS?

TIA
[originally posted on jforum.net by radar]
Migrated From Jforum.net
Ranch Hand

Joined: Apr 22, 2012
Posts: 17424
Try reindexing by message id if the date option doesn't work for you.

Rafael
[originally posted on jforum.net by Rafael Steil]
Migrated From Jforum.net
Ranch Hand

Joined: Apr 22, 2012
Posts: 17424
Hi,
HAPPY NEW YEAR!!
I tried with the ID instead of date but it works only for few messages.
It indexes only some messages and skips others.

MDT
[originally posted on jforum.net by radar]
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
 
subject: Reindex