URLyBird: Please Help! Non-deterministick error with database record reading

[Pawel Solarski]

Hi, I have a strange error occuring very rare during the tests of the database file record reading, with the usage of RadomAccessFile.

Then I execute the JUnit test when many threads trying to read random record and get this strange result:

//this record read correct
Client6 retrieved record 11:
[Palace                                                          , Digitopolis                                                     , 4   , Y, $180.00 , 2003/05/06,         ]
//now the following record read incorrect!
Thread:Client13 record 20 don't match:
expected
[Bed & Breakfast & Business                                      , Xanadu                                                          , 6   , N, $140.00 , 2005/09/28,         ]
actual
[Bed & Breakfast & Business                                      , Xanadu                                                          , Xana, d, u       ,           ,         ]

I start with populating the static cache of database in the JUnit during the junit class loading (similar to JUnit4 beforeClass()):

  
static{
path = "data/db-1x3.db";
database = new DBFileAccess(path);
				int[] all = database.getAll(); //gets the array of record numbers
				records = new HashMap<Integer, String[]>();
				for (int i = 0; i < all.length; i++) {
					try {
						int recNo = all[i];
						records.put(recNo, database
								.read(recNo));
					} catch (RecordNotFoundException e) {
						fail(e.getMessage());
					}
				}
}

After that, many threads are created, each one trying to read some random record:

public void testRead_ManyThreads() throws Throwable {
		int clientCount = 20;
		RecordReader[] clients = new RecordReader[clientCount];
		Random r = new Random();
               //create threads for record reading
		for (int i = 0; i < clients.length; i++) {
			clients[i] = new RecordReader(
					"Client" + (i + 1),
					r.nextInt(database
							.getLastRecordNumber()),
					database);
		}
		//let the thread start reading
		for (Thread thread : clients) {
			thread.start();
		}
		//check until last thread finished read
		for (Thread thread : clients) {
			thread.join();
		}
		
		//check if everything was correct
		int failedReads = 0;
		int incorrectData = 0;
		for (RecordReader client : clients) {
			if (client.isFailed()) {
				failedReads++;
				System.out.println("Thread:" + client.getName()
						+ " could not read record:"
						+ client.getRecNo());
			}
			if (!Arrays.equals(records.get(client.getRecNo()),
					client.getReadRecord())) {
				incorrectData++;
				System.out
						.println("Thread:"
								+ client
										.getName()
								+ " record "
								+ client
										.getRecNo()
								+ " don't match:\nexpected\n"
								+ Arrays
										.toString(records
												.get(client
														.getRecNo()))
								+ "\nactual\n"
								+ Arrays
										.toString(client.getReadRecord()));
			}
		}
		assertEquals(0, failedReads);
		assertEquals(0, incorrectData);
	}
    

And the simple code of RecordReader thread:

 private class RecordReader extends Thread {
		private static final long DELAY = 0000;
		private int recNo;
		private String[] record;

		public int getRecNo() {
			return recNo;
		}

		public String[] getReadRecord() {
			return record;
		}

		private DBFileAccess db;
		private boolean failed = false;

		public boolean isFailed() {
			return failed;
		}

		public RecordReader(String name, int recNo, DBFileAccess db) {
			super(name);
			this.recNo = recNo;
			this.db = db;
		}

		public void run() {
			try {
				Thread.sleep(DELAY);
				record = db.read(recNo);
				System.out.println(getName()
						+ " retrieved record " + recNo
						+ ":\n"
						+ Arrays.toString(record));
			} catch (Throwable e) {
				failed = true;
			}
		}
	};  

It happens very rare and with different records, no matter if a set the thread count to 10 or to 100. I looked through the log and found no other threads trying to change the file pointer. Do you have any idea of what can be wrong?

[Matheus Mendes]

Hi Pawel,

Man I don't know the Urly Bird spec but the problem that you mention seems to be a field length problem. I had a problem as yours on my project, I wasn't counting the first byt that indicates the total length, So sometimes my updates and selects came like yours are coming.

Well, it's just a wondering.... Try to log even more details and post here...

Hope you found this problem soon, wish you lucky

[Jari Timonen]

Have you tried with only one thread? If it does not fail, your threading/locking has issues. Without seeing actual reading of the file, it's hard to determine what's going wrong.

(remember forum rules about pasting entire sections of actual submission code)