• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

A question about ByteBuffer

 
Ranch Hand
Posts: 240
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,
This is from the ByteBuffer doc of j2sdk:


An implementation of the Java platform may optionally support
the creation of direct byte buffers from native code via JNI.
If an instance of one of these kinds of buffers refers to an
inaccessible region of memory then an attempt to access that
region will not change the buffer's content and will cause an
unspecified exception to be thrown either at the time of the
access or at some later time.



I open a direct ByteBuffer, use it and close it , then, open it , use it ... until there is a NullPointerException after I open it and try to use it. What is the problem? Thank you for reply.
Best,
Damu
 
Bartender
Posts: 1872
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I Damu,
You should forget about using direct buffers. The doc states also :

A direct byte buffer may be created by invoking the allocateDirect factory method of this class. The buffers returned by this method typically have somewhat higher allocation and deallocation costs than non-direct buffers. The contents of direct buffers may reside outside of the normal garbage-collected heap, and so their impact upon the memory footprint of an application might not be obvious. It is therefore recommended that direct buffers be allocated primarily for large, long-lived buffers that are subject to the underlying system's native I/O operations. In general it is best to allocate direct buffers only when they yield a measureable gain in program performance.


Record bytebuffers are typically not large neither long-lived. So I think that using direct buffers for them just would add overhead.
Best,
Phil.
 
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Well, you might have a single direct buffer that you re-use for each method call. This would avoid the expense of reallocating a buffer each time - you just need to clear() it. But you also need to synchronize to ensure that only one thread is using the buffer at a time. Many people here have designs that already have such synchronization, in which case it would be faily easy to reuse a buffer. Many others do not though. I would say that you should first consider how you want to handle thread safety issues and locking / unlocking of records. There are many different designs possible here. If, based on other considerations, you decide on a design that does synchronize each of the methods that would access a ByteBuffer, then you may indeed be able to re-use a driect ByteBuffer rather than allocating a new one each time, and re-using the direct buffer will probably be faster. However I don't think you should let the ByteBuffer re-use determine how/if you set up synchronization. That's a potentially complex issue for you to decide, and optimizing the time to create buffers is a very minor thing which should really be an afterthought, IMO. Focus on a design which is both simple and thread-safe first. If, after you've built it and tested it and are sure you understand it, you can see that it's possible to add a a re-used direct ByteBuffer to the design without too much trouble, great, do it, and see if it makes a noticable difference. But it's most likely not worth the time and trouble to change your synchronziation just so you can re-use a buffer. You'll have enough other things to worry about, IMO.
 
Philippe Maquet
Bartender
Posts: 1872
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Jim,
Thank you for those precisions. I agree with all of them ( ). I use myself a record ByteBuffer that I reuse, but only in a context (open) I know it will not be shared. In other cases, I allocate a new buffer each time I need one. In my design, if I had to share such a buffer, I would have to synchronize its access (as you explained above) and it would be a bottleneck. I guess that in your design (where all records are in memory) it may be different. Anyway, if you need to allocate record buffers on demand as I must do, I think that the docs are clear : allocating direct buffers (especially for tiny buffers as our record buffer is BTW) should be slower than allocating buffers on the Java heap.
Best,
Phil.
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I guess that in your design (where all records are in memory) it may be different.
Actually allocate a new buffer each time too; I'm just saying you could re-use a buffer. In my case the use of record-level synchronization means the existing sync would not offer sufficient protection for a single ByteBuffer which is used for different records. But I know many others here don't sync at record level; they sync on the Data instance (or something equivalent). In this case, a single Data-level ByteBuffer would be protected, so if my design used more Data-level sync, I'd probably put in a re-used direct ByteBuffer. But it doesn't, I haven't, and the difference in performance would be pretty trivial anyway in my case. Once I had full caching + RMI, the RMI is a bigger bottleneck than anything else, so these other performance considerations have no real effect. For me anyway...
 
Philippe Maquet
Bartender
Posts: 1872
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
OK. Reading you above, I thought you had such a shared buffer at the record level, storing some rough record view instead of the "translated" record view. But as you don't, such a buffer is (after having filled in your cache) only used for writes. BTW, do you allow concurrent writes ? If not, you could use such a shared buffer for writes, right ?
Best,
Phil.
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I thought you had such a shared buffer at the record level, storing some rough record view instead of the "translated" record view
No, I just save Strings, not ByteBuffers.
BTW, do you allow concurrent writes ?
Yes, as long as they're different records. This provides no real benefit; it's just a natural side effect from the record-level sync - actions on other records may proceed concurrently. And the only reson the record-level sync was never simplified away is that it's the only way to really achieve the "consume no cycles" requirement for locking. Though that's not really such an important requirement, still it's there and I can achieve it, so why not?
If not, you could use such a shared buffer for writes, right ?
Right.
I also could use a single shared buffer for reads, since that's done only once at startup, by a single thread - so protecting the buffer would be trivial. But again, with full caching you do so little I/O that it doesn't really matter how fast it is.
 
Philippe Maquet
Bartender
Posts: 1872
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Jim,

it's the only way to really achieve the "consume no cycles" requirement for locking.


If you mean avoiding the "notify all", I claim that it's not the only way ! I have no "cache-all" cache, but my LockManager class grants locks in FIFO order to clients waiting on their own lock object, consuming ... in the meantime. But you know all that already

I also could use a single shared buffer for reads, since that's done only once at startup, by a single thread - so protecting the buffer would be trivial.


That what I do and explained above ("I use myself a record ByteBuffer that I reuse, but only in a context (open) I know it will not be shared.").
All best,
Phil.
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If you mean avoiding the "notify all", I claim that it's not the only way!
No, what you describe is still essentially record-level sync as far as I meant the term. You've got one monitor per locked record which you sync on, as opposed to syncing on a single shared monitor for all records. You can have record-level sync with or without full caching; those are separable issues. There are many ways to implement this stuff; when I referred to record-level sync I was speaking generally about any strategy where there's one monitor per locked record; this allows you to use notify() rather than notifyAll().
 
Bigwood Liu
Ranch Hand
Posts: 240
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi friends,
I synch at data level, and used a reused byte buffer, every operation associated is synchronized.
The problem is, the buffer is valid a moment ago, but became null after that.
Best,
Damu
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Um, OK, let's try this again...
[Damu]: I open a direct ByteBuffer, use it and close it , then, open it , use it ... until there is a NullPointerException after I open it and try to use it.
How do you "open" and "close" and then again "open" the same ByteBuffer? What methods are you using? Let's see some code.
[ November 17, 2003: Message edited by: Jim Yingst ]
 
Bigwood Liu
Ranch Hand
Posts: 240
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Jim, thank you for reply.
I open the database, create a RandomAccessFile, and get a FileChannel from it, and map the data part to a ByteBuffer, this is how I open a buffer. Then I search the buffer for record. When I close it, I do as following:

I tried many times with desirable result, and got a NullPointerException at last. ???
Best,
Damu
[ November 18, 2003: Message edited by: damu liu ]
 
Bigwood Liu
Ranch Hand
Posts: 240
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I think it is a bug of java .
The problem always occur on file channel or byte buffer or random access file. I think the problem is in the close method.
Is it possible that a static Object that is alive become null after a while without any operation on it?
[ November 18, 2003: Message edited by: damu liu ]
 
Bigwood Liu
Ranch Hand
Posts: 240
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Can you explain it?
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
When I close it, I do as following:
What is "it"? The ByteBuffer? The FileChannel? The RandomAccessFile? Previously you said you "close" the ByteBuffer. That's impossible. So now I have no idea if your problem is with the ByteBuffer or the FileChannel or the RandomAccessFile. The only think I see you doing with the ByteBuffer is setting the reference dataBuffer equal to null. Along with setting several other refernces to null. Hmmm, a clue...
I think it is a bug of java .
I doubt it. A NullPointerException is usually caused because you have a reference that is null. I see you have code that explicitly sets references to null, in the section you quoted. I have no idea if you ever set those references to anything other than null, because you didn't show any other code. But my guess is that you're getting NullPointerException because, well, you set the references to null. Perhaps if you look at the stack trace you can find out exactly which line is throwing the exception. That will confirm exactly which reference we're really talking about.
 
Bigwood Liu
Ranch Hand
Posts: 240
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Jim, Thank you!
"It" means database. When I close the database, some reference is set to null. But I pass new object to them when I open database. So they should not be null.
In addition, I tried once in a loop, byte buffer that was not null a moment ago became null at the next loop.
Best,
Damu
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Looks like you're storing these references in instance variables which are accessed by more than one thread. If one thread calls close() and then another thread tries to access the instance variable again, suddenly it's null. That's the problem. You probably need to either add some more synchronization, or just use only local variables for storing your buffer, channel, etc. It sounds like you're just destorying and recreating them all the time anyway - so why use an instance variable? If you want to use an instance variable, stop setting it to null when other threads may still need it.
 
When you have exhausted all possibilities, remember this: you haven't - Edison. Tiny ad:
a bit of art, as a gift, that will fit in a stocking
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic