• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

What problem is solved by ByteBuffer

 
Ranch Hand
Posts: 595
6
jQuery Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,

I work on a project that involves high volume data transactions. I would like to know what problem does ByteBuffer solves and if I can use it for my component.
 
Saloon Keeper
Posts: 15510
363
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
ByteBuffer is a tool that can be used to manipulate big blocks of binary data. It pairs a piece of memory with some variables that do bookkeeping, so you can perform relative operations without having to worry that you are reading from or writing to the wrong memory locations. It also provides some methods that will interpret the binary data as Java primitives.

For instance, here's how you could read some primitives from a file written on a Little Endian platform:

As you can see, there is quite some bookkeeping involved in this code, and you have to mess with low level bit operations.

Here's what you can do with ByteBuffer:

Another advantage is that ByteBuffer can be allocated directly in kernel space using allocateDirect(). When reading data from a file that way, you can read Java primitives directly from the block of memory that the disk controller writes to, instead of having that memory copied to user space first. This is actually only useful in very specific cases, so you'll have to profile your application to see if it will improve performance.
 
Sheriff
Posts: 22783
131
Eclipse IDE Spring VI Editor Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:It pairs a piece of memory


Or disk, through MappedByteBuffer.
 
s ravi chandran
Ranch Hand
Posts: 595
6
jQuery Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If I have a message packet serialized to byte[] which is of small size, but the volume of this packet will be high( max range can reach to 20-10K packets per second). Will using bytebuffer help in reducing the latency in read / write operations?
 
Java Cowboy
Posts: 16084
88
Android Scala IntelliJ IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
That's hard to say in general, without knowing what your code does now exactly with byte arrays. It depends how big the packets are and what exactly has to be done with them, but processing 10-20K packets per second should not be a problem for a modern computer.
 
s ravi chandran
Ranch Hand
Posts: 595
6
jQuery Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Structure of the class is this:



This object is currently being converted to ByteMessage and pushed to a queue which is consumed by a Subscriber which will parse it and process the data.

Normal scenario the rate of publish is 10-20 entries, but during stress test it can reach millions.

Is it a scenario where bytebuffer might improve the performance?

 
Jesper de Jong
Java Cowboy
Posts: 16084
88
Android Scala IntelliJ IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I can't say because I don't know how you are converting this to a ByteMessage.

But more importantly: Do you actually know that you have a performance problem in the part where it converts this object to a ByteMessage? If not, then it's probably not worth the time trying to change it.
 
Stephan van Hulst
Saloon Keeper
Posts: 15510
363
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Going by your class name, it doesn't appear to me that a ByteBuffer will be very useful to you. ByteBuffer is mostly interesting when dealing with high volume low level operations, such as in video or real-time processing.

You say your project involves high volume data transactions. Maybe you can elaborate on that a bit?
 
s ravi chandran
Ranch Hand
Posts: 595
6
jQuery Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
This is for a financial project where we have stock exchange market data updates being published as and when we receive it.

For stress testing of our component, the volume of entries can reach  millions of entries per second.

 
Stephan van Hulst
Saloon Keeper
Posts: 15510
363
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
How is the data published? Are you talking to a web service?
 
author
Posts: 23951
142
jQuery Eclipse IDE Firefox Browser VI Editor C++ Chrome Java Linux Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:
Another advantage is that ByteBuffer can be allocated directly in kernel space using allocateDirect(). When reading data from a file that way, you can read Java primitives directly from the block of memory that the disk controller writes to, instead of having that memory copied to user space first.



It doesn't have to be kernel space. A direct byte buffer is also great for communicating with C libraries that shares a buffer with the Java side. Since direct byte buffers are not affected by the garbage collector... meaning that it doesn't move in memory... both sides can share data easily (and without taking an expensive JNI call).

Henry
 
s ravi chandran
Ranch Hand
Posts: 595
6
jQuery Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Message is published through JMS.

So, is it a realistic expectation that if we have a stress test scenario, we would benefit from using ByteBuffer ( Direct / Non Direct ).
 
Stephan van Hulst
Saloon Keeper
Posts: 15510
363
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
That's exactly what your stress test is for in the first place. Try conventional methods. If they are too slow for your stress test, you try non-conventional methods.
 
Rob Spoor
Sheriff
Posts: 22783
131
Eclipse IDE Spring VI Editor Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I don't think a ByteBuffer is the tool if you just need to convert an object to byte[] (which is what BytesMessage needs) and back. BytesMessage already has the methods you need to write and read your String and primitive fields. For the enums, I'd write and read the name (using valueOf to convert back).

I've actually done the same in a project once - I just have conversion from ByteMessage to my classes and vice versa, that uses the read and write methods of BytesMessage itself.
 
s ravi chandran
Ranch Hand
Posts: 595
6
jQuery Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Rob Spoor wrote:I don't think a ByteBuffer is the tool if you just need to convert an object to byte[] (which is what BytesMessage needs) and back. BytesMessage already has the methods you need to write and read your String and primitive fields. For the enums, I'd write and read the name (using valueOf to convert back).

I've actually done the same in a project once - I just have conversion from ByteMessage to my classes and vice versa, that uses the read and write methods of BytesMessage itself.


So there is no significant benefit as long as I am using it for ByteMessage?
 
Rob Spoor
Sheriff
Posts: 22783
131
Eclipse IDE Spring VI Editor Chrome Java Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
At least I don't see it.
 
Consider Paul's rocket mass heater.
reply
    Bookmark Topic Watch Topic
  • New Topic