permaculture playing cards*
The moose likes Java in General and the fly likes Precision and scale of a decimal value Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Java in General
Bookmark "Precision and scale of a decimal value" Watch "Precision and scale of a decimal value" New topic
Author

Precision and scale of a decimal value

Sindhu Kodoor
Ranch Hand

Joined: Sep 03, 2010
Posts: 65
How do we check the precision and scale of a decimal value in Java? In java decimal means a double value for a precision of 1 , and a scale of 7 , I am checking it this way :



The max value of a double with precision 1 and scale 7 is 9.9999999, the reason I am doing this is anything above this precision and scale it should allow, but if it is below precision and scale then it should discard

for example : 2.3333333 is good input , but 0.333 is bad input
Mike Simmons
Ranch Hand

Joined: Mar 05, 2008
Posts: 2994
    
    9
First, and most importantly, the > operator, as you're using it, has nothing to do with the precision or scale of the numbers it compares. Abandon that code. Delete it, forget it, and move on. The solution to your problem may eventually use a > somehow, but not that way. Begin anew.

What sort of "input" are you looking at, in the first place your code encounters it? Is it already a Java primitive such as double? Or is it a String? Or a BigDecimal? Or a stream of raw bytes that needs to be converted to a String, or a double, or something else? The exact format makes a difference here.

If you're given a double (primitive), there is really no way to tell how precise it it; that information has already been lost. If you get a double value of 42, you can't tell whether that was 42.0, 42.00, 42.00000000, etc. All of those are just 42.0, as a double.

If your're given a String, and that String represents a decimal value, then you may have enough information. The length of the String is a good start. Subtract 1 if the String contains a '.' character. However this is still risky - if the input was "1000", how accurate is it really? Is that 1 significant digit, or 4? I don't think there is any one answer to this that works for all situations - but there may be one answer that works for you, for whatever you are trying to do.

If you're given a BigDecimal, this question would have been solved simply by looking at the API.

If you're given an input stream (bytes) of some sort, you need to know more about how the information was originally encoded, and how to convert it to a String, or a double, or an int, or a BigDecimal, or something else. If you have no idea, my first guess would be that it's intended to be a String and you need to create a Reader of some sort, like a FileReader or InputStreamReader. But that guess may be wrong.
Sindhu Kodoor
Ranch Hand

Joined: Sep 03, 2010
Posts: 65
ok, the input is a String , and I looked up the BigDecimal examples and found that we have methods called precision and scale to check the value , so first I convert the input string for example "1.1234567" and convert to a BigDecimal :



so we have a [8,7] as the precision - scale . I need to do the following with this value now :

1. I need to allow value for the scale being above or below 7 example 1.12345678
2. But I need to be careful about the values befor the "." decimal point it should be "1" of length 1 not more not less than that.

With precision I cannot check only the int part of the decimal, it is the whole length.

Can you please suggest about the pow(int arg0) method? will this solve my problem?
Mike Simmons
Ranch Hand

Joined: Mar 05, 2008
Posts: 2994
    
    9
Sindhu Kodoor wrote:1. I need to allow value for the scale being above or below 7 example 1.12345678
2. But I need to be careful about the values befor the "." decimal point it should be "1" of length 1 not more not less than that.

Excuse me? Are you saying you can't handle a value like 2.0 or 1000000.0? Really?

Sindhu Kodoor wrote:With precision I cannot check only the int part of the decimal, it is the whole length.

Can you please suggest about the pow(int arg0) method? will this solve my problem?

I really have no idea what you're thinking here, sorry. The train left the tracks some time ago.
Jan Hoppmann
Ranch Hand

Joined: Jul 19, 2010
Posts: 142

If you don't want more or less than 1 digit before the decimal seperator try to check if it is more or less than 1. That should be fairly simple and straightforward. Just think in what cases there would be more or less than 1 digit, and check if the String class offers some method to divide Strings and check their length.


Life is full of choices. Sometimes you make the good ones, and sometimes you have to kill all the witnesses.
Sindhu Kodoor
Ranch Hand

Joined: Sep 03, 2010
Posts: 65
Excuse me? Are you saying you can't handle a value like 2.0 or 1000000.0? Really?


I do not know the tone , but it seems like an underestimate!! , you dint seem to get the problem.


I have a requirement , of a range of [8,7] , which means before the decimal point it should have 1 digit and after the decimal point the scale should be 7 digits . And also , if the scale is having less than or more than 7 digits also it is ok , but the digit before the decimal point should be restricted to only "1".

Using BigDecimal , I can get the value of the int part ( toBigInteger) but it would not give its length or size. IS there a way to get the length of the interger part?
Jan Hoppmann
Ranch Hand

Joined: Jul 19, 2010
Posts: 142

You could always call toString on it, but wouldn't it be easier to check the value of the Integer? Hint: All values greater or equal 10 have more than one digit.
Sindhu Kodoor
Ranch Hand

Joined: Sep 03, 2010
Posts: 65
yes true! Thank you
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Precision and scale of a decimal value
 
Similar Threads
float or double
Oracle jdbc driver getColumnType return Number
Decimals not converting to correct fractions
rounding off in java
converting float hex to int