I have a little issue trying to combine 4 digits together to give me the correct decimal value. First let me start with my code.
long firsttwo, secondtwo, combined;
firsttwo = 0x0C;
secondtwo = 0x6C;
The Decimal value of 0C: 12
The Decimal value of 6C: 108
But the Decimal value of all 0C6C: 3180
Now how do I get all the digits into one variable to be able to convert it to decimal correctely? Because if I just convert firsttwo by itself then secondtwo by itself I don't get the same final total. Thanks!
You need to shift the most significant byte when combining:
combined = (firsttwo << 8) | secondtwo;
this sets combined to 0x0c6c.
Related
I have the following piece of code. I want to format numbers with string templates. One variable has 2 decimal places, the other 4 decimal places but they represent the same number 50000 (fifty thousand).
The first number is correctly formatted (German representation) 50.000,00, the other one however is formatted as 5 million 5.000.000,00!
DATA: lv_p2 TYPE p LENGTH 9 DECIMALS 2,
lv_p4 TYPE p LENGTH 14 DECIMALS 4.
START-OF-SELECTION.
lv_p2 = '50000'.
lv_p4 = lv_p2.
SET COUNTRY 'DE'.
"This is correctly formatted as 50.000,00
WRITE |{ lv_p2 NUMBER = ENVIRONMENT CURRENCY = 'EUR' }|.
"This is on the other hand interpreted as five million! 5.000.000,00
WRITE |{ lv_p4 NUMBER = ENVIRONMENT CURRENCY = 'EUR' }|.
Is this documented somewhere? What am I doing wrong here?
EDIT:
It looks like the problem is with the addition CURRENCY. If I don't use it, then the number is correctly formatted.
WRITE |{ lv_p4 NUMBER = ENVIRONMENT }|.
or WRITE |{ lv_p4 NUMBER = ENVIRONMENT DECIMALS = 2 }|.
Anyway looks like some kind of a bug.
I believe this behaviour is documented.
ABAP Documentation - WRITE, format_options - CURRENCY cur
When CURRENCY is added:
"For data objects of type p, the decimal places determined by the
definition of the data type are ignored completely. Independently of
the actual value and without rounding, decimal separators and
thousands separators are inserted between the digits in the places
determined by cur."
Shortly: if CURRENCY is added (by WRITE), the number of decimal places is determined by the currency (in this case EUR has 2 decimal places), so the value 50.000,0000 will be 5.000.000,00. Same length (9 digits) only the number of decimals will be different.
I am in WinCE7 and to get the current time, I am using GetLocalTime(&systemTime);. This function gives the value of current time. Now if the milliseconds is 81, it displays it as 81 due to which the error occurs when I subtract two time values. For ex: time1 : 12:34:13:851 & time2: 12:34:14:81. Now I need to subtract seconds and milliseconds. So using sprintf, I am extracting seconds and milliseconds and putting them in time1 & time2 :
sprintf(time1,"%d.%d",systemTime.wSeconds,systemTime.wMilliseconds)
sprintf(time2,"%d.%d",systemTime.wSeconds,systemTime.wMilliseconds)
I am converting time1 & time2 into float using atof.Now time1 is 13.851 and time2 is 14.81. The milliseconds of time2 is actually 081 but it displays 81 so while subtracting it consider it as 810 which gives wrong values.
time2--> 14.810 14.081
time1--> 13.851 13.851
-------- ---------
result 0.959(wrong) 0.23(correct)
So to remove this error I thought of counting the digits of milliseconds and if it is 2 then add 0 at starting. So I did:
double digits = (floor (log10 (abs (milliseconds))) + 1); //calculate digits
if(digits == 2) //if milliseconds contains 2 digits, we need to add 0 at starting
{
sprintf(newMS,"0%d",milliseconds); //adding 0 to milliseconds
finalMilliseconds = atoi(newMS); //newMS is in char so converting it into integer and storing the value in finalMilliseconds
}
The problem occurs here. Lets say milliseconds = 18, so newMS = 018 but finalMilliseconds is again 18.
Please suggest any other way of conversion or any other way of adding 0 at starting
According to the documentation of SYSTEMTIME from MSDN:
It is not recommended that you add and subtract values from the
SYSTEMTIME structure to obtain relative times. Instead, you should
Convert the SYSTEMTIME structure to a FILETIME structure.
Copy the resulting FILETIME structure to a ULARGE_INTEGER structure.
Use normal 64-bit arithmetic on the ULARGE_INTEGER value.
The example here will give you some idea on how to get started.
It seems to me the simplest solution is to borrow what you need.
You already have integers. If you're subtracting two systemTime values, t2 from t1, say,
if( t1.wMilliseconds < t2.wMilliseconds ) {
t1.wMilliseconds += 1000;
t1.wSeconds--;
}
Or, just perform the subtraction. If the result's wMlliseconds is negative, adjust as above.
Take care to ensure t1 > t2, though. You don't want -1.25 = 0.0 - 0.75.
Instead of putzing with strings, if you want a float, make one:
double time1 = t1.wSeconds + 0.001 * t1.wMilliseconds;
C does the conversion for you. It's faster, more direct, and less error-prone than going through strings.
Another way of dealing with these marvels of lost leading or trailing zeroes (found in time and longlat), is to right pad the string with say four zeroes .i.e your newMS+"0000", and take the leftmost four characters.
You then have a number (as text) ranging from "0000" to "9990".
Put a "1" in front of it, then you can easily and unambiguously convert to an integer between 10000 and 19990.
Then you can add and subtract as you like.
Clumsy? Yes indeed :) But I have had to do weird tricks like this when GPS longitude readings go from 11.59(funny numbers) to 12.00
I need to insert numbers with decimals into a SQL Server 2008 database. It seems like decimal() is the correct data type to use, however, I'm having trouble understanding it exactly.
I found this script (scroll down for decimal):
http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=95322
Which allows me to test different decimal settings against numbers, and there's some I don't understand why they pass or fail. The way I understand it, when using decimal(precision, scale), precision is the number of digits to the left of the decimal and scale is the number of digits to the right of the decimal. Using this function, I don't understand why some are passing and why some are failing.
SELECT dbo.udfIsValidDECIMAL('2511.1', 6, 3)
I have 4 digits on the left and 1 on the right, yet this fails.
SELECT dbo.udfIsValidDECIMAL('10.123456789123456789', 18, 17)
SELECT dbo.udfIsValidDECIMAL('10.123456789123456789', 18, 16)
The first one fails, the second one passes. There are 18 digits after the decimal point, so it seems like both should fail (or pass and SQL truncates the number).
Maybe I have a fundamental misunderstanding in how decimal() is supposed to work?
DECIMAL(6,3) means: 6 digits in all, 3 of which to the right of the decimal point.
So you have 3 digits before, 3 digits after the decimal point, and of course it cannot handle 2511.1 - that's got four digits to the left of the decimal point. You'd need DECIMAL(7,3) to handle that.
See the MSDN documentation on DECIMAL:
decimal[ (p[ ,s] )] and numeric[ (p[ ,s] )]
p (precision)
The maximum total number of decimal digits that can be stored, both to the left and to the right of the decimal point. The precision
must be a value from 1 through the maximum precision of 38. The
default precision is 18.
s (scale)
The maximum number of decimal digits that can be stored to the right of the decimal point. Scale must be a value from 0 through p.
Scale can be specified only if precision is specified. The default
scale is 0; therefore, 0 <= s <= p. Maximum storage sizes vary, based
on the precision.
Precision is the number of digits that can be stored total.
So the number to the left of the decimal will be precision - scale.
For example, your first example will fail because you are only allowing for three places to the left of the decimal:
SELECT dbo.udfIsValidDECIMAL('2511.1',6,3)
cast(10.123456789123456789 as decimal(18,17))
A precision of 18 & scale of 17 allows just 1 digit to the left of the decimal place, but there are 2 in that example.
cast(10.123456789123456789 as decimal(18,16)
Has room for 2 digits so succeeds.
In sybase, how do you format the result from a select statement for a numeric field, to have no commas and 4 decimal digits.
For Example:
344,500.39495 = 344500.3949
492,930,948.39 = 492930948.3900
You can convert and format a number or a float to string with the str() function.
The second parameter is the total length of the number, including the decimal point, and the third parameter is the length after decimal point.
str(yourDecimalNumber, 20, 4)
You could use ROUND ( numeric-expression, integer-expression )
numeric-expression : The number, passed to the function, to be rounded.
integer-expression : A positive integer specifies the number of significant digits to the right of the decimal point at which to round. A negative expression specifies the number of significant digits to the left of the decimal point at which to round.
I'm trying to write an insert statement for a SQL Server table that inserts the value 1 into a decimal field. The field is of the type decimal(10, 10) which, as far as I understand, means that it can have up to 10 digits altogether, and up to 10 of those digits can be after the decimal point. But, when I try to run the insert statement I get the following error:
Arithmetic overflow error converting int to data type numeric.
If I change the data type of the field to decimal(11, 10), it suddenly works. What am I not understanding here? What am I doing wrong?
decimal(10, 10) means all decimal places, no digits to the left of the decimal point!
see here: http://msdn.microsoft.com/en-us/library/aa258832(SQL.80).aspx_
decimal[(p[, s])]
p (precision) Specifies the maximum total number of decimal digits
that can be stored, both to the left
and to the right of the decimal point.
The precision must be a value from 1
through the maximum precision. The
maximum precision is 38. The default
precision is 18.
s (scale) Specifies the maximum number of decimal digits that can be
stored to the right of the decimal
point. Scale must be a value from 0
through p. Scale can be specified only
if precision is specified. The default
scale is 0; therefore, 0 <= s <= p.
Maximum storage sizes vary, based on
the precision.
decimal(11,10) gives you 1 digit the the left of the decimal and 10 to the right, so integer 1 fits now!
EDIT
when using: decimal(p,s), think of p as how many total digits (regardless of left or right of the decimal point) you want to store, and s as how many of those p digits should be to the right of the decimal point.
DECIMAL(10,5)= 12345.12345
DECIMAL(10,2)= 12345678.12
DECIMAL(10,10)= .1234567891
DECIMAL(11,10)= 1.1234567891