Decimal (10,9) variable can't hold the number 50 (SQL Server 2008) - sql-server

This one is pretty straightforward. Why does the code below cause the error below?
declare #dTest decimal(10, 9)
set #dTest = 50
Error:
Msg 8115, Level 16, State 8, Line 3
Arithmetic overflow error converting int to data type numeric.
According to the MSDN documentation on decimal(p, s), p (or 10 in my case) is the "maximum total number of decimal digits that can be stored, both to the left and to the right of the decimal point" whereas s (or 9 in my case) is the "maximum number of decimal digits that can be stored to the right of the decimal point."
My number, 50, has only 2 digits total (which less than the maximum 10), and 0 digits to the right of the decimal (which is less than the maximum 9), therefore it should work.
I found this question about essentially the same issue, but no one explained why the documentation seems to conflict with the behavior. It seems like the s dimension is actually being interpreted as the fixed number of digits to the right of the decimal, and being subtracted from the p number, which in my case leaves 10 - 9 = only 1 digit remaining to handle the left side.
Can anyone provide a reasonable way to interpret the documentation as written to match the behavior?
EDIT:
I see some explanations below, but they don't address the fundamental problem with the wording of the docs. I would suggest this change in wording:
For "p (precision)" change "The maximum total number of decimal digits that can be stored" to read "The maximum total number of decimal digits that will be stored".
And for "s (scale)" change "The maximum number of decimal digits that can be stored to the right of the decimal point." to "The number of decimal digits that will be stored to the right of the decimal point. This number is substracted from p to determine the maximum number of digits to the left of the decimal point."
I'm going to submit a bug report to Connect unless some one has a better explanation.

10 - 9 is 1. DECIMAL(10, 9) can hold a number in the format 0.000000000. 50 has two digits before the decimal point, and is therefore out of range. You quoted it yourself:
According to the MSDN documentation on decimal(p, s), p (or 10 in my case) is the "maximum total number of decimal digits that can be stored, both to the left and to the right of the decimal point" whereas s (or 9 in my case) is the "maximum number of decimal digits that can be stored to the right of the decimal point."

I submitted a bug report to Connect: Misleading documentation on the decimal data type

A reasonable way to interpret the documentation is that trailing decimal zero digits are not ignored. So your number has 9 decimal digits to the right of the decimal point, and they all happen to be 0.

DECIMAL(10, 9) is a fixed precision and scale numeric data type. This means that it always stores the same number of digits to the right of the decimal point. So the data type you specified can only store numbers with one digit to the left of the decimal point and 9 digits to the right. Obviously, 50 does not fit in a number of that format.

Go though the link below.
http://msdn.microsoft.com/en-gb/library/ms190476.aspx
Precision is the number of digits in a number. Scale is the number of digits to the right of the decimal point in a number. For example, the number 123.45 has a precision of 5 and a scale of 2.

Related

Eiffel: REAL_32.to_double gives a strange value

Trying to transform a real_32 to real_64, I'm getting
real_32: 61.55
real_64: 61.54999923706055
Am I wrong with the to_double function?
This is expected. In the particular example, the binary representation of the decimal 61.55 with single and double precision respectively is:
REAL_32: 0 10000100 11101100011001100110011
REAL_64: 0 10000000100 1110110001100110011001100110011001100110011001100110
As you can see, the trailing pattern 0011 is recurrent and should go ad infinitum to give a precise value.
When REAL_32 is assigned to REAL_64, the trailing 0011s are not added automatically, but filled with zeroes instead:
REAL_32: 0 10000100 11101100011001100110011
REAL_64: 0 10000000100 1110110001100110011001100000000000000000000000000000
In decimal notation, this corresponds to 61.54999923706055. What is essential here, 61.54999923706055 and 61.55 have exactly the same binary representation when using single precision floating numbers. You can check it yourself with print ({REAL_32} 61.55 = {REAL_32} 61.54999923706055). In other words, the results you get are correct, and the two values are the same. The only difference is that when REAL_32 is printed, it is rounded to lower number of meaningful decimal digits.
This is the reason why accounting and bookkeeping software never uses floating-point numbers, only integer and decimal.
As a workaround working for getting from JSON into typescript deserialization, the following worked:
a_real_32.out.to_real_64

Round of Accurately from the last value after decimal

I have stuck (again) and looking for smart human beings of planet earth to help me out.
Background
I have an application which distributes the amounts to some users in a given percentage. Say I have $35000 and it will distribute the amounts to 3 users (A, B and C) in some ratio. So the amount distributed will be
A - 5691.05459265518
B - 14654.473815207
C - 14654.4715921378
which totals up to $35000
The Problem
I have to provide the results on the front end in 2 decimal spaces instead of float. So I use the round function of SQL Server with the precision value of 2 to convert these to 2 decimal spaces with rounding. But the issue is that when I total these values this comes out to be $34999.9999 instead of $35000.
My Findings
I searched a bit and found
If the expression that you are rounding ends with a 5, the Round()
function will round the expression so that the last digit is an even
number. Here are some examples:
Round(34.55, 1) - Result: 34.6 (rounds up)
Round(34.65, 1) - Result: 34.6 (rounds down)
So technically the answer is correct but I am looking for a function or a way to round of the value exactly what it should have been. I found that if I start rounding off (if the value is less than 5 then leave the previous number else increment the previous digit by 1 ) from the last digit after the decimal and keep on backtracking while I am left with only 2 decimal places.
Please advise.

Excel: How to check for repeated numbers inside a cell

I have an excel spreadsheet with numbers from 000 to 999 and am trying to find repeated numbers inside a cell.
(So for example, printing 1 if the number is 022 , 555 or 115 and 0 if it isn't)
So far, I have not been able to find a solution.
Feel free to ask for more information and thanks in advance.
This will do: =IF(COUNT(SEARCH(REPT({0,1,2,3,4,5,6,7,8,9},2),A1))>0,1,0)
Note: If value in cell A1 contains 2 repeated digits it will show 1 else 0. You can customize the repetition limit by changing 2 in the part 8,9},2).
You could try this one if you wanted to find repeated digits not necessarily next to each other:-
=IF(MAX(LEN(A1)-LEN(SUBSTITUTE(A1,{0,1,2,3,4,5,6,7,8,9},"")))>1,1,0)
If the numbers are stored as 3-digit numbers and you wanted it to work for (e.g.) 001, would need:-
=IF(MAX(LEN(TEXT($A1,"000"))-LEN(SUBSTITUTE(TEXT($A1,"000"),{0,1,2,3,4,5,6,7,8,9},"")))>1,1,0)
If your data is in Range "A1:A100" and you want to locate repeated numbers in the range for instance, enter =IF(COUNTIF(A:A,A1)>1,1,0) in cell B1 and fill down. But if you want to check repetitions of specific numbers like 022, 555 or 115, enter =IF(OR(AND(A1=022,COUNTIF(A:A,A1)>1),AND(A1=555,COUNTIF(A:A,A1)>1),AND(A1=115,COUNTIF(A:A,A1)>1)),1,0) in cell B1 and fill down.
being a number, use arithmetics to break it into digits and then check if all are different.
the formula is
=INT(NOT(AND(INT(A1/100)<>INT(MOD(A1,100)/10),INT(A1/100)<>MOD(A1,10),INT(MOD(A1,100)/10)<>MOD(A1,10))))
let's analyze it step by step
first, INT(A1/100) extracts the first digit (the integer division by 100); then INT(MOD(A1,100)/10) extracts the second digit (the integer division by 10 of the modulo 100); and MOD(A1,10) extracts the last digit (the modulo 10).
next there are the three comparisons of difference <> first with second, second with third and first with third, combined with AND() and finally take the result, negate it NOT() and transforming it into an integer 0 or 1 with INT()

ValueError: matplotlib display text must have all code points < 128 or use Unicode strings

In my code I get an array like this:
array(['2.83100e+07', '2.74000e+07', '2.79400e+07'],dtype='|S11')
How can I "cut" my values like:
2.83100e+07 --> 2.831 ?
Best regards!
using a for loop and round(n)
In [23]: round(66.66666666666,4)
Out[23]: 66.6667
In [24]: round(1.29578293,6)
Out[24]: 1.295783
help on round():
round(number[, ndigits]) -> floating point number
Round a number to a given precision in decimal digits (default 0 digits). This always returns a floating point number. Precision may be negative

Representing Year as Roman Number [duplicate]

This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
How to convert integer value to Roman numeral string?
Back in my college days, when learning C and all i had come across a question of representing a year in its corresponding Roman Numeral form. There was no solution in that text, as it was among some extra questions to ponder. All i could think of was using modulus operator and a bunch of ifs. I was wondering if some one could give me a proper solution. A simple algorithm or explanation of the logic used would be appreciated.
Here's logic for numbers less than 4,000. (See below for what to do above that.) There's a basic, 4-step algorithm that is applied at each level of magnitude.
Determine the number of thousands in the number: floor(number/1000). Output that many "M"s and subtract that many thousands from the number. At this point, the number is less than 1,000.
If the number is >= 900, output "CM" and subtract 900.
If the number is >= 500, output "D" and subtract 500.
If the number is >= 400, output "CD" and subtract 400.
At this point, the number is guaranteed to be < 400. We follow a similar pattern to reduce the number to less than 40:
Determine the number of hundreds in the number, output that many "C"s and subtract that many hundreds from the number. At this point, the number is less than 100.
If the number is >= 90, output "XC" and subtract 900.
If the number is >= 50, output "L" and subtract 50.
If the number is >= 40, output "XL" and subtract 40.
At this point, the number is guaranteed to be < 40. We repeat exactly the same logic using "X", "IX", "V", and "IV". We finish by using the number (guaranteed to be < 4) as the count of how many "I"s to output.
For larger numbers, the logic is still the same, we just use the standard symbols with bars on top. Each bar represents 1,000 times the value without that bar. (So V with a bar is 5,000, etc.)

Resources