I saw Explain BigInt Like I'm Five, but I already understand what a BigInt is. I want to know how to make one though. I am trying to pick apart BigInt.js (the v8 bigint.cc is too large and I'm not familiar with C++).
For myself and perhaps others in the future, could one explain what the data model looks like for a BigInt that supports arbitrary sized integers? Basically, what is the object and its properties. I get that there are all the arithmetic functions implemented in unique ways for the BigInt, but I don't see what the kernel is. What is the essence of the structure of the BigInt? Perhaps this one will be slightly easier to grok.
A BigInt works exactly like you learned about integers in school, except the "digits" are not based on 10 symbols, they are based on 4294967296 (or 18446744073709551616, or specifically for ECMAScript 9007199254740991).
The kernel of the data model is simply a list of "digits" that are themselves fixed-size integers and a sign bit (or alternatively, the first "digit" is itself signed). Everything else would be a performance optimization.
In pseudo-code, it would look something like this:
record BigInt
sign: boolean
digits: sequence[unsigned_integer]
or this:
record BigInt
first_digit: signed_integer
digits: sequence[unsigned_integer]
Again, if you write down an integer in base-10, you write it as a sequence of digits and a sign, i.e. writing the current year, you would write: 2, 0, 1, 9, signifying (from right-to-left)
9 * 10^0 = 9
+ 1 * 10^1 = 10
+ 0 * 10^2 = 000
+ 2 * 10^3 = 2000
====
2019
Or, maybe you would write 7, E, 3, signifying (from right-to-left)
3_16 * 10_16^0
+ E_16 * 10_16^1
+ 7_16 * 10_16^2
which is the same as
3_16 * 16_10^0
+ E_16 * 16_10^1
+ 7_16 * 16_10^2
which is the same as
3_10 * 16_10^0 = 3_10
+ 14_10 * 16_10^1 = 224_10
+ 7_10 * 16_10^2 = 1792_10
=======
2019_10
And a BigInt is represented in exactly the same way, except the base is (much) larger.
Related
The below TSQL statements are returning different values based on the order of #Size and #Value.The first statement returns 1687.500000 but the second one 1687.600000.
I am guessing it's because of some rounding but I can't really figure out myself. Any help would be really appreciated.
DECLARE #Amount DECIMAL(20,4) = 2,
#PriceDiff DECIMAL(25,10) = 0.421875,
#Size DECIMAL(16,4)= 200000.0000,
#Value DECIMAL(25,15)= 0.010000000000000
SELECT #Amount * #PriceDiff * #Size * #Value AS FinalValue
SELECT #Amount * #PriceDiff * #Value * #Size AS FinalValue
From Management studio
The reason you are experiencing the rounding error is due to the way that SQL Server determines the result of multiplication on a Decimal type's precicion and scale. see here
Also, SQL Server's order of operation for the same/equal-precedence operators is LTR .
Given that the first step is to multiple #Amount * #PriceDiff. According to that link the precision and scale would be:
precision = 20 + 25 + 1 = 46
scale = 4 + 10 = 14
resulting data type = Decimal(46, 14)
This result is over the max allowed precision for a Decimal so things get a little sticky. At the bottom of that link you will see:
The result precision and scale have an absolute maximum of 38. When a result precision is greater than 38, the corresponding scale is reduced to prevent the integral part of a result from being truncated.
Reading up on this further you'll find that instead of just lopping off all of the decimal places to make a Decimal(38,0) or allowing all precision to decimal with a Decimal(38,38) SQL server makes a big fat guess and makes it a Decimal(38,6).
All is well and good though as our result so far is 0.84375 and that fits fine into our new Decimal(38,6) container.
You can see now that if we multiply this by your #Size we will still be within the limits of a Decimal(38,6) with a result of 168750. So we are still good even with the Precision and Scale math and the resulting rounding to a scale of 6 that will occur.
However, if we take that 0.84375 result and multiple by it #Value we get:
precision = 38 + 25 + 1 = 64
scale = 6 + 15 = 21
result = Decimal(64, 21)
Which means we are back at forcing it into a Decimal(38,6)... And 0.0084375 doesn't fit, so it's rounded to 0.08438.
In using the T-SQL ROUND function I noticed what seems like weird behavior. It looks like the ROUND function only looks at the first digit to the right of the digit to be rounded. If I round -6.146 to one decimal I get -6.1. I would have thought it would start at the right and round each digit as it works its way to the left, like this: -6.146 -> -6.15 -> -6.2
I've observed the same behavior with Excel’s round function too.
The query below illustrates what I am describing. I may simply use the nested ROUND functions as shown below but I'm curious if there’s a better way and which approach is considered mathematically correct.
DECLARE #Num AS FLOAT
SET #Num = -6.1463
SELECT #Num [OriginalVal], ROUND(#Num, 1, 0) [SingleRound]
, ROUND(ROUND(ROUND(#Num, 3, 0), 2, 0), 1, 0) [NestedRound]
Results
OriginalVal | SingleRound | NestedRound
-6.1463 | -6.1 | -6.2
I think the basic rule of thumb is, in rounding, you look at the 1 digit immediately to the right of the place you are rounding to. You do not extend it all the way to the very end of the right of the decimal.
http://math.about.com/od/arithmetic/a/Rounding.htm
I was trying to round some fields. When I have 59 days, I want to change it to 60.
The problem is that when I use this code, the 59 is changed to 30 because the round it is 1.
select round(1.9666,0)*30, round(59/30,0)*3'
The result of that query is 60 for the first field and 30 for the second one. The problem is that when I've tried:
select 59/30
The result is 1 and I need the entire answer that is 1.9666...
How can I make it?
Because the number you are dividing by is an INT (the data type of the left side is irrelevant), SQL Server will return an INT as the answer.
If you want a number with a decimal place as your result, you'll need to divide by one.
Don't cast to a FLOAT as the answer is probably not what you want (floats are generally not accurate and are 'approximations'):
SELECT 59 / CAST(30 AS FLOAT) -- = 1.96666666666667
CAST the right-hand side of the division to a DECIMAL:
SELECT 59 / CAST(30 AS DECIMAL(10, 2)) -- = 1.96666
SELECT cast(59 AS FLOAT) / cast(30 AS FLOAT)
Because the original figures are whole numbers, SQL presumes you want a whole number output.
To ensure you get one with the decimal places, you need to first change the data type from an integer int to a floating point float.
This is what the CAST command does.
EDIT: Commenter suggests you cast to DECIMAL instead. The principle is the same, but you need to supply more arguments. To cast to a decimal use something like:
cast(59 as DECIMAL(18, 3))
The first argument (the 18) is the total number of figures you want to permit in the decimal. The second argument (the 3) is the number you want after the decimal point.
The suggestion that it's more accurate is correct - as you'll see if you run the SELECT statements in this answer one after the other. But in this particular case, it only makes a tiny difference.
Using SQL Server 2012...
I have two columns:
Price [decimal(28,12)]
OustandingShares [decimal(38,3)] -- The 38 is overkill but alas, not my call.
When I do an ALTER TABLE I get a resulting computed column as a [decimal(38,6)]. I need the datatype to be [decimal(28,12)].
ALTER TABLE [xyz].MyTable
ADD Mv AS OustandingShares * Price
How can I effectively get 12 decimals of scale on this computed column? I've tried doing convert on the OutstandingShares to 12 decimal places as well as wrapping a convert around the OutstandingShares * Price. The only thing I get is a computed field at [decimal(28,12)] with six trailing zeros.
Thoughts?
The Fix
This does what you want:
CONVERT(DECIMAL(28,12), (
CONVERT(DECIMAL(15, 3), [OustandingShares])
* CONVERT(DECIMAL(24, 12), [Price])
)
)
Test with this:
SELECT CONVERT(DECIMAL(28,12),
(CONVERT(DECIMAL(24,12), 5304.987781883689)
* CONVERT(DECIMAL(15,3), 3510.88)));
Result:
18625175.503659806036
The Reason
The computation is being truncated due to SQL Server's rules for how to handle Precision and Scale across various operations. These rules are detailed in the MSDN page for Precision, Scale, and Length. The details we are interested in for this case are:
Operation: e1 * e2
Result precision: p1 + p2 + 1
Result scale *: s1 + s2
Here the datatypes in play are:
DECIMAL(28, 12)
DECIMAL(38, 3)
This should result in:
Precision = (28 + 38 + 1) = 67
Scale = 15
But the max length of the DECIMAL type is 38. So what gives? We now need to notice that there was a footnote attached to the "Result scale" calculation, being:
* The result precision and scale have an absolute maximum of 38. When a result precision is greater than 38, the corresponding scale is reduced to prevent the integral part of a result from being truncated.
So it seems that in order to get the Precision back down to 38 it chopped off 9 decimal places.
And this is why my proposed fix works. I kept the "Scale" values the same as we don't want to truncate going in and expanding them serves no purpose as SQL Server will expand the Scale as appropriate. The key is in reducing the Precision so that the truncation would be non-existent or at least minimal.
With DECIMAL(15, 3) and DECIMAL(24, 12) we should get:
Precision = (15 + 24 + 1) = 40
Scale = 15
40 is over the limit so reduce by 2 to get down to 38, which means reduce the Scale by 2 leaving us with a true "Result Scale" of 13, which is 1 more than we need and will even be seeing.
Use cast() or convert(). Something like:
ALTER TABLE [xyz].MyTable ADD Mv AS cast(OustandingShares * Price as decimal(12, 6)
or whatever type you want it to be.
EDIT:
Oh, I think I'm getting the idea. The problem is the calculation itself. In that case, do the conversion before the multiplication, so you don't have to depend on SQL Server's (arcane) rules for conforming decimal types.
ALTER TABLE [xyz].MyTable
ADD Mv AS cast(OustandingShares as decimal(28, 12) * cast(Price as decimal(28, 12))
I believe what is happening in your case is that the maximum precision on the calculated result exceeds the allowed thresholds, so the scale is reduced accordingly. This is explained at the bottom of this page.
I have the following query:
DECLARE #A as numeric(36,14) = 480
DECLARE #B as numeric(36,14) = 1
select #B/#A
select cast(#B as decimal)/cast(#A as decimal)
Why does the first calculation returns 0.002083 and the second one returns 0.00208333333333333?
Isn´t numeric(36,14) good enough to have a good precision (just as the second query)?
If I use only numeric, instead of numeric(36,14), I have a good precision again:
select cast(#B as numeric)/cast(#A as numeric)
You can calculate precision and scale by yourself using this documentation from SQL Server Books online.
I tried to calculate precision and scale for your case (operation=division, p=36, s=14) and I got a pretty strange results...
precision of the result: [p1 - s1 + s2 + max(6, s1 + p2 + 1)] -> 36-14+14+max(6,14+36+1)=36+51=87
scale of the result : [max(6, s1 + p2 + 1)] -> max(6,14+36+1)=51
In this situation precision is greater than 38 and in this case (as stated in the documentation)
*The result precision and scale have an absolute maximum of 38. When a result precision is greater than 38, the corresponding scale is
reduced to prevent the integral part of a result from being truncated.
scale must be reduced by (87-38=) 49, that is (51-49=) 2 ...
I think that minimum scale length is 6 (because of expression scale=[max(6, s1 + p2 + 1)]) and it can't be reduced lower than 6 - that we have as a result (0.002083).
Just contributing for the understanding of the problem (going deeper on #Andrey answer), the things could be tricky, depending on the order of calculations.
Consider the variables:^
DECLARE #A as NUMERIC(36,19) = 100
DECLARE #B as NUMERIC(36,19) = 480
DECLARE #C as NUMERIC(36,19) = 100
Calculating A/B*C
If you want to calculate A/B*C, using the formulaes, we have:
A/B is of type NUMERIC(38,6) --> as calculated by #Andrey
The result will be 0.208333 (with scale of 6)
Multiplying by 100, we will get 20.833300
Calculating A*C/B
The result of A*C is 10000 of type NUMERIC(38,6). Diving by C, the result will be 20.833333 of type NUMERIC(38,6)
Then, the result may vary depending on the order of calculation (the same problem was pointed in https://dba.stackexchange.com/questions/77664/how-does-sql-server-determine-precision-scale).