Sybase IQ - Float to Numeric/Decimal conversion changes the scale of the output - sybase

I have a follow up question with regards to a similar issue I'm having, as the one posted on this thread: Why when converting SQL Real to Numeric does the scale slightly increase?.
I'm using Sybase IQ 16. I have a column called - StrikePrice with float datatype in a table called ProductTb. For a certain record, the value in the table for this column is StrikePrice=22411.39.
When I try to convert it in either decimal or numeric as follows : convert(decimal(31,5), StrikePrice) OR convert(numeric(31,5), StrikePrice), this gives a result as - 22411.39189.
How do I make sure that this shows the value as - 22411.39000 instead of 22411.39189? Is there any workaround that I can use to achieve this?
Thanks in advance for your help.
Example of what I'm looking for:
When I have a value of let's say : 0.0090827, the desired output needed is : 0.00908 i.e. limited to 5 digits after the decimal. And when I have a value of : 22411.39, the desired output is : 22411.39000.
Adding more details
This(above explained issue) seems to work fine on Sybase ASE and the results are what I expect on ASE. The issue is only on Sybase IQ. See details below:
Sybase ASE (15.7):
select TradeNum, StrikePrice as "Original Strike Price in Database", convert(numeric(31, 5), StrikePrice) as "Converted Strike Price"
from ProductTb where TradeNum in (8463676,72372333)
TradeNum Original Strike Price in Database Converted Strike Price
8463676 61.65 61.65000
72372333 85.6 85.60000
72372333 85.6 85.60000
Sybase IQ (16):
select TradeNum, StrikePrice as "Original Strike Price in Database", convert(numeric(31, 5), StrikePrice) as "Converted Strike Price"
from ProductTb where TradeNum in (8463676,72372333)
TradeNum Original Strike Price in Database Converted Strike Price
72372333 85.6 85.59999
72372333 85.6 85.59999
8463676 61.65 61.64999

Looks like you have a couple SQL function options:
round()
SELECT ROUND( 123.234, 1 ) FROM iq_dummy
=> 123.200
truncnum()
SELECT TRUNCNUM( 655, -2 ) FROM iq_dummy
=> 600
SELECT TRUNCNUM( 655.348, 2 ) FROM iq_dummy
=> 655.340
I don't have access to an IQ instance at the moment but I'm thinking something like the following may work:
convert(decimal(31,5), round(StrikePrice,2))
convert(numeric(31,5), truncnum(StrikePrice,2))
UPDATE
The updated question appears to state there are 2 different rounding requirements based on the value in the StrikePrice column:
if < 1.0 then keep 5 decimal places of accuracy
if > 1.0 then keep 2 decimal places of accuracy
NOTE: I'm guessing at the threshold here (< 1.0 vs > 1.-) as this has not been explicitly stated in the question; if OP decides the threshold is something other than 1.0 this should be easy enough to address ... just change the 1.0 in the query (below) to the new threshold value
This logic can be implemented via a case statement.
I don't have access to an IQ instance but the following ASE example should be easy enough to convert to IQ:
create table testtab (a float)
go
insert testtab values (61.649999) -- > 1.0 so round to 2 decimal places
insert testtab values (0.2234234) -- < 1.0 so round to 5 decimal places
go
select convert(numeric(31,5),
case when a < 1.0
then round(a,5)
else round(a,2)
end
)
from testtab
go
--------------
61.65000 -- rounded to 2 decimal places
0.22342 -- rounded to 5 decimal places

Related

Snowflake DIV0 returns different result than regular division

Why are these results different, comparing Snowflake division vs DIV0? It appears DIV0 is rounding down instead of up. Is there any way to make the DIV0 return the same number as the other results?
Snowflake query:
SELECT 75026 / 99999 AS divide_by,
DIV0(75026, 99999) AS div_0,
NVL(75026 / NULLIF(99999, 0), 0) AS div_nvl
Snowflake result:
DIVIDE_BY DIV_0 DIV_NVL
0.750268 0.750267 0.750268
Calculator result:
0.7502675026750268
Another way you can implement NVL based approach
set num=75026;
set den=0;
select iff($den=0,0,$num/$den)

How to achieve 2 decimal places even on zero values in Teradata SQL Assistant?

Below is the code in MS SQL Server
FORMAT(CAST(a.[Sample_Column] as numeric(10,2)), 'C', 'en-US')
I want to achieve the same in Teradata, for which I tried the below:
TRIM(TO_CHAR (cast(tmp."Sample_Column" as decimal(10,2)), '$99,999,999.00' ))
Issue is when the Sample column has values as '0', its populating '.00' in Teradata. I want it to be populated as '0.00' instead.
You must specify 0 in the format to avoid leading blanks.
TRIM(TO_CHAR (cast("Sample_Column" as decimal(10,2)), '$999,999,990.00'))
If "Sample_Column" is numeric, there's no need for a CAST:
TRIM(TO_CHAR ("Sample_Column", '$999,999,990.00'))
The currency sign might be supplied using NLS_PARAMS:
TRIM(TO_CHAR ("Sample_Column", 'L999,999,990.00', 'NLS_CURRENCY = ''$'''));

math - how to properly use MDAS rule in SQL Server

i'm getting the net value of a query. So far here is my problem. Whereas the Cost EUL UL is the only column while NetValue is a query only opened upon the computation.
Cost EUL UL NetValue
9000 5 1 7380 // this is what I want to be displayed
My formula is like this in programming
temp=cost-(.10*cost);
temp2=temp/EUL;
temp3=temp2*UL;
temp4=temp3-cost
so far here is my query
Cost*.10/eul*n_asset_ul //which will be the new column NetValue
and the output of my query is
Cost EUL UL NetValue
9000 5 1 180
Can somebody help?
EDIT: Answers will sometime/always be negative. So i also needed it to be absolute value
Are you sure the NetValue is equal to cost-(.10*cost))/EUL*UL?
It is a sample:
SELECT (Cost-(0.1*Cost))/EUL*UL AS NetValue1 --you r formula
,Cost-((Cost-(0.1*Cost))/EUL*UL) AS NetValue2
,Cost*(1.0-(0.9/EUL*UL)) AS NetValue3
FROM (VALUES( 9000,5,1)) t(Cost,EUL,UL)
NetValue1 NetValue2 NetValue3
-------------- --------------- ---------------------------------------
1620.000000000000 7380.000000000000 7380.000000000000

similar to STGeomFromText function

In ArcGis geological point represent in mssql database as hexa values
eg : 0x**7214**0000010C00000000004C0D4100000000004C0D41
I used mssql geospatial function as below
$query1="DECLARE #Point GEOMETRY
SET #Point = geometry::STGeomFromText('POINT (240000 240000)',0) INSERT INTO main (id, mname, pdata) VALUES (1,'update_1',#Point)";
but it retuns hexa values as 0x**0000**0000010C00000000004C0D4100000000004C0D41
first 4 characters are different from the required outcome , is there any otherway to get that ? I dont have much idea about geospatial function. String replace is not possible thanks.
I am expecting 0x72140000010C00000000004C0D4100000000004C0D41
I assume you're using SRID 5234, which is for Sri Lanka? In your STGeomFromText you have 0 as the second parameter, which is the SRID. Set that to 5234 and you will get the correct output.

Camel MyBatis decimal precision to MSSQL Server

Am having a bit of trouble with decimal precision an insert statement in myBatis.
Its a list request that sometimes contains values that have a null in some rows...
Effectively what is happening is when we pass a null value through, the decimal precision in other rows gets rounded to the nearest Integer. If no nulls are passed, then the data works fine
So our mapper file is something like...
<insert id="insertSales" parameterType="java.util.List">
insert into [dbo].[landing_sales_data]
(
[cust_code]
,[sales_value]
)
values
<foreach item="sales" collection="list" open="(" close=")" separator="),(">
#{sales.custCode},
#{sales.value,jdbcType=DECIMAL,numericScale=2}
</foreach>
</insert>
Which should be producing some SQL like
insert into landing_sales_data
(
[cust_code]
,[sales_value]
)
values(1,11.11),(2,null)
But the end result is that we get 11.00 in the table.
We have specified the jdbcType and the numeric scale in the mapper... and a trace on the SQL server confirms that it is sending 11.11 through to the database.
The problem appears that mybatis is setting the parameter that it creates for the null as with a 0 decimal precision that affects non related rows.
any ideas? i hope we are just missing something.
thanks
Observed the same thing.
Workaround:
Set the decimal values to something like 0.00 before calling MyBatis.
It's better if we can set where we process the data.
But we can do samething in mapper as well.
Ex:
<insert id="insertSales" parameterType="java.util.List">
insert into [dbo].[landing_sales_data]
(
[cust_code]
,[sales_value]
)
values
<foreach item="sales" collection="list" open="(" close=")" separator="),(">
#{sales.custCode},
<if test="sales.value!=null">
#{sales.value,jdbcType=DECIMAL,numericScale=2}
</if>
<if test="sales.value==null">
0.00,
</if>
</foreach>

Resources