Does anyone know if it's possible to do the below in one query instead of doing a numrows?
$select1 = "SELECT service FROM UPSServices WHERE code = '$serviceCode' AND ship_from_code = '$shipFrom'";
$result = mssql_query($select1);
//print_r($result);
if(mssql_num_rows($result) == 0) {
$select2 = "SELECT service FROM UPSServices WHERE code = '$serviceCode' AND ship_from_code IS NULL";
$result = mssql_query($select2);
}
while ($service = mssql_fetch_array($result)) {
return $service['0'];
}
You are using SQL Server, so you can do this. It is slightly more complicated than you might expect. The following version counts the number of valid values. If this is greater than 0, then the NULLs are filtered out.
select service
from (SELECT service,
count(ship_from_code) over () as NumValues
FROM UPSServices
WHERE code = '$serviceCode' AND (ship_from_code = '$shipFrom' OR ship_from_code IS NULL)
) t
where (NumValues > 0 and service is not NULL) or (NumValues = 0 and service is NULL)
limit 1
I think you need value with NULL shipCode if specified shipCode does not exists
SELECT service FROM UPSServices WHERE
code = '$serviceCode' AND (ship_from_code = '$shipFrom' OR ship_from_code IS NULL)
ORDER BY ship_from_code"
Order by will make sure that if your ship code exists then its at top
Related
I'm working through a stored procedure and wondering if there's a way to retrieve the anticipated result column list from a sql statement before fully executing.
Scenarios:
dynamic SQL
a UDF that might vary the columns outside of our control
EX:
//inbound parameter
SET QUERY_DEFINITION_ID = 12345;
//Initial statement pulls query text from bank of queries
var sqlText = getQueryFromQueryBank(QUERY_DEFINITION_ID);
//now we run our query
var cmd = {sqlText: sqlText };
stmt = snowflake.createStatement(cmd);
What I'd like to be able to do is say "right - before you run this, give me the anticipated column list" so I can compare it to what's expected.
EX:
Expected: [col1, col2, col3, col4]
Got: [col1]
Result: Oops. Don't run.
Rationale here is that I want to short-circuit the execution if something is missing - before it potentially runs for a while. I can validate all of this after the fact, but it would be really helpful to stop early.
Any ideas very much appreciated!
This sample SP code shows how to get a list of columns that a query will project into the result before you run the query. It should only be used for large, long running queries because it will take a few seconds to get the column list.
There are a couple of caveats. 1) It will only return the names of the columns. It won't tell you how they were built, that is, whether they're aliased, direct from a table, calculated, etc. 2) The example query I used is straight from the Snowflake documentation here https://docs.snowflake.com/en/user-guide/sample-data-tpcds.html#functional-query-definition. For convenience, I minimized the query to a single line. The output of the columns includes object qualifiers in addition to the column names, so V1.I_CATEGORY, V1.D_YEAR, V1.D_MOY, etc. If you don't want them to make it easier to compare names, you can strip off the qualifiers using the JavaScript split function on the dot and take index 1 of the resulting array.
create or replace procedure EXPLAIN_BEFORE_RUNNING()
returns string
language javascript
execute as caller
as
$$
// Set the context for the session to the TPC-H sample data:
executeNonQuery("use schema snowflake_sample_data.tpcds_sf10tcl;");
// Here's a complex query from the Snowflake docs (minimized to one line for convienience):
var sql = `with v1 as( select i_category, i_brand, cc_name, d_year, d_moy, sum(cs_sales_price) sum_sales, avg(sum(cs_sales_price)) over(partition by i_category, i_brand, cc_name, d_year) avg_monthly_sales, rank() over (partition by i_category, i_brand, cc_name order by d_year, d_moy) rn from item, catalog_sales, date_dim, call_center where cs_item_sk = i_item_sk and cs_sold_date_sk = d_date_sk and cc_call_center_sk= cs_call_center_sk and ( d_year = 1999 or ( d_year = 1999-1 and d_moy =12) or ( d_year = 1999+1 and d_moy =1)) group by i_category, i_brand, cc_name , d_year, d_moy), v2 as( select v1.i_category ,v1.d_year, v1.d_moy ,v1.avg_monthly_sales ,v1.sum_sales, v1_lag.sum_sales psum, v1_lead.sum_sales nsum from v1, v1 v1_lag, v1 v1_lead where v1.i_category = v1_lag.i_category and v1.i_category = v1_lead.i_category and v1.i_brand = v1_lag.i_brand and v1.i_brand = v1_lead.i_brand and v1.cc_name = v1_lag.cc_name and v1.cc_name = v1_lead.cc_name and v1.rn = v1_lag.rn + 1 and v1.rn = v1_lead.rn - 1) select * from v2 where d_year = 1999 and avg_monthly_sales > 0 and case when avg_monthly_sales > 0 then abs(sum_sales - avg_monthly_sales) / avg_monthly_sales else null end > 0.1 order by sum_sales - avg_monthly_sales, 3 limit 100;`;
// Before actually running the query, generate an explain plan.
executeNonQuery("explain " + sql);
// Now read the column list from the explain plan from the result set.
var columnList = executeSingleValueQuery("COLUMN_LIST", `select "expressions" as COLUMN_LIST from table(result_scan(last_query_id())) where "operation" = 'Result';`);
// For now, just exit with the column list as the output...
return columnList;
// Your code here...
// Helper functions:
function executeNonQuery(queryString) {
var out = '';
cmd = {sqlText: queryString};
stmt = snowflake.createStatement(cmd);
var rs;
rs = stmt.execute();
}
function executeSingleValueQuery(columnName, queryString) {
var out;
cmd1 = {sqlText: queryString};
stmt = snowflake.createStatement(cmd1);
var rs;
try{
rs = stmt.execute();
rs.next();
return rs.getColumnValue(columnName);
}
catch(err) {
if (err.message.substring(0, 18) == "ResultSet is empty"){
throw "ERROR: No rows returned in query.";
} else {
throw "ERROR: " + err.message.replace(/\n/g, " ");
}
}
return out;
}
$$;
call Explain_Before_Running();
I'm trying to do a "database side" bulk copy (i.e. SELECT INTO/INSERT INTO) using linq2db. However, my code is trying to bring the dataset over the wire which is not possible given the size of the DB in question.
My code looks like this:
using (var db = new MyDb()) {
var list = db.SourceTable.
Where(s => s.Year > 2012).
GroupBy(s => new { s.Column1, s.Column2 }).
Select(g => new DestinationTable {
Property1 = 'Constant Value',
Property2 = g.First().Column1,
Property3 = g.First().Column2,
Property4 = g.Count(s => s.Column3 == 'Y')
});
db.Execute("TRUNCATE TABLE DESTINATION_TABLE");
db.BulkCopy(new BulkCopyOptions {
BulkCopyType = BulkCopyType.MultipleRows
}, list);
}
The generated SQL looks like this:
BeforeExecute
-- DBNAME SqlServer.2017
TRUNCATE TABLE DESTINATION_TABLE
DataConnection
Query Execution Time (AfterExecute): 00:00:00.0361209. Records Affected: -1.
DataConnection
BeforeExecute
-- DBNAME SqlServer.2017
DECLARE #take Int -- Int32
SET #take = 1
DECLARE #take_1 Int -- Int32
SET #take_1 = 1
DECLARE #take_2 Int -- Int32
...
SELECT
(
SELECT TOP (#take)
[p].[YEAR]
FROM
[dbo].[SOURCE_TABLE] [p]
WHERE
(([p_16].[YEAR] = [p].[YEAR] OR [p_16].[YEAR] IS NULL AND [p].[YEAR] IS NULL) AND ...
...)
FROM SOURCE_TABLE p_16
WHERE p_16.YEAR > 2012
GROUP BY
...
DataConnection
That is all that is logged as the bulkcopy fails with a timeout, i.e. SqlException "Execution Timeout Expired".
Please note that running this query as an INSERT INTO statement takes less than 1 second directly in the DB.
PS: Anyone have any recommendations as to good code based ETL tools to do large DB (+ 1 TB) ETL. Given the DB size I need things to run in the database and not bring data over the wire. I've tried pyspark, python bonobo, c# etlbox and they all move too much data around. I thought linq2db had potential, i.e. basically just act like a C# to SQL transpiler but it is also trying to move data around.
I would suggest to rewrite your query because group by can not return first element. Also Truncate is a part of the library.
var sourceQuery =
from s in db.SourceTable
where s.Year > 2012
select new
{
Source = s,
Count = Sql.Ext.Count(s.Column3 == 'Y' ? 1 : null).Over()
.PartitionBy(s.Column1, s.Column2).ToValue()
RN = Sql.Ext.RowNumber().Over()
.PartitionBy(s.Column1, s.Column2).OrderByDesc(s.Year).ToValue()
};
db.DestinationTable.Truncate();
sourceQuery.Where(s => s.RN == 1)
.Insert(db.DestinationTable,
e => new DestinationTable
{
Property1 = 'Constant Value',
Property2 = e.Source.Column1,
Property3 = e.Source.Column2,
Property4 = e.Count
});
After some investigation I stumbled onto this issue. Which lead me to the solution. The code above needs to change to:
db.Execute("TRUNCATE TABLE DESTINATION_TABLE");
db.SourceTable.
Where(s => s.Year > 2012).
GroupBy(s => new { s.Column1, s.Column2 }).
Select(g => new DestinationTable {
Property1 = 'Constant Value',
Property2 = g.First().Column1,
Property3 = g.First().Column2,
Property4 = g.Count(s => s.Column3 == 'Y')
}).Insert(db.DestinationTable, e => e);
Documentation of the linq2db project leaves a bit to be desired however, in terms of functionality its looking like a great project for ETLs (without horrible 1000s of line copy/paste sql/ssis scripts).
I need to be able to do an IF ELSE in the having clause of a select statement. Here is an excerpt of what I am trying to do:
HAVING
IF #Vendor IS NOT NULL
THEN (COUNT(Iron_1.SN) > 0) AND [User].UserIdentifier = #Vendor AND Location_1.LocationName = #LocationName
ELSE
(COUNT(Iron_1.SN) > 0) AND Location_1.LocationName = #LocationName
I want certain filters based on if the user provided a value for a query parameter or not.
Try this instead
HAVING COUNT(Iron_1.SN) > 0 AND Location_1.LocationName = #LocationName AND
(#Vendor IS NULL OR (#Vendor IS NOT NULL AND [User].UserIdentifier = #Vendor))
or this slightly simplier version
HAVING COUNT(Iron_1.SN) > 0 AND Location_1.LocationName = #LocationName AND
(#Vendor IS NULL OR [User].UserIdentifier = #Vendor)
try the two and go for the one you like the best and works
The following select returns an empty result set, although it shoudn't:
$res = $GLOBALS['TYPO3_DB']->exec_SELECTquery('*', 'tx_xmluploader_xml_import_tree', 'xml_import_id='.$xml_import_id);
$xml_import_id is set. And it works if I remove the where clause..
Thanks
I still don't understand why it doesn't work.. A simple workaround suggested by a coleague:
// select all from the db
$res = $GLOBALS['TYPO3_DB']->exec_SELECTquery('*', 'tx_xmluploader_xml_import_tree');
while( $entry = $GLOBALS['TYPO3_DB']->sql_fetch_assoc() )
{
if( $entry['xml_import_id'] == $xml_import_id ) {
....
}
}
First, make sure the following is set in localconf.php:
$TYPO3_CONF_VARS['SYS']['sqlDebug'] = '1';
$TYPO3_CONF_VARS['FE']['debug'] = '1';
Then try
$res = $GLOBALS['TYPO3_DB']->SELECTquery('*', 'tx_xmluploader_xml_import_tree', 'xml_import_id='.$xml_import_id);
t3lib_div::debug($res);
Result is the output of the query in the frontend. You can then execute it in MySQL for debugging.
a) make sure $xml_import_id actually has a value (one which is in the database as well)
b) Try this:
$res = $GLOBALS['TYPO3_DB']->exec_SELECTquery(
'*',
'tx_xmluploader_xml_import_tree',
"xml_import_id='".$xml_import_id."'"
);
How do you process the result?
How does your expected $xml_import_id value look like?
cu
Roman
if (freightCostTotal == 0)
{
freightCount = 1;
dataGridView1.Rows[i].Cells[iFreightCount].Value = "1";
if (freightCostDefinedTotal != 0)
{
freightCostTotal = Convert.ToDecimal(pcsg.Freight_CountColumn.Table.Rows[0]["Fixed Freight"].ToString());
}
else
{
if (!String.IsNullOrEmpty(dataGridView1.Rows[i].Cells[iCost_Prior_ReCompute].Value.ToString()))
Cost_Prior_ReCompute = Convert.ToDecimal(dataGridView1.Rows[i].Cells[iCost_Prior_ReCompute].Value.ToString());
//freightCostPercentTotal = 0.05M;
//decimal costFreight = Convert.ToDecimal(pcsg.Freight_CountColumn.Table.Rows[0]["Cost"].ToString());
freightCostTotal = (Cost_Prior_ReCompute * freight_perc_);
}
dataGridView1.Rows[i].Cells[iFreight].Value = String.Format("{0:0.00}", freightCostTotal);
}
The following is not working. This is my first attempt at doing sql math
select freightCostTotal, Cost_Prior_ReCompute],freight_perc_, FixedFreight,
if (freightCostTotal == 0) then freightCount = 1;
if (freightCostDefinedTotal != 0) then freightCostDefinedTotal = FixedFreight else
set freightCostTotal = (Cost_Prior_ReCompute * freight_perc_);
from VF_CasINV_Cost
Thanks,
Jerry
Not sure what you're after exactly, but here's a start below. FYI -- your conditional on Cost_Prior_Recompute needs a default / else value -- I just put a '?' in there for now, since I can't tell what it should be.
Also -- I can't tell if freigntCostTotal is an actually field in your database that you're trying to update, or if you plan to just compute it based on other fields each time. So far, I'm assuming you're just going to compute it.
So this is how you would select -- keeping in mind (I assume from reading your code) you are looking for data where the frieghtCostTotal has NOT been updated in the database yet.
SELECT
1 as freightCount,
CASE WHEN freightCostDefinedTotal <> 0
THEN FixedFreight
ELSE ISNULL(Cost_Prior_ReCompute,?) * freight_perc_
END as freigntCostTotal,
Cost_Prior_ReCompute,
freight_perc_,
FixedFreight
FROM
VF_CasINV_Cost
WHERE
freightCostTotal = 0
Alternatively, you could just pull the data out like this, but it will get all data, not just the freightCostTotal = 0 data.
SELECT
1 as freightCount,
freigntCostTotal,
Cost_Prior_ReCompute,
freight_perc_,
FixedFreight
FROM
VF_CasINV_Cost
Here's how you would update. BTW -- If you're new to updating with SQL, I'd highly recommend you test this out on some non-production data first. Or at least, back-up your table first.
UPDATE VF_CasINV_Cost
SET freightCostTotal = CASE WHEN freightCostDefinedTotal <> 0
THEN FixedFreight
ELSE ISNULL(Cost_Prior_ReCompute,?) * freight_perc_
END
WHERE freightCostTotal = 0