I have a query where many columns could be blank or null. They actually have longer names than the example below which I am using as an example:
select *
from table1
where field1 is not null and field1 != '' and
field2 is not null and field2 != ''
...etc
It gets tiresome having to type
x is not null and x != ''.
Is there some way to specify "x is not null and x != ''"?
Like for Java with
StringUtils.isNotEmpty(x)
I use
where isnull(x, '') <> ''
a lot. I find it a bit easier to "understand" than nullif.
-- EDIT ---------------------------------------
I missed that they were all ANDed together. So, if all N fields must be non-null and not empty, assuming that all fields are strings (varchars), this should do it:
where isnull(field1 + field2 + field3 + ... + fieldN, '') <> ''
First, the strings are concatenated together:
If any are null, the result will be null
If none are null and all are empty, the result will be an empty string
Else, the result will be a non-empty string
Next, the results are isnulled:
If the concatenated value is null, it is set to an empty string
Else, you get the concatenated contents (empty or not-empty string)
Last, compare that with the empty string:
If True, then either all are empty or one or more is null
If False, none are null and at least one is not empty
Try
WHERE NULLIF(field1, '') IS NULL
For SQL Server, I would use COALESCE for this:
WHERE COALESCE(field1, '') > ''
ISNULL also works
If you want to exclude rows where every field is null or blank you can do it like this:
WHERE COAlESCE(Field1,Field2,Field3,Field4,Field5,'') <> ''
Related
output--
I have written the query in snowflake to generate Json file, from the query output want to remove fields which has NULL. OBJECT_CONSTRUCT is not working properly for some column its not passing NULL value where else for some column its giving null value as result.
Input-
Json remove any field which has value NULL or blank.
{"DIFID":122,"DIF_FLAG":"NULL","DIF_TYPE":"asian/white","FOCAL_COUNT":2370,"REFERENCE_COUNT":17304},
Required Output-
Json remove any field which has value NULL or blank.
{"DIFID":122,"DIF_TYPE":"asian/white","FOCAL_COUNT":2370,"REFERENCE_COUNT":17304},
query-
select distinct ITEMSTATID,object_construct(
'DIFID',DIFID,
'DIF_TYPE',DIF_TYPE,
'DIF_FLAG',DIF_FLAG,
'FOCAL_COUNT',FOCAL_COUNT::integer,
'REFERENCE_COUNT',REFERENCE_COUNT::integer,
'DIF_METHOD',DIF_METHOD,
'DIF_VALUE',DIF_VALUE)
DIF
from DEV_IPM.STAGEVAULT.DIF_STATISTICS;
For string column and 'NULL' as string literal column's value is not skipped:
CREATE OR REPLACE TABLE DIF_STATISTICS
AS
SELECT 1 AS ITEMSTATID,
122 AS DIFID,
'NULL' AS DIF_FLAG, -- here
'asian/white' AS DIF_TYPE,
2370 AS FOCAL_COUNT,
17304 AS REFERENCE_COUNT;
Output:
The value is definitely stored as TEXT:
SELECT null AS DIF_FLAG, 'NULL' AS DIF_FLAG;
On the left: true NULL on the right: NULL string
If it the case then it should be nullified NULLIF(DIF_FLAG, 'NULL') before passing to OBJECT_CONSTRUCT function:
SELECT ITEMSTATID,
object_construct(
'DIFID',DIFID,
'DIF_TYPE',DIF_TYPE,
'DIF_FLAG',NULLIF(DIF_FLAG, 'NULL'),
'FOCAL_COUNT',FOCAL_COUNT::integer,
'REFERENCE_COUNT',REFERENCE_COUNT::integer) AS DIF
FROM DIF_STATISTICS;
Previous answer before column details were provided (also plausible):
It is working as intended:
NULL Values
Snowflake supports two types of NULL values in semi-structured data:
SQL NULL: SQL NULL means the same thing for semi-structured data types as it means for structured data types: the value is missing or unknown.
JSON null (sometimes called “VARIANT NULL”): In a VARIANT column, JSON null values are stored as a string containing the word “null” to distinguish them from SQL NULL values.
OBJECT_CONSTRUCT
If the key or value is NULL (i.e. SQL NULL), the key-value pair is omitted from the resulting object. A key-value pair consisting of a not-null string as key and a JSON NULL as value (i.e. PARSE_JSON(‘NULL’)) is not omitted.
For true SQL NULL values, that column is ommitted:
CREATE OR REPLACE TABLE DIF_STATISTICS
AS
SELECT 1 AS ITEMSTATID,
122 AS DIFID,
NULL AS DIF_FLAG,
'asian/white' AS DIF_TYPE,
2370 AS FOCAL_COUNT,
17304 AS REFERENCE_COUNT;
SELECT ITEMSTATID,
object_construct(
'DIFID',DIFID,
'DIF_TYPE',DIF_TYPE,
'DIF_FLAG',DIF_FLAG,
'FOCAL_COUNT',FOCAL_COUNT::integer,
'REFERENCE_COUNT',REFERENCE_COUNT::integer) AS DIF
FROM DIF_STATISTICS;
Output:
Probably the data type of the column DIFID is VARIANT/OBJECT:
CREATE OR REPLACE TABLE DIF_STATISTICS
AS
SELECT 1 AS ITEMSTATID,
122 AS DIFID,
PARSE_JSON('NULL') AS DIF_FLAG, -- here
'asian/white' AS DIF_TYPE,
2370 AS FOCAL_COUNT,
17304 AS REFERENCE_COUNT;
Output:
I use copy command of snowflake which is below returns a file with content json
copy into #elasticsearch/product/sf_index
from (select object_construct('id',id, alpha,'alpha')from table limit 1)
file_format = (type = json, COMPRESSION=NONE), overwrite=TRUE, single = TRUE, max_file_size=5368709120;
data is
id alpha
1 null
the output file is
{
"id" :1
}
but I need to have the null values
{
"id" : 1,
"alpha" : null
}
You can use the function OBJECT_CONSTRUCT_KEEP_NULL.
Documentation: https://docs.snowflake.com/en/sql-reference/functions/object_construct_keep_null.html
Example:
select OBJECT_CONSTRUCT_KEEP_NULL('id',id, alpha,'alpha')
Will it be possible for you to check programmatically if the value is null and it is null use the below
select object_construct('id',1,'alpha',parse_json('null'));
Per SnowFlake documentation
If the key or value is NULL (i.e. SQL NULL), the key-value pair will be omitted from the resulting object. A key-value pair consisting of a not-null string as key and a JSON NULL as value (i.e. PARSE_JSON(‘NULL’)) will not be omitted.
The other option is, just send it without the null attribute in Elastic and then take care of the retrieval from Elastic.
How about this
select object_construct('id',id, 'alpha', case when alpha is not null then alpha else 'null' end )from table limit 1;
case should be supported by the copy command.
"null" is a valid in json document as per this SO
Is null valid JSON (4 bytes, nothing else)
Ok another possible way is this using union
select object_construct('id',id, 'alpha', parse_json('NULL') )from table where alpha is null
union
select object_construct('id',id, 'alpha', alpha )from table where alpha is not null;
select object_construct('id', id,'alpha', IFNULL(alpha, PARSE_JSON('null'))) from table limit 1
Use IFNULL to check if the value is null and replace with JSON 'null'
With my select below, if blank string is passed in I get the following error: Null or empty full-text predicate
in my DBAdapter when fetching rows from the database. If I provide a value, such as Well, I do not get results when I should as Well is in the r.[Desc] column. If I pass in Well One, I get: Syntax error near 'one' in the full-text search condition 'Well one'.
If I pass in One, I get nothing.
I've read similar questions here and have not seen a pattern where the value passed in can be nothing, the beginning of the column data, a word in the middle of the column data or more than one word in any order of the column data. I thought Contains returns the row if the column contains the value or part of the value passed in.
What am I doing wrong?
if #Drawing = ''
set #Drawing = null
if #ItemName = ''
set #ItemName = null
if #CF3 = ''
set #CF3 = null
if #Desc = ''
set #Desc = null
if #Design = ''
set #Design = null
if #MaxPSI = 0
set #MaxPSI = null
Select distinct
,r.[DRAWING]
,r.[DESC]
,r.[OP_PSI]
,r.[MAX_PSI]
,r.[MAX_TEMP]
,r.[Insulated]
,r.[DESIGN]
From Ref r
inner join Eng e on e.[DRAWING] = r.[DRAWING]
where r.SurveyNumber = #SurveyNumber
And (rtrim(#Drawing) is NUll or rtrim(r.DRAWING) like rtrim(#Drawing) + '%')
And (rtrim(#Design) is NUll or rtrim(r.DESIGN) like rtrim(#Design) + '%')
And (rtrim(#MaxPSI) is NUll or rtrim(r.MAX_PSI) like rtrim(#MaxPSI) + '%')
And (rtrim(#CF3) is NUll or rtrim(e.CF3) like rtrim(#CF3) + '%')
And (rtrim(#ItemName) is NUll or rtrim(e.ITEM_NAME) like rtrim(#ItemName) + '%')
AND ((#Desc = '""') OR CONTAINS( (r.[Desc]), #Desc))
I think you can try checking for empty as follows:
AND ((#Desc = '""' OR #Desc = '') OR CONTAINS( (r.[Desc]), #Desc))
I suspect the empty predicate may be getting passed as '' instead of "".
Have not used Contains much but from section in the doc [here][1] it seems you either need operators between the words or need to wrap expression in double quotes "". So what you can do is try passing params like this:
AND ((#Desc = '""' OR #Desc = '') OR CONTAINS( (r.[Desc]), '"'+#Desc+'"'))
What possible purpose is
rtrim(#Drawing) is NUll
If it is NULL the rtrim has no meaning. Besides comparison of string basically does an rtrim.
I'm trying to run the following case statement in SQL:
CASE
WHEN P.Photos_Cloned_From IS NULL
THEN 'http://www.toolboxbarn.com/v/vspfiles/photos/'+P.ProductCode+'-2T.jpg'
ELSE 'http://www.toolboxbarn.com/v/vspfiles/photos/'+P.Photos_Cloned_From+'-2T.jpg'
END AS image_link,
The statements works, but only for those records that are not NULL. For items that are NULL, the statement is not returning the THEN condition.
Any suggestions?
Try this and let us know what happens.
'http://www.toolboxbarn.com/v/vspfiles/photos/'+
COALESCE(P.Photos_Cloned_From,P.ProductCode,'DEFAULT')+'-2T.jpg'
AS image_link,
Using coalesce here is better than the case statement. Some platforms can optimize coalesce and it lets you easily make a default value.
#hogan's Suggestion is a great one to save code, but ultimately it should have the same result as your case statement if you don't introduce his new 'default' case. It is most likely that the ProductCode is also NULL is that a possibility?
Why string + NULL = NULL because NULL is an unknown in sql most db platforms will nullify the entire value when null is aggregated or concatenated.
So Think of the following test cases:
Id ProductCode Photos_Cloned_From
1 1 ClonedFrom
2 2 Null
3 NULL Null
The results of your case expression and Hogan's Suggestion would be:
1) ELSE 'http://www.toolboxbarn.com/v/vspfiles/photos/ClonedFrom-2T.jpg'
2) WHEN 'http://www.toolboxbarn.com/v/vspfiles/photos/2-2T.jpg'
3) WHEN NULL
CASE WHEN ISNULL(P.Photos_Cloned_From,'') = '' THEN 'http://www.toolboxbarn.com/v/vspfiles/photos/'+P.ProductCode+'-2T.jpg'
ELSE 'http://www.toolboxbarn.com/v/vspfiles/photos/'+P.Photos_Cloned_From+'-2T.jpg'
END AS image_link,
SELECT
F.Name
, F.Address
, F.InDate
, ( CASE WHEN (F.Date_Down IS NULL and F.Cod_Down = 0 )THEN 0 ELSE -1 END) as 'sn_Down'
FROM
Folders F
I have a table with with details of my companies products, and one of the columns has categories for the products but also empty strings for products that have not yet been categorized.
I'm trying to filter out the category 'trash' and leave the rest including the empty ones, but when I apply the filter, the empty ones are disappearing too...
is this normal? how can I filter out the 'trash' and leave everything else including empty values?
I'm using a WHERE clause like this:
WHERE
table.month = 8 AND table.category <> 'trash'
Probably you have entries with category NULL. Thus, they will not be returned, because by definition the comparison null <> 'trash' is neither true nor false, but a WHERE clause will only return records where all conditions are true.
You could modify your statement to this:
WHERE table.month = 8 AND ISNULL(table.category, '') <> 'trash'
or this:
WHERE table.month = 8 AND (table.category is null OR table.category <> 'trash')
This former replaces null values with empty strings, which should return what you want.