So I'm trying to make a query which goes through varbinary data. The issue is that I can't really finish what I'm trying to achieve. What you should know about the column is varbinary(50) and the patterns that occur have no specific order in writing, meaning every prefix could be anywhere as long it has 3 bytes(0x000000) First one is the prefix second and third are value data that I'm looking to check if its within the range i like. All the data is written like this.
What I've tried:
DECLARE #t TABLE (
val VARBINARY(MAX)
)
INSERT INTO #t SELECT 0x00000100000000000000000000000000000000000000000000000000
INSERT INTO #t SELECT 0x00001000000000000000000000000000000000000000000000000000
INSERT INTO #t SELECT 0x00010000000000000000000000000000000000000000000000000000
INSERT INTO #t SELECT 0x00100000000000000000000000000000000000000000000000000000
INSERT INTO #t SELECT 0x00000f00000000000000000000000000000000000000000000000000
declare #pattern varbinary(max)
declare #pattern2 varbinary(max)
set #pattern = 0x0001
set #pattern2 = #pattern+0xFF
select #pattern,#pattern2
SELECT
*
FROM #t
WHERE val<#pattern
OR val>#pattern2
This was total bust the patterns were accurate up to 2 symbols if I were to use 4 symbols as pattern it would work only if the pattern is in predefined position. I've tried combination of this and everything below.
WHERE CONVERT(varbinary(2), val) = 0xdata
also this:
select *
from table
where CONVERT(varchar(max),val,2) like '%data%'
Which works great for searching exact patterns, but not for ranges, I need some combination of both.
I'm aware I could technically add every possible outcome 1 by 1 and let it cycle through all the listed possibilities, but there has to be a smarter way.
Goals:
Locating the prefix(first binary data pair)
Defining a max value after the prefix, everything above that threshold to be listed in the results. Let's say '26' is the prefix, the highest allowed number after is '9600' or '269600'. Basically any data that exceeds this pattern '269600' should be detected example '269700'.
or query result would post this:
select * from table where CONVERT(varchar(max),attr,2) like
'%269700%'
I need something that would detect this on its own while i just give it start and end to look in between like the highest number variation would be '26ffff', but limiting it to something like 'ff00' is acceptable for what I'm looking for.
My best guess is 2 defined numbers, 1 being the allowed max range
and 2nd for a cap, so it doesn't go through every possible outcome.
But I would be happy to whatever works.
I'm aware this explanation is pretty dire, but bear with me, thanks.
*Update after the last suggestion
SELECT MIN(val), MAX(val) FROM #t where CONVERT(varchar(max),val,2) like '%26%'
This is pretty close, but its not sufficient i need to cycle through alot of data and use it after this would select only min or max even with the prefix filter. I believe i need min and max defined as a start and end range where the query should look for.
**Update2
I'm afraid you would end up disappointed, its nothing that interesting.
The data origin is related to a game server which stores the data like this. There's the predefined prefixes which are the stat type and the rest of the data is the actual numeric value of the stat. The data is represented by 6 characters data intervals. Here is a sample of the data stream. Its always 6-6-6-6-6 as long there's space to record the data on since its capped at 50 characters.
0x0329000414000B14000C14000D0F00177800224600467800473C00550F00000000000000000000000000
**Update3
Yes the groups are always in 3byte fashion, yes my idea was exactly that use the first byte to narrow down the search and use then use the second 2 bytes to filter it. I just don't know how to pull it off in an effective way. I'm not sure if i understood what u meant by predictively aligned assuming you meant if stat/prefix/header would always end up at the same binary location, if that's correct the answer is no. If the 3byte pattern is violated the data becomes unreadable meaning even if you don't need the extra byte you have to count it otherwise the data breaks example of a working data.
0x032900'041400'
example of a broken data:
0x0329'041400'
The only issue i could think is when the prefix and part of the value are both true example:
0x262600
Unless the query is specifically ordered to read the data in 3byte sequence meaning it knows that the first byte is always a prefix and the other 2 bytes are value.
Q:Can that be used as an alignment indicator so that the first non-zero byte after at least 3 zero bytes indicates the start of a group?
A:Yes, but that's unlikely I mean it although possible it would be written in order like:
0x260000'270000'
It wouldn't skip forward an entire 3byte group filled with no data. This type of entry would occur if someone were to manually insert it to the db, the server doesn't make records with gaps like those as far I'm aware:
0x260000'000000'270000'
To address your last comment that's something I don't know how to express it in a working query, except for the boneheaded version which would be me manually adding every possible number within my desired range with +1bit after that number. As you can imagine the query would look terrible. That's why I'm looking for a smarter solution that I cannot figure out how to do so by my self.
select * from #t
where (CONVERT(varchar(max),val,2) like '%262100%' or
CONVERT(varchar(max),attr,2) like '%262200%' or
etc...)
This may be a partial answer from which you can build.
The following will split the input data up into 3-byte (6 hex character) groups. It then extracts the first byte as the key, and several representations of the remaining two bytes as values.
SELECT S.*, P.*
FROM #t T
CROSS APPLY (
SELECT
N.Offset,
SUBSTRING(T.val, N.Offset + 1, 3) AS Segment
FROM (
VALUES
(0), (3), (6), (9), (12), (15), (18), (21), (24), (27),
(30), (33), (36), (39)
) N(Offset)
WHERE N.Offset < LEN(T.val) - 3
) S
CROSS APPLY(
SELECT
CONVERT(TINYINT, SUBSTRING(S.Segment, 1, 1)) AS [Key],
CONVERT(TINYINT, SUBSTRING(S.Segment, 2, 1)) AS [Value1],
CONVERT(TINYINT, SUBSTRING(S.Segment, 3, 1)) AS [Value2],
CONVERT(SMALLINT, SUBSTRING(S.Segment, 2, 2)) AS [Value12],
CONVERT(SMALLINT, SUBSTRING(S.Segment, 3, 1) + SUBSTRING(S.Segment, 2, 1)) AS [Value21]
) P
Given the following input data
0x0329000414000B14000C14000D0F00177800224600467800473C00550F00000000000000000000000000
--^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----
The following results are extracted:
Offset
Segment
Key
Value1
Value2
Value12
Value21
0
0x032900
3
41
0
10496
41
3
0x041400
4
20
0
5120
20
6
0x0B1400
11
20
0
5120
20
9
0x0C1400
12
20
0
5120
20
12
0x0D0F00
13
15
0
3840
15
15
0x177800
23
120
0
30720
120
18
0x224600
34
70
0
17920
70
21
0x467800
70
120
0
30720
120
24
0x473C00
71
60
0
15360
60
27
0x550F00
85
15
0
3840
15
30
0x000000
0
0
0
0
0
33
0x000000
0
0
0
0
0
36
0x000000
0
0
0
0
0
See this db<>fiddle.
DECLARE #YourTable table
(
Id INT PRIMARY KEY,
Val VARBINARY(50)
)
INSERT #YourTable
VALUES (1, 0x0329000414000B14000C14000D0F00177800224600467800473C00550F00000000000000000000000000),
(2, 0x0329002637000B14000C14000D0F00177800224600467800473C00550F00000000000000000000000000);
SELECT Id, Triplet
FROM #YourTable T
CROSS APPLY GENERATE_SERIES(1,DATALENGTH(T.Val),3) s
CROSS APPLY (VALUES (SUBSTRING(T.Val, s.value, 3))) V(Triplet)
WHERE Triplet BETWEEN 0x263700 AND 0x2637FF
This works only with '22 sql server because of 'generate_series'
DECLARE #YourTable table
(
Id INT PRIMARY KEY,
Val VARBINARY(50)
)
INSERT #YourTable
VALUES (1, 0x0329000414000B14000C14000D0F00177800224600467800473C00550F00000000000000000000000000),
(2, 0x0329002637000B14000C14000D0F00177800224600467800473C00550F00000000000000000000000000);
SELECT Id, Triplet
FROM #YourTable T
JOIN (VALUES (1),(4),(7),(10),(13),(16),(19),(22),(25),(28),(31),(34),(37),(40),(43),(46),(49)) Nums(Num) ON Num <= DATALENGTH(T.Val)
CROSS APPLY (VALUES (SUBSTRING(T.Val, Num, 3))) V(Triplet)
WHERE Triplet BETWEEN 0x263700 AND 0x2637FF
This one works on older versions without "generate_series"
The credit is to #Martin Smith from stackexchange
https://dba.stackexchange.com/questions/323235/varbinary-pattern-search
The int value is '1192060800000000' then following select statement works properly ie. length 16
SELECT DATEADD(day,clinicaldate/1000000/3600/24,'01-01-1960') FROM document
Refered - https://www.qvera.com/kb/index.php/743/can-change-clinicaldate-from-centricity-document-into-date
If the length is greater than or equal to '18' and having (-) minus sign at the beginning of the int (-58635680520000000)
then it gives the following error-
'Adding a value to a 'datetime' column caused an overflow'
I would like to know how I can ge the result for 18 digit int or how can we exclude such records in the query?
cast(clinicaldate as varchar(20)), use substring() to extract the section you actually want to work with, then do your math.
I am trying to get last numeric part in the given string.
For Example, below are the given strings and the result should be last numeric part only
SB124197 --> 124197
287276ACBX92 --> 92
R009321743-16 --> 16
How to achieve this functionality. Please help.
Try this:
select right(#str, patindex('%[^0-9]%',reverse(#str)) - 1)
Explanation:
Using PATINDEX with '%[^0-9]%' as a search pattern you get the starting position of the first occurrence of a character that is not a number.
Using REVERSE you get the position of the first non numeric character starting from the back of the string.
Edit:
To handle the case of strings not containing non numeric characters you can use:
select case
when patindex(#str, '%[^0-9]%') = 0 then #str
else right(#str, patindex('%[^0-9]%',reverse(#str)) - 1)
end
If your data always contains at least one non-numeric character then you can use the first query, otherwise use the second one.
Actual query:
So, if your table is something like this:
mycol
--------------
SB124197
287276ACBX92
R009321743-16
123456
then you can use the following query (works in SQL Server 2012+):
select iif(x.i = 0, mycol, right(mycol, x.i - 1))
from mytable
cross apply (select patindex('%[^0-9]%', reverse(mycol) )) as x(i)
Output:
mynum
------
124197
92
16
123456
Demo here
Here is one way using Patindex
SELECT RIGHT(strg, COALESCE(NULLIF(Patindex('%[^0-9]%', Reverse(strg)), 0) - 1, Len(strg)))
FROM (VALUES ('SB124197'),
('287276ACBX92'),
('R009321743-16')) tc (strg)
After reversing the string, we are finding the position of first non numeric character and extracting the data from that position till the end..
Result :
-----
124197
92
16
hie am trying to select the integer value before the char C in my SQL database table which contains the information below.
240mm2 X 15C WIRING CABLE
150mm2 X 3C flex
10mm2 x 4C swa
so far i have used the query
select left ('C',CHARINDEX ('C',product_name)) from product
and i get 'C' on my results which is correct. Now am stuck does anyone know how i can modify the above select query to get a result which only lists the integers for eg
15
3
4
Two observations: the integer before "C" has a space before it and there is no space between the integer and "C".
If these are generally true, then you can do what you want using substring_index():
select substring_index(substring_index(product_name, 'C', 1), ' ', -1) + 0 as thenumber
The + 0 simply converts the value to a number.
If you're doing this in SQL Server you could try the following:
Select Substring(product_name,
PATINDEX('% [0-9]%',product_name) + 1,
PATINDEX('%[0-9]C%',product_name) - PATINDEX('% [0-9]%',product_name)
) as num
from Product
This assumes that there is a space before the number and always a C after the number with no space.
It works out the starting point and then the length based on the start and end and performs a substring with the results.
You could use a combination of instring and substring.
First get the position of the C
Then substring till C
It goes like this:
SELECT INSTR('foobarbar', 'bar');
= 4
And then you select substring from 1 to 4.
I have a SQL entry that is of type hex (varbinary) and I want to do a SELECT COUNT for all the entries that have this hex value ending in 1.
I was thinking about using CONVERT to make my hex into a char and then use WHERE my_string LIKE "%1". The thing is that varchar is capped at 8000 chars, and my hex is longer than that.
What options do I have?
Varbinary actually works with some string manipulation functions, most notably substring. So you can use eg.:
select substring(yourBinary, 1, 1);
To get the first byte of your binary column. To get the last bit then, you can use this:
select substring(yourBinary, len(yourBinary), 1) & 1;
This will give you zero if the bit is off, or one if it is on.
However, if you really only have to check at most the last 4-8 bytes, you can easily use the bitwise operators on the column directly:
select yourBinary & 1;
As a final note, this is going to be rather slow. So if you plan on doing this often, on large amounts of data, it might be better to simply create another bit column just for that, which you can index. If you're talking about at most a thousand rows or so, or if you don't care about speed, fire away :)
Check last four bits = 0001
SELECT SUM(CASE WHEN MyColumn % 16 IN (-15,1) THEN 1 END) FROM MyTable
Check last bit = 1
SELECT SUM(CASE WHEN MyColumn % 2 IN (-1,1) THEN 1 END) FROM MyTable
If you are wondering why you have to check for negative moduli, try SELECT 0x80000001 % 16
Try using this where
WHERE LEFT(my_string,1) = 1
It it's text values ending in 1 then you want the Right as opposed to the Left
WHERE RIGHT(my_string,1) = 1