Null bitmap is set when there is no null columns - sql-server

I am learning PAGE structure and currently I'm stuck at the NULL bitmaps.
create table dbo.ro
(
ID int not null,
Col1 varchar(8000) null,
Col2 varchar(8000) null
);
insert into dbo.ro(ID, Col1, Col2) values
(1,replicate('a',8000),replicate('b',8000));
So currently there is no NULL values, let's see DBCC info:
DBCC IND(test, 'ro', 1);
DBCC PAGE('test',1, 408,3);
So I am interested in the following part
30000800 01000000 03005002
30 - BitsA
00 - BitsB
0800 - Fdata length
01000000 - Fixed data (ID = 1)
0300 - number of columns
50 - NULL bitmap
Why is it 50 and not 00? There is no NULL values in the record ...

The correct answer was in the comment and I upvoted it, but you still have a question so maybe I should explain you what does it mean.
The bottom three bits are 0. The other bits should be ignored.
If you expand 50 into a binary you've got 01010000.
The only bits of interest are the bottom three bits, these correspond to 3,2,1 columns that are not null. The other bits should be ignored means the server knows the number of colums, it's 3, and it cares only about 3 bits in this mask. Other bits are not set, for that they should be ignored. They just contain a garbage.

Related

Varbinary search with min and max pattern type

So I'm trying to make a query which goes through varbinary data. The issue is that I can't really finish what I'm trying to achieve. What you should know about the column is varbinary(50) and the patterns that occur have no specific order in writing, meaning every prefix could be anywhere as long it has 3 bytes(0x000000) First one is the prefix second and third are value data that I'm looking to check if its within the range i like. All the data is written like this.
What I've tried:
DECLARE #t TABLE (
val VARBINARY(MAX)
)
INSERT INTO #t SELECT 0x00000100000000000000000000000000000000000000000000000000
INSERT INTO #t SELECT 0x00001000000000000000000000000000000000000000000000000000
INSERT INTO #t SELECT 0x00010000000000000000000000000000000000000000000000000000
INSERT INTO #t SELECT 0x00100000000000000000000000000000000000000000000000000000
INSERT INTO #t SELECT 0x00000f00000000000000000000000000000000000000000000000000
declare #pattern varbinary(max)
declare #pattern2 varbinary(max)
set #pattern = 0x0001
set #pattern2 = #pattern+0xFF
select #pattern,#pattern2
SELECT
*
FROM #t
WHERE val<#pattern
OR val>#pattern2
This was total bust the patterns were accurate up to 2 symbols if I were to use 4 symbols as pattern it would work only if the pattern is in predefined position. I've tried combination of this and everything below.
WHERE CONVERT(varbinary(2), val) = 0xdata
also this:
select *
from table
where CONVERT(varchar(max),val,2) like '%data%'
Which works great for searching exact patterns, but not for ranges, I need some combination of both.
I'm aware I could technically add every possible outcome 1 by 1 and let it cycle through all the listed possibilities, but there has to be a smarter way.
Goals:
Locating the prefix(first binary data pair)
Defining a max value after the prefix, everything above that threshold to be listed in the results. Let's say '26' is the prefix, the highest allowed number after is '9600' or '269600'. Basically any data that exceeds this pattern '269600' should be detected example '269700'.
or query result would post this:
select * from table where CONVERT(varchar(max),attr,2) like
'%269700%'
I need something that would detect this on its own while i just give it start and end to look in between like the highest number variation would be '26ffff', but limiting it to something like 'ff00' is acceptable for what I'm looking for.
My best guess is 2 defined numbers, 1 being the allowed max range
and 2nd for a cap, so it doesn't go through every possible outcome.
But I would be happy to whatever works.
I'm aware this explanation is pretty dire, but bear with me, thanks.
*Update after the last suggestion
SELECT MIN(val), MAX(val) FROM #t where CONVERT(varchar(max),val,2) like '%26%'
This is pretty close, but its not sufficient i need to cycle through alot of data and use it after this would select only min or max even with the prefix filter. I believe i need min and max defined as a start and end range where the query should look for.
**Update2
I'm afraid you would end up disappointed, its nothing that interesting.
The data origin is related to a game server which stores the data like this. There's the predefined prefixes which are the stat type and the rest of the data is the actual numeric value of the stat. The data is represented by 6 characters data intervals. Here is a sample of the data stream. Its always 6-6-6-6-6 as long there's space to record the data on since its capped at 50 characters.
0x0329000414000B14000C14000D0F00177800224600467800473C00550F00000000000000000000000000
**Update3
Yes the groups are always in 3byte fashion, yes my idea was exactly that use the first byte to narrow down the search and use then use the second 2 bytes to filter it. I just don't know how to pull it off in an effective way. I'm not sure if i understood what u meant by predictively aligned assuming you meant if stat/prefix/header would always end up at the same binary location, if that's correct the answer is no. If the 3byte pattern is violated the data becomes unreadable meaning even if you don't need the extra byte you have to count it otherwise the data breaks example of a working data.
0x032900'041400'
example of a broken data:
0x0329'041400'
The only issue i could think is when the prefix and part of the value are both true example:
0x262600
Unless the query is specifically ordered to read the data in 3byte sequence meaning it knows that the first byte is always a prefix and the other 2 bytes are value.
Q:Can that be used as an alignment indicator so that the first non-zero byte after at least 3 zero bytes indicates the start of a group?
A:Yes, but that's unlikely I mean it although possible it would be written in order like:
0x260000'270000'
It wouldn't skip forward an entire 3byte group filled with no data. This type of entry would occur if someone were to manually insert it to the db, the server doesn't make records with gaps like those as far I'm aware:
0x260000'000000'270000'
To address your last comment that's something I don't know how to express it in a working query, except for the boneheaded version which would be me manually adding every possible number within my desired range with +1bit after that number. As you can imagine the query would look terrible. That's why I'm looking for a smarter solution that I cannot figure out how to do so by my self.
select * from #t
where (CONVERT(varchar(max),val,2) like '%262100%' or
CONVERT(varchar(max),attr,2) like '%262200%' or
etc...)
This may be a partial answer from which you can build.
The following will split the input data up into 3-byte (6 hex character) groups. It then extracts the first byte as the key, and several representations of the remaining two bytes as values.
SELECT S.*, P.*
FROM #t T
CROSS APPLY (
SELECT
N.Offset,
SUBSTRING(T.val, N.Offset + 1, 3) AS Segment
FROM (
VALUES
(0), (3), (6), (9), (12), (15), (18), (21), (24), (27),
(30), (33), (36), (39)
) N(Offset)
WHERE N.Offset < LEN(T.val) - 3
) S
CROSS APPLY(
SELECT
CONVERT(TINYINT, SUBSTRING(S.Segment, 1, 1)) AS [Key],
CONVERT(TINYINT, SUBSTRING(S.Segment, 2, 1)) AS [Value1],
CONVERT(TINYINT, SUBSTRING(S.Segment, 3, 1)) AS [Value2],
CONVERT(SMALLINT, SUBSTRING(S.Segment, 2, 2)) AS [Value12],
CONVERT(SMALLINT, SUBSTRING(S.Segment, 3, 1) + SUBSTRING(S.Segment, 2, 1)) AS [Value21]
) P
Given the following input data
0x0329000414000B14000C14000D0F00177800224600467800473C00550F00000000000000000000000000
--^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----
The following results are extracted:
Offset
Segment
Key
Value1
Value2
Value12
Value21
0
0x032900
3
41
0
10496
41
3
0x041400
4
20
0
5120
20
6
0x0B1400
11
20
0
5120
20
9
0x0C1400
12
20
0
5120
20
12
0x0D0F00
13
15
0
3840
15
15
0x177800
23
120
0
30720
120
18
0x224600
34
70
0
17920
70
21
0x467800
70
120
0
30720
120
24
0x473C00
71
60
0
15360
60
27
0x550F00
85
15
0
3840
15
30
0x000000
0
0
0
0
0
33
0x000000
0
0
0
0
0
36
0x000000
0
0
0
0
0
See this db<>fiddle.
DECLARE #YourTable table
(
Id INT PRIMARY KEY,
Val VARBINARY(50)
)
INSERT #YourTable
VALUES (1, 0x0329000414000B14000C14000D0F00177800224600467800473C00550F00000000000000000000000000),
(2, 0x0329002637000B14000C14000D0F00177800224600467800473C00550F00000000000000000000000000);
SELECT Id, Triplet
FROM #YourTable T
CROSS APPLY GENERATE_SERIES(1,DATALENGTH(T.Val),3) s
CROSS APPLY (VALUES (SUBSTRING(T.Val, s.value, 3))) V(Triplet)
WHERE Triplet BETWEEN 0x263700 AND 0x2637FF
This works only with '22 sql server because of 'generate_series'
DECLARE #YourTable table
(
Id INT PRIMARY KEY,
Val VARBINARY(50)
)
INSERT #YourTable
VALUES (1, 0x0329000414000B14000C14000D0F00177800224600467800473C00550F00000000000000000000000000),
(2, 0x0329002637000B14000C14000D0F00177800224600467800473C00550F00000000000000000000000000);
SELECT Id, Triplet
FROM #YourTable T
JOIN (VALUES (1),(4),(7),(10),(13),(16),(19),(22),(25),(28),(31),(34),(37),(40),(43),(46),(49)) Nums(Num) ON Num <= DATALENGTH(T.Val)
CROSS APPLY (VALUES (SUBSTRING(T.Val, Num, 3))) V(Triplet)
WHERE Triplet BETWEEN 0x263700 AND 0x2637FF
This one works on older versions without "generate_series"
The credit is to #Martin Smith from stackexchange
https://dba.stackexchange.com/questions/323235/varbinary-pattern-search

Update one certain bit in a bitmask field (SQL Server)

I have an int column containing 8 bits in my SQL Server database. How can I update certain bit without affecting others?
For example, I have a value
11010000
and I want to set bit1 and bit2 to 1, so it would become
11010011
Looked through bitwise operators but couldn't find proper solution.
My goal is not only to update certain bit, but also to avoid database locks.
So when transaction1 updates bit1 in a certain record, another transaction2 could update bit2 in the same field of the same record at the same time.
Is this possible? Or is using 8 separate bit columns the only way?
What you are looking for are the Bitwise operations. To always turn on the proper bit, use a bitwise OR. So, to turn on bits 1 and 2 (total value = 3), use an statement like this (assumes the value in #value is 208, or 11010000 binary):
SET #value = #value | 3
-- or, the alternate form
SET #value |= 3
Other operators are the Bitwise AND (SET #value = #value & 3), and the bitwise NOT (SET #Value = #value ~ 3), and the Bitwise XOR (SET #value = #value ^ 3).
That said, having eight bit fields is easier logically for a new programmer. I don't need to find something special to see that the field ShowCurrencySymbol is the flag for showing a currency symbol, as opposed to finding out what the fifth bit in a byte does. And, since the fields are compacted internally so that eight one-bit, non-nullable fields = one byte of space used (adding NULL takes two bits per bit).
Finally, you can't have two transactions simultaneously update a field on the same row. While one update is occurring, the row will be locked, preventing the other update from processing. If you really want something like this, you will have to use a much more expansive method - a separate BitValues or Flags table, something like this:
CREATE TABLE Flags (
RowID int not null,
FlagName varchar(16) Not Null,
BitValue bit not null,
CONSTRAINT PK_Flags PRIMARY KEY (RowID, FlagName)
);
Then you read and write your flags from this table separate from the row.
A pseudo-bitset value is when you have an integer or string that, when selected or printed, looks like a bitset. Something like this:
DECLARE #bits varchar(8) = '1010'
SELECT Right(8, REPLICATIE('0', 8) + #bits
Value returned: 00001010
For this type of bitset, you need to remember that you are not setting the bits, but are representing the bits instead, and you need to set the representation of the bits in the same way. For example, the function below can be used to set bits in a string representation of your bitstring:
CREATE FUNCTION dbo.SetBitInString
(
#Source char(8),
#Position int,
#Action char(1) = 'S',
#Value bit = NULL
)
RETURNS Char(8)
AS
BEGIN
DECLARE #work Char(8) = '00000000'
SET #work = Right(#work + ISNULL(LTRIM(RTRIM(#Source)), ''), 8)
SET #Work =
CASE #Action
WHEN 'S'-- Set
THEN Stuff(#Work, #Position, 1, '1')
WHEN 'R' -- Reset
THEN Stuff(#Work, #Position, 1, '0')
WHEN 'X' -- XOR value with position
THEN STUFF(#Work, #Position, 1, CAST(CAST(SubString(#Work, #Position, 1) as int) ^ ISNULL(#Value, 0) as CHAR(1)))
WHEN 'Q'
THEN '1'
---- add other options as needed - this is a quick example.
ELSE #Action
END
IF (#Action = #Work)
RAISERROR('Bad Action (%s) Submitted', 13, 1, #Action)
RETURN #Work
END
Reading a bit is a simple SUBSTRING. Constants can be defined for the meaning of each bit (e.g. DECLARE #ShowTotalPrice = 4 -- The 4th bit is the Show Total Price flag)
This should give you enough to go on if you want to use this style of setting, where the displayed value is your bitset represented in 1s and 0s.

datetime2 storage size

Using SQL Server 2008 R2, SP2
The docs says that datetime2 takes 6, 7 or 8 bytes depending witch precision you use
I need to store a large amount of data in binary form (concatenated values) and I love the idea of using only 6 bytes for each datetime, however when I try:
declare #_dt_p0 datetime2(0) = '2012-05-18 11:22:33'
select CONVERT(varbinary, #_dt_p0), LEN(CONVERT(varbinary, #_dt_p0))
declare #_dt_p4 datetime2(4) = '2012-05-18 11:22:33'
select CONVERT(varbinary, #_dt_p4), LEN(CONVERT(varbinary, #_dt_p4))
declare #_dt_p7 datetime2(7) = '2012-05-18 11:22:33'
select CONVERT(varbinary, #_dt_p7), LEN(CONVERT(varbinary, #_dt_p7))
It's clearly taking one extra byte, what I'm doing wrong?
I don't think I can explain why the length / datalength of a varbinary conversion is 7 instead of 6 (Mikael later found that the convert to varbinary adds the precision as an extra byte), but I don't know why you think that's a valid test anyway. I can confirm that 6 bytes are stored on the page when you are using an actual column (though null overhead for the row will be different depending on whether the column is nullable). How can I prove this?
USE tempdb;
GO
CREATE TABLE dbo.x
(
d1 DATETIME2(0) NULL,
v1 VARBINARY(32) NULL,
d2 DATETIME2(0) NOT NULL,
v2 VARBINARY(32) NOT NULL
);
declare #d datetime2(0) = '2012-05-18 11:22:33';
INSERT dbo.x(d1, v1, d2, v2)
SELECT #d, CONVERT(VARBINARY(32), #d), #d, CONVERT(VARBINARY(32), #d);
SELECT DATALENGTH(d1), DATALENGTH(v1),
DATALENGTH(d2), DATALENGTH(v2) FROM dbo.x;
Results:
6 7 6 7
So, the datetime2 columns are 6 bytes, but the varbinary columns are 7 bytes. Regardless of nullability. We can look closer by actually inspecting the page. Let's find all the pages in the heap for this table:
DBCC IND('tempdb', 'dbo.x', 0);
Partial results on my system (yours will be different):
PagePID PageType
283 10
311 1
So now let's look at Page 311:
DBCC TRACEON(3604, -1);
DBCC PAGE(2, 1, 311, 3);
And we can see that the datetime2 columns indeed occupy 6 bytes on the page:
Slot 0 Column 1 Offset 0x4 Length 6 Length (physical) 6
d1 = 2012-05-18 11:22:33
v1 = [Binary data] Slot 0 Column 2 Offset 0x19 Length 7 Length (physical) 7
v1 = 0x00f99f00b0350b
Slot 0 Column 3 Offset 0xa Length 6 Length (physical) 6
d2 = 2012-05-18 11:22:33
v2 = [Binary data] Slot 0 Column 4 Offset 0x20 Length 7 Length (physical) 7
v2 = 0x00f99f00b0350b
Good day,
firstly i want to say that this is the best question : "I don't know why you think that's a valid test anyway", since the answer is that this is not valid test!
You can read all about the issue, including the explanation of DateTime2 real stored format, and why this is mistake to examine the result of "CONVERT to binary" and assume that this is the same as the actual stored data! It is not accurate in variable-length data like varchar or nvarchar and it is not accurate in DateTime2 as well. The only way to examine and get the real stored format is examine the data itself using DBCC PAGE.
http://ariely.info/Blog/tabid/83/EntryId/162/Examine-how-DateTime2-type-stored-in-the-data-file.aspx
I hope this is useful :-)

REAL column holding values outside documented range

According to MSDN, the range for REAL values is - 3.40E + 38 to -1.18E - 38, 0 and 1.18E - 38 to 3.40E + 38. However, I have quite a few values beyond that range in my table.
The following query returns lots of very small values and no very large ones:
SELECT MyColumn ,
*
FROM data.MyTable
WHERE MyColumn <> 0
AND ( MyColumn < CONVERT(REAL, 1.18E-38)
OR MyColumn > CONVERT(REAL, 3.40E+38)
)
AND ( MyColumn < CONVERT(REAL, -3.40E+38)
OR MyColumn > CONVERT(REAL, -1.18E-38)
)
It is easy to show how these values end up in the table. I cannot insert them directly:
CREATE TABLE a(r REAL NULL);
GO
INSERT INTO a(r) VALUES(4.330473E-39);
GO
SELECT r FROM a
GO
DROP TABLE a;
----
0.0
But I can divide two columns and get and outside of range value:
CREATE TABLE a
(
r1 REAL NULL ,
r2 REAL NULL ,
r3 REAL NULL
) ;
GO
INSERT INTO a
( r1, r2 )
VALUES ( 4.330473E-38, 1000 ) ;
GO
UPDATE a
SET r3 = r1 / r2 ;
SELECT r1 ,
r2 ,
r3
FROM a
r1 r2 r3
------------- ------------- -------------
4.330473E-38 1000 4.330433E-41
So I guess MSDN gives wrong ranges of valid data, correct?
Am I missing anything?
Several people suggested that this is a bug.
What part of this behavior exactly is a bug. is it:
Wrong constants documented in MSDN and used in DBCC, as well as wrong threshold for rounding down.
Update being able to save wrong values
Books Online documents only the normal range for single- and double-precision floating point numbers. The IEEE 754 rules also specify floating-point numbers closer to zero than the smallest non-zero normal value, known variously as denormalized, denormal, and subnormal numbers. From that last link:
Denormal numbers provide the guarantee that addition and subtraction of floating-point numbers never underflows; two nearby floating-point
numbers always have a representable non-zero difference. Without
gradual underflow, the subtraction a−b can underflow and produce zero
even though the values are not equal. This can, in turn, lead to
division by zero errors that cannot occur when gradual underflow is
used.
SQL Server is following the rules for single-precision floating point calculations in the examples posted. The bug may be that DBCC checks only for normal values, and throws an incorrect error message when it encounters a stored denormal value.
Example producing a denormal single-precision value:
DECLARE
#v1 real = 14e-39,
#v2 real = 1e+07;
-- 1.4013e-045
SELECT #v1 / #v2;
Example showing a stored float denormal passes DBCC checks:
CREATE TABLE dbo.b (v1 float PRIMARY KEY);
INSERT b VALUES (POWER(2e0, -1075));
SELECT v1 FROM b; -- 4.94065645841247E-324
DBCC CHECKTABLE(b) WITH DATA_PURITY; -- No errors or warnings
DROP TABLE dbo.b;
This is a bug in SQL Server. The last script you post is a nice repro. Add one line to it at the end:
DBCC CHECKDB WITH data_purity
This fails with:
Msg 2570, Level 16, State 3, Line 1 Page (1:313), slot 0 in object ID
357576312, index ID 0, partition ID 1801439851932155904, alloc unit ID
2017612634169999360 (type "In-row data"). Column "r3" value is out of
range for data type "real". Update column to a legal value.
This proves it is a bug. I suggest you file a bug with Microsoft Connect for SQL Server.

Should I use an inline varchar(max) column or store it in a separate table?

I want to create a table in MS SQL Server 2005 to record details of certain system operations. As you can see from the table design below, every column apart from Details is is non nullable.
CREATE TABLE [Log]
(
[LogID] [int] IDENTITY(1,1) NOT NULL,
[ActionID] [int] NOT NULL,
[SystemID] [int] NOT NULL,
[UserID] [int] NOT NULL,
[LoggedOn] [datetime] NOT NULL,
[Details] [varchar](max) NULL
)
Because the Details column won't always have data in it. Is it more efficient to store this column in a separate table and provide a link to it instead?
CREATE TABLE [Log]
(
[LogID] [int] IDENTITY(1,1) NOT NULL,
[ActionID] [int] NOT NULL,
[SystemID] [int] NOT NULL,
[UserID] [int] NOT NULL,
[LoggedOn] [datetime] NOT NULL,
[DetailID] [int] NULL
)
CREATE TABLE [Detail]
(
[DetailID] [int] IDENTITY(1,1) NOT NULL,
[Details] [varchar](max) NOT NULL
)
For a smaller data type I wouldn't really consider it, but for a varchar(max) does doing this help keep the table size smaller? Or I am just trying to out smart the database and achieving nothing?
Keep it inline. Under the covers SQL Server already stores the MAX columns in a separate 'allocation unit' since SQL 2005. See Table and Index Organization. This in effect is exactly the same as keeping the MAX column in its own table, but w/o any disadvantage of explicitly doing so.
Having an explicit table would actually be both slower (because of the foreign key constraint) and consume more space (because of the DetaiID duplication). Not to mention that it requires more code, and bugs are introduced by... writing code.
alt text http://i.msdn.microsoft.com/ms189051.3be61595-d405-4b30-9794-755842d7db7e(en-us,SQL.100).gif
Update
To check the actual location of data, a simple test can show it:
use tempdb;
go
create table a (
id int identity(1,1) not null primary key,
v_a varchar(8000),
nv_a nvarchar(4000),
m_a varchar(max),
nm_a nvarchar(max),
t text,
nt ntext);
go
insert into a (v_a, nv_a, m_a, nm_a, t, nt)
values ('v_a', N'nv_a', 'm_a', N'nm_a', 't', N'nt');
go
select %%physloc%%,* from a
go
The %%physloc%% pseudo column will show the actual physical location of the row, in my case it was page 200:
dbcc traceon(3604)
dbcc page(2,1, 200, 3)
Slot 0 Column 2 Offset 0x19 Length 3 Length (physical) 3
v_a = v_a
Slot 0 Column 3 Offset 0x1c Length 8 Length (physical) 8
nv_a = nv_a
m_a = [BLOB Inline Data] Slot 0 Column 4 Offset 0x24 Length 3 Length (physical) 3
m_a = 0x6d5f61
nm_a = [BLOB Inline Data] Slot 0 Column 5 Offset 0x27 Length 8 Length (physical) 8
nm_a = 0x6e006d005f006100
t = [Textpointer] Slot 0 Column 6 Offset 0x2f Length 16 Length (physical) 16
TextTimeStamp = 131137536 RowId = (1:182:0)
nt = [Textpointer] Slot 0 Column 7 Offset 0x3f Length 16 Length (physical) 16
TextTimeStamp = 131203072 RowId = (1:182:1)
All column values but the TEXT and NTEXT were stored inline, including the MAX types.
After changing the table options and insert a new row (sp_tableoption does not affect existing rows), the MAX types were evicted into their own storage:
sp_tableoption 'a' , 'large value types out of row', '1';
insert into a (v_a, nv_a, m_a, nm_a, t, nt)
values ('2v_a', N'2nv_a', '2m_a', N'2nm_a', '2t', N'2nt');
dbcc page(2,1, 200, 3);
Note how m_a and nm_a columns are now a Textpointer into the LOB allocation unit:
Slot 1 Column 2 Offset 0x19 Length 4 Length (physical) 4
v_a = 2v_a
Slot 1 Column 3 Offset 0x1d Length 10 Length (physical) 10
nv_a = 2nv_a
m_a = [Textpointer] Slot 1 Column 4 Offset 0x27 Length 16 Length (physical) 16
TextTimeStamp = 131268608 RowId = (1:182:2)
nm_a = [Textpointer] Slot 1 Column 5 Offset 0x37 Length 16 Length (physical) 16
TextTimeStamp = 131334144 RowId = (1:182:3)
t = [Textpointer] Slot 1 Column 6 Offset 0x47 Length 16 Length (physical) 16
TextTimeStamp = 131399680 RowId = (1:182:4)
nt = [Textpointer] Slot 1 Column 7 Offset 0x57 Length 16 Length (physical) 16
TextTimeStamp = 131465216 RowId = (1:182:5)
For completion sakeness we can also force the one of the non-max fields out of row:
update a set v_a = replicate('X', 8000);
dbcc page(2,1, 200, 3);
Note how the v_a column is stored in the Row-Overflow storage:
Slot 0 Column 1 Offset 0x4 Length 4 Length (physical) 4
v_a = [BLOB Inline Root] Slot 0 Column 2 Offset 0x19 Length 24 Length (physical) 24
Level = 0 Unused = 99 UpdateSeq = 1
TimeStamp = 1098383360
Link 0
Size = 8000 RowId = (1:176:0)
So, as other have already commented, the MAX types are stored inline by default, if they fit. For many DW projects this would be unnacceptable because the typical DW loads must scan or at least range scan, so the sp_tableoption ..., 'large value types out of row', '1' should be used. Note that this does not affect existing rows, in my test not even on index rebuild, so the option has to be turned on early.
For most OLTP type loads though the fact that MAX types are stored inline if possible is actually an advantage, since the OLTP access pattern is to seek and the row width makes little impact on it.
None the less, regarding the original question: separate table is not necessary. Turning on the large value types out of row option achieves the same result at a free cost for development/test.
Paradoxically, if your data is normally less than 8000 characters, I would store it in a separate table, while if the data is greater than 8000 characters, I would keep it in the same table.
This is because what happens is that SQL Server keeps the data in the page if it allows the row to sit in single page, but when the data gets larger, it moves it out just like the TEXT data type and leaves just a pointer in the row. So for a bunch of 3000 character rows, you are fitting less rows per page, which is really inefficient, but for a bunch of 12000 character rows, the data is out of the row, so it's actually more efficient.
Having said this, typically you have a wide ranging mix of lengths and thus I would move it into its own table. This gives you flexibility for moving this table to a different file group etc.
Note that you can also specify it to force the data out of the row using the sp_tableoption. varchar(max) is basically similar to the TEXT data type with it defaulting to data in row (for varchar(max)) instead of defaulting to data out of row (for TEXT).
You should structure your data into whatever seems the most logical structure and allow SQL Server to perform its optimizations as to how to physically store the data.
If you find, through performance analysis, that your structure is a performance problem, then consider performing changes to your structure or to storage settings.
Keep it inline. The whole point of varchar is that it takes up 0 bytes if it's empty, 4 bytes for 'Hello', and so on.
I would normalize it by creating the Detail table. I assume some of the entries in Log will have the same Detail? So if you normalize it you will only be storing an FK id INTEGER instead of the text for every occurrence if you stored the text on the Detail table. If you have reasons to de-normalize do it, but from your question I don't see that being the case.
Having a nullable column costs 2 bytes for every 16 of them. If this is the only (or 17th, or 33nd, etc) nullable column in the table, it will cost you 2 bytes per row, otherwise nothing.

Resources