hopefully a simple question to settle my intrigue more than anything. I have setup Filestream on our SQL Server. From what I have read I was expecting to see the GUID in the database row match the filestream file name in the file path... but they dont match. Is there any other wizardry going on that I am missing?
Database Table:
where column id is the GUID and FileData is the Filestream column.
When I then go to the location where these BLOBS are being stored, I expect to see these GUIDs as the filenames:
Filestream Files
I am just looking to understand the whole process of how Filestream works. I have done a bit of digging around so if anyone is able to fill in those gaps for me would be great.
It is a bit confusing. But below is what I eventually figured out.
Notice the column circled in red above. The type and name may be different.
DECLARE #BaseDirectory AS varchar(200) = '\\server-name\Instance\DocumentPages\';
SELECT TOP(10) #BaseDirectory + SUBSTRING(CONVERT(varchar(200), file_stream.PathName(0)), LEN('file_stream\') + CHARINDEX('file_stream\', CONVERT(varchar(200), file_stream.PathName(0)), 1), 36) + '.' + dp.file_type, d.DocumentID
FROM [Document].[dbo].[DocumentPages] AS dp
INNER JOIN [Document].[dbo].[Documents] AS d
ON d.stream_id = dp.stream_id
ORDER BY d.DocumentID DESC
This query uses a function called .PathName() on this column. This isn't how I have known SQL to typically work (usually, functions take the column value in as a parameter). But it does!
Because our system is using Availability Groups, I had to find the base directory and ignore what SQL Server was returning.
You may also try file_stream.PathName(0), file_stream.PathName(1), or file_stream.PathName(2) to try and find the base path. And if the path matches, remove the SUBSTRING logic!
Hopefully Helpful!
Related
Could anyone tell me how to change the Stored Procedure in the article below to recursively expand all the attributes of a json file (multiple JSON document schemas)?
https://support.snowflake.net/s/article/Automating-Snowflake-Semi-Structured-JSON-Data-Handling-part-2
Craig Warman's stored procedure posted in that blog is a great idea. I asked him if it was okay to refactor his code, and he agreed. I've used the refactored version in the field, so I know the SP well as well as how it works.
It may be possible to modify the SP to work on your JSON. It will depend on whether or not Snowflake types the JSON in your variant column. The way you have it structured, it may not type everything. You can check by running this SQL and seeing if the result set includes all the columns you need:
set VARIANT_TABLE = 'WEATHER';
set VARIANT_COLUMN = 'V';
with MAIN_TABLE as
(
select * from identifier($VARIANT_TABLE) sample (1000 rows)
)
select distinct REGEXP_REPLACE(REGEXP_REPLACE(f.path, '\\[(.+)\\]'),'[^a-zA-Z0-9]','_') AS path_name, -- This generates paths with levels enclosed by double quotes (ex: "path"."to"."element"). It also strips any bracket-enclosed array element references (like "[0]")
typeof(f.value) AS attribute_type, -- This generates column datatypes.
path_name AS alias_name -- This generates column aliases based on the path
from
MAIN_TABLE,
LATERAL FLATTEN(identifier($VARIANT_COLUMN), RECURSIVE=>true) f
where TYPEOF(f.value) != 'OBJECT'
AND NOT contains(f.path, '[');
Be sure to replace the variables to your table and column names. If this picks up the type information for the columns in your JSON, then it's possible to modify this SP to do what you need. If it doesn't but there's a way to modify the query to get it to pick up the columns, that would work too.
If it doesn't pick up the columns, based on Craig's idea I decided to write type inference for non variant (such as strings from CSV log files without type information). Try the SQL above and see what results first.
There is a legacy application that uses a table to translate job names to filenames. This legacy application queries it as follows:
SELECT filename FROM aJobTable WHERE jobname = 'myJobName'
But in reality those jobnames always match the filenames (e.g. 'myJobName.job' is the jobname but also the filename) That makes this table appear unnecessary. But unfortunately, we cannot change the code of this program, and the program just needs to select it from a table.
That's actually a bit annoying. Because we do need to keep this database in sync. If a jobname is not in the table, then it cannot be used. So, as our only way out, right now we have some vbscripts to synchronize this table, adding records for each possible filename. As a result, the table just 2 columns with identical values. -- We want to get rid of this.
So, we have been dreaming about some hack that queries the data with the jobname, but just always returns the jobname again, like a copy/mirror query. Then we don't actually have to populate a table at all.
"Exploits"
The following can be configured in this legacy application. My hunch is that these may open the door for some tricks/hacks.
use of either MS Access or SQL Server (we prefer sql server)
The name of the table (e.g. aJobTable)
The name of the filename column (e.g. filename)
The name of the jobname column (e.g. jobname)
Here is what I came up with:
If I create a table-valued function mirror(a) then I get pretty close to what I want. Then I could use it like
SELECT filename FROM mirror('MyJobName.job')
But that's just not good enough, it would be if I could force it to be like
SELECT filename FROM mirror WHERE param1 = 'MyJobName.job'
Unfortunately, I don't think it's possible to call functions like that.
So, I was wondering if perhaps somebody else knows how to get it working.
So my question is: "How can you create a table (or other object) that always returns the value passed to its WHERE-clause, like a mirror."
It's kinda hard to answer not knowing the code that the application use, but if we assume it only takes strings and concatenate them without any tests whatsoever, I would assume code like this: (translated to c#)
var sql = "SELECT "+ field +" FROM "+ table +" WHERE "+ conditionColumn +" = '"+ searchValue +"'";
As this is an open door for SQL injection, and given the fact that SQL Server allows you two ways of creating an alias - value as alias and alias = value,
you can take advantage of that and try to generate an SQL statement like this:
SELECT field /* FROM table WHERE conditionColumn */ = 'searchValue'
So field should be "field /* ",
and conditionColumn should be "conditionColumn */"
table name doesn't matter, you could leave an empty string for it.
Hope someone can help with this question. I have a table with image file names sorted in it and some are missing the directory prefix. eg: (without folder info): "some_imagname.jpg" and then with the folder : "/photos/another_imagname.jpg".
I would like to run an update on the table, so that all image names have the folder prefixed added to it, where it is missing.
It's a long story as to how this happened, but suffice to say, I really need to get this updated quite soon.
Many thanks
Hans
A simple update statement with concatenation using the + operator and a filter to exclude the records that are already prefixed:
UPDATE table
SET column = '/photos/' + column
WHERE column NOT LIKE '/photos/%'
With the little information given i'd say:
UPDATE table
SET name_column = '/photos/' + name_column
WHERE name_column NOT LIKE '/photos%';
A database that a client of mine has was hacked. I am in the process of trying to rebuild the data. The site is running classic ASP with a SQL Server database. I believe I have found where the weak point was for the hackers and removed that entry point for now.
Every text colummn in the database was appended with some html markup and inline script/js tags.
Here is an example of a field:
all</title><script>
document.write("<style>.aq21{position:absolute;clip:rect(436px,auto,auto,436px);}</style>");
</script>
<div class=aq21>
<a href=http://samedaypaydayloansonlineelqmt.com >same day payday loans online</a>
<a href=http://samedaypaydayloan
This example was in the Users table in the UserRights column. The initial value was all, but then you can see the links that were appended.
I need to write a regex script that will search through all fields in each column of each table in the database and remove this extra markup.
Essentially, if I try to match </table>, then that string and everything that appends it can be replaced with a blank string.
All of these appended strings are the same for each field in the same column. However, there are multiple columns in each table.
This is what I have been doing so far, replacing the hacked part, but a nice regex would probably help me out, though my regex skills.... well suck.
UPDATE [databasename.[db].[databasetable]
set
UserRights = replace(UserRights,'</title><script>document.write("<style>.aq21{position:absolute;clip:rect(436px,auto,auto,436px);}</style>");</script><div class=aq21><a href=http://samedaypaydayloansonlineelqmt.com >same day payday loans online</a><a href=http://samedaypaydayloan','');
Any regex help and/or tips are appreciated.
This is what I ended up doing (big thanks to #Bohemian):
I went through each table and checked which column was affected. Then I ran the following script on each column:
UPDATE [tablename]
set columnname = substring(columnname, 1, charindex('/', columnname)-1)
where columnname like '%</%';
If the column had any markup in it, then I ended up manually updating those records manually. (lucky for me there was only a couple of records).
If anyone has any better solutions, please feel free to comment.
Thanks!
Since the bad stuff starts with a <, and that is an unusual character to typically find, I would use normal text functions, something like this:
update mytable set
mycol = substr(mycol, 1, charindex('<', mycol) - 1)
where mycol like '%<%';
And methodically do this with every column of every table.
Note that I'm only guessing at the right function to use, since I'm unfamiliar with SQL Server, but you get idea.
I welcome someone editing the SQL to improve it.
I have an informix database consisting of a large number of tables.
I know that there is a string "example" somewhere inside some table, but don't know which table it is or which column it is. (I know this is a very rare case)
Because of the large number of tables, there is no way to look for it manually. How do i find this value inside this large database? Is there any query to find it?
Thanks in advance!
Generally, you have two approaches.
One way would be to dump each table to individual text files and grep the files. This can be done manually on a small database, or via a script in the case of a larger one. Look into UNLOAD and dbaccess. Example in a BASH script: (you'll need to generate the table list in the script either statically or via a query.)
unload_tables () {
for table in ${TABLE_LIST} do
dbaccess database_name << EOF
unload to "${OUT_PATH}/${table}/.out"
select * from $table;
EOF
done
}
Or, this is a little more complicated. You can create a specific SELECT (filtering each column by "example") for each table and column in your db in a similar automated fashion using systables and syscolumns, then run each sql.
For example, this query shows you all columns in all tables:
SELECT tabname, colno, colname, coltype, collength
FROM systables a, syscolumns b
WHERE a.tabid = b.tabid
It is easy to adapt this such that the SELECT return a proper formatted SQL string that allows you to query the database for matches to "example". Sorry, I don't have a full solution at the ready, but if you google for "informix systables syscolumns", you will see lots of ways this information can be used.
In Informix, determining the columns that contain character data is fiddly, but doable. Fortunately, you're unlikely to be using the esoteric features such as DISTINCT TYPE that would make the search harder. (Alphadogg's suggestion of unload - using dbexport as I note in my comment to his answer - makes sense, especially if the text might appear in a CLOB or TEXT field.) You need to know that types CHAR, NCHAR, VARCHAR, NVARCHAR, and LVARCHAR have type numbers of 0, 13, 15, 16 and 43. Further, if the column is NOT NULL, 256 is added to the number. Hence, the character data columns in the database are found via this query:
SELECT t.owner, t.tabname, c.colname, t.tabid, c.colno
FROM "informix".systables t, "informix".syscolumns c
WHERE t.tabid = c.tabid
AND c.coltype IN (0, 13, 15, 16, 43, 256, 269, 271, 272, 299)
AND t.tabtype = 'T' -- exclude views, synonyms, etc.
AND t.tabid >= 100 -- exclude the system catalog
ORDER BY t.owner, t.tabname, c.colno;
This generates the list of places to search. You could get fancy and have the SELECT generate a string suitable for resubmission as a query - that's left as an exercise for the reader.