I have the following code which is working with no errors and returning the expected output when I print the results of the pyodbc cursor I created.
cnxn = pyodbc.connect(MY_URL)
cursor = cnxn.cursor()
cursor.execute(
'''
CREATE TABLE tablename(
filename VARCHAR(100),
synopsis TEXT,
abstract TEXT,
original TEXT,
PRIMARY KEY (filename)
)
'''
)
for file in file_names_1:
try:
query = produce_row_query(file, tablename, find_tag_XML)
cursor.execute(query)
except pyodbc.DatabaseError as p:
print(p)
result = cursor.execute(
'''
SELECT filename,
DATALENGTH(synopsis),
DATALENGTH(abstract),
original
FROM ml_files
'''
)
for row in cursor.fetchall():
print(row)
However, no new tables are showing up in my actual MS SQL server. Am I missing a step to push the changes or something of that nature?
You need to commit changes or else they will not be updated in your actual database.
cnxn.commit()
I need to copy data from an old database to a newer one.
Both of these databases have a user setup table with the primary key of "USER ID".
The problem is, in the old database the users didn't have the domain in the name, but in the new one they have.
Example:
Primary Key old DB: USER1
Primary Key new DB: DOMAIN\USER1
This prevents a standard WHERE clause to update the correct user because it can't find it due to the domain being added.
My code:
'FROM [' + #src_DB + '].dbo.[' + #src_table + '] as src '
'WHERE [' + #dest_DB + '].dbo.[' + #dest_table + '].[User ID] = ' + #domain_name + 'src.[User ID]'
printing the result:
WHERE [Destination_DB].dbo.[Destination_Table].[User ID] = DOMAIN\src.[User ID]
The problem is it doesn't add the DOMAIN to the value but rather to the statement...
How can I add the Domain to the actual value of src.[User ID]?
I think there's a dot missing, and you should use QUOTENAME
'WHERE ' + QUOTENAME(destination_table) + '.[User ID] = ' + QUOTENAME(#domain) + '.' + QUOTENAME(source_table) + '.[User ID]'
Whenever you create a SQL statement dynamically it's a good idea to print it out, copy it into a new query window and check for syntax errors...
UPDATE You: Yes, both databases are in the same server
An object can be (fully) specified with
ServerName.DatabaseName.Schema.ObjectName
A table's column would add one more .ColumnName
When both objects live on the same server you can let the first part away.
Objects of the same database let this part away.
Objects of the default schema might be called with the ObjectName alone.
But if you state a DatabaseName you must also state a SchemaName!
Use QUOTENAME() to add the brackets and add just the dots via string concatenation (or use CONCAT()-function).
UPDATE 2 Did I get this wrong completely?
After you comment I think I understand it now: You want to compare the values of both [USER ID] columns, but the new is DOMAIN\MyUserId while the older was just MyUserId.
You have two approaches
Add the Domain\ as string to the value of [User ID]
Use SUBSTRING([User ID],CHARINDEX('\',[UserID])+1,1000) to cut the newer value down to the naked value of [User ID]
For the first something like this
'WHERE [' + #dest_DB + '].dbo.[' + #dest_table + '].[User ID] = ''' + #domain_name + ''' + src.[User ID]'
The second is quite clumsy with dynamically created SQL...
Recently I have posted a question, it contains some syntax error, now the code is running without error, thanks to #Arulkumar.
But now I am facing one more problem, data from excel sheet is storing properly on to SQL Server database, but when I press refresh button or if I go to that link again in my application, data is repeating in the database. Means again it is retrieving values from excel and storing same data again in the database.
How can I avoid data repetition. Can any one please help me to solve this issue? Code and excel sheet sample is there in the above mentioned link.
You need MERGE statement
request.query('MERGE [mytable] as target USING (SELECT SalesPersonID, TerritoryID FROM OPENROWSET(' +
'\'Microsoft.ACE.OLEDB.12.0\', \'Excel 12.0;Database=D:\\sample\\test\\data\\1540_OPENROWSET_Examples.xls;HDR=YES\', ' +
'\'SELECT SalesPersonID, TerritoryID FROM [SELECT_Example$]\')' +
' ) as source' +
' ON target.SalesPersonID = source.SalesPersonID' +
' WHEN MATCHED THEN UPDATE SET TerritoryID = source.TerritoryID' +
' WHEN NOT MATCHED THEN INSERT (SalesPersonID, TerritoryID) VALUES (source.SalesPersonID, source.TerritoryID);'
,function(err,recordset){
if(err) console.log(err)
It will update TerritoryID if there is already row with same SalesPersonID and insert row if there is no matches in mytable.
If you need join on both fields change this:
ON target.SalesPersonID = source.SalesPersonID
On this:
ON target.SalesPersonID = source.SalesPersonID AND target.TerritoryID = source.TerritoryID
And after that - remove this string because it doesn't need anymore:
'WHEN MATCHED THEN UPDATE SET TerritoryID = source.TerritoryID' +
I have just recently began using OPENROWSET to insert images into a table. Previously, I would specify the path to each image (1 image = 1 INSERT statement), and use PHP to generate the image's binary string:
INSERT INTO nopCommerce..Picture (PictureBinary, MimeType, SeoFilename, AltAttribute, TitleAttribute, IsNew)
VALUES (
(
SELECT *
FROM OPENROWSET(BULK '" . $this->image['path'] . "', SINGLE_BLOB) AS Binary
),
'" . $this->image['mime_type'] . "',
'" . $this->image['seo_filename'] . "',
'" . $this->image['alt'] . "',
'',
0
)
However, I am trying to insert all images with a single query. So, I have began storing the path to each image into a table, and now I need to insert each one as I did before (just using the table's path field instead of a PHP string). But, when I attempt the following:
INSERT INTO nopCommerce..Picture (PictureBinary, MimeType, SeoFilename, AltAttribute, TitleAttribute, IsNew)
SELECT
(
SELECT *
FROM OPENROWSET(BULK ImagePath, SINGLE_BLOB) AS Binary
),
MimeType,
Name,
Alt,
'',
0
FROM nopRMS..Product_Image_Mappings
I receive the following error:
Msg 102, Level 15, State 1, Line 5
Incorrect syntax near 'ImagePath'.
So, I tried adding quotes around the column's name (to no avail):
INSERT INTO nopCommerce..Picture (PictureBinary, MimeType, SeoFilename, AltAttribute, TitleAttribute, IsNew)
SELECT
(
SELECT *
FROM OPENROWSET(BULK 'ImagePath', SINGLE_BLOB) AS Binary
),
MimeType,
Name,
Alt,
'',
0
FROM nopRMS..Product_Image_Mappings
Msg 4860, Level 16, State 1, Line 1
Cannot bulk load. The file "ImagePath" does not exist.
There has to be a way to accomplish this, I just cannot find the proper syntax online anywhere. Does anyone know how to tell SQL Server to get the path (string) from dbo.Product_Image_Mappings.ImagePath?
UPDATE
I forgot to give you an example of a value that dbo.Product_Image_Mappings.ImagePath would return. It's paths like \\DEREK\WebImages\1\ca-82300.jpg...
UPDATE
It appears that Eirikur Eiriksson has provided a solution in this thread, but this looks like an overly-complicated method of achieving the same end...
UPDATE (Attempt Using Eirikur Eiriksson's Method)
DECLARE #SQL_STR NVARCHAR(MAX) = N'';
SELECT #SQL_STR = STUFF(
(
SELECT
N'
UNION ALL
SELECT '
+ N'(SELECT X.BulkColumn FROM OPENROWSET(BULK '
+ NCHAR(39) + im.ImagePath + NCHAR(39)
+ N', SINGLE_BLOB) AS X) AS PictureBinary,'
+ NCHAR(39) + im.MimeType + NCHAR(39)
+ N' AS MimeType,'
+ NCHAR(39) + im.Name + NCHAR(39)
+ N' AS SeoFilename,'
+ NCHAR(39) + REPLACE(im.Alt, '''', '''''') + NCHAR(39)
+ N' AS AltAttribute,'
+ N'NULL AS TitleAttribute,'
+ N'0 AS IsNew'
FROM nopRMS..Product_Image_Mappings im
FOR XML PATH(''), TYPE
).value('.[1]','NVARCHAR(MAX)'),1,12,N''
)
INSERT INTO nopCommerce..Picture (PictureBinary, MimeType, SeoFilename, AltAttribute, TitleAttribute, IsNew)
EXEC (#SQL_STR);
This kinda worked, but it only inserted 42 rows (out of 7200+)... I need this to be 100% accurate :( I admit though, I may need to change something about this query, but I don't know anything about it (aside from the basic INSERT, SELECT, etc.)
Maybe don't use OPENROWSET? What you are wanting can be handled in a much simpler and cleaner manner using SQLCLR. You can create a Scalar UDF to just read the contents of a file and return that as a VARBINARY(MAX). Then it will fit in nicely to your existing query. For example:
INSERT INTO nopCommerce.dbo.Picture (PictureBinary, MimeType, SeoFilename,
AltAttribute, TitleAttribute, IsNew)
SELECT
dbo.GetBinaryFile([ImagePath]) AS [PictureBinary],
MimeType,
Name,
Alt,
'',
0
FROM nopRMS.dbo.Product_Image_Mappings;
And how much code does it take for dbo.GetBinaryFile()? Here it is:
using System;
using System.Data.SqlTypes;
using System.IO;
using Microsoft.SqlServer.Server;
[return:SqlFacet(MaxSize = -1)]
[SqlFunction(IsDeterministic = false, IsPrecise = true)]
public static SqlBytes GetBinaryFile([SqlFacet(MaxSize = 1000)] SqlString FilePath)
{
if (FilePath.Value.Trim().Equals(String.Empty))
{
return SqlBytes.Null;
}
return new SqlBytes(File.ReadAllBytes(FilePath.Value));
}
And the T-SQL wrapper object is the following (please note the WITH RETURNS NULL ON NULL INPUT line as it skips execution if NULL is passed in, hence no need to check for FilePath.IsNull in the C# code :-)
CREATE FUNCTION [dbo].[GetBinaryFile](#FilePath NVARCHAR(1000))
RETURNS VARBINARY(MAX)
WITH RETURNS NULL ON NULL INPUT
AS
EXTERNAL NAME [CSVParser].[CSVUtils].[GetBinaryFile];
The Assembly will need to be marked as WITH PERMISSION_SET = EXTERNAL_ACCESS. Many people go the easy route of setting the database property of TRUSTWORTHY to ON in order to accomplish this, but that is a security risk and isn't even necessary. Just do the following and you can set the Assembly to EXTERNAL_ACCESS while keeping TRUSTWORTHY set to OFF:
Sign the Assembly.
Create an Asymmetric Key in master from that DLL.
Create a Login (also in master) from that Asymmetric Key.
Grant the new Login the EXTERNAL ACCESS ASSEMBLY permission.
You can find detailed instructions on how to do this in Visual Studio / SSDT (SQL Server Data Tools) in the following article that I wrote: Stairway to SQLCLR Level 7: Development and Security (that site does require free registration in order to view the content).
Also, for anyone that does not want to bother with creating / deploying the Assembly, a similar function is available (though not for free) in the Full version of the SQL# library (which I created, and while many functions are free, the File_* file system functions are only in the Full version).
I know there are a few about similar topics on here but I can't find one related to my issue, this is it:
I have a table with an ID Column and a QRCode column. each time an item is added the primary key auto increments. The QRCode will scan in to be like the following:
"http://somewebsite.com/12345/987654321"
i want to be able to remove the "http://somewebsite.com/" from the string, I know how to do this in C# however I am unsure of how to do this in Sql Server. any guidance would be great, thanks
Regular formats are like the following, and used in the example below.
"http://somewebsite.com/12345/456564654"
"http://somewebsite.com/12345/989886765"
"http://somewebsite.com/12346/987654321"
the query returns the following results:
SELECT
REPLACE
(
REPLACE(QRCode, 'http://somewebsite.com/', '')
,'/', ' '
) AS QRCode
FROM
QRTable
WHERE
QRCode LIKE '%http://somewebsite.com/%'
"12345 456564654"
"12345 989886765"
"12346 987654321"
Now i need to update the table with those new results however as there's 3 results, i get the error message "Subquery returned more than 1 value". is there a way to replace the selected values in the table with the ones that exist based on the primary key field?
**Removed previous example
A more complete answer based on your updated question. This removes the first portion of the URL as well as the trailing / so that you get your desired output.
DECLARE #Variable VARCHAR(50)
SET #Variable = 'http://somewebsite.com/12345/456564654'
SET #Variable =
REPLACE
(
REPLACE(#Variable, 'http://somewebsite.com/', '')
,'/', ' '
)
PRINT #Variable
Output = 12345 456564654
Looking at your SQL statement you want this:
SELECT
REPLACE
(
REPLACE(QRCode, 'http://somewebsite.com/', '')
,'/', ' '
) AS QRCode
FROM
QRTable
WHERE
QRCode LIKE '%http://somewebsite.com/%'