We track IPs that attack our site. First attack, we temp block them. Tf they ever attack again then we permanently blacklist them. Information for each attack by each IP is stored in perpetuum. Twice daily, reports with an Excel spreadsheet with all pertinent information is emailed to various people, and then the information is manually added to a massive spreadsheet. We've recently spun up a new box with SQL server and I've added all of the existing information to a table in the new database.
As I'm new to this, I would like to know if there is a way to send the daily spreadsheets to this new sql server and have it parse out the excel attachment and update our master tracking table. The spreadsheet will always have the same structure (15 columns and header and footer rows) with varying row quantities, and of course it matches the existing table structure.
I've been googling it and am only able to find queries (ba dum tish) on how to make SQL export to excel and send an email with Database Mail. Can't find anything on sending en email TO sql server and having it process an attachment.
You can make use of the SQL Server Integration Services(SSIS). You can write an SSIS package that import the data from the given Excel spreadsheet to a table and
then from that table you can write insert or update statements to your production table. You can use "Data Flow task" to Import the Data from the excel file and then write an " Execute SQL Task" which will update the values to Production table. Remember that you will have to keep the Excel file in the same folder all time (or else you can use dynamic statements to get the file name dynamically using Variables). Once you have completed the package you can schedule the package as an SQL Server Job which will run periodically and hence the data will be automatically updated.
Please refer this video for a basic idea about SSIS :
Import Data From Excel to SQL Server Using SSIS
Twice daily, reports with an Excel spreadsheet with all pertinent information is emailed to various people,
Try saving the File to a location and then use SSMS Export,Import Wizard ..This package can be saved and set to Run Daily
Here is a step by step tutorial covering the same..
https://www.mssqltips.com/sqlservertutorial/203/simple-way-to-import-data-into-sql-server/
Related
I have created an SSIS package for my database running on SQL Server 2014.
This SSIS package extracts data from an Excel workbook (residing in a shared folder on my Windows Server) and feeds a table in my database. I have tested the package (through a SQL Job) and it works fine.
The logic behind the creation of this package is to pull data on a daily basis from that Excel workbook (the latter being updated on a daily basis). It does have a date column; so, data updated are tagged with the relevant date.
Since this SQL job will be executed on a daily basis, the table will be filled with duplicates (that is, data from Day 1 will be repeated on Day 2 and so on).
How can I deal this issue? I was thinking on adding a T-SQL flow to my SSIS package to delete the contents of the Table before it pulls the data from the Excel Workbook. I guess this would work but I am on the look out for a more elegant solution.
Any help would be appreciated.
I'm moving an application from an Access database to a SQL Server database.
The current Access database contains 5 'linked' Excel files (reports that come from SAP) which are refreshed daily by overwriting each file with the new SAP report. In this way, through a bunch of transformation/queries, the data ends up in the appropriate table in the way we want to store the data.
Is a setup similar to this possible using SSIS? I've watched tutorials about uploading Excel files into a table, but essentially I need to do this:
SAP Export > Save (overwrite) to Network File > MS Access link exists and new data is transformed through many 'stored procedures (action queries)' > Data moved to appropriate table.
Appreciate any YouTube/Google/Links/Reading about doing this in SSIS. Regards!
This issue occurred and I would like to know possible causes:
During text file import the SSIS package created tables in the wrong database instead of writing in the correct tables in the correct database.
Details: There is an application which uses SSIS packages to import text files daily.
Those file records are stored in tables in the application database and they are available until the next set of files is received, when the new records will replace them.
The records are also stored in another database where they are kept, with import date information.
The tables keeping the same information in the two databases have different names.
The SSIS package contains:
two database connections
a file connection
and a dataflow task with:
- one flat file Source
- one multicast
- two OLE DB destinations
Execution: dtexec
The package has been running for a year now.
There is NO table creation task.
I connect the output of the multicast to already created tables in the database.
Issue: one time the SSIS package wrote the records in the storage database tables and, instead of writing in the application database tables, it created the application database tables in the storage database and wrote the records there.
What could have caused it?
Thanks in advance.
I have a stored procedure that I am going to run every weekend, it produces a result set that I need to export into an Excel file.
For the above problem I want to automate this process, so I am going to create a SQL Job and I am going to run this stored procedure every weekend so that that generated Excel file is sent to my reporter.
For this I need steps to export the result set data to an Excel file.
And also is it possible to send that Excel file to the specific mail while running the job itself?
So, you might try your luck on https://dba.stackexchange.com/, but in my experience a SQL Agent job running a stored procedure could be coaxed to return CSV or XML - and those could end up in Excel, but there are missing links. I think the missing links would involve programming and potentially 3rd party tools to avoid using Excel's COM API.
I'd strongly recommend your pursuing SQL Server Reporting Services. It is included free with your edition of SQL and includes the ability to
run reports on a schedule (subscriptions),
format the results as an Excel file
distribute the results via email
You'd take your query and use it as the data source for a "report" and use the report wizard to create a very simple table with the results.
Avoid page headers (or footers) that span columns - this will keep the excel output cleaner.
References
Stack Overflow: reporting-services-export-to-excel-with-multiple-worksheets
Technet: Reporting Services
I've created an SSIS package with the purpose of deleting and then recreating an Excel sheet, and then inserting data from a SQL database into that Excel sheet.
What I currenty have is:
1.) An Execute SQL task to delete 'Sheet1' from a local Excel file.
2.) An Execute SQL task to recreate 'Sheet1' in the same Excel file.
3.) A Data Flow task that uses an OLE DB Source to a 2005 SQL Server database.
4.) An Excel Destinaton (using an Excel Connection Manager) which points at / maps the SQL Server columns to the local Excel spreadsheet.
So essentially I'm deleting the Excel Sheet to remove old data, recreating it, and then trying to import the SQL data into it.
This worked well when I ran it the first time. When I try to rerun it however, all the tasks run successfully, the correct number of rows are shown to move to the Excel destination on the data flow tab, but the Excel spreadsheets are blank.
Can anyone suggest why this might be happening?
Thanks, Gavin
Is this the only sheet in your Excel file? If so, it might be easier to use the File System Task component to delete the entire file before creating your sheet (and therefore file) in your Execute SQL statement.