I'm planning doing some migration and I need to know where all the attachment files is stored. In this forum, there are some answers, but the binary value for column ir_attachment.db_datas is set to null for every record.
As alredy mentioned in the comment section, the default location if not defined otherwise in odoo.conf file data_dir directory or Odoo service command --data-dir option, is ~/.local/share/Odoo/filestore/ODOO_DB_NAME or /var/lib/Odoo/filestore/ODOO_DB_NAME. If defined otherwise using any of the options, you will find the filestore directory at that defined location.
I cannot import a database in AWS RDS because of this commands in my sql file:
SET ##SESSION.SQL_LOG_BIN= 0;
SET ##GLOBAL.GTID_PURGED=/*!80000 '+'*/ '';
SET ##SESSION.SQL_LOG_BIN = #MYSQLDUMP_TEMP_LOG_BIN;
Are they important ? whiteout them there is no error.
log_bin_trust_function_creators parameter is set to 1 in a custom parameter.
FYI: MySql 5.7 and 8, same error
ERROR 1227 (42000) at line 20: Access denied; you need (at least one of) the SUPER, SYSTEM_VARIABLES_ADMIN or SESSION_VARIABLES_ADMIN privilege(s) for this operation
SET ##SESSION.SQL_LOG_BIN=0;
This is telling MySQL not to put these INSERT statements into the binary log. If you do not have binary logs enabled (not replicating), then it's not an issue to remove it. As far as I know, there is no way to enable this in RDS; I'm actually trying to figure out a way, which is how I found your question.
SET ##GLOBAL.GTID_PURGED=/*!80000 '+'*/ '';
Did you execute a RESET MASTER on the database where the dump originated from? Check here for an explanation of this value: gtid_purged
SET ##SESSION.SQL_LOG_BIN = #MYSQLDUMP_TEMP_LOG_BIN;
This is setting the ##SESSION.SQL_LOG_BIN variable back to the original setting; you should've seen another line like: SET #MYSQLDUMP_TEMP_LOG_BIN = ##SESSION.SQL_LOG_BIN;
If you're simply recovering this table into a new database that isn't writing to a binary log (for replication), it's safe to remove these lines. Hope this helps!
Started a new SSIS project, and forgot to set the default protection level to "Don't save sensitive" (our standard) Now midway through the project and made the changes (at project level and in each package.) When checked all packages are Don't Save Sensitive and the project is Don't Save Sensitive, however when attempting a build, I get
Project consistency check failed. The following inconsistencies were
detected: PACKAGE1.dtsx has a different ProtectionLevel than the
project. PACKAGE2.dtsx has a different ProtectionLevel than the
project. ... PACKAGE(N).dtsx has a different ProtectionLevel than
the project.
(it lists every package in the project even though they all match the Project level protection.)
I suspect you ran into the same issue I did. I corrected all my packages via the API so that they all indicated they were DTS:ProtectionLevel="0" which is unprotected.
The project (.dtproj) file also has a protection level which gets set to DontSaveSensitive. <SSIS:Project SSIS:ProtectionLevel="DontSaveSensitive" xmlns:SSIS="www.microsoft.com/SqlServer/SSIS">
The mismatch for me was that inside the project file, it keeps track of far too much information about each package so if you scroll down, you'll see an entry per package like
<SSIS:Property SSIS:Name="ProtectionLevel">3</SSIS:Property> or whatever the default number is. Make that 0 in the file (search and replace). Save the project file and it'll now build.
You might need to perform a Build All to get it to build. I suspect that VS/SSDT is trying to use the extra data it stores in the .dtproj file to determine whether it needs to validate all the packages in a project. Since we hand edited the file, it didn't trip whatever sensor would normally be flipped to signal a full recompile was needed.
billinkc's answer didn't work for me because changing the value with a text editor doesn't change it correctly. The following MSDN page explains that there is a cmd line tool to manage this:
http://technet.microsoft.com/en-us/library/cc879310.aspx
Like so:
for %f in (*.dtsx) do dtutil.exe /file %f /encrypt file;%f;2;strongpassword
It will change every module in the project to the protection level specified in the second to last value. If it is 0, don't store values, then you don't need the password and can get rid of the last semicolon and everything after.
The following MSDN article has a table with the numbers for each protection level, as used with dtutil:
http://technet.microsoft.com/en-us/library/ms141747.aspx
Microsoft have broken this so that even using the DTUTIL utility to change package protections doesn't fix the project file metadata.
I had to apply to manual fix to the project file to change metadata storing a copy of the Package Protection level of the packages to the same as that of the project and packages.
Thought through? Probably not.
Get list of packages (that have already been deployed and create DTUTIL statements. Put them in a batch file and execute from a command line.
This only works for deployed packages as we are looking at SSISDB and not the Project folder
USE SSISDB
DECLARE #projName VARCHAR(250) = 'Sales'
DECLARE #FolderPath VARCHAR(1000) = 'E:\ssis_' + #projName
DECLARE #DtutilString VARCHAR(1000) =
'"C:\Program Files\Microsoft SQL Server\130\DTS\Binn\dtutil.exe"/file "'+ #FolderPath +'\XXX" /encrypt file;"'+ #FolderPath +'\XXX";0 /quiet'
SELECT DISTINCT
REPLACE(#DtutilString, 'XXX', pack.[name])
-- SELECT *
FROM internal.packages AS pack
INNER JOIN
[internal].[projects] AS proj
ON pack.project_id = proj.project_id
WHERE proj.name = 'ssis_' + #projName
I have some old VB Script code that I can modify but I cannot migrate the app to a new language...
This VB Script basically used to connect to a SQL 2000 Server and manipulate the connections of the package before running it, which would output a flatfile database locally.
Now I do not have a DTS package, I just have an SSIS package.
The code used to be this:
dim DTScon
dim DTSpkg
set DTSpkg = Server.CreateObject("DTS.Package")
DTSpkg.LoadFromSQLServer "mysqlserver","myuser","mypass",dts.DTSSQLStgFlag_Default,,,,"MyPackageName"
set DTScon = DTSpkg.Connections.Item("Conn1")
set DTScon.UserId = "conn_username"
set DTScon.Password = "conn_password"
set DTScnp = DTScon.ConnectionProperties.Item("Data Source");
DTScnp.Value = "c:\path\to\output\flatfile"
I am now trying to modify the code to
dim DTScon
dim DTSpkg
set DTSpkg = Server.CreateObject("DTS.Application")
DTSpkg.LoadFromSQLServer "mysqlserver","myuser","mypass",dts.DTSSQLStgFlag_Default,,,,"MyPackageName"
set DTScon = DTSpkg.Connections.Item("Conn1")
set DTScon.UserId = "conn_username"
set DTScon.Password = "conn_password"
set DTScnp = DTScon.ConnectionProperties.Item("Data Source");
DTScnp.Value = "c:\path\to\output\flatfile"
First error I got is:
Microsoft VBScript runtime error '800a01b6'
Object doesn't support this property or mother: 'DTSpkg.LoadFromSQLServer'
/main.asp.241
This I am guessing will be one of the many hurdles I will probably have to overcome.
However, I have struggled to find a solution to this by Google searching.
Has anyone got an idea of what I need to do to get IIS running this new code?
Or any problems I am likely to face trying to do this?
Your best option would be to change your SSIS packages to use Package Configurations for the Connection Managers. If you use SQL Server configurations, when the package is loaded and executed, it will change the Connection Managers based on what it finds in the SQL Server configuration table. So your steps would be to alter the data in that table, then start the packages - no direct manipulation of the packages is necessary.
I want to get the results of a stored procedure and place them into a CSV file onto an FTP location.
The catch though is that I cannot create a local/temporary file that I can then FTP over.
The approach I was taking was to use an SSIS package to create a temporary file and then have a FTP Task within the pack to FTP the file over, but our DBA's do not allow temporary files to be created on any servers.
in reply to Yaakov Ellis
I think we will need to convince the DBA's to let me use at least a share on a server that they do not operate, or ask them how they would do it.
in reply to Kev
I like the idea of the CLR integration, but I don't think our DBA's even know what that is lol and they would probably not allow it either. But I will probably be able to do this within a Script Task in an SSIS package that can be scheduled.
This step-by-step example is for others who might stumble upon this question. This example uses Windows Server 2008 R2 server and SSIS 2008 R2. Even though, the example uses SSIS 2008 R2, the logic used is applicable to SSIS 2005 as well. Thanks to #Kev for the FTPWebRequest code.
Create an SSIS package (Steps to create an SSIS package). I have named the package in the format YYYYMMDD_hhmm in the beginning followed by SO stands for Stack Overflow, followed by the SO question id, and finally a description. I am not saying that you should name your package like this. This is for me to easily refer this back later. Note that I also have two Data Sources namely Adventure Works and Practice DB. I will be using Adventure Works data source, which points to AdventureWorks database downloaded from this link. Refer screenshot #1 at the bottom of the answer.
In the AdventureWorks database, create a stored procedure named dbo.GetCurrency using the below given script.
CREATE PROCEDURE [dbo].[GetCurrency]
AS
BEGIN
SET NOCOUNT ON;
SELECT
TOP 10 CurrencyCode
, Name
, ModifiedDate
FROM Sales.Currency
ORDER BY CurrencyCode
END
GO
On the package’s Connection Manager section, right-click and select New Connection From Data Source. On the Select Data Source dialog, select Adventure Works and click OK. You should now see the Adventure Works data source under the Connection Managers section. Refer screenshot #2, #3 and #4.
On the package, create the following variables. Refer screenshot #5.
ColumnDelimiter: This variable is of type String. This will be used to separate the column data when it is written to the file. In this example, we will be using comma (,) and the code is written to handle only displayable characters. For non-displayable characters like tab (\t), you might need to change the code used in this example accordingly.
FileName: This variable is of type String. It will contain the name of the file. In this example, I have named the file as Currencies.csv because I am going to export list of currency names.
FTPPassword: This variable is of type String. This will contain the password to the FTP website. Ideally, the package should be encrypted to hide sensitive information.
FTPRemotePath: This variable is of type String. This will contain the FTP folder path to which the file should be uploaded to. For example if the complete FTP URI is ftp://myFTPSite.com/ssis/samples/uploads, then the RemotePath would be /ssis/samples/uploads.
FTPServerName: This variable is of type String. This will contain the FTP site root URI. For example if the complete FTP URI is ftp://myFTPSite.com/ssis/samples/uploads, then the FTPServerName would contain ftp://myFTPSite.com. You can combine FTPRemotePath with this variable and have a single variable. It is up to your preference.
FTPUserName:This variable is of type String. This will contain the user name that will be used to connect to the FTP website.
ListOfCurrencies: This variable is of type Object. This will contain the result set from the stored procedure and it will be looped through in the Script Task.
ShowHeader: This variable is of type Boolean. This will contain values true/false. True indicates that the first row in the file will contain Column names and False indicates that the first row will not contain Column names.
SQLGetData: This variable is of type String. This will contain the Stored Procedure execution statement. This example uses the value EXEC dbo.GetCurrency
On the package’s Control Flow tab, place an Execute SQL Task and name it as Get Data. Double-click on the Execute SQL Task to bring the Execute SQL Task Editor. On the General section of the Execute SQL Task Editor, set the ResultSet to Full result set, the Connection to Adventure Works, the SQLSourceType to Variable and the SourceVariable to User::SQLGetData. On the Result Set section, click Add button. Set the Result Name to 0, this indicates the index and the Variable to User::ListOfCurrencies. The output of the stored procedure will be saved to this object variable. Click OK. Refer screenshot #6 and #7.
On the package’s Control Flow tab, place a Script Task below the Execute SQL Task and name it as Save to FTP. Double-click on the Script Task to bring the Script Task Editor. On the Script section, click the Edit Script… button. Refer screenshot #8. This will bring up the Visual Studio Tools for Applications (VSTA) editor. Replace the code within the class ScriptMain in the editor with the code given below. Also, make sure that you add the using statements to the namespaces System.Data.OleDb, System.IO, System.Net, System.Text. Refer screenshot #9 that highlights the code changes. Close the VSTA editor and click Ok to close the Script Task Editor. Script code takes the object variable ListOfCurrencies and stores it into a DataTable with the help of OleDbDataAdapter because we are using OleDb connection. The code then loops through each row and if the variable ShowHeader is set to true, the code will include the Column names in the first row written to the file. The result is stored in a stringbuilder variable. After the string builder variable is populated with all the data, the code creates an FTPWebRequest object and connects to the FTP Uri by combining the variables FTPServerName, FTPRemotePath and FileName using the credentials provided in the variables FTPUserName and FTPPassword. Then the full string builder variable contents are written to the file. The method WriteRowData is created to loop through columns and provide the column names or data information based on the parameters passed.
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
using System.Data.OleDb;
using System.IO;
using System.Net;
using System.Text;
namespace ST_7033c2fc30234dae8086558a88a897dd.csproj
{
[System.AddIn.AddIn("ScriptMain", Version = "1.0", Publisher = "", Description = "")]
public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
#region VSTA generated code
enum ScriptResults
{
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
};
#endregion
public void Main()
{
Variables varCollection = null;
Dts.VariableDispenser.LockForRead("User::ColumnDelimiter");
Dts.VariableDispenser.LockForRead("User::FileName");
Dts.VariableDispenser.LockForRead("User::FTPPassword");
Dts.VariableDispenser.LockForRead("User::FTPRemotePath");
Dts.VariableDispenser.LockForRead("User::FTPServerName");
Dts.VariableDispenser.LockForRead("User::FTPUserName");
Dts.VariableDispenser.LockForRead("User::ListOfCurrencies");
Dts.VariableDispenser.LockForRead("User::ShowHeader");
Dts.VariableDispenser.GetVariables(ref varCollection);
OleDbDataAdapter dataAdapter = new OleDbDataAdapter();
DataTable currencies = new DataTable();
dataAdapter.Fill(currencies, varCollection["User::ListOfCurrencies"].Value);
bool showHeader = Convert.ToBoolean(varCollection["User::ShowHeader"].Value);
int rowCounter = 0;
string columnDelimiter = varCollection["User::ColumnDelimiter"].Value.ToString();
StringBuilder sb = new StringBuilder();
foreach (DataRow row in currencies.Rows)
{
rowCounter++;
if (rowCounter == 1 && showHeader)
{
WriteRowData(currencies, row, columnDelimiter, true, ref sb);
}
WriteRowData(currencies, row, columnDelimiter, false, ref sb);
}
string ftpUri = string.Concat(varCollection["User::FTPServerName"].Value,
varCollection["User::FTPRemotePath"].Value,
varCollection["User::FileName"].Value);
FtpWebRequest ftp = (FtpWebRequest)FtpWebRequest.Create(ftpUri);
ftp.Method = WebRequestMethods.Ftp.UploadFile;
string ftpUserName = varCollection["User::FTPUserName"].Value.ToString();
string ftpPassword = varCollection["User::FTPPassword"].Value.ToString();
ftp.Credentials = new System.Net.NetworkCredential(ftpUserName, ftpPassword);
using (StreamWriter sw = new StreamWriter(ftp.GetRequestStream()))
{
sw.WriteLine(sb.ToString());
sw.Flush();
}
Dts.TaskResult = (int)ScriptResults.Success;
}
public void WriteRowData(DataTable currencies, DataRow row, string columnDelimiter, bool isHeader, ref StringBuilder sb)
{
int counter = 0;
foreach (DataColumn column in currencies.Columns)
{
counter++;
if (isHeader)
{
sb.Append(column.ColumnName);
}
else
{
sb.Append(row[column].ToString());
}
if (counter != currencies.Columns.Count)
{
sb.Append(columnDelimiter);
}
}
sb.Append(System.Environment.NewLine);
}
}
}
Once the tasks have been configured, the package’s Control Flow should look like as shown in screenshot #10.
Screenshot #11 shows the output of the stored procedure execution statement EXEC dbo.GetCurrency.
Execute the package. Screenshot #12 shows successful execution of the package.
Using the FireFTP add-on available in FireFox browser, I logged into the FTP website and verified that the file has been successfully uploaded to the FTP website. Refer screenshot #13.
Examining the contents by opening the file in Notepad++ shows that it matches with the stored procedure output. Refer screenshot #14.
Thus, the example demonstrated how to write results from database to an FTP website without having to use temporary/local files.
Hope that helps someone.
Screenshots:
#1: Solution_Explorer
#2: New_Connection_From_Data_Source
#3: Select_Data_Source
#4: Connection_Managers
#5: Variables
#6: Execute_SQL_Task_Editor_General
#7: Execute_SQL_Task_Editor_Result_Set
#8: Script_Task_Editor
#9: Script_Task_VSTA_Code
#10: Control_Flow_Tab
#11: Query_Results
#12: Package_Execution_Successful
#13: File_In_FTP
#14: File_Contents
If you were allowed to implement CLR integration assemblies you could actually use FTP without having to write a temporary file:
public static void DoQueryAndUploadFile(string uri, string username, string password, string filename)
{
FtpWebRequest ftp = (FtpWebRequest)FtpWebRequest.Create(uri + "/" + filename);
ftp.Method = WebRequestMethods.Ftp.UploadFile;
ftp.Credentials = new System.Net.NetworkCredential(username, password);
using(StreamWriter sw = new StreamWriter(ftp.GetRequestStream()))
{
// Do the query here then write to the ftp stream by iterating DataReader or other resultset, following code is just to demo concept:
for (int i = 0; i < 100; i++)
{
sw.WriteLine("{0},row-{1},data-{2}", i, i, i);
}
sw.Flush();
}
}
Is there a server anywhere that you can use where you can create a temporary file? If so, make a web service that returns an array containing the contents of the file. Call the web service from the computer where you can create a temporary file, use the contents of the array to build the temp file and ftp it over.
If there is no where at all where you can create a temporary file, I don't see how you will be able to send anything by FTP.
Try using a CLR stored procedure. You might be able to come up with something, but without first creating a temporary file, it might still be difficult. Could you set up a share on another machine and write to that, and then ftp from there?
Script from the FTP server, and just call the stored proc.
The catch though is that I cannot create
a local/temporary file that I can then FTP over.
This restriction does not make any sense, try to talk to DBA nicely and explain it to him/her. It is totally reasonable for any Windows process or job to create temporary file(s) in appropriate location, i.e. %TEMP% folder. Actually, SSIS runtime itself often creates temporary files there - so if DBA allows you to run SSIS, he is allowing you to create temporary files :).
As long as DBA understands that these temporary files do not create problem or additional workload for him (explain that he does not have to set special permissions, or back them up, etc), he should agree to let you create them.
The only maintenance task for DBA is to periodically clean %TEMP% directory in case your SSIS job fails and leaves the file behind. But he should do this anyway, as many other processes may do the same. A simple SQL Agent job will do this.