In SSIS, I am trying to pull in table data from a large Data warehouse for some analysis in Excel. I have written the following query:
SELECT journal_name AS Journal_Name,
division AS DIV,
account AS Acct,
cost_center AS CC,
effective_date AS Effective_Date,
effective_month AS Month,
effective_year AS Year,
posted_date AS Posted_Date,
source AS Source,
line_description AS Line_Description,
category AS Category,
entered_amount AS E_Entered_Amount,
entered_currency AS E_Currency,
functional_amount AS Fuctional_Amount,
functional_currency AS F_Currency
FROM corp.source
WHERE ( (division = 'C301' )
AND ( account LIKE '%' )
AND ( cost_center = '8D' )
AND ( effective_month BETWEEN '01' AND '10' )
AND ( effective_year = '2019' ) )
ORDER BY posted_date
I am doing this with ODBC Source as I have taken Oracle connector info from a Microsoft Access tool. I run the parse in the ODBC screen and I look at the preview and get no issues. However, when I try to run this package just to make sure the connection works, I get:
===================================
Package Validation Error (Package Validation Error)
===================================
Error at Expression Task [Microsoft.SqlServer.ExpressionTask]: A null or empty expression is not valid. Specify an expression.
Error at Expression Task: There were errors during task validation.
(Microsoft.DataTransformationServices.VsIntegration)
------------------------------
Program Location:
at Microsoft.DataTransformationServices.Project.DataTransformationsPackageDebugger.ValidateAndRunDebugger(Int32 flags, IOutputWindow outputWindow, DataTransformationsProjectConfigurationOptions options)
at Microsoft.DataTransformationServices.Project.DtsPackagesFolderProjectFeature.ExecuteTaskOrPackage(ProjectItem prjItem, String taskPath)
I'm not quite sure what column is causing the error. I have dealt with this exact same data in the past and was able to change all of the data coming in the Advanced Editor into Wide Strings [WSTR] when using OLE DB, then used data conversion to fix it and it worked very well. However, every time I try and change the type of the external data in this ODBC source advanced editor, it keeps resetting to the original data types. Is there a better way to do this, or is there a reason I cannot reset that incoming data to WSTR to deal with it?
Thanks.
Related
I work with large databases that needs to be stored into a server.
So, to work with them on Rstudio I have to open a connection to my Microsoft SQL Server with the dbConnect function :
conn <- dbConnect(odbc(),"myconnection",uid="***",pwd="***",schema="dbo",access="readonly")
and in order to use dplyr, I have to create data references with the tbl function :
data <- tbl(conn, "data")
But one of the online dataframe contains a columns that I can't read because I dont have the access, but I can read everything else.
The SQL query behind the tbl() function is :
SELECT * FROM data
and this is my problem.
Even when I try to select a specific column it doesn't work (see below), so I can't create my references and I can't work.
select(tbl(conn, "data"), "columnX")
=
SELECT columnX FROM data
I think this is the tbl() function and the call of "SELECT *" that blocks me.
Do you know what can I do ? Is there smilar functions that could resolve my problem ?
If you know the columns that you have access to, then one option is to bypass the default access SELECT * FROM ... with your own SQL query.
A remote table is defined by two components:
The database conneciton
The query to the database
When you connect with the default approach tbl(conn, 'data') then it defaults to a query SELECT * FROM data.
But here is another approach:
custom_query = 'SELECT columnX FROM data'
remote_table = tbl(conn, dbplyr::sql(customer_query))
Maybe it is a case of me looking at this for too long. But I have this Oracle update query I am trying to run, I have verified the query works with hardcoded values on SQL Developer, howver when I run it from my flow Mule it fails.can anybody tell me what I am doing wrong?
Here is the query:
<db:update config-ref="DBConf" doc:name="abcd">
<db:dynamic-query><![CDATA[UPDATE myTable
SET TYPE= 'Entry',
ENTERED_DATE=SYSDATE,
ENTRY_BY= 2345,
ENTRY_DATE=TO_DATE('#[flowVars.entryDate]','YYYY-MM-DD')
WHERE ID = 'abcd1234']]>
</db:dynamic-query>
</db:update>
the flowVars.entryDate value is '2017-05-10'
This throws the following Error:
Message : ORA-01841: (full) year must be between -4713 and +9999, and not be 0
(java.sql.SQLDataException). Message payload is of type: Integer
Now the same query works like I said in SQL Developer but not in Mule, Can anybody provide any input
You can find the same problem answer in the following link:
Oracle: year must be between -4713 and +9999, and not be 0
Try this once. TO_DATE('2012-05-12','yyyy-mm-dd').
Delete the quotes for the #[flowvar.entrydate]
Using Symfony with Sql Server and from what I've read, it seems that the connection to the database is not stable.
As soon as I use the orderBy method I get an error :
Here's an example :
$qStores =
$this->getManager()
->createQueryBuilder()
->select('rpdv')
->from('MainBundle:PointDeVenteReference', 'rpdv')
->andWhere( 'rpdv.partenaireClient = :id_partner ' )
->setParameter( 'id_partner', $this->getUser()->getPartenaire()->getIdPartenaire() )
->orderBy( 'rpdv.idPointDeVenteReference' , 'DESC' )
->setFirstResult( 0 )
->setMaxResults( 30 );
$stores = new Paginator( $qStores, FALSE );
And the error :
An exception has been thrown during the rendering of a template ("An exception occurred while executing
'SELECT DISTINCT TOP 30 id_point_de_vente_reference0
FROM ( SELECT p0_.id_point_de_vente_reference AS id_point_de_vente_reference0,
p0_.reference AS reference1,
p0_.date_derniere_modification AS date_derniere_modification2,
p0_.blocage AS blocage3
FROM point_de_vente_reference p0_
WHERE p0_.id_partenaire_client = ?
ORDER BY p0_.id_point_de_vente_reference DESC ) dctrn_result
ORDER BY id_point_de_vente_reference0 DESC'
with params [2829]:SQLSTATE[42000]:
[Microsoft][SQL Server Native Client 11.0][SQL Server]
The ORDER BY clause is invalid in views, inline functions, derived tables, subqueries, and common table expressions,
unless TOP, OFFSET or FOR XML is also specified.") in MainBundle:Default:store/list.html.twig at line 79.
I tried to change the class SQLServerPlatform with corrections found on the net, without success.
Do you have any idea?
Thx !
Edit :
This bug appears to be related to the Paginator with the second parameter to true. Passing it to false, I have no error
The dctrn_result is a derived table. From the error message, you can not use an order by. I do not know symfony2, but the code going to the database engine is invalid.
Craftydba
I am very new to Microsoft SQL Server and am using 2012 Management Studio. I get the error above when I try to export query results to an excel file using the wizard. I have seen solutions posted elsewhere for this error but do not know enough to figure out how to implement the solutions recommended. Can somebody please walk me through one of these solutions step by step?
I believe my problem is that the SQL Server Import and Export Wizard Does Not Recognise Varchar and NVarchar which I believe is the data type for the columns that I am receiving errors for.
Source Type 200 in SQL Server Import and Export Wizard?
http://connect.microsoft.com/SQLServer/feedback/details/775897/sql-server-import-and-export-wizard-does-not-recognise-varchar-and-nvarchar#
Query:
SELECT licenseEntitlement.entID, licenseEntitlement.entStartDate, entEndDate, quote.quoteId, quote.accountId, quote.clientId, quote.clientName, quote.contactName,
quote.contactEmail, quote.extReference, quote.purchaseOrderNumber, quote.linkedTicket
FROM licenseEntitlement INNER JOIN
quote ON quote.quoteId = SUBSTRING(licenseEntitlement.entComments, 12, PATINDEX('% Created%', licenseEntitlement.entComments) - 12)
inner join sophos521.dbo.computersanddeletedcomputers on computersanddeletedcomputers.name = entid and IsNumeric(computersanddeletedcomputers.name) = 1
WHERE (licenseEntitlement.entType = 'AVS') AND (licenseEntitlement.entComments LIKE 'OV Order + %') and entenddate < '4/1/2014'
ORDER BY licenseEntitlement.entEndDate
Error:
TITLE: SQL Server Import and Export Wizard
------------------------------
Column information for the source and the destination data could not be retrieved, or the data types of source columns were not mapped correctly to those available on the destination provider.
[Query] -> `Query`:
- Column "accountId": Source data type "200" was not found in the data type mapping file.
- Column "clientId": Source data type "200" was not found in the data type mapping file.
- Column "clientName": Source data type "200" was not found in the data type mapping file.
- Column "contactName": Source data type "200" was not found in the data type mapping file.
- Column "contactEmail": Source data type "200" was not found in the data type mapping file.
- Column "extReference": Source data type "200" was not found in the data type mapping file.
- Column "purchaseOrderNumber": Source data type "200" was not found in the data type mapping file.
- Column "linkedTicket": Source data type "200" was not found in the data type mapping file.
If any more details are needed please let me know
So, implementing the suggestion at the StackOverflow link you gave, of turning the query into a View, here's an example of what that could look like (with some code formatting ;) --
CREATE VIEW [dbo].[test__View_1]
AS
SELECT LIC.entID, LIC.entStartDate, entEndDate,
quote.quoteId, quote.accountId, quote.clientId, quote.clientName,
quote.contactName, quote.contactEmail, quote.extReference,
quote.purchaseOrderNumber, quote.linkedTicket
FROM [dbo].licenseEntitlement LIC WITH(NOLOCK)
INNER JOIN [dbo].quote WITH(NOLOCK)
ON quote.quoteId = SUBSTRING(LIC.entComments, 12,
PATINDEX('% Created%', LIC.entComments) - 12)
INNER JOIN sophos521.dbo.computersanddeletedcomputers COMPS WITH(NOLOCK)
ON COMPS.name = entid and IsNumeric(COMPS.name) = 1
WHERE (LIC.entType = 'AVS')
AND (LIC.entComments LIKE 'OV Order + %')
and (entenddate < '4/1/2014')
ORDER BY LIC.entEndDate
GO
Then, you would export from test__View_1 (or whatever real name you choose for it), as if test__View_1 was the table name.
FYI, after the first time you've executed the above -- after you've "created" the view -- then from then on, the view's first line (during modifications) changes, from CREATE VIEW, to ALTER VIEW.
((And, aside from the bug question... in your WHERE clause, did you intend entComments LIKE 'OV Order + %', or was that really intended to be entComments LIKE 'OV Order%'? I've made that change, in the alternative example code, below.))
Note: if you're going to be exporting repeatedly (or re-using) the output from one run, and especially if your query is slow or hogs the machine... then instead of a VIEW, you might prefer a SELECT INTO, to create a table once, which can be quickly re-used. (I would also choose SELECT INTO rather than CREATE VIEW, when developing a one-time-only query for export.)
IF EXISTS (SELECT * FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME = 'zz_LIC_ENT_DETAIL')
DROP TABLE [dbo].zz_LIC_ENT_DETAIL
SELECT LIC.entID, LIC.entStartDate, LIC.entEndDate,
quote.quoteId, quote.accountId, quote.clientId, quote.clientName,
quote.contactName, quote.contactEmail, quote.extReference,
quote.purchaseOrderNumber, quote.linkedTicket
INTO [dbo].zz_LIC_ENT_DETAIL
FROM [dbo].licenseEntitlement LIC WITH(NOLOCK)
INNER JOIN [dbo].quote WITH(NOLOCK)
ON quote.quoteId = SUBSTRING(LIC.entComments, 12,
PATINDEX('% Created%', LIC.entComments) - 12)
INNER JOIN sophos521.dbo.computersanddeletedcomputers COMPS WITH(NOLOCK)
ON COMPS.name = LIC.entid and IsNumeric(COMPS.name) = 1
WHERE (LIC.entType = 'AVS')
AND (LIC.entComments LIKE 'OV Order%')
and (LIC.entenddate < '4/1/2014')
ORDER BY LIC.entEndDate
Then, you would of course export from table zz_LIC_ENT_DETAIL (or whatever table name you chose).
Hope that helps...
It might be easier to right click query results window and choosing Save Results As (CSV)..
To append the column names in the first row you'd also need to modify your query in this way (note the cast for int or datetime columns):
select 'col1', 'col2', 'col3'
union all
select cast(id as varchar(10)), name, cast(someinfo as varchar(28))
from Question1355876
I'm trying to build an XE in order to find out which of our internal apps (that don't have app names and thus show up as .Net SQLClient Data Provider) are hitting particular servers. Ideally, I'd like to get the name of the Client and Database , but not sure if I can do that in one XE.
I figured for ease of use, I'd use a histogram/asynchronous_bucketizer, and save counts of what's trying to hit and how often. However, I can't seem to get it work on 2012, much less 2008. If I use sqlserver.existing_connection it works, but only gives me the count when it connects. I want to get counts during the day and see how often it occurs from each server, so I tried preconnect_completed. Is this the right event?
Also, and part of the reason I'm using XE, is that those servers can get thousands of calls a minute.
Here's what I've come up with thus far, which works but only gives me current SSMS connections that match - obviously, I'll change that to the .Net SQLClient Data Provider.
CREATE EVENT SESSION UnknownAppHosts
ON SERVER
ADD EVENT sqlserver.existing_connection(
ACTION(sqlserver.client_hostname)
WHERE ([sqlserver].[client_app_name] LIKE 'Microsoft SQL Server Management%')
)
ADD TARGET package0.histogram
( SET slots = 50,
filtering_event_name='sqlserver.existing_connection',
source_type=1,
source='sqlserver.client_hostname'
)
WITH(MAX_DISPATCH_LATENCY =1SECONDS);
GO
Aha! It's login, not preconnect_starting or preconnect_completed.
CREATE EVENT SESSION UnknownAppHosts
ON SERVER
ADD EVENT sqlserver.login(
ACTION(sqlserver.client_hostname)
WHERE ([sqlserver].[client_app_name] LIKE 'Microsoft SQL Server Management%')
)
ADD TARGET package0.histogram
( SET slots = 50,
filtering_event_name='sqlserver.login',
source_type=1,
source='sqlserver.client_hostname'
)
WITH(MAX_DISPATCH_LATENCY =1SECONDS);
GO
Then to query it, some awesome code I made horrid:
-- Parse the session data to determine the databases being used.
SELECT slot.value('./#count', 'int') AS [Count] ,
slot.query('./value').value('.', 'varchar(20)')
FROM
(
SELECT CAST(target_data AS XML) AS target_data
FROM sys.dm_xe_session_targets AS t
INNER JOIN sys.dm_xe_sessions AS s
ON t.event_session_address = s.address
WHERE s.name = 'UnknownAppHosts'
AND t.target_name = 'Histogram') AS tgt(target_data)
CROSS APPLY target_data.nodes('/HistogramTarget/Slot') AS bucket(slot)
ORDER BY slot.value('./#count', 'int') DESC