I am trying to get from my query history the pct_table_scan as
select
query_id
, (partitions_scanned / nullif(partitions_total,0)) * 100 as pct_table_scan
from table(information_schema.query_history());
from information_schema.query_history(). However, I see in the documentation that the partitions_% columns are available in the query_history from the account usage , which I don't have access to, but they are missing from the information_schema.query_history, which I can access.
Is there any way I can access the partitions_% columns from the information schema?
As you see in the documentation, these columns are not available in the output of the table function (information_schema.query_history), so you can't access this information through the function.
You may submit a feature request to Snowflake Support to add these columns to the query_history table function.
Related
I have set of views in the two schema under one database , i wanted to know which views are getting most used by external access like (Power BI , BO..) . Is there any way in snowflake to identify that?
Everytime an external tool accesses a Snowflake object (Views in your case), this is captured in Query History in Snowflake.
To compare which Views are most used, you can compare the number of times each of those Views is accessed.
Following SQL query accesses the query history, and filters by View name, and gives the number of times a View was accessed, in a given timeframe:
select count(1) as "ACCESS_COUNT" from snowflake.account_usage.query_history where start_time between '<yyyy-mm-dd hh:mm:ss>' and '<yyyy-mm-dd hh:mm:ss>' and database_name = '<db_name>' and query_text ilike '%<view_name>%' ;
Please note that the result will also include the count a View was accessed by any other users as well (and not just by external tools)
To refine your search, you can add more filters. Here's the list of columns in query_history View.
I am trying to take the count from the object Lead using SOQL.
When I am hitting the below mention query, I am getting the results.
select 'Lead' as source_table,count(*) as source_count from Lead
But when I am trying to give where condition with this query, then it is throwing me the error.
select 'Lead' as source_table,count() as source_count from Lead where
CreatedDate > 2020-02-24T09:43:51Z
Is there anything that I am missing.
Your query is not a valid SOQL, where exactly are you doing it? In real Salesforce (Apex, developer console, Salesforce APIs) or do you work on some copy of data imported to MSSQL for example?
Even your basic form (select 'Lead' as source_table,count(*) as source_count from Lead) will not parse OK in SOQL. There's no AS keyword and no dummy columns.
Closest would be
SELECT COUNT(Id) source_count
FROM Lead
WHERE CreatedDate > 2020-02-24T09:43:51Z
There will be no artificial column with "Lead" in it. If you need some key-value thing across multiple database tables you'd have to process results in Apex, maybe make a helper class to store results, maybe a Map<String, Integer> would be enough. But if you're doing it as API access then much better idea would be to access the dedicated APIs
getUpdated REST API - has "updated" and "deleted" versions, if you really need "created" it's not that great unless you filter them out later somehow
record count across multiple objects - but it's really count in table, similar to Setup -> Storage Usage. No way to pass a WHERE clause
ask your SF admin to build a report that does what you need (you don't need details, just a count, right?), then access the results with Analytics API. Or even build the report on the fly: example
I am using the Salesforce SOQL snap in a SnapLogic integration between our Salesforce instance and an S3 bucket.
I am trying to use a SOQL query in the Salesforce SOQL snap field "SOQL query*" to return the column names of an object. For example, I want to run a SOQL query to return the column names of the "Account" object.
I am doing this because SOQL does not allow "Select *". I have seen code solutions in Apex for this, but I am looking for a way to do it using only a SOQL query.
You want to query metadata? Names of available tables, names of columns you can see in each table, maybe types instead of real Account/Contact/... data, correct?
You might have to bump the version of the API up a bit, current is 47 / 48 so some objects might not be visible in your current one. Also - what API options you have? SOAP, REST? Is "Tooling API" an option? Because it has very nice official FieldDefinition table to pull this.
It's not perfect but this could get you started:
SELECT EntityDefinition.QualifiedApiName, QualifiedApiName, DataType
FROM FieldDefinition
WHERE EntityDefinition.QualifiedApiName IN ('Account', 'Contact', 'myNamespace__myCustomObject__c')
I don't see the table in the REST API reference but it seems to query OK in Workbench so there's hope.
Generally try to Google around about EntityDefinition, FieldDefinition, EntityParticle... For example this is a decent shot at learning which tables are visible to you:
SELECT KeyPrefix, QualifiedApiName, Label, IsQueryable, IsDeprecatedAndHidden, IsCustomSetting
FROM EntityDefinition
WHERE IsCustomizable = true AND IsCustomSetting = false
Or in a pinch you could try to see which fields your user has permission to query. It's bit roundabout way to do it but I have no idea which tables your connector can "see".
Starting from API version 51.0 there's the FIELDS() function available: it lets you query all fields of a given object similar to "SELECT *"
Example:
SELECT FIELDS(ALL) FROM User LIMIT 200
Reference:
https://developer.salesforce.com/docs/atlas.en-us.soql_sosl.meta/soql_sosl/sforce_api_calls_soql_select_fields.htm
Is there any option to create a custom Profile Request for SSIS Data Profiling Task?
At the moment there are 5 standard profile requests under SSIS Data Profiling task:
Column Null Ratio Profile Request
Column Statistics Profile Request
Column Length Distribution Profile Request
Column Value Distribution Profile Request
Candidate Key Profile Request
I need to add another one (Custom one) to get summary of all numeric values.
Thanks in advance for your helps.
Based on this Microsoft Documentation, SSIS Data profiling Task has only 5 main profiles (listed on your question) and there is no option to add a custom profile.
For a similar reason, i will create an Execute SQL Task to achieve that, you can use the aggregate functions you need and ISNUMERIC function in the where clause :
SELECT MAX(CAST([Column] AS BIGINT)) -- Maximum value
,MIN(CAST([Column] AS BIGINT)) -- Minimum value
,COUNT(Column) -- Count values
,COUNT(DISTINCT [Column]) -- Count distinct values
,AVG(CAST([Column] AS BIGINT)) -- Average
,SUM(CAST([Column] AS BIGINT)) -- Sum
FROM TABLE
WHERE ISNUMERIC([Column]) = 1
I think what you want to do here is create a computed column that is populated with your source column only if IsNumeric(SourceColumn) = 1.
Then create a profile task using Column Value Distribution Profile Request on the computed column, with ValueDistributionOption set to AllValues.
Edit:
To further clarify, the computed column doesn't have to be a task in SSIS, although that's how I was thinking about it when I came up with my answer. You could ALTER the table you want to profile, adding the computed column, and then create the Profile Task as I explained above.
I was also under the assumption that you wanted to profile the values of a single column. If you're wanting to do this for multiple columns, or need to profile the summary values aggregated from details records, then this answer may not be the best solution.
As the title says, I'm trying to filter a SQL table by the values in a collection. I'm attempting to use the Filter() function and an in formula to achieve this.
ClearCollect(NewCollection, Filter('SqlTable', ID in ExistingCollection.ID))
... where ID is a column belonging to 'SqlTable'.
I'm getting the error "Right side of 'in' operator is not a column name". My Collection 'ExistingCollection' has data in it, and the SqlTable also has data.
Possible issue may be the size of the Expenses table, it has 20k plus rows, but this filter should return a very small subset of that, ~200 rows. Am I running into the 5000 row limit here? Or is this kind of filtering not possible?
You can ignore the error message, as it is not correct. In any case it is not an error but a blue underline warning. This means that the query will not be delegated so that results will only come from the first 500 records of SqlTable.
You can test this by including in ExistingCollection.ID a value that is present in the first 500 records of SqlTable.
PowerApps documentation currently says that the in operator is delegated for SQL Server, but in fact it is not in the usage as a membership operator.
Please see point 5 of this reference.
The reference talks about Azure, but the same limitations apply to SQL Server.