I Use DataCollection on SSMA\Object Explorer\Management\DataCollection in 2 day ago.
But each time I get report from Data Collection, no data available to display. The following image is a sample Server Activity History report of data collection reports.
this query will confirm if a snapshot was actually taken on that date: (change the name of your Data Management Warehouse)
SELECT [snapshot_time_id]
,[snapshot_time]
FROM [DATA_MANAGEMENT_WAREHOUSE].[core].[snapshot_timetable_internal]
WHERE CAST(snapshot_time AS DATE) = CAST('2014-06-19' AS DATE)
if there is no result set, run without the filter to confirm when the snapshots first started and what the interval is.
SELECT [snapshot_time_id]
,[snapshot_time]
FROM [DATA_MANAGEMENT_WAREHOUSE].[core].[snapshot_timetable_internal]
Then, if you have no results, check to see if the Agent service is on. I believe it could be related to one of these things.
EDIT: added another query to check for wait stats.
SELECT [wait_type]
,[waiting_tasks_count]
,[wait_time_ms]
,[signal_wait_time_ms]
,[collection_time]
,[snapshot_id]
FROM [DATA_MANAGEMENT_WAREHOUSE].[snapshots].[os_wait_stats]
Related
We have an API that queries an Influx database and a report functionality was implemented so the user can query data using a start and end date.
The problem is that when a longer period is chosen(usually more than 8 weeks), we get a timeout from influx, query takes around 13 seconds to run. When the query returns a dataset successfully, we store that in cache.
The most time-consuming part of the query is probably comparison and averages we do, something like this:
SELECT mean("value") AS "mean", min("value") AS "min", max("value") AS "max"
FROM $MEASUREMENT
WHERE time >= $startDate AND time < $endDate
AND ("field" = 'myFieldValue' )
GROUP BY "tagname"
What would be the best approach to fix this? I can of course limit the amount of weeks the user can choose, but I guess that's not the ideal fix.
How would you approach this? Increase timeout? Batch query? Any database optimization to be able to run this faster?
In such cases where you allow user to select in days, I would suggest to have another table that stores the result (min, max and avg) of each day as a document. This table can be populated using some job after end of the day.
You can also think changing the document per day to per week or per month, based on how you plot the values. You can also add more fields like in your case, tagname and other fields.
Reason why this is superior to using a cache: When you use a cache, you can store the result of the query, so you have to compute for every different combination in realtime. However, in this case, the cumulative results are already available with much smaller dataset to compute.
Based on your query, I assume you are using InfluxDB v1.X. You could try Continuous Queries which are InfluxQL queries that run automatically and periodically on realtime data and store query results in a specified measurement.
In your case, for each report, you could generate a CQ and let your users to query it.
e.g.:
Step 1: create a CQ
CREATE CONTINUOUS QUERY "cq_basic_rp" ON "db"
BEGIN
SELECT mean("value") AS "mean", min("value") AS "min", max("value") AS "max"
INTO "mean_min_max"
FROM $MEASUREMENT
WHERE "field" = 'myFieldValue' // note that the time filter is not here
GROUP BY time(1h), "tagname" // here you can define the job interval
END
Step 2: Query against that CQ
SELECT * FROM "mean_min_max"
WHERE time >= $startDate AND time < $endDate // here you can pass the user's time filter
Since you already ask InfluxDB to run these aggregates continuously based on the specified interval, you should be able to trade space for time.
I am trying to receive a list of all opportunities that are created/updated/deleted since a given date. When I run query: SELECT Id FROM Opportunity WHERE SystemModStamp >= '2022-09-20' AND isDeleted = TRUE ALL ROWS I get a response stating "message": "ALL ROWS not allowed in this context", "errorCode": "MALFORMED_QUERY".
Going over the salesforce documentation I found out using queryAll() api method we can also get the opportunities which are created/updated/deleted. But I am not able to find an example of how to use this api. Any help in this direction would be highly appreciated.
Salesforce queryAll documentation: https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/resources_queryall.htm
Have you tried using queryAll instead of query? Start with
/services/data/v54.0/queryAll?q=SELECT+Id,+Name,+isDeleted,+StageName+FROM+Opportunity+ORDER+BY+isDeleted+DESC+LIMIT+20
and experiment, remove the ORDER BY, put filter by date...
SELECT Id, Name, isDeleted
FROM Opportunity
WHERE LastModifiedDate >2022-09-01T00:00:00Z
There's also dedicated "data replication API" which you can use to get opportunities updated / deleted in given time window (imagine an oppty that's edited 3 times in September, once in October - the last modified date will happily say October, how would you retrieve the fact it was touched in September too?). It'll give you just the IDs, you'd have to collect them and send the query for actual data - but it's something.
We have ssrs 2008 r2 rdls. I open an SSRS report and keep it open in my browser for more than 30 mins. If I now click any drill downs either through an action or goto report. I don't want the report get me that info. It should state that the report is expired.
Is there a solution already available . Do future version SQL server have a solution for this.
There not a built in way to do this natively in SSRS.
I haven't heard of anyone asking about a requirement like this before so I doubt that there's something already in the works to add it.
I don't know JavaScript enough to know whether or not you can add logic to the JavaScript URL to where it would create a blank link or link to a Page Expired page.
="javascript:void(window.open('https://na5.salesforce.com/','_blank'))"
You could create a hidden datetime parameter on the main report and set its default value to the current time: =Now(). Let's call this parameter #MainReportRuntime.
Make this same hidden parameter on your subreports and pass #MainReportRuntime through to the subreports.
Modify the SQL of the subreport to only return data when the date is within half an hour of the main report runtime and add the report expiry message when it is greater than half an hour old:
SELECT SomeField
FROM SomeTable
WHERE #MainReportRuntime >= DateAdd(mi, -30, Current_Timestamp)
UNION ALL
SELECT 'Report has expired - please rerun report to get current data'
WHERE #MainReportRuntime < DateAdd(mi, -30, Current_Timestamp)
I have a pipeline that daily extracts newly updated data from an SQLDB table into a .tsv file in Data Lake Store.
That data will be used later to append/update to an existing table.
To select only the newly updated rows from SQLDB, I used a sqlReaderQuery to select from a table with a WHERE clause ("WHERE DATEDIFF(day,c.UpdatedOn,GETDATE())=1") at the end which allows me to select only rows that have been updated (c.UpdatedOn) one day before the execution of the pipeline.
The problem is:
Imagine that the pipeline is paused for 1 week for maintenance. Then, when I resume the pipeline, it will only retrieve the rows added in the last day of the previous week...
Is there any way I can use a system variable that indicates the last time the pipeline ran?
So I could replace the GETDATE() in the WHERE clause for the system variable, for example:
WHERE DATEDIFF(day,c.UpdatedOn,DateOfPipelineLastExecution)<0
Thank you
Each time your activity runs it runs with respect to a specific window of time, and you should use the SliceStart and SliceEnd variables in your SQL Query to extract the data for that window. See https://learn.microsoft.com/en-us/azure/data-factory/v1/data-factory-scheduling-and-execution
https://learn.microsoft.com/en-us/azure/data-factory/v1/data-factory-functions-variables#data-factory-system-variables
As David says, you should use SliceStart and SliceEnd variables so that each slice gets you the information you need for everyday, no matter what day its executed on. For the example you gave, if you stop the pipeline for 1 week and then resume it, the pipeline will copy 7 slices corresponding to each day.
Your where clause should look something like this:
$$Text.Format( 'SELECT ... WHERE DATEDIFF(day,c.UpdatedOn, \\'{0:yyyyMMdd-HH}\\'')=1, SliceEnd)
We have a SQL statement that executes in 0:07 seconds in Management Studio, its a query that returns some data like:
[Company Department] - [Exprense Type] - [Year/Month] - [Foreseen Budget] - [Expenses]
It's a 53k lines result. One calculated field is created on the dataset (Budget - Expenses).
In the SSRS report we have a matrix that shows the data like:
Columns: [Year/Month] (Group)
[Foreseen Budget] [Expenses] [Foreseen Budget - Expenses]
Lines: [Company Department][Expense Type]
When I try to generate the report I get 5 minutes in SSRS.
In the log I get :
TimeDataRetrieval:440
TimeProcessing: 8056
TimeRendering:39603
Start Time: 2016-08-24 16:40:26.030
End Time: 2016-08-24 16:41:14.220
It's like 1 minute in SSRS, which is acceptable, but in real execution the report only appears 5 minutes later.
What is happening, and how can I improve the real execution time?
What format are you rendering as? Are you viewing the rows from browser? It will take quite a bit of time for the SSRS engine to render and draw up 53K rows.
Here are a few suggestions:
You could make use of pagination - Only show a few rows at a time? Change the datasource to bring back a few rows at a time selecting from a temp table as Tyron suggested
Have you tried to use report process caching? This could help reduce the processing time
Ive described an example on this post:
SSRS with custom paging
As some suggestions came, I got that the problem was the total lines and then group then by date.
So I made a sollution in 2 steps:
1. I made a temptable with all the results
2. I worked in summarize the data in the table not in SSRS
Doing that I got a 1 minute report rendering.
Thanks folks for the help!