I have SSAS and SSRS 2008R2. The end goal is to get the report with Daily MarketValue for each Portfolio and Security Combination. MarketValue has SCOPE calculation to Select Last Existing for Date Dimension. If the SCOPE is removed the query still takes 6 min t complete. and with SCOPE statement it timeout after 1 hour. here is my query
SELECT
NON EMPTY
{[Measures].[MarketValue]} ON COLUMNS
,NON EMPTY
{
[Portfolio].[PortfolioName].[PortfolioName].ALLMEMBERS*
[Effective Date].[Effective Date].[Effective Date].ALLMEMBERS*
[Security].[Symbol].[Symbol].ALLMEMBERS
}
DIMENSION PROPERTIES
MEMBER_CAPTION
,MEMBER_UNIQUE_NAME
ON ROWS
FROM EzeDM
WHERE
(
[AsOn Date].[AsOn Date].&[2014-06-17T06:32:41.97]
,[GoldenCopy].[Frequency].&[Daily]
,[GoldenCopy].[GoldenCopyType].&[CitcoPLExposure]
,[GoldenCopy].[PointInTime].&[EOP]
,[GoldenCopy].[PositionType].&[Trade-Date]
);
The SCOPE statement I have for MarketValue Measure is
SCOPE
[Effective Date].[Effective Date].MEMBERS;
THIS =
Tail
(
(EXISTING
[Effective Date].[Effective Date].MEMBERS)
,1
).Item(0);
END SCOPE;
Security DIM has around 4K values. Portfolio DIM has around 100 Values and EffectiveDate DIM has around 400 values.
If I remove the EffectiveDate from the cross join the query is taking less than 2 seconds.
So far I have tried different combinations and found that the slowness is due to the cross join between DIM with large values in them. but then I am thinking is 4000 values in DIM is actually large? people must have done the same reporting efficiently right?
Is this a SCOPE calculation? If so why does it get slower only when EffectiveDate is in the cross join?
Appreciate any help.
EDIT:1
Adding some more details about the current environment if that helps :
We do not have Enterprise version and currently we do not have any plans to ask our clients to upgrade to Enterprise version.
Security Dimension has around 40 attribute but 2 of them will always have data and at most "up to 6" may have any data. not sure if Attribute being not used in MDX query still affects the query performance "regardless it has data or not"
After reading the "Chris Webb" blog on MDX query improvements I notice the property is true for ALL Attributes in ALL Dimension.
"AttributeHierarchyEnabled = True"
For testing I have marked FALSE to all except currently I am using.
I do not have any aggregations defined on cube and I have started with building Aggregations using "Design Aggregations" wizard. after that I profile the same reporting query and didn't see any tick for "get data from Aggregations" event.
So currently I am working on preparing/testing "Usage Based Aggregation"
EDIT:2
So I created the log table with 50% logging sampling and ran 15-20 different reporting queries Client is expecting to run and saw some data in log table. I used the Wizard for Usage Based Aggregation and let SSAS finds out Estimated Row Count.
it was strange that it did not generate any aggregations.
I also tried the approach of changing the Aggregation property to LastChild As Frank suggested and it worked great but then I realize I can not pick LastChild Value for MarketValue for all Dimension. it is Additive across Security Dimension but not across Time.
I would assume that getting rid of the whole SCOPE statement and instead setting the AggregateFunction property of the measures to LastChild or LastNonEmpty would speed up the calculation. This would require [Effective Date] to be the first dimension tagged as time, and you need SQL Server Enterprise edition for these AggregateFunctions to be available.
Related
We are using EF Core 1.1, in an ASP.NET Core app, where the following LINQ query takes about 45 seconds to a minute on its first execution. After the first execution, the subsequent executions seem to work fine.
Question: How can we improve the performance of this query. User waiting for about 45 seconds or more gives him/her an impression that probably the ASP.NET page displaying the query is broken and user moves on to another page:
var lstProjects = _context.Projects.Where(p => p.ProjectYear == FY && p.User == User_Name).OrderBy(p => p.ProjectNumber).ToList();
Execution Plan in SQL Server Query Editor: The table has 24 columns one of which is of type varchar(255), four are of type varchar(75). Others are of types int, smalldatetime, bit etc. All of the columns are needed in the query. But the WHERE clause filters the data to return about 35 rows out of about 26,000.
More details on Execution Plan
Updated comment to answer.
When using Code First there still needs to be a consideration for indexing based on the common queries run in high-traffic areas of the application. The index scan across the PK amounts to little more than a table scan so an index across the Project Year + UserName would give a boost in performance and should be considered if this is expected to be used a bit or is performance sensitive. Regardless of DB First or Code First, developers need to consider profiler results against the database in order to optimize indexing based on realistic usage. Normally the execution plan will return back suggestions for indexing. Index suggestions should appear just below the SQL statement in the execution plan. From the screen shot it isn't clear whether one was suggested or not, as there might have been a scrollbar to the right of the SQL under the pop-up stats.
In cases where a query is returning slow results but no suggestions, try re-running the query with altered parameters with the execution plan to exclude SQL from picking up pre-compiled queries.
I have one complex report which fetches records from multiple tables.
I saw at many places that SSRS does not allow multiple data tables returned from single stored procedure, that is the reason I created one stored procedure and I created six dataset for report that was filtered from shared dataset, but when I ran below query it shows that my procedure was executed for six times and that might causing the performance issue.
SELECT TOP 100 *,Itempath,parameters,
TimeDataRetrieval + TimeProcessing + TimeRendering as [total time],
TimeDataRetrieval, TimeProcessing, TimeRendering,
ByteCount, [RowCount],Source, AdditionalInfo
FROM ExecutionLog3 where ItemPath like '%GetServiceCalls%'
ORDER BY Timestart DESC
To get rid of this, I removed all dataset filters and applied filter on tablix. After that I can see that procedure was called only one time. But that does not affect the performance much.
Now, question that is coming to me is how exactly I can improve performance of SSRS report.
Note: My Query executes in 13 seconds of time and report takes almost 20 mins to execute.
Please help me to resolve this issue.
Regards,
Dhaval
I always found that SSRS filters on large tables to take forever and that any text wildcards performed even more poorly.
My advise would be to do all the "grunt work" except sorts in SQL and then do any sorts in SSRS.
Part of you problem may be that you have a large dataset and you are performing wildcard searches which don't play well with Indexes when you have the wildcard at the start of the like statement (e.g. like '%... ).
We discovered that in our .NET 4.51/nHibernate/SQL 2008 app that when we do a single large query, we get all the data, but when we page against certain large results sets, rows are missed.
(In our testing, when we do pages of 200 per page, against a result set that is 1000 or more rows, we see the problem.)
We tried the app using the SQL2000 and SQL2005 nHibernate dialects, and saw no difference in behavior. (The query syntax does evolves to use newer constructs in SQL2005, but the same data is missing when using SQL2005 dialect.)
nHibernate generates SQL queries.
We are doing paging.
If we page by 100, the way nHibernate generates the successive SQL queries is:
TOP 100 // gives us first 100
TOP 200 // gives us 2nd 100 of this block
etc
The above, without an ORDER BY / sorting at nHibernate level, has the end result that some rows never surface to nHibernate. We surmise this is due arbitrary sorting by SQL, so that rows are "moving around" within the pages (and thus "hide" from our application code).
If we do the nHibnerate query as a single shot (returning all rows), we see all the data. (This query below is generated by nhibernate.)
Will adding an order by clause (aka nHibernate sorting) help?
SELECT top 33
... <field list>
FROM
salesOrder this_ left outer join [Item] Item2_ on this_.ItemId=Item2_.ItemId
WHERE this_.AccountId = #p0
and this_.ModifiedAt > #p1
and this_.ModifiedAt <= #p2
The database is quite large, but is not fragmented, and it performance is great. The problem has only been seen on very large tables (100 million rows) and even then only when the query returns > 1000 rows AND is paged.
Is an Order By clause (a sort at the nHibernate level) the solution, or is there another path we need to take?
The answer is: ORDER BY is required.
IOW, use a sort order (on an indexed field for best results!) on the nHibernate query.
I've a Cognos report in which I've cascading prompts. The Hierarchy is defined in the image attached.
The First Parent (Division) fills the two cascading child in 3-5 seconds.
But when I select any Policy, (that will populate the two child beneath) it took around 2 minutes.
Facts:
The result set after two minutes is normal (~20 rows)
The Queries behind all the prompts are simple Select DISTINCT Col_Name
Ive created indexes on all the prompt columns.
Tried turning on the local cache and Execution Method to concurrent.
I'm on Cognos Report Studio 10.1
Any help would be much appreciated.
Thanks,
Nuh
There is an alternative to a one-off dimension table. Create a Query Subject in Framework for your AL-No prompt. In the query itself, build a query that gets distinct AL-No (you said that is fast, probably because there is an index on AL-No). Wrap that in a select that does a filter on ' #prompt('pPolicy')#' (assuming your Policy Prompt is keyed to ?pPolicy?)
This will force the Policy into the sql before it is sent to the database, but wrapping on the distinct AL-No will allow you to use the AL-No index.
select AL_NO from
(
select AL_NO, Policy_NO
from CLAIMS
group by AL_NO, Policy_NO
)
where Policy_NO = #prompt('pPolicyNo')#
Your issue is just too much table scanning. Typically, one would build a prompt page from dimension-based tables, not the fact table, though I admit that is not always possible with cascading prompts. The ideal solution is to create a one-off dimension table with these distinct values, then model that strictly for the prompts.
Watch out for indexing each field, as the indexes will not be used due to the selectivity of the values. A compound index of the fields may work instead. As with any time you are making changes to the DDL - open SQL profiler and see what SQL Cognos is generating, then run an explain plan before/after the changes.
I'm new to MDX and trying to solve the following problem. Investigated calculated members, subselects, scope statements, etc but can't quite get it to do what I want.
Let's say I'm trying to come up with the MDX equivalent to the following SQL query:
SELECT SUM(netMarketValue) net,
SUM(CASE WHEN netMarketValue > 0 THEN netMarketValue ELSE 0 END) assets,
SUM(CASE WHEN netMarketValue < 0 THEN netMarketValue ELSE 0 END) liabilities,
SUM(ABS(netMarketValue)) gross
someEntity1
FROM (
SELECT SUM(marketValue) netMarketValue, someEntity1, someEntity2
FROM <some set of tables>
GROUP BY someEntity1, someEntity2) t
GROUP BY someEntity1
In other words, I have an account ledger where I hide internal offsetting transactions (within someEntity2), then calculate assets & liabilities after aggregating them by someEntity2. Then I want to see the grand total of those assets & liabilities aggregated by the bigger entity, someEntity1.
In my MDX schema I'd presumably have a cube with dimensions for someEntity1 & someEntity2, and marketValue would be my fact table/measure. I suppose i could create another DSV that did what my subquery does (calculating net), and simply create a cube with that as my measure dimension, but I wonder if there is a better way. I'd rather not have 2 cubes (one for these net calculations and another to go to a lower level of granularity for other use cases), since it will be a lot of duplicate info in my database. These will be very large cubes.
I think you should leave the aggregation logic to the cube--that's what it does best.
In your case, I would create an account dimension, and then I would add Account Intelligence. However, this only works for the Enterprise edition of SQL Server (2005 and above).
If you happen to have the standard edition, the canonical way to do this is to use unary operators.
That's the way we used to do it with Sql Server 2000, and here you have a great example.
I think what you want is not two cubes, but one cube with two fact tables (sometimes called a constellation schema). The question was written months ago so I won't elaborate more here unless someone asks for more info.