I want to do a running total with power query like I did with Tableau software before. Does anyone have ideas, thanks in advance!
Apologies for the very late answer - this challenge has been nagging at me for many months.
There are few solutions floating around forums and blogs but they all seem to need pages of custom M code. Some also cant meet the common requirement of needing to restart the running total when a group changes.
So I came up a with a technique you can build without writing any custom code - you just click around in the Power Query window and add one very simple Custom Column.
The key steps are:
Add an Index column
Group By the column(s) that define your groups, and add an "All Rows" column
Duplicate that "All Rows" column
Expand both the original and copy of the "All Rows" column (at this point you have a "cross-product" result of every combination of rows, within each group)
Add a Custom Column "Cumulative" to determine which of the combination rows to include in the Running Total, e.g. [Index] >= [#"All Rows - Copy.Index"]
Filter on the Custom Column "Cumulative" = TRUE
Group By the original columns and Sum the copy of the target field
I built a working solution which you can download from my OneDrive and try out:
http://1drv.ms/1AzPAZp
It's the file: Power Query demo - Running Total
Very easy.
Step 1: Add an index
#"Added Index" = Table.AddIndexColumn(#"**Prior Step**", "Index", 0, 1)
Step 2: Add a running total
#"Added Running Total" = Table.AddColumn(#"Added Index", "Running Total, each List.Sum(List.FirstN(#"Added Index"[**Column to Add Values From**],[Index]+1)))
Using Excel 2016,
Add Index named "RunningTotal"
Go to Query -> Edit -> Add Reference Query (Index must be a column within Source**
Go to View -> Advanced Editor and paste the below code to create a function:
Rename the query to fnRunningTotal
= (tbl as table, sumcolumn as text, rowindex as number) =>
let
#"Removed Other Columns" = Table.SelectColumns(tbl,{sumcolumn, "RunningTotal"}),
#"Filtered Rows" = Table.SelectRows(#"Removed Other Columns", each [RunningTotal] <= rowindex),
#"Renamed Columns" = Table.RenameColumns(#"Filtered Rows",{{sumcolumn, "Temp"}}),
#"Grouped Rows" = Table.Group(#"Renamed Columns", {}, {{"FinalRunningTotal", each List.Sum([Temp]), type number}}),
FinalRunningTotal = Record.Field(#"Grouped Rows"{0},"FinalRunningTotal")
in
FinalRunningTotal
To use the function:
= Table.AddColumn(Source, "RunningTotalAmount", each fnRunningTotal(Source,"MarketValue",[RunningTotal]))
I mean this code from Adam works but with adding " behind "Running Total:
Step 1: Add an index
"Added Index" = Table.AddIndexColumn(#"Prior Step", "Index", 0, 1)
Step 2: Add a running total
"Added Running Total" = Table.AddColumn(#"Added Index", "Running Total", each List.Sum(List.FirstN(#"Added Index"[Column to Add Values From],[Index]+1)))
I think if it is possible in Power Query it would be very complicated. It's just the wrong tool. Use Power Query to bring the data into Power Pivot and create the running total there instead. It is a standard pattern. See this excellent guide: http://www.daxpatterns.com/cumulative-total/
I believe the best technique so far is creating a list and then creating another column using List.Sum. This does not require any complicated codes. Just point and click and will solve the issue.
More details:
https://www.sumproduct.com/blog/article/power-query-one-route-to-a-running-total
My favorites
Cumulative on single column, no grouping
#"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1),
#"Added Cum Total" = Table.AddColumn(#"Added Index", "CumTotal", each List.Sum(List.FirstN(#"Added Index"[ColumnNameDoingCumulative],[Index]+1))),
Cumulative on a single column, multiple grouping
#"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1),
#"Added Custom" = Table.AddColumn(#"Added Index","CumTotal",(i)=>List.Sum(Table.SelectRows(#"Added Index", each [Group1Column]=i[Group1Column] and [Group2Column]=i[Group2Column] and [Index]<=i[Index]) [ColumnNameDoingCumulative]), type number ),
Related
if I import an .xlsx file from Salesforce into Power BI, there are instances where there are merged rows (number may vary) for a particular record.
Essentially there are merged rows that appear as one row and when one tries to import this data, the field will show it as 'Blank' when in fact it is non-empty.
Here is an example:
http://s000.tinyupload.com/?file_id=31165804604333631590
How would I write the DAX query for this? Is this the best route?
Thanks for your help.
Edit the query using Power Query, and use Table.FillDown to fill the blanks.
For example:
let
Source = Excel.Workbook(File.Contents("C:\Temp\Salesforce.xlsx"), null, true),
Sheet1_Sheet = Source{[Item="Sheet1",Kind="Sheet"]}[Data],
#"Promoted Headers" = Table.PromoteHeaders(Sheet1_Sheet, [PromoteAllScalars=true]),
#"Filled Down" = Table.FillDown(#"Promoted Headers",{"Vendor", "Shipping City"})
in
#"Filled Down"
Using MS Access and I have two tables, one is categories and the other is content.
My initial SQL statement, included below,takes a count of each content associated to a category and returns the count associated with each category.
So for each CATEGORY, I'm simply trying to return another count in which I count CONTENT that have a specific user level and are not deleted for each CATEGORY.
Below is what I am struggling with as I am not certain you can actually use COUNT like this.
COUNT(IIf([CONTENT.isDeleted]=0,1,0)) - COUNT(IIf([CONTENT.userLevel]=2)) AS userLevelCount
This is the full select statement with my addition but not working.
SELECT
CATEGORY.categoryId,
CATEGORY.categoryTitle,
CATEGORY.categoryDate,
CATEGORY.userLevel,
Last(CONTENT.contentDate) AS contentDate,
CATEGORY.isDeleted AS categoryDeleted,
COUNT(IIf([CONTENT.isDeleted]=0,1,0)) AS countTotal,
COUNT(IIf([CONTENT.isDeleted]=1,[CONTENT.contentID],Null)) AS countDeleted,
COUNT([CONTENT.categoryId]) - COUNT(IIf([CONTENT.isDeleted]=1,[CONTENT.contentID],Null))AS countDifference,
COUNT(IIf([CONTENT.isDeleted]=0,1,0)) - COUNT(IIf([CONTENT.userLevel]=2)) AS userLevelCount
FROM CATEGORY
LEFT JOIN CONTENT ON
CATEGORY.categoryId = CONTENT.categoryId
GROUP BY
CATEGORY.categoryId,
CATEGORY.categoryTitle,
CATEGORY.categoryDate,
CATEGORY.userLevel,
CATEGORY.isDeleted
HAVING (((CATEGORY.isDeleted)=0))
ORDER BY
CATEGORY.categoryTitle
you should be able to use the following
SUM(IIf([CONTENT.isDeleted]=0,1,0)) - COUNT(IIf([CONTENT.userLevel]=2,1,NULL)) AS userLevelCount
COUNT will not count NULL, but it will count zero. SUM will calculate the sum of all 1's - that's a second way of achieving the same.
IIF exists in the newer SQL versions
I believe I found the solution
Count(IIf([CONTENT.userLevel]=2,[CONTENT.contentID],Null)) AS countDifference2
This will return the count difference for CONTENT for each CATEGORY that isn't deleted and has a specific user level.
I'm usually a PHP programmer, but I'm currently working on a project in MS Access 2003 and I'm a complete VBA newbie. I'm trying to do something that I could easily do in PHP but I have no idea how to do it in Access. The facts are as follows:
Tables and relevant fields:
tblItems: item_id, on_hand
tblProjects: project_id
tblProjectItems: project_id, item_id
Goal: Determine which projects I could potentially do, given the items on-hand.
I need to find a way to compare each project's required items against the items on-hand to determine if there are any items missing. If not, add the project to the list of potential projects. In PHP I would compare an array of on-hand items with an array of project items required, using the array_diff function; if no difference, add project_id to an array of potential projects.
For example, if...
$arrItemsOnHand = 1,3,4,5,6,8,10,11,15
$arrProjects[1] = 1,10
$arrProjects[2] = 8,9,12
$arrProjects[3] = 7,13
$arrProjects[4] = 1,3
$arrProjects[5] = 2,14
$arrProjects[6] = 2,5,8,10,11,15
$arrProjects[7] = 2,4,5,6,8,10,11,15
...the result should be:
$arrPotentialProjects = 1,4
Is there any way to do this in Access?
Consider a single query to reach your goal: "Determine which projects I could potentially do, given the items on-hand."
SELECT
pi.project_id,
Count(pi.item_id) AS NumberOfItems,
Sum(IIf(i.on_hand='yes', 1, 0)) AS NumberOnHand
FROM
tblProjectItems AS pi
INNER JOIN tblItems AS i
ON pi.item_id = i.item_id
GROUP BY pi.project_id
HAVING Count(pi.item_id) = Sum(IIf(i.on_hand='yes', 1, 0));
That query computes the number of required items for each project and the number of those items which are on hand.
When those two numbers don't match, that means at least one of the required items for that project is not on hand.
So the HAVING clause excludes those rows from the query result set, leaving only rows where the two numbers match --- those are the projects for which all required items are on hand.
I realize my description was not great. (Sorry.) I think it should make more sense if you run the query both with and without the HAVING clause ... and then read the description again.
Anyhow, if that query gives you what you need, I don't think you need VBA array handling for this. And if you can use that query as your form's RecordSource or as the RowSource for a list or combo box, you may not need VBA at all.
I am new to Sql Server Reporting Services. I have created my following report.
I want to remove/hide rows of Brand Total whenever it does not exist in Brand list. Like in following picture i want to remove/hide "Ethnic Total" whereas "Ethnic" Brand does not exist in "Sample Store 1".
Similary i want to reomve/hide rows of "Outfitters Total" and "Junior Total" from Section Two whereas "Outfitters" and "Junior" don't exist in "Sample Store 2".
This is the structure of my report.
And following is the expression for Net Qty of a Single Brand total.
=Sum(IIf(Fields!Brand.Value = "Outfitters", Fields!Quantity.Value, Nothing))
What should i do?
What condition should i write in expression for Row Visibility?
Thanks in Advance for help.
i hope the below comments you are looking for.
Step 1: select that particular row (Outfitlers Total, Junior Total,Ethnic Total,Store Total)
One at a time and right click and select Row Visibility Option.
Step 2 :
A Dialog box appears with 3 options
1.Show
2.Hide
3. Show or hide based on expression
Select option 3 and copy the below expression in the Expression dialog box.
=iif((Sum(IIf(Fields!Brand.Value = "Outfitters", Fields!Quantity.Value, Nothing))) is nothing ,True,False)
i hope So this will be helpful.
=IIF(Fields!TotalRems.Value=0, True, False)
Replace TotalRems with your correct field name
You can do this way:
=IIF(Fields!YourField.Value like "YourValue",false,true)
Replace "YourField" with your own one and also change "YourValue" to whatever you need.
NB, " " or '' not treated as NOTHING,
For more explanation:
SSRS – Hide Rows in a Group based on a Value
another possibility for the hiding expression is, to use a text box reference. In place of "Textbox1" in the expression below, you can use the name of the text box, which is in the crossing of column "Net Qty" and row "Ethnic Total" (or one of the other total rows you mentioned)
=Iif(IsNothing(ReportItems!Textbox1.Value),True,False)
I'm writing an NHibernate criteria that selects data supporting paging. I'm using the COUNT(*) OVER() expression from SQL Server 2005(+) to get hold of the total number of available rows, as suggested by Ayende Rahien. I need that number to be able to calculate how many pages there are in total. The beauty of this solution is that I don't need to execute a second query to get hold of the row count.
However, I can't seem to manage to write a working criteria (Ayende only provides an HQL query).
Here's an SQL query that shows what I want and it works just fine. Note that I intentionally left out the actual paging logic to focus on the problem:
SELECT Items.*, COUNT(*) OVER() AS rowcount
FROM Items
Here's the HQL:
select
item, rowcount()
from
Item item
Note that the rowcount() function is registered in a custom NHibernate dialect and resolves to COUNT(*) OVER() in SQL.
A requirement is that the query is expressed using a criteria. Unfortunately, I don't know how to get it right:
var query = Session
.CreateCriteria<Item>("item")
.SetProjection(
Projections.SqlFunction("rowcount", NHibernateUtil.Int32));
Whenever I add a projection, NHibernate doesn't select item (like it would without a projection), just the rowcount() while I really need both. Also, I can't seem to project item as a whole, only it's properties and I really don't want to list all of them.
I hope someone has a solution to this. Thanks anyway.
I think it is not possible in Criteria, it has some limits.
You could get the id and load items in a subsequent query:
var query = Session
.CreateCriteria<Item>("item")
.SetProjection(Projections.ProjectionList()
.Add(Projections.SqlFunction("rowcount", NHibernateUtil.Int32))
.Add(Projections.Id()));
If you don't like it, use HQL, you can set the maximal number of results there too:
IList<Item> result = Session
.CreateQuery("select item, rowcount() from item where ..." )
.SetMaxResult(100)
.List<Item>();
Use CreateMultiCriteria.
You can execute 2 simple statements with only one hit to the DB that way.
I am wondering why using Criteria is a requirement. Can't you use session.CreateSQLQuery? If you really must do it in one query, I would have suggested pulling back the Item objects and the count, like:
select {item.*}, count(*) over()
from Item {item}
...this way you can get back Item objects from your query, along with the count. If you experience a problem with Hibernate's caching, you can also configure the query spaces (entity/table caches) associated with a native query so that stale query cache entries will be cleared automatically.
If I understand your question properly, I have a solution. I struggled quite a bit with this same problem.
Let me quickly describe the problem I had, to make sure we're on the same page. My problem came down to paging. I want to display 10 records in the UI, but I also want to know the total number of records that matched the filter criteria. I wanted to accomplish this using the NH criteria API, but when adding a projection for row count, my query no longer worked, and I wouldn't get any results (I don't remember the specific error, but it sounds like what you're getting).
Here's my solution (copy & paste from my current production code). Note that "SessionError" is the name of the business entity I'm retrieving paged data for, according to 3 filter criterion: IsDev, IsRead, and IsResolved.
ICriteria crit = CurrentSession.CreateCriteria(typeof (SessionError))
.Add(Restrictions.Eq("WebApp", this));
if (isDev.HasValue)
crit.Add(Restrictions.Eq("IsDev", isDev.Value));
if (isRead.HasValue)
crit.Add(Restrictions.Eq("IsRead", isRead.Value));
if (isResolved.HasValue)
crit.Add(Restrictions.Eq("IsResolved", isResolved.Value));
// Order by most recent
crit.AddOrder(Order.Desc("DateCreated"));
// Copy the ICriteria query to get a row count as well
ICriteria critCount = CriteriaTransformer.Clone(crit)
.SetProjection(Projections.RowCountInt64());
critCount.Orders.Clear();
// NOW add the paging vars to the original query
crit = crit
.SetMaxResults(pageSize)
.SetFirstResult(pageNum_oneBased * pageSize);
// Set up a multi criteria to get your data in a single trip to the database
IMultiCriteria multCrit = CurrentSession.CreateMultiCriteria()
.Add(crit)
.Add(critCount);
// Get the results
IList results = multCrit.List();
List<SessionError> sessionErrors = new List<SessionError>();
foreach (SessionError sessErr in ((IList)results[0]))
sessionErrors.Add(sessErr);
numResults = (long)((IList)results[1])[0];
So I create my base criteria, with optional restrictions. Then I CLONE it, and add a row count projection to the CLONED criteria. Note that I clone it before I add the paging restrictions. Then I set up an IMultiCriteria to contain the original and cloned ICriteria objects, and use the IMultiCriteria to execute both of them. Now I have my paged data from the original ICriteria (and I only dragged the data I need across the wire), and also a raw count of how many actual records matched my criteria (useful for display or creating paging links, or whatever). This strategy has worked well for me. I hope this is helpful.
I would suggest investigating custom result transformer by calling SetResultTransformer() on your session.
Create a formula property in the class mapping:
<property name="TotalRecords" formula="count(*) over()" type="Int32" not-null="true"/>;
IList<...> result = criteria.SetFirstResult(skip).SetMaxResults(take).List<...>();
totalRecords = (result != null && result.Count > 0) ? result[0].TotalRecords : 0;
return result;