if I import an .xlsx file from Salesforce into Power BI, there are instances where there are merged rows (number may vary) for a particular record.
Essentially there are merged rows that appear as one row and when one tries to import this data, the field will show it as 'Blank' when in fact it is non-empty.
Here is an example:
http://s000.tinyupload.com/?file_id=31165804604333631590
How would I write the DAX query for this? Is this the best route?
Thanks for your help.
Edit the query using Power Query, and use Table.FillDown to fill the blanks.
For example:
let
Source = Excel.Workbook(File.Contents("C:\Temp\Salesforce.xlsx"), null, true),
Sheet1_Sheet = Source{[Item="Sheet1",Kind="Sheet"]}[Data],
#"Promoted Headers" = Table.PromoteHeaders(Sheet1_Sheet, [PromoteAllScalars=true]),
#"Filled Down" = Table.FillDown(#"Promoted Headers",{"Vendor", "Shipping City"})
in
#"Filled Down"
Related
I have a general ledger transactions down to the individual journal bookings. I am working on a process to rebuild these booking from the bottoms up using tables and other information that is located in other places. To start, I have an SSIS package that pulls in 3-4 different "Divisions" worth of data.
In one case, there are over 600k lines, and I'll need at most 50k. The 600k being loaded into a table takes a while. I was looking for a way to head that off. If I were doing it in SQL Server, I'd do something like:
SELECT * FROM C601
WHERE (COST_CENTER = 5U AND ACCOUNT = 1100001)
OR (COST_CENTER = 5U AND ACCOUNT = 1300001)
I'd have about 12-13 total WHERE items, but would reduce it to maybe 10% of the original items. Is there a way to filter the flat file loading in SSIS with far fewer items before I load the SQL Server table as I would with SQL above?
Use a Conditional Split Transformation
1st approach
Add a similar expression:
[COST_CENTER] = "5U" && ([ACCOUNT] = 1100001 || [ACCOUNT] = 1300001)
2nd approach
Or you can add two split expression as following:
COST_CENTER]!= "5U" || [ACCOUNT]!= 1100001
And
[COST_CENTER] != "5U" || [ACCOUNT] != 1300001
Then you can use the Conditional Split default output to get the desired result.
I want to do a running total with power query like I did with Tableau software before. Does anyone have ideas, thanks in advance!
Apologies for the very late answer - this challenge has been nagging at me for many months.
There are few solutions floating around forums and blogs but they all seem to need pages of custom M code. Some also cant meet the common requirement of needing to restart the running total when a group changes.
So I came up a with a technique you can build without writing any custom code - you just click around in the Power Query window and add one very simple Custom Column.
The key steps are:
Add an Index column
Group By the column(s) that define your groups, and add an "All Rows" column
Duplicate that "All Rows" column
Expand both the original and copy of the "All Rows" column (at this point you have a "cross-product" result of every combination of rows, within each group)
Add a Custom Column "Cumulative" to determine which of the combination rows to include in the Running Total, e.g. [Index] >= [#"All Rows - Copy.Index"]
Filter on the Custom Column "Cumulative" = TRUE
Group By the original columns and Sum the copy of the target field
I built a working solution which you can download from my OneDrive and try out:
http://1drv.ms/1AzPAZp
It's the file: Power Query demo - Running Total
Very easy.
Step 1: Add an index
#"Added Index" = Table.AddIndexColumn(#"**Prior Step**", "Index", 0, 1)
Step 2: Add a running total
#"Added Running Total" = Table.AddColumn(#"Added Index", "Running Total, each List.Sum(List.FirstN(#"Added Index"[**Column to Add Values From**],[Index]+1)))
Using Excel 2016,
Add Index named "RunningTotal"
Go to Query -> Edit -> Add Reference Query (Index must be a column within Source**
Go to View -> Advanced Editor and paste the below code to create a function:
Rename the query to fnRunningTotal
= (tbl as table, sumcolumn as text, rowindex as number) =>
let
#"Removed Other Columns" = Table.SelectColumns(tbl,{sumcolumn, "RunningTotal"}),
#"Filtered Rows" = Table.SelectRows(#"Removed Other Columns", each [RunningTotal] <= rowindex),
#"Renamed Columns" = Table.RenameColumns(#"Filtered Rows",{{sumcolumn, "Temp"}}),
#"Grouped Rows" = Table.Group(#"Renamed Columns", {}, {{"FinalRunningTotal", each List.Sum([Temp]), type number}}),
FinalRunningTotal = Record.Field(#"Grouped Rows"{0},"FinalRunningTotal")
in
FinalRunningTotal
To use the function:
= Table.AddColumn(Source, "RunningTotalAmount", each fnRunningTotal(Source,"MarketValue",[RunningTotal]))
I mean this code from Adam works but with adding " behind "Running Total:
Step 1: Add an index
"Added Index" = Table.AddIndexColumn(#"Prior Step", "Index", 0, 1)
Step 2: Add a running total
"Added Running Total" = Table.AddColumn(#"Added Index", "Running Total", each List.Sum(List.FirstN(#"Added Index"[Column to Add Values From],[Index]+1)))
I think if it is possible in Power Query it would be very complicated. It's just the wrong tool. Use Power Query to bring the data into Power Pivot and create the running total there instead. It is a standard pattern. See this excellent guide: http://www.daxpatterns.com/cumulative-total/
I believe the best technique so far is creating a list and then creating another column using List.Sum. This does not require any complicated codes. Just point and click and will solve the issue.
More details:
https://www.sumproduct.com/blog/article/power-query-one-route-to-a-running-total
My favorites
Cumulative on single column, no grouping
#"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1),
#"Added Cum Total" = Table.AddColumn(#"Added Index", "CumTotal", each List.Sum(List.FirstN(#"Added Index"[ColumnNameDoingCumulative],[Index]+1))),
Cumulative on a single column, multiple grouping
#"Added Index" = Table.AddIndexColumn(Source, "Index", 0, 1),
#"Added Custom" = Table.AddColumn(#"Added Index","CumTotal",(i)=>List.Sum(Table.SelectRows(#"Added Index", each [Group1Column]=i[Group1Column] and [Group2Column]=i[Group2Column] and [Index]<=i[Index]) [ColumnNameDoingCumulative]), type number ),
I have a single table with contact info and about 25 checkboxes. I have a form to input the contact info, and almost all the checkboxes are to specify which power tool was purchased, with the last two checkboxes specifying whether marketing should come via US Mail or Email.
My end-game is to have a CSV file containing the name, address, and email of a subset of all the records. Which records go in is simple: if I check "chainsaw" and "email", then everyone who didn't buy a chainsaw AND want email communication is filtered out. If I check "chainsaw" and "weedwacker" and "US mail" I want all records who bought EITHER a chainsaw OR a weedwacker OR both AND want US mail communications.
I envision using a form with checkboxes where checking any box filters the records to only include those records where that checkbox = yes in the table (for the power tools), and the mail or email filters it further, and then to generate a query that contains those records, then export as a CSV.
My Google-ing skills are good, my VBA coding skills are nonexistent (I'm an HTML/CSS/Joomla guy but got handed this mini project), and the results of searching Google and stackoverflow have yielded nothing that would guide me in the right direction.
I know the table structure is not optimal, but it is manageable for the person I'm making it for who would struggle with more than one table in Access. And it's based off of some archaic database software and they want the structure to stay the same.
You will need to build a sql statement in your code. It can start with:
sql = "select * from MyTable where ID > 0" 'the "Where ID > 0" is here so the remaining sql additions can all say AND
then have a series of if statments that will add to the sql statement:
if chkChainsaw = -1 then
sql = sql & " AND Chainsaw = 1"
end if
if chkweedwacker = -1 then
sql = sql & " AND weedwacker= 1"
end if
When you are done building your sql statement, you can build a recordset and export a csv file from that.
dim rs as recordset
set rs = currentdb.openrecordset(sql, dbopendynaset)
do while not rs.eof
'export data
rs.movenext
loop
I am trying to connect to a SQLite database and have a method that specifies a specific row from the database (the first column in the database is “ID” and is a primary key) then extract the information from a few other columns in that row and display them in text fields.
This will be used for a simple Trivia game I am making; I will later make a random method that will choose the row at random.
I have been struggling with this problem for several weeks and I have been through loads of tutorials but all of them deal with displaying the data in a table view, I want to display it simply on text fields in a View based app. I am fairly confused at this point so any help starting from loading the database to displaying the data in the text fields would be GREATLY APPRECIATED!
Thanks!
Link to libsqlite3.dylib (and import <sqlite3.h>) to access the power of SQLite. There are a number of lightweight Objective-C front ends and I suggest you pick one. In this example, I use fmdb (https://github.com/ccgus/fmdb) to read the names of people out of a previously created database:
NSString* docsdir = [NSSearchPathForDirectoriesInDomains(
NSDocumentDirectory, NSUserDomainMask, YES) lastObject];
NSString* dbpath = [docsdir stringByAppendingPathComponent:#"people.db"];
FMDatabase* db = [FMDatabase databaseWithPath:dbpath];
if (![db open]) {
NSLog(#"Ooops");
return;
}
FMResultSet *rs = [db executeQuery:#"select * from people"];
while ([rs next]) {
NSLog(#"%# %#",
[rs stringForColumn:#"firstname"],
[rs stringForColumn:#"lastname"]);
}
[db close];
/* output:
Snidely Whiplash
Dudley Doright
*/
That illustrates talking to the database; knowing SQL is up to you (and is a different topic). You can include a previously constructed SQLite file in your app bundle, but you can't write to it there; the solution is to copy it from your app bundle into another location, such as the Documents directory, before you start working with it.
Finally, to put strings into text fields (UITextField), set their text property. So for example instead of the while loop shown above, where I log the database results, I could use those results to set text field values:
myTextField.text = [rs stringForColumn:#"firstname"];
myOtherTextField.text = [rs stringForColumn:#"lastname"];
I'm writing an NHibernate criteria that selects data supporting paging. I'm using the COUNT(*) OVER() expression from SQL Server 2005(+) to get hold of the total number of available rows, as suggested by Ayende Rahien. I need that number to be able to calculate how many pages there are in total. The beauty of this solution is that I don't need to execute a second query to get hold of the row count.
However, I can't seem to manage to write a working criteria (Ayende only provides an HQL query).
Here's an SQL query that shows what I want and it works just fine. Note that I intentionally left out the actual paging logic to focus on the problem:
SELECT Items.*, COUNT(*) OVER() AS rowcount
FROM Items
Here's the HQL:
select
item, rowcount()
from
Item item
Note that the rowcount() function is registered in a custom NHibernate dialect and resolves to COUNT(*) OVER() in SQL.
A requirement is that the query is expressed using a criteria. Unfortunately, I don't know how to get it right:
var query = Session
.CreateCriteria<Item>("item")
.SetProjection(
Projections.SqlFunction("rowcount", NHibernateUtil.Int32));
Whenever I add a projection, NHibernate doesn't select item (like it would without a projection), just the rowcount() while I really need both. Also, I can't seem to project item as a whole, only it's properties and I really don't want to list all of them.
I hope someone has a solution to this. Thanks anyway.
I think it is not possible in Criteria, it has some limits.
You could get the id and load items in a subsequent query:
var query = Session
.CreateCriteria<Item>("item")
.SetProjection(Projections.ProjectionList()
.Add(Projections.SqlFunction("rowcount", NHibernateUtil.Int32))
.Add(Projections.Id()));
If you don't like it, use HQL, you can set the maximal number of results there too:
IList<Item> result = Session
.CreateQuery("select item, rowcount() from item where ..." )
.SetMaxResult(100)
.List<Item>();
Use CreateMultiCriteria.
You can execute 2 simple statements with only one hit to the DB that way.
I am wondering why using Criteria is a requirement. Can't you use session.CreateSQLQuery? If you really must do it in one query, I would have suggested pulling back the Item objects and the count, like:
select {item.*}, count(*) over()
from Item {item}
...this way you can get back Item objects from your query, along with the count. If you experience a problem with Hibernate's caching, you can also configure the query spaces (entity/table caches) associated with a native query so that stale query cache entries will be cleared automatically.
If I understand your question properly, I have a solution. I struggled quite a bit with this same problem.
Let me quickly describe the problem I had, to make sure we're on the same page. My problem came down to paging. I want to display 10 records in the UI, but I also want to know the total number of records that matched the filter criteria. I wanted to accomplish this using the NH criteria API, but when adding a projection for row count, my query no longer worked, and I wouldn't get any results (I don't remember the specific error, but it sounds like what you're getting).
Here's my solution (copy & paste from my current production code). Note that "SessionError" is the name of the business entity I'm retrieving paged data for, according to 3 filter criterion: IsDev, IsRead, and IsResolved.
ICriteria crit = CurrentSession.CreateCriteria(typeof (SessionError))
.Add(Restrictions.Eq("WebApp", this));
if (isDev.HasValue)
crit.Add(Restrictions.Eq("IsDev", isDev.Value));
if (isRead.HasValue)
crit.Add(Restrictions.Eq("IsRead", isRead.Value));
if (isResolved.HasValue)
crit.Add(Restrictions.Eq("IsResolved", isResolved.Value));
// Order by most recent
crit.AddOrder(Order.Desc("DateCreated"));
// Copy the ICriteria query to get a row count as well
ICriteria critCount = CriteriaTransformer.Clone(crit)
.SetProjection(Projections.RowCountInt64());
critCount.Orders.Clear();
// NOW add the paging vars to the original query
crit = crit
.SetMaxResults(pageSize)
.SetFirstResult(pageNum_oneBased * pageSize);
// Set up a multi criteria to get your data in a single trip to the database
IMultiCriteria multCrit = CurrentSession.CreateMultiCriteria()
.Add(crit)
.Add(critCount);
// Get the results
IList results = multCrit.List();
List<SessionError> sessionErrors = new List<SessionError>();
foreach (SessionError sessErr in ((IList)results[0]))
sessionErrors.Add(sessErr);
numResults = (long)((IList)results[1])[0];
So I create my base criteria, with optional restrictions. Then I CLONE it, and add a row count projection to the CLONED criteria. Note that I clone it before I add the paging restrictions. Then I set up an IMultiCriteria to contain the original and cloned ICriteria objects, and use the IMultiCriteria to execute both of them. Now I have my paged data from the original ICriteria (and I only dragged the data I need across the wire), and also a raw count of how many actual records matched my criteria (useful for display or creating paging links, or whatever). This strategy has worked well for me. I hope this is helpful.
I would suggest investigating custom result transformer by calling SetResultTransformer() on your session.
Create a formula property in the class mapping:
<property name="TotalRecords" formula="count(*) over()" type="Int32" not-null="true"/>;
IList<...> result = criteria.SetFirstResult(skip).SetMaxResults(take).List<...>();
totalRecords = (result != null && result.Count > 0) ? result[0].TotalRecords : 0;
return result;