Google Sheets Query - build referenced array source dynamically - arrays

I have many sheets in my spreadsheet (sheet1,sheet2,sheet3...) and I want to add them all to array, maybe based on any call range? Now I add them manually as below:
=query(
{
INDIRECT("sheet1!$A$3:$V");
INDIRECT("sheet2!$A$3:$V");
INDIRECT("sheet3!$A$3:$V") };
"SELECT Col2, Col3, Col4, ...[etc]")
I want to create any "Settings" sheet and put here all sheets that should be in array, like this:
=query(
{
get_all_sheets_names_from('settings!A1:A100'); // something like this
};
"SELECT Col2, Col3, Col4, ...[etc]")
Is it possible?
My attempts:
https://docs.google.com/spreadsheets/d/1ZMzu6FuVyAJiWfNIHW87OW1Vpg_mM_QdtGs9nq9UXCU/edit#gid=0
I would like the array with data sources to be taken from the G2:G column.
The example in column C shows how this can be done manually. However, I am looking for a solution so that in the query nothing has to be done so that the query can drag an array with the names of the data source from G2:G

1) What i Think
I think it is not possible to use "INDIRECT" in the query parameters, because "INDIRECT" returns a cell reference and the parameters {(); ()} in a query are fixed objects.
An "INDIRECT" on a complete query is not possible either, for the same reason: a query does not return a reference on a cell.
2) Limited soluce
the principle: case1: look in column G the 3rd line (3rd source), if empty then test case 2, otherwise apply the formula with 3 sources.
case 2: if 2nd source is empty then go to case 1, otherwise apply the formula with 2 sources
case 1: if empty then display "no sources" otherwise apply formula with 1 source
Formula
note 1 replace ESTVIDE (fr) by ISBLANK (eng) !!
note 2 : you can test with (G2="source1" and G3="source2),
but it works with G2="source3" and G3="source1"
=SI(ESTVIDE($G$4); SI(ESTVIDE($G$3); SI(ESTVIDE($G$1); "no source(s)";query({((INDIRECT("'"&G2&"'!A1:A5")))};"SELECT Col1")) ;query({(INDIRECT("'"&G2&"'!A1:A5"));(INDIRECT("'"&G3&"'!A1:A5"))};"SELECT Col1")) ;query({(INDIRECT("'"&G2&"'!A1:A5"));(INDIRECT("'"&G3&"'!A1:A5"));(INDIRECT("'"&G4&"'!A1:A5"))};"SELECT Col1"))
Online sheet
https://docs.google.com/spreadsheets/d/1sCwwFjpYKKzzAvVwmbMUWcmHSc1wY52XnHlFdT00A3U/edit?usp=sharing
Limitations
Off course, this is a formula with only 3 sources max !
It will be verry big and uggly with more sources...
Script
macro is the only solution ?
soluce with Macro
append this script,
it gets value sources values from G2:G30 (you need more...put G100..)
it create the formula and put it on H2
it read max 50 value in each source (see A1:A50 in source code)
it's not so hard to understand,
note : managing macro with GSheet is a another problem, if you needs advices, please post a comment.
link to live sheet :
https://docs.google.com/spreadsheets/d/14XaR-UsADUpCUCVWqeg0zCbfGy3CCvnwVxUhozjYocc/edit?usp=sharing
function formula6() {
var spreadsheet = SpreadsheetApp.getActive();
var values=spreadsheet.getRange('G2:G30').getValues();
var acSources="{";
for (var i = 0; (i < values.length) && (values[i]!=""); i++) {
if (i>0) { acSources+=";" }
acSources=acSources+'INDIRECT("'+values[i]+'!A1:A50")';
}
acSources=acSources+"}";
var formula='query('+acSources+';"SELECT Col1")';
spreadsheet.getRange('H2').activate();
spreadsheet.getCurrentCell().setFormula('='+formula);
};

dudes who copy-pasted INDIRECT function into Google Sheets completely failed to understand the potential of it and therefore they made zero effort to improve upon it and cover the obvious logic which is crucial in this age of arrays.
in other words, INDIRECT can't intake more than one array:
=INDIRECT("Sheet1!A:B"; "Sheet2!A:B")
nor convert an arrayed string into active reference, which means that any attempt of concatenation is also futile:
=INDIRECT(MasterSheet!A1:A10)
————————————————————————————————————————————————————————————————————————————————————
=INDIRECT("{Sheet1!A:B; Sheet2!A:B}")
————————————————————————————————————————————————————————————————————————————————————
={INDIRECT("Sheet1!A:B"; "Sheet2!A:B")}
————————————————————————————————————————————————————————————————————————————————————
=INDIRECT("{INDIRECT("Sheet1!A:B"); INDIRECT("Sheet2!A:B")}")
the only possible way is to use INDIRECT for each end every range like:
={INDIRECT("Sheet1!A:B"); INDIRECT("Sheet2!A:B")}
which means that the best you can do is to pre-program your array like this if only part of the sheets/tabs is existant (let's have a scenario where only 2 sheets are created from a total of 4):
=QUERY(
{IFERROR(INDIRECT("Sheet1!A1:B5"), {"",""});
IFERROR(INDIRECT("Sheet2!A1:B5"), {"",""});
IFERROR(INDIRECT("Sheet3!A1:B5"), {"",""});
IFERROR(INDIRECT("Sheet4!A1:B5"), {"",""})},
"where Col1 is not null", 0)
so, even if sheet names are predictable (which not always are) to pre-program 100+ sheets like this would be painful (even if there are various sneaky ways how to write such formula under 30 seconds)
an alternative would be to use a script to convert string and inject it as the formula
A1 would be formula that creates a string that looks like real formula:
=ARRAYFORMULA("=QUERY({"&TEXTJOIN("; "; 1;
FILTER(SNAME(1); SNAME(1)<>SNAME(0))&"!A1:A20")&"}; ""where Col1 is not null""; 0)")
then this script will take the string from A1 cell and it will paste it as valid formula into A2 cell:
function onEdit() {
var sheet = SpreadsheetApp.getActive().getSheetByName('query'); // MASTER SHEET NAME
var src = sheet.getRange("A1"); // COPY STRING FROM
var str = src.getValue();
var cell = sheet.getRange("A2"); // PASTE AS FORMULA INTO
cell.setFormula(str);
}
function SNAME(option) {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheet = ss.getActiveSheet()
var thisSheet = sheet.getName();
if(option === 0){ // ACTIVE SHEET NAME =SNAME(0)
return thisSheet;
}else if(option === 1){ // ALL SHEET NAMES =SNAME(1)
var sheetList = [];
ss.getSheets().forEach(function(val){
sheetList.push(val.getName())
});
return sheetList;
}else if(option === 2){ // SPREADSHEET NAME =SNAME(2)
return ss.getName();
}else{
return "#N/A"; // ERROR MESSAGE
};
}
of course, the script can be changed to onOpen trigger or with custom name triggered from the custom menu or via button (however it's not possible to use the custom function as formula directly)
this will cover all your needs to not edit the formula by adding references if new sheets are added. the only drawback is a recalculation of sheet name script... to do so you need to dismantle A1 formula for example by adding ' before the leading = pressing enter and then removing it to fix the formula
spreadsheet demo

Related

Google Sheet Query - Building Reference Array Dynamically

I have multiple tabs in a file and want to merge the same range to a master tab.
I did a Query
=QUERY({source1!AR5:AU;source1!AR5:AU}, "select Col1,Col2,Col3,Col4 where Col1 is not null order by Col1", 0)
But at the end I will have more and more tabs (around 30), and I don't want to change my query manually each time. how can i do?
I saw somewhere that I can use macros to create a function, do you have an idea of the code?
I just want to have something easy where I have added in another tab all my tab names
Try:
function myFunction() {
var ss = SpreadsheetApp.getActiveSpreadsheet();
var sheets = ss.getSheets();
var allSheets = [];
// Append more names if you want to exclude more
// Since 'query' is master sheet, excluded it as well
// Remove 'query' below if you want to include it in the formula
var excludedTabs = ['other', 'random', 'query'];
sheets.forEach(function (sheet){
var sheetName = sheet.getSheetName();
if (!excludedTabs.includes(sheetName)){
allSheets.push(sheetName);
}
});
// Set formula in A1 cell in "query" sheet
// Modify A1 to change the cell
var cell = ss.getSheetByName('query').getRange("A1");
cell.setFormula("=QUERY({" + allSheets.join('!AR5:AU;') + "!AR5:AU}, \"select Col1,Col2,Col3,Col4 where Col1 is not null order by Col1\", 0)");
}
This will set the formula to A1 of the sheet query, feel free to change where to set the formula by changing A1 and query.
You can also add sheets to be excluded. Append it on the excludedTabs array.
Sample Output:
Sample sheets were added to check the new sheet cases.
Expected formula was added. (excluding query, random and other sheets)

Loop function to fill Crosselling template

I need to create a file containing information on crosselling for my webshop.
In my example file you'll see the tab "Basic Data", this is the data available to me already. Column A contains certain products, column B shows the assigned (product)categories.
The Tab "Starting Point" shows how the final file will be structured and how the function should come into play. It would need to do the following:
Copy the first product from the Unique product list (D2) to A2 (already done here)
Paste a filterfunction into B2 (already done here)
This filterfunction lists all products that belong to the same category like Product 1 except for Product 1 itself
Apply a numerical position tag in tens in Column C to the whole range of products related to Product 1 (in this case B2:B4), starting from 10 (..20, 30, ff) and optimally randomize it. (already done here)
Drag down A2, respectively paste "Product 1" into all cells below A2 until the end of the result of the filterfunction in Columns B is reached (already done here).
Continue the loop by pasting "Product 2" into A5, pasting the filterfunction into B5 and so on.
In "Desired Result" you can see how the end result should look like in this example. There are only 8 products, but I'd need to be able to do this for hundreds of products, that's why a function is needed.
I hope somebody is able to help me here.
Answer
You can get your desired result using Google Apps Script as I suggested.
How to use it:
Open Apps Script clicking on Tools > Script editor and you will see the script editor. It is based on JavaScript and it allows you to create, access, and modify Google Sheets files with a service called Spreadsheet Service.
Paste the following code and click on Run
Code
function main() {
const ss = SpreadsheetApp.getActiveSpreadsheet()
const sheetResult = ss.getSheetByName('Desired Result')
const sheetBasic = ss.getSheetByName('Basic Data')
// write unique products
const val = "=ARRAYFORMULA(UNIQUE('Basic Data'!A2:A))"
sheetResult.getRange('D2').setValue(val)
// get basic data
const lastColumn = sheetBasic.getRange('A:A').getValues().filter(String).length; // number of products
const cat = sheetBasic.getRange('A2:B'+lastColumn).getValues() // product | category
const productGroup = [] // array to store data
// loop each product
for (var j=0; j<cat.length; j++){
const p1 = cat[j]
var k = 1
for (var i=0; i<cat.length; i++){
if (cat[i][1] == p1[1] && cat[i][0] != p1[0]){
const val = [p1[0],cat[i][0],k*10]
k = k + 1
productGroup.push(val)
}
}
}
var n = productGroup.length+1
sheetResult.getRange('A2:C'+n).setValues(productGroup)
}
Some comments
This solution does not randomize the position value. If you need it I can help you with that, but first I want to check that this solution fits you.
Let me know if you can obtain your desired result. Keep in mind that this solution uses the name of the sheets.
Reference
Google Apps Script
Spreadsheet Service
JavaScript

Apps Script (Google Sheets) How to perform .setHiddenValues with an Array

I have some values stored in an Array and now I want to remove ALL the values stored in this array from a filter.
The values are stored correctly in the array but I don't manage to remove the values from the filter.
The Array's name is HideValues
Here is some code:
var p = 0;
spreadsheet.getSheetByName('TEM Tool Data').getRange('\'TEM Tool Data\'!E1').activate();
var criteria = SpreadsheetApp.newFilterCriteria();
//Remove all PID´s from the filter
while (p < HideValues.length){
criteria.setHiddenValues([HideValues[p]]).build();
p++;}
//Filter
spreadsheet.getSheetByName('TEM Tool Data').getFilter().setColumnFilterCriteria(5, criteria);
//Copy filtered area
spreadsheet.getRange('A2:I1386').activate();
//Paste
spreadsheet.getSheetByName('Visualization').getRange('A5').activate();
spreadsheet.getRange('\'TEM Tool Data\'!A2:I1386').copyTo(SpreadsheetApp.getActiveRange(),
SpreadsheetApp.CopyPasteType.PASTE_NORMAL, false);
spreadsheet.getSheetByName('Visualization').getRange('J3').activate();
};
You don't need to loop through the array
If your array is something like var HideValues = [1,2,3,4,5];,
you can simply specify criteria.setHiddenValues(HideValues).build(); - without the while loop
Also:
To a filter, you should create it first (if not already done) and apply it to a range, not sheet:
var filter = spreadsheet.getSheetByName('TEM Tool Data').getDataRange().createFilter();
filter.setColumnFilterCriteria(3, criteria);
You should apply it to a range, not a sheet

Comparing multiple values in Google Sheets with App Script for loops

I would like to compare multiple values in a Google Sheet spreadsheet some using for loops in Google App Script. But i would like some advice on the best way to do it.
To explain below...
I have two spreadsheets, A "FOOD" table, and A "FOOD GROUP" table. 
I've written a for loop script that goes through the entire FOOD table.
If the key value of both tables matches, the script will update a column from the FOOD table with a column from the FOOD GROUP table.
The script works without issues. But it can only compare one column between the 2 tables at a time. I would like to modify this script so I can compare multiple columns at once, without having to create a for loop for each specified column.
I pasted my code below. I can also provide images of my spreadsheet if you need it. 
In any case, I'm new to coding, so any constructive feedback or insight to improve my script will be helpful. I'm happy to answer any questions if anything seems unclear.
function FoodGroup_Test() {
var Data = SpreadsheetApp.getActiveSpreadsheet();
var FoodGroupDataSheet = Data.getSheetByName("Food Groups") // "FoodGroup" sheet
var FoodGroupAllValues = FoodGroupDataSheet.getRange(2, 1, FoodGroupDataSheet.getLastRow()-1,FoodGroupDataSheet.getLastColumn()).getValues();
var FoodGroupDataLastRow = FoodGroupDataSheet.getLastRow();
var FoodDataSheet = Data.getSheetByName("Food") // "Food" sheet
var FoodAllValues = FoodDataSheet.getRange(2, 1, FoodDataSheet.getLastRow()-1,FoodDataSheet.getLastColumn()).getValues();
// Object to contain all FoodGroup column values
var Object = {};
for(var FO = FoodGroupAllValues.length-1;FO>=0;FO--) // for each row in the "FoodGroup" sheet...
{
Object[FoodGroupAllValues[FO][15]] = FoodGroupAllValues[FO][11]; // ...store FoodGroup ID Key value
}
for(var F = FoodAllValues.length-1;F>=0;F--) // for each row in the "Food" sheet...
{
var Food_FoodGroupKey = FoodAllValues[F][94]; // Store FoodGroup Key value.
// ...if the Food value dont match, update it with FoodGroup's value
if (Object[Food_FoodGroupKey] != FoodAllValues[F][95])
{
FoodAllValues[F][95] = Object[Food_FoodGroupKey];
}
}
// declare range to place updated values, then set it.
var FoodDestinationRange = FoodDataSheet.getRange(2, 1, FoodAllValues.length, FoodAllValues[0].length);
FoodDestinationRange.setValues(FoodAllValues);
}
FOOD GROUP Table
FOOD table
In order for your code to work as expected, you should do the following changes to your code:
Update your if condition to this one:
Object[Food_FoodGroupKey] != FoodAllValues[F][95] && object2[] != FAllValues[F][86])
In order to avoid the undefined problem use the following line of code:
FoodDataSheet.createTextFinder("undefined").replaceAllWith("");

Using setvalues with a for loop

I am currently trying to perform a "copyTo" or "setValues" action.
I need to copy one grid (Sheet1!J16:Q36) to another grid (Sheet2!J16:Q36)
But not every cell shall be copied. Only those values that are not identical to the Sheet1 values shall be copied.
I have tried the below code with success, but sadly the script takes ages.
I understand that a batch operation with getValues in an array will be quicker, but I lack the capability to do that script.
I also used a third grid which compared the values of sheet1 and 2 and returned 1 or 0. Only if the value 1 was shown, the cell was considered by the for loop. I take it that this is inefficient.
Thank you for your help. I appreciate it a lot.
var ratenprogramm = SpreadsheetApp.getActiveSpreadsheet();
var ratenprogrammmain = ratenprogramm.getSheetByName("Ratenprogramm");
var vorlageratenprogramm =
ratenprogramm.getSheetByName("VorlageRatenprogramm");
for(i=1;i<=21;i++)
{
for(j=1;j<=8;j++)
{
if(vorlageratenprogramm.getRange(37+i,9+j).getValue() == 1)
{
vorlageratenprogramm.getRange(15+i,9+j).copyTo(ratenprogrammmain.getRange(15+i,9+j),{contentsOnly: true});
}
}
}
As you have noticed, calling any external services, including methods
like getValue() make your script slow, see Apps Script Best
Practices.
Your code can be optimized by replacing the multiple getValue()
requests by a single getValues().
Within the nested loops you can specify a multiple amount of ranges
and values that can be written with the Advanced Sheets Service,
with the Sheets API method spreadsheets.values.batchUpdate into
the corresponding ranges of the destination sheet, see also
here.
Sample
function myFunction() {
var ratenprogramm = SpreadsheetApp.getActiveSpreadsheet();
var ratenprogrammmain = ratenprogramm.getSheetByName("Ratenprogramm");
var vorlageratenprogramm = ratenprogramm.getSheetByName("VorlageRatenprogramm");
var data=[];
var range=vorlageratenprogramm.getRange(15,9,21,8);
var values=range.getValues();
for(i=0;i<4;i++)
{
for(j=0;j<1;j++)
{
if(values[i+1][j] == 1)
{
var cell=range.getCell(i+1,j+1).getA1Notation();
data.push([{ range:'Ratenprogramm!'+ cell, values: [[values[i+1][j]]]}]);
}
}
}
var resource = {
valueInputOption: "USER_ENTERED",
data: data
};
Sheets.Spreadsheets.Values.batchUpdate(resource, spreadsheetId);
}
Keep in mind that if you have many different ranges, it might be
easier and faster to overwrite the sheet with the complete range,
rather than using nesting looping. E.g.
vorlageratenprogramm.getRange(15,9,21,8).copyTo(ratenprogrammmain.getRange(15,9,21,8),{contentsOnly:
true});.

Resources