I have a table in BigQuery that contains, among other fields, an ID column and an array column. Since bringing the table into PowerBi, the array is just some delimited strings that aren't terribly useful, so clearly, I'll need a view to base my PowerBi off of. So, what is the right way to handle this so that the array data is useful in PowerBi?
My solution is to create 2 views, the first being a view for all the columns except the array column, and the second being a view the contains the ID, and the unnested array column, thus creating a one-to-many that can be joined to the first view if the user wants the array data, knowing that doing such a join will likely give them duplicates of the non-array data in the result.
Is there a better way?
I have multiple tables with one common element (ID). The first table has is the only one that has the ID and First Name. In a dashboard, I am trying to create a filter where when I type in or select the First Name, all data from all tables relating to the First Name's ID would populate.
FYI, I don't want to join the data. My actual tables are big data from multiple sources I would like to find an alternative route to this.
I also tried to create a parameter for the First Name and then created a Calculated Field with
IF [Parameter]=[Name]
THEN [ID]
ELSE NULL
END
but this didn't work. I am very new to Tableau and I'm thinking that there must be a way where the choosing a Name would trigger the ID and its relevant data.
create multiple sheets with different tables where each sheet will be collected to individual table.
Now create a filter and apply the filter to all worksheets and then go to dashboard and apply filter, Dash board should change.
I know it's old technology (ancient now), but it's what I have to work with due to work.
I am able to create a Group and report and link it to a textbox to provide a collapsible report, with master data and detail data.
What I want to know, is it possible in SSRS2005 to create another sub-group to the first group?
i.e. Master record -> Detail -> Sub-details
Every time I try and add another detail row for example I only get one row of data in the sub-group, because it's tied to the Details Grouping. I cannot explicitly say "report grouped by this other subgroup" (where it offers you to create groups in the Group list).
Yes I am trying to do this in a table.
This is what I am after...
[+] Col1 Col2 Col3 Col4
[+] data data data data
Col1 Col2 Col3 ...
data data data ...
...
...
The [+] is what I want to set up to allow the expansion of another group within the first group.
The answer is not to try and pack too much into one reporting object. In this case the Table object.
I managed to have far more flexibility by placing the tables and fields inside a List object.
Try these...
Drop a List item into the report
Define the Dataset to your main dataset that contains all the data.
Note: For this to work you need a query that have as much as the master and detail data inside as ONE query, so obviously your master data will repeat as deep as it has to, to get to the lowest common dataset, which in my case was the action items per student.
The List object will act as the Master data reference for all your other objects inside of it.
Now you can play! Drop in a few textboxes to show the data for the master data you want to only show ONCE per "logical record". I'll let you ponder what that meant!
Now to show subgroups, you'll need Tables for each one. Drop a Table object
In each table (group) don't specify a dataset, as I said above, all the data comes from the List.
But for each table, you'll need to define your Details Grouping. Go ahead and simply state the group parameters you require for the sub-group. So in my case, I want to display ONE row of Students, but any amount of Actions each student has. So place a StudentID for the student and an ActionID for the actions as a combined grouping.
Repeat the above for any other groups, and define their groupings so you don't get repeatitive data. One table could only have one pivot or group, so just define the index for that inside the Details Grouping. Do not use the Add.. feature to add groups, because you'll be adding groups within subgroups and go into another level again! Beware.
I hope this made sense.
While surfing through 9gag.com, an idea (problem) came up to my mind. Let's say that I want to create a website where users can add diffirent kinds of entries. Now each entry is diffirent type and needs diffirent / additional columns.
Let's say that we can add:
a youtube video
a cite which requires the cite's author name and last name
a flash game which requires additional game category, description, genre etc.
an image which requires the link
Now all the above are all entries and have some columns in common (like id, add_date, adding_user_id, etc...) and some diffirent / additional (for example: only flash game needs description or only image needs plus_18 column to be specified). The question is how should I organize DB / code for controlling all of the above as entries together? I might want to order them, or search entries by add_date etc...
The ideas that came up to my mind:
Add a "type" column which specifies what entry it is and add all the possible columns where NULL is allowed for not related to this particular type columns. But this is mega nasty. There is no data integration.
Add some column with serialized data for the additional data but it makes any filtration a total hell.
Create a master (parent) table for an entry and separate tables for concrete entry types (their additional columns / info). But here I don't even know how I'm supposed to select data properly and is just nasty as well.
So what's the best way to solve this problem?
The parent table seems like the best option.
// This is the parent table
Entry
ID PK
Common fields
Video
ID PK
EntryID FK
Unique fields
Game
ID PK
EntryID FK
Unique fields
...
What the queries will look like will largely depend on the type of query. To, for example, get all games ordered by a certain date, the query will look something like:
SELECT *
FROM Game
JOIN Entry ON Game.EntryID = Entry.ID
ORDER BY Entry.AddDate
To get all content ordered by date, will be somewhat messy. For example:
SELECT *
FROM Entry
LEFT JOIN Game ON Game.EntryID = Entry.ID
LEFT JOIN Video ON Video.EntryID = Entry.ID
...
ORDER BY Entry.AddDate
If you want to run queries like the one above, I suggest you give unique names to your primary key fields (i.e. VideoID and GameID) so you can easily identify which type of entry you're dealing with (by checking GameID IS NOT NULL for example).
Or you could add a Type field in Entry.
I have an application with multiple "pick list" entities, such as used to populate choices of dropdown selection boxes. These entities need to be stored in the database. How do one persist these entities in the database?
Should I create a new table for each pick list? Is there a better solution?
In the past I've created a table that has the Name of the list and the acceptable values, then queried it to display the list. I also include a underlying value, so you can return a display value for the list, and a bound value that may be much uglier (a small int for normalized data, for instance)
CREATE TABLE PickList(
ListName varchar(15),
Value varchar(15),
Display varchar(15),
Primary Key (ListName, Display)
)
You could also add a sortOrder field if you want to manually define the order to display them in.
It depends on various things:
if they are immutable and non relational (think "names of US States") an argument could be made that they should not be in the database at all: after all they are simply formatting of something simpler (like the two character code assigned). This has the added advantage that you don't need a round trip to the db to fetch something that never changes in order to populate the combo box.
You can then use an Enum in code and a constraint in the DB. In case of localized display, so you need a different formatting for each culture, then you can use XML files or other resources to store the literals.
if they are relational (think "states - capitals") I am not very convinced either way... but lately I've been using XML files, database constraints and javascript to populate. It works quite well and it's easy on the DB.
if they are not read-only but rarely change (i.e. typically cannot be changed by the end user but only by some editor or daily batch), then I would still consider the opportunity of not storing them in the DB... it would depend on the particular case.
in other cases, storing in the DB is the way (think of the tags of StackOverflow... they are "lookup" but can also be changed by the end user) -- possibly with some caching if needed. It requires some careful locking, but it would work well enough.
Well, you could do something like this:
PickListContent
IdList IdPick Text
1 1 Apples
1 2 Oranges
1 3 Pears
2 1 Dogs
2 2 Cats
and optionally..
PickList
Id Description
1 Fruit
2 Pets
I've found that creating individual tables is the best idea.
I've been down the road of trying to create one master table of all pick lists and then filtering out based on type. While it works, it has invariably created headaches down the line. For example you may find that something you presumed to be a simple pick list is not so simple and requires an extra field, do you now split this data into an additional table or extend you master list?
From a database perspective, having individual tables makes it much easier to manage your relational integrity and it makes it easier to interpret the data in the database when you're not using the application
We have followed the pattern of a new table for each pick list. For example:
Table FRUIT has columns ID, NAME, and DESCRIPTION.
Values might include:
15000, Apple, Red fruit
15001, Banana, yellow and yummy
...
If you have a need to reference FRUIT in another table, you would call the column FRUIT_ID and reference the ID value of the row in the FRUIT table.
Create one table for lists and one table for list_options.
# Put in the name of the list
insert into lists (id, name) values (1, "Country in North America");
# Put in the values of the list
insert into list_options (id, list_id, value_text) values
(1, 1, "Canada"),
(2, 1, "United States of America"),
(3, 1, "Mexico");
To answer the second question first: yes, I would create a separate table for each pick list in most cases. Especially if they are for completely different types of values (e.g. states and cities). The general table format I use is as follows:
id - identity or UUID field (I actually call the field xxx_id where xxx is the name of the table).
name - display name of the item
display_order - small int of order to display. Default this value to something greater than 1
If you want you could add a separate 'value' field but I just usually use the id field as the select box value.
I generally use a select that orders first by display order, then by name, so you can order something alphabetically while still adding your own exceptions. For example, let's say you have a list of countries that you want in alpha order but have the US first and Canada second you could say "SELECT id, name FROM theTable ORDER BY display_order, name" and set the display_order value for the US as 1, Canada as 2 and all other countries as 9.
You can get fancier, such as having an 'active' flag so you can activate or deactivate options, or setting a 'x_type' field so you can group options, description column for use in tooltips, etc. But the basic table works well for most circumstances.
Two tables. If you try to cram everything into one table then you break normalization (if you care about that). Here are examples:
LIST
---------------
LIST_ID (PK)
NAME
DESCR
LIST_OPTION
----------------------------
LIST_OPTION_ID (PK)
LIST_ID (FK)
OPTION_NAME
OPTION_VALUE
MANUAL_SORT
The list table simply describes a pick list. The list_ option table describes each option in a given list. So your queries will always start with knowing which pick list you'd like to populate (either by name or ID) which you join to the list_ option table to pull all the options. The manual_sort column is there just in case you want to enforce a particular order other than by name or value. (BTW, whenever I try to post the words "list" and "option" connected with an underscore, the preview window goes a little wacky. That's why I put a space there.)
The query would look something like:
select
b.option_name,
b.option_value
from
list a,
list_option b
where
a.name="States"
and
a.list_id = b.list_id
order by
b.manual_sort asc
You'll also want to create an index on list.name if you think you'll ever use it in a where clause. The pk and fk columns will typically automatically be indexed.
And please don't create a new table for each pick list unless you're putting in "relationally relevant" data that will be used elsewhere by the app. You'd be circumventing exactly the relational functionality that a database provides. You'd be better off statically defining pick lists as constants somewhere in a base class or a properties file (your choice on how to model the name-value pair).
Depending on your needs, you can just have an options table that has a list identifier and a list value as the primary key.
select optionDesc from Options where 'MyList' = optionList
You can then extend it with an order column, etc. If you have an ID field, that is how you can reference your answers back... of if it is often changing, you can just copy the answer value to the answer table.
If you don't mind using strings for the actual values, you can simply give each list a different list_id in value and populate a single table with :
item_id: int
list_id: int
text: varchar(50)
Seems easiest unless you need multiple things per list item
We actually created entities to handle simple pick lists. We created a Lookup table, that holds all the available pick lists, and a LookupValue table that contains all the name/value records for the Lookup.
Works great for us when we need it to be simple.
I've done this in two different ways:
1) unique tables per list
2) a master table for the list, with views to give specific ones
I tend to prefer the initial option as it makes updating lists easier (at least in my opinion).
Try turning the question around. Why do you need to pull it from the database? Isn't the data part of your model but you really want to persist it in the database? You could use an OR mapper like linq2sql or nhibernate (assuming you're in the .net world) or depending on the data you could store it manually in a table each - there are situations where it would make good sense to put it all in the same table but do consider this only if you feel it makes really good sense. Normally putting different data in different tables makes it a lot easier to (later) understand what is going on.
There are several approaches here.
1) Create one table per pick list. Each of the tables would have the ID and Name columns; the value that was picked by the user would be stored based on the ID of the item that was selected.
2) Create a single table with all pick lists. Columns: ID; list ID (or list type); Name. When you need to populate a list, do a query "select all items where list ID = ...". Advantage of this approach: really easy to add pick lists; disadvantage: a little more difficult to write group-by style queries (for example, give me the number of records that picked value X".
I personally prefer option 1, it seems "cleaner" to me.
You can use either a separate table for each (my preferred), or a common picklist table that has a type column you can use to filter on from your application. I'm not sure that one has a great benefit over the other generally speaking.
If you have more than 25 or so, organizationally it might be easier to use the single table solution so you don't have several picklist tables cluttering up your database.
Performance might be a hair better using separate tables for each if your lists are very long, but this is probably negligible provided your indexes and such are set up properly.
I like using separate tables so that if something changes in a picklist - it needs and additional attribute for instance - you can change just that picklist table with little effect on the rest of your schema. In the single table solution, you will either have to denormalize your picklist data, pull that picklist out into a separate table, etc. Constraints are also easier to enforce in the separate table solution.
This has served us well:
SQL> desc aux_values;
Name Type
----------------------------------------- ------------
VARIABLE_ID VARCHAR2(20)
VALUE_SEQ NUMBER
DESCRIPTION VARCHAR2(80)
INTEGER_VALUE NUMBER
CHAR_VALUE VARCHAR2(40)
FLOAT_VALUE FLOAT(126)
ACTIVE_FLAG VARCHAR2(1)
The "Variable ID" indicates the kind of data, like "Customer Status" or "Defect Code" or whatever you need. Then you have several entries, each one with the appropriate data type column filled in. So for a status, you'd have several entries with the "CHAR_VALUE" filled in.