I am trying to send simulated data from kepware to azure Hub and then to Stream analytic. Next step would be directly to Power Bi but it seems like stream analytics is sending an array to Power Bi.
here is my data from kepware:
SELECT
event.[values].id as id,
event.[values].v as Measurements,
event.[values].t as Time,
event.[values].q as Quality
INTO PowerBiVisualizationOutput
FROM PowerBiVisualizationInput AS event
CROSS APPLY GetArrayElements(event.[values]) AS sensordata
I have tried so many different way of writing quarries but they all are giving me null in output. Can some one please help me
Regards
The syntax for using GetArrayElements is illustrated here.
Note how it returns a record/object for each item/row of your array, that in your case you named sensordata (it's not event. anymore). That record has 2 properties : ArrayIndex and ArrayValue, that are hardcoded (part of GetArrayElements). Here you want to grab your properties out of ArrayValue:
SELECT
sensordata.ArrayValue.id as id,
sensordata.ArrayValue.v as Measurements,
sensordata.ArrayValue.t as Time,
sensordata.ArrayValue.q as Quality
INTO PowerBiVisualizationOutput
FROM PowerBiVisualizationInput AS event
CROSS APPLY GetArrayElements(event.[values]) AS sensordata
Edit following our conversation in comments:
Looking at the first screenshot, the field name we're trying to unfold is [value] and not value, the square brackets [] are in there.
To reference such fields in a query, we can use double quotes: "[value]":
SELECT
sensordata.ArrayValue.id as id,
sensordata.ArrayValue.v as Measurements,
sensordata.ArrayValue.t as Time,
sensordata.ArrayValue.q as Quality
INTO PowerBiVisualizationOutput
FROM PowerBiVisualizationInput AS event
CROSS APPLY GetArrayElements("[values]") AS sensordata
Related
I have a table in Snowflake where one of the fields, called 'value' is sometimes plain text sometimes JSON, this field is stored as string in Snowflake
I created this view to get only the rows where there is a Json format
CREATE OR REPLACE VIEW tmp_events AS
SELECT PARSE_JSON(value) as json_data,
id
FROM SessionEvent
WHERE session_event_type_id=7;
Then I flatten the rows to create a new field
CREATE OR REPLACE VIEW tmp_events_step2 AS
SELECT id,
json_data:meta:selected::string AS choice
from tmp_events ,
LATERAL FLATTEN(input => tmp_events.json_data)
WHERE choice IS NOT NULL
Everything runs fine until now, I can preview data from these two views, no error and I get the results I was expecting.
The error comes when I try to get distinct values from choice
SELECT DISTINCT choice from tmp_events_step2;
Error parsing JSON: unknown keyword "brain", pos 6
This name Brain seems to come from my initial table without the WHERE statement.
If I run the query without DISTINCT there is no error.
Weird thing I noticed while trying to debug: when I put a limit in tmp_events_step2, the code works fine again, even though I put a limit that's bigger than the number of rows in the table
CREATE OR REPLACE VIEW tmp_events_step2 AS
SELECT id,
json_data:meta:selected::string AS choice
from tmp_events ,
LATERAL FLATTEN(input => tmp_events.json_data)
WHERE choice IS NOT NULL
LIMIT 10000000;
SELECT DISTINCT choice from tmp_events_step2;
What's the catch? Why does it work only with the limit?
The very simple answer to this is the built-in function TRY_PARSE_JSON()
Er, not. You seem to have problems with the Query optimizer that may do incorrect predicate pushdowns. One way to prevent the optimizer from doing this is to use the secure view option:
CREATE SECURE VIEW tmp_events_step2 ...
and file a support ticket...
We reported this error two years ago and they said they where not going to fix, because by hoisting the JSON access prior to running the filters in the WHERE clause that makes the cast valid/safe, impacted performance.
create table variant_cast_bug(num number, var variant);
insert into variant_cast_bug
select column1 as num, parse_json(column2) as var
from values (1, '{"id": 1}'),
(1, '{"id": 2}'),
(2, '{"id": "text"}')
v;
select * from variant_cast_bug;
select var:id from variant_cast_bug;
select var:id from variant_cast_bug where num = 1;
select var:id::number from variant_cast_bug where num = 1; -- <- broken
select TRY_TO_NUMBER(var:id) from variant_cast_bug where num = 1; -- <- works
Sometimes you can nest the select and it will work, and then you can add another SELECT layer around it, and do some aggregation and the cost explodes again.
The only two safe solutions are SERCURE VIEW as Hans mentions, but that is a performance nightmare.
Or to understand this problem and use TRY_TO_NUMBER or it's friends.
At the time this was made bad worse because JSON boolean values where not valid values to pass to TRY_TO_BOOLEAN..
One of the times we got burnt by this was after a snowflake release when code that had been running for a year, started getting this error, because it was complex enough the hoisting did not impact, and then after release it did. This is where Snowflake are rather responsive, and then rolled the release back, and we put TRY_TO on a chunk of already working SQL just to play it safe.
Please submit a support case for this issue.
My department uses a software tool that can use a custom component library sourced from Tables or Queries in an MS Access database.
Table: Components
ID: AutoNumber
Type: String
Mfg: String
P/N: String
...
Query: Resistors
SELECT Components.*
FROM Components
WHERE Components.Type = "Resistors"
Query: Capacitors
SELECT Components.*
FROM Components
WHERE Components.Type = "Capacitors"
These queries work fine for SELECT. But when users add a row to the query, how can I ensure the correct value is saved to the Type field?
Edit #2:
Nope, can't be done. Sorry.
Edit #1:
As was pointed out, I may have misunderstood the question. It's not a wonky question after all, but perhaps an easy one?
If you're asking how to add records to your table while making sure that, for example, "the record shows up in a Resistors query if it's a Resistor", then it's a regular append query, that specifies Resisitors as your Type.
For example:
INSERT INTO Components ( ID, Type, Mfg )
SELECT 123, 'Resistors', 'Company XYZ'
If you've already tried that and are having problems, it could be because you are using a Reserved Word as a field name which, although it may work sometimes, can cause problems in unexpected ways.
Type is a word that Access, SQL and VBA all use for a specific purpose. It's the same idea as if you used SELECT and FROM as field or table names. (SELECT SELECT FROM FROM).
Here is a list of reserved words that should generally be avoided. (I realize it's labelled Access 2007 but the list is very similar, and it's surprisingly difficult to find an recent 'official' list for Excel VBA.)
Original Answer:
That's kind a a wonky way to do things. The point of databases is to organize in such a way as to prevent duplication of not only data, but queries and codes as well
I made up the programming rule for my own use "If you're doing anything more than once, you're doing it wrong." (That's not true in all cases but a general rule of thumb nonetheless.)
Are the only options "Resistors" and "Capacitors"? (...I hope you're not tracking the inventory of an electronics supply store...) If there are may options, that's even more reason to find an alternative method.
To answer your question, in the Query Design window, it is not possible to return the name of the open query.
Some alternative options:
As #Erik suggested, constrain to a control on a form. Perhaps have a drop-down or option buttons which the user can select the relevant type. Then your query would look like:
SELECT * FROM Components WHERE Type = 'Forms![YourFormName]![NameOfYourControl]'
In VBA, have the query refer to the value of a variable, foe example:
Dim TypeToDel as String
TypeToDel = "Resistor"
DoCmd.RunSQL "SELECT * FROM Components WHERE Type = '" & typeToDel'"
Not recommended, but you could have the user manually enter the criteria. If your query is like this:
SELECT * FROM Components WHERE Type = '[Enter the component type]'
...then each time the query is run, it will prompt:
Similarly, you could have the query prompt for an option, perhaps a single-digit or a code, and have the query choose the the appropriate criteria:
...and have an IF statement in the query criteria.
SELECT *
FROM Components
WHERE Type = IIf([Enter 1 for Resistors, 2 for Capacitors, 3 for sharks with frickin' laser beams attached to their heads]=1,'Resistors',IIf([Enter 1 for Resistors, 2 for Capacitors, 3 for sharks with frickin' laser beams attached to their heads]=2,'Capacitors','LaserSharks'));
Note that if you're going to have more than 2 options, you'll need to have the parameter box more than once, and they must be spelled identically.
Lastly, if you're still going to take the route of a separate query for each component type, as long as you're making separate queries anyway, why not just put a static value in each one (just like your example):
SELECT * FROM Components WHERE Type = 'Resistor'
There's another wonky answer here but that's just creating even more duplicate information (and more future mistakes).
Side note: Type is a reserved word in Access & VBA; you might be best to choose another. (I usually prefix with a related letter like cType.)
More Information:
Use parameters in queries, forms, and reports
Use parameters to ask for input when running a query
Microsoft Access Tips & Tricks: Parameter Queries
• Frickin' Lasers
I have two view objects in Oracle ADF.
LineVO represents order lines -- with one line per product.
Products are differentiated by several attributes... say "model" and "color". So, VO #1 contains a row for each model/color combination.
ModelVO represents a model-level summary of the lines.
Both VOs have a "quantity" field (an Integer).
There is a ViewLink between them and each has a row accessor to the other.
I want to achieve two-way coordination between these two view objects, such that:
When a user queries data, ModelVO.Quantity equals the sum of LineVO.Quantity, for the associated rows
When a user updates any LineVO.Quantity, the ModelVO.Quantity is immediately updated to reflect the new total
When a user updates a ModelVO.Quantity, the quantity is spread among the associated LineVO rows (according to complex business logic which I hope is not relevant here).
I have tried many different ways to do this and cannot get it perfect.
Right now, I am working on variations where ModelVO.Quantity is set to a Groovy expression "LineVO.sum('Quantity')". Unfortunately, everything I try either has the summing from LineVO->ModelVO working or the spreading from ModelVO->LineVO working, but never both at the same time.
Can someone suggest a way to do this? I want to do it in the model layer (either a EO or VO or combination).
Nevermind.. it turns out to be simple:
ModelVO.Quantity must be set to a Groovy "LineVO.sum('Quantity')" and it must have a recalcExpression set to a method where I can control things so it only recalculates when I am changing a LineVO.Quantity value.
The reason my approach didn't work initially was because, when the user updated a LineVO.Quantity value and I wanted to recalculate, I was getting the ModelVO row by lineVORow.getModelVO()... i.e., via a view accessor.
Apparently, that returns some sort of internal copy of the row and not the actual row.
When I got the parent row via the applicationModule.getModelVO().getCurrentRow(), the whole thing works perfectly.
I've posted another question about why accessing the row via the view accessor did not work. That part is still a mystery to me.
I'm using an MDX query to pull information to support a set of reports. A high degree of detail is required for the reports so they take some time to generate. To speed up the access time we pull the data we need and store it in a flat Oracle table and then connect to the table in Excel. This makes the reports refresh in seconds instead of minutes.
Previously the MDX was generated and run by department for 100 departments and then for a number of other filters. All this was done in VB.Net. The requirements for filters have grown to the point where this method is not sustainable (and probably isn't the best approach regardless).
I've built the entire dataset into one MDX query that works perfectly. One of my sets that I cross join includes members from three different levels of hierarchy, it looks like this:
(
Descendants([Merch].[Merch CHQ].[All], 2),
Descendants([Merch].[Merch CHQ].[All], 3),
[Merch].[Merch CHQ].[Department].&[1].Children
)
The problem for me is in our hierarchy (which I can't change), each group (first item) and each department (second item) have the same structure to their naming, ie 15-DeptName and it's confusing to work with.
To address it I added a member:
MEMBER
[Measures].[Merch Level] AS
(
[Merch].[Merch CHQ].CurrentMember.Level.Name
)
Which returns what type the member is and it works perfectly.
The problem is that it updates for every member so none of the rows get filtered by NON BLANK, instead of 65k rows I have 130k rows which will hurt my access performance.
Can my query be altered to still filter out the non blanks short of using IIF to check each measurement for null?
You can specify Null for your member based on your main measure like:
MEMBER
[Measures].[Merch Level] AS
IIf(IsEmpty([Measures].[Normal Measure]),null,[Merch].[Merch CHQ].CurrentMember.Level.Name)
That way it will only generate when there is data. You can go further and add additional dimensions to the empty check if you need to get more precise.
How would one go about writing a query that selects items 2000-2010 out of a collection of 10000 objects in the data store.
I know that it can be done like this in GQL:
select * from MyObject limit 10 offset 2000
According to the documentation, when using an offset the engine will still fetch all the rows, only not return them, thus making the query perform in a way that corresponds linearly with the value of offset.
Is there any better way? Such as using a pseudo ROWNUM column like one could do in other types of data stores.
There's no way to efficiently page using offsets, except to cache the results. You can, however, use datastore cursors to implement paging using a 'bookmark' type approach.
Besides using cursors you can also use a sort order approach. For example:
SELECT * FROM MyObject ORDER BY field LIMIT 10;
for the first 10 objects and then for the next 10 objects, etc.
SELECT * FROM MyObject WHERE field > largestFieldValueFromPreviousResult ORDER BY field LIMIT 10;
Field could even be a key if you don't have another appropriate field. Here is a more complete example:
http://code.google.com/appengine/articles/paging.html