I am currently working on a query that is registered for Query Notifications. In accordance w/ the rules of Notification Serivces, I can only use Deterministic functions in my queries set up for subscription. However, GetDate() (and almost any other means that I can think of) are non-deterministic. Whenever I pull my data, I would like to be able to limit the result set to only relevant records, which is determined by the current day.
Does anyone know of a work around that I could use that would allow me to use the current date to filter my results but not invalidate the query for query notifications?
Example Code:
SELECT fcDate as RecordDate, fcYear as FiscalYear, fcPeriod as FiscalPeriod, fcFiscalWeek as FiscalWeek, fcIsPeriodEndDate as IsPeriodEnd, fcPeriodWeek as WeekOfPeriod
FROM dbo.bFiscalCalendar
WHERE fcDate >= GetDate() -- This line invalidates the query for notification...
Other thoughts:
We have an application controls table in our database that we use to store application level settings. I had thought to write a small script that keeps a record up to date w/ teh current smalldatetime. However, my join to this table is failing for notificaiton as well and I am not sure why. I surmise that it has something to do w/ me specifitying a text type (the column name), which is frustrating.
Example Code 2:
SELECT fcDate as RecordDate, fcYear as FiscalYear, fcPeriod as FiscalPeriod, fcFiscalWeek as FiscalWeek, fcIsPeriodEndDate as IsPeriodEnd, fcPeriodWeek as WeekOfPeriod
FROM dbo.bFiscalCalendar
INNER JOIN dbo.xApplicationControls ON fcDate >= acValue AND acName = N'Cache_CurrentDate'
Does anyone have any suggestions?
EDIT: Here is a link on MSDN that gives the rules for Notification Services
As it turns out, I figured out the solution. Basically, I was invalidating my query attempts because I was casting a value as a DateTime which marks it as Non-Deterministic. Even though you don't specifically call out a cast but do something akin to:
RecordDate = 'date_string_value'
You still end up w/ a Date Cast. Hopefully this will help out someone else who hits this issue.
This link helped me quite a bit.
http://msdn.microsoft.com/en-us/library/ms178091.aspx
A good way to bypass this is simply to create a view that just says "SELECT GetDate() AS Now", then use the view in your query.
EDIT : I see nothing about not using user-defined functions (which is what I've used the 'view today' bit in). So can you use a UDF in the query that points at the view?
Related
I have a table in Snowflake where one of the fields, called 'value' is sometimes plain text sometimes JSON, this field is stored as string in Snowflake
I created this view to get only the rows where there is a Json format
CREATE OR REPLACE VIEW tmp_events AS
SELECT PARSE_JSON(value) as json_data,
id
FROM SessionEvent
WHERE session_event_type_id=7;
Then I flatten the rows to create a new field
CREATE OR REPLACE VIEW tmp_events_step2 AS
SELECT id,
json_data:meta:selected::string AS choice
from tmp_events ,
LATERAL FLATTEN(input => tmp_events.json_data)
WHERE choice IS NOT NULL
Everything runs fine until now, I can preview data from these two views, no error and I get the results I was expecting.
The error comes when I try to get distinct values from choice
SELECT DISTINCT choice from tmp_events_step2;
Error parsing JSON: unknown keyword "brain", pos 6
This name Brain seems to come from my initial table without the WHERE statement.
If I run the query without DISTINCT there is no error.
Weird thing I noticed while trying to debug: when I put a limit in tmp_events_step2, the code works fine again, even though I put a limit that's bigger than the number of rows in the table
CREATE OR REPLACE VIEW tmp_events_step2 AS
SELECT id,
json_data:meta:selected::string AS choice
from tmp_events ,
LATERAL FLATTEN(input => tmp_events.json_data)
WHERE choice IS NOT NULL
LIMIT 10000000;
SELECT DISTINCT choice from tmp_events_step2;
What's the catch? Why does it work only with the limit?
The very simple answer to this is the built-in function TRY_PARSE_JSON()
Er, not. You seem to have problems with the Query optimizer that may do incorrect predicate pushdowns. One way to prevent the optimizer from doing this is to use the secure view option:
CREATE SECURE VIEW tmp_events_step2 ...
and file a support ticket...
We reported this error two years ago and they said they where not going to fix, because by hoisting the JSON access prior to running the filters in the WHERE clause that makes the cast valid/safe, impacted performance.
create table variant_cast_bug(num number, var variant);
insert into variant_cast_bug
select column1 as num, parse_json(column2) as var
from values (1, '{"id": 1}'),
(1, '{"id": 2}'),
(2, '{"id": "text"}')
v;
select * from variant_cast_bug;
select var:id from variant_cast_bug;
select var:id from variant_cast_bug where num = 1;
select var:id::number from variant_cast_bug where num = 1; -- <- broken
select TRY_TO_NUMBER(var:id) from variant_cast_bug where num = 1; -- <- works
Sometimes you can nest the select and it will work, and then you can add another SELECT layer around it, and do some aggregation and the cost explodes again.
The only two safe solutions are SERCURE VIEW as Hans mentions, but that is a performance nightmare.
Or to understand this problem and use TRY_TO_NUMBER or it's friends.
At the time this was made bad worse because JSON boolean values where not valid values to pass to TRY_TO_BOOLEAN..
One of the times we got burnt by this was after a snowflake release when code that had been running for a year, started getting this error, because it was complex enough the hoisting did not impact, and then after release it did. This is where Snowflake are rather responsive, and then rolled the release back, and we put TRY_TO on a chunk of already working SQL just to play it safe.
Please submit a support case for this issue.
My department uses a software tool that can use a custom component library sourced from Tables or Queries in an MS Access database.
Table: Components
ID: AutoNumber
Type: String
Mfg: String
P/N: String
...
Query: Resistors
SELECT Components.*
FROM Components
WHERE Components.Type = "Resistors"
Query: Capacitors
SELECT Components.*
FROM Components
WHERE Components.Type = "Capacitors"
These queries work fine for SELECT. But when users add a row to the query, how can I ensure the correct value is saved to the Type field?
Edit #2:
Nope, can't be done. Sorry.
Edit #1:
As was pointed out, I may have misunderstood the question. It's not a wonky question after all, but perhaps an easy one?
If you're asking how to add records to your table while making sure that, for example, "the record shows up in a Resistors query if it's a Resistor", then it's a regular append query, that specifies Resisitors as your Type.
For example:
INSERT INTO Components ( ID, Type, Mfg )
SELECT 123, 'Resistors', 'Company XYZ'
If you've already tried that and are having problems, it could be because you are using a Reserved Word as a field name which, although it may work sometimes, can cause problems in unexpected ways.
Type is a word that Access, SQL and VBA all use for a specific purpose. It's the same idea as if you used SELECT and FROM as field or table names. (SELECT SELECT FROM FROM).
Here is a list of reserved words that should generally be avoided. (I realize it's labelled Access 2007 but the list is very similar, and it's surprisingly difficult to find an recent 'official' list for Excel VBA.)
Original Answer:
That's kind a a wonky way to do things. The point of databases is to organize in such a way as to prevent duplication of not only data, but queries and codes as well
I made up the programming rule for my own use "If you're doing anything more than once, you're doing it wrong." (That's not true in all cases but a general rule of thumb nonetheless.)
Are the only options "Resistors" and "Capacitors"? (...I hope you're not tracking the inventory of an electronics supply store...) If there are may options, that's even more reason to find an alternative method.
To answer your question, in the Query Design window, it is not possible to return the name of the open query.
Some alternative options:
As #Erik suggested, constrain to a control on a form. Perhaps have a drop-down or option buttons which the user can select the relevant type. Then your query would look like:
SELECT * FROM Components WHERE Type = 'Forms![YourFormName]![NameOfYourControl]'
In VBA, have the query refer to the value of a variable, foe example:
Dim TypeToDel as String
TypeToDel = "Resistor"
DoCmd.RunSQL "SELECT * FROM Components WHERE Type = '" & typeToDel'"
Not recommended, but you could have the user manually enter the criteria. If your query is like this:
SELECT * FROM Components WHERE Type = '[Enter the component type]'
...then each time the query is run, it will prompt:
Similarly, you could have the query prompt for an option, perhaps a single-digit or a code, and have the query choose the the appropriate criteria:
...and have an IF statement in the query criteria.
SELECT *
FROM Components
WHERE Type = IIf([Enter 1 for Resistors, 2 for Capacitors, 3 for sharks with frickin' laser beams attached to their heads]=1,'Resistors',IIf([Enter 1 for Resistors, 2 for Capacitors, 3 for sharks with frickin' laser beams attached to their heads]=2,'Capacitors','LaserSharks'));
Note that if you're going to have more than 2 options, you'll need to have the parameter box more than once, and they must be spelled identically.
Lastly, if you're still going to take the route of a separate query for each component type, as long as you're making separate queries anyway, why not just put a static value in each one (just like your example):
SELECT * FROM Components WHERE Type = 'Resistor'
There's another wonky answer here but that's just creating even more duplicate information (and more future mistakes).
Side note: Type is a reserved word in Access & VBA; you might be best to choose another. (I usually prefix with a related letter like cType.)
More Information:
Use parameters in queries, forms, and reports
Use parameters to ask for input when running a query
Microsoft Access Tips & Tricks: Parameter Queries
• Frickin' Lasers
I am trying to query if a certain date belongs to a specific range of dates. Source code example:
billing_period_found = BillingPeriod.query(
ndb.AND(
transaction.date > BillingPeriod.start_date,
transaction.date < BillingPeriod.end_date)
).get()
Data definition:
class Transaction(ndb.Model):
date = ndb.DateProperty(required=False)
class BillingPeriod(ndb.Model):
start_date = ndb.DateProperty(required=False)
end_date = ndb.DateProperty(required=False)
Getting the following error:
TypeError: can't compare datetime.date to DateProperty
The message error does make sense because datetime is different from DateProperty. However, as you can see, the definition for transaction.date is not datetime, so I am not getting where this attempt to convert datetime to date is coming from. Anyways - If I figure out how to convert datetime to DateProperty, I guess it would fix the problem.
Any ideas on how to solve this?
Thanks!
The App Engine datastore does not allow queries with inequalities on multiple properties (not a limitation of ndb, but of the underlying datastore). Selecting date-range entities that contain a certain date is a typical example of tasks that this makes it impossible to achieve in a single query.
Check out Optimizing a inequality query in ndb over two properties for an example of this question, and, in the answer, one suggestion that might work: query for (in your case) all BillingPeriod entities with end_date greater than the desired date, perhaps with a projection to just get their key and start_date; then, select out of those only those with start_date less than the desired date, in your own application (if you only want one of them, then a next over the iterator will stop as soon as it finds one).
Edit: the issue above is problem #1 with this code; once solved, problem #2 arises -- as clearly listed at https://cloud.google.com/appengine/docs/python/ndb/queries, the property is ndb queries is always on the left of the comparison operator. So, one can't do date < BillingPeriod.end_date, as that would have the property on the right; rather, one does BillingPeriod.end_date > date.
Ok this sounds like a problem that someone else should have solved already, but I can't find any help on it, just that their should be a better way of doing it than using an unequal join.
I have a log file of session info with a Source IP, and I am trying to create a query, that actually runs, to combine the Log file with Geo-IP data to tell the DB where users are connecting from. my first attempt came to this:-
SELECT coco, region, city
FROM GT_Geo_IP
WHERE (IP_Start <= [IntIP] AND IP_End >=[IntIP])
ORDER BY IP_Start;
it seemed to run quite quick and returns the correct record for a given IP. but when I tried to combine it with the log data, like this:-
SELECT T.IP,G.coco,G.region, g.city
FROM GT_Geo_IP as G, Log_Table as T
WHERE G.IP_Start <= T.IntIP AND G.IP_End >= T.IntIP
ORDER BY T.IP;
it locks access for over 45 mins (pegging one of my cpu cores) before i finally decide i need some CPU back or i should actually have a go at something else. From hunting around and this is actually slower than i realize, I found this article and indexed both IP_Start and IP_End to optimize the Search, and based on it came up with this:-
SELECT TOP 1 coco, region, city
FROM GT_Geo_IP
WHERE G.IP_Start >= [IntIP]
ORDER BY G.IP_Start;
But with my SQL skills i cant work out how to combine it with my log data.
Basically the question is how do i use the better method with my log data to get the required result? or is there a better way to do it?
The GeoIP data is from IP2Location's LITE-DB3
I have thought about nested queries, but i couldn't work out how to construct it, I thought about using VBA, but i'm not sure it will be any quicker
#EnviableOne
Please try this SQL statement.
SELECT
L.IP,
(
SELECT TOP 1
CONCAT(coco, ', ', region, ', ', city)
FROM
GT_Geo_IP
WHERE
IP_Start >= L.intIP ORDER BY IP_Start
) AS Location
FROM
Log_Table AS L;
I've been banging my head on this issue for a little while now, and decided I should ask for help. I have a table which holds temperature/humidity chart recorder data (currently over 775,000 records) from which I am trying to run a statistical query against it. The problem is that this often will take up to two minutes, and sometimes will not come back at all - causing me to force close the program (Control-Alt-Delete). At first, I didn't have as much of a problem - it was only after I hit the magical 500k records mark that I started getting serious slowdowns, getting progressively worse as more data was compiled and imported into the table.
Here is the query (pass-through):
SELECT dbo.tblRecorderLogs.strAreaAssigned, Min(dbo.tblRecorderLogs.datDateRecorded) AS FirstRecorderDate, Max(dbo.tblRecorderLogs.datDateRecorded) AS LastRecordedDate,
Round(Avg(dbo.tblRecorderLogs.intTempCelsius),2) AS AverageTempC,
Round(Avg(dbo.tblRecorderLogs.intRHRecorded),2) AS AverageRH,
Count(dbo.tblRecorderLogs.strAreaAssigned) AS Records
FROM dbo.tblRecorderLogs
GROUP BY dbo.tblRecorderLogs.strAreaAssigned
ORDER BY dbo.tblRecorderLogs.strAreaAssigned;
Here is the table structure in which the chart data is stored:
idRecorderDataID Number Primary Key
datDateEntered Date/Time (indexed, duplicates OK)
datTimeEntered Date/Time
intTempCelcius Number
intDewPointCelcius Number
intWetBulbCelcius Number
intMixingGPP Number
intRHRecorded Number
strAssetRecorder Text (indexed, duplicates OK)
strAreaAssigned Text (indexed, duplicates OK)
I am trying to write a program which will allow people to pull data from this table based on Area Assigned, as well as start and end dates. With the dataset size I currently have, this kind of report is simply too much for it to handle (it seems) and the machine doesn't ever return an answer. I've had to extend the ODBC timeout to almost 180 seconds in any queries dealing with this table, simply because of the size. I could use some serious help, if people have some. Thank you in advance!
-- Edited 08/13/2012 # 1050 hours --
I have not been able to test the query on the SQL Server due to the fact that the IT department has taken control of the machine in question, and has someone logged into it full-time using the remote management console. I have tried an interim step to lessen the impact of the performance issue, but I am still looking for a permanent solution to this issue.
Interim step:
I created a local table mirroring the structure of the dbo.tblRecorderLogs SQL Server table, to which I do a INSERT INTO using the former SELECT statement as it's subquery. Then any subsequent statistical analysis is drawn from this 'temporary' local table. After the process is complete, the local table is truncated.
-- Edited 08/13/2012 # 1217 hours --
Ran the shown query on the SQL Server Management Console, took 1 minute 38 seconds to complete according to the query timer provided by the console.
-- Edit 08/15/2012 # 1531 hours --
Tried to run query as VBA DoCmd.RunSQL statement to populate a temporary table using the following code:
INSERT INTO tblTempRecorderDataStatsByArea ( strAreaAssigned, datFirstRecord,
datLastRecord, intAveTempC, intAveRH, intRecordCount )
SELECT dbo_tblRecorderLogs.strAreaAssigned, Min(dbo_tblRecorderLogs.datDateRecorded)
AS MinOfdatDateRecorded, Max(dbo_tblRecorderLogs.datDateRecorded) AS MaxOfdatDateRecorded,
Round(Avg(dbo_tblRecorderLogs.intTempCelsius),2) AS AveTempC,
Round(Avg(dbo_tblRecorderLogs.intRHRecorded),2) AS AveRHRecorded,
Count(dbo_tblRecorderLogs.strAreaAssigned) AS CountOfstrAreaAssigned FROM
dbo_tblRecorderLogs GROUP BY dbo_tblRecorderLogs.strAreaAssigned ORDER BY
dbo_tblRecorderLogs.strAreaAssigned
The problem arises when the code is executed, the query takes so long - it encounters Timeout before it finishes. Still hoping for a 'magic bullet' to fix this...
-- Edited 08/20/2012 # 1241 hours --
The only 'quasi' solution I've found is running the failed query repeatedly (sort of priming the pump, as it were) so that when the query is called again by my program - it has a relative chance of actually completing before the ODBC SQL Server driver times out. Basically, a filthy filthy hack - but I don't have a better one to combat this issue.
I've tried creating a view, which works on the server side - but doesn't speed things up.
The proper fields being aggregated are indexed properly, so I can't make any changes there.
I am only pulling information from the database that is immediately useful to user - no 'SELECT * madness' going on here.
I think I am, officially, out of things to try - aside from throwing raw computing horsepower at the problem, which isn't a solution right now as the item isn't live, and I have no budget to procure better hardware. I will post this as an 'answer' and leave it up until Sept 3rd - where if I do not have better answers, I will accept my own answer and accept defeat.
When I've had to run min/max functions on several fields from the same table I've often found it quicker to do each column separately as a subquery in the from line of the main/outer query.
So your query would be like this:
SELECT rLogs1.strAreaAssigned, rLogs1.FirstRecorderDate, rLogs2.LastRecorderDate, rLog3.AverageTempC, rLogs4.AverageRH, rLogs5.Records
FROM (((
(SELECT strAreaAssigned, min(datDateRecorded) as FirstRecorderDate FROM dbo.tblRecorderLogs GROUP BY strAreaAssigned) rLogs1
inner join
(SELECT strAreaAssigned, Max(datDateRecorded) as LastRecordedDate, FROM dbo.tblRecorderLogs GROUP BY strAreaAssigned) rLogs2
on rLogs1.strAreaAssigned = rLogs2.strAreaAssigned)
inner join
(SELECT strAreaAssigned, Round(Avg(intTempCelsius),2) AS AverageTempC, FROM dbo.tblRecorderLogs GROUP BY strAreaAssigned) rLogs3
on rLogs1.strAreaAssigned = rLogs3.strAreaAssigned)
inner join
(SELECT strAreaAssigned, Round(Avg(intRHRecorded),2) AS AverageRH, FROM dbo.tblRecorderLogs GROUP BY strAreaAssigned) rLogs4
on rLogs1.strAreaAssigned = rLogs4.strAreaAssigned)
inner join
(SELECT strAreaAssigned, Count(strAreaAssigned) AS Records, FROM dbo.tblRecorderLogs GROUP BY strAreaAssigned) rLogs5
on rLogs1.strAreaAssigned = rLogs5.strAreaAssigned
ORDER BY rLogs1.strAreaAssigned;
If you take your query and the one above, copy them into the same query window in SQL Server and run the estimated execution plan you should be able to compare them and see which one works better.