Postgres Sql - How to apply Offset on Timestamp - database

My offset-date-time object I store in the DB with 2 columns, one timestamp(UTC) column
and another corresponding offset.
For example, if I get: 2017-05-01T16:16:35+05:00, in the DB I will store this data in 2 columns the first timestamp will have the value in UTC (2017-05-01T11:16:35), and the offset column will have the +5 timezone in minutes so -300 in minutes.
Now I need select this data from DB, but I need to apply offset and again get the data that was entered: 2017-05-01T16:16:35+05:00.
I can achieve this in Java by selecting both values and applying offset.
But I want to do DB level?

For example, if I get: 2017-05-01T16:16:35-05:00, in the DB I will
store this data in 2 columns the first timestamp will have the value
in UTC (2017-05-01T11:16:35), and the offset column will have the -5
timezone in minutes so -300 in minutes.
https://www.postgresql.org/docs/current/datatype-datetime.html. see: UTC offset for PST (ISO 8601 extended format)
So 2017-05-01T16:16:35-05:00 is an timestamptz type value, therefore at utc timezone value should be 2017-05-01 21:16:35+00!
demo
create table test_timestamp(
org text,
tsz timestamptz,
ts timestamp,
offsettz interval
);
org as timestamp or timestamptz input text. First we assume that org text format ending with something like '1999-01-08 04:05:06-8:00', the pattern is last part is like [+/-]99:99. the last part refer to the offset time to the UTC.
tsz cast text to timestamptz
ts ignore timezone, only timestamp.
offsettz interval can be positive or negative. offsettz is the pattern [+/-]99:99 text cast to interval.
Then create a trigger, the only input is org text, all other 3 column would be computed via trigger.
CREATE OR REPLACE FUNCTION supporttsz ()
RETURNS TRIGGER
AS $$
BEGIN
NEW.tsz := (NEW.org)::timestamptz at time zone 'utc';
NEW.ts := (NEW.org)::timestamptz at time zone 'utc';
NEW.ts := NEW.ts::timestamp;
IF SUBSTRING((
RIGHT (trim(NEW.org) , 5)) , 1 , 1) = '-' THEN
NEW.offsettz := (
RIGHT (trim(NEW.org)
, 5))::interval;
elsif SUBSTRING((
RIGHT (trim(NEW.org) , 5)) , 1 , 1) = '+' THEN
NEW.offsettz := (
RIGHT (trim(NEW.org)
, 5))::interval;
elsif SUBSTRING((
RIGHT (trim(NEW.org) , 6)) , 1 , 1) = '-' THEN
NEW.offsettz := (
RIGHT (trim(NEW.org)
, 6))::interval;
elsif SUBSTRING((
RIGHT (trim(NEW.org) , 6)) , 1 , 1) = '+' THEN
NEW.offsettz := (
RIGHT (trim(NEW.org)
, 6))::interval;
ELSE
NEW.offsettz := '0::00'::interval;
END IF;
RETURN new;
END;
$$
LANGUAGE plpgsql;
CREATE TRIGGER tg_supporttsz_test_timestamp
BEFORE INSERT ON test_timestamp FOR EACH ROW
EXECUTE PROCEDURE supporttsz ();

Sorry, I can't add comment because of low score.
I just tried below script, hope it can help.
create table test (id int, col2 timestamp, col3 int);
insert into test values(1, '2017-05-01T11:16:35', -300);
select * from test;
select (col2 - col3 * INTERVAL '1 minute')::varchar(32) || (col3/60) :: varchar(20) ||':00' from test;
We can finally get below result:
2017-05-01 16:16:35-5:00

Related

Data Vault 2 - Hash diff and recurring data changes

I am having issues retrieving the latest value in a satellite table when some data is changed back to a former value.
The database is Snowflake.
As per Data Vault 2.0, I am currently using the hash diff function to assess whether to insert a new record in a satellite table, like this:
INSERT ALL
WHEN (SELECT COUNT(*) FROM SAT_ACCOUNT_DETAILS AD WHERE AD.MD5_HUB_ACCOUNT = MD5_Account AND AD.HASH_DIFF = AccHashDiff) = 0
THEN
INTO SAT_ACCOUNT_DETAILS (MD5_HUB_ACCOUNT
, HASH_DIFF
, ACCOUNT_CODE
, DESCRIPTION
, SOME_DETAIL
, LDTS)
VALUES (MD5_AE_Account
, AccHashDiff
, AccountCode
, Description
, SomeDetail
, LoadDTS)
SELECT DISTINCT
MD5(AccountId) As MD5_Account
, MD5(UPPER(COALESCE(TO_VARCHAR(AccountCode), '')
|| '^' || COALESCE(TO_VARCHAR(Description), '')
|| '^' || COALESCE(TO_VARCHAR(SomeDetail), '')
)) AS AccHashDiff
, AccountCode
, Description
, SomeDetail
, LoadDTS
FROM source_table;
The first time, a new record with AccountCode = '100000' and SomeDetail = 'ABC' is added:
MD5_HUB_ACCOUNT
HASH_DIFF
ACCOUNT_CODE
DESCRIPTION
SOME_DETAIL
LDTS
c81e72...
8d9d43...
100000
An Account
ABC
2021-04-08 10:00
An hour later, an update changes the value of SomeDetail to 'DEF', this is the resulting table:
MD5_HUB_ACCOUNT
HASH_DIFF
ACCOUNT_CODE
DESCRIPTION
SOME_DETAIL
LDTS
c81e72...
8d9d43...
100000
An Account
ABC
2021-04-08 10:00
c81e72...
a458b2...
100000
An Account
DEF
2021-04-08 11:00
A third update sets the value of SomeDetail back to 'ABC', but the record is not inserted in the satellite table, because the value of the hash diff is the same as the first inserted record (i.e. 8d9d43...).
If I query which is the latest record in the satellite table, the LDTS column tells me it's the one with 'DEF' which is not the desired result.
Instead, I should have a record with SomeDetail = 'ABC' and LDTS = '2021-04-08 12:00'.
What is the correct approach to this? If I add LoadDTS to the hash diff, a new record will be created each time an update is pushed, which is not the desired result either.
As you (and also the standard) mentionned, you need to compare to the last effective record.
I'm not an expert with Snowflake, but it might look like this :
INSERT ALL
WHEN (SELECT COUNT(*) FROM SAT_ACCOUNT_DETAILS AD WHERE AD.MD5_HUB_ACCOUNT = MD5_Account AND AD.HASH_DIFF = AccHashDiff AND AD.LDTS = (SELECT MAX(LDTS) FROM SAT_ACCOUNT_DETAILS MAD WHERE MAD.MD5_HUB_ACCOUNT = AD.MD5_HUB_ACCOUNT)) = 0
THEN ....
By adding "AD.LDTS = (SELECT MAX(LDTS) FROM....." to the query, you make sure you test against the latest data and not historical data

Query using a statement within a VARCHAR2 column

Is there a way for a select statement to include in the WHERE clause a statement that is contained within the table? For example, the following table:
CREATE TABLE test_tab(
date_column DATE,
frequency NUMBER,
test_statement VARCHAR2(255)
)
/
If
MOD(SYSDATE - DATE, frequency) = 0
were contained within the column test_statement, is there a way to select rows where this is true? The test_statement will vary and not be the same throughout the table. I am able to do this in PL/SQL but looking to do this without the use of PL/SQL.
This kind of dynamic SQL in SQL can created with DBMS_XMLGEN.getXML. Although the query looks a bit odd so you might want to consider a different design.
First, I created a sample table and row using your DDL. I'm not sure exactly what you're trying to do with the conditions, so I simplified them into two rows with simpler conditions. The first row matches the first condition, and neither row matches the second condition.
--Create sample table and row that matches the condition.
CREATE TABLE test_tab(
date_column DATE,
frequency NUMBER,
test_statement VARCHAR2(255)
)
/
insert into test_tab values(sysdate, 1, 'frequency = 1');
insert into test_tab values(sysdate, 2, '1=2');
commit;
Here's the large query, and it only returns the first row, which only matches the first condition.
--Find rows where ROWID is in a list of ROWIDs that match the condition.
select *
from test_tab
where rowid in
(
--Convert XMLType to relational data.
select the_rowid
from
(
--Convert CLOB to XMLType.
select xmltype(xml_results) xml_results
from
(
--Create a single XML file with the ROWIDs that match the condition.
select dbms_xmlgen.getxml('
select rowid
from test_tab where '||test_statement) xml_results
from test_tab
)
where xml_results is not null
)
cross join
xmltable
(
'/ROWSET/ROW'
passing xml_results
columns
the_rowid varchar2(128) path 'ROWID'
)
);
This calls for dynamic SQL, so - yes, it is PL/SQL that handles it. I don't think that SQL layer is capable of doing it.
I don't know what you tried so far, so - just an idea: a function that returns ref cursor might help, e.g.
SQL> create table test (date_column date, frequency number, test_statement varchar2(255));
Table created.
SQL> insert into test values (trunc(sysdate), 2, 'deptno = 30');
1 row created.
SQL> create or replace function f_test return sys_refcursor
2 is
3 l_str varchar2(200);
4 l_rc sys_refcursor;
5 begin
6 select test_statement
7 into l_str
8 from test
9 where date_column = trunc(sysdate);
10
11 open l_rc for 'select deptno, ename from emp where ' || l_str;
12 return l_rc;
13 end;
14 /
Function created.
Testing:
SQL> select f_test from dual;
F_TEST
--------------------
CURSOR STATEMENT : 1
CURSOR STATEMENT : 1
DEPTNO ENAME
---------- ----------
30 ALLEN
30 WARD
30 MARTIN
30 BLAKE
30 TURNER
30 JAMES
6 rows selected.
SQL>
A good thing about it is that you could save the whole statements into that table and run any of them using the same function.
You can try this
select * from test_tab where mod(sysdate - date, frequency) = 0;

Finding date/time intervals from multiple rows T-SQL

I have an audit table that stores a master table's "status" values in multiple rows, each row has a date/time stamp. Let's call the master table "Project" and it moves through different status changes: Open, Pending, Closed. However sometimes the Project can move back to Open after it's been closed. Data would be something like this:
ProjectId Date Status
1234 07-01-2015 Open
1234 07-03-2015 Pending
1234 07-05-2015 Closed
1234 07-06-2015 Open
I need to take a date parameter value, and determine what status the Project "1234" was in on that given date.
ex: Date=07-02-2015 would give me "Open"
Date=07-07-2015 would also give me "Open"
Date=07-03-2015 would give me "Pending"
The problem I'm having is that the SQL needs to look at the surrounding rows (if any)
1) Limit the rows to only rows with date inside the window.
2) Sort the result by date descending (putting the most recent at the top)
3) Select the first row
DECLARE #CheckDate DATETIME = '07-07-2015',
#ProjectId INT = 1234
-- To ignore time, set the check date to the first millisecond of the
-- next day and use a less than operator
SET #CheckDate = DATEADD(dd, DATEDIFF(dd, 0, #CheckDate) + 1, 0)
SELECT TOP 1
[Status]
FROM [Project]
WHERE [ProjectId] = #ProjectId
AND [Date] < #CheckDate
ORDER BY [Date] DESC
If you have 2012 or later, you can do this with Lead as follows:
declare #date datetime = '7-2-15' --or whatever
select ProjectID, Status from (
select *
, lead(date) over (partition by projectID order by date) as NextEvent
from MyTable) a
where #date between [date] and isnull(NextEvent, #date + 1)
Otherwise, you can approximate lead using row_number and a self-join.
Note, depending on how precise your dates are, you may want to use something like where #date >= date and #date < isnull(NextEvent, #date + 1) instead of between.

selecting records from a table where a given date falls between 2 dates (columns) from a table in SQL Server

EDIT -----------------
I'm going to try to clarify.
I have this table:
A user will input a start date and an end date.
Let's say the user inputs 2011-10-21 for the start and 2011-12-18 for the end.
This will bring back all the records:
But what if the user inputs 2011-10-22 and 2011-12-18?
The 2011-10-22 is BETWEEN the date range of the FIRST ROW 2011-10-21 (start) & 2011-10-23 (end).
SINCE IT IS BETWEEN THE DATE RANGE OF THE FIRST RECORD, I WANT to return that row as well. I DO NOT WANT TO EXCLUDE IT.
So if if I run a query like this:
select * from table where PayPeriodStart between '2011-10-22' and '2011-12-18'
I WANT to see this:
and NOT this:
PLEASE NOTE... the user can enter in any start or end date and it'll be at least 1 week.
So they can pick 2011-11-09 (start) and 2011-12-04 (end) for example.
You'll want the search criteria to return the rows where the period start date is before the end date criteria and the period end date is after the start date criteria. In other words:
declare #Table table (PayPeriodStart smalldatetime, PayPeriodEnd smalldatetime)
insert into #Table (PayPeriodStart, PayPeriodEnd)
select '2010-01-01', '2010-12-31' -- test before the criteria; is not returned
union select '2011-10-21', '2011-10-23' -- overlaps start date; is returned
union select '2011-10-24', '2011-11-06' -- fully in date range; is returned
union select '2011-11-07', '2011-11-20' -- fully in date range; is returned
union select '2011-11-21', '2011-12-04' -- fully in date range; is returned
union select '2011-12-05', '2011-12-18' -- overlaps end date; is returned
union select '2012-01-01', '2012-12-31' -- test after the criteria; is not returned
-- This yields all but the first row
select * from #Table where PayPeriodStart between '2011-10-22' AND '2011-12-10'
-- This will yield all the rows that overlap the search criteria
select * from #Table
where PayPeriodStart <= '2011-12-10' -- the end search date
and PayPeriodEnd >= '2011-10-22' -- the start search date
If you are looking for all records that are valid between '2011-10-22' AND '2011-12-10' , this is the query you should use
select * from table where ('2011-10-22' >= PayPeriodStart AND '2011-10-22' <= PayPeriodEnd) Or ('2011-10-22' < PayPeriodStart AND '2011-12-10' > PayPeriodEnd)
I have never been a fan of the between syntax for the edge case reason, so I tend to use this syntax instead:
select * from table where ('2011-10-22' <= PayPeriodStart AND PayPeriodStart <= '2011-12-10')
Same end result, but I can ensure I include the boundaries as needed.
Are you asking for
select * from table where '2011-10-22' between PayPeriodStart AND PayPeriodEnd
AND '2011-12-10' between PayPeriodStart AND PayPeriodEnd
If you want to get data based on first of the month and last of the month or for fixed time interval like 30 day window, you may have to use DateAdd, DateDiff functions to calculate dates.
Following example is based on FirstOfTheMonth, LastOfTheMonth calculation
your reference data
declare #refDate date = getdate()
select #min = dateadd(mm, datediff(mm, 0, #refDate), 0) -- First of the month
, #max = dateadd(dd, -1, dateadd(mm, datediff(mm, 0, #refDate) + 1, 0)) --Last of the month

SQL Server stored procedure datetime parameter on selecting data

I have a football league database and I am trying to get the current score of the teams on a specific date. If I type in a date in the past I want the score at that specific date - the points of each teams on that date.
In my database I have a table that includes all the matches and I have a table with the teams and their point (this table is actually the same as the current score).
The two tables are:
create table teams
(
id char(3) primary key,
name varchar(40),
nomatches int,
owngoals int,
othergoals int,
points int
)
create table matches
(
id int identity(1,1),
homeid char(3) foreign key references teams(id),
outid char(3) foreign key references teams(id),
homegoal int,
outgoal int,
matchdate datetime
)
I am trying to use a stored procedure where I have a datetime as a parameter to show the current score (the team table) at that date defined by the parameter.
Right now I'm selecting all the matches that is bigger (newer) than the date I want til score table from and subtracting the result of that match from the teams point.
But it seems to me that it is a lot of work to do for something that simple.
Does anyone have a better idea?
Why do you subtract? Seems to me that the correct way to go would be to take the league start date and calculate the scores from that day up to selected date.
I'm not sure why you would want to put such a simple query into a procedure, but essentially what you're asking is as follows:
CREATE PROCEDURE sp_goals_to_date(#todate DATETIME) AS
SELECT id, SUM(homegoal) AS homegoal, SUM(outgoal) AS outgoal FROM matches WHERE matchdate <= #mydate
What Fedor says is correct - it is more efficient to do a single calculation from the beginning of time, than to have to do a calc from two different tables.
I would first transform the matches table like this:
SELECT
teamid = CASE t.calchometeam WHEN 1 THEN m.homeid ELSE m.outid END,
owngoal = CASE t.calchometeam WHEN 1 THEN m.homegoal ELSE m.outgoal END,
othergoal = CASE t.calchometeam WHEN 0 THEN m.homegoal ELSE m.outgoal END,
points = CASE m.homegoal
WHEN m.outgoal THEN #drawpoints
ELSE (SIGN(m.homegoal - m.outgoal) + 1) / 2 ^ ~m.playedhome) * #winpoints
+ (SIGN(m.homegoal - m.outgoal) + 1) / 2 ^ m.playedhome) * #losspoints
END
FROM matches m
CROSS JOIN (
SELECT CAST(0 AS bit) UNION ALL
SELECT CAST(1 AS bit)
) AS t (calchometeam)
WHERE m.matchdate <= #givendate
Now it's easier to calculate the necessary totals:
SELECT
teamid,
nomatches = COUNT(*),
owngoals = SUM(owngoal),
othergoals = SUM(othergoal),
points = SUM(points)
FROM transformed_matches
GROUP BY
teamid
Next step would be to join the last result set to the teams table to get the team's names. And if you actually need that final step, you could, of course, perform the calculations the way you intended from the beginning, i.e. calculate only the stats you need to subtract from the current values, rather than the actual standings. So, using this inverted logic, the entire query might look like this:
WITH
transformed_matches AS (
SELECT
matchid = m.id,
teamid = CASE t.calchometeam WHEN 1 THEN m.homeid ELSE m.outid END,
owngoal = CASE t.calchometeam WHEN 1 THEN m.homegoal ELSE m.outgoal END,
othergoal = CASE t.calchometeam WHEN 0 THEN m.homegoal ELSE m.outgoal END,
points = CASE m.homegoal
WHEN m.outgoal THEN #drawpoints
ELSE (SIGN(m.homegoal - m.outgoal) + 1) / 2 ^ ~m.playedhome) * #winpoints
+ (SIGN(m.homegoal - m.outgoal) + 1) / 2 ^ m.playedhome) * #losspoints
END
FROM matches m
CROSS JOIN (
SELECT CAST(0 AS bit) UNION ALL
SELECT CAST(1 AS bit)
) AS t (calchometeam)
WHERE m.matchdate > #givendate
),
aggregated AS (
SELECT
teamid,
nomatches = COUNT(*),
owngoals = SUM(owngoal),
othergoals = SUM(othergoal),
points = SUM(points)
FROM transformed_matches
GROUP BY
teamid
)
SELECT
t.id,
t.name,
nomatches = t.nomatches - ISNULL(a.nomatches , 0),
owngoals = t.owngoals - ISNULL(a.orngoals , 0),
othergoals = t.nomatches - ISNULL(a.othergoals, 0),
points = t.points - ISNULL(a.points , 0)
FROM teams t
LEFT JOIN aggregated a ON t.id = a.teamid
Note: You didn't specify which kind of football you meant, but living in Europe, it was easier for me to assume association football rather than any other kind. Yet, because I was not sure, I decided to parametrise my query. That is why you can see all those #winpoints, #drawpoints and #losspoints placeholders. You can replace the variables with the actual constants, if you like, or you could leave the query parametrised in case you wanted to satisfy your curiosity as to what the team's standings would have been if a different scoring system were in effect.

Resources