Overview of situation: I have two databases (one is DB2 and one is MSSQL) and I am using an SSIS package to feed from one to the other via jobs. In our SQL table, datetime fields were set up as SmallDateTime (years and years ago, cannot change at this point in time yet to DateTime). We are now getting dates that are coming through as year 2099 (1/1/2099) which fails as SmallDateTime can only go MaxDate of 06/06/2079 11:59:59.
My/our solution is to use the Derived Column transform to check the date, and if it is over year 2078, make it null. It was also advised that check for null before checking date.
I tried doing this,
[Derived Column Name] [Derived Column ] [Expression]
[ MyDate ] [Replace "MyDate"] [MyDate == "" ? NULL(DT_WSTR,5) : MyDate]
[ VerifiedDates ] [Add As New Column] [VerifiedDates == YEAR((DT_DBDATE)MyDate) > = 2078 ? NULL(DT_WSTR,10) : MyDate]
But this did not work for two reasons. Not only was the expression wrong, it also would not allow me to replace the column of "MyDate" like I did in the first run. Can I not replace a column more than once? Do these tasks happen at the same time?
Due to that issue, I tried to just replace the dates via the expression
[ MyDate ][Replace "MyDate"][YEAR((DT_DBDATE)MyDate) >= 2078 ? NULL(DT_WSTR, 10) : MyDate]
as well as
[ MyDate ][Replace "MyDate"][MyDate == YEAR((DT_DBDATE)MyDate) >= 2078 ? NULL(DT_WSTR, 10) : MyDate]
But none of these seem to be the correct syntax... Can anyone point me to where I am off?
I'm also having trouble finding a good resource for the syntax, presently using this ref
Have you tried the DATEPART function instead
[ MyDate ][Replace "MyDate"][ DATEPART("yyyy", (DT_DBTIMESTAMP)MyDate) >= 2078 ? NULL(DT_WSTR, 10) : MyDate ]
Related
I'm running a query in snowflake to then export. I need to set/convert a date value to the following format 'yyyy-MM-ddThh:mm:ss' from 2022-02-23 16:23:58.805
I'm not sure what is the best way to convert the date format. I've tried using TO_TIMESTAMP, but keep getting the following error '1 too many arguments for function [TO_TIMESTAMP(FSA.LAST_UPDATED, 'yyyy-MM-ddThh:mm:ss')] expected 1, got 2'
This looks like a conversion issue. Please check datatype for your column last_updated. Also seems there is some typo in your question - for the time portion in format, use mi (hh:mi:ss).
Refer below -
select to_timestamp('2022-02-23 16:23:58.805'::TIMESTAMP,'yyyy-mm-dd hh:mi:ss.ff')
;
000939 (22023): SQL compilation error: error line 1 at position 7
**too many arguments for function
[TO_TIMESTAMP(TO_TIMESTAMP_NTZ('2022-02-23 16:23:58.805'), 'yyyy-mm-dd hh:mi:ss.ff')] expected 1, got 2**
select to_timestamp('2022-02-23 16:23:58.805'::string,'yyyy-mm-dd hh:mi:ss.ff');
TO_TIMESTAMP('2022-02-23 16:23:58.805'::STRING,'YYYY-MM-DD HH:MI:SS.FF')
2022-02-23 16:23:58.805
TO_TIMESTAMP is for string -> timestamp, TO_CHAR is for timestamp -> string of which the TO_CHAR( <date_or_time_expr> [, '<format>' ] ) form is the one you seem to be wanting.
this SQL show string -> timestamp -> formatted string
SELECT
'2022-02-23 16:23:58.805' as time_string,
to_timestamp(time_string) as a_timestamp,
to_char(a_timestamp, 'yyyy-MM-ddThh:mm:ss') as formating_string;
TIME_STRING
A_TIMESTAMP
FORMATING_STRING
2022-02-23 16:23:58.805
2022-02-23 16:23:58.805
2022-02-23T16:02:58
I have model I created on the fly for peewee. Something like this:
class TestTable(PeeweeBaseModel):
whencreated_dt = DateTimeField(null=True)
whenchanged = CharField(max_length=50, null=True)
I load data from a text file to a table using peewee, the column "whenchanged" contains all dates in a format of '%Y-%m-%d %H:%M:%S' as varchar column. Now I want to convert the text field "whenchanged" into a datetime format in "whencreated_dt".
I tried several things... I ended up with this:
# Initialize table to TestTable
to_execute = "table.update({table.%s : datetime.strptime(table.%s, '%%Y-%%m-%%d %%H:%%M:%%S')}).execute()" % ('whencreated_dt', 'whencreated')
which fails with a "TypeError: strptime() argument 1 must be str, not CharField": I'm trying to convert "whencreated" to datetime and then assign it to "whencreated_dt".
I tried a variation... following e.g. works without a hitch:
# Initialize table to TestTable
to_execute = "table.update({table.%s : datetime.now()}).execute()" % (self.name)
exec(to_execute)
But this is of course just the current datetime, and not another field.
Anyone knows a solution to this?
Edit... I did find a workaround eventually... but I'm still looking for a better solution... The workaround:
all_objects = table.select()
for o in all_objects:
datetime_str = getattr( o, 'whencreated' )
setattr(o, 'whencreated_dt', datetime.strptime(datetime_str, '%Y-%m-%d %H:%M:%S'))
o.save()
Loop over all rows in the table, get the "whencreated". Convert "whencreated" to a datetime, put it in "whencreated_dt", and save each row.
Regards,
Sven
Your example:
to_execute = "table.update({table.%s : datetime.strptime(table.%s, '%%Y-%%m-%%d %%H:%%M:%%S')}).execute()" % ('whencreated_dt', 'whencreated')
Will not work. Why? Because datetime.strptime is a Python function and operates in Python. An UPDATE query works in database-land. How the hell is the database going to magically pass row values into "datetime.strptime"? How would the db even know how to call such a function?
Instead you need to use a SQL function -- a function that is executed by the database. For example, Postgres:
TestTable.update(whencreated_dt=whenchanged.cast('timestamp')).execute()
This is the equivalent SQL:
UPDATE test_table SET whencreated_dt = CAST(whenchanged AS timestamp);
That should populate the column for you using the correct data type. For other databases, consult their manuals. Note that SQLite does not have a dedicated date/time data type, and the datetime functionality uses strings in the Y-m-d H:M:S format.
I'm struggling with the following task. I have a source file in sft and I must run an ETL JOB to compare the dates in this file (DateA) with the ones in my database (DATEB).
Could you please help me setting this condition:
Condition 1) If Date A is null, leave Date B as it is
Condition 2) If Date A is more recent than Date B, replace Date B with Date A
I was thinking of something like that, but i can't finish it:
"expression": "DATEA eq '' ? DATEB : Condition 2",
"field": "data_CHANNEL_EMAIL_ACCEPTANCEDATE"
}
Is that correct? How to I write the second condition?
I am trying to query a SQL Server database to check for the MAX date in one field and to select the next day for input into another process. I am able to query the database and pull back a date, but I can't convert it into a date object.
Here is what I tried:
tmpMaxDate <- sqlQuery(dbhandle, 'select MAX([date]) + 1 from dbo.My_Data ')
tmpMaxDate
1 2016-01-02
IsDate(tmpMaxDate)
[1] FALSE
maxDate <- as.Date(tmpMaxDate)
Error in as.Date.default(tmpMaxDate) :
do not know how to convert 'tmpMaxDate' to class “Date”
maxDate
NULL
IsDate(maxDate)
[1] FALSE
maxDate <- as.Date(tmpMaxDate, format = "%Y-%M-%D")
Error in as.Date.default(tmpMaxDate, format = "%Y-%M-%D") :
do not know how to convert 'tmpMaxDate' to class “Date”
IsDate(maxDate)
[1] FALSE
maxDate
NULL
The packages I am using are RODBC, chron, lubridate, and RSQLserver
I thought I needed to convert to a date object to use the date from SQL Server in R.
Any thoughts on why my date won't convert? Do I need to convert to be able to use that date to filter data in R?
DateVar <- sqlQuery(dbhandle,"SELECT CONVERT(varchar(10),GETDATE() + 1,101) as Date")
DateVar
NewDate <- as.Date(DateVar$Date, format = "%m/%d/%y")
NewDate
I think the trick for me to was give the date response a column name and then reference the column (being the specific string value needed to convert) name in the conversion.
class(NewDate)
will show as "Date"
I have a bunch of question on MDX queries because i just started to learn about OLAP and SSRS report with SQL report builder.
First question is, could you please give me a link to the best sites that have a tutorial on MDX Queries..
Second, i already deploy a sales cubes.. What is the best way to pass parameter to the query? I mean how if i would like to query my sales for last 12 weeks automatically without using parameters (auto generate if now is in week 30 than it shows the data from week 18 to 30, and so on)..
here is my query that generated automatically from designer:
SELECT NON EMPTY { [Measures].[Total Stick] } ON COLUMNS,
NON EMPTY
{
( [PALAPA Location].[LocationCode].[LocationCode].AllMembers * [PALAPA Fact Sales].[Year].[Year].AllMembers * [PALAPA Fact Sales].[Week].[Week].AllMembers )
} Dimension Properties MEMBER_CAPTION,
MEMBER_UNIQUE_NAME ON ROWS
FROM (
SELECT (
{ [Time_Dim].[Week].&[2015-06-21T00:00:00], [Time_Dim].[Week].&[2015-06-28T00:00:00], [Time_Dim].[Week].&[2015-07-05T00:00:00], [Time_Dim].[Week].&[2015-07-12T00:00:00], [Time_Dim].[Week].&[2015-07-19T00:00:00] } ) ON COLUMNS
FROM (
SELECT ( { [Time_Dim].[Year].&[2015-01-01T00:00:00] } ) ON COLUMNS
FROM [PALAPA_DSV]
)
)
WHERE ( [Time_Dim].[Year].&[2015-01-01T00:00:00], [Time_Dim].[Week].CurrentMember ) CELL Properties Value,
BACK_COLOR,
FORE_COLOR,
FORMATTED_VALUE,
FORMAT_STRING,
FONT_NAME,
FONT_SIZE,
FONT_FLAGS
Third question, i would like to set the default values at SQL Report builder parameter (Week dimension) with this code :
=”[TP DIM CALENDAR].[Date].&[” + Format(CDate(Parameters!FromParameter.Value),”yyyy-MM-dd”) +
“T00:00:00]”
But why it didnt work when i run the report? The parameter value is blank..
Thanks for your help !
Please have a look at these mdx functions
StrToMember
https://msdn.microsoft.com/en-us/library/ms146022.aspx
StrToSet
https://msdn.microsoft.com/en-us/library/ms144782.aspx
These and several other StrTo.. functions are used pretty extensively for passing in parameters.
In your example you need to wrap the whole string in something like this:
strToMember(
"[TP DIM CALENDAR].[Date].&[" +
Format(CDate(Parameters!FromParameter.Value),"yyyy-MM-dd") +
"T00:00:00]"
)
Found the solution here :
http://www.msbitips.com/?p=4
Thanks for your help whytheq :)