Related
I am trying to figure out how to get the last day of the month given with a SSIS Expression.
I receive the column as an Integer, then added a Data Conversion step to convert the YYYMM to a date format.
Then on derived column, my expression is:
REPLACE((DT_WSTR,10)(DT_DBDATE)DATEADD("D",-(DAY(DATEADD("M",1,[Copy of MONTH]))),DATEADD("M",1,[Copy of MONTH])),"-","")
And on final table I am getting the value 24520504.
Example: I am getting the value for the derived column from an .xls file.
On that file, I use the column MONTH and it is an Integer (example: 201711).
On the final table it is an Integer too, but it has the last day from 201711 (example: 20171131)
EDIT: After your help, I reformulate my expression but I still get an error:
REPLACE((DT_STR,10,1252)DATEADD("d",-1,DATEADD("m",1,(DT_DBDATE)(DT_STR,6,1252)[MONTH])),"-","")
Assuming that this column is being input to the Derived column as a DT_DBDATE date type, the following expression will get the last day of the month. This expression essentially parses out the year and month first, uses these to build a new date by adding one month to this date (now in the next month), and then subtracts one day, which is the last DATEADD functions role, and then returns the date back to the same month, but now on the last day. Finally, the outer DAY function returns the day.
DAY(DATEADD("day",-1,DATEADD("month",1,(DT_DBDATE)((DT_STR,4,1252)YEAR((DT_DBDATE)YourDateColumn) + "-" + RIGHT((DT_STR,2,1252)MONTH((DT_DBDATE)YourDateColumn),2) + "-01"))))
Update:
For the DT_R8 data type that stores the date using the YYYYDD format, use the following expression to return the last day of the month.
DAY(DATEADD("day",-1,DATEADD("month",1,(DT_DBDATE)SUBSTRING((DT_STR,12,1252)LEFT((DT_STR,8,1252)YourDateColumn,4) + "-" + SUBSTRING((DT_STR,8,1252)YourDateColumn,5,2) + "-01",1,12))))
where are you starting (this will return last date of month in date format)?
in SQL:
declare #YYYYMM as int = 201811
select dateadd(d,-1,dateadd(m,1,cast(cast(#YYYYMM as varchar(6))+'01' as date)))
This will take it all the way to int:
declare #YYYYMM as int = 201811
select cast(replace(cast(dateadd(d,-1,dateadd(m,1,cast(cast(#YYYYMM as varchar(6))+'01' as date))) as varchar(10)),'-','') as int)
In SSIS:
it's similar. I don't have a way to confirm it right now though. I'll edit later.
this should work...
periodo ='201803'
DATEADD("day",-1,DATEADD("month",1,(DT_DBDATE)((DT_STR,4,1252)LEFT(period,4) + "-" + RIGHT(period,2) + "-01")))
I received timestamp datasets which are in the format of yyyy-mm-dd HH:MM:SS.ms. I want to convert into yyyy-mm-dd HH:MM:SS format. Is there any way to select only in this format using matlab?
For example:
2012-08-01 00:10:00.0
should be:
2012-08-01 00:10:00
Please note that the millisecond values are all zero.
The general way would be to use datestr to convert it to your desired format.
dates = {'2012-08-01 00:10:00.1';
'2012-08-01 00:10:00.1'};
new = datestr(dates, 'yyyy-mm-dd HH:MM:SS');
% 2012-08-01 00:10:00
% 2012-08-01 00:10:00
Another approach would be that since all of your milliseconds are going to be zero (therefore you don't have to worry about rounding) you can just use a regular expression to remove the milliseconds component (anything after the decimal point)
new = regexprep(dates, '\..*', '')
This is likely going to be more performant as you don't need to perform the intermediate step of converting to either a datetime object or a date number.
Since the input and output format are the same except for the milliseconds, don't use date functions, but simple string operations:
% example dates
C = {'2012-08-01 00:10:00.0'
'2013-08-02 00:11:11.0'
'2014-08-03 00:12:22.0'
'2015-08-04 00:13:33.0'
'2016-08-05 00:14:44.0'};
% method 1
D = cellfun(#(x)x(1:end-2), C, 'UniformOutput', false);
% method 2 (same, but no cellfun)
D = char(C);
D = cellstr(D(:,1:end-2));
% method 3
D = regexp(C, '[^\.]*', 'match', 'once');
% method 4
D = regexprep(C, '\..*$', '');
Lets say you need this data in datetime objects anyway then i would do something like this:
inp = {'2012-08-01 00:10:00.0'; '2012-08-02 04:10:00.0'}; % Some datestrins
t = datetime(inp,'InputFormat','yyyy-MM-dd HH:mm:ss.0'); % Convert to datetimes
datestr(t, 'yyyy-mm-dd HH:MM:SS') % convert back to strings
For the input & output formatter see the documentation. I assume that the last part is always zero.
I am tryinng to loop over a series of dates in order to create the dates inbetween. This is to be done in steps of month, always displaying the last day of the respective month. The start and end dates are given (first_date and last_date), while the last_date should always refer to the end of the previous month.
The original dataset looks like the following:
customer id first_date last_date
xy 135 01.01.2000 25.03.2005
xy 247 19.03.2003 25.03.2005
ab 387 01.06.2010 30.12.2012
ab 128 01.05.2010 28.02.2011
...
My goal is to have a dataset which looks like this:
customer id date
xy 135 31.01.2000
xy 135 28.02.2000
...
xy 135 28.02.2005
xy 247 31.03.2003
xy 247 30.04.2003
...
xy 247 28.02.2005
I found the solution to iterate over days quite straightforward (see below), but I am struggling to implement the monthly steps and the end of month dates.
data want;
set have;
by customer id;
do day = first_date to last_date;
output;
end;
format day date9.;
run;
Thanks for your help!!
First, lets get some data:
data have;
attrib customer length=$10 informat=$10.
id informat=best.
first_date informat=ddmmyy10. format=ddmmyy10.
last_date informat=ddmmyy10. format=ddmmyy10.
;
input customer $
id
first_date
last_date
;
datalines;
xy 135 01.01.2000 25.03.2005
xy 247 19.03.2003 25.03.2005
ab 387 01.06.2010 30.12.2012
ab 128 01.05.2010 28.02.2011
;
run;
The intnx() function will come to the rescue here. We are going to create a new variable called date, and then use the intnx function to return the end of the month for that date. As long as that date is less than the end date, we will continue to output it to a dataset and then increment to the end of the following month.
data want;
format date ddmmyy10.;
set have;
date = intnx('month',first_date,0,'end');
do while (date le last_date);
output;
date = intnx('month',date,1,'end');
end;
drop first_date last_date;
run;
While I think Rob's answer is the right way to do this, it's probably helpful to see how to do it the way you were trying to.
Starting with this:
data want;
set have;
by customer id;
do day = first_date to last_date;
output;
end;
format day date9.;
run;
This gives you too many rows, right? So what you need to do is identify where in the month you are. There are a bunch of ways to do this. Several date functions (like INTNX and INTCK) could be used to tell you where you are; but the easiest is just to compare month(date) with month(date+1). When they're different, you're on the last day of a month!
data want;
set have;
by customer id notsorted;
do day = first_date to last_date;
if month(day) ne month(day+1) then output;
end;
format day date9.;
run;
(I added notsorted since Rob's example data was not sorted, and I'm lazy. Probably not needed in your real case.)
I would note that this probably isn't your ideal solution - Rob's is probably that, in terms of data steps - in terms of speed. This of course will iterate through every day rather than just once per month.
Another option if you have the dataset you created above - with one row per day - is to use PROC EXPAND, if you have the ETS module. It's very handy for things like this.
data intermediate;
set have;
by customer id notsorted;
do day = first_date to last_date;
output;
end;
format day date9.;
run;;;
Here's your day-level data. Then below is the PROC EXPAND statement, asking for monthly data, aligned at the end. id day; identifies the time series variable, and by customer id notsorted; is the normal by statement (what variables identify the observations), with notsorted so they don't have to be in order relative to each other.
proc expand data=intermediate out=want from=day to=month align=end;
id day;
by customer id notsorted;
run;
This gives a slightly different solution than Rob's and my other solution, because it does give you the final row for each if it's not at the end of a month (and does set that final row to the end of the month). If that's desired, great, and our solutions can easily be adapted to give that; if it's not desired, you'll have to remove it afterwards.
You can do this with a simple iterative DO loop by using the date interval functions. Subtract one from the number of intervals to make it end at the last day of the previous month.
data want ;
set have ;
do offset=0 to intck('month',first_date,last_date)-1;
date=intnx('month',first_date,offset,'e');
output;
end;
format date yymmdd10.;
run;
I need an expression which will convert a text (VARCHAR) column to a DATETIME if, and only if, it matches dd/MM/yyyy, d/MM/yyyy, dd/M/yyyy or d/M/yyyy. If it doesn't match then I want a NULL.
I have this...
CASE ISDATE([DateField])
WHEN 1 THEN CONVERT(DATETIME,[DateField],103)
ELSE NULL
END
However this fails for '15/04/76' for example - with a "Conversion failed when converting datetime from character string" error - whereas I would want it to return NULL
Example output
'1/6/1976' -> 1976-06-01
'01/06/1976' -> 1976-06-01
'13/06/2001' -> 2001-06-13
'06/13/2001' -> NULL
'13/06/76' -> NULL
Is there a way of forcing ISDATE to validate a given format?
The documentation seems to suggest so...
ISDATE is deterministic only if used with the CONVERT function, the
CONVERT style parameter is specified and style is not equal to 0, 100,
9, or 109.
But ISDATE only takes one argument, so how do I "use it with CONVERT function" if I am not doing so already?
You could do a nested case statement here. The first could check to see if you have a 10 character string 2 for day, 2 for month, 4 for year and 2 for separators = 10 characters.
SET DATEFORMAT DMY;
Case When DateField Like '%/%/[0-9][0-9][0-9][0-9]'
Then Case When IsDate(DateField) = 1
Then CONVERT(DATETIME,[DateField],103)
End
End
Revised: I changed the code to use a like search which forces there to be a /YYYY at the end of the string, and then does an IsDate check to allow for a single day and/or month.
Well, first off, why on earth are you storing datetime values in a varchar column? This is a cardinal sin for a variety of reasons, not the least of which is that you get no validation whatsoever that the data is (or is convertible to) a datetime. You should also consider validating the input, even if you leave the column as varchar, so you don't have such a wide variety of potential formats that you want to consider valid.
So here is one way, borrowing a bit from #G Mastros:
DECLARE #f TABLE(i INT, d VARCHAR(32));
INSERT #f VALUES
(1,'15/04/76'),
(2,'15/04/1976'),
(3,'1/3/1976'),
(4,'1/3/76'),
(5,'15/3/1976'),
(6,'22/22/22'),
(7,'Yesterday');
SET DATEFORMAT DMY;
SELECT i, d, d2 = CASE WHEN ISDATE(d) = 1
AND d LIKE '%/[0-9][0-9][0-9][0-9]'
THEN CONVERT(DATETIME, d, 103) END
FROM #f;
Results:
i d d2
- ---------- -----------------------
1 15/04/76 NULL
2 15/04/1976 1976-04-15 00:00:00.000
3 1/3/1976 1976-03-01 00:00:00.000
4 1/3/76 NULL
5 15/3/1976 1976-03-15 00:00:00.000
6 22/22/22 NULL
7 Yesterday NULL
PS this will be a great case for TRY_CONVERT in SQL Server 2012. It does exactly what you're asking - it tries to convert to the specified data type; if it can't, it returns NULL.
Thanks for the responses folks.
I've done it like this...
CASE ISDATE([DateField]) WHEN 1 THEN
CASE WHEN SUBSTRING([DateField],LEN([DateField])-4,1) = '/' THEN
CASE WHEN CHARINDEX('/',[DateField],LEN([DateField])-3)=0 THEN
CONVERT(datetime, [DateField] , 103)
END
END
END
which is pretty nasty business so would still appreciate something neater!
But this doesn't work either - it still errors on mm/dd/yyyy format dates!
Scrap that last comment - it does seem to work now? Probably something to do with SET DATEFORMAT
I have struct as:
struct stored
{
char *dates; // 12/May/2010, 10/Jun/2010 etc..
};
// const
struct stored structs[] = {{"12/May/2010"}, {"12/May/2011"},
{"21/May/2009"}, {"13/May/2011"},
{"10/May/2011"}, {"19/May/2011"}};
What I want to do is to sort struct 'stored' by stored.dates.
qsort(structs, 9, sizeof(struct stored*), sortdates); // sortdates function
I'm not quite sure what would be a good way to sort those days? Compare them as c-strings?
I would convert the dates to numbers using something like:
year * 10000 + month * 100 + day;
and then do a simple numeric comparison (and for month, you'll need to map from Jan to 1, Feb to 2, etc.).
If you're doing a lot of comparisons, you may want to cache the numeric equivalent in the structure.
If you convert the dates to the format YYYYMMDD (as in 20100314), you can compare them as a string or as an integer (after conversion).
ISO 8601 formatted dates ("YYYYMMDD" or "YYYY-MM-DD" etc.) are trivially comparable as C strings. Your format is not - would changing the format of the date strings be an option?
PS: If you get rid of the "-", you could even store the date as plain 32bit integer. Depending on what your application does with those dates, that might be an additional bonus.
You can't compare these as strings, but you can compare substrings. Compare the years, and if they aren't equal you have your answer. Next compare the months, you'll need some kind of table to order the months by name. Finally if the months are the same, compare the days.