I am using the Go runtime to store entities in the Appengine Datastore sequenced by the time they were added; to store the timestamp in the key I am formatting the timestamp using the time.Time.String() method and storing the string version as the key.
To get the Time back again on retrieval I use time.Parse:
time.Parse("2006-01-02 15:04:05.000000000 +0000 UTC", the_timestamp)
In unit testing this functionality independent of an app (using cmdline - goapp test) my tests retrieve the timestamp in its entirety no problem.
But when I import the package doing this into an appengine app and test it (using cmdline - goapp serve) the timestamp is stored with its Year field set to "0000"
When you are converting your time to string before saving into datastore, the extra 0s at the end of time are removed.
So,
2009-11-10 23:00:00.12300000 +0000 UTC
is converted into
2009-11-10 23:00:00.123 +0000 UTC
Now when you retrieve it from datastore and use the Parse function, it tries to match upto 8 digits after decimal. Hence the error.
So, while converting the time into string, you need to Format the string so that 0s are not lost.
const layout = "2006-01-02 15:04:05.000000000 +0000 UTC"
t := time.Now()
tFormat := t.Format(layout)
http://play.golang.org/p/elr28mfMo8
Related
We have our time saved without a timezone offset inside our SQL Server database. The time is actually Central Europe Standard Time but due to no timezone offset in the time string it is treated as UTC. This now creates a bunch of problems regarding daylight saving times.
My question would be: is there a way to retroactively convert the Time with offset to the correct CEST Time.
For example my time string in my database is '2022-10-30 02:00' and should be converted to '2022-10-30 00:00+2' as well as '2022-10-30 03:00' to '2022-10-30 01:00+1'.
There is an option to convert a datetime object to another timezone with "AT TIME ZONE" but this didn't help much due to the date objects being treated as UTC in the database, but we need to convert them to the UTC+[offset] format. Also due to the daylight saving time changing timezone offsets during the year, we can't subtract the timezone offsets with a set value.
For example, using a date and time control, the user selects a date and time, such that the string representation is the following:
"6-25-2012 12:00:00 PM"
It so happens that this user is in the EST time zone. The string is passed to the server, which translates it into a .NET DateTime object, and then stores it in SQL Server in a datetime column.
When the date is returned later to the browser, it needs to be converted back into a date, however when the above string is fed into a date it is losing 4 hours of time. I believe this is because when not specifying a timezone while creating a JavaScript date, it defaults to local time, and since EST is -400 from GMT, it subtracts 4 hours from 12pm, even though that 12pm was meant to be specified as EST when the user selected it on a machine in the EST time zone.
Clearly something needs to be added to the original datetime string before its passed to the server to be persisted. What is the recommended way of doing this?
Don't rely on JavaScript's Date constructor to parse a string. The behavior and supported formats vary wildly per browser and locale. Here are just some of the default behaviors if you use the Date object directly.
If you must come from a string, try using a standardized format such as ISO8601. The date you gave in that format would be "2012-06-25T12:00:00". The easiest way to work with these in JavaScript is with moment.js.
Also, be careful about what you are actually meaning to represent. Right now, you are passing a local date/time, saving a local/date/time, and returning a local date/time. Along the way, the idea of what is "local" could change.
In many cases, the date/time is intended to represent an exact moment in time. To make that work, you need to convert from the local time entered to UTC on the client. Send UTC to your server, and store it. Later, retrieve UTC and send it back to your client, process it as UTC and convert back to local time. You can do all of this easily with moment.js:
// I'll assume these are the inputs you have. Adjust accordingly.
var dateString = "6-25-2012";
var timeString = "12:00:00 PM";
// Construct a moment in the default local time zone, using a specific format.
var m = moment(dateString + " " + timeString, "M-D-YYYY h:mm:ss A");
// Get the value in UTC as an ISO8601 formatted string
var utc = m.toISOString(); // output: "2012-06-25T19:00:00.000Z"
On the server in .Net:
var dt = DateTime.Parse("2012-06-25T19:00:00.000Z", // from the input variable
CultureInfo.InvariantCulture, // recommended for ISO
DateTimeStyles.RoundtripKind) // honor the Z for UTC kind
Store that in the database. Later retrieve it and send it back:
// when you pull it from your database, set it to UTC kind
var dt = DateTime.SpecifyKind((DateTime)reader["yourfield"], DateTimeKind.Utc);
// send it back in ISO format:
var s = dt.ToString("o"); // "o" is the ISO8601 "round-trip" pattern.
Pass it back to the javascript in moment.js:
// construct a moment:
var m = moment("2012-06-25T19:00:00.000Z"); // use the value from the server
// display it in this user's local time zone, in whatever format you want
var s = m.format("LLL"); // "June 25 2012 12:00 PM"
// or if you need a Date object
var dt = m.toDate();
See - that was easy, and you didn't need to get into anything fancy with time zones.
Here, I think this is what you are looking for:
How to ignore user's time zone and force Date() use specific time zone
It seems to me that you can do something like this:
var date = new Date("6-25-2012 12:00:00 PM");
var offset = date.getTimezoneOffset(); // returns offset from GMT in minutes
// to convert the minutes to milliseconds
offset *= 60000;
// the js primitive value is unix time in milliseconds so this retrieves the
// unix time in milliseconds and adds our offset.
// Now we can put this all back in a date object
date = new Date(date.valueOf() + offset);
// to get back your sting you can maybe now do something like this:
var dateString = date.toLocaleString().replace(/\//g,'-').replace(',','');
Blame the JSON.Stringfy()... and do:
x = (your_date);
x.setHours(x.getHours() - x.getTimezoneOffset() / 60);
I am using a filter before sending the date to the server:
vm.dateFormat = 'yyyy-MM-dd';
dateToSendToServer = $filter('date')(dateFromTheJavaScript, vm.dateFormat);
We are using JavaMail API to send calendar entries. But the recipients of Outlook have time zone issues, as meetings show wrong timings. In general our approach is as follows:
First of all we have,
SimpleDateFormat iCalendarDateFormat = new SimpleDateFormat("yyyyMMdd'T'HHmmss");
we then use iCalendarDateFormat.setTimeZone(TimeZone.getTimeZone(receiverTimeZone));
Finally, we use Calendar.getInstance() for start and end to manipulate Calendar fields,
and hence we have Date startDate = startTime.getTime();
Date endDate = endTime.getTime();
When we are about to send request as per icalendar specification we have ,
"DTSTAMP:" + iCalendarDateFormat.format(startDate) + "\n" +
"DTSTART:" + iCalendarDateFormat.format(startDate)+ "\n" "DTEND:" + iCalendarDateFormat.format(endDate)+ "\n"
Is this the correct approach?. Please comment.
Thanks
tl;dr
iCalendar format tracks the date-time separately from its intended time zone. You must juggle both parts appropriately.
Always use java.time classes. Never use legacy classes like Calendar & SimpleDateFormat.
Details
Caveat: I have not used iCalendar data before. So I may be incorrect in my understanding.
Looking at pages 31-33 of the RFC 5545 spec, it seems the authors of that spec assume you always want the date-time to be recorded separately from the time zone.
A moment, a point on the timeline, needs the context of a time zone or offset-from-UTC. For example, "noon on the 23rd of January next year, 2021" is not a moment. We do not know if you mean noon in Tokyo Japan, noon in Toulouse France, or noon in Toledo Ohio US — all very different moments, several hours apart.
To provide the context of an offset, a date and time must be accompanied by a number of hours-minutes-seconds such as 08:00. For an offset of zero hours-minutes-seconds, use +00:00.
2021-01-23T12:00:00+00:00
As an abbreviation of an offset of zero, +00:00, the letter Z can be used, pronounced “Zulu”. For example:
2021-01-23T12:00:00Z
But, strangely, the iCalendar spec wants to track the date and the time-of-day separate from the time zone. So this:
2021-01-23T12:00:00
…and a time zone field elsewhere:
America/New_York
And the iCalendar spec opts for the harder-to-read “basic” variation allowed by ISO 8601, which minimizes the use of delimiters. So this:
20210123T120000
For such a string, we must parse as a LocalDateTime. This class represents a date with a time-of-day but lacking any time zone or offset-from-UTC.
DateTimeFormatter f = dateTimeFormatter.ofPattern( "uuuuMMdd'T'HHmmss" ) ;
String input = "20210123T120000" ; // “Basic” variation of ISO 8601 format.
LocalDateTime ldt = LocalDateTime.parse( input , f ) ;
To determine a moment, we must apply a time zone. I assume iCalendar uses proper time zone names (Continent/Region format) and not the 2-4 letter pseudo-zones such as PST, CST, IST, and so on.
String zoneName = receiverTimeZone ; // Variable name taken from your code example, though you neglected to show its origins.
ZoneId z = ZoneId.of( zoneName ) ;
Apply the zone to get a ZonedDateTime, a moment, a point on the timeline.
ZonedDateTime zdt = ldt.atZone( z ) ;
Going the other direction, let's start with the current moment.
ZoneId z = ZoneId.of( "Africa/Tunis" ) ;
ZonedDateTime now = ZonedDateTime.now( z ) ;
And generate string values for iCalendar.
DateTimeFormatter f = dateTimeFormatter.ofPattern( "uuuuMMdd'T'HHmmss" ) ;
String iCal_DateTime = now.format( f ) ;
String iCal_ZoneName = now.getZone().toString() ;
Never use the terrible legacy date-time classes bundled with the earliest versions of Java: Calendar, GregorianCalendar, java.util.Date, SimpleDateFormat, and so on. These were supplanted years ago by the modern java.time classes defined in JSR 310.
Hard to tell without seeing the actual content of your iCalendar file, along with the expected start and end datetime with timezone information but you seem to be generating the DTSTART in floating time (datetime with local time). Although your code sample seems to imply that you have access to the recipient's timezone (receiverTimezone), this is a very fragile approach.
Instead, you should use either the datetime with UTC time or the datetime with local time and timezone (where the timezone does not have to be the receiver timezone).
If the event is not recurring, the most simple approach is to use datetime with UTC time.
See https://www.rfc-editor.org/rfc/rfc5545#section-3.3.5 for the definition of each format.
I had same problem, for which I struggle lot. So below are my findings:
Outlook works smoothly with UTC Timezone. If we set date & time with UTC Timezone then outlook automatically converts this UTC Time into user corresponding Timezone. We will have to use 'Instant' object for DTSTART:, DTEND: and for DTSTAMP(Optional but recommended) also.
Quick Test just use "DTSTART:"+Instant.now() in ical String.
And in Java 8 for getting UTC Time java time API provides Instant.now() through which you can get your system time in UTC format. Java 8 also provides method like
a. Instant.ofEpochMilli() - This returns Instant which can directly use in ical Sting.
b. new Date().toInstant() Which returns UTC Instant object.
There are few scenarios where input date and time sources are different:
If you are fetching Date and Time from database then in this case database is not storing Timezone its only saving Date & Time. So first convert the Date & Time in that Timezone in which it was saved in database, in my case I was storing Date & Time after converting in 'EST' Timezone and Date value was of EST but time zone was not there in DB. So while fetching Date & Time value from DB I have appended Timezone in the Date value and then further converted to EPOC time using below method
public static long getEpocTimeWithTimezone(Date date) {
SimpleDateFormat simpleDateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
DateTimeFormatter dateTimePattern = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
LocalDateTime dateTime = LocalDateTime.parse(simpleDateFormat.format(date), dateTimePattern);
long epochInMilliSeconds = dateTime.atZone(ZoneId.of("America/New_York")).toEpochSecond() * 1000;
return epochInMilliSeconds;
}
Then Just Use as below code for ical String:
Instant startDt = Instant.ofEpochMilli(getEpocTimeWithTimezone(//pass your date here
)).truncatedTo(ChronoUnit.MINUTES);
Now set this Instant object(startDt) directly to "DTSTART:":
"DTSTART:"+startDt+"....then in same fashion "DTEND:" also.
In second scenario you have Date with Timezone (make sure after conversion you did not loses your actual Timezone, Like in 1st scenario after saving Date in DB we actually lost Timezone but it was showing Timezone IST that was dummy so be careful about this)
So in this case just assume myDateObject is Date object. So just get the Instant (which
will be in UTC) object from myDateObject by using toInstant() of Date class.
Instant startDt = myDateObject.toInstant().truncatedTo(ChronoUnit.MINUTES);
I am using .truncatedTo(ChronoUnit.MINUTES); because if we will not use this then
we might get some extra min or second in Meeting invite Time section.
So the final String for outlook mail should be some like:
.
.
.
"BEGIN:VEVENT\n"+
"DTSTART:"+startDt+"\n"+
"DTEND:"+endDt+"\n"+
.
.
.
VVI Note: Since Z is representation of UTC time zone, So just adding Z in the last of Time will not be UTC zoned Time, You will have to convert the Date & Time then only accurate time will come on Outlook. For verifying your Time is in UTC format or not just save the .ics attached file (which you got in Email) in local and check Date & Time are coming as DTSTART:2020-05-15T13:57:00Z or not If not then you are not converting the Date correctly in UTC.
I have an issue with getting dates back in the right timezone.
Firs at all in the local machine, everything works fine but not in the server: the server is hosted in USA and the clients mostly are in Australia.
So a date is sent from angular app("12/23/2015 11:00:00 AM") to the server, the server store a date as utc in the database, until this point everything is working(I checked and the date is stored in the right utc)
book.StartDateTime = TimeZoneInfo.ConvertTimeToUtc(DateTime.SpecifyKind(book.StartDateTime.Value, DateTimeKind.Unspecified), ToolsHelper.OlsonTimeZoneToTimeZoneInfo(locationDetails.TimeZone)); // book.CreatedDate.Value.ToUniversalTime();
The issue is:
When a client request some dates stored in the database. The date stored in the database is return back to the client like this:
bookview.StartDateTime = TimeZoneInfo.ConvertTimeFromUtc(DateTime.SpecifyKind(bookli.StartDateTime.Value, DateTimeKind.Utc), ToolsHelper.OlsonTimeZoneToTimeZoneInfo(deCompany.TimeZone));
I checked and at this point the date is "12/23/2015 11:00:00 AM" -> the conversion is right in the server(in the server side I put a log),
But in the angular is shown as "12/23/2015 10:00:00 PM"
So apparently the issue is when the date is transferred by the api to the client, maybe when is converted to JSON
I have tried different ways nothing work, I have removed "DateTime.SpecifyKind", I convert the date to string then back to datetime format and nothing seems to work.
What could I do?
A few things:
Your example is not complete, so I can only speculate on some areas. It would be better to show both sides, including how you load and parse the data in Angular, and what the data looks like over the wire.
You shouldn't be sending dates back and forth in a locale-specific format like "12/23/2015 11:00:00 AM". You might use those in your UI, but they're not appropriate over the wire (in your JSON). Instead, you should be using ISO8601/RFC3339, such as "2015-12-23T11:00:00Z". (You probably are already doing this if you're using WebAPI.)
A DateTime object when serialized to ISO8601 format is coupled with the associated DateTimeKind in the Kind property.
If the Kind is Utc, then the ISO8601 string will end with a Z.
If the Kind is Local, then the ISO8601 string will end with the machine-local offset for that timestamp, such as -08:00.
If the Kind is Unspecified, then the ISO8601 string will not have either Z or offset - which means that it cannot unambiguously represent a specific moment in time.
This is ultimately the cause of the error. You're converting the DateTime to another time zone, which leaves it with Unspecified kind, which then gets serialized without an offset - so on the client side that gets interpreted (probably) in the local time zone of the browser.
A better approach is to use DateTimeOffset instead of DateTime. Then you don't have to worry about Kind, and the offset is always present. If you change your bookview.StartDateTime to a DateTimeOffset type, you can do something like this to fix the problem:
DateTimeOffset dto = new DateTimeOffset(bookli.StartDateTime.Value, TimeSpan.Zero);
bookView.StartDTO = TimeZoneInfo.ConvertTime(dto, ToolsHelper.OlsonTimeZoneToTimeZoneInfo(deCompany.TimeZone));
That will ensure that the offset gets persisted in the data.
On the client side, pay careful attention to how the ISO string gets parsed. if it's loaded into a Date object, then it will indeed be converted to the client's time zone. Instead, you might take a look at moment.js for client-side time formatting. In particular, use moment.parseZone to keep the value in the same offset that it was presented. For example:
var s = moment.parseZone("2015-12-31T11:00:00+00:00").format("L LT"); // "12/31/2015 11:00 AM"
In commented code, you also showed a call to DateTime.ToUniversalTime - be very careful with that. If the source kind is Unspecified, it is treated as Local. Therefore, the local time zone of the computer is reflected in the converted value. It's better to avoid ToUniversalTime and ToLocalTime entirely. Use only the conversion methods on TimeZoneInfo.
ToolsHelper.OlsonTimeZoneToTimeZoneInfo isn't something we know about either. But I'll assume it does a CLDR mapping similar to this one. However, if you're working with Olson time zones anyway, the better approach would be to not use TimeZoneInfo at all. Instead, use Noda Time, with it's native support for tzdb time zones.
Here's what I'm trying to do: the user submits a time in pacific, once submitted I use .replace to set the timezone to Pacific.
Pacific = time.USTimeZone(-8, "Pacific", "PST", "PDT")
addEvent.date = addEvent.date.replace(tzinfo=Pacific)
Once i've set the tzinfo, I'm doing a put. According to the python documentation of google appengine it says:
"If the datetime value has a tzinfo attribute, it will be converted to the UTC time zone for storage. Values come back from the datastore as UTC, with a tzinfo of None. An application that needs date and time values to be in a particular time zone must set tzinfo correctly when updating the value, and convert values to the timezone when accessing the value."
However, when I do a put(), i get the following error:
WARNING 2012-10-06 21:10:14,579 tasklets.py:399] initial generator _put_tasklet(context.py:264) raised NotImplementedError(DatetimeProperty date can only support UTC. Please derive a new Property to support alternative timezones.)
WARNING 2012-10-06 21:10:14,579 tasklets.py:399] suspended generator put(context.py:703) raised NotImplementedError(DatetimeProperty date can only support UTC. Please derive a new Property to support alternative timezones.)
Please note I am using NDB
Ok, so after doing that I assumed that maybe NDB doesn't automatically convert it into UTC. So then I tried to convert it to UTC using the following code:
class UTC(tzinfo):
def utcoffset(self, dt):
return timedelta(0)
def tzname(self, dt):
return str("UTC")
def dst(self, dt):
return timedelta(0)
and now I still get the same error even after I convert the pacific time to UTC and set the tzinfo name as "UTC".
Could really use a ton of help here...
thanks!
The solution is to remove the tzinfo completely from the time after converting to UTC.
timestamp = timestamp.replace(tzinfo=None)
Here my working example:
if my_date.utcoffset():
my_date = (my_date - my_date.utcoffset()).replace(tzinfo=None)
I ran into this using http://hnrss.org/. This was my solution.
import datetime
from dateutil import parser
your_date_string = 'Mon, 24 Oct 2016 16:49:47 +0000'
your_date = parser.parse(your_date_string)
# your_date is now a datetime.datetime object
your_date_minus_tz = your_date.replace(tzinfo=None)
Now try your put() and you should see 2016-10-24 16:49:47 in the Cloud datastore.