Attributes

Instances of datetime.time and datetime.datetime classes will get a new attribute fold with two possible values: 0 and 1.

Constructors

The __new__ methods of the datetime.time and datetime.datetime classes will get a new keyword-only argument called fold with the default value 0. The value of the fold argument will be used to initialize the value of the fold attribute in the returned instance.

Methods

The replace() methods of the datetime.time and datetime.datetime classes will get a new keyword-only argument called fold. It will behave similarly to the other replace() arguments: if the fold argument is specified and given a value 0 or 1, the new instance returned by replace() will have its fold attribute set to that value. In CPython, any non-integer value of fold will raise a TypeError, but other implementations may allow the value None to behave the same as when fold is not given. 4 (This is a nod to the existing difference in treatment of None arguments in other positions of this method across Python implementations; it is not intended to leave the door open for future alternative interpretation of fold=None.) If the fold argument is not specified, the original value of the fold attribute is copied to the result.

4

PyPy and pure Python implementation distributed with CPython already allow None to mean “no change to existing attribute” for all other attributes in replace().

C-API

Access macros will be defined to extract the value of fold from PyDateTime_DateTime and PyDateTime_Time objects.

int PyDateTime_GET_FOLD(PyDateTime_DateTime *o)

Return the value of fold as a C int.

int PyDateTime_TIME_GET_FOLD(PyDateTime_Time *o)

Return the value of fold as a C int.

New constructors will be defined that will take an additional argument to specify the value of fold in the created instance:

PyObject* PyDateTime_FromDateAndTimeAndFold(
    int year, int month, int day, int hour, int minute,
    int second, int usecond, int fold)

Return a datetime.datetime object with the specified year, month, day, hour, minute, second, microsecond and fold.

PyObject* PyTime_FromTimeAndFold(
    int hour, int minute, int second, int usecond, int fold)

Return a datetime.time object with the specified hour, minute, second, microsecond and fold.

What time is it?

The datetime.now() method called without arguments will set fold=1 when returning the second of the two ambiguous times in a system local time fold. When called with a tzinfo argument, the value of the fold will be determined by the tzinfo.fromutc() implementation. When an instance of the datetime.timezone class (the stdlib’s fixed-offset tzinfo subclass, e.g. datetime.timezone.utc) is passed as tzinfo, the returned datetime instance will always have fold=0. The datetime.utcnow() method is unaffected.

Conversion from naive to aware

A new feature is proposed to facilitate conversion from naive datetime instances to aware.

The astimezone() method will now work for naive self. The system local timezone will be assumed in this case and the fold flag will be used to determine which local timezone is in effect in the ambiguous case.

For example, on a system set to US/Eastern timezone:

>>> dt = datetime(2014, 11, 2, 1, 30)
>>> dt.astimezone().strftime('%D %T %Z%z')
'11/02/14 01:30:00 EDT-0400'
>>> dt.replace(fold=1).astimezone().strftime('%D %T %Z%z')
'11/02/14 01:30:00 EST-0500'

An implication is that datetime.now(tz) is fully equivalent to datetime.now().astimezone(tz) (assuming tz is an instance of a post-PEP tzinfo implementation, i.e. one that correctly handles and sets fold).

Conversion from POSIX seconds from EPOCH

The fromtimestamp() static method of datetime.datetime will set the fold attribute appropriately in the returned object.

For example, on a system set to US/Eastern timezone:

>>> datetime.fromtimestamp(1414906200)
datetime.datetime(2014, 11, 2, 1, 30)
>>> datetime.fromtimestamp(1414906200 + 3600)
datetime.datetime(2014, 11, 2, 1, 30, fold=1)

Conversion to POSIX seconds from EPOCH

The timestamp() method of datetime.datetime will return different values for datetime.datetime instances that differ only by the value of their fold attribute if and only if these instances represent an ambiguous or a missing time.

When a datetime.datetime instance dt represents an ambiguous time, there are two values s0 and s1 such that:

datetime.fromtimestamp(s0) == datetime.fromtimestamp(s1) == dt

(This is because == disregards the value of fold – see below.)

In this case, dt.timestamp() will return the smaller of s0 and s1 values if dt.fold == 0 and the larger otherwise.

For example, on a system set to US/Eastern timezone:

>>> datetime(2014, 11, 2, 1, 30, fold=0).timestamp()
1414906200.0
>>> datetime(2014, 11, 2, 1, 30, fold=1).timestamp()
1414909800.0

When a datetime.datetime instance dt represents a missing time, there is no value s for which:

datetime.fromtimestamp(s) == dt

but we can form two “nice to know” values of s that differ by the size of the gap in seconds. One is the value of s that would correspond to dt in a timezone where the UTC offset is always the same as the offset right before the gap and the other is the similar value but in a timezone the UTC offset is always the same as the offset right after the gap.

The value returned by dt.timestamp() given a missing dt will be the greater of the two “nice to know” values if dt.fold == 0 and the smaller otherwise. (This is not a typo – it’s intentionally backwards from the rule for ambiguous times.)

For example, on a system set to US/Eastern timezone:

>>> datetime(2015, 3, 8, 2, 30, fold=0).timestamp()
1425799800.0
>>> datetime(2015, 3, 8, 2, 30, fold=1).timestamp()
1425796200.0

Aware datetime instances

Users of pre-PEP implementations of tzinfo will not see any changes in the behavior of their aware datetime instances. Two such instances that differ only by the value of the fold attribute will not be distinguishable by any means other than an explicit access to the fold value. (This is because these pre-PEP implementations are not using the fold attribute.)

On the other hand, if an object’s tzinfo is set to a fold-aware implementation, then in a fold or gap the value of fold will affect the result of several methods: utcoffset(), dst(), tzname(), astimezone(), strftime() (if the “%Z” or “%z” directive is used in the format specification), isoformat(), and timetuple().

Combining and splitting date and time

The datetime.datetime.combine() method will copy the value of the fold attribute to the resulting datetime.datetime instance.

The datetime.datetime.time() method will copy the value of the fold attribute to the resulting datetime.time instance.

Pickles

The value of the fold attribute will only be saved in pickles created with protocol version 4 (introduced in Python 3.4) or greater.

Pickle sizes for the datetime.datetime and datetime.time objects will not change. The fold value will be encoded in the first bit of the 3rd byte of the datetime.datetime pickle payload; and in the first bit of the 1st byte of the datetime.time payload. In the current implementation 8 these bytes are used to store the month (1-12) and hour (0-23) values and the first bit is always 0. We picked these bytes because they are the only bytes that are checked by the current unpickle code. Thus loading post-PEP fold=1 pickles in a pre-PEP Python will result in an exception rather than an instance with out of range components.

A non-technical answer

• Alice: Bob - let’s have a stargazing party at 01:30 AM tomorrow!
• Bob: Should I presume initially that Daylight Saving Time is or is not in effect for the specified time?
• Alice: Huh?

• Bob: Alice - let’s have a stargazing party at 01:30 AM tomorrow!
• Alice: You know, Bob, 01:30 AM will happen twice tomorrow. Which time do you have in mind?
• Bob: I did not think about it, but let’s pick the first.

(same characters, an hour later)

• Bob: Alice - this Py-O-Clock gadget of mine asks me to choose between fold=0 and fold=1 when I set it for tomorrow 01:30 AM. What should I do?
• Alice: I’ve never hear of a Py-O-Clock, but I guess fold=0 is the first 01:30 AM and fold=1 is the second.

A technical reason

While the tm_isdst field of the time.struct_time object can be used to disambiguate local times in the fold, the semantics of such disambiguation are completely different from the proposal in this PEP.

The main problem with the tm_isdst field is that it is impossible to know what value is appropriate for tm_isdst without knowing the details about the time zone that are only available to the tzinfo implementation. Thus while tm_isdst is useful in the output of methods such as time.localtime, it is cumbersome as an input of methods such as time.mktime.

If the programmer misspecified a non-negative value of tm_isdst to time.mktime, the result will be time that is 1 hour off and since there is rarely a way to know anything about DST before a call to time.mktime is made, the only sane choice is usually tm_isdst=-1.

Unlike tm_isdst, the proposed fold attribute has no effect on the interpretation of the datetime instance unless without that attribute two (or no) interpretations are possible.

Since it would be very confusing to have something called isdst that does not have the same semantics as tm_isdst, we need a different name. Moreover, the datetime.datetime class already has a method called dst() and if we called fold “isdst”, we would necessarily have situations when “isdst” is zero but dst() is not or the other way around.

Backward Compatibility

It has been suggested that backward compatibility can be improved if the default value of the fold flag was None which would signal that pre-PEP behavior is requested. Based on the analysis below, we believe that the proposed changes with the fold=0 default are sufficiently backward compatible.

This PEP provides only three ways for a program to discover that two otherwise identical datetime instances have different values of fold: (1) an explicit check of the fold attribute; (2) if the instances are naive - conversion to another timezone using the astimezone() method; and (3) conversion to float using the timestamp() method.

Since fold is a new attribute, the first option is not available to the existing programs. Note that option (2) only works for naive datetimes that happen to be in a fold or a gap in the system time zone. In all other cases, the value of fold will be ignored in the conversion unless the instances use a fold-aware tzinfo which would not be available in a pre-PEP program. Similarly, the astimezone() called on a naive instance will not be available in such program because astimezone() does not currently work with naive datetimes.

This leaves us with only one situation where an existing program can start producing different results after the implementation of this PEP: when a datetime.timestamp() method is called on a naive datetime instance that happen to be in the fold or the gap. In the current implementation, the result is undefined. Depending on the system mktime implementation, the programs can see different results or errors in those cases. With this PEP in place, the value of timestamp will be well-defined in those cases but will depend on the value of the fold flag. We consider the change in datetime.timestamp() method behavior a bug fix enabled by this PEP. The old behavior can still be emulated by the users who depend on it by writing time.mktime(dt.timetuple()) + 1e-6*dt.microsecond instead of dt.timestamp().

Analogy with tm_isdst

The time.mktime interface allows three values for the tm_isdst flag: -1, 0, and 1. As we explained above, -1 (asking mktime to determine whether DST is in effect for the given time from the rest of the fields) is the only choice that is useful in practice.

With the fold flag, however, datetime.timestamp() will return the same value as mktime with tm_isdst=-1 in 99.98% of the time for most time zones with DST transitions. Moreover, tm_isdst=-1-like behavior is specified regardless of the value of fold.

It is only in the 0.02% cases (2 hours per year) that the datetime.timestamp() and mktime with tm_isdst=-1 may disagree. However, even in this case, most of the mktime implementations will return the fold=0 or the fold=1 value even though relevant standards allow mktime to return -1 and set an error code in those cases.

In other words, tm_isdst=-1 behavior is not missing from this PEP. To the contrary, it is the only behavior provided in two different well-defined flavors. The behavior that is missing is when a given local hour is interpreted as a different local hour because of the misspecified tm_isdst.

For example, in the DST-observing time zones in the Northern hemisphere (where DST is in effect in June) one can get

>>> from time import mktime, localtime
>>> t = mktime((2015, 6, 1, 12, 0, 0, -1, -1, 0))
>>> localtime(t)[:]
(2015, 6, 1, 13, 0, 0, 0, 152, 1)

Note that 12:00 was interpreted as 13:00 by mktime. With the datetime.timestamp, datetime.fromtimestamp, it is currently guaranteed that

>>> t = datetime.datetime(2015, 6, 1, 12).timestamp()
>>> datetime.datetime.fromtimestamp(t)
datetime.datetime(2015, 6, 1, 12, 0)

This PEP extends the same guarantee to both values of fold:

>>> t = datetime.datetime(2015, 6, 1, 12, fold=0).timestamp()
>>> datetime.datetime.fromtimestamp(t)
datetime.datetime(2015, 6, 1, 12, 0)

>>> t = datetime.datetime(2015, 6, 1, 12, fold=1).timestamp()
>>> datetime.datetime.fromtimestamp(t)
datetime.datetime(2015, 6, 1, 12, 0)

Thus one of the suggested uses for fold=-1 – to match the legacy behavior – is not needed. Either choice of fold will match the old behavior except in the few cases where the old behavior was undefined.

Strict Invalid Time Checking

Another suggestion was to use fold=-1 or fold=None to indicate that the program truly has no means to deal with the folds and gaps and dt.utcoffset() should raise an error whenever dt represents an ambiguous or missing local time.

The main problem with this proposal, is that dt.utcoffset() is used internally in situations where raising an error is not an option: for example, in dictionary lookups or list/set membership checks. So strict gap/fold checking behavior would need to be controlled by a separate flag, say dt.utcoffset(raise_on_gap=True, raise_on_fold=False). However, this functionality can be easily implemented in user code:

def utcoffset(dt, raise_on_gap=True, raise_on_fold=False):
    u = dt.utcoffset()
    v = dt.replace(fold=not dt.fold).utcoffset()
    if u == v:
        return u
    if (u < v) == dt.fold:
        if raise_on_fold:
            raise AmbiguousTimeError
    else:
        if raise_on_gap:
            raise MissingTimeError
    return u

Moreover, raising an error in the problem cases is only one of many possible solutions. An interactive program can ask the user for additional input, while a server process may log a warning and take an appropriate default action. We cannot possibly provide functions for all possible user requirements, but this PEP provides the means to implement any desired behavior in a few lines of code.