Local time is set to UTC

Hi There,
I have a Raspberry Pi Pico W and I have it connected to my wifi network no problem. First thing I want to do is to set the time to the correct local time.
I found an example that uses these commands

        ntptime.settime()
        time.localtime(ntptime.time())

This seems to work sometimes but fail others which seems to be some kind of timing issue. I put a sleep() in there and a try / except that loops a few times if it fails so I seem to have sorted that bit out but it sets the time to UTC.

I just commented out the second line (time.localtime) and it didn’t make any difference at all.
If I comment out both lines Thony sets the local time correctly if I disconnect and reconnect to the device.

My searchers are going around in circles and I’m not making any progress. I don’t care about making changes for Daylight Savings time.
I only want to log the time of day that events occurred so it would be adequate for me to simply add 10 hours to the time as my timezone is east coast of Australia.

When I read the micropython doc here
It describes the command like this
ntptime.settime(timezone=8 , server = 'ntp.ntsc.ac.cn')
When I try and use this command I receive the following error
unexpected keyword argument 'timezone’
Do I need to do something silly like add 10 hours of seconds to the epoch time somehow?
Can someone give me an idea where I’m going wrong?
Thanks
David

4 Likes

Hey David,

This is probably not the solution, but with python, I always like to start out checking for extra tabs and whitespaces before looking into anything else. I’ve been tripped up many times looking for errors that are practically invisible.

Blayden

3 Likes

Some Linux guru should step in here, but as I understand it a user has a an offset to UTC, but Linux timestamps everything in UTC and converts it to local time when a user accesses it. So you need to set up a timezone for your account, then set the machine time to UTC. NTP knows nothing of local time zones (I believe) so always deals in UTC.

3 Likes

Hi David,

I had a dig through the Micropython git and couldn’t find much documentation for the ntptime module - it seems to have been deprecated but left inside current builds.

To get your project back on track you can query an NTP server directly:

Adapting the code slightly from this tutorial: Raspberry Pi Pico W | Connecting to the Internet - Tutorial Australia

We can generate an integer list using the following code:

import network
import requests

... Connect to your WiFi ...

print("\n\n2. Querying the current GMT+0 time:")
r = urequests.get("http://date.jsontest.com") # Server that returns the current GMT+0 time.
#print(r.json())
time_lst = [int(i) for i in r.json()['time'].split(':')[:2]]

print(time_lst)
5 Likes

Hi Alan,
Thanks for responding. This is a Raspberry Pi pico project not a regular Raspberry Pi.
A Micropython interpreter is installed on the board and Micropython programs any simply executed. So no users or Linux. They are interesting little things to play around with. It can alternately run C/C++ programs but they need to be pre-compiled.
David

2 Likes

Hi Blayden,
That’s worth a look. I also doubt it will solve it but stranger things have happened. :slight_smile:
I might make a bare bones example to demonstrate the problem.
Thanks
David

3 Likes

OK, starting from zero knowledge of this is difficult. The documentation is a bit chaotic but I did see a clear statement - micropython has no support for time zones. It appears the Pi Pico micropython is a port from different hardware (ESP32?) but even that is not clear.
I looked at https://www.engineersgarage.com/micropython-esp8266-esp32-rtc-utc-local-time/ which states " ntptime.settime() : used to synchronize the local time maintained by the RTC or an internal clock source to the UTC standard time. Some port-specific MicroPython ntptime modules let the settime() method take two arguments, timezone and server. The ntptime module of the official MicroPython firmware does not accept any arguments. The epoch time for the UTC standard is January 1, 1970 00:00:00. The host server used by the MicroPython ntptime module is “pool.ntp.org”

So it seems the way to go is get the UTC time with ntptime.time(), figure out what offset is required and set the local time with machine.RTC (to set the real time clock - I assume there is one)

Reaching an ntp server is not guaranteed, they could be busy, network delays too long, etc. So that may be why it works sometimes and fails other times.

2 Likes

Hi Alan,
I have come to roughly the same conclusions as you re ntptime.time(). Your reference to documentation chaos is probably also accurate. It’s hard to know if I’m reading documentations that actually applies to the hardware I have.
Also I think the term Real Time Clock is used fairly loosely.
My understanding is the pico has a clock built in but it is not battery backed so when the power is turned off the time setting is lost.
The IDE “Thonny” automatically sets the internal clock when the system is powered on which can give the illusion the time setting is permanent.

ntptime.settime() retrieves the time and sets the internal clock.

I do have a RTC Module “Pico-RTC-DS3231” that I wish to use in my project. My project uses two Pico’s that communicate with each other via LORA. The remote unit has the RTC-DS3231 and sets it’s internal clock from that. The local unit has wifi and can use ntp. If I want to compare logs down the track the time stamps will need to roughly match which is why I’m heading down this rabbit hole :slight_smile:

It’s all a learning experience for me. :slight_smile:

I think all I really need to figure out now is how to add 10 hours to the time retrieved by the ntptime.settime() command and store that.
Or more likely use ntptime.settime() to set the clock then update the clock by adding 10 hours to the time that has been set.
I’ll see what I can figure out from machine.RTC

Here is my test code so far in case anyone is interested.

secret.py

ssid = 'Your SSID'
password = 'Your Password'

main.py

# A simple example that:
# - Connects to a WiFi Network defined by "ssid" and "password" defined in secrets.py
# - Sets the system time from the internet using ntp

# - Performs a GET request (loads a webpage)
# - Queries the current time from a server

import network   # handles connecting to WiFi
import urequests # handles making and servicing network requests
import secrets   # file containing ssud='' and password=''
import time
import ntptime
import debug
import errno
import formatDateTime

# Debug
# 0=off
# 1=Low
# 2=Med
# 3=Rediculous
DEBUG = 1
LOGTOFILE = True

debugCounter = 0    # Continuous global counter of debug log entries

# Connect to network
# Fill in your network name ssid and password in secrets.py
if(DEBUG >=1):
    debugCounter = debug.debug(DEBUG, debugCounter, "main()    ", "Connect to WIFI", LOGTOFILE)

wlan = network.WLAN(network.STA_IF)
wlan.active(True)

try:
    wlan.connect(secrets.ssid, secrets.password)
    time.sleep(5)
except Exception as errorMsg:
    debugCounter = debug.debug("ERROR", debugCounter, "main()    ", "Connect to Network: Error=" + str(errorMsg), LOGTOFILE)

# Set the date time from the network.
if(DEBUG >=1):
    timeStamp=str(formatDateTime.formattedTime())
    debugCounter = debug.debug(DEBUG, debugCounter, "main()    ", "Time before update=" + str(timeStamp), LOGTOFILE)

attempts=0
while attempts <3:
    try:
        ntptime.settime()
        break
    except Exception as errorMsg:
        attempts=attempts+1
        debugCounter = debug.debug("ERROR", debugCounter, "main()    ", "Set Time from Network: Attempt="+ str(attempts) +"Error=" + str(errorMsg))

if(DEBUG >=1):
    # print("Local time after synchronization:%s" %str(time.localtime()))
    timeStame=str(formatDateTime.formattedTime())
    debugCounter = debug.debug(DEBUG, debugCounter, "mail()    ", "Time after update=" + str(timeStamp), LOGTOFILE)

# Example 1. Make a GET request for google.com and print HTML
# Print the html content from google.com
# print("1. Querying google.com:")
# r = urequests.get("http://www.google.com")
# print(r.content)
# r.close()

# Example 2. urequests can also handle basic json support! Let's get the current time from a server
# print("\n\n2. Querying the current GMT+0 time:")
# r = urequests.get("http://date.jsontest.com") # Server that returns the current GMT+0 time.
# print(r.json())

debug.py

import time

def debug(debugLevel, debugCounter, methodName, details, logToFile):
    if(debugCounter==0):
        logText="Count\tLevel\tMethodName\tDetails"
        print(logText)
        if(logToFile):
            filelog(logText)
        logText="=================================================================================="
        print(logText)
        if(logToFile):
            filelog(logText)
            
    debugCounter = debugCounter + 1
    logText=str(debugCounter) + "\t" + str(debugLevel) + "\t" + methodName + "\t" + str(details)
    print(logText)
    if(logToFile):
        filelog(logText)
    return debugCounter

def filelog(logtext):
    file=open("datalog.csv","a")           # creation and opening of a CSV file in Write mode
    file.write((str(time.ticks_ms())+"~")+str(logtext)+"\n")          # Writing data in the opened file
    file.close()                           # The file is closed

formatDateTime.py

import time

def formattedTime():
    now=time.localtime()
    #timeString=print("{}:{}".format(now[3], now[4]))
    timeString=str(now[3]) + ":" + str(now[4]) + ":" + str(now[5])
    return timeString

def formattedDate():
    now=time.localtime()
    #dateString=("{}/{}/{}".format(now[2], now[1], now[0]))
    dateString=str(now[2]) + "/" + str(now[1]), + "/" + str(now[0])
    return dateString
2 Likes

Ttransfer time from one to the other. That way they will agree even if not the right time. If the RTC was on the node with Wifi and set occasionally from ntp then all would be sweet. RTC in computers will run normally run weeks with only seconds deviation, it looks like the DS3231 is similar. Or don’t bother with ntp at all, just check the time every now and then and set the RTC manually. Depends on how accurate the log needs to be.

1 Like

Hi Alan,
I think the DS3213 is surprisingly accurate. It even had a built in thermometer to allow for variations in temperature. Not that it matters all that much. An interesting stand alone solution would be to sync one clock from the DS3213 then send the time across to the other via Lora. Not of it really matters that much. It’s just interesting to get things right.

1 Like

I wrote the below code to set the Pico W RTC from the ntp server. All the work is done in the function set_time(). I wrote it as a function so that it could get called every 24 hours to reset the time. It gives UTC. If you want a time zone change the value of NTP_DELTA. The print(rtc.datetime()) should give say (2022, 12, 11, 6, 3, 26, 17, 0)

from time import sleep
import machine
import network
import socket
import struct
import time
import secrets

rp2.country('AU')

def set_time(): #Set RTC
    NTP_QUERY = bytearray(48)
    NTP_QUERY[0] = 0x1B
    addr = socket.getaddrinfo("pool.ntp.org", 123)[0][-1]
    s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    try:
        s.settimeout(1)
        res = s.sendto(NTP_QUERY, addr)
        msg = s.recv(48)
    finally:
        s.close()
    val = struct.unpack("!I", msg[40:44])[0]
    t = val - NTP_DELTA    
    tm = time.gmtime(t)
    machine.RTC().datetime((tm[0], tm[1], tm[2], tm[6] + 1, tm[3], tm[4], tm[5], 0))

wlan = network.WLAN(network.STA_IF)
wlan.active(True)
wlan.connect(secrets.ssid, secrets.password)

# Wait for connect or fail
max_wait = 10
while max_wait > 0:
    if wlan.status() < 0 or wlan.status() >= 3:
        break
    max_wait -= 1
    print('waiting for connection...')
    sleep(1)

# Handle connection error
if wlan.status() != 3:
    raise RuntimeError('network connection failed')
else:
    print('connected')
    status = wlan.ifconfig()
    print( 'ip = ' + status[0] )

NTP_DELTA = 2208988800 #UTC
rtc = machine.RTC()
set_time()
print(rtc.datetime())
2 Likes

Hi Fractal,
I have also been working on this problem.
I have come up with a very similar solution following an example I found on line. Perhaps our solutions have a similar “heritage” :wink:
I’m just completing testing and plan to post it here soon.
Thanks for your input. If my testing fails I know what I’ll be testing next!
David

1 Like

Hi Guys,

I have my Pico W network date update code working consistently now. It includes a Time Zone offset and ignores day light saving.

I have found that this process can make more sense if you turn off time sync within Thonny. Details are in the main program comments.

I have left my debugging code in. Just set the debug level to zero to disable it. I usually do my debugging this way as it is sometimes useful to turn it back on down the track. There is also a debug to file option when running without a console.
I have included several methods in separate files as they will be useful to include in other programs.
I notice when I execute the program directly on the Pico as main.py, if I inspect the log file (datalog.scv) the initial date is 01/01/2021. I have not see this date mentioned in any discussion posts anywhere.
Even with the Thonny date time settings disabled I still get different result when running within Thonny.
Anyway, the code below is now working and will be useful in my project as will the reusable debugging and date formatting code.
Hope this helps someone else.

main.py

# A simple example that:
# - Connects to a WiFi Network defined by "ssid" and "password" in secrets.py
# - Sets the system time from the internet using ntp protocol.

# - Optional
# - Performs a GET request (loads a webpage)
# - Queries the current time from a server

# Thonny automatically sets the device clock every time the the program is executed.
# To avoid total confusion during testing change the following settings to false
# From the menut - Tools -> Open Thonny Data Folder
# Edit the Configuration.ini
# Change these two settings

# sync_time = True
# local_rtc = True

# Found another example here
# https://gist.github.com/aallan/581ecf4dc92cd53e3a415b7c33a1147c
# That has been modified and saved into a module ntpClient.py
# It accepts a time zone offset and saves the time to the system clock.

import network   # handles connecting to WiFi
import urequests # handles making and servicing network requests
import secrets   # file containing ssud='' and password=''
import time
import ntpClientTZ
import machine

import debug
import errno
import formatDateTime

#import socket
#import struct
#import sys

# Debug
# 0=off
# 1=Low
# 2=Med
# 3=Rediculous
DEBUG = 3
LOGTOFILE = True
TIMEZONE = 10
NtpServer = "au.pool.ntp.org"
debugCounter = 0    # Continuous global counter of debug log entries

# Connect to network
# Fill in your network name ssid and password in secrets.py
if(DEBUG >=1):
    debugCounter = debug.debug(1, debugCounter, "main()    ", "Connect to WIFI", LOGTOFILE)

wlan = network.WLAN(network.STA_IF)
wlan.active(True)

try:
    wlan.connect(secrets.ssid, secrets.password)
    time.sleep(5)
except Exception as errorMsg:
    debugCounter = debug.debug("ERROR", debugCounter, "main()    ", "Connect to Network: Error=" + str(errorMsg))

# Set the date time from the network.
if(DEBUG >=1):
    timeStamp=str(formatDateTime.formattedTime())
    debugCounter = debug.debug(1, debugCounter, "main()    ", "Time before update=" + str(timeStamp), LOGTOFILE)
    dateStamp=str(formatDateTime.formattedDate())
    debugCounter = debug.debug(1, debugCounter, "main()    ", "Date before update=" + str(dateStamp), LOGTOFILE)
attempts=1
while attempts <=3:
    try:
        if(DEBUG >=1):
            debugCounter = debug.debug(1, debugCounter, "main()    ", "Update Attempt=" + str(attempts), LOGTOFILE)

        ntpClientTZ.setTime(TIMEZONE, NtpServer, DEBUG, LOGTOFILE)
        break
    except Exception as errorMsg:
        attempts=attempts+1
        debugCounter = debug.debug("ERROR", debugCounter, "main()    ", "Set Time from Network: Attempt="+ str(attempts) +"Error=" + str(errorMsg), LOGTOFILE)

if(DEBUG >=1):
    #print("Local time after synchronization:%s" %str(time.localtime()))
    timeStamp=str(formatDateTime.formattedTime())
    debugCounter = debug.debug(1, debugCounter, "main()    ", "Time after update=" + str(timeStamp), LOGTOFILE)
    dateStamp=str(formatDateTime.formattedDate())
    debugCounter = debug.debug(1, debugCounter, "main()    ", "Date after update=" + str(dateStamp), LOGTOFILE)


# Example 1. Make a GET request for google.com and print HTML
# Print the html content from google.com
# print("1. Querying google.com:")
# r = urequests.get("http://www.google.com")
# print(r.content)
# r.close()

# Example 2. urequests can also handle basic json support! Let's get the current time from a server
# print("\n\n2. Querying the current GMT+0 time:")
# r = urequests.get("http://date.jsontest.com") # Server that returns the current GMT+0 time.
# print(r.json())

ntpClientTZ.py

import time

def debug(debugLevel, debugCounter, methodName, details, logToFile):
    if(debugCounter==0):
        logText="Count\tLevel\tMethodName\tDetails"
        print(logText)
        if(logToFile):
            filelog(logText)
        logText="=================================================================================="
        print(logText)
        if(logToFile):
            filelog(logText)
            
    debugCounter = debugCounter + 1
    logText=str(debugCounter) + "\t" + str(debugLevel) + "\t" + methodName + "\t" + str(details)
    print(logText)
    if(logToFile):
        filelog(logText)
    return debugCounter

def filelog(logtext):
    file=open("datalog.csv","a")           # creation and opening of a CSV file in Write mode
    file.write((str(time.ticks_ms())+"~")+str(logtext)+"\n")          # Writing data in the opened file
    file.close()                           # The file is closed
    

formatDateTime.py

import time
import ntptime

def formattedTime():
    now=time.localtime()
    timeString=zfl(str(now[3]),2) + ":" + zfl(str(now[4]),2) + ":" + zfl(str(now[5]),2)
    return timeString

def formattedDate():
    now=time.localtime()
    dateString=zfl(str(now[2]),2) + "/" + zfl(str(now[1]),2) + "/" + zfl(str(now[0]),2)
    return dateString

def zfl(inputString, width):
    # Zero padd the specified number of digits
    padded = '{:0>{w}}'.format(inputString, w=width)
    return padded

debug.py

import time
import ntptime

def formattedTime():
    now=time.localtime()
    timeString=zfl(str(now[3]),2) + ":" + zfl(str(now[4]),2) + ":" + zfl(str(now[5]),2)
    return timeString

def formattedDate():
    now=time.localtime()
    dateString=zfl(str(now[2]),2) + "/" + zfl(str(now[1]),2) + "/" + zfl(str(now[0]),2)
    return dateString

def zfl(inputString, width):
    # Zero padd the specified number of digits
    padded = '{:0>{w}}'.format(inputString, w=width)
    return padded

secrets.py

ssid = 'YourNetworkName'
password = 'YourNetworkPassword'
3 Likes

NTP is complex. I have a webserver running PHP and I find it easiest to grab the date/time in JSON format. It is only a few lines of easy to understand python code. Swapping from NTP to local time is straightforward. The accuracy is good enough for my applications.

If you have a search I described the method in a post here.

https://forum.core-electronics.com.au/t/guide-by-michael-raspberry-pi-pico-w-connecting-to-the-internet/14879
4 Likes

The only reason I persevered with ntp is because it’s been around for ever and the servers are pooled so it’s not likely to break when I turn my back.
Perhaps that json server is more than just a single machine.
I think most of my confusion was from thonny updating the time automatically. I assumed it was only doing it at startup but I think every time I changed it, thonny “fixed” it. I disabled thonny time update in the .ini file and tested with the usb disconnected and it worked as expected then.
Thanks
David

2 Likes

NTP is the way to go. As David says, it has been around for ever. The DNS entry pool.ntp.org is backed by thousands of machines, and work is being done to make this regionally aware (I think at the moment it is a round robin by the DNS and you get one anywhere in the world. NTP looks at ping time and makes an educated guess about network delays). Although Fractal says NTP is complex (and it is), it is likely Pico W uses SNTP - Simple Network Time Protocol - which just asks “what’s the time?”. The full NTP client is complex as it tries to sync the local machine within a few milliseconds. However, modern PCs have the capacity to do this with (to a PC) trivial resources. SNTP is not necessarily that accurate, but usually the inaccuracy is much less than a second, usually ‘good enough’ for a microprocessor based application.

Just reading, it seems there are two real time clocks - one in the Pi itself that is active when power is applied, and one that is in a hat with battery backup. Is this true, and if so, is it confusing?

2 Likes

Hi Alan,

I can absolutely see this being confusing! Often microcontrollers like the RP2040 have an RTC peripheral, but it’s not battery backed, the power management is up to you. I think you’d use it in a low-power application where you had to turn the Pico on at certain times, and leave just the RTC running other times:
image

The DS3213 is a much simpler option than this for most people - it has its own power source you don’t need to worry about, it’s own clock source, and has no extra power draw than what is absolutely needed. You’d only go for the inbuilt RTC to minimise the cost/size of your RP2040 product I think.

Anyway, great work everyone in this thread, I’m sure this’ll come in handy for someone down the line!
-James

There is an Australian pool of ntp servers.
It’s named au.pool.ntp.org
It is a round robin thing within the Australian servers .

I don’t really care about sub second timing. Within a few seconds is still overkill for me. My internet connection has a +600ms ping time anyway. :slight_smile:
It will be handy if I can sync my two pico to be roughly the same if I ever need to compare logs on both devices.
I think I’ll sync the pico wifi from ntp then send the time via Lora to the remote pico.
The remote pico will also request the time when it boots. Sounds like an interesting solution anyway :slight_smile:
David

1 Like

Allan,
I forgot to comment on clock usage scenarios and how I deal with them. To simplify any code updates down the track I set the internal clock from the battery backed clock when the system starts up. From then on everything uses the internal system clock.
That way if the time source changes down the track you will only ever need to make one change. Plus all your programs come out the same because you have a rule to follow. :slight_smile:
People talk about the internal clock drifting over time. I suppose if that is critical to your project you could re-sync it periodically.
David

For iot I send data to a webserver using http with something like:

r = urequests.get(url = 'https://www.myserver.com/iot.php?date='+date+'&time='+time+'&temp='+tempC+'&humidity='+humidity+'&pressure='+pressure).text

Normally the micropython looks for the http 200 ok response for success.

It occurred to me that the webserver could also return the date/time as JSON for a trivial extra amount of return data. So the date/time can be checked on every data transfer. Overkill, but ultra simple and very cool. viz

j = json.loads(r.text)
rtc.datetime((j["year"], j["month"], j["day"], j["dayofweek"], j["hours"], j["mins"], j["secs"], j["subseconds"]))
print(rtc.datetime())

My ntp server micropython code posted above uses SNTP - Simple Network Time Protocol. It took me hours to debug the code and get it working so it was anything but simple. Even after extending the response time it occasionally timed out although using a local ntp pool may fix that.

3 Likes