Piicodev - detecting which sensors are attaced

Support Raspberry Pi
21

Hi @Liam et al,

And an operational note re the tests code.

It CANNOT detect an actual address conflict on a given i2c bus. This is a characteristic of the bus itself.

However, if you use the how_many_connected() function, and it returns (say) 4, when you know and can see 5 devices on the bus then you do have something conflicting.

I tested this by changing the ASW on my ENS160 air quality sensor from the On position (which correctly separated it from an Adafruit LTR390 ambient/UV light sensor with the following results

# ASW ON  set On to avoid conflict with LTR390 at 0x53
show_int()
[16, 60, 82, 83, 119]
show_hex()
['0x10', '0x3c', '0x52', '0x53', '0x77']
how_many_connected()
5
# ASW OFF   ENS160 default
rescan()
[16, 60, 83, 119]
show_int()
[16, 60, 83, 119]
show_hex()
['0x10', '0x3c', '0x53', '0x77']
how_many_connected()
4
# OOPS this is wrong, 'cos I see 5 devices connected

Murray

2 Likes
22

Hi @Liam et al ( and @Michael )

I have just added code to support a user defined ā€˜libraryā€™ of non-PiicoDev devices.

The format of the library nested dictionaries is critical, and it must be defined before trying to use it ā€¦ Duh!

:exploding_head: Still WIP but here is a dump of the head of a program and the result of the run - first without the external definitions, then with the external definitions tested

##################################
# the top of the main.py
#

# Read air quality metrics from the PiicoDev Air Quality Sensor ENS160
# Shows three metrics: AQI, TVOC and eCO2

from PiicoDev_ENS160 import PiicoDev_ENS160 # import the Air Quality Sensor
from PiicoDev_BME280 import PiicoDev_BME280 # import the Atmospheric Sensor
from PiicoDev_Unified import sleep_ms       # a cross-platform sleep function
from PiicoDev_SSD1306 import create_PiicoDev_SSD1306

from PiicoDev_Unified import PiicoDev_test

mytests = PiicoDev_test()

mytests.show_hex()

print('-- details --')
mytests.details('long')

#
#define an extra list of 'foreign' devices
#
extern_list: dict = {
    0x53: {		# 83.  0x53
        'what': 'Ambient Light-UV Sensor',
        'long_name': 'Adafruit LTR390 Ambient Light-UV Sensor',
        'short_name': 'LTR390'},
    }

print('-- details w/ external list --')

mytests.details('long', extern_list)

#
# end of code extract
######################################

######################################
# Results
#

['0x10', '0x3c', '0x52', '0x53', '0x77']

-- details --
16 0x10 PiicoDev VEML6030 Colour Sensor
16 0x10 PiicoDev VEML6030 Ambient Light Sensor (ASW off) <<<<< Possible conflict
60 0x3c PiicoDev OLED Module SSD1306
82 0x52 PiicoDev Real Time Clock (RTC) RV3028
82 0x52 PiicoDev Air Quality Sensor ENS160 (ASW on) <<<<< Possible conflict
83 0x53 PiicoDev Air Quality Sensor ENS160 (ASW off)
119 0x77 PiicoDev BME280 Atmospheric Sensor

-- details w/ external list --
16 0x10 PiicoDev VEML6030 Colour Sensor
16 0x10 PiicoDev VEML6030 Ambient Light Sensor (ASW off) <<<<< Possible conflict
60 0x3c PiicoDev OLED Module SSD1306
82 0x52 PiicoDev Real Time Clock (RTC) RV3028
82 0x52 PiicoDev Air Quality Sensor ENS160 (ASW on) <<<<< Possible conflict
83 0x53 PiicoDev Air Quality Sensor ENS160 (ASW off)
83 0x53 Adafruit LTR390 Ambient Light-UV Sensor <<<<< EXTERNAL LIST <<<<< Possible conflict
119 0x77 PiicoDev BME280 Atmospheric Sensor

The assumption re devices in the external list is that conflicts are possibleā€¦
Canā€™t (yet) determine otherwise

Still WIP - not yet pushed to repo.

Murray

2 Likes
23

Hi @Liam and @Michael

repository updated
This is the description of the external dictionary and use from the code

        
        details()               - prints 'human name' of the connected ID's e.g. 'OLED Module'
            details('what')     - prints 'human name' of the connected ID's e.g. 'OLED Module'       
            details('short')    - prints 'short_name' of the connected ID's e.g. 'SSD1306'
            details('long')     - prints 'long_name' of the connected ID's e.g. 'PiicoDev OLED Module SSD1306'
        *****
        The details() function can also access a user defined dictionary of devices from other manufacturers.
        The user dictionary MUST be in the same format as the internal dictionaries
        
            extern_list: dict = {
                0x53: {		# 16.  0x10
                    'what': 'Ambient Light-UV Sensor',
                    'long_name': 'Adafruit LTR390 Ambient Light-UV Sensor',
                    'short_name': 'LTR390'},
                <nextID>: {    # possible comment
                    'what': 'simple description',
                    'long_name': 'lengthy description',
                    'short_name': 'MFR code'},
            }

        then calling the details() function as below _AFTER_ the user dictionary is defined, will
        display this if the LTR390 is in the connected devices list
        
            details('what', extern_list ) - prints 'human name' of the connected ID's e.g. 'Ambient Light-UV Sensor'       
            details('short', extern_list) - prints 'short_name' of the connected ID's e.g. 'LTR390'
            details('long', extern_list)  - prints 'long_name' of the connected ID's e.g. 'Adafruit LTR390 Ambient Light-UV Sensor'

Also cleaned up the print code to use common internal functions rather than cut/paste code blocks

(still to remove the commented out bits :exploding_head:

Murray

2 Likes
24

Hi @Liam and @Michael

The ā€˜finalā€™ push to the repo done ā€¦ v 1.0 exists :partying_face:

Comments etc expanded / cleaned up. Dead lines removed etc etc ā€¦

Murray

3 Likes
25

Hi Murray,

Looks amazing!

I can see the pull request there, Michaelā€™s still away but Iā€™ve got even more messages headed his way as soon as he returns!

Liam

1 Like
26

Hi Liam,

Any feedback from Michael re my PiicoDev pull request?

Murray

1 Like
27

Hey Murray,

Michael is out sick today, but when he gets back Iā€™ll be sure to get him to check in on this :slight_smile:

Cheers,
Sophia

1 Like
28

Hi Murray,

So sorry we havent gotten back to your (and Jamesā€™s) amazing PR yet, Iā€™ve got more messages headed his way.

Liam

29

Oh @Murray125532 what a bloody ripper! Thank you for this significant contribution to PiicoDev!
I really like the structure youā€™ve implemented - it appears to respect the existing PiicoDev_Unified structure and only bring extras to the party. Nice :smiley:
Iā€™d love to discuss how we can get this worked into PiicoDev, for for that weā€™ll have toā€¦

Letā€™s get into the weeds, or - the challenges of Unifiying

The unfortunate constraint that PiicoDev_Unified has to navigate is the flash memory size for Micro:bit v2 which is so small that we can just fit a minified version of PiicoDev_Unified and minified versions of one or two device libraries. This means that all changes to Unified have to be well considered to not increase the flash consumption any further. Your contribution is over 400 lines of added code.

OK so why not have multiple versions of Unified?

We knew early on that we wanted to maintain only one version of PiicoDev_Unified.py to streamline development, simplify testing and so Unified actually does what it says on the tin. That means that the same code has to work across multiple boards. To create separate versions for each piece of hardware introduces a lot of project risk and burden in maintaining and testing multiple files.

Then we found we outgrew the flash size of the Micro:bit v2, so we implemented the minifier which takes the original PiicoDev_Unified.py file, strips the comments, whitespace, and shrinks variable names where it can. The smaller ā€œminifiedā€ file is generated automatically every time a commit is made and all Micro:bit guids point to this file.

Great. We have sustainable deployment for Unified, where we edit and test one Piicodev_Unified.py file, and a special version gets made for Micro:bit that is functionally identical, just smaller and a bit obfuscated.

The awkward thing now is that Unified is more-or-less stuck at its present size until Micro:bit release a larger flash variant.

Hope is not lost

Weā€™ve done device libraries before that have a set of essential functions, and extended functions in another file. The PiicoDev RFID Module presented this exact problem: essential functions that are non-negotiable (reading NFC tags) and nice extras that fewer users would need (writing complex data to and from the tag). The PiicoDev_RFID.py contains essential functionality, and PiicoDev_RFID_Expansion.py file can be used by larger-memory devices. This obviously isnā€™t our preferred way to deploy devices since it breaks a lot of patterns, but in this case it was necessary.

Perhaps itā€™s time we used this idea with PiicoDev_Unified.py + PiicoDev_Unifed_Expansion.py?
What do you think? :smiley:

1 Like
30

Hi @Michael

Thanks for the appreciation.

I can think of two possible ways forward

  • the ā€˜simpleā€™ solution of yours where the Unified and Test functions are separate and only included as needed
  • a two-part implementation of your concept where a subset of the ā€˜most usefulā€™ functions become part of PiicoDev_Unified.py, and the rest of the functions with all the dictionaries and lookups in the PiicoDev_Unifed_Expansion.py module (maybe call it PiicoDev_Lookup.py or similar).

The ā€˜most usefulā€™ functions I am thinking would be the initialisation, and then begin at the top of this list, and ( space permitting ) extend appropriately down to the end of the list

 Available functions
    -------------------
        clear()                 - clears the list of connected i2c devices
        rescan()                - clears and rescans the default i2c bus and repopulates the list
        show_int()              - prints the list of connected ID's in DECIMAL detected by the original/most recent scan
        show_hex()              - prints the list of connected ID's in HEXADECIMAL detected by the original/most recent scan
        is_ID_connected(id)     - returns 1 if the ID is in the list, otherwise 0
        how_many_connected()    - returns count of detected ID's

This solution may actually include all the code into the expansion module anyway, and rename the class of the minimal selected functions in the main Unified library as

class PiicoDev_minitest()

so that either set of tests could be instantiated without collision.

Hmmm, and hindsight is 20/20 - I think that I need to add a ā€˜defaultā€™ bus parameter to the functions so that the alternate i2c bus can be considered ā€¦ more thinking needed here too (sigh) - and more lines of codeā€¦

cheers
Murray

1 Like
31

Take a look at our implementation of the RFID module @Murray125532 - you will find that to avoid duplication there are placeholder classes/functions initialised in the base file with a ā€˜not implementedā€™ warning. Then when the expansion file is included, the functions are redefined.

This avoids having funny edge cases like the minitest() function you proposed

the same could be true for the eg. description dictionaries, or they could remain undefined in the base file since theyā€™ll never get accessed without the overlying functions to do so.

1 Like
32

Interesting approach in the RFID modules ā€¦

However going way back to the original posters issue, maybe the appropriate implementation would be to add these minimal lines to the existing Unified module

    # add the appropriate form of this to the Unified init function
    self.i2c = I2C(id=id, scl=scl, sda=sda, freq=freq)
    self.connected = self.i2c.scan()     # especially this line
        
    # rescan() - clears and rescans the default i2c bus and repopulates the list
    def rescan(self):
        self.connected = []
        self.connected = self.i2c.scan() 
    
    # show_hex() - prints the list of connected ID's in HEXADECIMAL detected by the original/most recent scan
    def show_hex(self):
        print( [hex(i) for i in self.connected] )

so that at any time the user can run the show_hex() function to see what addresses are detected. and as I discussed above - while it wonā€™t panic if there are conflicts - if you know that that you have 3 devices connected and only two addresses show up, then your initial check will be to look at the address switches etc, and then for an open / missing connection.

For those wanting a bigger dive into the address mapping / what is connected / making their own dictionaries etc they could then choose to include PiicoDev_Lookup.py .

Hopefully the 5 or 6 ā€˜minimalā€™ lines wont break the Microbit, and this basic quick test is considered useful, as it is ā€˜always thereā€™ and can be called from the REPL and from their own programs as needed.

If the ā€˜bigger stuffā€™ is needed for development at the REPL, or for inclusion in their own codebase, just include it and initialise it.

Murray

EDIT
This is like picking the right hammer for the right task :stuck_out_tongue_closed_eyes:

1 Like
33

@Murray125532 i agree this is a great staple to have an eagle-eyed view of the bus.

I suppose the implementation would only have to change at import eg.

from PiicoDev_Unified import sleep_ms

becomes either

from PiicoDev_Unified import *

or

from PiicoDev_Unified import sleep_ms, scan

I took the liberty of implying a single scan function would perhaps be friendlier than rescan and show_hex

1 Like
34

A single scan() function works for me in the main Unified codebase

def scan(self)
    print( [hex(i) for i in self.i2c.scan() ] )

and keep all the rest ā€˜as isā€™ in a standalone module for the ā€˜serious geeksā€™ :smiley:

1 Like
35

If you want to put it in as a PR and get that good contribution credit I wonā€™t begrudge you!

Likewise if youā€™d like to proceed with restructuring Unified to include the additional functions as a separate file that could be a scalable way to proceed.

If you choose not to, Iā€™ll be happy to put in a ā€œsuggested forksā€ section in the readme to highlight your fork / commit. In that way, PiicoDev can stay vanilla and simple on the main repo but at least give people options for these awesome user addons.

1 Like
36

Hi @Michael

I have created a second PR with just the scan() command added to PiicoDev_Unified.py to provide the minimal information that would satisfy the original poster as discussed above.

If you update / release it to PiicoDev_Unified.py it can be used as below

Added very simple scan() command available to userland REPL and user codebase.

from PiicoDev_Unified import scan
from machine import I2C
scan()

Returns:
[nn,nn,...] # list of addresses discovered on i2c bus

The full suite of lookup functions can be added to your suggested forks list for use and needed.

cheers
Murray

3 Likes
37

PR14 has been merged. Thanks murray :smiley: Iā€™ll leave all the history and discussion on the PR

the changes have been merged onto main and are ready for use on Pico + Micro:bit
The features will appear for Raspberry Pi when we perform a python package (PyPI) update - on the release of a new module or bugfix

4 Likes