S.send(data1, data2, data3) to TTN on LoPy4

Support Micropython
1

Guys,
Not really sure where to post this but Im seeking help again…

Following on from this:

and this:

and this: https://core-electronics.com.au/tutorials/encoding-and-decoding-payloads-on-the-things-network.html

I want to be able to send Battery Voltage and 2x Temperature readings to TTN, ideally in one s.send.

I can send the data as 3 lines of code, i.e:

s.send(bat_volts)
s.send(temp1)
s.send(temp2)

but this seems problematic making sense of the data when it pops into a TTN event in Node-Red (via TTN).

I was hoping to be able to:

s.send(3 bits of data all at once)

That way, when the TTN-event in Node-Red is triggered I can query and manipulate the data as appropriate.

I have looked at s.send(bytes([1, 2, 3])) and this seems to give the answer Im looking for in TTN, but I have no idea yet how to combine and ustruct.pack the 3 variables (floats).

I also read here (ustruct) about ustruct.pack(fmt, v1, v2,). This appears to pack v1 & v2 together but then I cant unpack the two values correctly with ustruct,unpack(fmt, buffer).

To try and show my thinking so far:

cst = 1.234
cat = 10.2
print("CST = ", cst)
print("CAT = ", cat)
# encode the packet, so that it's in BYTES (TTN friendly)
cst_pack = ustruct.pack('f', cst)
cat_pack = ustruct.pack('f', cat)
combined_pack = ustruct.pack('f', cst, cat)

print("cst_pack:", cst_pack)
print("cat_pack:", cat_pack)
print("combined_pack:", combined_pack)

print("Now lets decode...")
print ("Unpacked cst is:", ustruct.unpack('f', cst_pack)[0])
print ("Unpacked cat is:", ustruct.unpack('f', cat_pack)[0])

print ("Unpacked Combined is:", ustruct.unpack('f', combined_pack)[0])
#print ("Unpacked Combined is:", ustruct.unpack('f',combined_pack)[1])

This appears to show that combined_pack is the result of packing the two variables together but I have no ideas on how to un-pack them back to the two variables again…

Any chance you can give me some pointers please?

Thx
J

2

Hi Jon,

The combined pack puts the two byte values together basically. You can use unpack to get a tuple containing the values.

ustruct is almost identical to the Python library ‘struct’. The info provided ustruct in the documentation only points out the differences. For great examples check out the struct documentation:
https://docs.python.org/3/library/struct.html

3

In my limited experience, the “u” on the front is meant to represent the Greek letter “mu”, which is often used to represent “micro”.

So if you use uXYZ, on the LoPy, there is probably an XYZ for Python. Hope this helps.

1 Like
4

Thanks guys, I’ll have another look at it today and see if I can figure it out.

5

Success:)

Code snippets:

def adc_battery():
    #... Routine removed
	return adc_median

def temp_measure():
    #.. Routine Removed
    return Current_Soil_Temp, Current_Air_Temp

adc_median, Current_Soil_Temp, & Current_Air_Temp are all floats.

To pack up into bytes Ive used:

    send_packet = ustruct.pack("3f", *[temps[0], temps[1], lipo_voltage])

And as a check, to unpack:

    print ("Unpacked Send_packet is:", ustruct.unpack('<3f',send_packet))

The unpack routine outputs “Unpacked Send_packet is: (15.0, 15.25, 3.8908)” So, I can now pack up the 3 variables and correctly unpack them in Atom.

Uploading to TTN with:

s.send(send_packet)

This works successfully too.

Now onto TTN decoder problems!!

By way of a disclaimer, I have never done anything in Javascript before - its like a foreign language to me:(

The payload comes through TTN as 00007141000075418FB97840 and the data input (pre-packing in MicroPython was (15.0625, 15.3125, 3.8908) and post packing was b’\x00\x00qA\x00\x00uA\xdc\x02y@’

All my Googling so far has talked about unpacking one float and presenting that as say “temp: 1.234”.

How on earth do I go about taking b’\x00\x00qA\x00\x00uA\xdc\x02y@’ and decoding into temp1: 15.0625, temp2: 15.3125, volts: 3.8908?

This one really has me beat guys so any help would be gratefully received.

Thx
Jon

6

Hi Jon,

Here is a tutorial I’ve made about decoding payloads:
https://core-electronics.com.au/tutorials/the-things-network-ttn/encoding-and-decoding-payloads-on-the-things-network.html

There is a float decode bit in there, but it is only for a single float. Adding more floats to it to decode is quite easy though!

Here is the code (I’ve tested this with your values and it works)

function Decoder(bytes, port) {

  // Based on https://stackoverflow.com/a/37471538 by Ilya Bursov
  function bytesToFloat(bytes) {
    // JavaScript bitwise operators yield a 32 bits integer, not a float.
    // Assume LSB (least significant byte first).
    var bits = bytes[3]<<24 | bytes[2]<<16 | bytes[1]<<8 | bytes[0];
    var sign = (bits>>>31 === 0) ? 1.0 : -1.0;
    var e = bits>>>23 & 0xff;
    var m = (e === 0) ? (bits & 0x7fffff)<<1 : (bits & 0x7fffff) | 0x800000;
    var f = sign * m * Math.pow(2, e - 150);
    return f;
  }  

  // Test with 0082d241 for 26.3134765625
  return {
    // Take bytes 0 to 4 (not including), and convert to float:
    Temp1: bytesToFloat(bytes.slice(0, 4)),
    Temp2: bytesToFloat(bytes.slice(4, 8)),
    Voltage: bytesToFloat(bytes.slice(8, 12))
    
  };
}

A brief explanation:
The bytestofloat function converts bytes from 32 bit interger to a float (through some processes I don’t fully understand)
You don’t really need to know how it’s doing it if you know how to use it though. Here we call the function, and save the result as Temp1. You are selecting bytes 0-3 (four total), to decode into a float using the function.

// Take bytes 0 to 4 (not including), and convert to float:
Temp1: bytesToFloat(bytes.slice(0, 4))

The payload received is just all the bytes together in order, so you identify the slice of the bytes you want to decode and store to each variable.
If you had both integers and floats in the same string of bytes you could add the integer decode function and call that particular slice of the pie by picking the two bytes that represent it.

Let me know if you have any questions!

7

OMG! Thanks so much. I spent a good part of yesterday looking at this and couldnt figure it out.

I did find some examples that did the bytes.slice thing but I couldnt make the numbers work - I had (incorrectly) assumed that I could paste b’\x00\x00qA\x00\x00uA\xdc\x02y@’ into the payload test field and that this is the same as 00007141000075418FB97840. Turns out not to be the case!!

I am learning that its the simple things that often trip me up, for example:

  1. Its not well documented that the 3f in the struct.pack code means to expect 3 floats - all text refers to just ‘f’.
  2. This example - I never thought to try to paste the payload data into the payload test field. Doh!

Thanks Stephen, very much appreciated.

Now onto my final hurdle (for now) - to get node red to populate 3 charts from the three bits of data Node Red is now seeing:)

R
Jon

1 Like
8

Hi guys,
Thought Id update with the final piece of the puzzle.

It was relatively straight forward, the only bit of code I needed to develop was the function to seperate msg.payload into 3 seperate values. That code looks like this:

var msg1 = {};
var msg2 = {};

var a = msg.payload

msg.payload = a.Battery_Volts;
msg.topic = 'Battery Volts';

msg1.payload = a.Air_Temp;
msg1.topic = 'Air Temp';

msg2.payload = a.Soil_Temp;
msg2.topic = 'Soil Temp';

return [msg, msg1, msg2];

Im not sure if its the most efficient, but it works.

00%20pm
Image 7-6-19 at 6.00 pm.jpg725×255 54.7 KB

9

Once you understand how a floating point number is stored, it all makes sense.

The float consists of three parts:

  1. The sign 1 bit
  2. The mantissa - the number part
  3. The exponent - the power to multiply by.

So the function first of all rearranges the 4 bytes into a word with bits in the right place. As it says, the LSB is first, and the MSB is in the 4th byte. Hence the need for shifting the bytes left to the correct place in the word.

Once that is done, split the word into the three parts of a float, and voila. Some maths and you have the correct result.

Computers really are simple - but only if you know the silly tricks they play to make up for being so simple. :):laughing:
:grin: