Bit Fields

ELI5 and Why They're Useful

Created: 2021-04-30

Updated: 2022-12-03

After several years of programming in Python, I'd occasionally come across code like this snippet for Arduino, which is written in C:

for(int i = 0; i < DATA_WIDTH; i++)
    {
        bitVal = digitalRead(dataPin);

        // Set the corresponding bit in bytesVal.
        bytesVal |= (bitVal << ((DATA_WIDTH-1) - i));

        //Pulse the Clock (rising edge shifts the next bit).
        digitalWrite(clockPin, HIGH);
        delayMicroseconds(PULSE_WIDTH_USEC);
        digitalWrite(clockPin, LOW);
    }

I knew this was bit-shifting but didn't grasp the "how" or "why". I recently came across some OpenCV code that deals with managing window flags - and it finally clicked.

When we work with integers in Python, we think of them as a single value. In reality, this value is represented by multiple bytes:

import sys
>>> (sys.maxsize).bit_count()
63

For cases where we all we care about is True/False flags, a single integer can be made to represent many flags. And more importantly, for any combination of these True/False flags, we get a unique integer value.

Bit Fields are powerful structures for managing several True/False flags that interact to create unique values!

'Bit' of Structure First

We're working in binary - so a bit can be 1 or it can be 0. Think of these as:

• 1 = True
• 0 = False

Where it gets interesting is when we want to encode several True/False statements in one structure. For instance, the input from a keyboard.

Frequently, your computer is receiving multiple, simultaneous keystrokes (Shift, Alt, Space)

I used to deal with situations like these with control flow that looked like:

if 'a' in keys:
    output = 'a'
    if 'alt' in keys:
        output = 'b'
        if 'ctrl' in keys:
            output = 'c'
        elif 'space' in keys:
            ...

That makes me anxious. It's fine for simple conditions but can quickly spiral out of control

Let's break this problem down by looking at an example using a simple game controller. We have:

• Movement Pad (Forward, Back, Left, Right)
• A Button
• B Button

For a given input:

• Movement - 5 Possible States (Including None)
• A Button - 2 Possible States
• B Button - 2 Possible States

Which would look like:

Way of the Bitshifter

{'A': 1, 'B': 1, 'Left': 0, 'Right': 0, 'Up': 0, 'Down': 0} is easier to read

We could also represent this as tuple: (1, 1, 0, 0, 0, 0)

But if we're taking that step, we might as well go for a Bit Field 0b110000

Where 0b is the binary prefix in Python.

So the input is ... 48 ??

We can also lookup an individual field

A B Left Right Up Down

1 1 0 0 0 0

Builtin Extensibility

For my MindRef Project I've needed to interact with Java classes from Python to support Android.

The premise is that we register a Java callback with the Java class we are invoking. So when we call one of its methods, we can receive the result in our Python application. This would require:

• Java
  • Public interface with a onComplete method and compatible signature
  • Getter and Setter functions to add the callback coming from Python side
  • Additional boilerplate to call the associated method
• Python
  • PythonJavaClass from pyjnius with a onComplete method
  • A manager class for creates and associates the callback.

After 3-4 methods, I realized this would not be sustainable.

My solution was to consolidate everything into one interface on the Java side and one PythonJavaClass on the Python side. And add the convention that a single integer was to be the first parameter to every method.

In the Python code, I have a function that expects the key, and *args. I can determine context from the key:

class MindRefCallCodes(IntEnum):
    MIRROR = 1 << 0  # Mirroring a filesystem
    READ = 1 << 1  # Read without writing
    WRITE = 1 << 2  # Creating a resource
    REMOVE = 1 << 3  # Removing a resource
    CATEGORIES = 1 << 4  # Operating on categories
    NOTES = 1 << 5  # Operating on notes
    EXTERNAL_STORAGE = 1 << 6  # Target of operation is external storage
    APP_STORAGE = 1 << 7  # Target of operation is app storage

So if the key is 36, I know that I need to handle the result of 'WRITE' and 'CATEGORIES', which I interpret as 'Write a Category'