Software

Introduction

Just like the 2017 and 2018 NodeConf EU badges, your NodeWatch runs a version of the Espruino embedded JavaScript interpreter. And just like those badges, you can connect to it from the Web IDE in Google Chrome and program its NRF52832 BLE chip directly with no intermediate compilation steps.

Using Espruino also means you have access to a very large base of existing code libraries, examples and tutorials, going back several years, created by both the Espruino team and the community.

Bangle.js specific Reference

Whilst you can use all of the reference docs on the main site, there is also a specific one for Bangle.js which includes all of its new features, including TensorFlow Lite here.

Open Source

Espruino has always been Open Source and this continues with Bangle.js. You can acess all of the main Espruino code here and the Bangle.js specific Apps source here.

Connecting to your Bangle.js over Bluetooth

If you are not using HID to control your phone/PC and want to program your watch with the Web IDE, you need to ensure your device is connectable. Go into the Settings menu and check that "Bluetooth" is On and "Programmable" is On. Then scroll to "Make Connectable" and select "Yes". You should also note the last 4 characters of your Bluetooth address. Then exit out of Settings to the main Clock screen.

Music and keyboard control of your PC or phone

You can optionally install the HID and Music Control Apps from the Bangle.js Appstore if you want to control your computer over Bluetooth. Also enable Bluetooth and HID in the Settings App. You can then use the three buttons to go up/down tracks and play/pause music in the music app or go up/down in a presentation etc in the HID app when you have paired the watch with your phone or computer.

Note that you cannot use HID and simultaneously use the Web IDE for programming.

As of June 2020, the HID functionality is still quite buggy and regularly causes the watch to reboot and lose its time. We don't recommend keeping it enabled permanently yet.

Switching between HID User Mode and IDE Developer Mode

You cannot use both HID and the Web IDE together and you need to follow some specific steps to avoid confusing your computer about what the Bangle.js device is. If you have already been using HID mode to emulate a keyboard or music controller, then you need to do the following before you can connect with the Web IDE.

  • Go into Bluetooth settings on your computer
  • Delete Bangle.js (or its equivalent Bluetooth address) from the list of known devices. On Mac, you must right click it in the list. The X symbol only disconnects and is not sufficient.
  • Turn off HID in the settings on your Watch
  • Long press BTN3 to get back to the standard clockface
  • When you go into the Web IDE now and try to connect, you should see the device listed. If not, you may have to do a reset of the Watch by very-long-pressing BTN1+BTN2 until your watch buzzes.
  • If you want to go back to HID mode, ensure you are not connected on the IDE and repeat the process of deletion and re-connection in the correct mode.

Bluetooth REPL

You can execute code directly in a REPL on the device in the Web IDE. Simply click the yellow Connect Button, select Web Bluetooth and then select your device. Once the Connect Button goes green, you can start typing directly on the Left Hand Side.

TensorFlow Lite for Microcontrollers

You can read all about TensorFlow Lite here and TensorFlow Lite for Microcontrollers here over on their main site.

NodeWatch is the first smartwatch with TensorFlow Lite built-in! What's particularly exciting about our implementation is that you don't have to do a huge install of compilers and tools to make use of it. All of that messy work has been done for you. So you can focus on building and training your models and uploading them on to the watch for immediate execution.

Note that TensorFlow Lite for Microcontrollers is quite new and has a very limited set of features compared to full TensorFlow. Make sure you read up on those before you start building anything new.

TensorFlow on Bangle.js should be considered experimental. You may run into issues like "out of memory" errors at this stage in its development.

Hello World Sine Wave with TF Lite

There's a full set of instructions for how to create and deploy the TF Hello World example to Bangle.js here. It should give you a good understanding of how you can create your own ML implementations on Bangle.js using TF Lite.

Gesture Control of your PC or phone

We're so incredibly excited by this aspect of the project. Once TensorFlow Lite for Microcontrollers was ported to Bangle.js and ran under Espruino, we set to work on adding gesture control using Machine Learning.

There is a demo App in the Bangle.js App Store which adds some default gestures trained with other people's data. To use it, you need to:

  • Enable Bluetooth
  • Go to Settings > HID
  • Enable Human Interface Device mode (HID)
  • On your computer, pair with your watch (again getting the right Bluetooth address from the settings menu)
  • Open a presentation in Powerpoint or Google Sheets and go into presentation mode
  • Open the Gesture app
  • Swipe your arm left and right to change slides!!!

We have other gestures built-in like double-clap and up-up but we won't have bound them to any action on the PC. This is something you could try to do yourself as an exercise.

In this Workshop you can learn how to add gesture control to your NodeWatch and customise those gestures with your training data.

Andreas Madsen has written a detailed blogpost here on how the gesture model was built for Bangle.js.

Google Colab from Workshop

The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more.

Normally Jupyter runs directly on your computer. One easy way of setting it up is to use an all-in-one installation like Anaconda. However to avoid having to do this for NodeConf, we are using Google Colab.

Colab is a free cloud service based on Jupyter Notebooks that supports limited free GPU. Not only is this a great tool for improving your coding skills, but it also allows anyone to develop deep learning applications using popular libraries such as PyTorch, TensorFlow, Keras, and OpenCV.

Your Notebooks can live on Google Drive and all of the execution of the code happens on the Google Cloud.

One key aspect of the NodeWatch workshop is a walkthrough of a Google Colab that the NearForm Research Data Science team created. In it you'll learn how to add a new gesture to your device from scratch. You won't need any experience of ML, TF, Jupyter or Colab. In many cases, you'll simply be pressing a Play button beside some code and looking at the output.

You can access the full Jupyter Notebook (always with the extension .ipynb) for Google Colab that we used in the Workshop in our Gestures Repo here. Whilst GitHub can do a nice preview of it, you need to import it into Colab to run it. Ignore the README on that Repo unless you are running it locally.

Using gestures in your code

If ‘.tfmodel’ and ‘.tfnames’ files exist, the Bangle.js will automatically run the model against gestures when the watch is moved, and will output the result with a ‘Bangle.on(‘aiGesture’ …)’ event.

Installing your Apps in flash on the NodeWatch

Follow the steps here to add your app to your Bangle.js menu.

BLE to MQTT bridge

You can use MQTToBLE on Raspberry Pis and/or ESP32s to setup a BLE to MQTT bridge so that your watch can send and receive MQTT messages over the internet. This also works very nicely with Node-RED.