Software

Introduction

Just like the 2017 and 2018 badges, your NodeWatch runs a version of the Espruino embedded JavaScript interpreter. Just like with those badges, you can connect to it from the Web IDE in Google Chrome and program its NRF52832 BLE chip directly with no intermediate steps.

Using Espruino also means you have access to a very large base of existing code libraries, examples and tutorials, going back several years, created by both the Espruino team and the community.

Bangle.js specific Reference

Whilst you can use all of the reference docs on the main site, we also have a specific one for Bangle.js which includes all of its specific features, including TensorFlow Lite here.

Open Source

Espruino has always been Open Source and this continues with Bangle.js. You can acess all of the main Espruino code here and the Bangle.js specific Apps source here.

Connecting to your Bangle.js over Bluetooth

If you are not using HID to control your phone/PC and want to program your watch with the Web IDE, you need to ensure your device is connectable. Go into the Settings menu and check that "Bluetooth" is On and "Programmable" is On. Then scroll down to "Make Connectable" and select "Yes". You should also note the last 4 characters of your Bluetooth address.

Bluetooth HID, Music Control

You can install the HID App and Music Control App from the Bangle.js App store. These enable you to control a PC over Bluetooth by either emulating a keyboard or music controls. Make sure to enable HID and Bluetooth in your settings before launching.

In this Workshop you can learn how to add gesture control to your NodeWatch and customise those gestures with your training data. See below.

Switching between HID User Mode and IDE Developer Mode

You cannot use both HID and the Web IDE together and you need to follow some specific steps to avoid confusing your computer about what the Bangle.js device is. If you have already been using HID mode to emulate a keyboard or music controller, then you need to do the following before you can connect with the Web IDE.

  • Go into Bluetooth settings on your computer
  • Delete Bangle.js (or its equivalent Bluetooth address) from the list of known devices. On Mac, you must right click it in the list. The X symbol only disconnects and is not sufficient.
  • Turn off HID in the settings on your Watch
  • Long press BTN3 to get back to the standard clockface
  • When you go into the Web IDE now and try to connect, you should see the device listed. If not, you may have to do a reset of the Watch by very-long-pressing BTN1+BTN2 until your watch buzzes.
  • If you want to go back to HID mode, ensure you are not connected on the IDE and repeat the process of deletion and re-connection in the correct mode.

Bluetooth REPL

You can execute code directly in a REPL on the device in the Web IDE. Simply click the yellow Connect Button, select Web Bluetooth and then select your device. Once the Connect Button goes green, you can start typing directly on the Left Hand Side.

TensorFlow Lite for Microcontrollers

You can read all about TensorFlow Lite here and TensorFlow Lite for Microcontrollers here over on their main site.

NodeWatch is the first smartwatch with TensorFlow Lite built-in! What's particularly exciting about our implementation is that you don't have to do a huge install of compilers and tools to make use of it. All of that messy work has been done for you. So you can focus on building and training your models and uploading them on to the watch for immediate execution.

Note that TensorFlow Lite for Microcontrollers is quite new and has a very limited set of features compared to full TensorFlow. Make sure you read up on those before you start building anything new.

TensorFlow on Bangle.js should be considered experimental. You are likely to run into issues like "out of memory" errors quite regularly at this early stage in its development.

Hello World Sine Wave with TF Lite

There's a full set of instructions for how to create and deploy the TF Hello World example to Bangle.js here. It should give you a good understanding of how you can create your own ML implementations on Bangle.js using TF Lite.

Gesture Control of your PC or phone

We're so incredibly excited by this aspect of the project. Once TensorFlow Lite for Microcontrollers was ported to Bangle.js and ran under Espruino, we set to work on adding gesture control using Machine Learning. There will be a lot more information about it in the coming weeks.

We will be releasing an App in the Bangle.js App Store shortly to quickly install some default gestures trained with other people's data. When that is available (very shortly). You will need to:

  • Enable Bluetooth
  • Go to Settings > HID
  • Enable Human Interface Device mode (HID)
  • On your computer, pair with your watch (again getting the right Bluetooth address from the settings menu)
  • Open a presentation in Powerpoint or Google Sheets and go into presentation mode
  • Open the Gesture app
  • Swipe your arm left and right to change slides!!!

We will have other gestures built in like double-clap and up-up but we won't have bound them to any action on the PC. This is something you could try to do yourself as an exercise.

You can follow the Workshop to get started.

Google Colab from Workshop

The Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more.

Normally Jupyter runs directly on your computer. One easy way of setting it up is to use an all-in-one installation like Anaconda. However to avoid having to do this for NodeConf, we are using Google Colab.

Colab is a free cloud service based on Jupyter Notebooks that supports limited free GPU. Not only is this a great tool for improving your coding skills, but it also allows anyone to develop deep learning applications using popular libraries such as PyTorch, TensorFlow, Keras, and OpenCV.

Your Notebooks can live on Google Drive and all of the execution of the code happens on the Google Cloud.

One key aspect of the NodeWatch workshop is a walkthrough of a Google Colab that the NearForm Research Data Science team created. In it you'll learn how to add a new gesture to your device from scratch. You won't need any experience of ML, TF, Jupyter or Colab. In many cases, you'll simply be pressing a Play button beside some code and looking at the output.

You can access the full Jupyter Notebook (always with the extension .ipynb) for Google Colab that we used in the Workshop in our Gestures Repo here. Whilst GitHub can do a nice preview of it, you need to import it into Colab to run it. Ignore the README on that Repo unless you are running it locally.

Using gestures in your code

If ‘.tfmodel’ and ‘.tfnames’ files exist, the Bangle.js will automatically run the model against gestures when the watch is moved, and will output the result with a ‘Bangle.on(‘aiGesture’ …)’ event.

Saving your Apps to storage on the NodeWatch

By default anything you type or copy/paste directly on the Web IDE REPL is running from RAM and is not persisted. Don’t change the communications settings to ‘save to flash’ - it will overwrite the bootloader and loading menu (you can get it back by installing the Bootloader from http://banglejs.com/apps)

Apps are loaded from storage via a bootloader stored using E.setBootCode. To develop we’d recommend:

Leave the IDE’s upload setting at the default ‘Save on Send’ -> to RAM setting Upload your code using the upload button When it’s working, wrap it in:

require(“Storage”).write(“-myapp”,`...your code, in this string`)

Then upload.

If you also write a JSON file like:

require("Storage").write('+myapp',{
  name:"My App",
  src:"-myapp"
  //optional icon:"*myapp",
});

Then the watch’s menu will show your app. The icon should be another file containing a 48px icon generated by http://www.espruino.com/Image+Converter saved as 1 bit, 4 bit or 8 bit Web Palette “Image String”

To add it to the ‘app loader’ just check out the readme in https://github.com/espruino/BangleApps - you just add the files you made above (without require("Storage").write), and add an entry to ‘apps.json’

Add your App to the Appstore

Even better, once you are happy with your App, you can submit it via PR for inclusion in the official Bangle.js App Store!

Clocks / Widgets

We'd love to see an ecosytem of Watchfaces evolve, like the old Pebble one.

You can set type:"clock" or "widget" in the JSON for your app. Generally clocks should leave lines 0..23 free for Widgets, and if you're making a widget it should be 24px high or less.

BLE to MQTT bridge

You can use MQTToBLE on Raspberry Pis and/or ESP32s to setup a BLE to MQTT bridge so that your watch can send and receive MQTT messages over the internet. This also works very nicely with Node-RED.

Gotchas for Developers

  • There’s a global called ‘g’ which is the display. It’s an instance of https://banglejs.com/reference#Graphics - By default there is no offscreen buffer - so ‘g.flip()’ won’t do anything - but it doesn’t cause a problem either
  • GPS+Screen on uses about 100mA, so it’ll drain the watch's battery in about 3 hours. With no GPS and normal usage you’ll get days out of it.