Further Developments
Native iOS App
The current control panel is a React web app served over HTTPS. While this works well and can be installed as a PWA on iOS, a native Swift app would unlock several improvements:
- Bluetooth provisioning – configure WiFi credentials on Pico W devices without needing a serial connection
- On-device beat detection – use the iPhone microphone and Core ML to run beat tracking directly, removing the need for a Raspberry Pi server in simpler setups
- Haptic feedback – tap along with the beat using the Taptic Engine
- Widgets and Live Activities – show current tempo and beat count on the Lock Screen
- Better offline support – native networking with Bonjour/mDNS for automatic server discovery on the local network
- AirPlay / Spotify integration – sync to audio playback metadata instead of (or in addition to) microphone input
The SwiftUI client (at ios/) is already a thin replacement for the web dashboard on iOS and macOS — it talks to the same REST API and uses a Keychain-backed bearer token. The bigger remaining work is on-device beat detection (likely Accelerate-framework FFT plus a Core ML port of the BTrack onset detection model) and the Bluetooth provisioning / Bonjour discovery flows above.
Beat Tracking Frameworks
“Traditional” Signals-based Beat Tracking
| Library | Language | Bindings | Beat Tracking | Online Beat Tracking | Downbeat Tracking |
|---|---|---|---|---|---|
| BTrack | C++ | Python | ✅ | ✅ | ❌ |
| Aubio | C | Python | ✅ | ❓ | ❌ |
| Essentia | C++ | Python | ✅ | ❓ | ❌ |
ML Based Beat Tracking
Papers
- https://dida.do/blog/machine-learning-approaches-for-time-series
- https://paperswithcode.com/sota/online-beat-tracking-on-gtzan?p=a-novel-1d-state-space-for-efficient-music
- 1d Space
- Beatnet
- Böck & Schedl
- https://program.ismir2020.net/static/final_papers/223.pdf
- https://archives.ismir.net/ismir2016/paper/000186.pdf
- https://tempobeatdownbeat.github.io/tutorial/intro.html
Results: Offline results for Beatnet are rather disappointing compared to Btrack. In particular, because the model is trained on master copies with no noise, it struggles to identify beats on audio recorded with a microphone.
Libraries
https://madmom.readthedocs.io/en/latest/ https://github.com/mjhydri/1D-StateSpace