GuruCat
Around the beginning of the pandemic, I was asked if I could make an app that could use a cats face to generate a ‘psychic reading’. As I suddenly had a lot more free time, I thought I would take on this slightly ridiculous challenge!
Firstly, this meant learning how to train a neural net to detect cat faces, and then how to program in Unity/C# – which seemed the ideal platform for real-time detection, and for 3d graphics, instead of something like React Native which doesn’t allow for multithreading, and isn’t particularly suited to anything other than 2d.
The neural net was the hardest part of this. Off the shelf trained models existed, but were not good enough for public use, and were not optimized for running on mobile devices.
After some experiments with HOG detection, (Histogram of oriented Gradients), and SVM, (Support Vector Machines), I discovered some very recent advances in a model called YOLO, (You Only Look Once). Experimenting with this led to much better results. In the end I trained a very accurate model in a version of YOLO, called YOLOX-Nano.
Training was performed on Google Colab ultra fast GPU/TPU, using several thousand annotated images. The implementation in Unity uses a multi-threaded combination of OpenCV, and NatDevice for real-time on-device detection, using the phone’s camera.
On successful detection, the app then generates a social media friendly and sharable video directly in the app, and provides native phone sharing capability to whatever social network apps you have installed on the phone.
It also uses OneSignal for notifications, and Google AdMob for adverts.
Published in the App store and Google Play.
