Osiyo. Dohiju? ᎣᏏᏲ. ᏙᎯᏧ? Hey, welcome back.
One of the elements about rapid prototyping is at some point you’re going to have to switch from prototyping to better development. That time came last week for me. I was taking my OpenCV code to detect an object and I needed to pass that to the page so ThreeJS could place a 3D object at that location. The issue, which I knew was coming, was that the OpenCV code is in Python and ThreeJS is on the front end. So how was I going to get an image and then do the OpenCV work and send that data to the front end too.
At this point, I have some options. I can work on a way to pass the info to the front end, which isn’t probably that bad but I’ll still have to work on the camera perspective between the two. I can do all of the work on the front end. After all, there are a lot of tutorials with LeapMotion, OpenCV, and ThreeJS out there. This might be a viable option, except that I’m having issues getting the objects to display where I want them in my view. Again, this could be because of the camera perspective between the two frameworks. I could do all of the development work in C/C++/Rust. If I were going to go this route I’d choose Rust because I hate CMake on Windows and I don’t use the Visual Studio compiler. If someone built a solution file I might use VS. I’d like the solution to work easily on all manner of machines hence Rust. Another option is to build it all in Python. And the final option is to build it in some conglomeration of everything. Like Rust for some elements, core libraries in C/C++, maybe some GLUT code. There are benefits to each approach. What I want is something simple.
The simplest approach, for me, would be for everything to be in Python. The TTS, STT, OpenCV, OpenGL, and LeapMotion all working in Python then the display is sent to the page. Why not just a “native” Python window? Well, I want the configuration of the front end to be highly manageable. I still want the display to be configurable for the user. I can still do some HTML GUI/HUD elements if I want. However, I really want 3D GUI/HUD in OpenGL. ThreeJS and WebGL do present some nice features and these are always possibilities if I make a lightweight front end I can continue with the work I’ve already started. In my opinion, I think it’s better that I have as much processing done in Python or compiled code and use the browser to display all of it. I think, in the future, this will translate better. This adds a layer of complexity that I wasn’t intending on. It is less complex than learning a whole new programming language like Rust to implement a bunch of other work. I like Rust a lot. However, I’d rather work on the programming and not spending time debugging something that I don’t have a full grasp on.
Up to this point working on Windows has been fine since most of the installs were Python commands. Now, I have to switch to Ubuntu in order to work because it contains all of the compiler info I need. For now, I will have to use Ubuntu until I can figure out a better way to work on the code. I am writing an installer to pull down the libraries needed and build.
Before I make the final decision on how I’m going to work this, I also have to consider the RealSense cameras that I’m going to be adding.
This post has been a mixed bag of crap. I have been exhausted with the snow storms, kids at home all the time, and power outages. I forgot that I was also looking at Babylon.js for 3D web renders. I did a lot of research last night and I think I’m going to use it. There are some good examples of integrating LeapMotion, BabylonJS, and OpenCV and how to make things work better. It’s not ideal because I will have to rewrite a lot of Python OpenCV code… however, I think it will be better in the long run than trying to merge technologies. We’ll see what the lag is. I may still have some code run on the backend because STT is still spotty through the browser. We’ll just have to see. I’ll know more once I build my POC.
I guess the next blog I write will be about a POC with Flask, BabylonJS, OpenCV, and LeapMotion.
Until next time. Dodadagohvi. ᏙᏓᏓᎪᎲᎢ.
IN MY MIND BY WINK DOUBLEGUNS 