Osiyo. Dohiju? Hey. Welcome Back.
I’ve been stuck the last year or so on recording videos. I liked recording videos but editing them became too much of a PITA. I just set up a streaming computer with multiple camera views including an overhead cam. It’s similar to my static setup that I was using for board game reviews. In this case I’m going to just stream my content instead of writing and editing videos. When the stream is over I’ll edit it for content and make youtube videos out of it. The plan is to go through all of the collectibles I own (which is quite a few) and show them in-depth. In addition, I’m going to stream some of my Burn Notice transitions videos. I’ll have the videos set up so I can go through and narrate them. I stopped on these last year because I wasn’t sure where fair use comes into play and I didn’t have time to figure it all out.
I’m working on LARC v2 which is the Launch Assist and Recovery Computer – a lot of that work will go directly into SERINDA v3. I’m starting over on the SERINDA project, essentially. I want my HUD/HMD working again. I have all of the components still. I even have the Latte Panda available for running code on Windows. I have started outlining my requirements for this project and the steps I want to take to get it working. I’m going to be writing individual pieces of code that do what I want for that section. Then later I’ll integrate into one application. I started with this approach before then I got side tracked by many life events.
I made mistakes with SERINDA v2 when I went with nodejs. I think I had the correct approach for most of it – the mistake I made was not using python for processing. I liked having the browser act as the screen to do displays. I can work in webpages easily. I think, however, I’m going to go with React for this application and then utilize node or something else for GPIO calls. I also have the LattePanda so I don’t have to do any of this on the RaspberryPI. I can do it all in Windows. However, I want this to be something anyone can do so I’m really going to work to keep it all in the RaspberryPi. At this point in development though I want it to work.
I have a few obstacles to overcome. The first is display. I’ve ordered a tOLED however it’s monochrome so we’re talking data readout only. If I want to do anything non monochromatic I’ll have to take one of my previous AR setups and put those to work – which means they’ll be bulky at best. BUT that’s not the biggest concern. The second obstacle is VOX. I want to be able to give voice commands and have it do the work correctly. Now, using Chrome I can overcome a lot of this with the libraries I’ve written already. I still like the idea of using the browser for the display because I can customize it. The third item piggy backs off that. With everything I want to do it really seems like the browser is the way to go to display the work. That means I need to seamlessly work with Python for OpenCV (or even C++) I’m not going to use opencv4nodejs – there are far more examples with Python that I can understand (or even C++) and I am not going to take the time to do something that is only beneficial to a few people. Don’t get me wrong opencv4nodejs is a great library – it’s not what I’m going to use for this project. I might go back and do work with it… but not for now. I also intend to use some of the AR packages out there for React. Now, I would like for this to work on all devices. At this time, however, I really don’t care what devices it works on as long as it works. I can go back and do something with it at a later time. I am, more than likely, going with https://viromedia.com/viroreact for devices. There are several elements I need to consider, such as maybe the phone or device acts as the augmented reality camera and overlay while it sends a video feed to the pi or lattepanda which then does opencv calculations and the like for additional overlays I want. I don’t know yet. So the final obstacle, at this time, is gestures. Since I’ve been working on this project so long I’ve done quite a few amazing feats with the code. I’ve been able to detect road signs and read them as I drive by. I have been able to calculate ground speed and relay that to an Arduino that then displayed the data on an OLED along with other information. I’ve done voice commands with openCV. I’ve done neural network training with OpenCV to be able to detect and identify objects. The one element I’ve not completed is gestures. I have a LeapMotion and that may still work, but idk. I have gloves I taped each finger with a different color so I could test gesture controls. This isn’t a big deal yet because there are so many more items I want to do first. However, I do want gestures to be able to control the display as well as gesture movements to be able to crop an area for the camera to take a picture or what have you. Maybe use gestures to scroll a pdf. I’m not sure, yet. That’s kind of the beauty of this. I can make it up as I go along. I can use Minority Report as a guide and go from there.
So just where do I start? I think I start with a couple of experiments to link Python, Express, and React together into an app. I’ve done some work like this before. I don’t want to tax the raspberrypi too badly on processing. I don’t know what it will be able to handle for streaming as it is. So, I am going to start with this. The idea is that I want Express running to handle REST requests. React will be my primary interface for display. With this, that means, my code for sending voice commands to the server will still work, but now I’ll use React instead of angular. The Express code sitting on top of the old MEAN stack should still be fine though. The next step is that I need to work with OpenCV in Python (and I might go to C++ or Groovy or something later, but Python as I said has a lot of examples and I’ve worked with it the most) and get that to stream to the browser. The next part is to send voice commands from the application to Python to start and stop scripts in some way. This was originally why I went with OpenCV4NodeJs so I could control the app in one location. The issue was that I spent too much time debugging what was going on with the processing rather than actually coding the app. I may have mentioned this before; I spent years perfecting my game engine work and never wrote a game. When I left the game engine wrapper I realized I wished I’d spent more time working on the game than the supporting architecture. That’s where I am now. I want my working examples to be combined into the app.
The start is going to be, recap time, Express set up to process REST requests and manage GPIO through NodeJS. Python (or C++ or Groovy) OpenCV. ReactJS for the front end. And finally the webkitSpeechRecognition for processing speech commands to send to the service so it can run an app. App is ambiguous. App refers to the whole application but also I call each sub program an app. So if I want to open a PDF that set of code is a program but I’d call it an app. So I may be confusing. I should call them like a subroutine or program or something else. For now, it is what it is.
So tonight I’m going to get the base set up (NodeJS, Express, Groovy, OpenCV, Java, and a base React app with typescript) and I’m doing this in a Kali Linux VM for the moment, but I’ll get it set up on a Pi soon.
Until next time. Dodadagohvi
IN MY MIND BY WINK DOUBLEGUNS 