Osiyo. Dohiju? Hey, welcome back.
First, 2 years working on this and waiting for technology is nothing compared to 30 years to get to this point. Finally after all these years I have an AR view with OpenCV overlay. It’s not perfect, however, it is amazing.
I was looking at WebXR last night to see if I could do some quick AR/MX work in a browser without much in the way of libraries. Simply put, I wanted to use HTML and not convert everything to React, Angular, or some other framework. I wanted Flask to serve up the content and on an AR enabled browser the OpenCV portions would display on their own and not in a 3D world. I also didn’t want to use Unity to create something then navigate in it. I will be adding AR/MX functionality along the way, but primarily I wanted a display source. So my display source is my phone. To start with you can check out these links to view the direction I started in https://immersive-web.github.io/dom-overlays/ and https://github.com/immersive-web/dom-overlays/blob/master/explainer.md
Using this code, plus the code I have for the current SERINDA display I am able to view the components in an AR-lite setting. AR-lite because OpenCV isn’t exactly displaying AR, I’m just manipulating the HTML so it’s more or less transparent. Also, right now, I am running the code on my computer (Mac and PC) and I have to have Chrome open for voice commands. However, I can have a tiny display on my Raspberry Pi that I can click commands and when they run they obviously affect all viewers so that gets reflected back to the AR/MX browser as I’d expect.
Most of this is good. Like I’m not complaining because 4 years of research and work and 30 years of wanting this to work have finally come around.
Here are the bad parts. I’ve already said I need Chrome open. I don’t want that. I want to be able to interact with the audio and not have a browser open except on the phone. So maybe I have a microphone to the computer and the phone for AR/MX is just the display. I would like to not have to use my phone in the future. To be able to have a display with a camera view that shows AR as it should be. idk how i’m going to do that… maybe there’s an AR/MX browser for Raspberry Pi that I’ve not looked for or some other display technology I can utilize. The final item, for now, is the OpenCV displays. Commands are taking forever to get through the code and show up. They do show up though. Along with that the Audio for running some commands is failing after my adjustment to the snips code so I could process numerical data and not just string data.
I’ll fix a few of these items and then put up a video displaying what it can do. For the display I need to fix the view so it’s smaller. I didn’t account for the display being so big in the viewer. I can certainly make it a lot smaller and still readable. I need to figure out opacity. Setting the opacity of a div to anything tends to break the overlay so it does not even show up. It would be nice to be able to load some text and set the opacity to something else so I can read. As well it might be nice to be able to set the view as one or both eyes. So maybe I can read with one eye and have some view of the world but also have a full view of the world with the other eye. I don’t know yet. The fact that I’m now contemplating something I was looking forward to is pretty exciting. Now the fun part of experimentation begins.
Until next time. Dodadagohvi.