XR – MR – How Did I Get To This Point?

ᎣᏏᏲ. ᏙᎯᏧ? Hey, welcome back!

At age 5, I knew only two things – I wanted to be a firefighter and fly helicopters. I knew exactly what led to me wanting to be a firefighter. I don’t remember how I found out about helicopters and pilots. It was probably a TV show my parents watched or a movie, and I picked up on it.

I would ride in the car and imagine flying down the freeway behind our car and over bridges because somehow I was aware enough to know I couldn’t go under. I would also pretend the house was on fire, drive up in the engine and put it out, save people on the roof, and I had a Dalmatian. Watching 101 Dalmations a few years later was like an awesome story about a breed of dog I wanted.

My love of helicopters was further expanded by the movie Blue Thunder, scenes from Magnum PI where TC was flying in Vietnam, and Airwolf. The fantastic technology of being able to move a helmet that directs the forward guns captivated me. I didn’t know how it could be done; I also knew that at least part of that tech was, indeed, real. I came up with various scenarios where the tech of Blue Thunder and Airwolf were miniaturized.

Then came the cartoons MASK and GI Joe, which had a lot of cool displays, voice activation stuff, and much more that thoroughly intrigued me. This lit a bunch of ideas for me. And Batman (1989) and the Zorro TV show (with Duncan Regehr) ignited my tinkering and fascination. I spent a lot of time designing helmets, HUDs, features, and more.

There was an ad on the radio and TV for patenting your idea. I realize it was probably a scam where they take your idea and then make their own stuff. However, I drew it up, came up with the design specs, and called. They needed a name, so I called it the Monohelm. It was a dumb name, but I had to think of something immediately. It was an Army helicopter pilot helmet with features like HUD, cameras on all 4 sides, a wrist-mounted keypad, and many features involving the HUD.

https://www.flighthelmet.com/product/PRS4-SPH4G.html

Then came Universal Soldier, which changed my applications a little bit more.

I had all the specs, design, HUD, and more planned out. I contacted some manufacturers. There were a lot of issues with my design, mostly that I wanted them to be smaller than possible in 1992. The cost for coming up with some possible items was $435,000, almost a million dollars today.

I still pursued ideas and fantasized about this HUD/HMD. In 2002, I started writing games in OpenGL. I worked on flight simulator displays and much more. Then, in 2005 or so, I got involved with JMonkeyEngine and used it to create games and ideas. I figured I could use these skills to eventually build the HUD/HMD I wanted.

2012 would change everything I thought about having someone else do much of my microcontroller work. Along came Arduinos and Raspberry PI. Through this experimentation with this hardware, I thought they had more power than they did, but I built my first working HUD (repo). It wasn’t impressive, but I put it together and made my wife drive me around the neighborhood while I tested and debugged it. Finally, electronics were in my wheelhouse for making prototypes.

Then, in 2014-2015, I discovered the power of OpenCV. From that point on, I’ve worked to get various HUD work done. In 2018, I discovered some open-source projects that dealt with personal assistants, and I haven’t stopped since.

At this point, all of my ideas and dreams are coming together. I’ve bought so much hardware and written so much software. When I started working on SERINDA in 2018, I had hoped to be able to get to this point faster, but I had to wait for technology. Elements like displays weren’t advanced enough. OLEDs weren’t good enough to display full-color video. SBCs weren’t fast enough, nor could they push video the way you needed. I bought commercial HUD elements to test…, but none of them worked out. I built multiple versions and finally landed on two off-the-shelf designs, allowing me to refine my work. I attempted to use LeapMotion and a project called Triton to figure out how to translate my work correctly.

I worked hard on cameras, linking everything and attempting to get positional data. I bought Intel RealSense cameras D435i and T265. Then, Intel decided to EOL the T265.

I’m very happy with my work on SERINDA so far. It’s allowed me to do things that I never thought I’d actually be able to get to.

What I didn’t realize was that when OAK-D was first released, I bought one. I put it in storage with my other XR hardware. I was frustrated with Intel cameras and attempting to link the two together, so I fired up my OAK-D, which happened to be the answer to my needs. What I found is that most of the stuff I wanted to do and had individual scripts to work on were already done with OAK-D and so much easier than I had been doing it. Now, whole sections of code can be removed because they’re useless. I can focus on perfecting my work instead of trying to figure out the baseline toolkit.

So SERINDA v3, aka Graceland, will be advancing SERINDA with the newfound tech and the culmination of all of my experiments into my dream.

If you’ll excuse me I have some work to do

Until next time. Dodadagohvi. ᏙᏓᏓᎪᎲᎢ.