Issue No. 14

This week's newsletter is more of an experience report of something I've been trying on and off for a year.

I've been experimenting with working in virtual reality!

After reading "Working From Orbit - VR Productivity In (or Above) a WFA World", I was super intrigued by the idea of working in VR.

So I went crazy and bought an Oculus (now Meta) Quest 2 headset.

Here is my experience report 💬.

Getting started with working in VR

I tried many apps to get my computer screen into my VR headset. After trying most of them, Immersed has turned out to be the best for my use case.

Immersed even has Linux support. However, using it on Linux was a hassle, and it works better on my Mac.

Things were pretty sucky initially because it was hard to see my keyboard and trackball in VR. Immersed had a solution that sort of worked - they could render your keyboard in VR by getting you to press different keys on it and then detect where the keyboard was. But, unfortunately, that didn't work well for me because I use an ortholinear keyboard.

You would think this wouldn't have been an issue for me because I can touch type, but I occasionally glance down at my keyboard. You can try this for yourself. Just put a tea towel over your hands on your keyboard to see what the experience is like for you. Not really usable, IMHO.

So I stopped using the app for a while, and most of my VR  time was spent playing games.

There were other issues with my experience.

Being in VR takes time to get used to. Honestly, my face was red at the start, my eyes didn't feel very good, and I had to take regular breaks. However, things are much better now that I'm used to VR, and I've found a good head strap (see tips below). I can sustain much longer sessions in VR because the headset isn't being pressed against my face as hard as it needed to be before to have a good fit, with other straps to get a good fit.

The Immersed app was buggy. I spent most of my time in VR yak shaving, trying to get the screens set up just so, then something would happen, and I would lose my setup. So frustrating. Immersed is a small startup, and I wonder whether it will continue to exist. Thankfully they continue to make improvements. (I still worry for their future as they tried to pivot to sell "virtual land" in their app - that move feels pretty bizarre to me!).

The resolution isn't excellent. I've done some hacks (see my tips below) to make things usable. Still, even then, the resolution of my virtual monitors is not super amazing.

Hand tracking didn't work well when I first started out. Trying to move a screen around in VR was a pain. So I went back to the controllers. But then I was regularly fumbling around for the controllers to change something. It wasn't ideal.

The app drains the battery of my Quest super quickly, so my sessions were short.

So does working in VR suck?

If we were to stop there, I would simply tell you that VR sucks and isn't ready for work unless you are super motivated to stick at it.

Then Meta released the passthrough API that allows you to create little windows into the real world. This upgrade finally let me see my keyboard and trackball perfectly (as well as my cup of coffee or a drink).

Gamechanger.

Meta's improvements to the Quest software haven't stopped there either. Hand tracking is way better, and they seem to be moving forward with improvements with every release that makes a difference. Although, as a user, I always wish they went faster!

With each improvement in the Quest's software and the Immersed app, I've found myself going back and trying it more often.

I can concentrate better in a spaceship circling the earth than at my desk in our spare room. I can now type away for a couple hours on a newsletter post in the evening with some chilled beats in the background and feel less distracted because of the environment that I'm in.

My tips to get the most out of working in VR

The experience with a stock Quest 2 can be significantly improved with a few modifications.

Here are my tips to get the best out of working in Immersed on the Quest 2:

• Change the facial interface - I have this one (affiliate link) from VR Cover.
• Change the head strap - I think the BoboVR (affiliate link) is the most comfortable and has a battery that allows you to significantly extend your sessions.
• Hardwire your computer to your network (you should be doing this anyway, as it makes a massive difference for video calls).
• Get a WiFi6 wireless router (ideally 6E so you can be fully future-proof).
• Get SideQuest and tweak your settings. For example, I've turned my CPU and GPU to the max and turned off Fixed Foveated Rendering Off. These settings seem to give me a better experience.
• I tried having lots of screens around me, but I found the whole experience to be laggy with extra screens, so nowadays, I just have one (sometimes two or three) in VR. If you try this yourself, you can try different approaches to see what works best.
• ptom or Paul Tomlinson, who wrote the article that got me into all of this, also has good tips in his article, and the Immersed Discord is a good place for recommendations.

These changes allow you to have the most comfortable and least laggy experience using Immersed.

Would I recommend this to most people? No!

Would I recommend this to fellow geeks who want to experiment and try new ways of working? Sure!

I took a recording towards the end of writing this article so you can see my experience while I've been drafting this article while seated in my spaceship that is circling the earth.

As you can see from the video, hand gestures aren't always perfect (gesture recognition seemed worse while I was recording the footage than usual). Also, you need to resize your screens when you change the environment (Immersed allows you to have different presets). However, I have to say I really like the spaceship environment, and I don't really change much. However, I also like the cave environment (not in the video).

So things are not perfect, but they are usable!

Things that still suck for the basic solo Immersed experience:

• Joining a zoom call from VR - Immersed has a virtual webcam that shows your avatar from any angle in VR, but it is a poor experience. Hand gestures come across on the video, but the technology isn't there for eyes or facial expressions, so it is a bizarre experience for everyone else to talk to you. I've tried it only once because the experience is too odd for other folks.
• Multiple screens - your mileage may vary. Still, I found many screens to be laggy and slow, so I prefer to use only one screen (similar to my regular desk setup these days).

Is the future of work in VR?

It might sound weird to say this, but I think more and more of us will end up working in VR and AR (Augmented Reality). VR is crossing over into the mainstream, and it will enable better ways of working together in the future.

Most of this article has been about my solo experience working in VR. VR could give remote workers the sense of community and connectedness currently missing when you aren't in an office face-to-face with your co-workers. VR has the potential to bridge this gap. We aren't there, but I can see a future where we regularly interact in "the office" over VR. VR has the potential to allow us to have workshops as intense and creative as the ones I've experienced when you get a team of folks around a whiteboard with a bunch of stickies. We just need the hardware and software to develop enough to allow that.

I don't think we will all be in VR 100% of the time, working and playing together. However, I believe we will start spending more time working in VR (and AR) as technology makes it a more accessible experience.

Some things that still need improving to make this fully accessible:

• A high-speed link with almost no latency between my computer and my VR headset. It took a while (and a router upgrade) to get my setup to where it is (around 4 milliseconds latency). A wireless dongle would be great (spoiler alert, there is one on its way).
• Higher resolution displays and much upgraded graphical rendering - honestly, the resolution in VR is nothing compared to my laptop's screen. I have to move my head around a lot because the way these headsets work, only the centre of where I'm looking is rendered in detail.
• Full-colour passthrough in high definition - currently, we only have black and white passthrough, which is not ideal.
• Automatic IPD adjustment - IPD is the distance between your eyes. Quest 2 only has three settings. I've never been convinced that the setting I have mine on is perfect. The IPD of my eyes sits in between the settings.
• Eye tracking would make a big difference for presence and rendering where I'm looking in higher resolution.
• Face tracking - like eye tracking, I think face tracking plus photorealistic avatars will be a game changer because you won't have to leave VR to join a zoom call.

And a lot more, but the list above will get us to a place where I think we can consider giving out VR headsets instead of monitors for employees who prefer to work in VR.

The Quest Pro is just around the corner, and it looks like it will address quite a few of the items on my list above, so I'm excited about what the future has in store for working in VR.

That is all for this week folks, have a great week!

Jeremy (he/him)

Don't ignore your dreams, don't work too much, say what you think, cultivate friendships, and be happy.

PS Yes this issue of my newsletter was written in a single three hour VR session!

📌 Follow me on LinkedIn and Twitter.