CESI
Question
Asked 6 September 2015
What toolkits are available for developing 3D VR UIs for general purpose applications (apart from game engines)?
3D game engines like Unity or Unreal can be easily used to develop 3D UI for VR devices like the Oculus. I don't think the game engines would be suited for general-purpose applications, where you might want to integrate lots of other arbitrary technologies. Are there popular 3D UI toolkits that you can use for developing such interfaces and that you could develop on using C++ or C#?
Most recent answer
Hello,
For developing UI in 3D VR and Unity, you can look at VRTK. It is open source. https://vrtoolkit.readme.io/
Technically, you can use the integrated UI of Unity and convert it to a VR UI by placing a VRTK script on your UI panel.
Moreover, VRTK offers a lot of integrated interaction for building application. And it is compatible with Oculus and HTC.
Vincent
All Answers (13)
Sipal S.p.A.
Every 3D game engine, typically, provides some tools for 3D UI. In my research group we use OpenSceneGraph (written in C++ language) which have a module, called osgWidget, for 3D user interfaces.
We have also published a paper on the development of a custom 3DUI interface for design review (see link 1) for immersive design review based on pie-menu concept.
In literature you can find many references on 3DUI and in particular the book "3D User Interface" (see link 2).
Conference Paper A Novel 3D User Interface for the Immersive Design Review
2 Recommendations
University of South Australia
Have you looked at OpenFrameworks? http://openframeworks.cc
There may be some 3D add-on libraries that are particularly helpful for building 3D UIs. See http://www.ofxaddons.com/
It should be relatively easy to build some nice 3DUI with just a little C++ coding.
As mentioned above OpenSceneGraph is also a very good option, and Unity3D has good plug-in support for a range of different 3DUI widgets.
1 Recommendation
Fraunhofer Austria Research GmbH
To add to the given answers, you might also consider X3D - it is an open standard for deploying 3D applications to the web (there are some standalone implementations available as well).
See e.g. http://www.web3d.org/example for some examples.
1 Recommendation
University of Rennes
Did you consider developping a plugin for one of the Autodesk's 3D modellers (Maya, 3DsMax, MotionBuilder). They provide an C++ API which you can use to integrate your stuff into the modeller.
Obviously you will need a license for the modeller. But the great advantage is that you will then benefit from all the tools (timeline, animation curves, etc) that are already integrated in the modeller.
National Polytechnic Institute
Hi!! Maybe other one is Blender (https://www.blender.org/). Blender is free and open source software. Regards
1 Recommendation
Cineca
I would recommend Blend4web https://www.blend4web.com/en/
Amaris https://www.amaris.com/
Game engines are for building ... well games, they can have uses beyond that but that's not "their intended purpose". You indirectly point out one of their major problems has they are they are not truly modular, i.e., you can't trivially replace a component or module of a game engine as you can't trivially pick up several modules (one for loading modules, one for simulating physics and so forth) and glue them in a somewhat accepted common interface.
Regardless, game engines today (at least the ones you pointed out) don't implement such components but integrate (plug-in) with existing ones.
P.S. Blender is a modeling tool that evolved to a game engine, yet still mostly used as a free alternative to tools such as maya.
University of Alabama in Huntsville
Thank you everyone for your answers!
I found openframeworks quite interesting, but since it's quite bare-bones, a lot of work in building 3D VR UIs would be sort of reinventing the wheels (Unity 5, in contrast already has a robust GUI system). The problem with game engines, like Gonçalo Amador pointed out, is the lack of control in the lower level operations like loading/streaming assets, custom rendering, etc. I do not have experience with the plug-in mechanism in Unity and what it allows/does not allow one to achieve, but it looks worth exploring.
Btw, has anyone had experience with the Oculus SDK for Windows?
University of Geneva
Hello Sabin,
In my opinion, there are two shades in your question:
1. Is it possible to use game engines (which have 3D UI components, but lack a complete 3D UI framework) to build general purpose application ?
2. Are there existing integration of 3D UI toolkit in C++ or C# to build a general purpose application ?
Correct me if am wrong, but I will try to answer these questions.
1) Yes. Especially for virtual reality, immersive or not, a game engine is the best solution by far. What is a general purpose application in virtual reality ? A game without score, laser and boss ?
By using Unity or Unreal, you can achieve what you want. Both provide 3D UI features, in terms of 3D components in the virtual space. But there is no specific integration or structure such as a complete 3D UI menu for a HMD with only head-tracking... You will have to find a plugin for this or build it yourself.
2) I cannot point you to a 3D UI toolkit. This is a really interesting topic, which will gain more and more attention. As stated in the previous answers, there are some solutions, but I do not know them.
My advice is to go with game engine and find or build a 3D UI solution.
Best regards,
Naëm
2 Recommendations
Universidade da Coruña
We have developed several projects using OpenScenegraph
like the Empty Museum, that you can check at: http://videalab.udc.es/en/virtual_reality
1 Recommendation
University of Liverpool
Hello Sabim,
I would agree with Naëm's post. Game engines are capable of producing general purpose applications. We have been using them for this purpose for some time now. In fact the great thing about them is that they allow some tasks to be completed quickly and relatively easily using blueprints (In UE4) and for more complex tasks coding can be incorporated. They have the benefit of offering higher quality rendering and shaders, easier character integration and better animation tools. I would also recommend both Unity and UE4 for VR development.
Hope that helps.
Simon
1 Recommendation
University of Alabama in Huntsville
Ok, I've toyed a bit with Unity UI (which is really easy to use). Seems like game engines are the way to go. Unity UI can be used in VR with a little adapter code, as explained in the Oculus blog (see link).
CESI
Hello,
For developing UI in 3D VR and Unity, you can look at VRTK. It is open source. https://vrtoolkit.readme.io/
Technically, you can use the integrated UI of Unity and convert it to a VR UI by placing a VRTK script on your UI panel.
Moreover, VRTK offers a lot of integrated interaction for building application. And it is compatible with Oculus and HTC.
Vincent
Related Publications
1 In the last few years, Virtual Reality (VR) is assuming a prominent role and is recognized as a pivotal technology in various sectors. However, transmission of immersive videos produced by real-time 360°c ameras or stored on remote servers to reproduce 3D environments, or streamed by video games accessible through headsets, would require a lot of...
In the last few years, Virtual Reality (VR) is assuming a prominent role and is recognized as a pivotal technology in various sectors. However, transmission of immersive videos produced by real-time 360° cameras or stored on remote servers to reproduce 3D environments, or streamed by video games accessible through headsets, would require a lot of n...
Recently, Virtual reality (VR) technologies have started to be used increasingly in the field of education, as in many other fields. With the widespread use of virtual reality applications, there is a need to investigate the effects of virtual reality applications in the field of education. The results obtained from these researches can contribute...