Digitally Enhance Your Workforce - Theorem Solutions

Digitally Enhance Your Workforce

Engineering and Manufacturing

Digital Transformation and Industry 4.0 are terms that are currently being spoken about a lot. New roles have been created in organisations with these terms forming part of job titles.  But what does an Industry 4.0 or digital transformation person actually do?  What is Industry 4.0? And surely, the world is already digital, isn’t it? People already have access to smart phones, virtual reality, have more computing power than it took to put a man on the moon on their wrists, and even a smart refrigerator!  How much more “digital” is there?

Let’s look at Industry 4.0, or the 4th Industrial Revolution, in relation to the process of manufacturing.  Whilst the ‘smart’ products just mentioned exist, the processes that create many everyday goods is still very 20th century.

The first industrial revolution introduced mechanisation through the use of water and steam powered equipment.  The second introduced mass production and assembly lines that were powered by electricity, whilst the third introduced computers.  The fourth builds on this with concepts such as a “smart” or “connected” factory.

The smart factory uses technologies that rely on “big data” (complex information that traditional data-processing applications are unable to manage effectively) to connect machines.  Through the implementation of technologies such as Block Chain, IoT (Internet of Things) and AI (Artificial Intelligence), computers and automation enhance the manufacturing process with smart and autonomous systems powered by data and machine learning.

Smart factories use the internet and the cloud to connect the real and virtual worlds, resulting in the seamless integration of cyber-physical systems at all levels, which allows the whole production process to be monitored.

A cyber-physical system is another new term that will soon become a common phrase in relation to Industry 4.0 and Digital Transformation. It is a system that integrates cyber components (namely, sensing, computation) and human users, connecting them to the internet and to each other.

Cyber-Physical Systems

When computers were introduced, they were highly disruptive due to the addition of an entirely new technology.  Now, and into the future, computers are connected and communicate with each other to make decisions without human involvement.

A combination of the technologies mentioned above make Industry 4.0 possible and the smart factory a reality.  As a result of the support of smart machines that constantly get smarter (as they get access to more data), factories will become more efficient and less wasteful.

Whilst machines can cope with all of this “big data” these factories are still today overseen and operated by people.  The average person cannot compute this level of information, so processes need to be implemented to humanise what is going on around us. 

The human brain works in an entirely different manner to a computer, and in order to gain the best value from the necessary information, it needs to be placed in context with its natural environment.

Smart factories use technologies that rely on "big data" to connect machines

This is where the value of extended reality (XR) comes into a digital transformation strategy.  This is how to integrate the human-being into 21st century processes!

Theorem Solutions’ earlier white paper “Closing the cognitive gap” explores this in detail.  However, to summarise this in just a couple of lines, the human brain works in three dimensions.  Accordingly, it is unnatural to expect the brain to process 3D information that is not in context and at full scale.  Shrinking information on to a two-dimensional display creates cognitive load, which is the mental effort required to process any given type of information.

So, in order not to decrease the productivity of a digitally enhanced manufacturing process, organisations need to “digitally enhance your human workforce”.

This can be done via extended reality (XR) the umbrella term for augmented, mixed and virtual reality.  It depends on the use-case, which of these extended realities is most appropriate.  For example, to design a smart factory layout, Virtual Reality may be the most appropriate. 3D models of all the equipment required to design the layout can be brought into a virtual building, to ensure that it will all fit, and that maximum value is gained from the given space.

A good example of Mixed Reality would be to holographically project (via a headset like the Microsoft HoloLens 2) the virtual object into a real world setting, to either check that something will fit into a space, or that it will interface with the equipment around it.

Augmented Reality helps provide additional context to the real world.  For example, overlaying CAD and its associated meta data onto a real-world assembly.

All of these do the same thing; they provide 3D content in context with the environment in which they are to be used, in a manner that the human brain is comfortable with.

Leonardo da Vinci once wrote that a poet would be "overcome by sleep and hunger before [being able to] describe with words what a painter is able to [depict] in an instant."  This is considered to be one of the origins of the phrase "A picture is worth a thousand words."  Which, applying a 21st century interpretation to, “images in context are far more easily understood than data that is not”.

Until such time as machines can be relied upon to operate completely unattended (assuming that this is something that is actually desirable; science fiction, such as The Matrix and Terminator movies, give the impression that this is highly unadvisable) there will be this need for a cyber-physical relationship, and in order to bring people onto the same playing field as the machines let’s make it easier to consume information!

Before rushing out to buy head-mounted displays, there are some things that organisations need to think about. First and foremost, a viable use case that warrants a digital overhaul needs to be identified before any other steps are taken.

It is commonplace today for businesses to set up their own POC (proof of concept) project within an in-house department that will start to explore this.  The great thing about this methodology is that something tangible can be set up really quickly, particularly as games-engines from Unity and Unreal Engine lend themselves to this task.  However, users should be mindful of what they are trying to achieve, as it is easy to create technical debt by doing this.

The problem with the POC is that invariably it will not be scalable.  It may work for the one particular project, but to roll out the XR capability, teams may find themselves having to do a lot of re-work.

Therefore, the decision of whether to do a POC with an in-house development team, is a significant one.

Can you reuse the digital assets that were used to design the product?

Furthermore, it is important to consider where the digital assets come from, and how will they get into XR?  There are many fantastic examples of really high-end video content on the internet and social media showing the art of the possible, but many of these have spent days or weeks in graphics studios creating something that looks really good, but below the surface, the content is somewhat lacking.

Often the graphics may have been carefully created by a graphic artist or 3D modeller in a product like Autodesk’s 3DS MAX, which is fine for creating a promo-video, but not so helpful for creating a scalable enterprise-grade process.

Therefore, having a process-and the underlying technology-that will allow users to re-use the digital assets that were used to design the product in the first place, will definitely help.

Earlier in this paper, the concept of technical debt was introduced. Technical debt describes what happens when software development teams take actions to expedite the delivery of a piece of functionality, or a project, which will later need to be re-worked. In other words, it’s the result of prioritizing a quick delivery over a perfect solution.

The question is therefore: use in-house development, or purchase an off-the-shelf solution?  The answer to this probably lies in the initial question. What is the use-case?

If creating a one-off experience for one product, then doing something in-house is not a bad choice.  But to create a solution for reviewing the design of many products, or one product over many iterations, then having a process that can read the source 3D CAD and prepare it for XR, is an absolute must.

It may be that a business actually needs a hybrid approach.  In some instances the off-the-shelf product which has been made by experts in 3D CAD is the right way to go, but for other use-cases, as long as 3D data can be loaded into the games-engine, then building an experience in house might be  a better alternative.

But, there is another thing to consider; many businesses probably already have a Product Lifecycle Management process.  How will that work with XR?

XR, The Disruptor to PLM

Many organisations will have had a robust and well-established product lifecycle management process in place for many years.  Designs are created, checked in, and out, as they are developed with all of the associated metadata being carefully tied together so throughout the products life, everything is kept where it should be.

But along comes this potentially disruptive force that is XR.  The PLM manager is carefully controlling all the data that relates to the company’s products, and then a new department is asking for the CAD models which they want to take outside of this process.  Moreover, they intend to create new digital assets from this CAD for use in an XR experience!

The challenge to the PLM manager now, is how can they possibly be in control of this?

Product Lifecyle Circle: Design, Manufacturing, Distribution, Customer, End of Life

There are solutions that can connect to the PLM system and “check-out” the CAD model so that other users are aware that the product (or an element of it) is being worked on.  The Theorem XR Visualization Pipeline can do just this. 

Moreover, following an immersive design review, the associated media (notes, photos, videos) can be saved and returned to the designer.  So not all is lost then?

Maybe, but unfortunately, it’s more complicated than that.  These new digital assets have been created and used for a process, and information as an output from this has been documented, but this cannot be checked back into the PLM system.  Unlike with CAD data, where there is a CAD manager that knows exactly what it is looking at, there is currently no XR asset manager.  Accordingly, the PLM doesn’t know what this new type of asset is, nor what to do with it.

The important thing to consider before an organisation starts to implement an XR solution is whether all other processes have been identified, and the appropriate stakeholders have been engaged with, before implementing something that can be considered to be highly disruptive to an otherwise fine process.

In the case of a PLM process, until such a time that someone has created an add-on that will manage your new XR assets, these will need to be managed outside of this process.

However, this does not mean that organisations should not adopt XR. The benefits of working in context and at full scale are significant, and many companies that have implemented this will point to any ROI given.  For example, one of Theorem’s customers are expected to make savings of almost £400K in the first year, giving an expected full return on investment in just 2 months.  Surely, with this level of pay-back, it is worth having a look at your existing PLM processes?

 

A piece that talks about Industry 4.0, Digital Transformation and cyber-physical systems would be amiss without talking about the Digital Twin.  This is another term that has generated lots of buzz over the last few years, and again is synonymous with Industry 4.0 language; but what is it?

A Digital Twin is a digital replica of the real-world environment, which allows users to monitor, model and simulate activities as if they were working with the real-life counterpart.  It is the twin of the real world.

Invariably, as it is a twin of the real world, it must be three-dimensional.  This means, that it must be based on some type of 3D asset, and thus originate in the world of 3D CAD.

A digital twin, is however, more complex than just the 3D data because of it’s metadata.  It requires a link to the other Industry 4.0 technologies that were mentioned earlier in the paper.  For a digital twin to be fully effective it must respond in the same way that its real-life counterpart would.  Accordingly, links (most probably via webservices) to IoT and/or Blockchain technologies is a necessity.

For example, if there is a sensor in equipment that alerts the user to an issue, then this should also be identifying the same issue in the digital counterpart.  This is important if the maintenance crew are off-site, so they can see in the digital twin what the problem is.  This could be incredibly useful in something that is usually unmanned, such as an offshore wind turbine.

 

By having a digital twin that alerts users to a problem, engineers can look to see how to resolve the issue before starting to work on a task.  This means that the correct tools, and any necessary spare parts can be organised beforehand. 

Moreover, if this is something that may require rehearsing, then the digital twin can be used to achieve this.  Unlike the real asset which may be in a hazardous environment (such as a wind turbine in the North Sea), then the fact that the engineer is fully ready is hugely important. Not only will this prevent potentially unnecessary repeat visits, but it also means that staff will be safe and won’t cause damage to themselves, or an organisations incredibly expensive asset.

The Connected Worker

The connected worker is one of the areas that Theorem Solutions have focused on; the Guides Experience allows the user to be trained on the maintenance, assembly and disassembly of a product in either VR or MR.  Coupled with the CAD overlay capability, Theorem can snap digital geometry to the real-life counterpart!

Research by Microsoft has found that staff who have been trained to carry out complex tasks in Mixed Reality (with the HoloLens) have been 25% more productive than those that have not.  Moreover, due to the collaborative properties of the device, they found that 80% of companies that implemented Mixed Reality said that collaboration across cross-functional teams had been improved. 

A connection to your remote workforce can boost productivity by up to 25%

There is still another way that the visualisation of a digital twin can help.  In addition to managing the asset, and training staff to work on it, digital twins can be used to provide remote assistance.  This is where an expert, who is remote to the person with the issue that needs resolving, can provide remote support. By utilizing the camera(s) on the device, assistance can be provided to the field-based user.

Theorem Solutions realised that there is lots of scope for utilizing its 30 years of 3D CAD knowledge to provide innovative solutions to assist with this new connected, digital future.  It was through the appreciation that a person needs to have their data in context and at full-scale to fully understand it, and thus to not introduce cognitive load, that led to the creation of Theorem XR.

Unlike other XR based companies, Theorem initially focused on the important matter of getting 3D CAD assets into an XR experience in a simple, and efficient manner.  To achieve this, the Visualization Pipeline was developed.  This is a server-based technology which can support drag-n-drop of 3D CAD, as well as being connected to a PLM system so it can check-out CAD models. 

The Visualization Pipeline operates by using Theorem’s CADTranslate (previously CADverter) technology which has been used by many OEMs in manufacturing to translate 3D assets from one format to another to improve collaboration throughout the supply-chain and, therefore, the efficiency of production processes.Circle with segments showing the Visualisation Pipeline's output capabilities. Lightweight Variables: glTF, JT, 3D PDF, 3D XML, STEP, and Creo View. Photorealistic Rendering: DS 3DEXCITE and Autodesk VRED. In-house XR: using Unity or Unreal Engine. Commercial Virtual Reality: CAVE and Powerwalls. Theorem's use case focused XR suite: Visualisation, Product Review, Factory Layout, Guides, and CAD Overlay

After initially thinking about how to get 3D CAD into XR, the Theorem software development team recognized that the technology is limited by the available hardware.  Head Mounted Displays (HMDs) are now very cost effective, and the image resolution and field of view are improving all the time, however the workstations to power them can be very expensive.  Accordingly, Theorem adopted the Unity Games Engine which exploits the power of the gaming range of graphics processors as these are optimised for this use, and are significantly more cost effective than the really high-end professional grade equivalents which have traditionally been the technology of choice for the Powerwall and CAVE VR user.  Adopting this hardware approach meant that the entire solution could be brought in at a much lower price-point for the consumer.

Due to the decision on the choice of technology wrapper (Unity) and the associated benefits of the hardware that this delivered, a requirement to optimize the data was identified.  In large scenes, in particular a factory layout, the user may be loading many different models which, could include a full product tree as part of the assembly.  In some instances, this level of data is unnecessary.  Moreover, the need in a design review to interrogate an assembly right down to the very last nut, bolt and washer may also be unnecessary.  Accordingly, the Theorem development team introduced features to make a more efficient experience.  Tools to collapse the product hierarchy, shrink-wrap and part-cull were introduced. 

Additionally, by having models in your scene that are too heavy (detailed) the user experience is negatively impacted.  The result is a laggy experience which is uncomfortable for the wearer; frame refresh rates become low and this can result in VR sickness (where physical motion and the rendered scene become out of sync which confuses the senses). 

In developing the Visualization Pipeline, Theorem were very keen to mitigate against this, which is why different options were created for use at the point of file conversion.  The issue today with XR is that it is a constant fight for the best compromise.  In an ideal world, all of the options would be ticked, but in reality, there is only so much computing power available, especially when using a Mixed Reality headset that does the rendering on the device.

A key element of Theorem’s CADTranslate product line is the ability to provide collaboration around a 3D asset.  Being able to translate it into a format used by the supply chain means that collaboration around the design is far more easily achieved.  However, this data, whilst 3D, is displayed on a 2D monitor.  To really benefit from the design concept a 3D approach is necessary.

It was this point of collaboration, which led to the requirement to ensure that all of Theorem XR users were able to collaborate around a 3D model, regardless of location, or chosen device.

The location element is of significant importance; with supply chains rarely being located in the same city, or even the same country, it is incredibly important that teams can collaborate around a design concept regardless of where they are.  By hosting a collaboration session on the Visualization Pipeline server, users can join a session and participate in the review regardless of location.  Users will appear as an avatar in the scene. This avatar represents the users location with respect to the 3D model and shows the other users what they are looking at.

Control can be passed to anyone on the session, not just the person that set it up, so ideas/thoughts can be shared.  Moreover, audio, video and typed notes can all be created and saved for review after the session.

80% of companies that have implemented Mixed Reality said that collaboration has improved across their cross-functional teams

The second point, which is unique to Theorem, is that the ability to collaborate is not restricted to just VR users.  Mixed Reality users (those with a HoloLens 2) can also participate, in the same session as the VR users.  These users appear as avatars in exactly the same way that a VR user does.  Moreover, those collaborating with them will also appear as avatars, located in the room wherever they are in relation to the assembly.  So, regardless of the output device, anyone can join in with an immersive collaboration session.

Moreover, if users don’t have access to a MR or VR device, then they can still join the session by desktop, however in doing so, the benefits of the session being in context and at full-scale are not realised (users can still see everything, including any collaborators avatars).

Use-case Focused Approach

Having developed the architecture to effectively manage and handle 3D assets, as well as the ability to collaborate, Theorem considered why a user might be needing to work in context at full-scale.  Unlike other XR product offerings, Theorem has looked in detail at the features that are required for each use case and provided those that are necessary, whilst omitting a full feature-set as it was felt that consumers do not want to pay for features that they will not be using. This has led to a portfolio of XR Experiences which do exactly what the user needs.

Visualization Experience

Often an individual user’s requirement is nothing more than to be able to visualize their 3D data at full-scale.  Having a host of tools that the user will rarely, if ever, need was felt to be unnecessary.  So, this Experience, was designed with the primary purpose to allow the user to look at an assembly at full scale so that the size, and interfaces with the environment around it, can be appreciated.

However, rather than just this ability, a handful of useful tools have been bundled with this application which allow the user to move or hide the components in the assembly in order to see what’s behind or inside them, as well as the abilities to revolve and dynamically section it. 

The Desktop Review (the desktop version of the Visualization Experience) comes included with the Visualization Pipeline and with certain CADTranslate products.  By having a VR headset connected to the CAD workstation, users can benefit from a full-scale review of the model, whilst sitting at a desk!

Design Review Experience

A design review is a formal process and is a frequent project milestone.  As this is a specific exercise, with set goals, Theorem provided a set of specific features with the Product Review Experience.  These are focused on providing a collaborative experience- as this is not an exercise done solo- and to document the decisions made during the review.

To document the design review, 3 options are provided; written note, video clips and images.  The ability to leave written notes in XR has always been tricky, however with tools such as finger tracking on the HoloLens 2, it has now never been easier to type in an immersive headset! 

If typing isn’t enough, users are able to grab a snapshot or record a video clip of what was discussed-which is incredibly powerful.  It makes it much easier for the designer to understand when back at their desk, being able to view an image of a partially dissected assembly, or play back the discussion.

This Experience App includes the core features from the Visualization Experience, so the ability to interact with the model, such as moving components and measuring, remains as these are often required during a design review.

Factory Layout Experience

The Factory Layout Experience provides users with all the functionality requires to optimize factory layouts through immersive visualization. Creating new layouts can be expensive and disruptive, so prototyping a concept in Theorem XR makes the process more flexible, modular, and easier to reconfigure without the traditional expense. Existing, or new, library components can be imported and repositioned using ‘grab and move’ capabilities and with a simple snap-to functionality for easy and accurate object positioning.

To look at the entire factory, it can be shrunken down to a table-top view.  This is particularly useful when using Mixed Reality as it is difficult to see something so large at full scale.

Visual Digital Twin Experience

The Visual Digital Twin app provides the ability to see the original design overlaid on top of the physical component that has been manufactured.  This allows a user to use their physical prototype, check it against the original 3D design, and report any issues.

This use case can be beneficial at any point along the design and manufacturing process, for example checking a 3D printed prototype, or as a final verification of a full production part before assembly.

The application provides object occlusion and transparency support which, in addition to looking at the 3D geometry, the user can mark parts of the model as transparent, or occlude them, to review the physical object underneath.

Visual Digital Twin works with any model that the user has uploaded to the Visualization Pipeline.  Unlike other applications this is not a one-off (or hard-baked) experience for just a single product/item, meaning that this is easily scalable and ready for an enterprise level roll-out.

Moreover, the application will run on the most common mobile platforms allowing current devices to be used without the cost of purchasing more.

Training Guides Experience

The Training Guides Experience enables production operatives to optimize production processes and accelerate production readiness through XR technologies.

Training Guides enables a user to author 2D content and link it to 3D Theorem XR content.  The content is a series of steps or stages.  Each step when played out, will display the 2D content attached to the 3D geometry.  Guides enables the processes to create work instructions or a training exercise.

Training Guides provide the 3D content at full scale and in the context of the physical parts (AR/MR), or digital context using Virtual Reality.  It also has the capability to replay animations of the work(build)/training sequence. 

Furthermore, it can also be used in conjunction with CAD overlay, where the digital geometry can be snapped-to the physical item.  This can then provide instructions with digital markers pointing at real life objects with instructions to tighten, undo, remove, replace, position, etc.

Theorem XR is a suite of Experiences for specific use-cases, with features that are required to deliver the benefits that customers need. Theorem XR has been developed to digitally enhance the workforce through providing users with the information that is required, at full scale and in context, to perform tasks in the most efficient way possible.

The solution was developed on the premise that not one CAD vendor or headset/output device is the best at everything.  Being CAD vendor neutral, as Theorem have always been, remains a core value to the way that Theorem operate. This approach has been carried over to XR devices. Theorem will work with VR headsets from HTC, Oculus and several others, and will also work with Mixed Reality devices from Microsoft.  Why? Because, the choice of output device is based around two things; the use case and personal preference.  So why should Theorem Solutions, as a software vendor, force users to choose one over another?

By starting to look into the world of enterprise XR as part of a Digital Transformation strategy, users need to consider which problem they are looking to solve and make sure that the solution can deliver the benefits that need to be achieved.  Theorem’s specialist consultants have many years of experience in helping customers do just that.

And, whilst undertaking research and analysis into the features of XR, if a video online looks too good to be true-with experiences appearing incredibly “real”- then it probably isn’t real.  Or, at the very least, scalable.  Graphic artists can create amazing scenes using rendering technologies, but often these are one-offs which cannot easily be replicated. 

However, if a digital asset has been uploaded to the Theorem XR Visualization Pipeline then it can be used in any of the experiences.  And if users change the design and upload the new CAD model, the XR Experience will be able provide the full scale, in context experience again, and again.

Training staff in Mixed Reality can boost productivity by 25%

Theorem XR is a truly enterprise solution that won’t lock users down to a particular 3D CAD system or a specific device; Theorem XR will help organisations be Industry 4.0 ready and gain that competitive edge over competitors.

Reach out to schedule time with the team

We're here to help

Talk to our XR team today. If you aren't sure where to start or which device would most suit your use case; perhaps you don't have a specific use case defined yet? We can help you to make sense of the market and what is available today with an unbiased and neutral opinion.

If you are at the next stage of your XR journey and have already chosen your use case and device, but are struggling to get existing data on to your HMD, we can demonstrate how a simple drag and drop solution can be implemented.

At whichever stage you are at in your XR journey, we are here to help.