NVIDIA has partnered with GE HealthCare to drive innovation in autonomous imaging, with a focus on advancing AI-powered X-ray technologies and ultrasound applications.

Building autonomy into systems like X-ray and ultrasound requires medical imaging systems to understand and operate in the physical world. This enables the automation of complex workflows such as patient placement, image scanning and quality checking.

To accomplish this, GE HealthCare, a pioneering partner, is using the new NVIDIA Isaac™ for Healthcare medical device simulation platform, which includes pretrained models and physics-based simulations of sensors, anatomy and environments. The platform accelerates research and development workflows, enabling GE HealthCare to train, test and validate autonomous imaging system capabilities in a virtual environment before deployment in the physical world.

“The healthcare industry is one of the most important applications of AI, as the demand for healthcare services far exceeds the supply,”

said Kimberly Powell, vice president of healthcare at NVIDIA.

“We are working with an industry leader, GE HealthCare, to deliver Isaac for Healthcare, three computers to give lifesaving medical devices the ability to act autonomously and extend access to healthcare globally.”

Expanding Access to Imaging With Physical AI

Ultrasounds and X-ray are the most common and widely used diagnostic imaging systems, yet nearly two-thirds of the global population lack access. Enhancing imaging systems with robotic capabilities will help expand access to care.

NVIDIA and GE HealthCare have been working together for nearly two decades, building innovative image-reconstruction techniques across CT and MRI, image-guided therapy and mammography.

“GE HealthCare is committed to developing innovative technologies that redefine and enhance patient care,”

said Roland Rott, president and CEO of Imaging at GE HealthCare.

“We look forward to taking advantage of physical AI for autonomous imaging systems with NVIDIA technology to improve patient access and address the challenges of growing workloads and staffing shortages in healthcare.”

Isaac for Healthcare Closes Gap Between Simulation and Reality

NVIDIA will also support other customers with Isaac for Healthcare for use cases including simulation environments. Simulation environments enable robotic systems to safely learn skills in a physically accurate virtual environment for real-world situations, such as surgery, that would otherwise be impossible to replicate.

Isaac for Healthcare is a physical AI platform built on NVIDIA’s three computers for robotics: NVIDIA DGX™, NVIDIA Omniverse™ and NVIDIA Holoscan. It includes AI models fine-tuned for healthcare robotics that can understand, act and see using enhanced vision and language processing. It also has a simulation framework for developers to accurately simulate medical environments and provides seamless deployment on NVIDIA Holoscan, an edge AI computing platform, to power robotic decision-making in the real world, in real time.

Simulation options for medical sensors are often limited. With Isaac for Healthcare, developers can now access physics-based digital twins of medical environments, allowing them to import custom sensors, instruments and even anatomies to teach robots how to respond to various scenarios. These virtual environments help close the gap between simulation and real-world implementation, and enable rapid digital prototyping.

Isaac for Healthcare allows for multi-scale simulation ranging from microscopic structures and surgery suites to full hospital facilities. Easy policy training in simulation allows robotic systems to learn how to respond in various medical scenarios in the operating room, and how to best support physician decision-making and patient care.

Healthcare Robotics Ecosystem Rapidly Expands

Isaac for Healthcare can help speed the development of robotic healthcare solutions by simulating complex medical scenarios, training AI models and optimizing robotic applications like surgery, endoscopy and cardiovascular interventions. Early adopters include Moon Surgical, Neptune Medical and Xcath.

Isaac for Healthcare is enabling ecosystem partners to seamlessly integrate their simulation tools, sensors, robot systems and medical probes into a domain-specific simulation environment. Among early ecosystem partners are Ansys, Franka, ImFusion, Kinova and Kuka.

Issac for Healthcare is now available in early access.

Vision Pro allows users to interact with digital content in a way that creates the feeling that it is physically present in their space. Vision Pro’s breakthrough design incorporates an ultra-high resolution display system that features 23 million pixels across two displays, and Apple’s custom silicon in a unique dual-chip design to ensure every experience feels like it’s happening in front of the user’s eyes in real time.

“Today marks the beginning of a new era for computing. Just as the Mac introduced us to personal computing, and iPhone introduced us to mobile computing, Apple Vision Pro introduces us to spatial computing”,

said Tim Cook, Apple’s CEO.

“Creating our first spatial computer required invention across nearly every facet of the system. Through a tight integration of hardware and software, we designed a standalone spatial computer in a compact wearable form factor that is the most advanced personal electronics device ever.”,

said Mike Rockwell, Apple’s vice president of the Technology Development Group.

Apple Vision Pro is changing the way users interact with their favorite apps, capture and relive memories, enjoy TV shows and movies, and connect with others on FaceTime.

Spatial computing makes new types of games possible with titles that can span a spectrum of immersion and bring gamers into all-new worlds. Users can also play over 100 Apple Arcade games on a screen as large as they want, with incredible immersive audio and support for popular game controllers.

A user’s world can transcend the size of a physical room with landscapes that can help them focus or reduce clutter in busy spaces. A twist of the digital crown allows the user to control how present or immersed they are in the environment.

One of the most interesting features is Optic ID is a new secure authentication system that analyzes a user’s iris under various invisible LED light exposures, and then compares it to the enrolled Optic ID data that is protected by the Secure Enclave to instantly unlock Apple Vision Pro. A user’s Optic ID data is fully encrypted, is not accessible to apps, and never leaves their device, meaning it is not stored on Apple servers.

Where a user looks stays private while navigating Apple Vision Pro, and eye tracking information is not shared with Apple, third-party apps, or websites. Additionally, data from the camera and other sensors is processed at the system level, so individual apps do not need to see a user’s surroundings to enable spatial experiences. EyeSight also includes a visual indicator that makes it clear to others when a user is capturing a spatial photo or video.

In a recent GQ profile, Cook reaffirmed his belief that incorporating digital elements into the real world can improve communication and connection between people.

He stressed that AR should be used to bring digital elements into the real world, not to bring people into digital spaces. He still maintains his view that people should not be encouraged to “live their whole lives” in a virtual world.

The vision laid out by Cook in the profile coincides with the launch of Apple’s new mixed reality headset, which will reportedly cost around $3,000 and prioritize “collaborative presence.”

The headsets will allow users to see and interact with digital elements in the real world. Cook explained to GQ that Apple is in no rush to develop the headset because the company wants to control the underlying technology to ensure innovation.

Cook also highlighted Apple’s commitment to environmental sustainability and its efforts to help people have better relationships with technology.

“My philosophy is that if you’re looking at the phone more than you’re looking into someone’s eyes, you’re doing something wrong.”

Cook said.

In the age of mixed reality, Apple aims to provide tools to help people spend more time in the real world and do more real things instead of spending their lives in the virtual world.

The tech giant has the mission and desire to differentiate itself from other tech companies that prioritize time spent staring at a phone or computer screen and digital experiences over real-world interaction. The company’s upcoming mixed-reality headset represents a significant opportunity for Apple to demonstrate how technology can enhance, not diminish, human connections.

Apple was expected to announce its mixed reality headset at this year’s WWDC conference. However, according to some reports, the company may not be ready for that just yet, and employees have expressed concern about compromises with the hardware and that the economic environment could lead to muted reception rather than the “iPhone moment” Apple is hoping for.

Apple is working on a second headset project that is designed solely for AR and will be similar to standard glasses that a user can wear throughout their day to complement the world around them. This project is still in its early stages and is expected to take at least a few years before it becomes technologically feasible.

The Snap Lens studio incorporated Ray Tracing for developers last April to improve the immersion and realism of AR content. The latest feature for Snap Lens will provide developers with advanced light reflection tools for virtual content, creating super-realistic digital assets.

These will allow users to add extra realism when viewing objects such as jewellery, clothing, machinery, furniture and more. People around the world use Snap Lens for a number of use cases, including trying on augmented reality clothing, shopping, filters, leisure and gaming, and more.

The world’s first to use the technology, Tiffany & Co, has started using Snap’s Ray Tracing in its Snapchat-based Tiffany Lock Lens. Potential customers can use the AR filter to try on the firm’s lockable bracelets and purchase items without leaving the app, XR Today further reports.

According to NVIDIA, ray tracing simulates lighting effects in virtual scenes and objects with cutting-edge rendering tools. These accurately display shadows, lighting, light refraction, and reflections for people using digital assets and immersive content.

Lighting adjusts to the field of view (FoV) and the angle of the viewing position in a headset or device. Depending on the positioning of real-time 3D (RT3D) assets and scenery, ray tracing will change the lighting from their respective sources in the environment.

Ray tracing tools from NVIDIA, Epic Games, and Unity Technologies also use ray tracing processes to boost efficiency rather than monitoring all of the light trajectories from their respective sources.

The most frequent use cases for ray tracing take place in film, television, and video games. Additional sources include the architectural, engineering, and construction (AEC) industries.

It’s currently only available for the Unity engine, which is widely used in VR. Support is planned for Unreal Engine, Wwise, and FMOD.

Presence Platform
The Presence Platform is a collection of development tools and programming interfaces that enable hands and voice interaction and augmented reality capabilities with Quest 2 and Quest Pro, for example.

Applications of the new immersive audio features include virtual reality, augmented reality and mixed reality. For the latter, Meta’s Quest 2 and Quest Pro VR headsets add computer graphics to the video image from their front-facing cameras.

In addition to Meta devices, the new audio SDK supports “almost any standalone mobile VR device” as well as PC VR (e.g. Steam VR) and third-party devices.

New features
New features include better handling of the head, outer ear, and torso filtering effects that greatly affect sound in the real world: The Head-Related Transfer Function (HRTF) is designed to mimic authentic audio perception accurately. Without it, sounds in your immediate environment will sound unnatural.

The Spatial Audio Rendering and Room Acoustics features build on the previous Oculus Spatializer and will continue to be developed. The system is much better suited for use in VR than the built-in audio systems in popular game engines, which are primarily designed for consoles and PCs, Meta said in its developer blog.

The new Audio SDK offers both flexibility and ease of use. According to Meta, even developers with no audio experience will be able to mix audio, which is essential for immersion.

What else we need to know
The previous Oculus Spatializer will continue to be supported in Unreal Engine, FMOD or Wwise or for those who prefer a native API solution. Meta does not recommend upgrading in these areas yet.

For new projects in Unity Engine, Meta recommends using the new “XR Audio SDK” to better maintain applications in the long run or to try out experimental features.

The report likens Apple’s tool to Minecraft and Roblox, which allow anyone to easily create 3D tools and worlds. Apple will reportedly use technology it acquired from Montreal-based startup Fabric Software in 2017, and customers will be able to distribute the AR apps they create on the App Store alongside developers.

“With the software tools, Apple hopes that even people who aren’t proficient in computer code will be able to tell the headset via the Siri voice assistant to create an AR app, which can then be made available for download from Apple’s App Store. The tool could, for example, allow users to create an app with virtual animals that move around a room and over or around real objects without having to design the animal from scratch, program its animations and calculate its movement in a 3D space with obstacles.”

The report notes.

Apple already offers some AR creation tools like Reality Composer on the iPhone and iPad that can be added to the headset.

We should bear in mind, however, that the report refers to sources that last saw demonstrations of the tool in 2021. That is, the report is referring to older information, and it’s possible that Apple’s plans have changed at least a bit in the time since then.

According to inside sources, Apple executives are emphasizing health and wellness, including suggestions for AR apps that help with meditation and exercise.

Apple’s headset is expected to launch later this year and will reportedly have an iOS-like interface with an app grid, advanced eye and hand tracking, and the ability to function as an external display for Macs.

The investment will be used to accelerate the company’s growth in Europe, the Middle East and Africa and to fund expansion into the North American market as the company leads the way in the new enterprise virtual reality as a service (VRaaS) training space.

“In the digital age, Gemba is an impactful solution to the biggest challenge faced by transformational leaders—the roll-out of powerful and effective training at scale, in a compelling and cost-efficient way,”

said Gemba CEO, Nathan Robinson.

Through its proprietary Virtual Reality (VR) offering combining software, content, and service, Gemba helps global enterprises seeking to train distributed workforces across multiple sites through highly immersive learning programs, simulated factory walks, and live training events – simply by putting on a VR headset.

“Gemba is set to change the way global organizations train their workforces across all critical learning and development areas – from interactive, on-the-job skills training and safety all the way to leadership training.”

added Gregg Hill, Co-founder and General Partner, Parkway Venture Capital.

On January 20, Microsoft announced it was shutting down an early metaverse-like VR app it acquired a few years ago, AltSpaceVR, after transferring much of the useful metaverse technology to its enterprise mixed reality tool, Microsoft Mesh.

As far as enterprises are concerned, training is probably one of the most obvious wins in the commercialisation of virtual reality. Actually practicing new skills in VR, especially hands-on skills in the field or in highly specialized fields like medicine, is likely to attract more visual learners.

In particular, Gemba seeks adoption among employers with multiple operating locations, virtual reality being a standardized, hands-on approach that doesn’t make executives hard pressed for time travel to learn.

The particular post explains that the company’s even bigger ambition now is a more open, accessible and safe form of immersion in the metaverse.

“Looking to the future, we see opportunities for VR to spread beyond consumers into enterprises. We hope to create a platform with Mesh that provides the most opportunities for everyone involved, including creators, partners and customers”

reads the post.

One of the first virtual reality social networking apps is AltspaceVR, which allows users to join 3D chat rooms, play games, watch movies and attend events using headsets and online platforms.

Microsoft acquired the firm as part of a goal to create its own mixed reality ecosystem when it was on the brink of failure in 2017 due to financial issues. The tech giant said customers can now pull their data and content from the VR platform as AltspaceVR has ceased operations.

With its Mesh platform, a tool for creating collaborative apps for users in augmented reality, Microsoft is still in the race for the metaverse.

As VR platforms develop, it’s extremely important to assess their security and see if they’re meeting the demands of their customers. This includes helping people in connecting with others who share common interests while also ensuring that the platforms they use are free of harassment and inappropriate behavior, as noted at Gizbot.

The removed AltspaceVR Social Hubs

The main reason why social hubs in AltspaceVR were created is to “help cultivate new friendships” and to introduce the platform to its users. Users were free to interact with other AltspaceVR users and  to invite each other into private spaces or gatherings.

However, Microsoft decided to shut these hubs down because of the lack of regulation which might result in harassment or unwanted attention.

AltspaceVR is about to be connected with Microsoft Family Safety in the coming months as a result of the Microsoft account requirement. This will allow parents to approve or limit access of their kids to AltspaceVR if acquired.

While social centers will be deleted from AltspaceVR, the app will require a Microsoft account to employ. Microsoft started to test account connections with AltspaceVR before one year, and it turned out that it might become a requirement very soon. Apparently, Microsoft is attempting to anticipate moderating concerns that are part of the metaverse drive.

Photo Credits: Gizbot

AltspaceVR was bought by Microsoft in 2017. The social hub hasn’t made any modifications to the service until now.

In fact a lot of specialist suggest that AltspaceVR is part of  Microsoft’s larger metaverse project, rather than a critical component of the metaverse’s future.

On the other hand, Mark Zuckerberg is still considering that Metaverse is the future of Facebook. The company plans to invest at least $10 billion in the metaverse by 2022.

One is true- the next generation of internet will be much more about open environment where users will be able to create their own metaverse worlds.

The tech giant hosted the event Wednesday on Decentraland, a cryptocurrency-focused virtual world that users can create, explore and trade in. In metaverse players are using headsets or smart glasses which allow them to live, work and play in a virtual world.

Decentraland can be accessed via a desktop browser and it’s a place where users create an avatar which they can navigate around the blockchain-powered virtual world using a mouse and keyboard.

The Samsung event took place in a virtual building called Samsung 837X that the company has built on Decentraland and is designed to be a replica of its flagship New York experience center.

CNBC and other companies struggled to find the 837X building and when we did, many of us were unable to gain access to it.

Problems in Decentraland

When creating an avatar on Decentraland, it lands in a sort of atrium where clouds appear to be gliding across the floor. There’s a round pool in the middle that has a worrying vortex in the center.

On the opposite side of the pool, there were three large boards. An ad for Samsung 837X hang on the “crowd” board. Once clicked, you’re given the option to “jump in.”

I went in too Sam, same result for me. Pretty awful brand experience let alone for a product launch pic.twitter.com/8PQbIkkBYM

— Drew Benvie (@drewb) February 9, 2022

After jumping in, you’re transported to Samsung’s little world on Decentraland and you can see the 837X building with a pizza store next door.

CNBC state, that they have noticed a large line of people at the main entrance to the building. People were struggling to get in. Some users were getting their avatars to jump on other people’s heads as they clambered to the front of the queue but it didn’t help. The doors wouldn’t open and the chatbox was again full of pleas for help.

According to rumours, a YouTuber had managed to find a way in, while a CNET journalist wrote on Twitter that they had managed to gain access by switching to the “ATHENA” server. It wasn’t immediately obvious how to do this.

After around 30 minutes of trying to access Samsung’s building in the metaverse, CNBC gave up and went back to the real world.