Specter Glasses: Guide to the Otherworld Using an MR headset at the Mizuki Museum

2022年9月
Indoor AR Cloud system using Immersal SDK & Hololens 2

Kadinche in cooperation with the Mizuki Shigeru Museum and Mizuki Production, will conduct a demonstration experiment of the “Specter Glasses: Guide to the Otherworld” using a mixed reality (MR) headset at the Mizuki Shigeru Museum (Sakaiminato City, Tottori Prefecture) from September 23rd to 25th, 2022. During the same period, visitors to the museum will be able to borrow a headset free of charge to experience the “Specter Megane’s Guide to the Otherworld ” inside the museum.

At the Mizuki Shigeru Museum, visitors can experience the background of the cartoonist Mizuki Shigeru and the world of his works. The “Yokai Megane (Specter Glasses)” is an advanced interactive museum guide that allows visitors to hear about aspects of Mizuki that are still unknown to them and to see other worlds that were previously invisible to them. The guide is Buckbeard, a Yokai master who will show you around the museum, demonstrating his knowledge of the world.

Main functions:
Spatial voice guidance: voice guidance that automatically plays when you approach an exhibit.
Extended information pop-up: Displays spatial information about yokai in the yokai cave
Photo shooting: Commemorative photo shooting of spatial contents at the Yokai Plaza.

We will use Microsoft HoloLens 2 as our MR heasets, a pair of mixed reality smart glasses that are worn on the head to enable hands-free recognition of real space and placement of 2D/3D objects in space. We also utilize Immersal SDK (software development kit), Immersal’s VPS (Visual Positioning System), as self-positioning estimation technology. This technology enables highly accurate recognition of the museum’s location and exhibits. Anttoni Vesterinen, CEO of Immersal, says: “We are very happy to be part of this incredible art project, enabling new ways of experiencing art. AR technology can open new possibilities in museum and art sector as this pilot proves.”

The purpose of this experiment is to verify the usefulness and return on investment of this next-generation guide system for indoor spaces such as memorial halls and museums by creating actual content and developing an MR system and having visitors experience it.

Demonstration Experiment Overview
Demonstration experiment name: ” Specter Glasses: Guide to the Otherworld” using MR headsets
Period: September 23 (Fri.) – 25 (Sun.), 2022, 9:30 – 17:00 (reception until 30 minutes before closing)
Place: Mizuki Shigeru Museum, 5 Honmachi, Sakaiminato City, Tottori Prefecture, 684-0025, Japan
Organizer: Mizuki Shigeru Museum
Developed by: Kadinche Corporation
Creative Direction: Domain
Technology Partners: Microsoft Japan & Immersal
Cooperation: Mizuki Production

Profile of Mizuki Shigeru
Born in 1922, Mizuki Shigeru is a manga artist and a leading authority on yokai (Japanese monsters) and yokai research. Born in 1922, he grew up in Sakaiminato City, Tottori Prefecture. During the Pacific War, he went to Rabaul, a fierce battlefield, and lost his left arm in a bombing. After being demobilized, he became a picture-story artist and later became a comic book artist. His representative works include “Gegege no Kitaro,” “Kappa no Sanpei,” and “Akuma-kun.

About the Mizuki Shigeru Museum
A museum where visitors can experience Mizuki Shigeru’s life and the world of his works. Located on the east side of Mizuki Shigeru Road in Sakaiminato City, Tottori Prefecture, the museum opened on March 8, 2003 (Mizuki Shigeru’s 8th birthday). The museum opened on March 8, 2003 (Mizuki Shigeru’s 81st birthday), and is one of the most popular tourist facilities in Tottori Prefecture, attracting many visitors. The museum features an exhibition space with familiar yokai such as Gegege no Kitaro and a recreation of Mizuki Shigeru’s workplace, making it a must-see facility even for fans of Mizuki Shigeru. This is where the yokai researcher and manga artist Mizuki Shigeru got his start.

About Kadinche Corporation
Kadinche Corporation is an XR software development and content production company headquartered at 14-23 Daikanyama-cho, Shibuya-ku, Tokyo, Japan, founded on August 8, 2008. http://xs267448.xsrv.jp/kadinche/

Meta Kabuki: Kabuki in a Virtual Space Where Real and Virtual Merge

2022年1月
First in Kabuki History: Kabuki in a Virtual Space Where Real and Virtual Merge

As a commemorative production for the opening of the Daikanyama Metaverse Studio, “META Kabuki Genji Memories” was streamed live and archived on a virtual space for the first time in Kabuki history.

This is the first Kabuki production to use virtual production (a filming technique that fuses 3DCG backgrounds and VFX with images shot in the real world in real time), and is based on the immortal masterpiece of classical literature, “The Tale of Genji”. The real-time synthesis of the virtual backgrounds of the Heian period and the live-action images of the kabuki actors creates an immersive experience that makes the audience feel as if they are in the Heian period. Kabuki actors Ichitaro Nakamura and Hayato Nakamura will take on this new dimension of Kabuki performance. Nakamura Ichitaro will play the role of the five families and also serve as the general director. Hayato Nakamura takes on the role of Hikaru Genji for the first time in his career.

In addition, the staff of E&H Production, a studio established in 2021 by Seonghu Park, known for directing “GARO-VANISHING LINE-“, “GOH”, and “JUJYUKI KARASEN”, will participate in the production as directors. This is a new type of entertainment that combines Kabuki, virtual production technology, and animation.

Development of Sword Art Online – Ex Chronicles – Online Edition

2021年12月

Kadinche Corporation, in collaboration with Sony Group Corporation, developed the basic technology for the VR event “Sword Art Online – Ex Chronicles – Online Edition” which was held from November 20, 2021.

The event was held as a VR event where visitors could enjoy heated battles with bosses appearing in the work and a wide variety of gimmicks and content unique to VR.

Kadinche Corporation developed an MMO front-end system using the Unity game engine and a back-end system capable of withstanding high loads using Microsoft Azure. These systems made it possible for multiple people to enter the VR space at the same time and enjoy contents and communication such as voice chat in real time.

Experiment of peer support and remote fitness system for cancer patients using VR technology

2020年7月

Kadinche Co., Ltd. (Headquarters: Shinagawa-ku, Tokyo; Representative Director: Soko Aoki), which develops space-time and human expansion technology, is a peer support and remote fitness system (commonly known as “VR cancer” for VR patients) that uses VR technology. In order to verify the effectiveness of “Peer Support”), we will start a demonstration experiment with the cooperation of Associate Professor Masahiko Sumitani, Director of Palliative Care Department, University of Tokyo Hospital.

About development of “VR cancer peer support”

“Peer support” is especially focused on cancer patients, where patients who experience the same illness in the medical field share worries and anxieties and support each other while sharing the wisdom and information to positively lead their lives thereafter. is spreading.
Normally, patients’ meetings and meetings by medical institutions were the main focus, but due to the recent spread of new coronavirus infections, it has become difficult to provide face-to-face opportunities, so opportunities on the Internet such as SNS and chat are increasing. I will.
However, in SNS and chat, the interaction between patients tends to be weaker than in the face-to-face type, and there are also voices who point out that they are unsatisfactory because it is not possible to conduct hands-on programs such as yoga and gymnastics.
Therefore, by making full use of VR technology, it is possible to develop a virtual space that allows you to experience athletics while allowing you to interact while maintaining anonymity while giving you the illusion that you are facing each other, even though you are participating remotely.

About “VR cancer peer support”system

You can see the introduction video of “VR Peer Support” below.

“VR cancer peer support” has two main functions.
1) Ability to interact with people who are remote (experience collaborators) through conversation and hand movements
2) Ability to perform fitness (upper limb movement) in a virtual space (3D CG)
Both functions can be executed in the same virtual space, so it is also possible to combine 1) and 2) to “exercise fitness while sharing conversations and hand movements”. In addition, the fitness is “daily change”, “stretch”, “boxer size”, “trunk”, etc. are prepared, and 3D CG space according to each fitness is prepared. The trainer’s model gymnastics displayed during fitness displayed the movements measured by motion tracking in 3D CG space.

Fitness training on the beach
Fitness training in the forest

Conducting verification tests for cancer patients

With the cooperation of Associate Professor Masahiko Sumitani, Director of Palliative Care Department, University of Tokyo Hospital, who has been supervised from the development stage, we will start a demonstration experiment at the University of Tokyo Hospital to verify its effectiveness. have become.
In carrying out the demonstration experiment, several cancer patients who were visiting the hospital were invited to participate, and participants were asked to lend an HMD (head-mounted display) to take home with them, and an exercise program prepared in a virtual space Participants are invited to participate in opportunities to interact with each other and to examine the changes from before the experiment.

Communication among the medical doctor & patients
Fitness Program

Future development

Based on the verification results of the verification test, we will further implement the functions that better meet the needs of patients and aim for practical use. We are also considering the application of this activity for cancer patients as a role model in the future in the preventive care field for other diseases, injuries, and seniors.
In addition to the above R&D, we are also focusing on the development of an “online self-fitness system” to promote the habit of exercising at home, centered on senior citizens. We will continue to make efforts to contribute to maintaining and improving the health of people in Japan.

Comment from Associate Professor Masahiko Sumitani, Director, Department of Palliative Care, University of Tokyo Hospital.

We would like to provide a solution in which patients can train by themselves.
Director, Department of Palliative Care, University of Tokyo Hospital
Associate Professor Masahiko Sumitani

In developed countries in Europe and America, including Japan, the number of patients suffering from cancer is increasing year by year, and one in two people in Japan experience cancer. Although medical treatment results for cancer have improved due to advances in medicine, various anxieties and troubles in daily life caused by having cancer may not be solved by only medical staff such as hospitals and clinics. .. In addition, patients who have been successfully treated for cancer often suffer from prolonged physical upset and anxiety.
As a specialist in palliative care and supportive care, we will be in charge of treating various problems that patients have, and in cooperation with cancer therapists, we can reasonably continue the optimal cancer treatment for each patient. I am treating with the goal of. We also provide therapeutic intervention for patients who have undergone cancer treatment.
In addition to such medical treatment at the medical field, peer support (counseling by cancer survivors), in which cancer patients support each other, helps patients with cancer that cannot be covered by medical care. Is spreading as a support for. Many peer support activities have become popular in Japan, but it’s still embarrassing to go to other patients, it is embarrassing to interact with other patients directly, I often listen to my voice.
Therefore, we have jointly developed VR (Virtual Reality) Cancer Peer Support that allows cancer patients to participate more in peer support activities. VR eliminates the physical distance traveled, allowing patients to participate in their favorite places. Also, by using an avatar (human model doll), there is no embarrassment when face-to-face, and lively communication is possible even indirectly.
In addition, the “Fitness” program we provide to cancer patients through VR Cancer Peer Support. It is known that exercise habits in daily life can reduce the risk of cancer development and recurrence, and reduce side effects associated with cancer treatment. I think it’s also true that “I don’t have it,” or “I started, but I was shaved for three days.” From the standpoint of a specialist in cancer palliative care and supportive care, I continue to have the desire to have cancer patients exercise every day, and I would like to realize that with VR cancer peer support. I am.
I hope that patients will be able to manage their own health so that they can live a healthy life while having cancer and after graduating from cancer.

3D model production of Higashiyama Zoo and Botanical Garden by photogrammetry

2020年6月

We took thousands of photos of Higashiyama Zoo and Botanical Garden (Nagoya, Aichi, Japan) with a handheld camera from a general passage and produced a 3D model of the animal house by photogrammetry. This is a joint work with Kawaguchi Laboratory , Faculty of Engineering, Graduate School of Engineering, Nagoya University.

Photogrammetry is a useful method for creating 3D models of existing buildings and spaces, although it has many constraints such as selection of appropriate objects and places, shooting conditions, software processing performance, and weight reduction of model data. The 3D model archive of historical buildings and archeological materials, including the game industry and construction industry, are also attracting attention in the academic and educational fields.

Developed software for "TSUNAGATE", a platform for IoT construction sites

2020年6月

Kadinche has developed software for “TSUNAGATE”, a platform for IoT of construction sites at Takenaka Corporation.
“TSUNAGATE” can connect various ICT tools to a cloud management system for centralized management by building a network (wireless LAN) environment at the construction site via a temporary distribution board that enables Internet connection. It is a platform realized by the cooperation of Takenaka Corporation’s original IoT distribution board and our cloud technology.
Normally, at the construction site, power for construction is supplied by a temporary power line, but by applying “TSUNAGATE” which can communicate using the temporary power line, ICT tools such as network cameras and digital signage can be applied. This eliminates the need for separate communication wiring. It also enables the realization of IoT at the construction site, such as centralized management of those ICT tools, remote control of lighting, and collection of sensor information.
We have developed “TSUNAGATE” for Cloud and software.
“TSUNAGATE CLOUD” “TSUNAGATE CLOUD” is a system that monitors and controls the functions installed in “TSUNAGATE BOX” located at the construction site.
Since it is built on the cloud, “TSUNAGATE BOX” on the construction site can be centrally managed from a remote location via a web browser.
We developed “TSUNAGATE CLOUD” by utilizing AWS IoT.

Other examples of using “TUNAGATE” (“TSUNAGATE VIEW”)
“TUNAGATE VIEW” uses power line communication (PLC), installs a camera on the ceiling and connects it to a temporary light, and the images around 360 degrees are displayed. It is a system that acquires and saves in “TSUNAGATE CLOUD”. By installing multiple units on the construction site, we have realized a “virtual site patrol” that allows you to change the viewpoint and the date and time to look around the site.
At our company, we developed software such as cloud development by utilizing software development utilizing a 360 degree camera, virtual tour, and “PanoPlazaTour”.
We will continue to combine 360-degree camera solutions and IoT technology to support the efficiency of site/construction management in the digital twin era.

8K VR Initiatives [NAB 2018]

2019年10月

Currently multi-lens 360° cameras able to shoot in 8K are available on the market. Since then it has also been possible to shoot 8K videos using single lens cameras. Kadinche Co., Ltd. has began the following 8K VR-related initiatives in collaboration with 8K camera maker Astro Design Co. Ltd. and super wide-angle lens manufacturer Internya Co. Ltd.

Monoscopic 8K VR

Using an Astro Design’s 8K camera and an internal 250° wide-angle lens, we established a production flow for shooting test images and viewing them with a head-mounted display. The equipment configuration used as of March 2018 is as shown in the photo below, but the camera and lens will change with future product development.

8K video tablet app

By using an 8K camera and a wide-angle lens, you can shoot a wide area in high resolution. For example, even if you shoot an entire baseball field, the resolution is such that individual players on the field can be seen clearly. Usually such content is displayed on a large TV screen. We thought about the ability to zoom-in on an area of interest by using a handheld tablet. We have developed a prototype system with the following configuration. A PC connected to a large-screen display plays the entire 8K video, and at the same time, it streams the 8K video to the tablet device connected locally via WiFi. The tablet can zoom in on the desired location of the video. The information (coordinates) for the zoomed-in screen area is sent to the PC as needed, and only that part is streamed back to the tablet.

Exhibited at the Astro Design booth at NAB 2018

These systems were exhibited at the Astro Design Booth (Convention Center Booth No .: Central Hall “C7515”) at NAB Show 2018, an exhibition of the National Broadcasting Corporation in Las Vegas, USA.

* About Astro Design Co. Ltd.
We are developing hardware and software solutions in the field of ultra-high-definition video technology and real-time high-speed digital signal processing technology such as 8K.
[Contact]
Astro Design Corporation Planning Department
Tatsunobu Kanamura (e-mail: kanemura@astrodesign.co.jp)
TEL: 03-5734-6100 FAX: 03-5734-6101

Providing VR SaaS solutions to Prince Hotels

2019年10月

Kadinche Co. Ltd. has partnered with Prince Hotel Co., Ltd. (Head office: Toshima-ku, Tokyo, President Shigeru Akasaka) in order to provide SaaS-type VR content management, distribution, and viewing systems.

Prince Hotel’s VR content is managed and distributed on Panoplaza – an AWS platform developed by Kadinche. The platform is accompanied by a GearVR app which is used for downloading and viewing VR contents. Panoplaza allows clients to start using the service with a minimal setup period of around a month and a monthly fee. This drastically cuts down development costs.

Using this service, Prince Hotel has created virtual tours using 360 ° videos and 360 °panoramic  photos on PANOPLAZA. By using VR capable devices (such as GearVR) it is possible to showcase all of the wedding facilities throughout Japan anywhere and anytime using VR.

Kadinche will continue to develop SaaS-type solutions for customers with similar needs for VR content management and distribution.

Areas of introduction: Sapporo, New Furano, Kawagoe, Takanawa, Shin-Yokohama, Hakone, Kamakura, Karuizawa, Otsu, Kyoto, Hiroshima
Hotel Names: Sapporo Prince Hotel, New Furano Prince Hotel, Kawagoe Prince Hotel, Grand Prince Hotel Takanawa, Shin Yokohama Prince Hotel, The Prince, Hakone Ashinoko, Kamakura Prince Hotel, Karuizawa Prince Hotel, Biwako Otsu Prince Hotel, Grand Prince Hotel Kyoto, Grand Prince Hotel Hiroshima

■ Related information
PRINCE WEDDING website (http://www.princehotels.co.jp/wedding/)

  GearVR App menu

GearVR Image

360 ° Panoramic picture

 

 

 

 

RaspberryPi VR x Cloud = surveillance camera【NAB 2018】

2019年10月

RaspberryPi VR x cloud service

We have developed a simple surveillance system using  RaspberryPi and 220 degree camera. By using RaspberryPi VR 220 Camera MOD KIT, we attached a 220 ° ultra-wide angle lens to the ultra-compact single board computer. The resulting system periodically uploads still images to the cloud service Entapano VR.

Simple surveillance camera system using a RaspberryPi 220 degree camera

RaspberryPi has a reputation for being a small and inexpensive computer. It is possible to create a simple surveillance system by attaching a 220 degree wide-angle camera to it and uploading the photographs to the Internet . Our prototype system takes and sends a photo once every 5 minutes by using the following equipment configuration.
・ Raspberry Pi RASPBERRY PI 3 Model B
・ Entaniya RaspberryPi VR 220 Camera MOD KIT
・ Entaniya RaspberryPi camera case (prototype)
・ Entapano VR (cloud service)
When it comes to surveillance systems, there is an impression that the cost of implementation is quite high due to the cost of both hardware and software. By combining RaspberryPi and cloud services, the setup becomes much easier and cheaper.

Raspberry Pi RASPBERRY PI 3 Model B + VR 220 Camera MOD KIT

Cloud service: Entapano VR


Exhibition at NAB 2018

This system was demoed at the Entaniya booth during the NAB Show 2018, an exhibition of the American Broadcasting Corporation in Las Vegas, USA.

Simultaneous 360 VR video playback system

2019年10月

In recent times , 360-degree videos and live-action VR have been used by companies for personnel training and also as entertainment content for tourists. For such scenarios, the ability for simultaneous playback across multiple head-mounted has been requested by clients. The simultaneous playback system developed by our company allows for such method of operation. The system consists of an operator’s Windows PC and multiple HMD devices such as smartphones, Gear VR and Oculus Go all connected via wireless LAN and operating synchronously.
Once the operator starts a video playback on his terminal, the video via also start on the connected HMDs. This allows for everyone to watch the same video at the same time.

Functions

  • Connection between the operator’s terminal and the HMD (requires all the devices to be on the same wireless LAN, connection to the Internet isn’t required)
  • Select a video for playback (must be stored in a specific folder)
  • Simultaneous playback on PC and HMDs
  • Fast forward and rewind (10 seconds, 30 seconds)
  • Connection status display (PC only)
  • Connect to the operator’s terminal (Smarthphone, HMD only)
  • Monoscopic & Stereoscopic view toggle (Smartphone, HMD only)
  • Viewpoint reset (Smartphone, HMD only)

Target devices

  • Smartphone VR using Android
  • Smartphone VR using iOS
  • Gear VR
  • Oculus Go

Setup

  1. Copy the video you want to play to the operator’s PC’s video folder and the Movies folder in Oculus Go’s internal storage
  2. Start the PC app
  3. Write down the “Server IP Address” at the bottom of the PC app
  4. Launch the Oculus Go app
  5. Tap the gear icon at the bottom of the Oculus Go app screen, and enter the IP address of the operator’s PC that you wrote down earlier.
  6. Wait for the Oculus Go app to connect to the PC app.
  7. If the connection fails because the PC application is not running, you can reconnect by tapping the connection button (cloud icon) at the top of the screen.
  8. If the cloud icon at the top of the Oculus Go app screen is has a check mark and the IP address of the Oculus Go device is displayed on the USER tab of the PC app, the setup is complete.

Method of operation
When you play a video on the PC app, the following actions will be reflected in the Oculus Go app.

  • The video you are playing
  • Play/Pause
  • Fast forward/Rewind

Furthermore the following operation is possible:

  • The Oculus Go app can be operated independently even while connected to the PC app.
  • When the Oculus Go app is operated independetly a gear icon that toggles the menu will be display in the lower end of the screen
  • When an operation is performed on the PC app, it will be forcefully replicated on the Oculus Go. This is done by design.

Use Cases

  • Tottori Prefecture: Use for Hoshitori Prefecture Project
  • Sakai Railway Company
  • Individual Clients