In this article, we propose a VR-based immersive self-training system for volleyball beginner’s receiving, by which a virtual training environment is given to a practitioner with a head-mounted-display (HMD). Not wearing motion capture markers and haptic devices, a practitioner has free hands to receiving a virtual ball, so that the proposed system is expected him/her to be able to play receiving motion as the same as in the real world, in order to enhance skill upgrading effectively.
KEYWORDS: Visualization, Head-mounted displays, Virtual reality, Safety, Space operations, Control systems, 3D modeling, 3D displays, Visual process modeling, Video
Among characteristic club activities in Japanese fisheries high schools, demonstration of dismantling tunas and other fishes dismantling is attractive to consumers as well as to students. To improve the situation where it is difficult to provide sufficient practice opportunities due to high cost, the previous research proposed a virtual reality support system of self-study for tuna dismantling. The system provides visual sense through Head-Mounted-Display (HMD) with interactive manipulation of some kinds of knives by both hands. Implementing additional functions into the previously proposed system, this article proposes a system that enables effective repetitive practice and allows students to learn exactly dismantling procedure and operation, through the experience of the virtual world.
In this paper, we propose a system to represent a 3D virtual house based on the given floorplan images, through which a user can understand intuitively and efficiently interior furnishing condition. The proposed system adopts a learning-based approach in the use of a convolutional neural network with customized dataset, which allows us to acquire high-accuracy semantic information of diverse components of the house, e.g., walls, toilet and area of balcony, from input floorplan image. Then, constructing an interactive virtual environment with Head-Mounted-Display (HMD), the system enables the user to adjust arbitrarily the position of furniture with various sizes, and prompts whether the selected furniture can be placed in a specific position through an intuitive way of expression.
In this article, a hybrid type of virtual keyboard with multimodal feedback using a physical tabletop (or any hard plane around the user) is proposed. A user, wearing a VR Head-Mounted-Display (HMD), sees a keyboard in the given virtual world. Tracking his/her finger’s motion on the tabletop, the proposed system determines the key pressed by him/her and gives multimodal feedback to him/her. A pair of changing colors and playing sounds are given to the user as visual and audio feedback, in conjunction with physical tactile feedback from the tabletop. The system is expected to improve performance of keyboard input in the virtual environment.
A gas burner is often used in science experiments in Japanese lower secondary school education, and its correct operating procedures are included in the study content. However, students are given only inadequate learning opportunities for the operating procedures with the actual gas burner, due to time and instrumentation constraints. Therefore, this article proposes an operation learning system of a gas burner using virtual reality technology with high realistic sense to supplement the learning with the real one. The proposed system provides interactivity through a dial controller and a pair of hand controllers as an input device, and a stereoscopic HMD as an output device. The dial controller imitates an adjusting screw, so that a learner can easily get a realistic sense of turning screws for air and gas. Also judging whether the user’s operation is correct or not, the system displays necessary information on the virtual scene of the gas burner for learning to promote the learner’s understanding.
A Piano is a popular musical instrument. Development of playing piano requires frequent and suitable practice. However, the learners often have difficulty on space and time for practice due to less portable size and weight of the piano. In this article, a portable and virtual-reality (VR) based self-training system of the piano is proposed, which is installed on a standalone see-through Head-Mounted-Display (HMD). A learner wearing the HMD can see both of superimposed virtual piano keyboard and virtual his/her own fingers. According to his/her keying by the fingers, the system represents the situation acoustically. Evaluation of training outcome (number of incorrect keying) and subjective assessment shows advantages of the proposed system in comparison of a piano playing application on a tablet device.
Among characteristic club activities in Japanese fisheries high schools, demonstration of dismantling tunas and other fishes dismantling is attractive to consumers as well as to students. In spite of the extracurricular education, the school has difficulties to provide enough chances the students to practice of the dismantling due to their cost. To improve the situation, this article proposes a virtual reality support system of self-study for tuna dismantling. The system provides visual sense through Head-Mounted-Display (HMD) with interactive manipulation of some kinds of knives by both hands. Through the experience of the virtual world, students can study correct procedure with safety and accuracy of the dismantling. From the user test for fisheries high school students, the system shows its effectiveness on the learning outcome of the dismantling.
Regular evacuation drills from fire have the advantage that many people can participate at the same time. However, during training, you just walk down the corridors and passageways as usual, and it lacks a realistic feel, especially with flames and smoke. Since the number of facilities with realistic flames and smoke is limited, it is difficult to conduct drills with such realities in schools and workplaces. In this paper, we propose a system that uses mixed reality (MR) technology to simulate fire and smoke during an evacuation drill at a school or workplace. The proposed system uses a transparent head-mounted-display (HMD) to move through the real world, so that the system provides trainees with a highly realistic image of flames and smoke superimposed on the evacuation route. In addition, the appropriateness of the evacuation action is assessed by recording the route and time of the action, and the results are presented to the trainees at the end of the exercise to improve the effectiveness of the exercise.
The correct posture and procedure are essential for effective and injury-free strength training. Although it is necessary to know the differences between the postures and procedures of the trainer and the trainee, it is difficult for the trainee to know them during training without the presence of the trainer. Therefore in this article, we propose a mixed reality (MR) system which uses motion capture to recognize a trainee's posture on a HMD. Through the system, the trainee can train while looking his/her own postures on a HMD. Also the PC with the motion sensor can be placed some distance away since the HMD is connected with the PC wirelessly.
The so-called “Kanbun,” the Japanese reading of the classical Chinese text, has been studied as a type of the classical Japanese in secondary education in Japan. In Kanbun the word order is changed based on symbols called “return mark,” which is converted into a Japanese reading. This change is often so complicated that it may cause phobias in learning for junior and senior high school students in Japan. Therefore in this paper, a visualization system of step-by-step change of word order is proposed, which recognizes the return marks on the classical Chinese text and visually superimposes the converted text beside the original one, in the use of the augmented reality (AR) technology. The proposed system is applicable to any Kanbun sentences so that it is expected to contribute to improved student understanding.
This article proposes a training support system of holding chopsticks both for beginners and people with the wrong holding. Although eating with chopsticks is a part of East and Southeast Asian cultures, many people does not hold the chopsticks correctly. Since the posture of the holding is complex form in three dimensional space, two dimensional media such as textbooks and videos is insufficient for learning and training. Therefore, this article proposes a system, which acquires a trainee’s holding posture from a camera and a contactless motion sensor, and evaluates and represents the correctness of the posture in comparison of the correct one. Since the augmented reality (AR) technology is introduced into the representation, the system makes the trainee understand the difference easily.
Internet of Things (IoT) devices increases rapidly. In addition, there are more opportunities for users who do not understand security status of IoT devices to use these. Therefore, this article proposes a visualization system with mixed reality (MR) technology, which helps users understand security status of IoT devices around them. The proposed system introduces an image recognition technology into obtaining type and location of the device. Based on both of location of the devices and the relevant security information stored in database, the proposed system overlays color as security level and messages of recommended action on the devices. If the user reacts correctly, the reactions are stored as the action history for the device in the system. Then, the proposed system updates the security status of the devices based on the action history and the relevant security information stored in database. Installed on a head-mounted-display (HMD), the proposed system expresses appropriate security status of IoT devices more intuitively than the conventional visualization such as log-text or graph. Also, the system can express the vulnerabilities related to positional relation of the devices. Such intuitive and position-related representation of the security is expected even for non-expert to act adequately.
This article proposes a virtual reality (VR) system to equip knowledge and skill of the tuna dismantling, they have to learn it in hands-on style as well as in conventional lecture and self-study styles. However, high cost of tuna gives them less chances of the practical work. Therefore, a VR based system is proposed as a supplemental tool, in which the students can study and practice dismantling the tuna in a virtual space. The proposed system consists of PC, a HMD for VR and controllers for both hands. One controller is assumed to be a cutting knife, and the another is used for tracking the position of the other hand. The system encourages the user to cut the virtual tuna with the controller as the cutting knife. From position and direction of the knife when the knife is cut in, the system determines the correctness of the cutting. If the cutting is within normal range, the system represents the cutting and encourage the user to proceed. If not, the system give caution by message and/or vibration to the user and encourages him/her to re-cut in. The system has another mode called "test mode." In the test mode, the system gives no caution/hints to the user, while the system determines the correctness of the user's action (cutting). The accumulated determination is converted to score, so that the user can easily understand his/her level of achievement. Thus, the system is expected to be effective as a supplemental and continuous study tool.
Acquiring microscope operation skills has been contained by Japanese secondary education. Although many chances of observation with the microscope operation must inspire students to study deeply science, the curriculum allows short time and small budget. In this paper, a virtual reality (VR) system of self-learning for the microscope operation is proposed, which makes inexperienced students equip fundamental skill of the operation. The system, composed by a computer, a head-mounted-display (HMD) and a dial type input device, display the student a virtual microscope in the center of the virtual world. In the virtual world, the student can manipulate the virtual microscope with controllers and a dial type input device, as well as he/she can see the procedures of the operation. The operation to focus with focus knob is especially important in the procedure, so the system adopts a dial type input device to experience under the condition more similar to the reality by the sense of touch. The system displays how to operate a microscope by visual information in texts and figures and judges if user’s operation is correct by comparing result of user’s operation and the correct one stored in the system such as the position and inclination of virtual objects. If the user operates the virtual microscope incorrectly, the system represents an alert with audiovisual stimulation. The system is expected to help users to learn microscope operation skills practically at relatively low cost.
While Japanese calligraphy has been contained by Japanese elementary and secondary education curriculum, teachers in charge of the calligraphy education are not always proficient, since skill of the calligraphy is unnecessary for teachers’ license. To overcome the problematic situation, not only training of teachers but also an ICT-based support system for the students to learn the skill by themselves. Therefore, this paper proposes a system that combines a self-study support system for inexperienced people to learn Japanese calligraphy skills, and a virtual writing system for practicing calligraphy without using ink in a virtual environment. The proposed system is a kind of the augmented reality (AR) system, which consists of a computer, a head-mounted-display (HMD) and a non-contact motion sensor. Firstly, the system introduces AR technology into visualization of expert’s motion data, which is previously recorded from an expert’s action. Secondly, on the system users practice calligraphy by imitating expert’s brush motion on virtual paper. To obtain user's motion, the motion sensor obtains the position and tilt of the brush at each frame. Simultaneously, the system simulates the handwriting from these data and superimposes it on the tip of the brush using AR technology. After user’s practice, the system calculates the differences in brush position, tilt and speed between expert’s brush motion and user’s brush motion by using DP matching and encourages user to improve his/her motion, finally. This system is expected to overcome problematic situations in calligraphy practice.
For beginners of sign language, this article proposes the arm motion learning support system to analyze the difference between the example of teacher's motion and user's motion, and to urge the user to improve his/her motion. Using virtual reality (VR) system composed by head-mounted display and motion tracking hardware/software, the proposed system measures how similar between the example motion and user's motion, and gives him/her the result of analysis, followed by the motion replay as visual feedback. For the user's easy understanding, the system scales the character according to the user's physique. Also the system adopts the sight cursor input for hands-free input. Questionary investigation after user test by 21 people shows that the proposed method has the competitive advantage against the conventional learning methods (textbooks and videos) especially in interactivity including visual feedback, depth sense in posing, motivation in learning, and easy user-interface. Detection of fingers' motion should be discussed in the next step. Since heavy and/or bulky devices are unacceptable for the purpose of the self-learning, a suitable device or other methods will be surveyed.
This article proposes a self-training system of chest compression, one of the important methods of emergency life-saving, with augmented reality (AR) technology, which decreases barrier of the practical learning and contributes to improvement of the learner’s skill. The system is built on a see-through head-mounted display (HMD) with camera and computer. For a user wearing the system who practices the chest compression with a training manikin by him/herself, it provides feedback on posture, depth and rate superimposed on the HMD. Making the user easily learn the chest compression by his/herself in real-time without his/her additional actions/movements such as confirmation of his/her operation by video, the system is expected to contribute to a practical and fundamental self-learning tool of the emergency life-saving operation.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.