Methodology for the Development of Augmented Reality Applications: MeDARA. Drone Flight Case Study

Associated Data

Data Availability Statement

Not applicable.

Abstract

Industry 4.0 involves various areas of engineering such as advanced robotics, Internet of Things, simulation, and augmented reality, which are focused on the development of smart factories. The present work presents the design and application of the methodology for the development of augmented reality applications (MeDARA) using a concrete, pictorial, and abstract approach with the intention of promoting the knowledge, skills, and attitudes of the students within the conceptual framework of educational mechatronics (EMCF). The flight of a drone is presented as a case study, where the concrete level involves the manipulation of the drone in a simulation; the graphic level requires the elaboration of an experiential storyboard that shows the scenes of the student’s interaction with the drone in the concrete level; and finally, the abstract level involves the planning of user stories and acceptance criteria, the computer design of the drone, the mock-ups of the application, the coding in Unity and Android Studio, and its integration to perform unit and acceptance tests. Finally, evidence of the tests is shown to demonstrate the results of the application of the MeDARA.

Keywords:

augmented reality applications, educational mechatronics, intelligent and adaptive learner interfaces, drone simulator, educational games, edutainment and gamification

1. Introduction

With the Fourth Industrial Revolution, the augmented reality (AR) approach allowed new solutions and provided systems with new intelligence capabilities. With this, the representation of information is possible without losing the perception of the real world [1].

Makhataeva and Varol investigated the main developments in AR technology and the challenges due to camera location issues, environment mapping and registration, AR applications in terms of integration, and subsequent improvements in corresponding fields of robotics [2]. Augmented reality is a technology that complements the perception of and interaction with the world and allows the user to experience a real environment augmented with additional information generated by a computer. It is developed in three phases: (1) In the first, the environment is recognized. (2) Then, the virtual information provided is processed, mixed, and aligned. (3) Finally, the activation is carried out, which is based on the projection of the virtual images. Some of the main applications of augmented reality are in manufacturing operations, design, training, sales, and services (see ) [3].

Open in a separate window

In recent years, there has been a rapid growth in the mobile device market that has allowed the emergence of new types of user–machine interaction that are very useful in three-dimensional environments through touch screens. These new possibilities, together with the expansion of computer systems and the appearance of cloud computing, have made possible the appearance of numerous online applications for the design and visualization of three-dimensional models [4].

Liu and Li, in 2021, applied this technology to aerial vehicles to carry out building inspections [5]. Using AR with semiautonomous aerial systems for infrastructure inspection has enabled an extension of human capabilities by improving their ability to access hard-to-reach areas [6]. In Liu and Li, 2021, an AR solution is presented that integrates the UAV inspection workflow with the building information model (BIM) of the building of interest, which is used to direct navigation in conjunction with aerial video during an inspection [5]. Furthermore, remote interactive training of drone flight control with a first-person view is possible through mixed reality systems. A remote trainer carries out the design and management of training scenarios in a virtual environment [7]. The effects of virtual reality, augmented reality, and mixed reality as training enhancement methods are described in Kaplan, 2021 [8].

Augmented reality has been introduced to the educational sector to generate a learning experience that motivates and facilitates interaction with industrial systems that are expensive and complex to acquire since there are institutions with little or no equipment and spaces destined for the new technologies [9,10]. There are several relevant properties and interactions of AR for educational use in learning spaces, in addition to various learning theories such as constructivism, social cognitive theory, makes connections, and activity theory [11].

The main contribution of this work is the design and application of a methodology for developing AR mobile applications that include the concrete, graphic, and abstract levels as part of a macrolearning process. With this, it is possible to generate a new teaching–learning strategy whose key element is the Experiential Storyboard (it is worth mentioning that we are the first to define this new concept) and the test of the developed AR mobile application. Experiential Storyboard establishes that the participant first lives the experience of manipulating a physical and/or virtual object with instructions given in colloquial language. Afterward, a storyboard is generated that includes everything in the user experience and that will be the entrance to the mobile application software development process, which is highly known by the engineering community and combines tools and software for the construction of comprehensive theoretical and practical learning. In addition, this application framework allows the creation of intelligent and safe work environments. This can be implemented in industries and schools that seek continuous personnel training so that they can later manipulate robots and actual machinery without causing damage to the device or damage to the infrastructure or personnel involved.

The paper is organized as follows: Section 2 presents the methodology for the development of augmented reality applications (MeDARA) based on the Educational Mechatronics Conceptual Framework (EMCF), while the application of the MeDARA to the Drone Flight Case Study is in Section 3. Section 4 presents the results of applying the methodology. Section 5 establishes a discussion on the applications of AR in education, and the conclusions close the paper in Section 6.

4. Results

In order to establish a mathematical framework, the descriptive statistical analysis of the initial contact assessment for the augmented reality mobile application test for the experimental and control group is performed. It is worth mentioning that data consider the participant’s necessary time to understand the operation of the mobile application. The t-test is used to determine if there is a significant difference between the means of the two groups. The hypotheses tested by the independent samples t-test are

H0:therearenodifferencesbetweenthemeans:μx=μyH1:iftherearedifferencesbetweenthemeans:μx≠μy

The conditions of an independent samples t-test for hypothesis testing are Independence, Normality, and Homoscedasticity.

The Shapiro–Wilk test finds significant evidence that the data come from populations with a normal distribution (see ).

Table 3

Initial Evaluation TestWp-ValNormalExperimental0.9636550.619153TrueControl0.9680420.713114TrueOpen in a separate window

Several tests allow comparing variances. Since the normality criterion is met, one of the recommended tests is the Bartlett test. shows that no significant evidence is found (for alpha = 0.05) and that the variances are equal between both populations.

Table 4

Tp-ValEqual_varBartlett9.3083280.002281FalseOpen in a separate window

Therefore, the t-test with Welch’s correction must be performed. The results are shown in where Dof is the abbreviation of Degrees of freedom.

Table 5

TDofAlternativep-ValCI95%Cohen-dBF10Powert-test−6.190927.2369two-sided1.23419×10−6[−19.37,−9.73]1.95773 3.603×104 0.999976Open in a separate window

Given that the p-value (1.28132×10−6) is less than the alpha level of significance (0.05), there is sufficient evidence to consider that there is a fundamental difference between the learning time of the application keypad of the individuals of the control group and individuals in the experimental group. The effect size measured by Cohen’s d is large (1.9577).

5. Discussion

The object of analysis presents the principles of Industry 4.0 expressively in higher education courses described in the description of the professional profile, field of activity, or curriculum and the assumptions are intertwined as a pedagogical proposal [22].

As Atamanczuc and Siatkowski (2019) point out, changes in the world of work have led to greater precariousness in working conditions and labor relations, as well as in the lives of workers. However, this is not announced in the principles of the so-called Industrial Revolution [23]. It is necessary, therefore, to reflect on the impacts of this new “industrial revolution” on the increase in productive capacity and the possibilities of emancipation or the subordination of workers.

It is possible to understand the learning itinerary as a route in which the user can learn specific material. Its approach has been expressed in terms of a guided visit of learning material [24]; a formative structure providing open and dynamic processes [25]; a guide on how students learn the content [26]; and the knowledge organizers of teachers and students and sequencing of content that fits the student’s profile [27].

Considering the impact on the students’ education, certain personal learning itineraries have been explained in various studies, which highlight different conclusive aspects such as the following: allowing the teacher and the students to have real control in the subject organization [24]; implementing learning itineraries to improve student perception of the classes [24]; using learning itineraries in a linear or flexible way, favoring the teaching–learning process [12,18]. Note that the flexible learning design requires teaching competencies and induction processes regarding the technological mediation used for students [26,28].

The implementation of robotics and computational thinking in education and the decision to include robotics and PC content is not neutral but, rather, has evident political–economic motivations, such as the following:

• Encouragement of more technical, computerized, and specialized careers (STEM careers);

• Inclusion of business in the educational system through “philanthropy”;

• Increasing incorporation of robots into society;

• Movement of capital from the public to the private sector;

• Normalization, by the education sector, of the company discourse that this “has to be so”;

• Involvement of companies, through concrete projects, in academic life.

The forms of knowledge representation used by the students to solve problems according to their cognitive style are not exclusive. They only evidence the preference for the forms of codification that, according to their dimension, generate information recall. From this perspective, it is important to emphasize that the context of the subject is technical; therefore, it favors the use of representations based on artifacts. From the point of view of navigation in the pathway, because it configures inputs, it delivers complete control to the learner, and the teacher configures their role as the mediator between the pathway and the learner [29]. Regarding the learning outcome, the study has revealed a relationship between the implementation of the learning itinerary, mediated by AR, for the mechatronics course and the learning outcomes [30,31]. Finally, it is essential to emphasize the contribution of this research in terms of scientific references that establish a relationship between the use of personal learning itineraries and augmented reality in the training of students, where academic performance is improved in addition to the research process. For future work, incorporating mixed reality and extending the applications are proposed.

6. Conclusions

Implementing the MeDARA through the three levels of macrolearning of the EMCF—concrete, graphic, and abstract—shows its effectiveness. The student was capable of developing an AR mobile application using an existing drone flight simulator app, the experiential storyboard, and the programming tools.

The final functional model was verified when implementing the tests, within which unit and acceptance tests were performed. Each of the model’s aspects in augmented reality, the buttons’ functionality, the drone’s size, and design in Unity of the drone were validated, while the acceptance tests determined if changes were made within the design.

In addition, the results show that when the initial contact assessment with the developed AR mobile application occurs, there is a real difference between the learning time of the application buttons of the control group and individuals in the experimental group. This difference means that the buttons in the AR mobile application can be improved to make it more intuitive to the users.

The present innovation used in augmented reality for education corresponds to the process type since the proposal offers a form of teaching that differs from other educational proposals. Incorporating augmented reality in the learning process is innovative because it implies a paradigm shift in how learning is approached through the implementation of the EMCF—incorporating technologies as tools that support the process of academic formation. Moreover, AR mobile applications can be used to simulate an automation process in the industry.

Abbreviations

The following abbreviations are used in this manuscript:

MeDARADevelopment of Augmented Reality ApplicationsEMCFEducational Mechatronics Conceptual FrameworkARAugmented RealityUAVUnmanned Aerial VehicleBIMBuilding Information ModelSCARASelective Compliant Articulated Robot ArmAUTOC-ARExtreme ProgrammingXPExtreme ProgrammingCADComputer-Aided DesignUMLUnified Modeling LanguagePCPersonal ComputerSDKSoftware Development KitJDKJava Development KitRAMRandom-Access MemoryMBMegabyteGBGigabyteROMRead-Only MemoryPNGPortable Network GraphicsKBKilobyteSTEMScience, Technology, Engineering, and MathematicsFPSFrames per second

Funding Statement

This work is funding by Laureate Education Inc. through the 2018–2019 David Wilson Award for Excellence in Teaching and Learning. Besides, we would like to thank the Mexican National Council of Science and Technology CONACYT for the scholarships 227601 and 338079.

Author Contributions

Conceptualization, L.F.L.-V. and Y.A.-M.; methodology, L.F.L.-V., M.A.C.-M. and R.C.-N.; software, R.H.-Q. and Y.A.-M.; validation, C.A.G.-G., M.A.Z.-A. and N.F.-V.; formal analysis, M.A.C.-M.; investigation, R.H.-Q., L.F.L.-V., M.A.C.-M. and R.C.-N.; resources, Y.A.-M.; data curation, R.C.-N., N.F.-V. and C.A.G.-G.; writing—original draft preparation, R.H.-Q., L.F.L.-V., M.A.C.-M. and Y.A.-M.; writing—review and editing, M.A.Z.-A., N.F.-V. and R.C.-N.; visualization, R.H.-Q., M.A.C.-M. and R.C.-N.; supervision, N.F.-V. and C.A.G.-G.; project administration, M.A.Z.-A. and Y.A.-M.; funding acquisition, M.A.Z.-A. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.