Mapping out the future of VR and AR

Mobile World Congress
Shanghai, China
30 June – 1 July, 2017

The Yu Garden with the Shanghai Tower behind

The Yu Garden ( 豫园) with the Shanghai Tower (上海中心大厦) behind. Photo by Mark Pegrum, 2017. May be reused under CC BY 4.0 licence.

After flying up from Guilin on 29 June, I managed to catch the last two days of the Mobile World Congress in Shanghai. An enormous event that brought together technologists, marketers and investors, and showcased new technologies from phones to drones and robots to cars, it also hosted a series of summits on specific themes. I spent Friday 30 June at the VR and AR Summit, where industry speakers offered their perspectives on the latest developments and the current challenges facing VR and AR.

In his presentation, What is the future of VR & AR?, Christopher Tam (from Leap Motion) argued that there are 5 key elements of VR and AR, namely immersion, imagination, availability, portability and interaction. Before the advent of VR/AR, it was as if our computing platforms only allowed us to peek at the possibilities through a tiny keyhole, but now we can open the door into a utopian world, he said.

Immersion needs high quality graphics and rapid refresh rates; imagination needs good content; but interaction is hard to measure. One way of measuring interaction is by considering human-machine interaction bandwidth. This is a fundamental factor to unlock the mainstream adoption of VR/AR and, while a lot of progress has been made on the other elements, this remains a bottleneck which the industry is currently focused on addressing. The leap from 1D to 2D computing required the invention of the mouse to accompany the keyboard. A mouse works for 2D because it allows one-to-one mapping; however, it is not sufficient in a 3D world, because in such a world we need to do more than moving, selecting, pointing or clicking. Interaction in a 3D world should be inspired by the way we interact with the real world; we should use the model of ‘bare hands’ interaction, given that this is our primary way of interacting with the real world. It is natural, universal, unencumbered, and accessible. In education, children can study in a hands-on style, with more fun and better retention; this is how children learn in the real world. In training, people can practise how to handle complex situations in hands-on ways. In commerce, consumers can enjoy the digital world and be impressed at the first try. In healthcare, we can enable diagnosis, physical therapies and rehabilitation; this moves the barrier between healthcare givers and their patients. In art and design, we can express ourselves by creating in a 3D manner with no restraints. In social relations, we can hang out and interact with friends. In entertainment, there will be easier, more intuitive controlling, and deeper immersion; users can become the protagonists in the stories we are telling, not just operating a person but becoming that person. Thus, hand tracking brings to life the advantages of VR/AR in almost all verticals. He concluded by demonstrating Leap Motion’s hand tracking technology.

In his presentation, The future of virtual reality in China, James Fong (from Jaunt China) suggested that VR is the next stage in a long human quest to experience and interact with captured and created realities; this stretches from cave art through painting, photography, gramophones, motion pictures, television and 3D films to AR and VR. He suggested that there is no need to separate VR and AR as they will merge soon. He briefly pointed out some questions of looming importance: we want Star Trek’s Holodeck or the Matrix experience, but we need to ask how this affects our humanity. Will we become isolated from each other? Will we appreciate human connections? Will we not want to leave the perfect VR/AR world?

In VR/AR storytelling, we can be part of a scripted narrative or take our own pathway through a free-form construct; engage in first-person participation or third-person observation; venture alone or interact with n-number of participants; and focus on private enjoyment or share experiences with family, friends and the world. It will however take a long time for high quality and compelling content to arrive, in part because VR will disrupt every element of content creation. We are used to third-person stories and it will take time to get used to first-person stories. We haven’t yet developed the creative language for working with VR. However, all of the major companies that run operating systems are moving to support VR natively, and this will usher in major developments.

He wrapped up by looking at the Chinese market, where there is no Google, Facebook, Amazon or Twitter, and where the market is dominated by local players like Baidu, WeChat, Weibo, iQiyi, Youku, Tencent, Alipay and WeChat Pay. Therefore a lot of international products don’t work in this country. Some challenges in China are the same as in the rest of the world (e.g., poor headset viewing experiences; market experimenting with live and 360) and some are different (VR experience centres/cafés in China keep interest high; content quality has not improved due to a lack of financing; and the camera and higher quality headset market is starting to pick up). He predicted that China could be the largest VR market in the world by 2018.

The slogan of the 2017 Mobile World Congress, Shanghai. Photo by Mark Pegrum, 2017. May be reused under CC BY 4.0 licence.

In a panel discussion moderated by Sam Rosen (ABI Research), with panel members Alvin Wang (Vive), James Fong (Jaunt China) and Christopher Tam (Leap Motion), it was suggested that 5G will make a big difference to VR/AR adoption because if processing is done online at high speed, we will be able to use much less bulky headsets with less drain on batteries. Alvin Wang mentioned that it will soon be possible to wear headsets that incorporate facial recognition and emotion recognition based on microgestures, allowing interviewers to sense whether an interviewee is nervous or lying, or teachers to sense whether a student understands. He claimed that one of the scarcest commodities in the world is good teachers, but AI technology can give everyone personalised access to the best teachers. He mentioned a project to put 360 cameras in MIT classes so that anyone in the world can join a class by high profile professors. James Fong talked about the power of VR to give people a sense of real-world events; he gave the example of being able to place viewers in the context of refugees arriving in another country, seeing the scale of the phenomenon, maybe being able to touch the boat the refugees arrived on, and thereby building more empathy than is possible with traditional news reports on TV.

In his presentation, The next big test for HMDs: Is the industry prepared?, Tim Droz (from SoftKinetic) said the aim of VR and AR is to take you somewhere other than your current location. There are two types of interaction which are theoretically possible in VR and AR environments; inbound interaction through sight, hearing, smell, taste, and haptics; and outbound interaction through the mind, gaze, facial expression, voice, touch, pushing, knocking, grabbing (etc), gesture, body expression, and locomotion. At the moment only a few of these are available, but as more are built into our equipment, it will become more bulky and unwieldy. However, for mass adoption, a lighter and more seamless experience is needed. He demonstrated some SoftKinetic hardware (like the time-of-flight sensor) and software (like human tracking and full body tracking software) which will make a contribution to interaction through hand movements. This greatly strengthens users’ sense of presence.

In his presentation, 360° and VR User Generated Content – Millions of 360° cameras and smartphones in 2017!, Patrice Roulet (from ImmerVision) suggested that it will soon become normal for everyday smartphones to be used to record and share 360 content, in such a way that it captures your entire environment and the entire moment. It will only take two clicks to share such content on social media. To capture this content, it’s necessary to have a very good lens (such as ImmerVision’s panomorph lens which provides a high quality image across the whole field of view, can be miniaturised for mobile devices, and allows multi-platform sharing and viewing), and advanced 360 image processing. The panomorph lens can be used for much more than capturing 360 images; the internet of things (IoT) is about to evolve from connected devices to smart devices, and this technology has the potential to play a role as part of artificial intelligence (AI) in the upcoming ‘Cambrian explosion’ of the IoT.

In his presentation, VR content: Where do we go next?, Andrew Douthwaite (from WEARVR) stated that one key question is what comes first: adoption of hardware or high quality content; it’s something of a chicken and egg situation. He showed an example of a rollercoaster VR experience on a headset linked to a desktop computer; he noted that many people initially experience some nausea due to the sensory conflict that arises from, for example, sitting still while immersed in a moving VR experience. The emergence of mobile VR is now bringing VR experiences to a much wider audience; Google Cardboard is currently the most widespread example. There is a lot of 360 content on YouTube, and games like Raw Data are helping to drive the industry forward. Google Earth VR is another great example and will help VR reach the mass market, and could impact travel and tourism. New software is now making it possible for users to create VR characters and then inhabit their bodies and act as those characters.

Important future developments are wireless and comfortable VR headsets and more natural input mechanisms, including hand presence. One problem is that much 360 video content is currently of low quality; there is no point in having high quality headsets unless there is also high quality content available. The future of content, he said, lies in storytelling and narrative-based content; social interaction; healthcare; property; training; education; tourism; therapy and mental health (e.g., mindfulness and meditation); serialised content; lifestyle and productivity (though this might be more AR); and WebVR (an open standard which is a kind of metaverse, allowing you to have VR experiences in your web browser).

In his presentation, VR marketing, Philip Pelucha (from 3D Redshift) suggested that the next generation of commerce will not be browser-based; he gave the example of a 360 video of a product leading to a pop-up store allowing customers to further engage with the product. Noting that we already have online universities, he asked how long before virtual reality universities appear. He mentioned that soon we won’t have to commute to work because our phones and laptops will turn the world into our virtual office. In fact, he said, this is already beginning to happen, and when today’s children grow up, they won’t understand why you would have to go to an office to work, or to a shop to buy something. He also spoke about one major area of current development as being language education; a VR/AR app for immersive learning, or to support you when travelling, could be extremely helpful.

In his presentation, Bring the immerse experience to entertainment, movie and live event, Francis Lam (from Isobar China) showcased innovative examples of 360 videos. He showed the B(V)RAIN headset that combines VR with neural sensors; as your emotions change, what you see changes. In effect, the hardware allows you to visualise your mental state, and this can have consequences such as the targets you face in a shooter game, or the taste combinations in drinks that are recommended to you.

He concluded with some issues for consideration. Bad VR, he pointed out, can make you feel sick, so it needs to be high quality and low latency. VR is not just about watching, but rather about experiencing; it is about how, from a first-person point of view, you can go into a scene and experience it. VR is not just visual; audio is important, but there can be other sensors and tactile feedback. We should also ask to what extent VR can be a shared experience, where someone wearing a headset can interact with others who are not. VR is good for communication, a point which is well understood by Facebook; for example, with VR you can make eye contact in a way that is not possible in video chat. VR can allow us to explore new possibilities, such as experimenting with genders. In fact, VR hasn’t arrived yet; there is much more development to happen. Finally, he stated, VR is really not content, it is a medium.

China Mobile slogan, 2017 Mobile World Congress, Shanghai

China Mobile display, 2017 Mobile World Congress, Shanghai. Photo by Mark Pegrum, 2017. May be reused under CC BY 4.0 licence.

There is no doubt that industry perspectives on new technologies differ in some ways from those usually heard at academic and educational conferences, but is important that there is an awareness, and an exchange, of differing views between technologists and educators. After all, we face many of the same challenges, and we stand to gain from collaboratively developing solutions that will work in the educational and other spheres.

From China to the world: Mobile tech in teacher education

MTech Conference
Guilin, China
27-29 June, 2017

Shanhu Lake, Guilin, China

Shanhu Lake, Guilin (杉湖, 桂林), China. Photo by Mark Pegrum, 2017. May be reused under CC BY 4.0 licence.

The inaugural MTech Conference, based on the MTech Project and its underpinning MTech Survey, drew together teacher educators from Asia and Europe to discuss how best to integrate mobile technologies in teacher education internationally. It is hoped that this will be the first in an ongoing series of collaborative events involving the MTech Network.

In our opening presentation, Mobile learning in teacher education: Beginning to build a global overview, Kevin Burden and I gave an overview of the MTech Project and the underpinning survey of technology use by teacher educators around the world. We outlined initial insights emerging from the first round of data collection, based on 96 responses, with a little under two thirds from Asia, and a little over a fifth from Europe. We showed for example that relative to the iPAC Mobile Pedagogical Framework (see image below), teacher educators typically report more evidence of personalisation and collaboration than authenticity in mobile learning activities.

iPAC Framework

iPAC Mobile Pedagogical Framework (Kevin Burden, 2017)

Interesting insights are also beginning to emerge around themes of seamlessness and intercultural learning. We invited attendees and their colleagues to complete the survey, which has now entered the second round of data collection, with the aim of increasing the overall number of responses and especially obtaining responses from regions of the world which are currently underrepresented in the data.

In a presentation reflecting the Chinese context at GXNU, Developing pre-service teachers’ ICT in education competencies and curriculum leadership, Xibei Xiong referred to the TPACK Framework in describing a proposed ICTs in education curriculum which should include TK, TPK, TCK, and TPCK. Curriculum leadership, she said, shapes teacher education programmes by providing supportive policies, managing the curriculum, and evaluating pre-service teachers’ learning outcomes. Teacher education programmes may in turn shape the practices of curriculum leaders in terms of changing the education system requirements. Curriculum leaders at university level have a role to play in policy formulation and resource allocation; at school level, they have a role in determining educational curriculum structure, course objectives and academic credit management; and at classroom level, they have a role in developing course content and pedagogy.

In a presentation from the Singaporean context, Understanding teachers’ design talk for the co-creation of seamless science inquiry, Ching Sing Chai discussed the TPACK Framework and its various revisions and extensions in recent studies, before coming to focus on TPASK (Technological Pedagogical And Science Knowledge). He suggested that teachers need to design instruction with technology in order to develop their TPK; they should learn through designing in a collaborative community; they should be supported with appropriate scaffolds; and finally they need to engage in reflective experiential learning. Design talk embedded in a dialogic design, he went on to say, is key to supporting the emergence of TPACK. Sustainability and scalability ultimately come through teachers, so teacher development is more and more important in today’s world.

He described a Singaporean study involving the  5E (Engagement, Exploration, Explanation, Elaboration and Evaluation) approach for science inquiry-based learning, used as a PCK framing. Teachers talked about designing lessons for Grade 3/4 students. Mobile devices were used in various ways, including for seamless science learning (for example, students taking pictures and explaining heat sources in their own houses). The software used included KWL, Sketchbook, MapIt, Blurb (from the University of Michigan) and other tools (Nearpod, PowerPoint, Google, etc). The content of teachers’ design discussions was analysed to identify references to TK, PK, CK, TPK, TCK, PCK, TPCK, and CTX (representing context). A lot of the initial discussion was about TK but this element declined over time; conversely, the amount of discussion involving PCK increased over time, as did the discussion involving TPASK (but this was at a much lower level). CTX featured strongly but also decreased over time. The resulting model is quite different from the theoretical TPACK model (see image below).

A possible depiction of TPASK in design

A possible depiction of TPASK in design (Ching Sing Chai, 2017)

In a presentation from the Hong Kong context, Cultivating academic integrity and ethics of university students with augmented reality mobile learning trails, Theresa Kwong and Grace Ng showcased the mobile AR TIEs (Trails of Integrity and Ethics) developed by HKBU and its partner institutions in a Hong Kong-government funded project (a project on which I am also a consultant). As she pointed out, this is learning in the style of Pokémon Go, but in fact this project began around 18 months before the release of Pokémon Go. It is all about linking the environment to relevant educational content, in this case related to themes of academic integrity and ethics. Given that students find these AR trails motivating and helpful in connecting theoretical content with their everyday lives, this is an approach which is highly relevant to present and future educators and teacher educators.

In a presentation from the Taiwanese context, Mobile learning x cloudclassroom = ?, Chun-Yen Chang suggested that the spread of mobile devices along with BYOD policies means that the moment is right to be implementing mobile learning. The Taiwanese Ministry of Education has run collaborative projects on mobile learning in schools, and has set up a Teaching Application Mall of educational apps. He went on to describe his CCR (CloudClassRoom) project which supports mobile-assisted anonymous quizzes and presents teachers with aggregated data. It can be used, he said, in museums, outdoors, online, and in the ‘Asian silent classroom’. Polling students before and after lessons can be an ideal way of tracking changes in their understandings.

In a presentation from the Australian context, Teaching teachers how to go mobile: What’s happening in Australia?, Grace Oakley suggested that although mobile technologies are being used in many Australian schools, mobile learning is not developing as quickly as might be hoped, nor are its boundaries being pushed. There are many policy barriers, she added, including duty of care issues, funding, behaviour management issues, cybersafety, testing regimes, school processes, and equity issues. She then illustrated some activities with mobile devices being carried out in primary schools: oral retelling with Puppet Pals; learning prepositions with a camera and the Book Creator app; media presentations with Tellagami; and mobile augmented reality learning trails created with FreshAiR. She wrapped up with a discussion of how digital technologies, digital literacies, and mobile learning are beginning to feature in initial teacher education courses as well as in resource platforms for practising teachers, such as the Digital Technologies Hub. She indicated that some pre-service teachers are beginning to create mobile learning activities for their students, but she concluded by asking whether they are getting enough opportunities to do so.

In a presentation from the Irish context, Mobile learning on an initial teacher education progamme – MGO programme, Seán Ó Grádaigh showcased the technological changes that have occurred in the last decade. Uber is the largest transport company in the world, but has no cars; Facebook, Twitter and WeChat are the largest content platforms in the world, but they produce no content; Alibaba is the largest shopping mall in the world, but it has no shops; and Netflix is the largest cinema in the world, but has no movie theatres. However, he argued, we haven’t seen a game-changing application in education yet. Still, given the speed of changes, we need to be educating students for the future.

There is a misconception that better technology – from kitchen mixers through cameras to gym equipment – will lead to changes by itself. The same is true in education. But what is required is a vision, a plan, professional development, and pedagogical (as opposed to technological) training. In terms of the technology available, most schools are way ahead of most teacher training programmes, a situation that needs to change.

He went on to suggest that using technology to facilitate reflective practice by pre-service teachers may be a game changer. His students are asked to do reflections – hot reflections straight after a class, and cold reflections where they later revisit their initial reflections – using text, audio, video, or videoconferencing. He showed an example of a teaching video with a voiceover where the pre-service teacher provided commentary on her performance. She then received feedback from two tutors. The five steps followed in this task are:

  • Students create and construct a lesson
  • Students deliver and record it
  • Students watch and analyse it
  • Students create a reflective voiceover on their video
  • Students receive feedback on their reflection from tutors

He continued by suggesting that digital technologies can help to recreate immersive learning contexts for language learning as well as other subjects. However, rather than passively listening or watching, it is better to build inquiry activities around multimedia materials like videos. Teachers and students can also become actively involved in multimedia creation. Involving students in ‘teach-back’ activities is a great way to check that they have understood what they are learning.

In another presentation anchored in the Hong Kong context but with wide global relevance, entitled Learning design and mobile technologies in STEM education, Daniel Churchill explained that STEM is an approach to learning that removes traditional barriers separating science, technology, engineering and mathematics, and integrates them into real-world, rigorous and relevant learning experiences for students. It aims to improve learning in STEM areas; improve teaching effectiveness; deal with the shortage of STEM professionals in the future; include minorities, achieve gender balance, and provide opportunities for low-income members of society; decrease unemployment; foster international competitiveness in the 21st century; and help provide solutions to internationally pressing problems. Ideally, STEM should be not only multidisciplinary (where concepts and skills are taught separately in each discipline but housed within a common theme) or interdisciplinary (where there is the introduction of closely linked concepts and skills from two or more disciplines with the aim of deepening understanding and skills) but transdisciplinary (where knowledge or skills from two or more disciplines are applied to real-world problems and projects with the aim of shaping the total learning experience). There are both scientific and engineering approaches to STEM; in the latter, there are phases of problem scoping, idea generation, design and construction, design evaluation, and redesign. Challenges include insufficient teacher training; insufficient teacher knowledge of STEM; insufficient funding; insufficient laboratory resources and technicians; insufficient community support and media coverage; preferences for music, sport, and academic subjects; a student focus on exam preparation; learning computer coding without any logical or systematic thinking; and a focus on rote memorisation and a lack of depth of conceptual understanding.

He went on to explore six key affordances of  mobile technologies for STEM:

  • multimodal content (e.g., in the form of dynamic, interactive learning objects)
  • linkage of technologies (i.e., a mobile phone can connect to a whole ecology of digital devices)
  • capture (e.g., taking photos or making videos, capturing GPS position and acceleration, etc)
  • representation (e.g., programming a robot, making a digital story, creating a presentation, etc)
  • analytical (i.e., processing and looking for patterns in data)
  • socially interactive

Combining these affordances, he suggested, leads to new learning possibilities. Key tools include robotics, 3D printing, and cognitive tools.

He concluded by saying that STEM should not be just another science, maths or technology class. Learning design based on (pre-) engineering tasks is the critical strategy for STEM education, he argued, and STEM can be conceptualised based on an interaction model. Mobile and emerging technologies are essential for enabling STEM: these include virtual reality, augmented reality, wearables, and so on.

In his presentation, Key issues in mobile learning: A research framework, Pedro Isaías spoke of a range of current developments and challenges in mobile learning. He began by talking about the ubiquitousness pillar of mobile learning. He described the development of mobile LMSs, but mentioned that they have generally not really been designed for mobile devices. He asked whether they can be truly mobile-friendly without compromising navigation. He went on to emphasise the importance of creating responsive designs by following these guidelines: use mobile-friendly layouts, compress content, concentrate on the essential, format your text, and test the course on several platforms.

He went on to address the authenticity pillar of mobile learning, stressing the role of wearable technologies in education: for example, for interactive simulations, facial recognition for identifying students, creating first-person videos, and enhancing game participation. The challenges include cost, design concerns, privacy issues, familiarisation with the interface – digital literacies are needed here – and technical challenges. Augmented reality, he said, also has an important role to play in promoting authentic learning: it increases student engagement, mediates between students and the world, supports problem solving, enhances motivation, and provides access to real-world scenarios. One challenge is that students may become overly focused on the technology rather than the learning, and there are cost implications. He illustrated his comments with a video about the SNHU (Southern New Hampshire University) AR app, and a video about simulated 3D objects generated from textbook images with Arloopa. Gamification, too, can contribute to authenticity. Gamification should not be about external rewards, but about learning objectives. It enhances student motivation, provides ubiquitous access to resources, facilitates authentic and situated learning, improves peer interaction, promotes technical literacy, and fosters teamwork. Mobile learning game essentials, he said, are: an introduction and logo, instructions, a game objective, questions, feedback and results.

He then addressed the personalisation pillar of mobile learning, which is linked to mobile learning analytics, artificial intelligence, and geolocation. There are some concerns around data privacy and informed consent with analytics. With mobile intelligent systems, advantages include the fact that students can be taught according to their knowledge; adaptive learning methods; individualised adaptive teaching; explanation of teaching content; and automatic generation of exercises. Some LMSs provide geolocation features: this allows delivery of content according to location, designing of location-based online content, reaching a global audience, and consideration of cultural differences. Geolocation examples include language-adaptable subtitles, scavenger hunts, and geocaching games.

Finally, he addressed the collaboration pillar, which is about social learning and the e-society. Mobile learning harnesses the potential of social learning, promoting collaboration, discussion and knowledge exchange. But it is important to consider the quality of the interactions, and to think about the role of the teacher in the students’ discussions. Mobile learning involves the production of multimedia content, allows ubiquitous access to information, encourages the development of digital literacy, and creates informed citizens. There may be some issues around data privacy and security, and we must ask whether an increasingly mobile society may lead to an expansion of the digital divide.

In a presentation looking at future developments in mobile learning through wearables, The research on pedagogical feedback tactics of affective tutoring system based on physiological responses, Qin Huang suggested that in time wearable devices will be able to detect humans’ real emotions by registering physiological signals. She gave details of a study making use of the OCC emotional classification model, which is one of the most complete models and the first structural model used in the field of artificial intelligence. With good calculability, she said, it is widely used in the field of emotional computing.

Sun Tower & Moon Tower, Guilin, China

Sun Tower (日塔) & Moon Tower (月塔), Guilin, China. Photo by Mark Pegrum, 2017. May be reused under CC BY 4.0 licence.

The conference concluded with an MTech Steering Group meeting to discuss future directions for the MTech Network, how to gather more responses to the MTech survey and collaboratively publish our research, and when and where to meet again for another conference event. It is likely that the second MTech Conference will be held in China in late 2018.

New hardware, new software, and new questions about learning

mLearn
Sydney, Australia
24-26 October, 2016

syd16b

Hyde Park, Sydney, Australia. Photo by Mark Pegrum, 2016. May be reused under CC BY 3.0 licence

After an absence of three years, it was great to be back at mLearn, which took place this year at the University of Technology Sydney. As always, this conference brought together an international spread of expertise and contemporary research in mobile learning, focused in 2016 on the theme of Mobile Learning Futures: Sustaining Quality Research and Practice in Mobile Learning. Presentations covered new hardware (such as wearables), new software (such as AR and VR interfaces), new strategies (such as gaming), new questions about mobile teaching and learning, and the intersection points between all of these. Many of these presentations are written up in the conference proceedings.

New hardware – in connection with  new software – was showcased in the presentation, The use of wearable technologies in Australian universities: Examples from environmental science, cognitive and brain sciences and teacher training, where Victor Alvarez, Matt Bower, Sara de Freitas, Sue Gregory and Bianca de Wit began by showcasing the Vandrico Wearables Database, which lists the main wearables available for different parts of the body (see Figure 1).

Vandrico Wearables Database. Source: http://vandrico.com/wearables/

Figure 1. Vandrico Wearables Database. Source: http://vandrico.com/wearables/

They went on to give some examples of the use of wearables at Australian universities.  The first example was Murdoch University’s Conserv-AR mixed reality mobile game to promote awareness of wildlife conservation in Western Australia; there is an augmented reality field trip followed by a visit to a conservation island in virtual reality. The second was Macquarie University’s Portable Teaching Laboratory, involving a gaming headset to monitor brain activity in the cognitive and brain sciences. The third was the University of New England’s Virtual Teacher project involving student teachers engaging in classroom roleplays in the virtual world Second Life as part of their preparation for their first professional experience placements. As the authors pointed out, wearable technologies can thus be used in a wide variety of different ways in a wide variety of different areas; in some ways, wearables involve more research complexities than handheld mobiles because there are so many possible variations in the hardware, software, and pedagogical approaches.

In the presentation, Perceived utility and feasibility of wearable technologies in higher education, Matt Bower, Daniel Sturman and Victor Alvarez mentioned key areas where wearables are being used, from medical diagnosis through aged care to the social implications of facial recognition augmented with personal information. They gave an overview of the educational affordances of wearable technologies, as showcased in Bower and Sturman’s 2015 article ‘What are the educational affordances of wearable technologies?‘ They then went on to discuss eight use cases of wearables that were rated for utility and feasibility in an international survey, noting that there were significant differences in many cases between perceived utility and perceived feasibility. Key issues surrounding wearable use mentioned by respondents were cost; technological issues; lack of pedagogical benefits; distraction or disruption; resistance to change; and privacy and legal issues. This is an area where there is really a considerable gap between potential utility and current feasibility in education, notably in terms of cost.

Contemporary software was showcased in the presentation, WhatsApp in mLearning: The (learning) medium is the message(r), where Christopher Pang spoke of the phenomenal rise in popularity of the OTT (over the top) platform, WhatsApp. He asked how habitual use of a mobile platform like WhatsApp shapes a learner’s practices. M-learning offers an additional platform for e-learning, he suggested, and can be a motivational aid to e-learning. Beyond this, it can support collaborative learning and informal learning, and supports the blurring of boundaries and role distances.

In this study, he created weekly replacement, supplementary and complementary tasks for business students, given to trial and control groups, followed up by self-reported questionnaires, revisiting of conversation threads, and selected interviews. However, even in the control group which was not specifically asked to use WhatsApp, students were already using it extensively.

Overall, he found that the use of the mobile app drove online completion and led to higher completion rates. Students demonstrated self-directedness and elements of lifelong learning. They were very willing to receive formative feedback through WhatsApp, including students who normally would not ask questions in class. Students also used WhatsApp groups for group sourcing of answers; the dilemma for a tutor in a WhatsApp group is whether to intervene or allow students to work out the answers for themselves. In conclusion, he noted that active WhatsApp students were likely to show greater learner negotiation, greater agency, and greater learning effectiveness; and were more likely to show a drive towards self-directed learning, to seek personalised learning and co-creation of learning opportunities, and to connect data to generate new learning.

In my own paper, On the path to situated learning: Embedding academic integrity via mobile augmented reality learning trails, co-authored with Eva Wong and Theresa Kwong, my colleagues from Hong Kong Baptist University, I spoke about the outcomes experienced to date, at approximately the midway point of a 3-year Hong Kong-government-funded project where AR TIEs (Trails of Integrity and Ethics)  have been developed to help students connect formal learning about integrity and ethics with the everyday situations they face on campus. The trails immerse students in collaborative problem-solving tasks centred on ethical dilemmas, addressed in real-world locations where such dilemmas might arise, with contextually appropriate digital advice and information available on hand. By allowing students to play out the consequences of their decisions, this approach is designed to complement classroom engagement and, in particular, to reinforce the links between theoretical learning and the practical application of such learning in everyday contexts. Results to date indicate the value of situated learning in helping students to integrate ethical understandings into their everyday study practices. At the same time, numerous challenges have arisen, leading to an ongoing reshaping of the trail designs as we seek to capitalise on the potential of mobile learning to turn academic integrity and ethics from a formal requirement into a set of considerations that inform students’ daily lives.

In another paper, Factors in designing an augmented reality m-learning trail with place-based pedagogy in residential education, my colleagues Kevin Yue, Lisa Law, Hiu Ling Chan, Jade Chan, Elaine Wong, Theresa Kwong and Eva Wong spoke about the Hall Tutors TIE (Trail of Integrity and Ethics), which is one of the subject-specific trails forming part of the same Hong Kong project outlined above. It was explained that ethical reasoning and judgement skills can be more effectively developed when linked with personal experiences. Therefore a learning trail was created in which student hall tutors explore a scenario-based story to help them develop a more personal understanding of their roles. The presenters used a visualiser to demonstrate the underpinning mobile app, giving the audience a sense of the digital screens, information and choices through which students move when taking the trail. Visualisations of keywords used by students in pre- and post-trail online discussions have revealed a shift from a focus on ‘rules’ to a focus on being a ‘role model’, suggesting a change of mindset among the student hall tutors, who seem to have developed a new sense of their roles.

In their presentation, Understanding the relationship  between augmented reality games and educational pedagogies, Christine Redman and Joanne Blannin discussed the educational potential of the AR game Ingress (an older but more complex game from the same company, Niantic, that created Pokémon Go; see Figure 2) in terms of motivation, learning theories, pedagogical strategies, 21st century skills, and a STEM focus. They are using Positioning Theory to understand people’s motivation to play and continue playing. The game requires players to move between the real and the virtual and to connect with other people. In the game, players receive constant and instant feedback, and there is a complex, multifaceted reward system. There are 16 levels, with each level taking longer than the last, and more badges are needed to move on. There are two teams, Green and Blue, which need to remain in communication, with team members collaboratively planning major goals.

ingress1

Figure 2. A comparison of Ingress and Pokémon Go player views. Source: https://goo.gl/7kTDgm

From an educational perspective, we can say that learners know where they are up to and can predict strategies to move on in the game; have clear intentions; have explicit success criteria; and have constant feedback on progress. Playing a game like this, the authors suggested, can lead to the development of enterprise skills, 21st century skills, and the 7C skills. In particular, the game rewards strategic thinking, problem solving, memory, spatial awareness, teamwork, communication, and leadership skills. Elements of geography and environmental awareness, history and architecture, mathematics and spatial skills, are also prominent in the game. It is played by people of all ages and there are numerous women in leading roles. Active participation in the game often involves learning, and sometimes also teaching others.

In the presentation, Location-based mobile learning games: Motivation for and engagement with the learning process, Roger Edmonds and Simon Smith suggested that GPS and maps can power up experiences with authentic location interaction, while storytelling and rich media deliver learning, personalisation and an emotional connection, and gameplay helps with retention and recollection of knowledge.  They described location-based mobile learning games created using the Mobile Learning Academy platform, which does not require programming knowledge; some have been created by lecturers, but students are now also generating their own games. Typically, the design stage of a game involves identifying and scoping out the game and creating context with a story. The development stage involves using gaming software to link rich media to places, and adding location-interaction tasks and gameplay, before testing and publishing. The play stage involves walking to places, triggering the activation of content and tasks, performing challenges, answering quizzes, uploading photos and notes, and finally sharing experiences via Facebook and Twitter.

In a study of students’ responses to the four lecturer-created games, engagement did not vary much between the four different disciplines, but whether the students thought they understood more about the topic did vary – key considerations were design factors (e.g., content, duration, level of difficulty, location, tasks, and competencies) and implementation strategies (how the game is integrated with tutorials or excursions, and whether it is mandatory or voluntary). In conclusion, location-based mobile games do provide active, authentic, engaging educational experiences in higher education, but the pedagogical benefits are influenced by game design factors and implementation strategies. Further information is available on the project’s companion website, Pedagogy Go.

In their presentation, Using mobile serious games technology to enhance student engagement and learning in a postgraduate ethics classroom, Gillian McGregor and Emma Bartle explored the opportunity for technology to contribute to the teaching and learning of applied psychology skills in the form of a serious game called How Do You Feel (which can be downloaded for Android devices here or played in the Firefox or Internet Explorer web browsers here). Intended to supplement rather than replace teaching in a professional psychology programme, the game involves a series of scenarios where clients present a variety of issues, allowing students to safely build up their skills in dealing with clients. In preliminary findings, it has been established that student engagement is greater when using the serious game than when reading a static case study. Students liked the connection to real life, being able to see the theory in practice, seeing examples of what psychologists could say when encountering different scenarios, and discussing the scenarios with peers.

In their presentation, A mobile learning framework for developing educational games and its pilot study for secondary mathematics education, Yanguo Jing and Alastair Craig described how they structured a game around GCSE maths skills, with each level of the game focusing on different skills. Students enjoyed the game and thought it helped them learn key concepts and skills. Learning theory and game design principles are fundamentally important in creating successful educational games. The future plan is to employ more social and multiplayer elements to increase the level of student engagement.

In their presentation, Survive with the VUVU on the Vaal: Eyetracking findings of a user interface evaluation of a mobile serious game for statistics education, Seugnet Blignaut, Gordon Matthew and Lizanne Fitchat suggested that balancing fun and teaching in serious games can be challenging. They described a game for students at a rural South African university which teaches everyday life skills alongside basic statistics. Eyetracking software provided quantitative data revealing where students were and were not focusing on the screen. Qualitative data revealed students’ concerns over the user interface (including for some students who were familiar with mobile technologies but not with a mouse when the game was played on a PC), game instructions (including the need to have these available throughout the game), 3D graphics (which were limited compared to commercial games), and the game challenges (with a need to individualise the levels and adjust them to players’ competencies). Two key lessons learned were that eyetracking devices and usability interviews are not unobtrusive and reduce players into subjects; and that students should be continuously involved in the conceptualisation and production of the game.

Key teaching and learning themes were flagged up in the paper, Does the mobility of mobile learners across locations affect memory?, where Chrysanthi Tseloudi and Immaculada Arnedillo-Sánchez opened by stating that mobile learning research focuses on the flow of learning as learners move through physical, technological, conceptual, social and temporal dimensions. This paper focused on the physical contexts, and asked whether learners’ memory is challenged when they try to recall learning from one context in a different context. Environmental elements can become encoded in memory along with the learning that is taking place; it may be a struggle to remember what we have learned in a different context where the same environmental cues are not present. This is a major challenge for mobile learning. Possible strategies include mentally reinstating the original learning context, i.e., essentially remembering the place you were in when learning (though learners vary in their ability to do this), or suppressing the surrounding context when learning (which may be difficult to do in an environment rich with stimuli, some of which might be relevant to the learning). Decontextualisation of learning may be a preferable approach; in other words, it may be more promising to learn in multiple contexts, and make the learning available in many different places.

In sum, should we really be trying to learn “anywhere” – and should we be learning in the exact place in which we need the information, or in many different places? This is currently unanswered. We need to research how much mobility is needed to facilitate decontextualisation, how artificial and real contexts interact, and what elements learners can manipulate to reinstate or vary their own contexts. In mobile learning research, they suggested, we should be investigating contextualisation in parallel with decontextualisation.

In an interesting follow-up discussion, Jocelyn Wishart raised the idea that a key advantage of mobile devices is allowing users to recreate contextualisation of learning through the multimodal records we make at the time when learning occurs. It was suggested by others that the context may sometimes but not always be relevant to learning, and that different strategies might be needed depending on the case. Kevin Burden commented that another advantage of mobile devices in learning is reducing the cognitive load because information can be partly offloaded to the device and carried with the learner.

In the presentation, Choosing between a student-generated animation or written assignment: Students know what they want, Hardy Ernst and Laurel Dyson talked about introducing a video-based assignment instead of a written assignment in a course, but although the quality of learning was similar, the videos were disruptive, time-consuming and not appreciated by all students. The following year students were given the choice between a video or written assignment, and it was found that students employed very individual learning strategies. It depended on students’ visual and digital literacy skills, time management, group work preferences, and engagement, with having a choice being more engaging for students. When asked in 2016 about the main reason for their choice of a video or written assignment, it was found that those who didn’t like group work chose the written assignment; other factors influencing the choice either way were students’ perceptions of their ability to manage time, interest, better learning opportunities, and leniency of marking (with many students thinking the videos would be more leniently marked). In a thematic analysis of students’ responses about why they chose the video option, key factors mentioned by students were interest and fun, as well as a belief that the visual mode is a good way to present knowledge, a wish to share ideas, and novelty; these are generally positive factors. Among the students who chose the written assignment, the key factors were working at their own pace and independent learning, as well as the time-consuming nature of making a video and past negative experiences with group work; here there are more negative factors mentioned. In sum, students demonstrated a solid understanding of their own abilities, allowing them to adopt deliberate individual learning strategies.

In his plenary which opened the final day, The role of education in identity transformation and acculturation, John Traxler raised some concerns around mobile learning. He spoke of two ‘elephants in the room’: the notion that mobile technologies are value-free conduits which are morally neutral and serve no-one’s particular interests; and the linked notion of the completion of the European project of modernity.

He spoke of the only partially successful inclusion agenda in Western higher education, which led to a massification process as non-traditional students were brought into education, accompanied by the introduction of computer laboratories as industralised workshops; in this context, mobile devices might represent a more flexible, user-friendly kind of industrialisation. He asked whether the process of acculturation into education adds to or replaces one’s sense of identity, in a process of ‘them’ becoming ‘us’. However, he speculated that with mobile technologies, there is more pressure from the outside world where mobile technologies are widely used, which is beginning to transform education from without – with ‘them’ perhaps starting to transform ‘us’.

Technology, he suggested, distorts the relationship between people and language because of the encoding of characters and the available input mechanisms. Moreover, computing is arguably underpinned by a programming paradigm which does not map well to many natural languages. Technology also has the effect of changing pedagogy, notably as international aid agencies have sought to make their educational missions scalable and sustainable through mobile devices, pushing them towards transmissive pedagogies rather than more constructivist pedagogies, and without taking into account locally relevant pedagogies. Furthermore, much of the education takes place in English. In a sense, technology is a Trojan horse for education, but education itself is a Trojan horse.

The hegemony of US technology, the English language, and European models of pedagogy may be especially challenging for cultures and languages which differ substantially from these; but is the hegemony of middle class values equally challenging for working class, non-traditional students? He spoke of the work of Richard Heeks on ICT4D 2.0, and the need to distinguish between:

  • pro-poor innovation (outside of but on behalf of poor communities)
  • para-poor innovation (working alongside poor communities)
  • per-poor innovation (within and by poor communities).

He went on to discuss the concept of epistemicide, where whole ways of looking at the world are killed off, starting with examples from the European 16th century. This is linked to the hegemony of the European university system around the world, with the University of Cape Town resembling the University of Florence, he suggested. In a different way, it is linked to the growing hegemony of mobile technologies, though the latter may also be producing a kind of postmodernity where knowledge can be generated outside the academy and everyone can discuss and share ideas. As Traxler commented in response to an audience question, the fundamental question may be whether the technology is hegemonic or enabling; and this may depend at least in part on whose hands it is in.

In her workshop, Debating the future for mobile learning in schools, Jocelyn Wishart mentioned that the use of mobile devices in schools varies enormously across the world, ranging from outright bans to an expectation that students will bring and use mobile devices. Mobile phones are also being used in a wide range of different ways, from ways that support learning to ways that distract students from it. She showcased a series of mobile phone policies from schools around the globe to demonstrate just how different the approaches taken by schools are. This was followed by a group discussion about how to balance up the benefits and drawbacks of using mobile devices in education.

In their workshop, The Handbook of Mobile Teaching and Learning, Aimee Zhang and Dean Cristol described the 2015 publication of this book through Springer, as well as outlining plans for a second edition. Given the number of new possibilities emerging in the field, as showcased in the papers at this conference, there will be no shortage of material to include in the new version! Some key emerging focus areas are likely to include wearables and AR/VR.

Jacarandas in blossom, Sydney, Australia. Photo by Mark Pegrum, 2016. May be reused under CC BY 3.0 licence

Jacarandas in blossom, Sydney, Australia. Photo by Mark Pegrum, 2016. May be reused under CC BY 3.0 licence

As always, then, this year’s mLearn Conference highlighted currently emerging themes around mobile learning, providing a snapshot of where we’re at, where we’re heading, and what our most pressing questions are.

From code literacy to robotics

KL2016B

KLCC, Kuala Lumpur, Malaysia. Photo by Mark Pegrum, 2016. May be reused under CC BY 3.0 licence.

Digital Education Show Asia
Kuala Lumpur, Malaysia
18-19 April, 2016

It was good to be back at the Digital Education Show Asia in KL for the second year running, especially given the heavy focus of this year’s event, the fourth in the series, on 21st century skills and digital literacies, including computational thinking and coding. Robotics, perhaps unsurprisingly, was also high on the agenda.

In my own paper on the first morning, Mapping the pathway from m-learning to digital literacies for ASEAN educators, I argued that in order for our students to get the most out of mobile learning, it is important for educators to help them develop their mobile literacy, and the individual literacies of which it is composed, including code literacy.

In his paper, Learning beyond boundaries – How coding shapes systematic problem solvers, Felix Lee suggested that coding and robotics have a role to play within the current context where we have to break down boundaries between subjects, develop creativity through problem-based learning, and let the students determine their learning paths through an interactive technology-enhanced curriculum.

In her talk, Advocacy of STEM education – Introducing computational thinking as the new literacy of the 21st century, Ng Puay San emphasised the importance of applied STEM education to support innovation in a global conceptual economy. She stressed the need for an integrated curriculum where different subjects like science, technology, engineering and maths connect with each other. She talked about computational thinking – which she described as a new literacy – as a framework within which students learn to reason about systems and problems, and which goes well beyond issues of hardware and software. She showcased the 3-year initiative Code for Change under Singapore’s Smart Nation Vision, designed to improve students’ skills in this area, and gave the example of a 4-year-old girl coding with Scratch. She wrapped up with an overview of the need for a quality and integrated curriculum, authentic assessment, leveraging of educational technology, continuous professional development for teachers, and partnership with the community, industries and home.

In a talk focusing on the Malaysian context, Digital.Tech@Schools: Empowering students to become digital innovators, Sumitra Nair opened by noting that 90% of all future jobs will require digital competencies, according to the EU Skills Report 2013. She indicated that there are currently initiatives to encourage digital innovation in Malaysia, but that these exist at the fringes of the formal curriculum; young people, she suggested, need to move from being consumers to creators of technology. The Digital.Tech@Schools initiative, piloted in 24 schools in December 2015, involves revising the ICT curriculum and training teachers; introducing co-curricular clubs; and running national-level competitions. The new curriculum focus is to be on algorithms, decomposition and debugging; coding and sequencing; and digital literacy – searching, analysing and curating content. The approach will involve thematic, activity- and project-based learning (combining unplugged and device-based learning). Co-curricular activities in the pilot included app development, Arduino and Scratch programming, and 3D printing. The initiative has now been endorsed by key decision makers. The focus in 2016 is on educator readiness for the curriculum roll-out.

In the follow-up panel, How do education leaders need to adapt according to a new technology-driven education system?, chaired by Eric Lam, a number of key points were raised, notably about the need to place pedagogy before technology; the centrality of the teacher’s role even within technology-enhanced education; the need to remember the human dimension of education; the importance of teachers employing creativity and design thinking to repurpose technological tools appropriately for learning; the advantages of having students sharing through online platforms; the need to have students use technology for communication and creation rather than just consumption, but to use more traditional tools when appropriate; and the key role played by the surrounding culture and context.

In his presentation, Visual  learning and emerging technologies – Rethinking 21st century literacy for a visual world, Emory Craig indicated, following Ron Bleed, that being visually literate is a must in the contemporary era. He noted the enormous potential of augmented reality (AR) in education, as well as of virtual reality (VR); at 90 frames per second, as in current high-end VR displays, he says, you cannot tell the difference between reality and visual media. He also described Facebook’s experiments in social VR, as well as Microsoft’s Hololens, which allows holographic teleportation. He concluded with some questions:

  • What new tools/vocabulary do we need to analyse visual media?
  • Can new media and VR create new ways of knowing? Can it create empathy? (And is this the final form of media, now that it has become immersive?)
  • What happens when media becomes as ‘real’ as the real world? (And how do we keep our critical distance?)
  • Will it foster new forms of collaborative learning?
  • How should current educational practices and institutions change in a highly visual and virtual world?

He also referred participants to the Digital Bodies site where he and his colleagues write about these kinds of new developments.

The roundtable discussion, Understanding how mobile and ubiquitous access technology can help to enable blended learning in your schools, led by Ian Pittman, began with a discussion of the wide range of possible definitions and interpretations of blended learning.  The topic of learning design arose quickly, as did the issue of the socioeconomic context and how the available technology impacts on learning designs, which always need to be customised to particular groups of learners in particular contexts.

Reflecting on less well-provisioned contexts in his paper, The burden of technology in education: Is there a more painless way?, Eric Lam mentioned that a key infrastructure  challenge is how to achieve e-learning with only short periods of stable internet access. He talked about the importance of downloading materials from the cloud when there is an internet connection, so they can later be used without a connection. E-learning should be afforded, he suggested, without the constant presence of the internet. While this problem may cease to exist in the future, it is a very real problem now. He showed a platform called PageWerkz designed to work under these conditions.

In his presentation, Outlining best practices on how to develop MOOC content, David Asirvatham suggested that the advantage of MOOCs is that they allow for on-demand and networked learning. It is important to begin by deciding whether to set up a cognitive-behaviourist xMOOC or a connectivist cMOOC, or to try to combine the two. MOOCs can be fully online, or used to support blended and/or flipped approaches. Creating a MOOC is a chance to explore new pedagogical approaches as well as new business models. He suggested that it would take 6-12 months to develop a MOOC from scratch, and that the cost might be around US $50,000. Ultimately it is a team effort involving the following roles: subject matter expert, instructional designer, script editor, graphic designer, camera operator, audio/video editor, and reviewer. He noted that learning objects, in the form of videos within a MOOC, should ideally be under 5 minutes long. It’s also important to consider how you will address the typical drop-out rate from MOOCs, which may be up to 80-90%.

In between the many papers, it was interesting to see how many companies are offering robotics hardware and associated programming software for education. Providers included Arduino Robotics (Malaysia), rero (Malaysia) and Pitsco Tetrix (USA), whose products can for example be used in robotics lessons and clubs in schools. There is also a push for the integration of robotics with STEM, such as by Abilix (China) and in the STEM with Robotics programme by CM Asia (Singapore). Meanwhile, the company Robotics Learning (Malaysia) was showcasing its programmes to help children learn to create robots in a problem-solving environment, combining elements of STEM and coding in an integrated learning context (see Figure 1). It was also interesting to learn about Malaysia’s annual National Robotics Competition for school students.

Robots2016B

Figure 1: Robots from Robotics Learning, Malaysia (2016)

My strongest impression of this conference is that, within a broader recognition of the importance of 21st century skills and digital literacies, there is a growing appreciation of the need to foreground computational thinking and code literacy, and to understand the role that can be played by robotics and programming in an integrated, STEM-oriented, problem-based approach to the curriculum. It will be interesting to see how these intersecting trends continue to evolve over coming years.

Skip to toolbar