An encouraging 85% of children and 74% of adults have played a musical instrument. The piano tops the list as the most popular musical instrument played by 5-14-year olds, showing 15% growth rate since 2014. Despite this, only 20%of music produced was using a smartphone or tablet (Making Music. 2014.) Currently within the music education industry, an opportunity presents itself to design an application allowing users to learn music from their electronic device. A versatile application, beneficial to all age groups and ability levels. Considering the cost of a digital piano ranges between £300-£1000 and £5000-£10000 for an Acoustic Grand Piano, this application is particularly beneficial to those without a disposable income.

Get Help With Your Essay

If you need assistance with writing your essay, our professional essay writing service is here to help!
Find out more about our Essay Writing Service

LeapMotion Controller, provides 3D tracking data of the hands through an easy to use application programme interface (API) (DiMartino. 2016). Currently 59% of universities globally utilise Leap Motion Controllers, 5 times more than any other input device (Colgan A, 2017). A clear market leader, identified for its educational attributes. The virtual 3D piano model will initially be designed using AutoDesk Maya. The application will be developed using Unity, creating a 3D piano playable by the assistance of the hand tracking LeapMotion Controller. Programmed in C#, enables the end-user to press action on the 88 keys to produce the key sound using clips of each 88 keys in creating a virtual learning technique. Design of aPHP/HTML5 website, will allow customers to explore and download the application along with the required executables. The website provides information regarding the necessary requirements to give the end-user the best performance of the Pianistic Application.

Keywords: 3D Piano, C#, LeapMotion Controller, Unity, virtual learning technique.

1. Introduction

Of the current piano based applications that are available from the Google Play Store or Apple Store, none completely fulfil the requirements because many of the applications are developed for fun recreational instead of music educational. With the extortionate price of purchasing a piano along with any further piano lessons undertaken, a limit number of people can play this instrument. My knowledge of this field is in the expertise on knowing how to play the piano myself, having a Grade 8 in this instrument is beneficial to me when it comes to developing this application. The learning curves which will be embarked when developing this Final Year Project is in learning on how to design the 3D Virtual Piano Model using software Autodesk Maya, and how to develop Pianistic using Unity with the combination of LeapMotion transmitting sound when pressing the keys for the development of the application. I will be scripting in C# for the LeapMotion Controller to interact with keys when pressed.

1.1 Aims and objectives

Theprimary aim of this project is to design and develop a virtual piano application for Windows, which uses hand tracking technology and provides access to all 88 keys giving the end-user a virtual learning technique.

Objectives

- To initially carry out a literature review, to review and analyze the market competitors, background to problem

- To review the recent developing technologies, utilized for running application diagnostics

- Identify and evaluate the user requirements, and costs implications associated with the application

- Evaluate the development methodologies associated with the development of the application

- To develop a working prototype of the Pianistic Application for Windows, utilizing the LeapMotion Controller

- To design a responsive and user-friendly PHP/HTML5 website, allowing users access to the LeapMotion application manual and other associated executables.

1.2 Literature review

1.2.1. Background to the Problem

Music has a positive impact in the social and personal development, (Hallam, S. 2010) Where piano practice can help people to develop skills such as discipline and concentration. (Kokotsaki, D., & Hallam, S. 2007). The problems with these applications which are available in the Google Play Store and Apple Store for piano such as Don’t Tap the White Tile, Piano Tiles 2 and Magic Piano are intended for fun recreational usage but does not keep them interested in acting as a serious learning music technique. Schellenberg, E. Glenn evaluates the results from the graph in Figure 1 to show how learning a musical instrument increases children’s level of intelligence.

Figure 1: The mean increase in Full Scale IQ (Wechsler Intelligence) for each 6-year olds who completed the study from learning a musical instrument to no lessons.

(Schellenberg, E. Glenn. 2004)

Studies indicate that the use of technologies such as computers and tablets have advantages in the success in their learning abilities. (Prensky M., 2001) (Figure 2) It raises their motivational performance and promotes the usage of the information and communication technologies (ICT). (Gutterman, B., Rahman, S., Supelano, J., Thies, L., and Yang, M. 2009) There is a growth in the time spent on apps, where music stands at 79%. (Saifi. 2017)

Figure 2: Shows the % of students reported how technology has been beneficial to the learning process.

(Prensky M., 2001)

1.2.2. Improvements of Technologies

1.2.2.1. Scripting Languages

Unity 3D offers its developers scripting languages built into the IDE (Integrated Development Environment): Unity JavaScript, C# and Boo. The scripting Language is supposed to allow developers to implement the same content independently of their language preferences which is C# for Pianistic Application. The Language is run mono to give faster compile times. Mono is open-source from .NET Framework to allow cross compatibility on different platforms. C# allows the programmer to have the complete and precise control over Unity JavaScript. C# does very little in the Background and requires the developer to manage the components manually. (Unity Technologies 2014)

1.2.2.2. LeapMotion 3D App Performance

If your building a demanding 3D Application, the two things that we to consider is the GPU and the CPU Cycles. When you are using your 3D Application, LeapMotion Controller provides 1/100^th Accuracy when detecting the movement of the hands, they work together to generate the desired frame rate. The details are handled by the GPU and the CPU handles rendering the draw cells. (Praveen, K. 2016) It’s important to see whether GPU and CPU too have many vertices to process. CPU handles the vertices to interact with the object and the GPU will handle the vertex shaders, if you have too many vertices within the 3D Scene it can cause the shader to have trouble writing the specific colour to the material of the object. Therefore, this can decrease the performance, which can drag the frame rate of the 3D App. (Praveen, K. 2016)

1.2.2.3. UI Design

Most Popular Piano Applications on Android and IOS do not provide the best UX experience because the screen size of the applications that Is being played on, in comparing of the desktop environments and small screens, Jones et al evaluates that when they are running applications on small screens the tasks are being performed much slower. (Jones, M., Buchanan, G., and Thimbleby, H 2003) Maniar et al. evaluated that mobile phones between 1.76-3.78-inch screens and it found that the small screen has significantly deteriorated the students learning effectiveness. (Maniar, N., Bennett, E., Hand, S., and Allan, G. 2008) Kim et al. tested three mobile devices from 3.5 -9.7-inch screen and the findings are that the largest screen led to much more enjoyment was compared to the smaller screen. (Kim, K.J., Sundar, S.S., and Park, E 2011)

1.2.2.4. Application Packaging and Development Lifecycle

Application Packaging has its benefits for app developers and users. Properly setting up all necessary installation files reduces any errors in software. I am using executable file (.exe) file for my Pianistic Application along with other executables that are needed such as the Leap Motion Control Panel and updated Leap Motion Drivers. Application Packaging is viewed as critical to effectively managing software on server, desktops and within any Windows Operating System. Therefore, we can streamline the software install, uninstall, patch and repairs. Application Packaging helps companies reducing their costs when going through each phase of the application lifecycle. (International Journal of Computer and Information Technology,2013)

Figure 3: The process of the Application Development Lifecycle in allowing programmers to develop an application with the need of cost savings and time reduction in both IT and Business Operations.

(International Journal of Computer and Information Technology,2013)

1.2.2.5. Website requirements

The Application Packaging for Pianistic requires a host and domain for promoting the application on the website so the users can download the application package. PHP Framework is the most popular programming Language for developing your Website where over 82% of all websites are being used for this server-side programming Language. (Goel R.,2014). Having a website can increase turnover of around 45% with around two million UK still don’t have a website. Design, Hosting and unique domain would cost £200-£2500 per year depending on the size of the company. (Expertmarket.co.uk. 2018) Websites must be following the GDPR (General Data Protection Regulation) legislation in knowing where the data is being protected and the companies knows where the data is stored and managed which is called Data Mapping. Websites encrypted in HTTPS ensures security, trust and to protect users against the man-in the middle attacks from insecure networks. (Denne, A. 2018)

1.2.3. Market Competitors

1.2.3.1. Virtual Piano LeapMotion

Liang, Patey, shah and Walters implented a virtual piano using input processed by LeapMotion Controller. The LeapMotion has two monochromatic IR Cameras and three LED’s to detect hand and finger movement which uses mathematics to infer three-dimensional Position Data. Instead the finger positional of the data is collected on a separate computer and send through the ethernet communication to Hard Processor System (HPS). The communication between positional data over the UDP connection on a separate workstation using Java. (Liang, Patey, shah and Walters. 2014).

Figure 4: Shows the Block Diagram of how processor interacts with various controllers.

(Liang, Patey, shah and Walters. 2014)

1.2.3.2. Glozart

Glozart is a 3D Graphics program that designed by James Delos Reyes. It can graphically stimulate the hands while playing the Virtual Piano. This is an educational program for inexperienced pianists to learn how to play the Piano by seeing and hearing a demonstration of how the piece is being played. The Application is written in C++ and using utilities OpenGL, GLUT Qt, MusicXML and MIDI. The Problems that were addressed during implentation where that to show real-time sequence with the graphic and sound synchronization, analysing a music file into simulation frames and vigorously select the position of the fingers and supply a responsive user interface. (Reyes J, 2007)

Figure 5: Shows Hands and the Keyboard for GLUT primitives for Glozart.

(Reyes J, 2007)

1.3 Summary of the report

Hallam, S. investigated that piano practice can help people to develop skills such as discipline and concentration. The results from Schellenberg, E. Glenn experiment showed that learning a musical instrument increases children’s level of intelligence. Many Piano applications which are developed for Android and IOS Mobile Devices are not designed for music education and therefore played as fun recreational usage. Saifi investigated that there is a growth in the time spent on apps, where music stands at 79%. Most Applications are played on smaller screens, as most of the Piano Applications which are downloaded from the IOS and Android Store, compatibility features will have an issue if you are looking to learn the piano of these mobile devices; Jones et al evaluates that when they are running applications on small screens the tasks are being performed much slower. Prensky M., studied that the use of these technologies using computers and tablets have the advantages in their learning abilities. Schools will promote the use of these technologies as it will raise their motivional performance and promotes the usage of the information and communication technologies (ICT). Currently 59% of universities globally utilise Leap Motion Controllers, which is clear market leader, identified for its educational attributes which is the aim of the application.

As LeapMotion and Unity came together to provide this hand tracking technology, the research and the experiments in developing an application, there will be new findings throughout the development stages of the application. The First Stage of Development will be the design the 3D Virtual Piano Model in AutoDesk Maya, the next stage will import the core assets into Unity 3D to integrate the movements of the hands of LeapMotion from the Controller with the 3D Image of the piano model. Integrating the key press with fingers to detect the key sound from the piano.

The Pianistic Application should be used by end-users of every age and all levels of ability. The Application will be compatible with Window 7,8.1 and 10. The user will play through Laptop/Desktop in detecting your both hands, with the connection of LeapMotion Controller using USB 2.0. This will be a music educational program for Windows which is a better alternative that are currently on the Android and IOS market. The Prototype Model should be tested and analysed vigorously do get the best results of performance and easy to use for end-user in following some instructions provided by the developer.

2 Analysis

The Analysis Chapter documents your work in determining a formal set of requirements to be used in delivering a suitable solution to the problem being tackled. The nature of this investigation will vary from project to project, but may involve the use of questionnaires, interviewing stakeholders.

The investigation of the problem and its potential solution will have led to the identification of a set of functional and non-functional system requirements, which should be presented, using whatever descriptive techniques are appropriate for the type of system and type of requirements involved. For example, functional requirements might be documented as user stories or use cases.

On establishing the requirements you should draw up a suitable project plan to formalise the different phases required in delivering a successful project.

Length: ~9 pages

3 Design

Design covers the user interface, software architecture, data definitions, algorithms and other high-level descriptions of the system you are proposing to create. Ideally, a good system design document is one that can be passed to someone else to implement.

It is expected that during the design phase various options will have been considered before any final decision was taken. These options should be identified and the rationale for each decision presented.

You are encouraged to use descriptions and models suitable for your own circumstances. For example,

Develop an Architectural Design for your proposed system.
In describing how the user will interact with your system you may want to present a block diagram identifying key parts of the user interface in addition to showing screen shots. If you considered HCI guidelines, you should explain how these influenced your design
If you are using web pages, you may want to include a map of your site, indicating how users navigate through it
If you are using a database, you will need to describe the design schema, including details of any normalisation involved. You may also want to include an entity-relationship diagram
If appropriate include a class diagram, identifying your main classes, their interactions and their purpose
For some processing it may be necessary to make use of complex algorithms, which should be described and illustrated appropriately (e.g. using sequence diagrams)

Length: ~5 pages

4 Testing

The content of testing chapter is largely self-explanatory.

Your approach to testing should then be described, identifying the test cases that you have used to verify the correctness of the software. These should demonstrate that your testing has been appropriate and thorough.

Length: ~ 4 pages

5 Evaluation

In the final chapter you should summarise your project work overall and evaluate it critically as follows. The Conclusions chapter, like the Introduction, should be freestanding, allowing the reader to understand what the project has achieved without studying other chapters.

Evaluate your project to assess the perceived value of your system to its intended users against the specified requirements. You are not expected to create a ‘perfect’ system so marks are awarded more for the thoroughness of your validation than any praise obtained. Indeed, criticism gives you an opportunity to explain the issue raised and suggest improvement.

6 Reflection

Reflect upon the project overall and indicate what lessons you have learned along the way. Clarify what you might do differently if faced with the same situation again. In particular, you should identify and discuss how the project plan evolved as the project progressed.

7 Conclusions

You should use this section to highlight the key conclusions, findings and contributions of your work.

Length: ~ 6 pages

8 References

Making Music. 2014. Making Music, Teaching, Learning and Playing in the UK https://gb.abrsm.org/en/making-music/4-the-statistics/44-how-learners-learn/
Colgan A., 2017. Leap Motion Blog. http://blog.leapmotion.com/vr-universities/
DiMartino. 2016. A Statistical Hand Gesture Recognition System Using the Leap Motion Controller. http://www.csun.edu/~renzo/MSAbstracts/MichaelDiMartino.pdf
Gutterman, B., Rahman, S., Supelano, J., Thies, L., and Yang, M. 2009. White Paper. Information Communication & Technology (ICT) in Education for Development. Disponível em: http://unpan1.un.org/intradoc/groups/public/documents/gaid/ unpan034975.pdf [accessed May 30th, 2016]
Duffy B., Fuller R., (2000), Journal of Applied Research in Intellectual Disabilities, Volume 13, Issue 2, pages 77–89
Schellenberg, E. Glenn. (2004). Music Lessons Enhance IQ. Psychological Science, 15(8), 511-514
Prensky M., 2001 Digital Natives, Immigrants Digital Natives – http://www.marcprensky.com/writing/default.asp
T. Mogg and B. B. Corporation, “Free mobile apps drain battery faster,” 2012, [accessed 3-April-2012]. [Online]. Available: http://www.bbc.co.uk/news/technology-17431109
Pathak, Y. C. Hu, and M. Zhang, “Where is the energy spent inside my app? fine grained energy accounting on smartphones with eprof,” in Proceedings of the sixth conference on Computer systems, ser. EuroSys ’12. New York,
NY, USA: ACM, 2012.
Clean Master Games. 2016. “Don’t Tap the White Tile”, Play Store. DOI=https://play.google.com/store/apps/details?id=com.umonistudio.tile
Jones, M., Buchanan, G., and Thimbleby, H. Improving web search on small screen devices. Interacting with Computers 15, 4 (2003), 479-49
Maniar, N., Bennett, E., Hand, S., and Allan, G. The effect of mobile phone screen size on video-based learning. Journal of Software, 3, 4 (2008), 51-61.
Kim, K.J., Sundar, S.S., and Park, E. The effect of Screen-Size and Communication Modality on Psychology of Mobile Device Users. In Proc. CHI 2011, ACM Press (2011), 1207-1212.
Liang, Patey, Shah and Walters. 2018. Leap Motion Piano: Design. [Online]. [11 October 2018]. Available from: http://www.cs.columbia.edu/~sedwards/classes/2014/4840/designs/Leap-Motion.pdf
Saifi. 2017. The 2017 Mobile App Market: Statistics, Trends, and Analysis. [Online]. [11 October 2018]. Available from: https://www.business2community.com/mobile-apps/2017-mobile-app-market-statistics-trends-analysis-01750346
International Journal of Computer and Information Technology (ISSN: 2279 – 0764)
Volume 02– Issue 03, May 2013
Goel R. 2014. Advantages and Benefits of PHP Language. [Online]. [12 October 2018]. Available from: www.webinfozone.com/articles/advantages-benefits-of-php-language-articleListing-1001
John Elliot, ah-lian kor, oluwafemi ashola omotosho. 2017. Energy Consumption in Smartphones: An Investigation of Battery and Energy Consumption of Media Related Applications on Android Smartphones. [Online]. [12 October 2018]. Available from: https://www.researchgate.net/publication/319954899_Energy_Consumption_in_Smartphones_An_Investigation_of_Battery_and_Energy_Consumption_of_Media_Related_Applications_on_Android_Smartphones
Reyes J. 2007. Calpolyedu. [Online]. [12 October 2018]. Available from: http://users.csc.calpoly.edu/~zwood/teaching/SrProj/GLozartWriteUp.pdf
Denne, A. 2018. GDPR Compliance for small business – a 28 STEP CHECKLISTS. [Online]. [12 October 2018]. Available from: https://www.3bweb.com/blog/gdpr-compliance
Expertmarket.co.uk. 2018. How Much Does a Website Cost in 2018? [Online]. [12 October 2018]. Available from: https://www.expertmarket.co.uk/web-design/how-much-does-website-cost
Unity Technologies. “Unity Scripting.” Unity. N.p., n.d. Web. 12 Feb. 2014.
http://unity3d.com/unity/workflow/scripting
Unity Technologies. “Unity Technologies Delivers Unity 3.” Marketwire. N.p., n.d. Web. 19 Jan. 2014. <http://www.marketwired.com/pressrelease/Unity-Technologies Delivers-Unity-3- 1325564.htm>.
Praveen, K. 2016. Unity Software Performance Optimizations for Games: Best Practices. [Online]. [14 October 2018]. Available from: https://software.intel.com/en-us/articles/unity-software-performance-optimizations-for-games-best-practices

9 Appendices

The appendices are an opportunity to provide secondary material in support of the description in the body of the report. In principle, the reader need not look at the appendices and no specific marks are awarded for this section.

Sample content: