In our latest music app “Sounds of Christmas by mDecks Music” we wanted to create the illusion of playing a virtual instrument with your iPhone in 3D space. The user would hit “AIR” as if it were Bells and the iPhone will produce sound (notes, chords, etc)
To achieve such effect, we had to user iPhone’s CoreMotion Technology which allows access to all the iPhone’s motion detection system. You can read more about it on my previous post Turning an iPhone into a Musical Instrument
Now, if you have ever worked with MIDI, Samplers and Sequencing software, you know what latency is. For those who do not know: Latency is the amount of time that takes all your software and hardware to turn an “action” (command, process, etc) into sound.
There’s always latency involved in sound production, regardless if you are using computers or not. The speed of sound is not infinite which could be interpreted as already having an embedded amount of latency ( you know how in a tennis match you see the ball being hit and much later you hear the sound.) There’s latency when you play a real Acoustic Piano, since your action of pressing a key needs to be mechanically translated into hitting a string with a hammer. Of course these conversions are so fast that the pianist would not notice them (or at least can cope with)
When using MIDI, software and samplers, we add a full new set of conversions that need to be done before sound is finally produced. Your action in the keyboard is turn into a midi signal, the midi signal is turn into a note request for the sampler which has to ask the computer to access the hard drive (or memory if possible) and read the sound file which is then processed into a music note sent to your sound card which converts that information into sound and output it via your speakers.
An acceptable latency is one that produces the sound fast enough for you to perceive it as being in real time, and does not affect your playing no matter how fast you play. Musicians are tuned to feel that delay much better than the average listener and a delay of 20ms is big enough to be noticeable. An acceptable latency for musicians is below 12ms.
If we want to use the iPhone motion to trigger notes we then have to treat that motion as if it were a key been played on a keyboard, then turn it into MIDI send it to a sampler, etc. It was crucial to create a method that could detect and process motion in a very short amount of time (<1ms) and the question was if iPhone’s CoreMotion was fast enough to make this possible.
After our first test we realized that not only iPhone’s CoreMotion is incredibly fast but it is also so precise that we were able to simulate virtual Bells begin hit by the iPhone at any tempo we wanted.
Christmas Sounds by mDecks Music turned also into a rhythmic and sight-reading training tool where the user can perform a song at any tempo and, we decided to include rhythm scores of all songs for music teachers and students since this is such an intuitive way of learning and practicing rhythms
You can print all scores from your iPhone within the app or, you may download the eBook in PDF format with all songs notated as rhythm.
Yesterday, after 14 days, a new version of our music app Rhythmic Vocabulary for All Instruments Vol. 1 was Rejected by the review team on Apple. This version of the app has an in-app purchase where the user can buy Vol. II within the app.
I made two mistakes when submitting the app for review:
- I forgot to upload a in-app screenshot, which is not optional and it is said to be used in the review process only.
- I did not add the In-App Purchase on the main screen for the app submission
In the email from Apple, they described generic problems with in-app purchases but they do not tell you specifically what the problem is. Apparently the most common problem is not having a restore button (or feature) for the user to retrieve the in-app purchases previously made when they re-install the app or they download a new version and, I knew that was not the problem with mine.
I wrote to apple and they promptly reply telling me to submit a new build of the app and to make sure the in-app purchases were included with the build. That meant: Add a screenshot, and add the in-app purchases that you want to include in the app (on the main page)
Now we have to wait for review for another 14 days or so.