Wednesday, December 26, 2012

C# XAudio2 Sound Playback for Windows Phone

     Let's begin with a small introduction to XAudio2:
     XAudio2 is a low-level audio API. It provides a signal processing and mixing foundation for games that is similar to its predecessors, DirectSound and XAudio. XAudio2 is the replacement for both DirectSound and XAudio.
     XAudio2 abstracts audio generation by separating sound data from "voice", allowing each voice to be filtered by programmable digital signal processing and effects processing functions. Voices can be "submixed" together into a single stream. There is always only one Mastering Voice that outputs the result using WASAPI.

     XAudio2 is primarily intended for developing high performance audio engines for games. For game developers who want to add sound effects and background music to their modern games, XAudio2 offers an audio graph and mixing engine with low-latency and support for dynamic buffers, synchronous sample-accurate playback, and implicit source rate conversion. Compared to WASAPI, XAudio2 requires only a minimum amount of code even for complex audio solutions. Compared to the Media Foundation engine, XAudio2 is a low-level, low-latency C++ API that is designed for use in games.
     XAudio2 cannot be used for background music - for this task you will have to use the IMFMediaEngine. XAudio2 cannot be used for capturing audio - for this task you will have to use WASAPI. Do not use XAudio2 for media playback. For that task you can use MediaElement
    XAudio2 is part of the DirectX api that is included in the new Windows Phone 8 SDK. The Api is shared between Windows 8 and Windows Phone 8 which means that you will be able to fully reuse your source code on both platforms. 

If you want to use XAudio2 for your C#/VB/HTML code you have two options:
1. Use SharpDX . SharpDX is a wrapper of the DirectX Api under .Net platform. Theoretically you can use it to call XAudio2 api directly from your managed code. Practically what happens is that the .Net CLR/GC on ARM seem to block native threads so your audio will shutter/glitch in certain conditions. I had the same problem when I was developing our Windows 8 game Kids' Orchestra and the audio had glitches even on a core i7 processor.
2. The other option, which from my experience works better, is to develop an Windows Phone Runtime Component that will manage the XAudio2 part and expose the needed methods/events to the managed code.

      To better understand how it is done I took the Windows 8 sample XAudio2 audio file playback sample C++ from MSDN and ported to Windows Phone 8 by splitting it in two projects: The C#/Xaml part for the UI and the "audio" project which is a Windows Phone Runtime component developed in C++.
     The porting was pretty easy. I only had to re-code the player class to make it "visible" to the managed code project and added an event that will tell you when a certain Source Voice has finished playing its buffer/sound (we have 7 sounds and each sound has a Source Voice associated to it). If you need further details on how to write a Windows Phone Runtime component in C++ have a look at this  MSDN Post

    This sample only plays Wav files that are resources in the C++ project. You could also dynamically generate sounds in managed code and pass the Wave/buffer data as a byte[] to the runtime component. Inside the native code you will then generate an XAUDIO2_BUFFER and submit it to a Source Voice for playing.

     I have attached the SOURCE CODE for the Windows Phone project. If you have problems with it don't hesitate to contact me.

NAMASTE!

12 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. Many thanks for the work you have shared here. I'm currently working on a WP7 app that streams its own generated audio by overriding a MediaElement but it is glitching on WP8 so this is probably the solution. The only trouble is I don't know C++ at the moment. Would your sample need much work to modify it. I assume I would need to implement a callback to keep filling the buffer?

    ReplyDelete
  3. Mark it won't work in WP7 as C++ was not available. Sorry

    ReplyDelete
  4. Brilliant,
    Where did you get the sounds from? I used your sample as a base for my own code. I used electro_adpcm.wav for the background music in my game just for convenience but I fell in love with it and would like to use it. I'll be selling my game on the Windows Phone Store soon. Are these wav files part of the public domain?

    Thanks!

    ReplyDelete
  5. Hi Matthew,
    The sounds are from the Windows 8 sample and I guess that are public domain.

    ReplyDelete
  6. can you have multiple sounds play in the background as well using XAudio2 (e.g. while you use other apps)?

    ReplyDelete
  7. This is great! Another task for me to master. Thank you very much for the update. I'll be waiting for more latest enhancements in C#.

    ReplyDelete
  8. Hi Dan,
    I am a new Windows Phone developer. I am creating an karaoke app, could you please suggest me some way to mix the music which is emitted from MediaElement and sound which is recorded from earphone ? Please help me, I really need this.
    Thanks very much!!!!

    ReplyDelete
  9. What specials those wav files are? Why I can't play my wav file (converted to Microsoft adpcm, mono with audacity)?

    ReplyDelete
  10. Nguyen: try adpcmencode.exe from WP8 SDK

    ReplyDelete
  11. I cannot get any fileformat to work inside this project? I have converted several audio files with different converters. used the adpcmencode and got an output file. But no file works in the project :S? Can you specify some information for making the audio files?

    ReplyDelete
    Replies
    1. I finally got it working first converting through Free Audio Converter, and then with adpcmencode.exe and it works :D

      Delete