Sunday, September 27, 2015

Using iBeacons with Windows 10 - Part I

It’s been a while since my last post on the blog, but it was such a busy period and sadly my blog is not one of my main priorities. Would really love to find more time for writing.
This post is focused on “consuming” iBeacon devices around us and also configuring your device as an iBeacon using the newly added functionality in the Universal Windows Platform target Windows 10 Build 10240. 
A little bit of introduction to the beacon/Bluetooth technology. The iBeacon functionality is built on-top of the Bluetooth Low Energy (BLE) and enables to create a region around an object. This allows client devices (Android, Windows, iOS) to determine when it has entered or left a region along with an estimation of proximity to the beacon. The BLE standard architecture (and not only) is a client-server one. The device that exposes the functionality is the server (in our case the beacon) and our device (phone, tablet) is the client. In order for a client to know that a server is in range the server advertise itself (sending bytes packages at certain intervals times on different radio channels). The battery life of an BLE device depends mainly of the advertising interval (this is because usually the device is idle and advertising). With one advertising packages each second the lifetime of a beacon device running on a CR2032 battery is about 1 year. The client on the other side is scanning the communications channels for Bluetooth advertising.

Here is how an advertising package looks like:

The  “standard sequence” includes: 

  • Preamble and Access Address is always the same 
  • Packet Data Unit
  • CRC to understand if the received package is correct or not.
The PDU can have a length between 8 and 39 bytes where the first 4 bits is the PDU type. For this post we are interested in two types ADV_IND – Connectable Undirected Advertising meaning a device that is advertising and it can be connected to and ADV_NONCONN_IND Non connectable undirected advertising is a device advertising but not connectable. If you want to better understand the advertising package read this post and the complete list of AD Type can be found here:
So we could consider a Beacon any Bluetooth Low Energy that is advertising. The iBeacon (and EddyStone but that we will address in another post) is just a standard advertising package. 

The structure of the iBeacon packages is:

The iBeacon prefix bytes are always the same:

iBeacon Prefix
Number of bytes that follow (AD Length)
AD Type
Flags type
26 bytes that follow in the second AD structure (AD Length)
Manufacturer specific data AD Type
4C 00
Company identifier 0x004C = Apple
Secondary ID that denotes a proximity beacon, which is used by all iBeacons
defines the remaining length to be 21 bytes

              The developers can configure for each iBeacon (using the tools that manufacturers provide) the UUID, Major, Minor and TX Power. 
        Here is how the advertising looks like from an Estimote iBeacon. The package was captured using Packet Sniffer from Texas Instruments:

              As you can see the device can be connected to because the advertising type is ADV_IND but this is not always true. If you have a generation 1 Texas Instruments SimpleLink SensorTag with the iBeacon firmware flushed you can switch between “normal” mode and iBeacon mode and the device in iBeacon mode cannot be connected (it is only an iBeacon without any service). Here is the package captured by the sniffer:

Btw if you want to update the Gen 1 SensorTag to the iBeacon firmware you can follow these instructions. I haven’t compiled the source code but downloaded the firmware from this link. You will need an Android or an iOS device even if right now the TI application is broken for both Android 5 and iOS 9 (I had to roll back to iOS 8 in order to update the SensorTag firmware). If you look at the two packges you will see that the SensorTag is missing and ADV sequence 02 01 06 but because we are going to filter using the ManufacturerData we will still intercept this type of iBeacona correctly.
Also remember that the only Beacon standard supported by all the mobile platforms iOS, Android, Windows is iBeacon.  Windows 10 and Android can actually support natively any type of Beacon.

              After the standard Prefix we have the actual data of the iBeacon:

16 bytes
Application developers should define a UUID specific to their app and deployment use case.
2 bytes
Further specifies a specific iBeacon and use case. For example, this could define a sub-region within a larger region defined by the UUID.
2 bytes
Allows further subdivision of region or use case, specified by the application developer.
TX Power
1 bytes
The calibrated(known) measured signal strength in Rssi at 1 meter

You might find on some websites that the Tx Power length is 2 bytes but it is not true. The Tx Power is 1 byte signed.
For the tests and development, I have used both an Estimote beacon and the SensorTag with the iBeacon firmware.

So now let’s get back to Windows 10 and the new SDK. We now have support for:
  • BluetoothLEAdvertisementWatcher - capturing advertising packages in the foreground
  • BluetoothLEAdvertisementPublisher – set a new advertising package for the
  • BluetoothLEAdvertisementWatcherTrigger – the same as the watcher only that it sets a trigger and the code associated with the trigger will run in the background even if the application is not in the foreground or running
  • BluetoothLEAdvertisementPublisherTrigger – the counter part of the advertising publisher for advertising in the background

IMPORTANT: even if a device advertises ADV_IND meaning that a client can connect to it the UWP Bluetooth functionality for Windows 10 build 10240 is still LIMITED because it can only connect to the device if it was previously paired with Windows. For the moment there is no way to pair to new devices from code so the application would open the Bluetooth window, the user have to manually pair with the device and then, when coming back to the application, we can connect to the device and enumerate services and characteristics.
If we want to monitor for iBeacons only in the foreground (while our application is running) we will use the BluetoothLEAdvertisementWatcher class. 
IMPORTANT: when you create a new project in order to be able to interact with the iBeacons from your UWP project you have to open the Package.appxmanifest file and enable Bluetooth  Capability:

Using BluetoothLEAdvertismentWatcher we can monitor the advertisement packages from any BLE device around us. Of course this is a little bit excessive if we only want/need to monitor a certain class of iBeacon. This can be done using the AdvertisementFilter property of the watcher. To make things easier I have developed a small helper class (that hopefully will grow in time) that will actually set the BluetoothLEAdvertisment with the desired iBeacon sequence.

The extension method uses an iBeaconData class as input:

public class iBeaconData
        public Guid UUID { get; set; }
        public ushort Major { get; set; }
        public ushort Minor { get; set; }
        public short TxPower { get; set; }
        public double Distance { get; set; }
        public short Rssi { get; set; }
        public iBeaconData()
            UUID = Guid.Empty;
            Major = 0;
            Minor = 0;
            TxPower = 0;
            Distance = 0;
            Rssi = 0;

To set the BluetootLEAdvertisement you just need to set the iBeacon data and pass it to the extension iBeaconSetAdvertisement. If we pass the class without setting any property we will monitor for all iBeacon advertisement. If we want to monitor the iBeacons with a certain UUID we just have to set the UUID property before calling the extension method. On the other hand if we set the Major property we will also have to set the UUID to which the Major corresponds. If you look at the source code you will see that the filters work in sequence: If we set the Minor we need to set the Major and UUID, if we set the Major we need to also set the UUID. This is because we are looking for certain patterns of bytes in the advertisement package. Here are some samples of the API callesìd:

// Create and initialize a new watcher instance.
watcher = new BluetoothLEAdvertisementWatcher();

// Monitor all iBeacons advertisment
watcher.AdvertisementFilter.Advertisement.iBeaconSetAdvertisement(new iBeaconData());

// Monitor all iBeacons with UUID
    new iBeaconData()
        UUID = Guid.Parse("{307f40b9-f8f5-6e46-aff9-25556b57fe6d}")

// Monitor all iBeacons with UUID and Major 
    new iBeaconData()
        UUID = Guid.Parse("{307f40b9-f8f5-6e46-aff9-25556b57fe6d}"),

// Monitor all iBeacons with UUID and Major 
    new iBeaconData()
        UUID = Guid.Parse("{307f40b9-f8f5-6e46-aff9-25556b57fe6d}"),
        Major = 18012,

The Advertisement filter is not the only filter that you can set for the watcher. You can have a look at all the other parameters of the class on MSDN.
The other helper method, iBeaconParseAdvertisement, helps parsing the received advertisement package and retrieve the iBeacon data: UUID, Major, Minor, TxPower and Distance. The Distance is an approximate one and it is calculated using the TxPower (the Rssi level measured at 1m) and the actual Rssi level. To better understand why is hard to have a really precise value read this blog post about Fundamentals of Beacon Ranging  (on iOS devices the distance is more precise because Apple is the only manufacturer of its devices so he can tune the bluetooth chipset settings to have similar results on all the devices meaning the Rssi level measured on the iPhone at 1m would be almost the same for the iPad or iPod).
In the sample I have subscribed the Received event handler of the watcher an simply called the method iBeaconParseAdvertisement to get the iBeacon data.

// Get iBeacon specific data
var beaconData = eventArgs.Advertisement.iBeaconParseAdvertisement(eventArgs.RawSignalStrengthInDBm);

Now let's see how we can scan for iBeacons from the background (while our application is not running). The SDK adds a new background trigger specific for BLE advertisment that is why it is called BluetoothLEAdvertisementWatcherTrigger. If you don't know how the background tasks work read read this: Creating and register a background task .
The main difference between scanning in the foreground and in the background from the BLE point of view is the minimum sampling period: when scanning in the foreground the value can be 0, while in background the minimum sampling interval is 1 second. We cannot have more than one event/second in the background. If more than one iBeacon has advertised in that period we will find all the advertisements packages using the Advertisements property. Remember also that the background task runs in a different process but in the same container (so the storage is shared).
Also in this case when setting the trigger I am calling the extension method that sets the filter for the advertisement data:

// Create and initialize a new trigger to configure it.
trigger = new BluetoothLEAdvertisementWatcherTrigger();
// Add the manufacturer data to the advertisement filter on the trigger:
trigger.AdvertisementFilter.Advertisement.iBeaconSetAdvertisement(new iBeaconData());

I've published the sample project on GitHub (here is the link ). The solution has 4 projects:
  • Beacons.Helper - UWP library that contains the extension methods
  • Beacons.Universal.Foreground - the sample for the  BluetoothLEAdvertisementWatcher
  • Beacons.Tasks - the runtime component that contains the background tasks
  • Beacons.Universal.Background the sample for register/unregister and monitor a background task that scans for iBeacons
The next blog post will be on configuring the device as an iBeacon. We will look at the new publisher API. 

If you have questions don't hesitate to contact me (you can find me on twitter or email - I have disabled the comments on the blog because there was a lot of SPAM)

I also recommend (if you have time) to watch the Build 2015 session of Kiran Pathakota :  Building Compelling Bluetooth Apps in Windows 10
Hope you've enjoyed the post!


Wednesday, June 10, 2015

Manually install Google Apps on Hyper-V Android emulator

    One of the "hot" new features of Visual Studio 2015 is the new Android Hyper-V emulator. It is an awesome feature as you have a highly performant Android emulator running on the same virtualization engine as the Windows Phone emulator. 
   It is an x86 virtual machine and similar to other Android Virtual machines like Genymotion, Xamarin Android Player the Google applications and services are not installed by default.
     For the other engines they usually support installing Google applications by drag-drop. You just have to download the Google Apps specific for the version of Android your emulator is running (you can download it from here ) and drop the package on the emulator running. For the moment this feature is not supported in the Hyper-V version (might still coming) but you can do it manually with a simple sequence of instructions. 
    This is what you need to do:

1. Download the GApps packages specific for the Android version your emulator is running
2. Unzip the file
3. Open an adb command prompt (if you have the Xamarin Tools installed in Visual Studio 2015 open the menu Tools->Android->Android Adb Command Prompt) and navigate where you have unzipped your files and go to the system folder from the archive.
4. Execute this set of commands:
  • adb remount
  • adb shell chmod 777 /system
  • adb push . /system

What this commands do is remount the system partition of the emulator to be writable and pushes the contents of the system folder from the unzipped package to the system folder on the emulator. This is the only things that you have to do. Just wait a little bit and you will see that the emultator starts to install everything needed. You might need to soft reset the Hyper-V emulator to make everything work (you should be actually prompted to login with your Google account). Before pushing the System folder content you can also delete the packages that you are not using.

Let me know if you have problems or if i works via email or twitter (disabled the comments on the blog because of all the SPAM I am getting)

P.S. If the links to the google apps package are not working just copy the package name and paste it in a search engine of you choice and you will easily find a mirror for the package.

Thursday, January 1, 2015

Using the Android Hyper-V Preview with Visual Studio 2013

   Happy New Year to you all! While I am anxiously waiting to know if I will be renewed today as an Microsoft MVP (btw being MVP is an awesome experience, the MVPSummit was a great experience even if I was expecting to see more new things so if you can get involved and try this experience). Still planning to do the post on BLE background agents but there are so many things to write that I have to find time to write it. Hopefully this will happen soon.
    So this small post is all about running the Android Hyper-V Preview Emulator that comes with the Visual Studio 2015 Preview as a stand-alone emulator and use it with your production machine running Visual Studio 2013 and the Xamarin tools. Even if the emulator is still in preview if you are doing Android and Windows Phone development on the same machine you won't have to reboot your machine without Hyper-V support in order to have a decent Android emulator. Strangely Microsoft decided for this preview that the emulator can install only if you have the preview of Visual Studio 2015. In theory you could run the preview along with Visual Studio 2013 but things are not always perfect so many of you will keep two different instances for VS 2013 and the Preview. In oreder to get the emulator work you will still have to install the preview of Visual Studio 2015 on some virtual machine or secondary machine and copy the needed files.
    The method is pretty simple. All you will need to do is to copy the folder "Microsoft Visual Studio Emulator for Android" that is located inside Program Files (x86) [ %PROGRAMFILES(X86)%] folder to the machine you want to run the emulator on. At this point you have everything you need. Copy the folder at the same location on the other machine. We still need to take two steps before launching the emulator for the first time. Inside the folder that you've just copied there are two files vsemu.vhd and sdcard.vhd that will be used to "generate" the two virtual machines (phone and tablet). Here are the two steps that you have to take:
  1.  Make sure that adb.exe is in your Path as it will be used by XDE.exe
  2. Create a new folder called Android inside the folder "%userprofile%\AppData\Local\Microsoft\XDE" and copy the two vhd each of them two times with the following names:
     vsemu.vhd copied as: and
     sdcard.vhd copied as and

     Now we are ready to start the emulator. You won't be able to start the emulator from Visual Studio 2013 the way VS 2015 Preview does but you can start it manually and once you've started it and the adb automatically connects to the emulator you will see it in Visual Studio as a deployable device. Let's not mention that you can actually use the accelerometer, the location, battery simulation and the screenshot (awesome).

     Here are the two commands that you need to run (make a batch on your desktop):

Phone Android Emulator:
"%PROGRAMFILES(X86)%\Microsoft Visual Studio Emulator for Android\1.0\XDE.exe" /name "VS Emulator Android - Phone" /vhd %userprofile%\AppData\Local\Microsoft\XDE\Android\ /video "720x1280" /memsize 1024 /telemetrySession "14.0;eyJBcHBJZCI6MTAwMSwiRGVmYXVsdENvbnRleHRJZCI6ImJkNmMxNjk2LTQ5ZjUtNDI4My1iZTk2LWUzNGNjZWYwZTA4YSIsIkhvc3ROYW1lIjoiRGV2MTQiLCJJZCI6ImIyOWZlODMwLThhOWMtNDE1Yy04MWExLWFlNTFkNzFiYTUwNSIsIklzT3B0ZWRJbiI6dHJ1ZSwiUHJvY2Vzc1N0YXJ0VGltZSI6NjM1NTU2MTAxODUyMjI0MzA1fQ=="

Tablet Android Emulator:
"%PROGRAMFILES(X86)%\Microsoft Visual Studio Emulator for Android\1.0\XDE.exe" /name "VS Emulator Android - Tablet" /vhd %userprofile%\AppData\Local\Microsoft\XDE\Android\ /video "1080x1920" /memsize 2048 /telemetrySession "14.0;eyJBcHBJZCI6MTAwMSwiRGVmYXVsdENvbnRleHRJZCI6ImJkNmMxNjk2LTQ5ZjUtNDI4My1iZTk2LWUzNGNjZWYwZTA4YSIsIkhvc3ROYW1lIjoiRGV2MTQiLCJJZCI6ImIyOWZlODMwLThhOWMtNDE1Yy04MWExLWFlNTFkNzFiYTUwNSIsIklzT3B0ZWRJbiI6dHJ1ZSwiUHJvY2Vzc1N0YXJ0VGltZSI6NjM1NTU2MTAxODUyMjI0MzA1fQ==" /noStart

You won't probably need the /telemetrySession but this way you could give back some feedback (I guess). Also the first time you run the emulator it will ask permission to configure the Internet on the device and of course you will have to accept.

Also while digging to make this work I found out that the source code of the Android Preview Emulator is published here: . In the same place there is also the source code for the Microsoft Wireless Display Adapter. It looks like it is running on Linux kernel with a pretty decent CPU and we might get some new features other than a simple Miracast adapter.

Until my next blog post: NAMASTE and again HAVE A GREAT NEW YEAR!