I didn't want to dedicate years to obtaining PhD in computer vision or spend my life doing statistics. But, like any other good developer, I'll just take the toolkit someone else came up with to solve the hard problems.

Enter the Kinect.

In reality, Microsoft has had a Speech API for quite some time and I've played with it in the past, but with the Kinect they've produced a beautiful piece of hardware that can do 3D depth sensing, speech recognition and directional detection, live skeletal tracking and more all in package that doesn't cost a lot more than a high-end web cam.  This first example doesn't actually require the Kinect, in fact, the code itself just uses the default audio input.  But it can be easily changed to use the Kinect audio stream.  Later projects I'm working on will use the Kinect camera's to do some hopefully neat things.

Since the Kinect hacking started my brain has been churning with ideas.  Of the most pragmatic, was the thought to tie in speech recognition to an MVVM application.  The nice thing about a well implemented screen using MVVM is that you have your UI described seperately in XAML on the front end and a class (ViewModel) containing your library of commands that can be executed.  Using a Command object you can tie a specific element, like a button, to perform a specific command like Save very cleanly and easily.

This clean seperation of concerns mean you don't really care how a command is invoked, whether it's a button press, keyboard shortcut, or voice command it all works the same.  Your ViewModel executes the command and the UI happily updates through the powerful databinding of XAML.

Aside from the obvious scifi references that this brings to mind, it could also help by making programs more accessible to the vision or mobility impaired.  Also, it could be just plain more efficient in some scenarios.

Most of the work is done for us by the commanding infrastructure in WPF.  So first I'd like ta take a look at how this implementation will be used.  Below is a standard button declaration with a command attached.

<Button Content="Save" Command="{Binding Save}" />

The other great thing about XAML is the extensibility, so by the time we've implemented this speech API the only thing that will change is this:

<Button Content="Save" Command="{Binding Save}"
             voice:SpeechCommand.Phrase="save" />

One simple property added and that's pretty much all the end developer needs to do.  The only other thing we need for using the speech recognition is something I call a SpeechCommand, which is basically just an implementation of the standard DelegateCommand found in MVVM frameworks.  The SpeechCommand acts exactly like the standard commands, but it is also the place for the Phrase AttachedProperty to live and is the glue that bridges the application to my wrapper around the Speech API.

In the next post I'll walkthrough how I built the app and post some source code.  Until then, I leave you with a screenshot.  Please note, that no mouse or keyboards were harmed (or used ) in the taking of this screenshot.

Some examples include, Binding, Static- and DynamicResource, Type, Static, etc.  You can look at the code below and see some examples:

<Grid x:Name="LayoutRoot" >
    <ContentControl Style="{StaticResource BgBlueTop}"/>
    <Border >
        <ItemsControl ItemsSource="{Binding Messages}">
             ...
        </ItemsControl>
    </Border>
</Grid>

Basically, the xaml parser can interpret an attribute as a string literal value or convert it to an object through some means.  Markup extensions allow you to do the deciding about how the value you're setting should be interpreted by the xaml parser.  For a much more in depth look at how this all works, a decent place to start is this link on MSDN:

http://msdn.microsoft.com/en-us/library/ms747254.aspx

One interesting use of MarkupExtensions I've been playing with is a reimplementation of the Binding markup extension to provide an alternative to the IValueConverter interface. Sometimes you really only need to do a conversion once or a couple times on a specific screen.  When you implement a value converter it feels like you are taking a tiny bit of view or business logic and stuffing it in an unrelated portion of your application.  That being said, a standardized library of value converters like boolean-to-visibility and so on can make developing on a project much easier to use.  But for the one off scenarios, it'd be nice to keep all that logic contained in your ViewModel and not have to worry about spinning up a new class for such a simple thing.

I used the post here as a starting point for my custom binding because working with Binding or BindingBase was not going so well.  From there I created my own Binding class and added a ConverterDelegate property.  This property looks for a method on the same DataContext of the binding and uses a generic IValueConverter behind the scenes to call that method of the DataContext.  This helps get rid of all those little one-off classes that you have to create for specific conversion scenarios.

The code is far from production ready but the gist of it is that a custom MarkupExtension overrides the ProvideValue method of the abstract MarkupExtension class. In the background I look up the method on my DataContext and use a IValueConverter that calls that method to do the conversion. So to wire up my converter I can simply do this:

<Window x:Class="CustomMarkup.Window1"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="clr-namespace:CustomMarkup"
    Title="Window1" Height="300" Width="300">
    <Grid>
        <Button Content="{local:ExBinding Path=ButtonText,
                ConverterDelegate=ToUpper}"
                Height="25" Width="100"/>
    </Grid>
</Window>

You can see I have a converter delegate called ToUpper and in my code behind or ViewModel I can simply write whatever methods I need to do conversions like this:

    public partial class Window1 : Window
    {
        public Window1()
        {
            DataContext = this;
            InitializeComponent();
        }

        // our converter method
        public object ToUpper(object value)
        {
            return (object)value.ToString().ToUpper();
        }

        public string ButtonText { get; set;}
    }

I can post the sample project if I get interest but the real point is to give a small showing of what's possible with custom MarkupExtensions in WPF and SL5.

Unfortunately, I chose wrong. I should have used the Enterprise version because that's what we are licensed for. In the end, they are the same version, but Enterprise is sold only to businesses and Ultimate is sold to home users.

So you think it would be an easy task to switch from one to the other, but the only officially supported way is to completely reinstall and start over. I had just spent the entire night setting up all my software, so I didn't really feel like doing it twice two nights in a row.

Luckily, there is a way to workaround the limitation, I've pulled and simplified from here: http://www.mydigitallife.info/2009/11/03/hack-to-in-place-downgrade-from-windows-7-ultimate-or-professional-to-less-premium-editions/

The condensed version is to:

1. Run RegEdit on your computer and navigate to HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\Current Version
2. In the right hand side change the word "Ultimate" to "Enterprise" under both the EditionID and ProductName entries
3. From there I inserted the Windows 7 DVD and ran setup and chose upgrade and everything worked flawlessly

Note: I reran the setup from inside of Windows. I did not boot from the DVD to do the upgrade so I don't know if it works that way or not.

I've had to dredge up old memories from my college physics classes, and I'm not entirely sure what I've got is hundred percent accurate, but I feel with proper setup it's pretty good.  The calculation is easy enough, to get the force you just multiply the mass times the acceleration of the object.  The accelerometers in the Wii remote are accurate to about  +/- 3 Gs with about a ten percent range of error.  Not too terrible for a fifty dollar bluetooth enable accelerometer you can pick up at any store in town.  The problem is, if you try to hold it in your hand you can generate massive amounts of acceleration well above 3 Gs.  So instead,  I tied the remote onto a heavy bag.  Much more mass, means you get the same amount of force with acceleration levels well within the remotes acceptable range.

Force = mass * acceleration

Because I'm not punching any harder, the force stays the same.  The heavy bag has a lot more mass, so the acceleration has to be a lot less to equal the same force.

The thing I'm not sure of is whether I have to take into account the fact the bag is on a chain so it actually swings up in an arc.  I don't know how much that affects the calculations, but I don't think it would be a lot with the small amount of time range we're talking about.

My next steps are to try and rig up some testing system to try to determine the accuracy of the measurements.  I also want to get the opinion of some physics people to get my calculations even more accurate.  It's been a fun project so far and I'd like to get it to the point where I could bring it into the gym and have my students play around with it a little bit.

The application itself is written in C#, using WPF, MVVM, and Wiimote Lib

I'll try to post any updates here as they develop.

]]> http://bignickolson.com/2010/08/11/wii-remote-force-calculator/feed/ 4 http://bignickolson.com/2010/04/27/uses-for-silverlight-reflection-pt-ii/ http://bignickolson.com/2010/04/27/uses-for-silverlight-reflection-pt-ii/#comments Tue, 27 Apr 2010 16:17:20 +0000 Nick http://blog.bignickolson.com/?p=288 So, as promised, here is the followup to what I was doing with Silverlight reflection that made me need access to Internal members of a class.

Localization in Silverlight is still an interesting story and everybody seems to have their own way of doing it.  The client I'm on uses Excel spreadsheets that load into a database. The localization data then pulls when a usercontrol loads and changes the controls data based on the locale and what was in the database.

This was very tedious for developers to setup.  You would create your UI and then have to go back and pull the default text for your controls and add them all to the spreadsheet.

Using reflection I can create an instance of all the user controls in my assembly and display them to the user in a listbox.  From there, when they select a user control, I can reflect on that type and display all the controls defined in the UserControl to the user.  When you define a control in xaml and give it a name, it becomes an internal member of a partial class generated by Visual Studio.

So, what I do is when you are all done with your UserControl you simply run my tool on the assembly and it reflects through and shows all the UserControls. When you select one, it uses reflection to create an instance with the default constructor, which initializes all your controls properties.

From there I grab all the controls through reflection and use some logic to get which properties of each I want to localize.  For TextBlocks I grab the Text property, for ContentControls it's the Content property, and so on.

Using that I generate a simple tab-delimited string with all the default localization for all the controls and the user can copy/paste right into Excel and move on to their next task and let someone else do the translating/localizing for them.

However, I have a scenario that would be greatly helped by being able to reflect on internal members.  The documentation I found on MSDN said that as long as the member was visible at compile time you could reflect on it.  This should include internal members in the same assembly or an assembly that's the target of InternalsVisibleTo.

I was having problems, at first I just tried setting the InternalsVisibleTo attribute in the AssemblyInfo.cs for my project to the project that had my reflection code. 

No such luck, even though I could access the members through normal means.  What I was missing was setting the BindingFlags attribute while calling GetMembers on my type. 

If you pass in BindingFlag.NonPublic | BindingFlags.Public | BindingFlags.Instance it should all work for you now.

Up next will be a post sometime about why I wanted this ability.

It seemed rebooting fixd the problem, but I'm not gonna reboot everytime I cradle and un-cradle.

So I found another solution that seems to work, when you cradle and nothing happens, uncradle the device again and open up Windows Mobile Device Center, formerly ActiveSync on XP.

Under Connection Settings, click the checkbox to disable DMA connections.  Then hit ok.  Once that is set, reopen the settings and re-enable DMA connections.  Once the settings are applied you can go into the Device Emulator Manager and cradle the device again, it should work now!

There are a number of ways to do this but the one I've settled on after tackling it on a few projects is an adorner that automatically overlays and control with any content you want. 

Other options include using the Popup control, which is problematic because popups are not part of the normal visual layout.  They are always on top of all other content and don't move when you resize or move the window, at least not automatically.  Another way you can do it is put everything inside a grid, and add the content you want to overlay with at the end of the Grid's content with no Row or Column specification.  You can set the visibility to collapsed and show or hide based on databinding or triggers, etc.  This works better than the popup for resizing, but is not as reusable.  Even though the adorner is a bit more code, I think it's more reusable and better than the Popup option.

The way I use it is I create a UserControl that will be my overlay, let's call it ProgressMessage.  I've got a Grid I want to overlay called LayoutRoot.  I can then call OverlayAdorner<ProgressMessage>.Overlay(LayoutRoot).  Now my grid will be overlaid with the ProgressMessage user control.  I've also provided an override of the Overlay method so you can actually pass in an instance of the content you want to overlay with.

I use a factory pattern and how IDisposable/using statements work to automatically create/remove the adorner.  You could also store the IDisposable that's returned and call Dispose later to remove the AdornerLayer

using (OverlayAdorner<ProgressMessage>.Overlay(LayoutRoot))
{
  // do some stuff here while overlaid
}

A couple of quick notes, because of the way WPF layout and hit-testing works, you should not have any height or width set on your overlay content, and the background needs to be non-transparent.  To get a semi-transparent background use the alpha-portion of the aRGB color format on your background.  So instead of Black, use #44000000 and that gives you a semi-transparent gray background.  Additionally, all these methods block mouse input, but the keyboard navigation remains active.  I've started playing with lost focus events and other methods to intercept losing focus and retain that.  Otherwise the user can tab through the controls underneath the overlay and activate them using arrow keys, enter and space bar.  You can either solve this, or once I straighten it out I'll post what I come up with

Here is the rest of the class, OverlayAdorner.cs

  /// <summary>
   /// Overlays a control with the specified content
   /// </summary>
   /// <typeparam name="TOverlay">The type of content to create the overlay from</typeparam>
   public class OverlayAdorner<TOverlay> : Adorner, IDisposable where TOverlay : UIElement, new()
   {
      private UIElement _adorningElement;
      private AdornerLayer _layer;

      /// <summary>
      /// Overlay the specified element
      /// </summary>
      /// <param name="elementToAdorn">The element to overlay</param>
      /// <returns></returns>
      public static IDisposable Overlay(UIElement elementToAdorn)
      {
         return Overlay(elementToAdorn, new TOverlay());
      }

      /// <summary>
      /// Overlays the element with the specified instance of TOverlay
      /// </summary>
      /// <param name="elementToAdorn">Element to overlay</param>
      /// <param name="adorningElement">The content of the overlay</param>
      /// <returns></returns>
      public static IDisposable Overlay(UIElement elementToAdorn, TOverlay adorningElement)
      {
         var adorner = new OverlayAdorner<TOverlay>(elementToAdorn, adorningElement);
         adorner._layer = AdornerLayer.GetAdornerLayer(elementToAdorn);
         adorner._layer.Add(adorner);

         return adorner as IDisposable;
      }

      private OverlayAdorner(UIElement elementToAdorn, UIElement adorningElement)
         : base(elementToAdorn)
      {
         this._adorningElement = adorningElement;
         if (adorningElement != null)
         {
            AddVisualChild(adorningElement);
         }
         Focusable = true;

      }

      protected override int VisualChildrenCount
      {
         get { return _adorningElement == null ? 0 : 1; }
      }

      protected override Size ArrangeOverride(Size finalSize)
      {
         if (_adorningElement != null)
         {
            Point adorningPoint = new Point(0,0);
            _adorningElement.Arrange(new Rect(adorningPoint, this.AdornedElement.DesiredSize));
         }
         return finalSize;
      }

      protected override Visual GetVisualChild(int index)
      {
         if (index == 0 && _adorningElement != null)
         {
            return _adorningElement;
         }
         return base.GetVisualChild(index);
      }

      public void Dispose()
      {
         _layer.Remove(this);
      }

   }

I've got some things I'd like to share, so expect more updates starting again soon.  Continue doing a lot of WPF and related work.  It's great fun and soon I hope to forget entirely what WinForms even looked like.