WPF SelectedItem working inconsistently....(MVVM data binding) - wpf

Question

I have a WPF ListBox that is a child of a ListView.
The ListBox has a 'SelectedItem' attribute that is bound to a property on my ViewModel SelectedTask.
When I click a row on my ListBox, then it updates SelectedTask. But only sometimes.....
By sometimes:
Every List Box row will update SelectedTask at least once
The ones in the "middle" of the screen will always update SelectedTask. I can hop from one row to another and back again, each time SelectedTask is updated
The ones at the top or bottom of the screen will update SelectedTask, but maybe only once. If I select another row and then return to this one, I can not get SelectedTask to be updated.
A highly cut-down/edited view of my XAML is as follows:
<ListView ItemsSource="{Binding WorkItems}" >
<ListView.View>
<GridView>
<GridViewColumn>
<GridViewColumn.CellTemplate>
<DataTemplate>
<Grid>
<Grid.RowDefinitions>
<RowDefinition/>
<RowDefinition />
</Grid.RowDefinitions>
<Grid.ColumnDefinitions>
<ColumnDefinition Width="70"/>
<ColumnDefinition Width="850"/>
</Grid.ColumnDefinitions>
/* Display properties from Work Item */
<TextBox Grid.Row="0" Grid.Column="0" Text="{Binding Code}" />
/* This SelectedItem works sometimes, but not all the time */
<ListBox Grid.Row="1" Grid.Column="0" Grid.ColumnSpan="2" ItemsSource="{Binding Tasks}" SelectedItem="{Binding Path=DataContext.SelectedTask, RelativeSource={RelativeSource FindAncestor, AncestorType={x:Type ListView}}}" >
<ListBox.ItemTemplate>
<DataTemplate>
<Grid Margin="0,0,0,5">
<Grid.RowDefinitions>
<RowDefinition/>
<RowDefinition />
<RowDefinition />
<RowDefinition />
</Grid.RowDefinitions>
<Grid.ColumnDefinitions>
<ColumnDefinition Width="*" />
</Grid.ColumnDefinitions>
<StackPanel Grid.Row="0" Orientation="Horizontal">
/* Display properties from Work Item Task */
<TextBlock Text="{Binding Path=Description}" />
</StackPanel>
</Grid>
</DataTemplate>
</ListBox.ItemTemplate>
</ListBox>
</Grid>
</DataTemplate>
</GridViewColumn.CellTemplate>
</GridViewColumn>
</GridView>
</ListView.View>
</ListView>
EDIT
I was asked what the SelectedTask property looked like:
public MyTaskType SelectedTask
{
get { return _selectedTask; }
set
{
_selectedTask = value;
NotifyPropertyChanged();
}
}