Microsoft Release Management - build fails with Internal Server Error - ms-release-management

Question

I am trying to implement RM for releasing a website (on prem). When I que a build or invoke the release from the RM client, i am getting the following error. I turned on logs for the client, server and monitor as described here.
Our setup:
TFS Version: TFS2010
RM Version: 2013
VS Version: 2013
Build controller. build Agent, RM Server, RM Client all on the same Windows Server 2008 R2 SP1 box. TFS on a separate box.
I would really appreciate any help in troubleshooting this.
Thanks.
Image of my template designer as requested in the comment below. :
11/16/2015 12:06:48 AM - Error - (9428, 12556) - The remote server returned an error: (500) Internal Server Error.: \r\n\r\n at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at Microsoft.TeamFoundation.Release.Data.WebRequest.PlatformHttpClient.EndGetResponse(IAsyncResult asyncResult)
at Microsoft.TeamFoundation.Release.Data.WebRequest.RestClientResponseRetriever.EndGetAsyncMemoryStreamFromResponse(IAsyncResult asyncResult, IPlatformHttpClient platformHttpClient)
at Microsoft.TeamFoundation.Release.Data.WebRequest.RestClientResponseRetriever.EndDownloadString(IAsyncResult asyncResult, IPlatformHttpClient platformHttpClient)
at Microsoft.TeamFoundation.Release.Data.WebRequest.RestClient.EndPost(IAsyncResult asyncResult)
at Microsoft.TeamFoundation.Release.Data.Proxy.RestProxy.HttpRequestor.<>c__DisplayClass1.<GetPostCaller>b__0(String url, String body)
at Microsoft.TeamFoundation.Release.Data.Proxy.RestProxy.BaseDeploymentControllerServiceProxy.PopulateTaggedActivities(String workflowXml, Int32 environmentId)
at Microsoft.TeamFoundation.Release.Data.Model.Release.BuildReleaseStages(ReleasePath selectedReleasePath, XElement applicationVersion)
at Microsoft.TeamFoundation.Release.Data.Model.Release.BringDataFromApplicationVersion()
at Microsoft.TeamFoundation.Release.Data.Model.Release.OnPropertyChanged(String propertyName, Boolean setDirty)
at Microsoft.TeamFoundation.Release.Data.Model.PropertyChangedBase.OnPropertyChanged(String propertyName)
at Microsoft.TeamFoundation.Release.Data.Model.Release.set_ApplicationVersionId(Int32 value)
at Microsoft.VisualStudio.Release.ViewModel.ViewModels.ReleaseViewModel.Initialize(Int32 modelId)
at Microsoft.VisualStudio.Release.ViewModel.ViewModels.ReleaseViewModel..ctor(String viewMode, Dictionary`2 popupParameters)
at Microsoft.VisualStudio.Release.ViewModel.ViewModels.ApplicationVersionsViewModel.CreateNewRelease(Int32 releaseTemplateId)
at Microsoft.VisualStudio.Release.ViewModel.ViewModels.ApplicationVersionsViewModel.CreateRelease(XElement selectedItem)
at Microsoft.VisualStudio.Release.ViewModel.ViewModels.ApplicationVersionsViewModel.CreateNewRelease(Object selectedItems)
at Microsoft.VisualStudio.Release.ViewModel.Helpers.RelayCommand.Execute(Object parameter)
at MS.Internal.Commands.CommandHelpers.CriticalExecuteCommandSource(ICommandSource commandSource, Boolean userInitiated)
at System.Windows.Controls.Primitives.ButtonBase.OnClick()
at System.Windows.Controls.Button.OnClick()
at System.Windows.Controls.Primitives.ButtonBase.OnMouseLeftButtonUp(MouseButtonEventArgs e)
at System.Windows.UIElement.OnMouseLeftButtonUpThunk(Object sender, MouseButtonEventArgs e)
at System.Windows.Input.MouseButtonEventArgs.InvokeEventHandler(Delegate genericHandler, Object genericTarget)
at System.Windows.RoutedEventArgs.InvokeHandler(Delegate handler, Object target)
at System.Windows.RoutedEventHandlerInfo.InvokeHandler(Object target, RoutedEventArgs routedEventArgs)
at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
at System.Windows.UIElement.ReRaiseEventAs(DependencyObject sender, RoutedEventArgs args, RoutedEvent newEvent)
at System.Windows.UIElement.OnMouseUpThunk(Object sender, MouseButtonEventArgs e)
at System.Windows.Input.MouseButtonEventArgs.InvokeEventHandler(Delegate genericHandler, Object genericTarget)
at System.Windows.RoutedEventArgs.InvokeHandler(Delegate handler, Object target)
at System.Windows.RoutedEventHandlerInfo.InvokeHandler(Object target, RoutedEventArgs routedEventArgs)
at System.Windows.EventRoute.InvokeHandlersImpl(Object source, RoutedEventArgs args, Boolean reRaised)
at System.Windows.UIElement.RaiseEventImpl(DependencyObject sender, RoutedEventArgs args)
at System.Windows.UIElement.RaiseTrustedEvent(RoutedEventArgs args)
at System.Windows.UIElement.RaiseEvent(RoutedEventArgs args, Boolean trusted)
at System.Windows.Input.InputManager.ProcessStagingArea()
at System.Windows.Input.InputManager.ProcessInput(InputEventArgs input)
at System.Windows.Input.InputProviderSite.ReportInput(InputReport inputReport)
at System.Windows.Interop.HwndMouseInputProvider.ReportInput(IntPtr hwnd, InputMode mode, Int32 timestamp, RawMouseActions actions, Int32 x, Int32 y, Int32 wheel)
at System.Windows.Interop.HwndMouseInputProvider.FilterMessage(IntPtr hwnd, WindowMessage msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at System.Windows.Interop.HwndSource.InputFilterMessage(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at MS.Win32.HwndWrapper.WndProc(IntPtr hwnd, Int32 msg, IntPtr wParam, IntPtr lParam, Boolean& handled)
at MS.Win32.HwndSubclass.DispatcherCallbackOperation(Object o)
at System.Windows.Threading.ExceptionWrapper.InternalRealCall(Delegate callback, Object args, Int32 numArgs)
at MS.Internal.Threading.ExceptionFilterHelper.TryCatchWhen(Object source, Delegate method, Object args, Int32 numArgs, Delegate catchHandler)
Below is what I find in the Microsoft.TeamFoundation.Release.Build.log
Microsoft.TeamFoundation.Release.Build starting process...
Processing using the following parameters:
TeamFoundationServerUrl: http://MyTFSServer.MyDomain.com:8080/tfs/MyCollection
TeamProjectName: MYProjectName
TeamProjectWithCollection: MyCollection/MYProjectName
BuildDefinitionName: MyBuildName
BuildNumber: MyBuildName_20151115.1
TargetStage:
Creating the release and waiting for deployment result...
ERROR: The remote server returned an error: (500) Internal Server Error.
at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at Microsoft.TeamFoundation.Release.Data.WebRequest.RestClientResponseRetriever.EndGetAsyncMemoryStreamFromResponse(IAsyncResult asyncResult, IPlatformHttpClient platformHttpClient)
at Microsoft.TeamFoundation.Release.Data.WebRequest.RestClientResponseRetriever.EndDownloadString(IAsyncResult asyncResult, IPlatformHttpClient platformHttpClient)
at Microsoft.TeamFoundation.Release.Data.WebRequest.RestClient.EndPost(IAsyncResult asyncResult)
at Microsoft.TeamFoundation.Release.Data.Proxy.RestProxy.BaseDeploymentControllerServiceProxy.PopulateTaggedActivities(String workflowXml, Int32 environmentId)
at Microsoft.TeamFoundation.Release.Data.Model.Release.BuildReleaseStages(ReleasePath selectedReleasePath, XElement applicationVersion)
at Microsoft.TeamFoundation.Release.Data.Model.Release.BringDataFromApplicationVersion()
at Microsoft.TeamFoundation.Release.Data.Model.Release.OnPropertyChanged(String propertyName, Boolean setDirty)
at Microsoft.TeamFoundation.Release.Workflow.Services.ReleaseWorkflowService.InitiateFromBuild(Uri teamFoundationServerUrl, String teamProject, String buildDefinition, String buildNumber, String targetStageName)
at Microsoft.TeamFoundation.Release.Build.Program.Release()
at Microsoft.TeamFoundation.Release.Build.Program.MainImplementation(String[] args, Action`1 consoleWriter, Action releaseAction)
Exit: 1
For the service, the text log is not getting generated. Following are the entries from the event log with Source "Release Management Monitor". Sorry for the formatting. This is the best I could do to copy from the machine event log:
Log Name: Application
Source: Microsoft Deployment Agent
Date: 11/16/2015 10:24:26 AM
Event ID: 0
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: MyRMServerMachine.MyDomain.com
Description:
Timestamp: 11/16/2015 10:24:26 AM
Message: Object reference not set to an instance of an object.: \r\n\r\n at Microsoft.TeamFoundation.Release.Data.Model.SystemSettings.LoadXml(Int32 id)
at Microsoft.TeamFoundation.Release.Data.Model.ModelFactory.Load[T](Int32 id)
at Microsoft.TeamFoundation.Release.DeploymentAgent.Services.Deployer.DeploymentEventFetcherBase.DeployNextComponent()
Category: General
Priority: -1
EventId: 0
Severity: Error
Title:
Machine: Log Name: Application
Source: Microsoft Deployment Agent
Date: 11/16/2015 10:24:26 AM
Event ID: 0
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: MyRMServerMachine.MyDomain.com
Description:
Timestamp: 11/16/2015 10:24:26 AM
Message: Object reference not set to an instance of an object.: \r\n\r\n at Microsoft.TeamFoundation.Release.Data.Model.SystemSettings.LoadXml(Int32 id)
at Microsoft.TeamFoundation.Release.Data.Model.ModelFactory.Load[T](Int32 id)
at Microsoft.TeamFoundation.Release.DeploymentAgent.Services.Deployer.DeploymentEventFetcherBase.DeployNextComponent()
Category: General
Priority: -1
EventId: 0
Severity: Error
Title:
Machine: MyRMServerMachine
Application Domain: ReleaseManagementMonitor.exe
Process Id: 1652
Process Name: C:\Program Files (x86)\Microsoft Visual Studio 12.0\Release Management\bin\ReleaseManagementMonitor.exe
Win32 Thread Id: 8176
Thread Name:
Extended Properties:
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft Deployment Agent" />
<EventID Qualifiers="0">0</EventID>
<Level>2</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2015-11-16T16:24:26.000000000Z" />
<EventRecordID>474148</EventRecordID>
<Channel>Application</Channel>
<Computer>MyRMServerMachine.MyDomain.com</Computer>
<Security />
</System>
<EventData>
<Data>Timestamp: 11/16/2015 10:24:26 AM
Message: Object reference not set to an instance of an object.: \r\n\r\n at Microsoft.TeamFoundation.Release.Data.Model.SystemSettings.LoadXml(Int32 id)
at Microsoft.TeamFoundation.Release.Data.Model.ModelFactory.Load[T](Int32 id)
at Microsoft.TeamFoundation.Release.DeploymentAgent.Services.Deployer.DeploymentEventFetcherBase.DeployNextComponent()
Category: General
Priority: -1
EventId: 0
Severity: Error
Title:
Machine: MyRMServerMachine
Application Domain: ReleaseManagementMonitor.exe
Process Id: 1652
Process Name: C:\Program Files (x86)\Microsoft Visual Studio 12.0\Release Management\bin\ReleaseManagementMonitor.exe
Win32 Thread Id: 8176
Thread Name:
Extended Properties: </Data>
</EventData>
</Event>
Application Domain: ReleaseManagementMonitor.exe
Process Id: 1652
Process Name: C:\Program Files (x86)\Microsoft Visual Studio 12.0\Release Management\bin\ReleaseManagementMonitor.exe
Win32 Thread Id: 8176
Thread Name:
Extended Properties:
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft Deployment Agent" />
<EventID Qualifiers="0">0</EventID>
<Level>2</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2015-11-16T16:24:26.000000000Z" />
<EventRecordID>474148</EventRecordID>
<Channel>Application</Channel>
<Computer>MyRMServerMachine.MyDomain.com</Computer>
<Security />
</System>
<EventData>
<Data>Timestamp: 11/16/2015 10:24:26 AM
Message: Object reference not set to an instance of an object.: \r\n\r\n at Microsoft.TeamFoundation.Release.Data.Model.SystemSettings.LoadXml(Int32 id)
at Microsoft.TeamFoundation.Release.Data.Model.ModelFactory.Load[T](Int32 id)
at Microsoft.TeamFoundation.Release.DeploymentAgent.Services.Deployer.DeploymentEventFetcherBase.DeployNextComponent()
Category: General
Priority: -1
EventId: 0
Severity: Error
Title:
Machine: MyRMServerMachine
Application Domain: ReleaseManagementMonitor.exe
Process Id: 1652
Process Name: C:\Program Files (x86)\Microsoft Visual Studio 12.0\Release Management\bin\ReleaseManagementMonitor.exe
Win32 Thread Id: 8176
Thread Name:
Extended Properties: </Data>
</EventData>
</Event>
Also, the following keeps getting logged from the monitor:
Log Name: Application
Source: Release Management Monitor
Date: 11/16/2015 10:26:36 AM
Event ID: 0
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: MyRMServer.MyDomain.com
Description:
Timestamp: 11/16/2015 10:26:36 AM
Message: An existing connection was forcibly closed by the remote host: \r\n\r\n at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
Category: General
Priority: -1
EventId: 0
Severity: Error
Title:
Machine: MyRMServer
Application Domain: ReleaseManagementMonitor.exe
Process Id: 1652
Process Name: C:\Program Files (x86)\Microsoft Visual Studio 12.0\Release Management\bin\ReleaseManagementMonitor.exe
Win32 Thread Id: 9896
Thread Name:
Extended Properties:
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Release Management Monitor" />
<EventID Qualifiers="0">0</EventID>
<Level>2</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2015-11-16T16:26:36.000000000Z" />
<EventRecordID>474201</EventRecordID>
<Channel>Application</Channel>
<Computer>MyRMServer.MyDomain.com</Computer>
<Security />
</System>
<EventData>
<Data>Timestamp: 11/16/2015 10:26:36 AM
Message: An existing connection was forcibly closed by the remote host: \r\n\r\n at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
Category: General
Priority: -1
EventId: 0
Severity: Error
Title:
Machine: MyRMServer
Application Domain: ReleaseManagementMonitor.exe
Process Id: 1652
Process Name: C:\Program Files (x86)\Microsoft Visual Studio 12.0\Release Management\bin\ReleaseManagementMonitor.exe
Win32 Thread Id: 9896
Thread Name:
Extended Properties: </Data>
</EventData>
</Event>
Log Name: Application
Source: Release Management Monitor
Date: 11/16/2015 10:26:36 AM
Event ID: 0
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: MyRMServer.MyDomain.com
Description:
Timestamp: 11/16/2015 10:26:36 AM
Message: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.: \r\n\r\n at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
at System.Net.Connection.ReadCallback(IAsyncResult asyncResult)
Category: General
Priority: -1
EventId: 0
Severity: Error
Title:
Machine: MyRMServer
Application Domain: ReleaseManagementMonitor.exe
Process Id: 1652
Process Name: C:\Program Files (x86)\Microsoft Visual Studio 12.0\Release Management\bin\ReleaseManagementMonitor.exe
Win32 Thread Id: 9896
Thread Name:
Extended Properties:
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Release Management Monitor" />
<EventID Qualifiers="0">0</EventID>
<Level>2</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2015-11-16T16:26:36.000000000Z" />
<EventRecordID>474200</EventRecordID>
<Channel>Application</Channel>
<Computer>MyRMServer.MyDomain.com</Computer>
<Security />
</System>
<EventData>
<Data>Timestamp: 11/16/2015 10:26:36 AM
Message: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.: \r\n\r\n at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
at System.Net.Connection.ReadCallback(IAsyncResult asyncResult)
Category: General
Priority: -1
EventId: 0
Severity: Error
Title:
Machine: MyRMServer
Application Domain: ReleaseManagementMonitor.exe
Process Id: 1652
Process Name: C:\Program Files (x86)\Microsoft Visual Studio 12.0\Release Management\bin\ReleaseManagementMonitor.exe
Win32 Thread Id: 9896
Thread Name:
Extended Properties: </Data>
</EventData>
</Event>
Log Name: Application
Source: Release Management Monitor
Date: 11/16/2015 10:26:36 AM
Event ID: 0
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: MyRMServer.MyDomain.com
Description:
Timestamp: 11/16/2015 10:26:36 AM
Message: The underlying connection was closed: An unexpected error occurred on a receive.: \r\n\r\n at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at Microsoft.TeamFoundation.Release.Data.WebRequest.RestClientResponseRetriever.EndGetAsyncMemoryStreamFromResponse(IAsyncResult asyncResult, IPlatformHttpClient platformHttpClient)
at Microsoft.TeamFoundation.Release.Data.WebRequest.RestClientResponseRetriever.EndDownloadString(IAsyncResult asyncResult, IPlatformHttpClient platformHttpClient)
at Microsoft.TeamFoundation.Release.Data.WebRequest.RestClient.EndGet(IAsyncResult asyncResult)
at Microsoft.TeamFoundation.Release.Data.Proxy.RestProxy.BaseWorkflowAutomationEngineServiceProxy.GetStalledDeploymentActivities(Int32 timeout)
at Microsoft.TeamFoundation.Release.Monitor.Services.MonitorServiceExtensions.FailAllV2StalledDeployment(MonitorService monitorService, String serviceName)
Category: General
Priority: -1
EventId: 0
Severity: Error
Title:
Machine: MyRMServer
Application Domain: ReleaseManagementMonitor.exe
Process Id: 1652
Process Name: C:\Program Files (x86)\Microsoft Visual Studio 12.0\Release Management\bin\ReleaseManagementMonitor.exe
Win32 Thread Id: 9896
Thread Name:
Extended Properties:
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Release Management Monitor" />
<EventID Qualifiers="0">0</EventID>
<Level>2</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2015-11-16T16:26:36.000000000Z" />
<EventRecordID>474199</EventRecordID>
<Channel>Application</Channel>
<Computer>MyRMServer.MyDomain.com</Computer>
<Security />
</System>
<EventData>
<Data>Timestamp: 11/16/2015 10:26:36 AM
Message: The underlying connection was closed: An unexpected error occurred on a receive.: \r\n\r\n at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at Microsoft.TeamFoundation.Release.Data.WebRequest.RestClientResponseRetriever.EndGetAsyncMemoryStreamFromResponse(IAsyncResult asyncResult, IPlatformHttpClient platformHttpClient)
at Microsoft.TeamFoundation.Release.Data.WebRequest.RestClientResponseRetriever.EndDownloadString(IAsyncResult asyncResult, IPlatformHttpClient platformHttpClient)
at Microsoft.TeamFoundation.Release.Data.WebRequest.RestClient.EndGet(IAsyncResult asyncResult)
at Microsoft.TeamFoundation.Release.Data.Proxy.RestProxy.BaseWorkflowAutomationEngineServiceProxy.GetStalledDeploymentActivities(Int32 timeout)
at Microsoft.TeamFoundation.Release.Monitor.Services.MonitorServiceExtensions.FailAllV2StalledDeployment(MonitorService monitorService, String serviceName)
Category: General
Priority: -1
EventId: 0
Severity: Error
Title:
Machine: MyRMServer
Application Domain: ReleaseManagementMonitor.exe
Process Id: 1652
Process Name: C:\Program Files (x86)\Microsoft Visual Studio 12.0\Release Management\bin\ReleaseManagementMonitor.exe
Win32 Thread Id: 9896
Thread Name:
Extended Properties: </Data>
</EventData>
</Event>
Log Name: Application
Source: Release Management Monitor
Date: 11/16/2015 10:22:15 AM
Event ID: 0
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: MyRMServer.MyDomain.com
Description:
Timestamp: 11/16/2015 10:22:15 AM
Message: Object reference not set to an instance of an object.: \r\n\r\n at Microsoft.TeamFoundation.Release.Monitor.Services.MonitorService.FailAllStalledDeployment()
Category: General
Priority: -1
EventId: 0
Severity: Error
Title:
Machine: MyRMServer
Application Domain: ReleaseManagementMonitor.exe
Process Id: 1652
Process Name: C:\Program Files (x86)\Microsoft Visual Studio 12.0\Release Management\bin\ReleaseManagementMonitor.exe
Win32 Thread Id: 9896
Thread Name:
Extended Properties:
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Release Management Monitor" />
<EventID Qualifiers="0">0</EventID>
<Level>2</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2015-11-16T16:22:15.000000000Z" />
<EventRecordID>474098</EventRecordID>
<Channel>Application</Channel>
<Computer>MyRMServer.MyDomain.com</Computer>
<Security />
</System>
<EventData>
<Data>Timestamp: 11/16/2015 10:22:15 AM
Message: Object reference not set to an instance of an object.: \r\n\r\n at Microsoft.TeamFoundation.Release.Monitor.Services.MonitorService.FailAllStalledDeployment()
Category: General
Priority: -1
EventId: 0
Severity: Error
Title:
Machine: MyRMServer
Application Domain: ReleaseManagementMonitor.exe
Process Id: 1652
Process Name: C:\Program Files (x86)\Microsoft Visual Studio 12.0\Release Management\bin\ReleaseManagementMonitor.exe
Win32 Thread Id: 9896
Thread Name:
Extended Properties: </Data>
</EventData>
</Event>