TF271001: The server response was: 4.4.1 Connection timed out type SmtpException. TFS Email Notifications Delayed or Not Sent

by Xavier Dilip Kumar 17. July 2013 19:53

Detailed Message:

TF271001: An error occurred while attempting to send an e-mail notification to the following address: email id. Further e-mail notification errors that occur within the next five minutes might not be logged. Verify that the e-mail notification settings are correct in the Team Foundation Administration Console.
Exception Message: Service not available, closing transmission channel. The server response was: 4.4.1 Connection timed out (type SmtpException)

Error :

TFS Email Notifications Delayed or Not Sent

Explanation:

I started experiencing problems with TFS alert notification mails. TFS Email Notifications were Delayed or Not received. The TFS alert notification was working fine for the past 2 years.

Solution:

Following are the investigation done to fix the notification problem

1. Verified the SMTP server and found out no problems with it. (Anonymous sender was enabled in Exchange for tfs@domain.com)
2. Verified whether "Visual Studio Team Foundation Background Job Agent" service is running in TFS App tier. Service was running fine. (Restarted the service too)


3. I ran the follwoing query to find tfs job failures in the tbl_JobHistory table in the Tfs_Configuration database

SELECT TOP 10 DATEADD(hour, -7, StartTime) as StartTimePDT, Result, ResultMessage
FROM [Tfs_Configuration].[dbo].[tbl_JobHistory] WHERE JobId = 'A4804DCF-4BB6-4109-B61C-E59C2E8A9FF7' AND Result <> 0
ORDER BY StartTime DESC

Note: The JobId GUID for the “Team Foundation Server Event Processing” job with the namespace: “Microsoft.TeamFoundation.JobService.Extensions.Core.NotificationJobExtension” is 'A4804DCF-4BB6-4109-B61C-E59C2E8A9FF7'

Query result:

"There were errors or warnings during notification delivery.    35573/35598 emails delivered.  0/0 soap notifications delivered.   25 errors.  0 warnings.    Set /Service/Integration/Settings/NotificationJobLogLevel in the TF registry to 2 to see errors and warnings"   

4. Queried [dbo].[tbl_EventNotification] table in the [Tfs_DefaultCollection] database to see for failures.

SELECT * FROM [Tfs_DefaultCollection].[dbo].[tbl_EventNotification]

Found many records with failed status.

5. Deleted all records with failed status

Delete FROM [Tfs_DefaultCollection].[dbo].[tbl_EventNotification] where [status] = 'failed'

Note : I dint see any harm in deleting all the records from FROM [Tfs_DefaultCollection].[dbo].[tbl_EventNotification]

Delete FROM [Tfs_DefaultCollection].[dbo].[tbl_EventNotification]

6. Queried [dbo].[tbl_EventNotification] table in the [Tfs_DefaultCollection] database to see the notification event backlogs count.

SELECT * FROM [Tfs_DefaultCollection].[dbo].[tbl_ClientEvent]

Found around 1000 pending events that need to be processed.

7. Deleted all pending events backlogs

Delete FROM [Tfs_DefaultCollection].[dbo].[tbl_ClientEvent] 

Verified the fix by trigging the new alerts, All the alerts got processed and also received the notification mails.

link to turn on verbose log

link to Decrease or Increase notification delay settings in TFS

 

Tags: ,

TFS | TFS 2010 | TFS 2012

Decrease or Increase notification delay settings in TFS

by Xavier Dilip Kumar 16. July 2013 21:39

Explanation:

By default notification delay is set to 120 seconds. The notifications are processed in batchs for every 2 minutes. The notification delay can be set in the TF registry
Note: The TFS registry mentioned is not the Windows Registry on the TFS server.

Solution:

The registry value can be changed using the PowerShell script or using the TfsReg tool

Following is the command to set the notification delay to 1 minute using the tool:

tfsreg.exe /server:http://tfs:8080/tfs /path:/Service/Integration/Settings/NotificationJobDelay /value:60

Note: Restart the TFS IIS app pool for the change to take effect.

Tags: ,

TFS | TFS 2010 | TFS 2012

TFS event 7005 and 7006 warnings

by Xavier Dilip Kumar 4. July 2013 10:47

Explanation:

The default threshold is set to 30 seconds, The warning occurs when the threshold exceeds.

Log Name:      Application
Source:        TFS Services
Date:          1/27/2013 8:53:37 AM
Event ID:      7006
Task Category: None
Level:         Warning
Detailed Message: There are no active requests for service host DefaultCollection that exceed the warning threshold of 30.
 
Log Name:      Application
Source:        TFS Services
Date:          1/27/2013 8:52:22 AM
Event ID:      7005
Task Category: None
Level:         Warning
Detailed Message: A request for service host DefaultCollection has been executing for 37 seconds, exceeding the warning threshold of 30.

Solution:

Increase the threshold limit, if you experience performance issues due to these warnings. execute the following SQL statements

use [<Collection DB name>]

DECLARE @registryUpdates typ_KeyValuePairStringTableNullable
INSERT @registryUpdates ([Key], [Value])
SELECT '#\Configuration\ServiceHostMonitor\TotalExecutionElapsedThreshold\', '300'
 
EXEC prc_UpdateRegistry 1, 'Test', @registryUpdates, 1

Tags: ,

TFS | TFS 2010 | TFS 2012

Error during serialization or deserialization using the JSON JavaScriptSerializer. The length of the string exceeds the value set on the maxJsonLength property.

by Xavier Dilip Kumar 2. July 2013 20:12

Explanation:

  • The Work item with large amount of data in the custom HTML fields fails to open after migration.
  • The error may occur for the build logs with large amount of data as well.

Workaround:

Try to modify web.config in \Program Files\Microsoft Team Foundation Server 11.0\Application Tier\Web Services to adjust the size.
The default length is 20MB, so try a larger value than that. The value below is for 50MB. Experimentation may be necessary to find the best value.

<appSettings>
<add key="maxJsonLength" value="52428800" />
</appSettings>

Value is specified in byte

Tags: ,

TFS | TFS 2010

specified argument was out of the range of valid values. Parameter name: column

by Xavier Dilip Kumar 2. July 2013 19:19

Explanation:
In a CI build (gated check-in) environment, associate a work item with a check-in to trigger a build. The build will partially succeeded with the following error.

 


Workaround:
Change “Associate Changesets and Work Items” property to False in the build definition.

 

Solution:
It’s a known bug in TFS 2012 update 2 but fixed in update 3 http://www.microsoft.com/en-us/download/details.aspx?id=38185

Tags: ,

TFS 2012

TF900560: Could not start build: The file exists.

by Xavier Dilip Kumar 25. June 2013 14:43

Explanation:

TFS2012 server process the build request but the build controller fails.

Workaround:

Reimaged the build controller,Since the controller was not responding to any command and also TFS admin console crashed.

Tags: , ,

TFS | TFS 2010 | TFS 2012

TF215097: An error occurred while initializing a build for build definition <Build Definition Name> rootArgumentValues

by Xavier Dilip Kumar 25. June 2013 14:27

Error:

The values provided for the root activity's arguments did not satisfy the root activity's requirements: ''Build Parameter':
The following keys from the input dictionary do not map to arguments and must be removed: <Parameter Name>. 
Please note that argument names are case sensitive. Parameter name: rootArgumentValues

Explanation:

The error appears when the build parameter is removed from the template template and still avaiable in the build tables.

The table tbl_BuildDefinition contains all information for a build definition. ProcessParameters column will contains the build parameter details

Solution:

Refresh the build definition and save it. In the process tab, there is a Refresh button that will reload the build definition and the process template and synchronize them

 

Tags: , ,

TFS | TFS 2010 | TFS 2012

tf400731: Only one Catalog resource of type 'Project Portal' can exist under 'Team Project'.

by Xavier Dilip Kumar 20. June 2013 13:14

Solution:

1. Remove the existing portal setting for team project as shown in the screenshort
2. Execute the following command to create a team project site

TFPT.EXE addprojectportal /collection:http://<server>:<port>/tfs/<TeamProjectCollection>

* /teamproject:<TeamProject> /processtemplate:"Microsoft Visual Studio Scrum 2.2"

 

Tags: , ,

SharePoint | TFS 2010 | TFS 2012

TF250061: The following site could not be attached to Team Foundation Server: The site is already associated with a team project on an installation of Team Foundation Server with a different ID.

by Xavier Dilip Kumar 20. June 2013 05:19

Explanation:

I had this problem when I migrated TFS 2010 collection to TFS 2012 instance and having the collection online on both TFS 2010 and TFS 2012 instances in parallel.

Solution:

To fix the problem, you need to disable the team project portal links either on TFS 2010 or TFS 2012 collection --> Team projects and run the SharePoint repair on the APP TIER using TFS administration console.

 Following are the steps I followed to fix the error in TFS 2012 instance

1. Open Team Explorer in Visual Studio --> Connect to TFS 2010 instance --> Select the team project that needs the fix --> goto "Portal Settings" --> Unchecked the "Enable Team Project Portal checkbox" --> click OK

 

 

2. Login to TFS 2012 App Tier --> Open TFS Administration console --> Select "Share Point Web Applications" --> Select the SharePoint URL --> click "Repair Connection" --> Repair option

All the permissions and connections to the team projects Share point sites should succeed.

(Optional)

3. Login to TFS 2010 App Tier --> Open TFS Administration console --> Select "Share Point Web Applications" --> Select the SharePoint URL --> click "Repair Connection" --> Repair option

All the permissions and connections to the team projects Share point sites should succeed.

 

Tags: , ,

SharePoint | TFS 2010 | TFS 2012

TF14098: Access Denied User [Domain]\User needs Label permission(s) for $/Source

by Xavier Dilip Kumar 18. June 2013 20:33

 

 

Explanation
The users should have project level permission to label any folders and files in version control using visual studio IDE. The error occurs when team project does not have project users/groups permissions set at project level but its set on folders.

Solution
Add the users/groups permission at project level.

 

Tags: , ,

TFS

About the author

My name is Xavier Dilip Kumar Jayaraj and I am a Software Configuration Management Engineer with a background in application development, Build & packaging using Install Shield.

Profile in LinkedIn

  

Quotes I Like

"Failure will never overtake me if my determination to succeed is strong enough."  - Dr. APJ. Abdul Kalam

"Always be yourself, express yourself, have faith in yourself, do not go out and look for a successful personality and duplicate it." - Bruce Lee

"Technology is just a tool. In terms of getting the kids working together and motivating them, the teacher is the most important." - Bill Gates

"Innovation distinguishes between a leader and a follower." - Steve Jobs

Disclaimer

The information provided here is based on my expreriences, troubleshooting and online/offline findings. It can be used as is on your own risk without any warranties and I impose no rights.

Month List