Searching External Systems using SharePoint 2010 Business Connectivity Services (BCS) within throttling limits

SharePoint 2010 Business Connectivity Services (BCS) within throttling limits. To solve this need to split records into chunks.
The LastIdSeen filter enables chunking for IDEnumerator methods. 
For  more information check url:
http://msdn.microsoft.com/en-us/library/ms492695%28v=office.12%29.aspx

Example
http://blogs.msdn.com/b/taj/archive/2010/08/24/searching-external-systems-using-sharepoint-2010-business-connectivity-services-bcs-within-throttling-limits.aspx

Business Connectivity Services (BCS) throttling limits

BCS can only fetch 2000 records at one fetch due to throttling issues. we can resolve this issues in two ways

1. Switch off the BCS throttling using powershell. This not the best way as it will cause performance issues in future.
   
   $bdcProxy = Get-SPServiceApplicationProxy | where {$_.GetType().FullName -eq (‘Microsoft.SharePoint.BusinessData.SharedService.’ + ‘BdcServiceApplicationProxy’)}
   $dbRule = Get-SPBusinessDataCatalogThrottleConfig -Scope Database -ThrottleType Items -ServiceApplicationProxy $bdcProxy
   Set-SPBusinessDataCatalogThrottleConfig -Identity $dbRule -Enforced:$false
2.  Modifying the BCS model to include the Key to Filter Methods.

 

SharePoint BDC Oracle Connection to database using username and password

If you want SharePoint BDC connect to Oracle database using explicit username and password:

1.  Set up an application definition in Secure Store Service with the Oracle credentials.
2.  Use AuthenticationMode of RdbCredentials.
3. When you use RdbCredentials as the authentication mode, you cannot use the RdbConnection User ID and RdbConnection Password properties, as these values are supplied by the Secure Store Service.(Lob connection will be as below)
   

   <LobSystemInstances>       <LobSystemInstance Name="OracleInstance">         <Properties>         <Property Name="AuthenticationMode" Type="System.String">RdbCredentials</Property>         <Property Name="DatabaseAccessProvider" Type="System.String">Oracle</Property>         <Property Name="RdbConnection Data Source" Type="System.String">YOUR_ORACLE_NET_SERVICE_NAME_HERE</Property>         <Property Name="SsoApplicationId" Type="System.String">SECURESTORE_ORACLE_APP_ID_HERE</Property>         <!-- Client Ship -->         <Property Name="SsoProviderImplementation"           Type="System.String">           Microsoft.Office.BusinessData.Infrastructure.SecureStore.LocalSecureStoreProvider,           Microsoft.Office.BusinessData, Version=14.0.0.0, Culture=neutral,           PublicKeyToken=71e9bce111e9429c</Property>        </Properties>       </LobSystemInstance>     </LobSystemInstances>

4. If you specify them, they are simply ignored. You must use Secure Store to supply the Oracle credentials.(Lob connection will be as below)
   

   <LobSystemInstance Name="PassThrough">           <Properties>             <Property Name="ShowInSearchUI" Type="System.String"></Property>             <Property Name="DatabaseAccessProvider" Type="System.String">Odbc</Property>             <Property Name="AuthenticationMode" Type="System.String">PassThrough</Property>             <Property Name="RdbConnection Driver" Type="System.String">{Oracle in OraClient11g_home1}</Property>             <Property Name="RdbConnection dbq" Type="System.String">XE_ORION</Property>             <Property Name="RdbConnection uid" Type="System.String">hr</Property>             <Property Name="RdbConnection pwd" Type="System.String">12345</Property>             <Property Name="RdbConnection Trusted_Connection" Type="System.String">yes</Property>           </Properties>         </LobSystemInstance>

SharePoint Fast Search2010 Incremental Crawl BCS

1. You can add these properties to your list methodInstance in bdc file and deploy again. 

   <MethodInstance Type="Finder" ReturnParameterName="DocumentReadList" Default="true" Name="DocumentReadList" DefaultDisplayName="ProductDocument Read List">          <Properties>        <Property Name="LastModifiedTimeStampField" Type="System.String">YourModifiedDateField</Property>   </Properties>   </MethodInstances>

   For IdEnumerator set __BdcLastModifiedTimestamp Property
   

   <Properties> <Property Name="__BdcLastModifiedTimestamp" Type="System.String">LastModifiedOn</Property> </Properties>

1. You can add these properties to your list method of entity using PowerShell:
   $entity = Get-SPBusinessDataCatalogMetadataObject  –BdcObjectType “Entity” –Name “EntityName” –Namespace “ModelNameSpace” -ServiceContext adminurl
   $methodinstance = $entity.MethodInstances | where {$_.Name –eq “ReadListName”}
   $methodinstance.Properties.Add("LastModifiedTimeStampField", YourModifiedDateField)
   $methodinstance.Update()

You can check sample of:

HOW TO: Create a Searchable SharePoint 2010 BDC .NET Assembly Connector Which Reads From A Flat File

Metadata Mappings Common Mistake

  When you want manage Metadata Mapping, Somebody use link exists in Queries and Results.
This link is not correct for SharePoint Fast Search2010, You should Manage Metadata mapping from Fast Search Administration.

Fast AutoDiscover Properties

Sometimes will find crawler discover my object crawler properties automatically.  You need to make sure Category Properties defined to allow auto discover. Check Auto Discover property from Fast Search Administration.(You can go to this from any of Fast Service Management)

Also you can set it from PowerShell:

Set-FASTSearchMetadataCategory  –Name “Category” –DiscoverNewProperties $flag

Flag is true/false

Set-FASTSearchMetadataCategory  –Name “Business Data” –DiscoverNewProperties $true

Also you can add your crawled properties manual using PowerShell:

$guid = [guid]::NewGuid()

New-FASTSearchMetadataCategory -Name MoreWeb -Propset $guid

New-FASTSearchMetadataCrawledProperty -Name ExtendedTitle -Propset $guid -Varianttype 31

Command does not work

Sometimes when you copy command from Site and try to run from PowerShell or Command Line.
Command Give error or doesn't give expected result, Although it is correct like command below:
stsadm -o backup -url sitecollectionurl -filename -backupfilepath
Although command appear correct there is some characters incorrect like '-',' '. To eliminate this problem you need to rewrote unascii character (dash, space) and rewrote again. Space here has ascii value is 140 instead of 32

Enable BCS Content Type to be crawled

When tried to crawl my business data catalogs and select Business data from content source, You didn’t find my business data.
Check External Content Type information. You will find its property Crawlable is No you need to update it to Yes like Figure Below:

This occurs because I didn’t define properties RootFinder or UseClientCachingForSearch to my list Method.

To Solve this problem you have two options:

1. You can add these properties to your list methodInstance in bdc file and deploy again.
2. You can add these properties to your list method of entity using PowerShell:
   
   $entity = Get-SPBusinessDataCatalogMetadataObject  –BdcObjectType “Entity” –Name “EntityName” –Namespace “ModelNameSpace” -ServiceContext adminurl
   $methodinstance = $entity.MethodInstances | where {$_.Name –eq “ReadListName”}
   $methodinstance.Properties.Add(“RootFinder”, “”)
   $methodinstance.Properties.Add(“UseClientCachingForSearch”, “”)
   $methodinstance.Update()

To Customize Refine Panel in UI Web Part

1. Add Metadata mapping for column you want to add to refine Panel:
   • Go to Fast Query Service and Select Manage from Control Panel.
   • From Left navigation, select Fast Search Administration.
   • Select Link Manage Properties to manage properties mapping between your metadata and crawled content.
   • You can add or manage property. To allow property to be used for refine panel make sure refiner property is checked.
2. To Add a metadata to Refine Panel:


• From Search Page Edit Refine Panel Web part.
• Form Filter category definition you add/remove columns.
• Make sure use default configuration is unchecked to allow you customize column
  
• Make sure use default configuration is unchecked to allow you customize column.
  
  

Add items to the autocomplete using PowerShell

If you want to add multiple keywords to autosuggestion of fast:

1. Create file with keywords separated by newline.
2. Run Command below to add elements from your file to auto suggest list.Get-content “Auto Suggest File Path” | foreach-object {   New-SPEnterpriseSearchLanguageResourcePhrase –SearchApplication “FastQuery” –Language en-US –Type QuerySuggestionAlwaysSuggest –Name $_ }
3. Use Commands below to refresh the cache:
   $timer=Get-SPTimerJob|? {$_.Name –eq “Prepare Query Suggestions”}
   $timer.RunNow()