Duplicate search results from custom module indexer

This forum is only for questions or discussions about working with the mojoPortal source code in Visual Studio, obtaining the source code from the repository, developing custom features, etc. If your question is not along these lines this is not the right forum. Please try to post your question in the appropriate forum.

Please do not post questions about design, CSS, or skinning here. Use the Help With Skins Forum for those questions.

This forum is for discussing mojoPortal development

This forum is only for questions or discussions about working with the mojoPortal source code in Visual Studio, obtaining the source code from the repository, developing custom features, etc. If your question is not along these lines this is not the right forum. Please try to post your question in the appropriate forum.

You can monitor commits to the repository from this page. We also recommend developers to subscribe to email notifications in the developer forum as occasionally important things are announced.

Before posting questions here you might want to review the developer documentation.

Do not post questions about design, CSS, or skinning here. Use the Help With Skins Forum for those questions.
This thread is closed to new posts. You must sign in to post in the forums.
7/11/2013 9:13:02 AM
Gravatar
Total Posts 9

Duplicate search results from custom module indexer

Hi,

Can anyone spot where I'm going wrong here. I've got a custom module that has two parts:

  1. A product admin page for adding product details that is accessed using an link in the administration page
  2. A product list control that links to a details page, which I've used the internal url manager to create search engine friendly urls to

In order to get the products searchable I've created an IndexBuilderProvider which, in essence, seems to work fine. Here's the code for the RebuildIndex routine:

    public override void RebuildIndex(PageSettings pageSettings, string indexPath)
    {
      if (WebConfigSettings.DisableSearchIndex) { return; }

      if ((pageSettings == null) || (indexPath == null))
      {
        return;
      }

      //don't index pending/unpublished pages
      if (pageSettings.IsPending) { return; }

      Guid featureGuid = new Guid("9164051a-7335-447a-90e1-37f73bd858e7");
      ModuleDefinition productListFeature = new ModuleDefinition(featureGuid);

      // get the list of modules on the site using the feature, although in all likelyhood there'll only be one
      List<int> moduleIds = new List<int>();
      using (IDataReader rdr = Module.GetModulesForSite(pageSettings.SiteId, featureGuid))
      {
        while (rdr.Read())
        {
          moduleIds.Add(Convert.ToInt32(rdr["ModuleId"]));
        }
      }

      foreach (int modId in moduleIds)
      {
        Module mod = new Module(modId);
        // get the pageModules using the Module
        List<PageModule> pageModules = PageModule.GetPageModulesByPage(pageSettings.PageId);
        foreach (PageModule pageModule in pageModules)
        {
          // cehck if the pageModule matches our custom module id
          if (pageModule.ModuleId == modId)
          {
            log.Info("CSafeProductsIndexBuilderProvider indexing page - " + pageSettings.PageName);
            // get the list of active products showing on the list page
            List<Product> allProds = ProductManager.GetHomeProducts();
            // for each product create an item index for the detail page
            foreach (Product product in allProds)
            {
              IndexItem indexItem = new IndexItem()
              {
                SiteId = pageSettings.SiteId,
                PageId = pageSettings.PageId,
                PageName = pageSettings.PageName,
                ViewRoles = pageSettings.AuthorizedRoles,
                ViewPage = product.Url,
                UseQueryStringParams = false,
                FeatureName = productListFeature.FeatureName,
                FeatureResourceFile = productListFeature.ResourceFile,
                ItemId = product.Id,
                ModuleId = modId,
                ModuleTitle = mod.ModuleTitle,
                Title = product.Name,
                Content = product.IntroText + " " + product.DetailText,
                PublishBeginDate = pageModule.PublishBeginDate,
                PublishEndDate = pageModule.PublishEndDate,
                FeatureId = featureGuid.ToString()
              };

              if (product.Deleted)
                IndexHelper.RemoveIndexItem(pageSettings.PageId, modId, product.Id);
              else
                IndexHelper.RebuildIndex(indexItem);
              log.Info(String.Format("CSafeProductsIndexBuilderProvider indexing pageId {0}, moduleId {1}, productid {2}", pageSettings.PageId, modId, product.Id));

            }

            // Create a single indexItem for the Product List page (use zero for the product id to differentiate it)
            string searchContent = string.Empty;
            allProds.FindAll(x => !x.Deleted).ForEach(x => searchContent += String.Format("{0} {1} ", x.Name, x.ShortDescription));

            IndexItem homeItem = new IndexItem()
            {
              SiteId = pageSettings.SiteId,
              PageId = pageSettings.PageId,
              PageName = pageSettings.PageName,
              ViewRoles = pageSettings.AuthorizedRoles,
              ViewPage = pageSettings.Url.Replace("~/", string.Empty),
              UseQueryStringParams = false,
              FeatureName = productListFeature.FeatureName,
              FeatureResourceFile = productListFeature.ResourceFile,
              ItemId = 0,
              ModuleId = modId,
              ModuleTitle = mod.ModuleTitle,
              Title = mod.ModuleTitle,
              Content = searchContent,
              PublishBeginDate = pageModule.PublishBeginDate,
              PublishEndDate = pageModule.PublishEndDate,
              FeatureId = featureGuid.ToString()
            };
            IndexHelper.RebuildIndex(homeItem);
            log.Info(String.Format("CSafeProductsIndexBuilderProvider indexing pageId {0}, moduleId {1}, productid {2}", pageSettings.PageId, modId, 0));

          }
        }

      }
    }


All seems to work fine and when I trigger the index rebuild I get the following log entries that I expect:

2013-07-11 14:19:39,446 INFO (null) - (null) - (null) - CSafeProducts.BusinessLogic.CSafeProductsIndexBuilderProvider - CSafeProductsIndexBuilderProvider indexing page - Products
2013-07-11 14:19:39,495 INFO (null) - (null) - (null) - CSafeProducts.BusinessLogic.CSafeProductsIndexBuilderProvider - CSafeProductsIndexBuilderProvider indexing pageId 8, moduleId 18, productid 1
2013-07-11 14:19:39,512 INFO (null) - (null) - (null) - CSafeProducts.BusinessLogic.CSafeProductsIndexBuilderProvider - CSafeProductsIndexBuilderProvider indexing pageId 8, moduleId 18, productid 2
2013-07-11 14:19:39,529 INFO (null) - (null) - (null) - CSafeProducts.BusinessLogic.CSafeProductsIndexBuilderProvider - CSafeProductsIndexBuilderProvider indexing pageId 8, moduleId 18, productid 4
2013-07-11 14:19:39,546 INFO (null) - (null) - (null) - CSafeProducts.BusinessLogic.CSafeProductsIndexBuilderProvider - CSafeProductsIndexBuilderProvider indexing pageId 8, moduleId 18, productid 6
2013-07-11 14:19:39,561 INFO (null) - (null) - (null) - CSafeProducts.BusinessLogic.CSafeProductsIndexBuilderProvider - CSafeProductsIndexBuilderProvider indexing pageId 8, moduleId 18, productid 7
2013-07-11 14:19:39,577 INFO (null) - (null) - (null) - CSafeProducts.BusinessLogic.CSafeProductsIndexBuilderProvider - CSafeProductsIndexBuilderProvider indexing pageId 8, moduleId 18, productid 5
2013-07-11 14:19:39,594 INFO (null) - (null) - (null) - CSafeProducts.BusinessLogic.CSafeProductsIndexBuilderProvider - CSafeProductsIndexBuilderProvider indexing pageId 8, moduleId 18, productid 3
2013-07-11 14:19:39,611 INFO (null) - (null) - (null) - CSafeProducts.BusinessLogic.CSafeProductsIndexBuilderProvider - CSafeProductsIndexBuilderProvider indexing pageId 8, moduleId 18, productid 0

These log entries only appear once, so I figure I'm only adding one index per product, but when I run a search on the product name then I get duplicate entries for both the detail page and the list page, as you can see here.

This is my first time playing with an IndexBuilderProvider so if anyone can point me in the right direction I'd really appreciate it.

Cheers, Martin.

 

 

7/11/2013 9:26:21 AM
Gravatar
Total Posts 9

Re: Duplicate search results from custom module indexer

Sorry, I've just realised that I have another problem first. The search results are only showing if you are logged in, despite the fact that the page settings are set to "All Users", so anyone clicking on the link above won't be able to see what I'm referring to.

Has anyone got any ideas as to why the those duplicate search results I'm talking about aren't showing up if you aren't logged in?

Cheers, Martin.

7/11/2013 9:49:09 AM
Gravatar
Total Posts 9

Re: Duplicate search results from custom module indexer

And for info I'm running MojoPortal 2.3.9.7 on MSSQL, with IIS on Windows Server 2008 R2 64 bit.

7/11/2013 9:50:37 AM
Gravatar
Total Posts 18439

Re: Duplicate search results from custom module indexer

Hi Martin,

Indexing issues can be difficult to diagnose, we have a utility page that you can drop into your site and navigate to that allows you to directly browse what is in the index which can be helpful in diagnosing the problem. It allows you to see the index item key and roles which may help you verify if the items are being indexed correctly ie each product should have a unique key in the index. 

If your module exists on multiple pages that can also result in duplicated items in the index since pageid becomes part of the key for the index item.

Also make sure you are using the latest version of mojoPortal.

You also should be adding the module view roles:

indexItem.ModuleViewRoles = module.ViewRoles;

Hope that helps,

Joe

7/11/2013 10:37:28 AM
Gravatar
Total Posts 9

Re: Duplicate search results from custom module indexer

Thanks Joe, the ModuleViewRoles has sorted my "All Users" access although I'm still getting the duplicates. I'm on the 2.3.9.7 release but have just seen your post about the changes to the indexing and haven't implemented that yet.

The module is definitely only published once on the site so I'll have a look at the IndexBrowser and post back if I find anything.

Thanks again for you help.

Cheers, Martin.

7/11/2013 11:28:11 AM
Gravatar
Total Posts 9

Re: Duplicate search results from custom module indexer

Just as a follow up, viewing the Index showed duplicate entries for each of the products as well, with the same Id too so clicking on the delete button to remove one of them deleted both.

However, adding the CreatedUtc and LastModUtc to the IndexItem (as your other post pointed out that we are supposed to) appears to have fixed the problem smiley.

 

 

You must sign in to post in the forums. This thread is closed to new posts.