Sitecore Corner

Recently we noticed that on our production environment the Crawling Log files were abnormally big – between 500 MB and 1 GB. After some digging we found out that they were just filling up with exceptions while crawling the media library. The exceptions look like this:

4576 10:48:07 WARN Could not compute value for ComputedIndexField: urllink for indexable: sitecore://master/{8EA15044-EE2C-41DB-81D6-0A9C42062814}?lang=en&ver=1
Exception: System.NullReferenceException
Message: Object reference not set to an instance of an object.
Source: Sitecore.Kernel
at Sitecore.Context.PageMode.get_IsNormal()
at adaptiveImages.AdaptiveImagesMediaProvider.GetMediaUrl(MediaItem item)
at Sitecore.ContentSearch.ComputedFields.UrlLink.ComputeFieldValue(IIndexable indexable)
at Sitecore.ContentSearch.LuceneProvider.LuceneDocumentBuilder.AddComputedIndexFields()

Considering the case that we have tons of media (over 2000 items) and languages (there is a separate exception for each language) we had some pretty serious index time performance issues and hard disk space problems.

After some debugging we found out that the Sitecore.Context.PageMode.IsNormal was throwing null reference. The huge problem was that there is no way to protect against null pointers here because you cannot check the Sitecore.Context and the Sitecore.Context.PageMode for nulls. The good part is that the PageMode.IsNormal method code is using the Sitecore.Context.Site (which is the actual null in this case) to check the PageMode.

public static bool IsNormal
{
    get
    {
        return Context.Site.DisplayMode == DisplayMode.Normal;
    }
}

With this information the fix became pretty trivial. As try-catch is pretty expensive operation – we just needed to check if the Context.Site is not null and the Context.Site.DisplayMode is normal. The fixes took place in the two GetMediaUrl overrides. Here is the code for the modified methods which were causing the exception.

  /// <summary>
        /// Gets a media URL.
        /// </summary>
        /// <param name="item">The media item.</param>
        /// <returns>
        /// The media URL.
        /// </returns>
        public override string GetMediaUrl(MediaItem item)
        {
            Assert.ArgumentNotNull(item, "item");

            // FIX FOR CHECKING THE PAGE MODE
            bool isNormal = Context.Site != null && Context.Site.DisplayMode == DisplayMode.Normal;

            //If media item is not an image or the page context is not normal, then return
            if (!IsImage(item) || !isNormal)
                return base.GetMediaUrl(item);

            MediaUrlOptions mediaUrlOptions = new MediaUrlOptions();

            return GetMediaUrl(item, mediaUrlOptions);
        }

        /// <summary>
        /// Gets the media URL.
        /// </summary>
        /// <param name="item">The item.</param>
        /// <param name="mediaUrlOptions">The media URL options.</param>
        /// <returns></returns>
        public override string GetMediaUrl(MediaItem item, MediaUrlOptions mediaUrlOptions)
        {
            Assert.ArgumentNotNull(item, "item");
            Assert.ArgumentNotNull(mediaUrlOptions, "mediaUrlOptions");

            // FIX FOR CHECKING THE PAGE MODE
            bool isNormal = Context.Site != null && Context.Site.DisplayMode == DisplayMode.Normal;

            //If media item is not an image or the page context is not normal, then return
            if (!IsImage(item) || !isNormal || Context.Database == null || Context.Database.Name != _database)
                return base.GetMediaUrl(item, mediaUrlOptions);

            //If resolution cookie is not set
            if (!IsResolutionCookieSet())
            {
                //If mobileFirst is set to FALSE or user agent is identifying as a desktop, return with largest break-point resolution
                if (!_mobileFirst || IsDesktopBrowser())
                {
                    mediaUrlOptions.MaxWidth = GetLargestBreakpoint();
                    return base.GetMediaUrl(item, mediaUrlOptions);
                }
                //Return with mobile-first breakpoint (Smallest)
                mediaUrlOptions.MaxWidth = GetMobileFirstBreakpoint();
                return base.GetMediaUrl(item, mediaUrlOptions);
            }

            // If Max-width is not set or Max-width is greater than the selected break-point, then set the Max-width to the break-point
            if (mediaUrlOptions.MaxWidth == 0 || mediaUrlOptions.MaxWidth > GetScreenResolution())
                mediaUrlOptions.MaxWidth = GetScreenResolution();

            // If Max-width is not set and the 'maxWidth' setting is not empty, then set the Max-width property to the maxWidth
            if (mediaUrlOptions.MaxWidth == 0 && !string.IsNullOrEmpty(_maxWidth))
            {
                int maxWidth = 0;
                if (int.TryParse(_maxWidth, out maxWidth))
                {
                    // If pixel ratio is normal
                    if (GetCookiePixelDensity() == 1)
                        mediaUrlOptions.MaxWidth = maxWidth;
                    else
                        mediaUrlOptions.MaxWidth = maxWidth * GetCookiePixelDensity();
                }

            }

            return base.GetMediaUrl(item, mediaUrlOptions);
        }

Basically we are using an isNormal variable to check the Context.Site display mode instead of using the Context.PageMode.IsNormal property:

            bool isNormal = Context.Site != null && Context.Site.DisplayMode == DisplayMode.Normal;

By doing this we kept the module’s consistency and logic as we just added a null check for the Context.Site. After the fix our logs became around 2 kb again and our indexing speed came back to normal !

Happy Log Fixing ! 🙂